Call REST API operations with Shared Key authorization

Article
12/08/2023

This article shows how to call an Azure Storage REST API operation by creating an authorized REST request using C#. After you learn how to call a REST API operation for Blob Storage, you can use similar steps for any other Azure Storage REST operation.

Prerequisites

The sample application lists the blob containers for a storage account. To try out the code in this article, you need the following items:

Install Visual Studio and include the Azure development workload. This example was built using Visual Studio 2019. If you use a different version, the guidance might vary slightly.
An Azure subscription. If you don't have an Azure subscription, create a trial account before you begin.
A general-purpose storage account. If you don't yet have a storage account, see Create a storage account.
The example in this article shows how to list the containers in a storage account. To see output, add some blob containers to the storage account before you start.

Download the sample application

The sample application is a console application written in C#.

Use git to download a copy of the application to your development environment.

git clone https://github.com/Azure-Samples/storage-dotnet-rest-api-with-auth.git

This command clones the repository to your local git folder. To open the Visual Studio solution, navigate to the storage-dotnet-rest-api-with-auth folder and open StorageRestApiAuth.sln.

About REST

Representational State Transfer (REST) is an architecture that enables you to interact with a service over an internet protocol, such as HTTP/HTTPS. REST is independent of the software running on the server or the client. The REST API can be called from any platform that supports HTTP/HTTPS. You can write an application that runs on a Mac, Windows, Linux, an Android phone or tablet, iPhone, iPod, or web site, and use the same REST API for all of those platforms.

A call to the REST API consists of a request made by the client, and a response returned by the service. In the request, you send a URL with information about which operation you want to call, the resource to act upon, any query parameters and headers, and depending on the operation that was called, a payload of data. The response from the service includes a status code, a set of response headers, and depending on the operation that was called, a payload of data.

About the sample application

The sample application lists the containers in a storage account. Once you understand how the information in the REST API documentation correlates to your actual code, other REST calls are easier to figure out.

If you look at the Blob Service REST API, you see all of the operations you can perform on blob storage. The storage client libraries are wrappers around the REST APIs, making it easy to access storage resources without using the REST APIs directly. Sometimes, however, you might want to use the REST API instead of a storage client library.

List Containers operation

This article focuses on the List Containers operation. The following information helps you understand some of the fields in the request and response.

Request Method: GET. This verb is the HTTP method you specify as a property of the request object. Other values for this verb include HEAD, PUT, and DELETE, depending on the API you're calling.

Request URI: https://myaccount.blob.core.chinacloudapi.cn/?comp=list. The request URI is created from the blob storage account endpoint https://myaccount.blob.core.chinacloudapi.cn and the resource string /?comp=list.

URI parameters: There are additional query parameters you can use when calling ListContainers. A couple of these parameters are timeout for the call (in seconds) and prefix, which is used for filtering.

Another helpful parameter is maxresults: if more containers are available than this value, the response body will contain a NextMarker element that indicates the next container to return on the next request. To use this feature, you provide the NextMarker value as the marker parameter in the URI when you make the next request. When using this feature, it's analogous to paging through the results.

To use additional parameters, append them to the resource string with the value, like this example:

/?comp=list&timeout=60&maxresults=100

Request Headers: This section lists the required and optional request headers. Three of the headers are required: an Authorization header, x-ms-date (contains the UTC time for the request), and x-ms-version (specifies the version of the REST API to use). Including x-ms-client-request-id in the headers is optional. You can set the value for this field to anything, and it's written to the storage analytics logs when logging is enabled.

Request Body: There's no request body for ListContainers. Request Body is used on all of the PUT operations when uploading blobs, including SetContainerAccessPolicy. Request Body allows you to send in an XML list of stored access policies to apply. Stored access policies are discussed in the article Using Shared Access Signatures (SAS).

Response Status Code: Tells of any status codes you need to know. In this example, an HTTP status code of 200 is ok. For a complete list of HTTP status codes, check out Status Code Definitions. To see error codes specific to the Storage REST APIs, see Common REST API error codes

Response Headers: These include Content Type; x-ms-request-id, which is the request ID you passed in; x-ms-version, which indicates the version of the Blob service used; and the Date, which is in UTC and tells what time the request was made.

Response Body: This field is an XML structure providing the data requested. In this example, the response is a list of containers and their properties.

Creating the REST request

For security when running in production, always use HTTPS rather than HTTP. For the purposes of this exercise, we use HTTP so you can view the request and response data. To view the request and response information in the actual REST calls, you can download Fiddler or a similar application. In the Visual Studio solution, the storage account name and key are hardcoded in the class. The ListContainersAsyncREST method passes the storage account name and storage account key to the methods that are used to create the various components of the REST request. In a real world application, the storage account name and key would reside in a configuration file, environment variables, or be retrieved from an Azure Key Vault.

In our sample project, the code for creating the Authorization header is in a separate class. The idea is that you could take the whole class and add it to your own solution and use it "as is." The Authorization header code works for most REST API calls to Azure Storage.

To build the request, which is a HttpRequestMessage object, go to ListContainersAsyncREST in Program.cs. The steps for building the request are:

Create the URI to be used for calling the service.
Create the HttpRequestMessage object and set the payload. The payload is null for ListContainersAsyncREST because we're not passing anything in.
Add the request headers for x-ms-date and x-ms-version.
Get the authorization header and add it.

Some basic information you need:

For ListContainers, the method is GET. This value is set when instantiating the request.
The resource is the query portion of the URI that indicates which API is being called, so the value is /?comp=list. As noted earlier, the resource is on the reference documentation page that shows the information about the ListContainers API.
The URI is constructed by creating the Blob service endpoint for that storage account and concatenating the resource. The value for request URI ends up being http://contosorest.blob.core.chinacloudapi.cn/?comp=list.
For ListContainers, requestBody is null and there are no extra headers.

Different APIs might have other parameters to pass in such as ifMatch. An example of where you might use ifMatch is when calling PutBlob. In that case, you set ifMatch to an eTag, and it only updates the blob if the eTag you provide matches the current eTag on the blob. If someone else has updated the blob since retrieving the eTag, their change isn't overridden.

First, set the uri and the requestPayload.

// Construct the URI. It will look like this:
//   https://myaccount.blob.core.chinacloudapi.cn/resource
String uri = string.Format("http://{0}.blob.core.chinacloudapi.cn?comp=list", storageAccountName);

// Provide the appropriate payload, in this case null.
//   we're not passing anything in.
Byte[] requestPayload = null;

Next, instantiate the request, setting the method to GET and providing the URI.

// Instantiate the request message with a null payload.
using (var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, uri)
{ Content = (requestPayload == null) ? null : new ByteArrayContent(requestPayload) })
{

Add the request headers for x-ms-date and x-ms-version. This place in the code is also where you add any additional request headers required for the call. In this example, there are no additional headers. An example of an API that passes in extra headers is the Set Container ACL operation. This API call adds a header called "x-ms-blob-public-access" and the value for the access level.

// Add the request headers for x-ms-date and x-ms-version.
DateTime now = DateTime.UtcNow;
httpRequestMessage.Headers.Add("x-ms-date", now.ToString("R", CultureInfo.InvariantCulture));
httpRequestMessage.Headers.Add("x-ms-version", "2017-07-29");
// If you need any additional headers, add them here before creating
//   the authorization header.

Call the method that creates the authorization header and add it to the request headers. The authorization header is created later in the article. The method name is GetAuthorizationHeader, which you can see in this code snippet:

// Get the authorization header and add it.
httpRequestMessage.Headers.Authorization = AzureStorageAuthenticationHelper.GetAuthorizationHeader(
    storageAccountName, storageAccountKey, now, httpRequestMessage);

At this point, httpRequestMessage contains the REST request complete with the authorization headers.

Send the request

Now that you have constructed the request, you can call the SendAsync method to send it to Azure Storage. Check that the value of the response status code is 200, meaning that the operation has succeeded. Next, parse the response. In this case, you get an XML list of containers. Let's look at the code for calling the GetRESTRequest method to create the request, execute the request, and then examine the response for the list of containers.

    // Send the request.
    using (HttpResponseMessage httpResponseMessage =
      await new HttpClient().SendAsync(httpRequestMessage, cancellationToken))
    {
        // If successful (status code = 200),
        //   parse the XML response for the container names.
        if (httpResponseMessage.StatusCode == HttpStatusCode.OK)
        {
            String xmlString = await httpResponseMessage.Content.ReadAsStringAsync();
            XElement x = XElement.Parse(xmlString);
            foreach (XElement container in x.Element("Containers").Elements("Container"))
            {
                Console.WriteLine("Container name = {0}", container.Element("Name").Value);
            }
        }
    }
}

If you run a network sniffer such as Fiddler when making the call to SendAsync, you can see the request and response information. Let's take a look. The name of the storage account is contosorest.

Request:

GET /?comp=list HTTP/1.1

Request Headers:

x-ms-date: Thu, 16 Nov 2017 23:34:04 GMT
x-ms-version: 2014-02-14
Authorization: SharedKey contosorest:1dVlYJWWJAOSHTCPGiwdX1rOS8B4fenYP/VrU0LfzQk=
Host: contosorest.blob.core.chinacloudapi.cn
Connection: Keep-Alive

Status code and response headers returned after execution:

HTTP/1.1 200 OK
Content-Type: application/xml
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 3e889876-001e-0039-6a3a-5f4396000000
x-ms-version: 2017-07-29
Date: Fri, 17 Nov 2017 00:23:42 GMT
Content-Length: 1511

Response body (XML): For the List Containers operation, this shows the list of containers and their properties.

<?xml version="1.0" encoding="utf-8"?>
<EnumerationResults
  ServiceEndpoint="http://contosorest.blob.core.chinacloudapi.cn/">
  <Containers>
    <Container>
      <Name>container-1</Name>
      <Properties>
        <Last-Modified>Thu, 16 Mar 2017 22:39:48 GMT</Last-Modified>
        <Etag>"0x8D46CBD5A7C301D"</Etag>
        <LeaseStatus>unlocked</LeaseStatus>
        <LeaseState>available</LeaseState>
      </Properties>
    </Container>
    <Container>
      <Name>container-2</Name>
      <Properties>
        <Last-Modified>Thu, 16 Mar 2017 22:40:50 GMT</Last-Modified>
        <Etag>"0x8D46CBD7F49E9BD"</Etag>
        <LeaseStatus>unlocked</LeaseStatus>
        <LeaseState>available</LeaseState>
      </Properties>
    </Container>
    <Container>
      <Name>container-3</Name>
      <Properties>
        <Last-Modified>Thu, 16 Mar 2017 22:41:10 GMT</Last-Modified>
        <Etag>"0x8D46CBD8B243D68"</Etag>
        <LeaseStatus>unlocked</LeaseStatus>
        <LeaseState>available</LeaseState>
      </Properties>
    </Container>
    <Container>
      <Name>container-4</Name>
      <Properties>
        <Last-Modified>Thu, 16 Mar 2017 22:41:25 GMT</Last-Modified>
        <Etag>"0x8D46CBD93FED46F"</Etag>
        <LeaseStatus>unlocked</LeaseStatus>
        <LeaseState>available</LeaseState>
        </Properties>
      </Container>
      <Container>
        <Name>container-5</Name>
        <Properties>
          <Last-Modified>Thu, 16 Mar 2017 22:41:39 GMT</Last-Modified>
          <Etag>"0x8D46CBD9C762815"</Etag>
          <LeaseStatus>unlocked</LeaseStatus>
          <LeaseState>available</LeaseState>
        </Properties>
      </Container>
  </Containers>
  <NextMarker />
</EnumerationResults>

Now that you understand how to create the request, call the service, and parse the results, let's see how to create the authorization header.

Creating the authorization header

Tip

Azure Storage supports Microsoft Entra integration for blobs and queues. Microsoft Entra ID offers a much simpler experience for authorizing a request to Azure Storage. For more information on using Microsoft Entra ID to authorize REST operations, see Authorize with Microsoft Entra ID. For an overview of Microsoft Entra integration with Azure Storage, see Authenticate access to Azure Storage using Microsoft Entra ID.

To learn more about authorization concepts, see Authorize requests to Azure Storage.

Let's distill that article down to exactly is needed and show the code.

First, use Shared Key authorization. The authorization header format looks like this:

Authorization="SharedKey <storage account name>:<signature>"

The signature field is a Hash-based Message Authentication Code (HMAC) created from the request and calculated using the SHA256 algorithm, then encoded using Base64 encoding.

This code snippet shows the format of the Shared Key signature string:

StringToSign = VERB + "\n" +  
               Content-Encoding + "\n" +  
               Content-Language + "\n" +  
               Content-Length + "\n" +  
               Content-MD5 + "\n" +  
               Content-Type + "\n" +  
               Date + "\n" +  
               If-Modified-Since + "\n" +  
               If-Match + "\n" +  
               If-None-Match + "\n" +  
               If-Unmodified-Since + "\n" +  
               Range + "\n" +  
               CanonicalizedHeaders +  
               CanonicalizedResource;

For Blob storage, you specify VERB, md5, content length, Canonicalized Headers, and Canonicalized Resource. You can leave the others blank for this example, but put in \n to specify that they're blank.

Canonicalization is a process of standardizing data that has more than one possible representation. In this case, you're standardizing the headers and the resource. The canonicalized headers are the headers that start with "x-ms-". The canonicalized resource is the URI of the resource, including the storage account name and all query parameters (such as ?comp=list). The canonicalized resource also includes any additional query parameters you might have added, such as timeout=60, for example.

Let's start with the two canonicalized fields, because they're required to create the Authorization header.

Canonicalized headers

To create this value, retrieve the headers that start with "x-ms-" and sort them, then format them into a string of [key:value\n] instances, concatenated into one string. For this example, the canonicalized headers look like this:

x-ms-date:Fri, 17 Nov 2017 00:44:48 GMT\nx-ms-version:2017-07-29\n

Here's the code used to create that output:

private static string GetCanonicalizedHeaders(HttpRequestMessage httpRequestMessage)
{
    var headers = from kvp in httpRequestMessage.Headers
        where kvp.Key.StartsWith("x-ms-", StringComparison.OrdinalIgnoreCase)
        orderby kvp.Key
        select new { Key = kvp.Key.ToLowerInvariant(), kvp.Value };

    StringBuilder headersBuilder = new StringBuilder();

    foreach (var kvp in headers)
    {
        headersBuilder.Append(kvp.Key);
        char separator = ':';

        // Get the value for each header, strip out \r\n if found, then append it with the key.
        foreach (string headerValue in kvp.Value)
        {
            string trimmedValue = headerValue.TrimStart().Replace("\r\n", string.Empty);
            headersBuilder.Append(separator).Append(trimmedValue);

            // Set this to a comma; this will only be used
            // if there are multiple values for one of the headers.
            separator = ',';
        }

        headersBuilder.Append("\n");
    }

    return headersBuilder.ToString();
}

Canonicalized resource

This part of the signature string represents the storage account targeted by the request. Remember that the Request URI is http://contosorest.blob.core.chinacloudapi.cn/?comp=list, with the actual account name (contosorest in this case). In this example, this is returned:

/contosorest/\ncomp:list

If you have query parameters, this example includes those parameters as well. Here's the code, which also handles additional query parameters and query parameters with multiple values. Remember that you're building this code to work for all of the REST APIs. You want to include all possibilities, even if the ListContainers method doesn't need all of them.

private static string GetCanonicalizedResource(Uri address, string storageAccountName)
{
    // The absolute path will be "/" because for we're getting a list of containers.
    StringBuilder sb = new StringBuilder("/").Append(storageAccountName).Append(address.AbsolutePath);

    // Address.Query is the resource, such as "?comp=list".
    // This ends up with a NameValueCollection with 1 entry having key=comp, value=list.
    // It will have more entries if you have more query parameters.
    NameValueCollection values = HttpUtility.ParseQueryString(address.Query);

    foreach (var item in values.AllKeys.OrderBy(k => k))
    {
        sb.Append('\n').Append(item.ToLower()).Append(':').Append(values[item]);
    }

    return sb.ToString();
}

Now that the canonicalized strings are set, let's look at how to create the authorization header itself. You start by creating a string of the message signature in the format of StringToSign previously displayed in this article. This concept is easier to explain using comments in the code, so here it is, the final method that returns the Authorization Header:

internal static AuthenticationHeaderValue GetAuthorizationHeader(
    string storageAccountName, string storageAccountKey, DateTime now,
    HttpRequestMessage httpRequestMessage, string ifMatch = "", string md5 = "")
{
    // This is the raw representation of the message signature.
    HttpMethod method = httpRequestMessage.Method;
    String MessageSignature = String.Format("{0}\n\n\n{1}\n{5}\n\n\n\n{2}\n\n\n\n{3}{4}",
                method.ToString(),
                (method == HttpMethod.Get || method == HttpMethod.Head) ? String.Empty
                  : httpRequestMessage.Content.Headers.ContentLength.ToString(),
                ifMatch,
                GetCanonicalizedHeaders(httpRequestMessage),
                GetCanonicalizedResource(httpRequestMessage.RequestUri, storageAccountName),
                md5);

    // Now turn it into a byte array.
    byte[] SignatureBytes = Encoding.UTF8.GetBytes(MessageSignature);

    // Create the HMACSHA256 version of the storage key.
    HMACSHA256 SHA256 = new HMACSHA256(Convert.FromBase64String(storageAccountKey));

    // Compute the hash of the SignatureBytes and convert it to a base64 string.
    string signature = Convert.ToBase64String(SHA256.ComputeHash(SignatureBytes));

    // This is the actual header that will be added to the list of request headers.
    AuthenticationHeaderValue authHV = new AuthenticationHeaderValue("SharedKey",
        storageAccountName + ":" + signature);
    return authHV;
}

When you run this code, the resulting MessageSignature looks like this example:

GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:Fri, 17 Nov 2017 01:07:37 GMT\nx-ms-version:2017-07-29\n/contosorest/\ncomp:list

Here's the final value for AuthorizationHeader:

SharedKey contosorest:Ms5sfwkA8nqTRw7Uury4MPHqM6Rj2nfgbYNvUKOa67w=

The AuthorizationHeader is the last header placed in the request headers before posting the response.

That covers everything you need to know to put together a class with which you can create a request to call the Storage Services REST APIs.

Example: List blobs

Let's look at how to change the code to call the List Blobs operation for container container-1. This code is almost identical to the code for listing containers, the only differences being the URI and how you parse the response.

If you look at the reference documentation for ListBlobs, you find that the method is GET and the RequestURI is:

https://myaccount.blob.core.chinacloudapi.cn/container-1?restype=container&comp=list

In ListContainersAsyncREST, change the code that sets the URI to the API for ListBlobs. The container name is container-1.

String uri =
    string.Format("http://{0}.blob.core.chinacloudapi.cn/container-1?restype=container&comp=list",
      storageAccountName);

Then where you handle the response, change the code to look for blobs instead of containers.

foreach (XElement container in x.Element("Blobs").Elements("Blob"))
{
    Console.WriteLine("Blob name = {0}", container.Element("Name").Value);
}

When you run this sample, you get results like the following:

Canonicalized headers:

x-ms-date:Fri, 17 Nov 2017 05:16:48 GMT\nx-ms-version:2017-07-29\n

Canonicalized resource:

/contosorest/container-1\ncomp:list\nrestype:container

Message signature:

GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-date:Fri, 17 Nov 2017 05:16:48 GMT
  \nx-ms-version:2017-07-29\n/contosorest/container-1\ncomp:list\nrestype:container

Authorization header:

SharedKey contosorest:uzvWZN1WUIv2LYC6e3En10/7EIQJ5X9KtFQqrZkxi6s=

The following values are from Fiddler: