Azure blob storage java generate SAS token always throw sr is mandatory. Cannot be empty error - azure

I want to use Java to generate a SAS token, this is what I do:
public static String GetSASToken(String resourceUri, String keyName, String key) {
long epoch = System.currentTimeMillis() / 1000L;
int week = 60 * 30 * 1000;
String expiry = Long.toString(epoch + week);
String sasToken = null;
try {
String stringToSign = URLEncoder.encode(resourceUri, "UTF-8") + "\n" + expiry;
String signature = getHMAC256(key, stringToSign);
sasToken =
"SharedAccessSignature sr=" + URLEncoder.encode(resourceUri, "UTF-8") + "&sig="
+
URLEncoder.encode(signature, "UTF-8") + "&se=" + expiry + "&skn="
+ keyName;
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
return sasToken;
}
public static String getHMAC256(String key, String input) {
Mac sha256_HMAC = null;
String hash = null;
try {
sha256_HMAC = Mac.getInstance("HmacSHA256");
SecretKeySpec secret_key = new SecretKeySpec(key.getBytes(), "HmacSHA256");
sha256_HMAC.init(secret_key);
Encoder encoder = Base64.getEncoder();
hash = new String(encoder.encode(sha256_HMAC.doFinal(input.getBytes("UTF-8"))));
} catch (InvalidKeyException e) {
e.printStackTrace();
} catch (NoSuchAlgorithmException e) {
e.printStackTrace();
} catch (IllegalStateException e) {
e.printStackTrace();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}
return hash;
}
This code is nothing but copied from microsoft website,
Then in main class:
String key = "xxxxxHh9jP1ZOTYZI/z1OCVeThsjK00sSc3TYUuiHJQ==";
String kn = "frankbucket";
String url = "https://frankbucket.blob.core.windows.net/mycontainer";
System.out.print(AzureSASTokenGenerator.GetSASToken(url, kn, key));
when I run this, I got below SAS:
SharedAccessSignature sr=https%3A%2F%2Ffrankbucket.blob.core.windows.net%2Fmycontainer&sig=xxxj7Fgbkz5OSag%2BzFQAzBkIdd3I1J9AmFwxjcQg%3D&se=1588299503&skn=frankbucket
In Javascript:
var blobUri = 'https://frankbucket.blob.core.windows.net';
var token =
"SharedAccessSignature sr=https%3A%2F%2Ffrankbucket.blob.core.windows.net%2Fmycontainer&sig=ZA5fgKKny5%2BzdffvdEmy6WdsqqpoMsssssYYM9ruXgAdo0%3D&se=1588299257&skn=frankbucket";
var blobService = AzureStorage.Blob.createBlobServiceWithSas(blobUri, token);
blobService.listBlobsSegmented('mycontainer', null, function(error, results) {
When I run it, I got below error:
<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
RequestId:ea315aac-001e-00a0-0700-0f2e4d000000
Time:2020-04-10T06:22:18.2383162Zsr is mandatory. Cannot be empty
I have no clue where is the issue, I got code form microsoft website, but it does not work.
Can anyone show me working example for this?
Hope to hear your advice.
Thanks

If you want to know how to create account sas token with java, please refer to the following code
public void callblobRestAPIWithSas() throws NoSuchAlgorithmException, InvalidKeyException, IOException {
// 1. create account sas token
SimpleDateFormat fmt = new SimpleDateFormat("yyyy-MM-dd");
fmt.setTimeZone(TimeZone.getTimeZone("UTC"));
Calendar cal = Calendar.getInstance();
cal.setTime(new Date());
cal.add(Calendar.DATE, -2);
String start = fmt.format(cal.getTime());
cal.add(Calendar.DATE, 4);
String expiry = fmt.format(cal.getTime());
String StorageAccountName = "blobstorage0516";
String StorageAccountKey = "";
String apiVersion="2019-07-07";
String resource ="sco";
String permissions ="rwdlac";
String service = "b";
String stringToSign = StorageAccountName + "\n" +
permissions +"\n" + // signed permissions
service+"\n" + // signed service
resource+"\n" + // signed resource type
start + "\n" + // signed start
expiry + "\n" + // signed expiry
"\n" + // signed IP
"https\n" + // signed Protocol
apiVersion+"\n"; // signed version
SecretKeySpec secretKey = new SecretKeySpec(Base64.getDecoder().decode(StorageAccountKey), "HmacSHA256");
Mac sha256HMAC = Mac.getInstance("HmacSHA256");
sha256HMAC.init(secretKey);
String signature=Base64.getEncoder().encodeToString(sha256HMAC.doFinal(stringToSign.getBytes("UTF-8")));
String sasToken = "sv=" + apiVersion +
"&ss=" + service+
"&srt=" + resource+
"&sp=" +permissions+
"&se=" + URLEncoder.encode(expiry, "UTF-8") +
"&st=" + URLEncoder.encode(start, "UTF-8") +
"&spr=https" +
"&sig=" + URLEncoder.encode(signature,"UTF-8");
//2. test the sas token
String resourceUrl="https://blobstorage0516.blob.core.windows.net/test/help.txt"; // your blob url
URL url = new URL(resourceUrl+"?"+sasToken);
OkHttpClient httpClient = new OkHttpClient().newBuilder().build();
Request request = new Request.Builder()
.url(url)
.method("GET", null)
.build();
okhttp3.Response response = httpClient.newCall(request).execute();
if(response.isSuccessful()){
System.out.println("The blob content : "+ response.body().string());
}
}

Related

Azure Cognitive Search, how to iterate over facets to bypass 100K limit

I understand the limitation of the API with the 100K limit, and MS's site says as a work around "you can work around this limitation by adding code to iterate over, and filter on, a facet with less that 100K documents per facet value."
I'm using the "Back up and restore an Azure Cognitive Search index" sample solution provided by MS. (https://github.com/Azure-Samples/azure-search-dotnet-samples)
But can some when tell me where or how I implement this "iterate loop" on a facet" The facetable field I'm trying to use is "tributekey" but I don't know where to place the code in the below. Any help would be greatly appreciated.
// This is a prototype tool that allows for extraction of data from a search index
// Since this tool is still under development, it should not be used for production usage
using Azure;
using Azure.Search.Documents;
using Azure.Search.Documents.Indexes;
using Azure.Search.Documents.Models;
using Microsoft.Extensions.Configuration;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
namespace AzureSearchBackupRestore
{
class Program
{
private static string SourceSearchServiceName;
private static string SourceAdminKey;
private static string SourceIndexName;
private static string TargetSearchServiceName;
private static string TargetAdminKey;
private static string TargetIndexName;
private static string BackupDirectory;
private static SearchIndexClient SourceIndexClient;
private static SearchClient SourceSearchClient;
private static SearchIndexClient TargetIndexClient;
private static SearchClient TargetSearchClient;
private static int MaxBatchSize = 500; // JSON files will contain this many documents / file and can be up to 1000
private static int ParallelizedJobs = 10; // Output content in parallel jobs
static void Main(string[] args)
{
//Get source and target search service info and index names from appsettings.json file
//Set up source and target search service clients
ConfigurationSetup();
//Backup the source index
Console.WriteLine("\nSTART INDEX BACKUP");
BackupIndexAndDocuments();
//Recreate and import content to target index
//Console.WriteLine("\nSTART INDEX RESTORE");
//DeleteIndex();
//CreateTargetIndex();
//ImportFromJSON();
//Console.WriteLine("\r\n Waiting 10 seconds for target to index content...");
//Console.WriteLine(" NOTE: For really large indexes it may take longer to index all content.\r\n");
//Thread.Sleep(10000);
//
//// Validate all content is in target index
//int sourceCount = GetCurrentDocCount(SourceSearchClient);
//int targetCount = GetCurrentDocCount(TargetSearchClient);
//Console.WriteLine("\nSAFEGUARD CHECK: Source and target index counts should match");
//Console.WriteLine(" Source index contains {0} docs", sourceCount);
//Console.WriteLine(" Target index contains {0} docs\r\n", targetCount);
//
//Console.WriteLine("Press any key to continue...");
//Console.ReadLine();
}
static void ConfigurationSetup()
{
IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
IConfigurationRoot configuration = builder.Build();
SourceSearchServiceName = configuration["SourceSearchServiceName"];
SourceAdminKey = configuration["SourceAdminKey"];
SourceIndexName = configuration["SourceIndexName"];
TargetSearchServiceName = configuration["TargetSearchServiceName"];
TargetAdminKey = configuration["TargetAdminKey"];
TargetIndexName = configuration["TargetIndexName"];
BackupDirectory = configuration["BackupDirectory"];
Console.WriteLine("CONFIGURATION:");
Console.WriteLine("\n Source service and index {0}, {1}", SourceSearchServiceName, SourceIndexName);
Console.WriteLine("\n Target service and index: {0}, {1}", TargetSearchServiceName, TargetIndexName);
Console.WriteLine("\n Backup directory: " + BackupDirectory);
SourceIndexClient = new SearchIndexClient(new Uri("https://" + SourceSearchServiceName + ".search.windows.net"), new AzureKeyCredential(SourceAdminKey));
SourceSearchClient = SourceIndexClient.GetSearchClient(SourceIndexName);
// TargetIndexClient = new SearchIndexClient(new Uri($"https://" + TargetSearchServiceName + ".search.windows.net"), new AzureKeyCredential(TargetAdminKey));
// TargetSearchClient = TargetIndexClient.GetSearchClient(TargetIndexName);
}
static void BackupIndexAndDocuments()
{
// Backup the index schema to the specified backup directory
Console.WriteLine("\n Backing up source index schema to {0}\r\n", BackupDirectory + "\\" + SourceIndexName + ".schema");
File.WriteAllText(BackupDirectory + "\\" + SourceIndexName + ".schema", GetIndexSchema());
// Extract the content to JSON files
int SourceDocCount = GetCurrentDocCount(SourceSearchClient);
WriteIndexDocuments(SourceDocCount); // Output content from index to json files
}
static void WriteIndexDocuments(int CurrentDocCount)
{
// Write document files in batches (per MaxBatchSize) in parallel
string IDFieldName = GetIDFieldName();
int FileCounter = 0;
for (int batch = 0; batch <= (CurrentDocCount / MaxBatchSize); batch += ParallelizedJobs)
{
List<Task> tasks = new List<Task>();
for (int job = 0; job < ParallelizedJobs; job++)
{
FileCounter++;
int fileCounter = FileCounter;
if ((fileCounter - 1) * MaxBatchSize < CurrentDocCount)
{
Console.WriteLine(" Backing up source documents to {0} - (batch size = {1})", BackupDirectory + "\\" + SourceIndexName + fileCounter + ".json", MaxBatchSize);
tasks.Add(Task.Factory.StartNew(() =>
ExportToJSON((fileCounter - 1) * MaxBatchSize, IDFieldName, BackupDirectory + "\\" + SourceIndexName + fileCounter + ".json")
));
}
}
Task.WaitAll(tasks.ToArray()); // Wait for all the stored procs in the group to complete
}
return;
}
static void ExportToJSON(int Skip, string IDFieldName, string FileName)
{
// Extract all the documents from the selected index to JSON files in batches of 500 docs / file
string json = string.Empty;
try
{
SearchOptions options = new SearchOptions()
{
SearchMode = SearchMode.All,
Size = MaxBatchSize,
Skip = Skip,
// ,IncludeTotalCount = true
// ,Filter = Azure.Search.Documents.SearchFilter.Create('%24top=2&%24skip=0&%24orderby=tributeId%20asc')
//,Filter = String.Format("&search=*&%24top=2&%24skip=0&%24orderby=tributeId%20asc")
//,Filter = "%24top=2&%24skip=0&%24orderby=tributeId%20asc"
//,Filter = "tributeKey eq '5'"
};
SearchResults<SearchDocument> response = SourceSearchClient.Search<SearchDocument>("*", options);
foreach (var doc in response.GetResults())
{
json += JsonSerializer.Serialize(doc.Document) + ",";
json = json.Replace("\"Latitude\":", "\"type\": \"Point\", \"coordinates\": [");
json = json.Replace("\"Longitude\":", "");
json = json.Replace(",\"IsEmpty\":false,\"Z\":null,\"M\":null,\"CoordinateSystem\":{\"EpsgId\":4326,\"Id\":\"4326\",\"Name\":\"WGS84\"}", "]");
json += "\r\n";
}
// Output the formatted content to a file
json = json.Substring(0, json.Length - 3); // remove trailing comma
File.WriteAllText(FileName, "{\"value\": [");
File.AppendAllText(FileName, json);
File.AppendAllText(FileName, "]}");
Console.WriteLine(" Total documents: {0}", response.GetResults().Count().ToString());
json = string.Empty;
}
catch (Exception ex)
{
Console.WriteLine("Error: {0}", ex.Message.ToString());
}
}
static string GetIDFieldName()
{
// Find the id field of this index
string IDFieldName = string.Empty;
try
{
var schema = SourceIndexClient.GetIndex(SourceIndexName);
foreach (var field in schema.Value.Fields)
{
if (field.IsKey == true)
{
IDFieldName = Convert.ToString(field.Name);
break;
}
}
}
catch (Exception ex)
{
Console.WriteLine("Error: {0}", ex.Message.ToString());
}
return IDFieldName;
}
static string GetIndexSchema()
{
// Extract the schema for this index
// We use REST here because we can take the response as-is
Uri ServiceUri = new Uri("https://" + SourceSearchServiceName + ".search.windows.net");
HttpClient HttpClient = new HttpClient();
HttpClient.DefaultRequestHeaders.Add("api-key", SourceAdminKey);
string Schema = string.Empty;
try
{
Uri uri = new Uri(ServiceUri, "/indexes/" + SourceIndexName);
HttpResponseMessage response = AzureSearchHelper.SendSearchRequest(HttpClient, HttpMethod.Get, uri);
AzureSearchHelper.EnsureSuccessfulSearchResponse(response);
Schema = response.Content.ReadAsStringAsync().Result.ToString();
}
catch (Exception ex)
{
Console.WriteLine("Error: {0}", ex.Message.ToString());
}
return Schema;
}
private static bool DeleteIndex()
{
Console.WriteLine("\n Delete target index {0} in {1} search service, if it exists", TargetIndexName, TargetSearchServiceName);
// Delete the index if it exists
try
{
TargetIndexClient.DeleteIndex(TargetIndexName);
}
catch (Exception ex)
{
Console.WriteLine(" Error deleting index: {0}\r\n", ex.Message);
Console.WriteLine(" Did you remember to set your SearchServiceName and SearchServiceApiKey?\r\n");
return false;
}
return true;
}
static void CreateTargetIndex()
{
Console.WriteLine("\n Create target index {0} in {1} search service", TargetIndexName, TargetSearchServiceName);
// Use the schema file to create a copy of this index
// I like using REST here since I can just take the response as-is
string json = File.ReadAllText(BackupDirectory + "\\" + SourceIndexName + ".schema");
// Do some cleaning of this file to change index name, etc
json = "{" + json.Substring(json.IndexOf("\"name\""));
int indexOfIndexName = json.IndexOf("\"", json.IndexOf("name\"") + 5) + 1;
int indexOfEndOfIndexName = json.IndexOf("\"", indexOfIndexName);
json = json.Substring(0, indexOfIndexName) + TargetIndexName + json.Substring(indexOfEndOfIndexName);
Uri ServiceUri = new Uri("https://" + TargetSearchServiceName + ".search.windows.net");
HttpClient HttpClient = new HttpClient();
HttpClient.DefaultRequestHeaders.Add("api-key", TargetAdminKey);
try
{
Uri uri = new Uri(ServiceUri, "/indexes");
HttpResponseMessage response = AzureSearchHelper.SendSearchRequest(HttpClient, HttpMethod.Post, uri, json);
response.EnsureSuccessStatusCode();
}
catch (Exception ex)
{
Console.WriteLine(" Error: {0}", ex.Message.ToString());
}
}
static int GetCurrentDocCount(SearchClient searchClient)
{
// Get the current doc count of the specified index
try
{
SearchOptions options = new SearchOptions()
{
SearchMode = SearchMode.All,
IncludeTotalCount = true
};
SearchResults<Dictionary<string, object>> response = searchClient.Search<Dictionary<string, object>>("*", options);
return Convert.ToInt32(response.TotalCount);
}
catch (Exception ex)
{
Console.WriteLine(" Error: {0}", ex.Message.ToString());
}
return -1;
}
static void ImportFromJSON()
{
Console.WriteLine("\n Upload index documents from saved JSON files");
// Take JSON file and import this as-is to target index
Uri ServiceUri = new Uri("https://" + TargetSearchServiceName + ".search.windows.net");
HttpClient HttpClient = new HttpClient();
HttpClient.DefaultRequestHeaders.Add("api-key", TargetAdminKey);
try
{
foreach (string fileName in Directory.GetFiles(BackupDirectory, SourceIndexName + "*.json"))
{
Console.WriteLine(" -Uploading documents from file {0}", fileName);
string json = File.ReadAllText(fileName);
Uri uri = new Uri(ServiceUri, "/indexes/" + TargetIndexName + "/docs/index");
HttpResponseMessage response = AzureSearchHelper.SendSearchRequest(HttpClient, HttpMethod.Post, uri, json);
response.EnsureSuccessStatusCode();
}
}
catch (Exception ex)
{
Console.WriteLine(" Error: {0}", ex.Message.ToString());
}
}
}
}
I tried adding a filter option in the ExportToJSON method but the request fails

Azure Blob Storage - Error trying to delete blob with API REST with C#

I'm working with Azure Blob Storage and C#. I need to delete a blob using the API REST, but I get the 403 Forbidden error. I'm using the same function to generate the authentication header for create a new blob, but it works for put but doesn't it for delete. This is my code:
public bool DeleteBlob(string containerName, string fileName)
{
string blobName = fileName;
string method = "DELETE";
string requestUri = $"https://{_accountName}.blob.core.windows.net/{containerName}/{blobName}";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(requestUri);
string now = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
request.Method = method;
request.Headers.Add("x-ms-version", "2020-10-02");
request.Headers.Add("x-ms-date", now);
request.Headers.Add("x-ms-delete-snapshots", "include");
request.Headers.Add("Authorization", AuthorizationHeader("DELETE", request, containerName, blobName));
bool result = false;
using (HttpWebResponse resp = (HttpWebResponse)request.GetResponse())
{
if (resp.StatusCode != HttpStatusCode.Accepted && resp.StatusCode != HttpStatusCode.OK)
{
throw new Exception(resp.StatusDescription);
}
result = true;
}
return result;
}
private string AuthorizationHeader(string method, HttpWebRequest request, string containerName, string blobName)
{
string urlResource = $"/{_accountName}/{containerName}/{blobName}";
string stringToSign = $"{method}\n\n\n{request.ContentLength}\n\n{request.ContentType}\n\n\n\n\n\n\n{GetCanonicalizedHeaders(request)}{GetCanonicalizedResource(request.RequestUri, _accountName)}";
HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(_accountKey));
string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
return string.Format("{0} {1}:{2}", "SharedKey", _accountName, signature);
}
private string GetAuthorizationHeader(string method, HttpWebRequest request, string now, string containerName, string blobName)
{
string urlResource = $"/{_accountName}/{containerName}/{blobName}";
var canonicalizedStringToBuild = string.Format("{0}\n{1}", now, $"/{_accountName}/" + urlResource);
string signature;
using (var hmac = new HMACSHA256(Convert.FromBase64String(_accountKey)))
{
byte[] dataToHmac = Encoding.UTF8.GetBytes(canonicalizedStringToBuild);
signature = Convert.ToBase64String(hmac.ComputeHash(dataToHmac));
}
return string.Format($"{_accountName}:" + signature);
}
private string GetCanonicalizedHeaders(HttpWebRequest request)
{
ArrayList headerNameList = new ArrayList();
StringBuilder sb = new StringBuilder();
foreach (string headerName in request.Headers.Keys)
{
if (headerName.ToLowerInvariant().StartsWith("x-ms-", StringComparison.Ordinal))
{
headerNameList.Add(headerName.ToLowerInvariant());
}
}
headerNameList.Sort();
foreach (string headerName in headerNameList)
{
StringBuilder builder = new StringBuilder(headerName);
string separator = ":";
foreach (string headerValue in GetHeaderValues(request.Headers, headerName))
{
string trimmedValue = headerValue.Replace("\r\n", String.Empty);
builder.Append(separator);
builder.Append(trimmedValue);
separator = ",";
}
sb.Append(builder.ToString());
sb.Append("\n");
}
return sb.ToString();
}
private string GetCanonicalizedResource(Uri address, string accountName)
{
StringBuilder str = new StringBuilder();
StringBuilder builder = new StringBuilder("/");
builder.Append(accountName);
builder.Append(address.AbsolutePath);
str.Append(builder.ToString());
NameValueCollection values2 = new NameValueCollection();
if (address.Query.Length > 0)
{
string query = address.Query.Remove(0, 1);
var queryParts = query.Split('&');
foreach (var queryPart in queryParts)
{
var parts = queryPart.Split('=');
values2.Add(parts[0], parts[1]);
}
ArrayList list2 = new ArrayList(values2.AllKeys);
list2.Sort();
foreach (string str3 in list2)
{
StringBuilder builder3 = new StringBuilder(string.Empty);
builder3.Append(str3);
builder3.Append(":");
builder3.Append(values2[str3]);
str.Append("\n");
str.Append(builder3.ToString());
}
}
return str.ToString();
}
Could anyone tell me what I'm doing wrong? As I commented for create a new blob working fine, but not for deletion. I don't know if the authentication header is different for delete (I know the method name is different) I have not found much information about that.
Thannks.
The reason your delete operation is failing with 403 error is because in your stringToSign you're assigning content length as 0 whereas it should be an empty string.
Please change your AuthorizationHeader method code to the code mentioned below and your delete blob operation should work just fine:
private string AuthorizationHeader(string method, HttpWebRequest request, string containerName, string blobName)
{
string urlResource = $"/{_accountName}/{containerName}/{blobName}";
var contentLength = request.ContentLength == 0 ? "" : request.ContentLength.ToString();
string stringToSign = $"{method}\n\n\n{contentLength}\n\n{request.ContentType}\n\n\n\n\n\n\n{GetCanonicalizedHeaders(request)}{GetCanonicalizedResource(request.RequestUri, _accountName)}";
HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(_accountKey));
string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
return string.Format("{0} {1}:{2}", "SharedKey", _accountName, signature);
}
What I have done above is set the content length value to an empty string if the content length is 0.
From this link (emphasis mine):
In the current version, the Content-Length field must be an empty
string if the content length of the request is zero. In version
2014-02-14 and earlier, the content length was included even if zero.

Problems with generating SAS manually for blob file

I'm trying to make a downloader attached to the service bus that is going to download the files from blob storage. But I'm having some problems with generating the SAS token manually, please se the error message below.
I'm getting error <AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail>
private static string createToken(string resourceUri, string keyName, string key)
{
var now = DateTime.UtcNow;
TimeSpan sinceEpoch = now - new DateTime(1970, 1, 1);
var time = 60 * 2;
var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + time);
var expiryDateString = now.AddSeconds(time).ToUniversalTime().ToString("u");
string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
var sasToken = String.Format(
CultureInfo.InvariantCulture,
"{0}?sp={1}&st={2}&se={3}&spr={4}&sv={5}&sr={6}&sig={7}",
resourceUri,
"r",
DateTime.UtcNow.ToUniversalTime().ToString("u"),
expiryDateString,
"https",
"2019-02-02",
"b",
HttpUtility.UrlEncode(signature));
return sasToken;
}
Is it the stringToSign? Or signature as a whole? I'm a but unusre, to maybe I need to use the HttpUtility.UrlEncoder on all fields?
Please try the code below, it generates a working sas token for me:
static void Main(string[] args)
{
string resourceUri = "https://xx.blob.core.windows.net/test1/1.txt";
string account_name= "storage account name";
string key = "storage account key";
string s1 = createToken(resourceUri,account_name,key);
Console.WriteLine(s1);
Console.ReadLine();
}
private static string createToken(string resourceUri, string account_name, string key)
{
var accountName = account_name;
var accountKey = key;
var start = DateTime.UtcNow.AddHours(-2).ToString("yyyy-MM-ddTHH:mm:ssZ");
var end = DateTime.UtcNow.AddHours(2).ToString("yyyy-MM-ddTHH:mm:ssZ");
var permission = "rwdlac";
var serviceType = "b";
var resourceTypes = "sco";
var protocol = "https";
var serviceVersion = "2019-02-02";
//here is the difference from your code.
var stringToSign = string.Format("{0}\n{1}\n{2}\n{3}\n{4}\n{5}\n{6}\n{7}\n{8}\n",accountName, permission,serviceType,resourceTypes,start,end,"",protocol,serviceVersion);
HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(accountKey));
string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
var sasToken = string.Format("?sv={0}&ss={1}&srt={2}&sp={3}&se={4}&st={5}&spr={6}&sig={7}", serviceVersion,
serviceType, resourceTypes, permission, end, start, protocol, HttpUtility.UrlEncode(signature));
var urlToListTables = resourceUri + sasToken;
return urlToListTables;
}
Please let me know if you still have any issues.

Google Docs API - Impersonate User File Download

Using the Google Docs Java API with a Google Apps account, is it possible to impersonate a user and download a file?
When I run the program below, it is clearly logging on to the domain and impersonating the user because it retrieves the details of one of the files and prints out the file title. However, when it tries to download the file, a ServiceForbiddenException is thrown.
If it is not possible with the Java API, does anyone know if it is possible for my program to write an HTTP request to download the file using the Protocol API?
public class AuthExample {
private static DocsService docService = new DocsService("Auth Example");
public static void main(String[] args)
throws Exception
{
String adminUser = args[0];
String adminPassword = args[1];
String authToken = args[2];
String impersonatedUser = args[3];
loginToDomain(adminUser, adminPassword, authToken);
URL url = new URL( "https://docs.google.com/feeds/" + impersonatedUser + "/private/full" );
DocumentListFeed feed = docService.getFeed(url, DocumentListFeed.class);
DocumentListEntry entry = feed.getEntries().get(0);
String title = entry.getTitle().getPlainText();
System.out.println( title );
String type = entry.getType();
if ( type.equals("document") )
{
String encodedAdminUser = URLEncoder.encode(adminUser);
String resourceId = entry.getResourceId();
String resourceIdNoPrefix = resourceId.substring( resourceId.indexOf(':')+1 );
String downloadUrl =
"https://docs.google.com/feeds/download/documents/Export" +
"?xoauth_requestor=" + encodedAdminUser +
"&docId=" + resourceIdNoPrefix +
"&exportFormat=doc";
downloadFile( downloadUrl, title + ".doc" );
}
}
private static void loginToDomain(String adminUser, String adminPassword, String authToken)
throws OAuthException, AuthenticationException
{
String domain = adminUser.substring( adminUser.indexOf('#')+1 );
GoogleOAuthParameters oauthParameters = new GoogleOAuthParameters();
oauthParameters.setOAuthConsumerKey(domain);
oauthParameters.setOAuthConsumerSecret(authToken);
oauthParameters.setOAuthType(OAuthType.TWO_LEGGED_OAUTH);
oauthParameters.setScope("https://docs.google.com/feeds/ http://spreadsheets.google.com/feeds/ http://docs.googleusercontent.com/");
docService.useSsl();
docService.setOAuthCredentials(oauthParameters, new OAuthHmacSha1Signer());
docService.setUserCredentials(adminUser, adminPassword);
}
// Method pasted directly from Google documentation
public static void downloadFile(String exportUrl, String filepath)
throws IOException, MalformedURLException, ServiceException
{
System.out.println("Exporting document from: " + exportUrl);
MediaContent mc = new MediaContent();
mc.setUri(exportUrl);
MediaSource ms = docService.getMedia(mc);
InputStream inStream = null;
FileOutputStream outStream = null;
try {
inStream = ms.getInputStream();
outStream = new FileOutputStream(filepath);
int c;
while ((c = inStream.read()) != -1) {
outStream.write(c);
}
} finally {
if (inStream != null) {
inStream.close();
}
if (outStream != null) {
outStream.flush();
outStream.close();
}
}
}
}
Impersonation will work as intended if you use Oauth2 with ServiceAccounts

getting 403 forbidden webexception for azure blob put request

I am using this code to upload some text directly to azure blob using rest api. I am getting an webexception 403 forbidden. Can someone plaese tell me where am i going wrong in my code
private String CreateAuthorizationHeader(String canonicalizedString, CloudBlob blob)
{
String signature = string.Empty;
using (HMACSHA256 hmacSha256 = new HMACSHA256())
{
Byte[] dataToHmac = System.Text.Encoding.UTF8.GetBytes(canonicalizedString);
signature = Convert.ToBase64String(hmacSha256.ComputeHash(dataToHmac));
}
String authorizationHeader = String.Format(CultureInfo.InvariantCulture, "{0} {1}:{2}", "SharedKeyLite", myaccount, signature);
return authorizationHeader;
}
private void PutBlob(String containerName, String blobName , CloudBlob blob)
{
String requestMethod = "PUT";
String urlPath = String.Format("{0}", blobName);
String storageServiceVersion = "2009-10-01";
String dateInRfc1123Format = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture);
// if (uploadPST.HasFile)
// {
string content = "Sample file";
// Stream content = uploadPST.FileBytes;
UTF8Encoding utf8Encoding = new UTF8Encoding();
Byte[] blobContent = utf8Encoding.GetBytes(content);
Int32 blobLength = blobContent.Length;
const String blobType = "BlockBlob";
/* String canonicalizedHeaders = String.Format(
"x-ms-blob-type:{0}\nx-ms-date:{1}\nx-ms-version:{2}",
blobType,
dateInRfc1123Format,
storageServiceVersion);*/
String canonicalizedHeaders = String.Format(
"x-ms-date:{0}\nx-ms-meta-m1:{1}\nx-ms-meta-m1:{2}",
dateInRfc1123Format,
"v1",
"v2");
String canonicalizedResource = String.Format("/{0}/{1}", myaccount, urlPath);
String stringToSign = String.Format(
"{0}\n\n{1}\n\n{2}\n{3}",
requestMethod,
"text/plain; charset=UTF-8",
canonicalizedHeaders,
canonicalizedResource);
String authorizationHeader = CreateAuthorizationHeader(stringToSign, blob);
Uri uri = new Uri(CloudStorageAccount.FromConfigurationSetting("DataConnectionString").BlobEndpoint + "/" + urlPath);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = requestMethod;
// request.Headers.Add("x-ms-blob-type", blobType);
request.Headers.Add("x-ms-date", dateInRfc1123Format);
// request.Headers.Add("x-ms-version", storageServiceVersion);
request.Headers.Add("Authorization", authorizationHeader);
request.ContentLength = blobLength;
using (Stream requestStream = request.GetRequestStream())
{
requestStream.Write(blobContent ,0 ,blobLength);
}
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
String ETag = response.Headers["ETag"];
}
// }
}
First, you construct an HMACSHA256 object using the default constructor -- this causes a random key to be generated and used for signing. What you want is the overload which accepts a string - and pass the azure account key.
Still, signing the request 'manually' can be tricky, as there are a lot of things to do and it's easy to mess up or forget something. Instead, I recommend you use the SignRequest method of StorageCredentialsAccountAndKey class (msdn doc) For instance;
// ...there exists a request object, and strings for the account name and key
var creds = new StorageCredentialsAccountAndKey(accountName, accountKey);
creds.SignRequest(request);
This will do everything needed to sign the request properly, including creating a canonicalized headers string, creating a date with the correct format, etc.

Resources