How to escape characters in Azure Table queries? - azure

I want to query for rows where a column named message starts with: metric="foo"
I tried encoding = and " with percentage and hex codes but it did not work.
Microsoft documentations says special characters must be encoded but does not tell how: https://learn.microsoft.com/en-us/rest/api/storageservices/querying-tables-and-entities#query-string-encoding
What should the query look like when value being compared contains special characters?

If you're using azure sdk, then the sdk already did the tricky for you.
In my test, I'm using the latest azure table storage sdk Microsoft.Azure.Cosmos.Table, version 1.0.4.
The test code:
static void Main(string[] args)
{
string connstr = "xxxx";
var storageAccount = CloudStorageAccount.Parse(connstr);
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("myCustomer123");
TableQuery<CustomerEntity> query = new TableQuery<CustomerEntity>();
string myfilter = TableQuery.CombineFilters(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "ivan"),
TableOperators.And,
//for metric="foo", like below.
TableQuery.GenerateFilterCondition("PhoneNumber", QueryComparisons.Equal, "metric=\"foo\"")
);
query.FilterString = myfilter;
var items = table.ExecuteQuery(query);
foreach (var item in items)
{
Console.WriteLine(item.RowKey);
Console.WriteLine(item.PhoneNumber);
}
Console.WriteLine("*****end******");
Console.ReadLine();
}
Test result:

If you want to use the parameter to filter results, you can use ?$filter=<your parameter>%20eq%20'<vaule>'. For example
var date = DateTime.Now.ToUniversalTime().AddYears(1).ToString("R");
var CanonicalizedResource = "/" + StorageAccountName + "/people";
var StringToSign = date + "\n" + CanonicalizedResource;
// List the containers in a storage account.
// ListContainersAsyncREST(StorageAccountName, StorageAccountKey, CancellationToken.None).GetAwaiter().GetResult();
var hmacsha = new HMACSHA256();
hmacsha.Key = Convert.FromBase64String(StorageAccountKey);
var sig= hmacsha.ComputeHash(UTF8Encoding.UTF8.GetBytes(StringToSign));
var sig1 = Convert.ToBase64String(sig);
Console.WriteLine(sig1);
String uri = "https://jimtestperfdiag516.table.core.windows.net/people" + "?$filter=PartitionKey%20eq%20'Jim'";
HttpClient client = new HttpClient();
var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, uri);
httpRequestMessage.Headers.Add("x-ms-date", date);
var str = "SharedKeyLite " + StorageAccountName + ":" + sig1;
httpRequestMessage.Headers.TryAddWithoutValidation("Authorization", str);
httpRequestMessage.Headers.Add("x-ms-version", "2017-04-17");
httpRequestMessage.Headers.Add("Accept", "application/json;odata=fullmetadata");
var results = client.SendAsync(httpRequestMessage).Result;
var response = results.Content.ReadAsStringAsync().Result;
var objs = JsonConvert.DeserializeObject(response);
Console.WriteLine(objs);

Related

How do we pass cosmos DB partition key definition while creating a cosmos DB SQL Container using Azure.ResourceManager.CosmosDB 1.0.1

I am trying to create a cosmos db partition using the below sample. I am using managed identity to authenticate my requests.
I am using Azure.ResourceManager v1.3.1 and Azure.ResourceManager.CosmosDB v1.0.1
While trying to run below snippet, I am getting:
Azure.RequestFailedException: 'Message: {"code":"BadRequest","message":"Message:
{"Errors":
["A Partition key definition is not specified in the request."]}
I am using .NET Framework 4.7.2
I am not sure how we to pass the partition key definition while creating a cosmos DB SQL Container. Can anyone please help here. Thanks.
private static async Task TestStub1()
{
var subscriptionId = "xxx";
var resourceGroupName = "xxx";
var accountName = "xxx";
var databaseName = "xxx";
var containerName = "xxx";
var throughputProperties = ThroughputProperties.CreateAutoscaleThroughput(4000);
var accountEndpoint = "xxx";
var location = AzureLocation.EastUS;
try
{
var tokenCredential = new DefaultAzureCredential();
var armClient = new ArmClient(tokenCredential);
var dbAccountIdentifier = new ResourceIdentifier($"/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DocumentDB/databaseAccounts/{accountName}");
var dbAccount = armClient.GetCosmosDBAccountResource(dbAccountIdentifier);
var databases = dbAccount.GetCosmosDBSqlDatabases();
var cosmosDBSqlDatabaseResourceInfo = new CosmosDBSqlDatabaseResourceInfo(databaseName);
var cosmosDBSqlDatabaseCreateOrUpdateContent = new CosmosDBSqlDatabaseCreateOrUpdateContent(location, cosmosDBSqlDatabaseResourceInfo);
var cosmosDBSqlResource = await databases.CreateOrUpdateAsync(Azure.WaitUntil.Completed, databaseName, cosmosDBSqlDatabaseCreateOrUpdateContent);
if (cosmosDBSqlResource.HasValue)
{
var cosmosDBSqlContainers = cosmosDBSqlResource.Value.GetCosmosDBSqlContainers();
var cosmosDBContainerPartitionKey = new CosmosDBContainerPartitionKey();
cosmosDBContainerPartitionKey.Kind = CosmosDBPartitionKind.Hash;
var cosmosDBSqlContainerResourceInfo = new CosmosDBSqlContainerResourceInfo(containerName);
cosmosDBSqlContainerResourceInfo.PartitionKey = cosmosDBContainerPartitionKey;
var cosmosDBSqlContainerCreateOrUpdateContent = new CosmosDBSqlContainerCreateOrUpdateContent(location, cosmosDBSqlContainerResourceInfo);
var cosmosDBSqlContainer = await cosmosDBSqlContainers.CreateOrUpdateAsync(WaitUntil.Completed, containerName, cosmosDBSqlContainerCreateOrUpdateContent);
}
}
catch (Exception ex)
{
throw;
}
}
This is a sample for Microsoft.Azure.Management.CosmosDB but should be the same.
Assuming a string parameter of partitionKey this is how you create a partition key definition.
PartitionKey = new ContainerPartitionKey
{
Kind = "Hash",
Paths = new List<string> { partitionKey },
Version = 1 //version 2 for large partition key
},
More samples like this at Azure Management Libraries for .NET for Azure Cosmos DB

Problems with generating SAS manually for blob file

I'm trying to make a downloader attached to the service bus that is going to download the files from blob storage. But I'm having some problems with generating the SAS token manually, please se the error message below.
I'm getting error <AuthenticationErrorDetail>Signature fields not well formed.</AuthenticationErrorDetail>
private static string createToken(string resourceUri, string keyName, string key)
{
var now = DateTime.UtcNow;
TimeSpan sinceEpoch = now - new DateTime(1970, 1, 1);
var time = 60 * 2;
var expiry = Convert.ToString((int)sinceEpoch.TotalSeconds + time);
var expiryDateString = now.AddSeconds(time).ToUniversalTime().ToString("u");
string stringToSign = HttpUtility.UrlEncode(resourceUri) + "\n" + expiry;
HMACSHA256 hmac = new HMACSHA256(Encoding.UTF8.GetBytes(key));
var signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
var sasToken = String.Format(
CultureInfo.InvariantCulture,
"{0}?sp={1}&st={2}&se={3}&spr={4}&sv={5}&sr={6}&sig={7}",
resourceUri,
"r",
DateTime.UtcNow.ToUniversalTime().ToString("u"),
expiryDateString,
"https",
"2019-02-02",
"b",
HttpUtility.UrlEncode(signature));
return sasToken;
}
Is it the stringToSign? Or signature as a whole? I'm a but unusre, to maybe I need to use the HttpUtility.UrlEncoder on all fields?
Please try the code below, it generates a working sas token for me:
static void Main(string[] args)
{
string resourceUri = "https://xx.blob.core.windows.net/test1/1.txt";
string account_name= "storage account name";
string key = "storage account key";
string s1 = createToken(resourceUri,account_name,key);
Console.WriteLine(s1);
Console.ReadLine();
}
private static string createToken(string resourceUri, string account_name, string key)
{
var accountName = account_name;
var accountKey = key;
var start = DateTime.UtcNow.AddHours(-2).ToString("yyyy-MM-ddTHH:mm:ssZ");
var end = DateTime.UtcNow.AddHours(2).ToString("yyyy-MM-ddTHH:mm:ssZ");
var permission = "rwdlac";
var serviceType = "b";
var resourceTypes = "sco";
var protocol = "https";
var serviceVersion = "2019-02-02";
//here is the difference from your code.
var stringToSign = string.Format("{0}\n{1}\n{2}\n{3}\n{4}\n{5}\n{6}\n{7}\n{8}\n",accountName, permission,serviceType,resourceTypes,start,end,"",protocol,serviceVersion);
HMACSHA256 hmac = new HMACSHA256(Convert.FromBase64String(accountKey));
string signature = Convert.ToBase64String(hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign)));
var sasToken = string.Format("?sv={0}&ss={1}&srt={2}&sp={3}&se={4}&st={5}&spr={6}&sig={7}", serviceVersion,
serviceType, resourceTypes, permission, end, start, protocol, HttpUtility.UrlEncode(signature));
var urlToListTables = resourceUri + sasToken;
return urlToListTables;
}
Please let me know if you still have any issues.

Create Folder and Update Title and Custom Field in Sharepoint Online

I try to create folder in sharepoint online, based on some of tutorial. the problem comes since creating folder is not given "Title" column value.
I want to create folder and also update column "Title".
here is the code for create folder
public string CreateDocumentLibrary(string siteUrl, string relativePath)
{
//bool responseResult = false;
string resultUpdate = string.Empty;
string responseResult = string.Empty;
if (siteUrl != _siteUrl)
{
_siteUrl = siteUrl;
Uri spSite = new Uri(siteUrl);
_spo = SpoAuthUtility.Create(spSite, _username, WebUtility.HtmlEncode(_password), false);
}
string odataQuery = "_api/web/folders";
byte[] content = ASCIIEncoding.ASCII.GetBytes(#"{ '__metadata': { 'type': 'SP.Folder' }, 'ServerRelativeUrl': '" + relativePath + "'}");
string digest = _spo.GetRequestDigest();
Uri url = new Uri(String.Format("{0}/{1}", _spo.SiteUrl, odataQuery));
// Set X-RequestDigest
var webRequest = (HttpWebRequest)HttpWebRequest.Create(url);
webRequest.Headers.Add("X-RequestDigest", digest);
// Send a json odata request to SPO rest services to fetch all list items for the list.
byte[] result = HttpHelper.SendODataJsonRequest(
url,
"POST", // reading data from SP through the rest api usually uses the GET verb
content,
webRequest,
_spo // pass in the helper object that allows us to make authenticated calls to SPO rest services
);
string response = Encoding.UTF8.GetString(result, 0, result.Length);
if (response != null)
{
//responseResult = true;
responseResult = response;
}
return responseResult;
}
I already tried to use CAML, but, the problem is, the list of sharepoint is big, so got the error prohibited access related to limit tresshold.
Please help.
Refer below code to update folder name.
function renameFolder(webUrl,listTitle,itemId,name)
{
var itemUrl = webUrl + "/_api/Web/Lists/GetByTitle('" + listTitle + "')/Items(" + itemId + ")";
var itemPayload = {};
itemPayload['__metadata'] = {'type': getItemTypeForListName(listTitle)};
itemPayload['Title'] = name;
itemPayload['FileLeafRef'] = name;
var additionalHeaders = {};
additionalHeaders["X-HTTP-Method"] = "MERGE";
additionalHeaders["If-Match"] = "*";
return executeJson(itemUrl,"POST",additionalHeaders,itemPayload);
}
function getItemTypeForListName(name) {
return"SP.Data." + name.charAt(0).toUpperCase() + name.slice(1) + "ListItem";
}

How to know in C# size of one container in windows azure?

How to know the size of a container in azure, using C # language. I have hundreds of terabytes of files, so I do not work recursively. I tried with the following code.
var key = Util.GetPassword("Storage Account Key");
var accountName = "mystorageaccount";
var connectionString = "DefaultEndpointsProtocol=https;AccountName="
+ accountName + ";AccountKey=" + key;
var account = CloudStorageAccount.Parse(connectionString);
var blobClient = account.CreateCloudBlobClient();
var blobContainer = blobClient.GetContainerReference("mycontainer");
long fileSize = 0;
foreach (CloudBlockBlob blobItem in blobContainer.ListBlobs())
{
fileSize += blobItem.Properties.Length;
}
fileSize.Dump();
Here are the codes you are looking for...
string key = "";
string accountName = "mystorageaccount";
string connectionString = "DefaultEndpointsProtocol=https;AccountName="
+ accountName + ";AccountKey=" + key;
var account = CloudStorageAccount.Parse(connectionString);
var blobClient = account.CreateCloudBlobClient();
var blobContainer = blobClient.GetContainerReference("mycontainer");
long fileSize = 0;
foreach(CloudBlobContainer container in blobClient.ListContainers())
{
fileSize += (from ICloudBlob blob in
container.ListBlobs(useFlatBlobListing: true)
select blob.Properties.Length).Sum();
}

Twitter request token returns 401 unauthorized. I cant find what I did wrong

I need to do twitter sign in for mobile apps. Thus far I have tried a lot of libraries including tweetsharp and linq2twitter. But I am getting nowhere. With all this trouble, I resorted to webclient calls, but I am once again stuck. When I make the call, I get the 401 Unauthorized error. I include my code here. Can you guys see anything wrong at all ? I am really stuck with this.
string oauth_signature_method = "HMAC-SHA1";
TimeSpan ts = DateTime.UtcNow - new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc);
string oauth_timestamp = Convert.ToInt64(ts.TotalSeconds).ToString();
string oauth_version = "1.0";
string oauth_consumer_key = "iD1232134AQ2Pb6Q";
string oauth_nonce = Convert.ToBase64String(new ASCIIEncoding().GetBytes(DateTime.Now.Ticks.ToString()));
SortedDictionary<string, string> sd = new SortedDictionary<string, string>();
sd.Add("oauth_version", oauth_version);
sd.Add("oauth_consumer_key", oauth_consumer_key);
sd.Add("oauth_nonce", oauth_nonce);
sd.Add("oauth_signature_method", oauth_signature_method);
sd.Add("oauth_timestamp", oauth_timestamp);
UrlEntity callback = new UrlEntity();
callback.Url = #"//tweeter";
string encodedCallbackUrl = HttpUtility.UrlEncode(callback.Url);
sd.Add("oauth_callback",encodedCallbackUrl);
WebClient wc = new WebClient();
wc.Headers.Add("User-Agent: randomAgent HTTP Client");
wc.Headers.Add("Host: api.twitter.com");
wc.Headers.Add(#"Accept: */*");
UrlEntity url = new UrlEntity();
url.Url = #"https://api.twitter.com/oauth/request_token";
string signature = CreateSignature(url, sd);
sd.Add("oauth_signature",signature);
string dataValues = "";
foreach (KeyValuePair<string, string> pair in sd)
{
dataValues += pair.Key + "=''" + pair.Value + "'',";
}
dataValues = dataValues.Substring(0, dataValues.Length - 1); // cuts off the last,
string headerVal = " Oauth " + dataValues;
wc.Headers.Add("Authorization",headerVal);
wc.UploadString(#"https://api.twitter.com/oauth/request_token", "");
wc.DownloadStringCompleted += WcOnDownloadStringCompleted;
Below is the code to make the signature.
public static string CreateSignature(UrlEntity url, SortedDictionary<string, string> sd)
{
string parameterString = "";
SortedDictionary<string, string> sd2 = new SortedDictionary<string, string>();
foreach (KeyValuePair<string, string> parameter in sd)
{
string encodedKey = HttpUtility.UrlEncode(parameter.Key);
string encodedvalue = HttpUtility.UrlEncode(parameter.Value);
sd2.Add(encodedKey,encodedvalue);
}
foreach (KeyValuePair<string, string> pair in sd2)
{
parameterString += pair.Key + "=" + pair.Value + "&";
}
parameterString = parameterString.Substring(0, parameterString.Length - 1); // cuts off the last &
string signatureString = "POST&";
var encodedUrl = HttpUtility.UrlEncode(url.Url); // percent encode URL
signatureString += encodedUrl + "&";
signatureString += HttpUtility.UrlEncode(parameterString); // percent encode entire parameter string
string consumerSecret = "supercorectConsumersecretfromtwitterwebsite";
string signingKey = HttpUtility.UrlEncode(consumerSecret) + "&";
HMACSHA1 hasher = new HMACSHA1(new ASCIIEncoding().GetBytes(signingKey));
string signatureStringFinal = Convert.ToBase64String(hasher.ComputeHash(new ASCIIEncoding().GetBytes(signatureString)));
return signatureStringFinal;
}
It is a bit messy, but I am stuck. The following links might help.
https://dev.twitter.com/docs/auth/implementing-sign-twitter
https://dev.twitter.com/docs/api/1/post/oauth/request_token
https://dev.twitter.com/docs/auth/creating-signature
EDIT:
I have taken a look at the request using fiddler.
The response message has this in it...
"Failed to validate oauth signature and token"
Which likely means the oauth signature is wrong. But I have looked and I have not seen anything.
I am not a .net expert but am setting up something similar for PHP and note that the Oauth attributes must be in alphabetical order. From a quick look the examples here don't appear to be. See https://dev.twitter.com/docs/auth/creating-signature.

Resources