In Cosmos DB v3, I'm getting an IOrderedQueryable<T> using GetItemLinqQueryable<T>. This allows me to write custom queries. The problem is I'd like to track request charges whenever a query is materialized. How can this be accomplished?
When I execute methods like ReadItemAsyncand ExecuteStoredProcedureAsync, the returned object has a RequestCharge property, but I need to detect charges with linq queries.
You can use the extension method ToFeedIterator on your IOrderedQueryable.
using Microsoft.Azure.Cosmos.Linq;
var query = container.GetItemLinqQueryable<MyClass>()
.Where(c => true)
.ToFeedIterator();
while (query.HasMoreResults)
{
var response = await query.ReadNextAsync();
Console.WriteLine(response.RequestCharge);
foreach (var myClassInstance in response)
{
// do stuff
}
}
edit: if you need count or any aggregate function:
var query = container.GetItemLinqQueryable<MyClass>()
.Where(c => true);
Response<int> x = await query.CountAsync();
Console.WriteLine(x.RequestCharge);
int count = x; // Autoboxing
You can find the full list of available extension functions on GitHub.
Related
I have a Saved Search based on Web Services Operations. I'm trying to access the data through a RESTlet and I have a working script deployment. This thing I can't figure out is the correct search type. I've tried a whole load of combinations but still get this error:
[code] => INVALID_RCRD_TYPE
[message] => The record type [WEBSERVICESOPERATIONS] is invalid.
Does anyone know the correct datatype I need to use. Perhaps there is a way of listing them all? Here is my RESTlet:
function getRESTlet(dataIn) {
var message = "";
var status = "OK";
var data = {};
var savedSearch = nlapiLoadSearch('WebServicesOperations', 'customsearch_mysearch');
var resultset = savedSearch.runSearch();
var returnSearchResults = [];
var searchid = 0;
do {
var resultslice = resultset.getResults(searchid, searchid + 1000);
for ( var rs in resultslice) {
returnSearchResults.push(resultslice[rs]);
searchid++;
}
} while (resultslice.length >= 1000);
return returnSearchResults;
} // Close function
Valid Scriptable/Searchable record types would be found in the Records Browser. Not all Record Types are available to be scripted or searched via scripting.
Web Services Operations does not appear to be supported in scripting.
I am using a User Defined Function across multiple collections within multiple Cosmos databases. Is there a way to store it somewhere and deploy it to all of these collections/databases at once? Or a way to update them all at the same time? Currently I am having to go through and manually update each UDF within each collection within each database.
You can write console application for updating UDF-
private async Task<string> CreateUDFAsync(string collectionUri, string udfName, string udfBody)
{
ResourceResponse<UserDefinedFunction> response = null;
try
{
var existingUdf = await this.cosmosDbClient.ReadUserDefinedFunctionAsync($"{collectionUri}/udfs/{udfName}");
existingUdf.Resource.Body = udfBody;
response = await this.cosmosDbClient.ReplaceUserDefinedFunctionAsync(existingUdf.Resource);
}
catch (DocumentClientException ex)
{
response = await this.cosmosDbClient.CreateUserDefinedFunctionAsync(collectionUri,
new UserDefinedFunction
{
Id = udfName,
Body = udfBody
});
}
return response.Resource.AltLink;
}
It will replace existing UDF and Create new in case missing
At the Cosmos DB resource model structure stored procedures, UDFs, merge procedures, triggers and conflicts are Container level resources.
You have to create them for each container.
I want to retrieve data from Kusto DB from c# app can any one help me on this.
I have knowledge on writing the Kusto queries but I need some help on pulling data from Azure Kusto DB hosted in Azure.
I tried the following code but it's not working:
var client = Kusto.Data.Net.Client.KustoClientFactory.CreateCslQueryProvider("https://help.kusto.windows.net/Samples;Fed=true");
var reader = client.ExecuteQuery("MyTable | count");
// Read the first row from reader -- it's 0'th column is the count of records in MyTable
// Don't forget to dispose of reader when done.
Could you please elaborate what's not working (what is the error message you're getting) with the code above?
In addition, a full (though simple) example can be found below:
// This sample illustrates how to query Kusto using the Kusto.Data .NET library.
//
// For the purpose of demonstration, the query being sent retrieves multiple result sets.
//
// The program should execute in an interactive context (so that on first run the user
// will get asked to sign in to Azure AD to access the Kusto service).
class Program
{
const string Cluster = "https://help.kusto.windows.net";
const string Database = "Samples";
static void Main()
{
// The query provider is the main interface to use when querying Kusto.
// It is recommended that the provider be created once for a specific target database,
// and then be reused many times (potentially across threads) until it is disposed-of.
var kcsb = new KustoConnectionStringBuilder(Cluster, Database)
.WithAadUserPromptAuthentication();
using (var queryProvider = KustoClientFactory.CreateCslQueryProvider(kcsb))
{
// The query -- Note that for demonstration purposes, we send a query that asks for two different
// result sets (HowManyRecords and SampleRecords).
var query = "StormEvents | count | as HowManyRecords; StormEvents | limit 10 | project StartTime, EventType, State | as SampleRecords";
// It is strongly recommended that each request has its own unique
// request identifier. This is mandatory for some scenarios (such as cancelling queries)
// and will make troubleshooting easier in others.
var clientRequestProperties = new ClientRequestProperties() { ClientRequestId = Guid.NewGuid().ToString() };
using (var reader = queryProvider.ExecuteQuery(query, clientRequestProperties))
{
// Read HowManyRecords
while (reader.Read())
{
var howManyRecords = reader.GetInt64(0);
Console.WriteLine($"There are {howManyRecords} records in the table");
}
// Move on to the next result set, SampleRecords
reader.NextResult();
Console.WriteLine();
while (reader.Read())
{
// Important note: For demonstration purposes we show how to read the data
// using the "bare bones" IDataReader interface. In a production environment
// one would normally use some ORM library to automatically map the data from
// IDataReader into a strongly-typed record type (e.g. Dapper.Net, AutoMapper, etc.)
DateTime time = reader.GetDateTime(0);
string type = reader.GetString(1);
string state = reader.GetString(2);
Console.WriteLine("{0}\t{1,-20}\t{2}", time, type, state);
}
}
}
}
}
My Stored Procedure: (I created it via Azure Script Explorer)
function GetAllResources() {
var collection = getContext().getCollection();
// Query documents and take 1st item.
var isAccepted = collection.queryDocuments(
collection.getSelfLink(),
'SELECT * FROM MultiLanguage as m',
function (err, docs, options) {
if (err) throw err;
// Check the feed and if empty, set the body to 'no docs found',
// else take 1st element from feed
if (!docs || !docs.length) getContext().getResponse().setBody('no docs found');
else getContext().getResponse().setBody(JSON.stringify(docs));
});
if (!isAccepted) throw new Error('The query was not accepted by the server.');
}
The sproc can be executed successfully from script explorer.
My C# code to call the sproc:
public async Task<IHttpActionResult> GetReources() {
client = new DocumentClient(new Uri(ConfigurationManager.AppSettings["endpoint"]), ConfigurationManager.AppSettings["authKey"]);
var collectionLink = UriFactory.CreateDocumentCollectionUri(DatabaseId, CollectionId);
//var docs = await client.ReadDocumentFeedAsync(collectionLink, new FeedOptions { MaxItemCount = 10 });
//var docs = from d in client.CreateDocumentQuery<Models.Resource>(collectionLink)
// select d;
StoredProcedure storedProcedure = client.CreateStoredProcedureQuery(collectionLink).Where(c => c.Id == "GetAllResources").AsEnumerable().FirstOrDefault();
Models.Resource docs = await client.ExecuteStoredProcedureAsync<Models.Resource>(storedProcedure.SelfLink);
foreach (var d in docs) {
Models.Resource a = new Models.Resource();
a = docs;
//a.id = d.id;
//a.Scenario = d.Scenario;
//a.Translations = d.Translations;
//a.LastModified = d.LastModified;
//a.ModifiedBy = d.ModifiedBy;
//a.LastAccessed = d.LastAccessed;
resources.Add(a);
}
return Ok(resources);
}
First, there is an error for the "foreach..." like said
foreach cannot operate on variables of type Models.Resource because it
doesn't contain a public definition of GetEnumerator.
Then I tried to modify my sproc to only return 1 result and remove the foreach line, then I got error said
Failed to deserialize stored procedure response or convert it to type
'Models.Resource'
I just want to return the result of the stored procedure as my defined class (Models.Resource). How to do this?
It can be simpler to get sproc by name using CreateStoredProcedureUri, like this:
const string endpoint = "https://your.service.azure.com:443/";
const string authKey = "<your magic secret master key>==";
var client = new DocumentClient(new Uri(endpoint), authKey);
Uri sprocUri = UriFactory.CreateStoredProcedureUri("databaseName", "collectionName", "GetAllResources");
var result = await client.ExecuteStoredProcedureAsync<string>(sprocUri);
The stored procedure above serializes results of the query (docs array) to string, if you keep it this way, the result of sproc would be string, which I guess you would need to manually deserialize to objects. You can do this simpler, just return docs from sproc and have result as objects (like Models.Resource[]), serialization would happen automatically.
If you change the sproc to return just one doc (e.g. do __.response.setBody(docs[0]) and Models.Resource represent one item, then the call is correct:
Models.Resource doc = await client.ExecuteStoredProcedureAsync<Models.Resource>(sprocUri);
Also, to // Query documents and take 1st item, I wouldn't recommend to use script as script has overhead of running JavsScript engine. Scripts kick in when you have bulk operations (to optimize for network traffic) or have business logic which makes sense to run on the server. To take 1st item you can do query from client like this: SELECT TOP 1 * FROM c. Typically you would WHERE and ORDER BY clause to that.
There is a number of docdb samples on github, for instance, https://github.com/Azure/azure-documentdb-dotnet/tree/master/samples/code-samples/ServerSideScripts and https://github.com/Azure/azure-documentdb-dotnet/tree/master/samples/code-samples/Queries.
Thanks,
Michael
All right, let's make sure we are on the same page.
I am using the sproc same as above.
I am using client code like this:
class Models
{
// This would have more properties, I am just using id which all docs would have.
public class Resource
{
[JsonProperty("id")]
public string Id { get; set; }
}
}
public async Task<IHttpActionResult> GetResources()
{
const string endpoint = "https://myservice.azure.com:443/";
const string authKey = "my secret key==";
var client = new DocumentClient(new Uri(endpoint), authKey);
Uri sprocUri = UriFactory.CreateStoredProcedureUri("db", "c1", "GetAllResources");
var serializedDocs = await client.ExecuteStoredProcedureAsync<string>(sprocUri);
Models.Resource[] resources = JsonConvert.DeserializeObject<Models.Resource[]>(serializedDocs);
return Ok(resources);
}
It works fine. Is this what you are doing?
I'm seeing a lot of exceptions in the collectionSelfLink when making DocumentDb call -- see image below.
I'm able to connect to DocumentDb and read data but these exceptions concern me -- especially in something that's pretty straight forward like a collectionSelfLink.
Any idea what may be causing them and how to fix them?
Here's the function that's using the selfLink
public async Task<IEnumerable<T>> ReadQuery<T>(string dbName, string collectionId, SqlQuerySpec query)
{
// Prepare collection self link
// IMPORTANT: This is where I'm seeing those exceptions when I inspect the collectionLink. Otherwise, I'm NOT getting any errors.
var collectionLink = UriFactory.CreateDocumentCollectionUri(dbName, collectionId);
var result = _client.CreateDocumentQuery<T>(collectionLink, query, null);
_client.CreateDocumentQuery<T>(collectionLink);
return await result.QueryAsync();
}
And here's the QueryAsync() extension method
public async static Task<IEnumerable<T>> QueryAsync<T>(this IQueryable<T> query)
{
var docQuery = query.AsDocumentQuery();
var batches = new List<IEnumerable<T>>();
do
{
var batch = await docQuery.ExecuteNextAsync<T>();
batches.Add(batch);
}
while (docQuery.HasMoreResults);
var docs = batches.SelectMany(b => b);
return docs;
}
So SelfLink is an internal property that is set by DocumentDB. It cannot be set by the user and will only be populated on resources that have been returned from a call to the server.
The UriFactory code that you are using is construction a link that can be used to execute operations, but it is not a SelfLink.
If you are looking at a SelfLink property on a newly initialized DocumentCollection() object the SelfLink will be null as it has not been persisted on the server yet. This would explain all those errors in debug watch.