Querying DocumentDB for a list of propertiers using Linq's Select - azure

Using Azure's DocumentDb and the .NET API, I have the following method which works great for retrieving lists of entire documents:
public async Task<IEnumerable<T>> GetItemsAsync<T>(Expression<Func<T, bool>> predicate)
{
IDocumentQuery<T> query = _Client.CreateDocumentQuery<T>(
UriFactory.CreateDocumentCollectionUri(_DatabaseId, _Collection),
new FeedOptions { MaxItemCount = -1 })
.Where(predicate)
.AsDocumentQuery();
List<T> results = new List<T>();
while (query.HasMoreResults)
{
var item = await query.ExecuteNextAsync<T>();
results.AddRange(item);
}
return results;
}
Now, I don't always want to return the entire document (especially considering the DocumentDb RU pricing model), so I thought I should be able to add a .Select projection like so:
public async Task<List<TResult>> GetItemsAsync<T, TResult>(Expression<Func<T, bool>> predicate, Expression<Func<T, TResult>> select)
{
IDocumentQuery<TResult> query = _Client.CreateDocumentQuery<T>(
UriFactory.CreateDocumentCollectionUri(_DatabaseId, _Collection),
new FeedOptions { MaxItemCount = -1 })
.Where(predicate)
.Select(select)
.AsDocumentQuery();
List<TResult> results = new List<TResult>();
while (query.HasMoreResults)
{
var item = await query.ExecuteNextAsync<TResult>();
results.AddRange(item);
}
return results;
}
Usage:
var rez = await _docs.GetItemsAsync<ApolloAssetDoc, Guid?>(x => x.MyVal == 5, x => x.ID);
But the second method always return 0 results. Obviously I'm barking up the wrong tree.
Any idea what the correct way to return a list of either dynamic objects for queries where more than one property is selected (eg "SELECT d.id, d.MyVal FROM Items d WHERE d.DocType=0")or a simple list where only a single property is selected (eg "SELECT d.id FROM Items d WHERE d.DocType=0")?

I could repro the issue, if there is no [JsonProperty(PropertyName = "id")] for ID property in the entity Class. If it is not included, please have a try to use the following code:
public class ApolloAssetDoc
{
[JsonProperty(PropertyName = "id")]
public Guid ID { get; set; }
public string MyVal { get; set; }
}
Note: The field is case sensitive.

Related

Is there a way to iterate through the fields in a row of a PXResultSet?

Is it possible to use a foreach loop in a BLC to iterate through the fields of a PXResultSet to get the FieldNames?
Is this doable? I can't seem to find a good way.
Thanks...
The PXResultset records are selected from a view. You can get the field names from the View.
Here's a full example:
public class SOOrderEntry_Extension : PXGraphExtension<SOOrderEntry>
{
public override void Initialize()
{
// Get field list from data view
var dataView = new PXSelect<SOOrder>(Base);
string fieldNames = string.Join(",", GetFieldNames(dataView.View, Base.Caches));
// You don't need result set to get field names
PXResultset<SOOrder> resultSet = dataView.Select();
throw new PXException(fieldNames);
}
public string[] GetFieldNames(PXView view, PXCacheCollection caches)
{
var list = new List<string>();
var set = new HashSet<string>();
foreach (Type t in view.GetItemTypes())
{
if (list.Count == 0)
{
list.AddRange(caches[t].Fields);
set.AddRange(list);
}
else
{
foreach (string field in caches[t].Fields)
{
string s = String.Format("{0}__{1}", t.Name, field);
if (set.Add(s))
{
list.Add(s);
}
}
}
}
return list.ToArray();
}
}
When run, this example will show the fields names used in the data view in Sales Order screen SO301000 as an exception.
Field names are contained in Cache object. If you really need to get field names from PXResultset you need to iterate the cache types in the result set.
Example for first DacType (0) of result set:
public class SOOrderEntry_Extension : PXGraphExtension<SOOrderEntry>
{
public override void Initialize()
{
var dataView = new PXSelect<SOOrder>(Base);
PXResultset<SOOrder> resultSet = dataView.Select();
foreach (PXResult result in resultSet)
{
Type dacType = result.GetItemType(0);
foreach (var field in Base.Caches[dacType].Fields)
PXTrace.WriteInformation(field);
}
}
}

Azure Cosmos Db document - want to use replacedocumentasync with a removed property in the document

// I am using code like below
Document doc = client.CreateDocumentQuery<Document>(collectionLink)
.Where(r => r.Id == "doc id")
.AsEnumerable()
.SingleOrDefault();
doc.SetPropertyValue("MyProperty1", "updated value");
Document updated = await client.ReplaceDocumentAsync(doc);
Want to remove "MyProperty2" from the document. How to do this?
It seems that you'd like to update MyProperty1 property and remove MyProperty2 property from your document, the following sample code is for your reference.
private async Task updateDoc()
{
string EndpointUri = "xxxxx";
string PrimaryKey = "xxxxx";
DocumentClient client;
client = new DocumentClient(new Uri(EndpointUri), PrimaryKey);
Document doc = client.CreateDocumentQuery<Document>(UriFactory.CreateDocumentCollectionUri("testdb", "testcoll"))
.Where(r => r.Id == "doc5")
.AsEnumerable()
.SingleOrDefault();
//dynamically cast doc back to your MyPoco
MyPoco poco = (dynamic)doc;
//Update MyProperty1 of the poco object
poco.MyProperty1 = "updated value";
//replace document
Document updateddoc = await client.ReplaceDocumentAsync(doc.SelfLink, poco);
Console.WriteLine(updateddoc);
}
public class MyPoco
{
public string id { get; set; }
public string MyProperty1 { get; set; }
}
My document:
Updated:
Edit:
this would remove "MyProperty3" and "MyProperty4" as well.
As you mentioned, setting property with null would be also a approach.
Document doc = client.CreateDocumentQuery<Document>(UriFactory.CreateDocumentCollectionUri("testdb", "testcoll"))
.Where(r => r.Id == "doc5")
.AsEnumerable()
.SingleOrDefault();
doc.SetPropertyValue("MyProperty2", null);
//replace document
Document updateddoc = await client.ReplaceDocumentAsync(doc.SelfLink, doc);
Thanks for replying. You got it what I was asking for.
I did not want to use MyPoco as using Microsoft.Azure.Document is the most flexible. Also, using this approach would also remove any MyProperty3 if it exists with MyPoco.
Using Document class only, following worked:
eachDoc.SetPropertyValue("MyProperty2", null);

Using CreateDocumentQuery to Azure DocumentDB to pass both field and value to query a collection

Assume I have a collection, with many documents, and I want to query that collection , where a field has a particular value. However, I do NOT know ahead of time which field we will be querying.
Here is an example of what I am trying to accomplish:
Document azureDocuments = client.CreateDocumentQuery(UriFactory.CreateDocumentCollectionUri(sDatabaseName, sCollectionCollection)).Where(f => f.MyField=MyValue).AsEnumerable().FirstOrDefault();
Has anyone done this successfully ?
Using CreateDocumentQuery to Azure DocumentDB to pass both field and value to query a collection
You could try to dynamically generate a predicate base on the fieldname and fieldvalue that user provides, and do filter based on the predicate. The following sample code is for your reference.
private async Task QueryDocs(string fieldname, string value)
{
Func<MyDoc, bool> criteria = d => d.id == d.id;
switch (fieldname)
{
case "id":
criteria = d => d.id == value;
break;
case "name":
criteria = d => d.name == value;
break;
//other cases...
}
string EndpointUri = "https://mydocumentdbtest.documents.azure.com:443/";
string PrimaryKey = "{primary key}";
DocumentClient client;
client = new DocumentClient(new Uri(EndpointUri), PrimaryKey);
var query = client.CreateDocumentQuery<MyDoc>(UriFactory.CreateDocumentCollectionUri("{databaseid}", "{collectionid}")).Where(criteria);
List<MyDoc> list = query.ToList();
}
public class MyDoc
{
public string id { get; set; }
public string name { get; set; }
}

How to retrieve only a predefined number of results in Azure Tables

I am trying to perform an azure table query.
My table (that saves logs) has thousands of rows of data, and it gets populated with more each second.
Right now i have only 1 partition key, but it doesn't affect the next question.
How can i get back lets say only the 100 latest results.
this is my Entity:
public MainServerLogEntity(string Message)
{
this.PartitionKey = "mainserverlogs";
this.RowKey = (DateTime.MaxValue.Ticks - DateTime.UtcNow.Ticks).ToString();
this.Message = Message;
this.Date = DateTime.UtcNow;
}
public MainServerLogEntity() { }
public string Message { get; set; }
public DateTime Date { get; set; }
Right now this is the query i am performing inside a web api i have:
[Route("MainServerLogs")]
[HttpGet]
public IEnumerable<MainServerLogEntity> GetMainServerLogs()
{
CloudTable table = AzureStorageHelpers.GetWebApiTable(connectionString, "mainserverlogs");
TableQuery<MainServerLogEntity> query = new TableQuery<MainServerLogEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "mainserverlogs"));
return table.ExecuteQuery(query);
}
But the problem is that i am getting alot of data, and i am requesting this api every few seconds in order to update the ui.
What should i do? is it possible to define in the query that i only want the 100 first rows?
If it is not possible then what other technique should i use?
Try implementing a .Take(100) on the query like so:
[Route("MainServerLogs")]
[HttpGet]
public IEnumerable<MainServerLogEntity> GetMainServerLogs()
{
CloudTable table = AzureStorageHelpers.GetWebApiTable(connectionString, "mainserverlogs");
TableQuery<MainServerLogEntity> query = new TableQuery<MainServerLogEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "mainserverlogs")).Take(100);
return table.ExecuteQuery(query);
}

Does the Azure Storage Table query entities really has number limitations?

From MSDN, it seems there's a limitation for the number of entities returned by the Query service:
A query against the Table service may return a maximum of 1,000 entities at one time and may execute for a maximum of five seconds.
But as I wrote a sample to show this issue, I didn't find any limitations for the number of returned entities, here is my key code:
public class DataProvider
{
public static string PartitionKey
{
get { return "PartitionKey"; }
}
public static IEnumerable<CustomerEntity> MoreThanThousandData()
{
var result = new List<CustomerEntity>();
for (int i = 0; i < 1200; i++)
{
result.Add(new CustomerEntity(PartitionKey, Guid.NewGuid().ToString())
{
Name = Guid.NewGuid().ToString(),
Age = new Random().Next(10, 70)
});
}
return result;
}
}
Insert 1200 entities to the table:
public class AfterOptimize
{
public void InsertDataToTable()
{
var cloudData = DataProvider.MoreThanThousandData();
Console.WriteLine("Plan to insert {0} entities to the table.", cloudData.Count());
InsertDataToTableInternal(AzureTableService.Table, cloudData);
}
private void InsertDataToTableInternal(CloudTable table, IEnumerable<ITableEntity> data)
{
var splitedData = data.Chunk(100);
Parallel.ForEach(splitedData, item =>
{
var batchInsertOperation = new TableBatchOperation();
foreach (var tableEntity in item)
{
batchInsertOperation.Add(TableOperation.Insert(tableEntity));
}
table.ExecuteBatch(batchInsertOperation);
});
}
}
Then, read from the table, the partition key are all the same here:
public void ReadCloudData()
{
InsertMoreThanOneThousandDataToTable();
var query =
new TableQuery<CustomerEntity>().Where(TableQuery.GenerateFilterCondition("PartitionKey",
QueryComparisons.Equal, DataProvider.PartitionKey));
var result = AzureTableService.Table.ExecuteQuery(query);
Console.WriteLine("Read {0} entities from table.", result.Count()); // output 1200
}
I only used the latest Azure storage .NET client API.
I'm not able to find a documentation link but ExecuteQuery method handles continuation token internally and will return all entities in a table. Thus the behavior you're seeing is correct.
If you run Fiddler when you are executing this code, you will notice multiple requests are sent to table service. First request would be without continuation token but in subsequent requests you will see NextPartitionKey and NextRowKey querystring parameters.

Resources