I'm recently using the Microsoft.Azure.Cosmos.Table API and I noticed there was a great method called InsertOrMergeOperation, however, I am wondering if there is any way that the return result could tell me whether my entity just did an insert or a merge/update operation. I checked the TableResult object but it doesn't return any useful information about that.
Is there anyone know if this Operation could give me what I want? If not, is there any other operation that could perform the same workflow?
The TableResule does not return a value to indicate if it's an insert or merge operation. If you want to get the info, you have 2 methods for that:
Method 1:use table query with partition key and rowkey to check if the records exists or not in table, then you know the following operation by InsertOrMerge() is insert or merge as per the result is null or not, sample code like below:
CloudTableClient tableClient = account.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("People");
TableQuery<CustomerEntity> query = new TableQuery<CustomerEntity>()
.Where(
TableQuery.CombineFilters(
TableQuery.GenerateFilterCondition("PartitionKey", QueryComparisons.Equal, "Smith"),
TableOperators.And,
TableQuery.GenerateFilterCondition("Email", QueryComparisons.Equal,"Ben#contoso.com")
));
await table.ExecuteQuerySegmentedAsync<CustomerEntity>(query, null);
Method 2:use try-catch block, and first do a insert operation, if it returns an "conflict" error message, means the following InsertOrMerge() operation is an merge operation, code like below:
TableOperation t1;
try
{
t1 = TableOperation.Insert(customer);
table.Execute(t1);
}
catch (Exception e)
{
if (e.Message.ToLower() == "Conflict".ToLower())
{
t1 = TableOperation.InsertOrMerge(customer);
table.Execute(t1);
}
}
Hope it helps.
Related
I am querying Azure DocumentDb (cosmos) for a document that is present in the container:
try{
doc = await client.ReadDocumentAsync(
GetDocumentUri("tenantString-campaignId"),
new RequestOptions
{
PartitionKey = new PartitionKey(tenantString)
});
}
catch(Exception e)
{
Console.WriteLine(e);
}
for this document:
tenantString-campaignId is the id you can see here and tenantString alone is the partition key this is under. The tenant itself was passed in as a string and I had that working but now I have changed to passing a Tenant object and parsing the string required from it I am not returning the document.
I have tried a few different variations of tenantString and id and I can generate either a DocumentClientException, Id does not exist, Exception or it fails silently; no exception and returns to the calling method where it causes a NullReferenceException as no document is returned.
As far as I can make out from debugging through this I am constructing all my data correctly and yet no document is returned. Does anyone have any idea what I can try next?
This syntax for the .NET SDK v2 is not correct. ReadDocumentAsync() should look like this.
var response = await client.ReadDocumentAsync(
UriFactory.CreateDocumentUri(databaseName, collectionName, "SalesOrder1"),
new RequestOptions { PartitionKey = new PartitionKey("Account1") });
You can see more v2 samples here
I have created a console app to delete documents from CosmosDB based on a partition that we have set up.
using (var client = new DocumentClient(new Uri(Config["CosmosEndpoint"]), Config["CosmosAuthkey"]))
{
var db = Options.Env.ToUpper().Equals("CI") ? Config["CiDatabase"] : Config["QaDatabase"];
var tenantString = $"{Options.Tenant}-{Options.Language}";
Log.Information($"Deleting {tenantString} from {db}-{Config["CosmosCollection"]}");
var query = client.CreateDocumentQuery<Document>(
UriFactory.CreateDocumentCollectionUri(db, Config["CosmosCollection"]),
"SELECT * FROM c",
new FeedOptions()
{
PartitionKey = new PartitionKey(tenantString)
}
).ToList();
Log.Information($"Found {query.Count} records for Tenant: {tenantString}");
if (query.Count > 0)
{
foreach (var doc in query)
{
Log.Information($"Deleting document {doc.Id}");
await client.DeleteDocumentAsync(doc.SelfLink,
new RequestOptions { PartitionKey = new PartitionKey(tenantString) });
Log.Information($"Deleted document {doc.Id}");
}
}
else
{
Log.Information($"Tenant: {tenantString}, No Search Records Found");
}
}
This never reaches the line Log.Information($"Deleted document {doc.Id}"); but also does not seem to throw an exception. I have wrapped the call in a try/catch for DocumentClient/ArgumentNull Exception in an attempt to see what it was but it just bombs on the delete call. It does, however, always delete one document.
This tells me that my config must be correct as I am connecting and querying and even deleting but not for all documents in the query. Even more strange is that I have copied this from another application that I wrote earlier where this code works.
Is there an upper limit on connecting meaning I need to delay my loop
some?
Why do I not see an exception when using a try/catch?
Or is there another reason that I only seem to be able to delete one document at a time with this code?
I'm trying to retrieve a list of entities from CRM, but I'd like to get each one with the related entities. So far, I've the following code:
FilterExpression filterExpression = new FilterExpression();
ConditionExpression condition = new ConditionExpression(Constants.ModifiedOnAttribute, ConditionOperator.GreaterEqual, lastSync);
filterExpression.AddCondition(condition);
QueryExpression query = new QueryExpression()
{
EntityName = entityName,
ColumnSet = new ColumnSet(attributesMetadata.Select(att => att.Name).ToArray<string>()),
Criteria = filterExpression,
Distinct = false,
NoLock = true
};
RetrieveMultipleRequest multipleRequest = new RetrieveMultipleRequest();
multipleRequest.Query = queryExpression;
RetrieveMultipleResponse response = (RetrieveMultipleResponse)proxy.Execute(multipleRequest);
In the variable response, I can see the EntityCollection attribute, but inside, Related entities always come empty.
I'd like to know if it is possible to retrieve the set of a given entities, with the related entities, using RetrieveMultipleRequest, rather than go one by one using RetrieveRequest.
One approach to retreive related entities data - adding LinkEntities to your query. Example below will make you an idea how to make this:
LinkEntity linkEntity = new LinkEntity("email", "new_emails", "activityid", "new_relatedemail", JoinOperator.Inner);
linkEntity.Columns.AddColumn("versionnumber");
linkEntity.Columns.AddColumn("new_emailsid");
linkEntity.EntityAlias = "related";
query = new QueryExpression("email");
query.ColumnSet.AddColumn("activityid");
query.ColumnSet.AddColumn("versionnumber");
query.Criteria.AddCondition("modifiedon", ConditionOperator.NotNull);
query.LinkEntities.Add(linkEntity);
And then you can access attributes from related entities using EntityAlias you specified above:
foreach (Entity entity in entities.Entities)
{
if ((long)(entity["related.versionnumber"] as AliasedValue).Value > 0)
{
stop = false;
}
}
The RetrieveMultipleRequest is for returning multiple instances of a particular type of entity. I have spent a year using the CRM SDK from C# and I have found no way of populating those related entity collections in a single query. This basically leaves you with two options:
Use the AliasedValue as SergeyS recommends. Remember when querying 1:Many relationships, be aware that you could be returning multiple results for the same parent entity. This is what I use most of the time.
Perform a second query for each relationship you want access to. You'll probably get better performance if you can use an IN statement in your second query, based on the results of the first, rather than performing a separate query for each result of the first.
Below is some pseudo code to show the difference.
var contacts = GetContacts();
// One Request to get the cars for the contacts
var cars = GetCarsWhereContactIdIn(contacts.Select( c => c.new_ContactId));
foreach(var c in contacts){
c.new_Cars.AddRange(cars.where(car => car.new_contactId = c.ContactId));
}
// Verses
var contacts = GetContacts();
foreach(var c in contacts){
// One Request for each contact
c.new_Cars.AddRange(GetCarsForContact(c.ContactId));
}
I'm working with Windows Azure Table Storage and have a simple requirement: add a new row, overwriting any existing row with that PartitionKey/RowKey. However, saving the changes always throws an exception, even if I pass in the ReplaceOnUpdate option:
tableServiceContext.AddObject(TableName, entity);
tableServiceContext.SaveChangesWithRetries(SaveChangesOptions.ReplaceOnUpdate);
If the entity already exists it throws:
System.Data.Services.Client.DataServiceRequestException: An error occurred while processing this request. ---> System.Data.Services.Client.DataServiceClientException: <?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
<code>EntityAlreadyExists</code>
<message xml:lang="en-AU">The specified entity already exists.</message>
</error>
Do I really have to manually query for the existing row first and call DeleteObject on it? That seems very slow. Surely there is a better way?
As you've found, you can't just add another item that has the same row key and partition key, so you will need to run a query to check to see if the item already exists. In situations like this I find it helpful to look at the Azure REST API documentation to see what is available to the storage client library. You'll see that there are separate methods for inserting and updating. The ReplaceOnUpdate only has an effect when you're updating, not inserting.
While you could delete the existing item and then add the new one, you could just update the existing one (saving you one round trip to storage). Your code might look something like this:
var existsQuery = from e
in tableServiceContext.CreateQuery<MyEntity>(TableName)
where
e.PartitionKey == objectToUpsert.PartitionKey
&& e.RowKey == objectToUpsert.RowKey
select e;
MyEntity existingObject = existsQuery.FirstOrDefault();
if (existingObject == null)
{
tableServiceContext.AddObject(TableName, objectToUpsert);
}
else
{
existingObject.Property1 = objectToUpsert.Property1;
existingObject.Property2 = objectToUpsert.Property2;
tableServiceContext.UpdateObject(existingObject);
}
tableServiceContext.SaveChangesWithRetries(SaveChangesOptions.ReplaceOnUpdate);
EDIT: While correct at the time of writing, with the September 2011 update Microsoft have updated the Azure table API to include two upsert commands, Insert or Replace Entity and Insert or Merge Entity
In order to operate on an existing object NOT managed by the TableContext with either Delete or SaveChanges with ReplaceOnUpdate options, you need to call AttachTo and attach the object to the TableContext, instead of calling AddObject which instructs TableContext to attempt to insert it.
http://msdn.microsoft.com/en-us/library/system.data.services.client.dataservicecontext.attachto.aspx
in my case it was not allowed to remove it first, thus I do it like this, this will result in one transaction to server which will first remove existing object and than add new one, removing need to copy property values
var existing = from e in _ServiceContext.AgentTable
where e.PartitionKey == item.PartitionKey
&& e.RowKey == item.RowKey
select e;
_ServiceContext.IgnoreResourceNotFoundException = true;
var existingObject = existing.FirstOrDefault();
if (existingObject != null)
{
_ServiceContext.DeleteObject(existingObject);
}
_ServiceContext.AddObject(AgentConfigTableServiceContext.AgetnConfigTableName, item);
_ServiceContext.SaveChangesWithRetries();
_ServiceContext.IgnoreResourceNotFoundException = false;
Insert/Merge or Update was added to the API in September 2011. Here is an example using the Storage API 2.0 which is easier to understand then the way it is done in the 1.7 api and earlier.
public void InsertOrReplace(ITableEntity entity)
{
retryPolicy.ExecuteAction(
() =>
{
try
{
TableOperation operation = TableOperation.InsertOrReplace(entity);
cloudTable.Execute(operation);
}
catch (StorageException e)
{
string message = "InsertOrReplace entity failed.";
if (e.RequestInformation.HttpStatusCode == 404)
{
message += " Make sure the table is created.";
}
// do something with message
}
});
}
The Storage API does not allow more than one operation per entity (delete+insert) in a group transaction:
An entity can appear only once in the transaction, and only one operation may be performed against it.
see MSDN: Performing Entity Group Transactions
So in fact you need to read first and decide on insert or update.
You may use UpsertEntity and UpsertEntityAsync methods in the official Microsoft Azure.Data.Tables TableClient.
The fully working example is available at https://github.com/Azure-Samples/msdocs-azure-data-tables-sdk-dotnet/blob/main/2-completed-app/AzureTablesDemoApplicaton/Services/TablesService.cs --
public void UpsertTableEntity(WeatherInputModel model)
{
TableEntity entity = new TableEntity();
entity.PartitionKey = model.StationName;
entity.RowKey = $"{model.ObservationDate} {model.ObservationTime}";
// The other values are added like a items to a dictionary
entity["Temperature"] = model.Temperature;
entity["Humidity"] = model.Humidity;
entity["Barometer"] = model.Barometer;
entity["WindDirection"] = model.WindDirection;
entity["WindSpeed"] = model.WindSpeed;
entity["Precipitation"] = model.Precipitation;
_tableClient.UpsertEntity(entity);
}
I'm getting a strange error while trying to select a row from a table under Windows Azure Table Storage. The exception "An item with the same key has already been added." is being thrown even though I'm not inserting anything. The query that is causing the problem is as follows:
var ids = new HashSet<string>() { id };
var fields = new HashSet<string> {"#all"};
using (var db = new AzureDbFetcher())
{
var result = db.GetPeople(ids, fields, null);
}
public Dictionary<string, Person> GetPeople(HashSet<String> ids, HashSet<String> fields, CollectionOptions options)
{
var result = new Dictionary<string, Person>();
foreach (var id in ids)
{
var p = db.persons.Where(x => x.RowKey == id).SingleOrDefault();
if (p == null)
{
continue;
}
// do something with result
}
}
As you can see, there's only 1 id and the error is thrown right at the top of the loop and nothing is being modified.
However, I'm using "" as the Partition Key for this particular row. What gives?
You probably added an object with the same row key (and no partition key) to your DataServiceContext before performing this query. Then you're retrieving the conflicting object from the data store, and it can't be added to the context because of the collision.
The context tracks all object retrieved from the Tables. Since entities are uniquely identified by their partitionKey/rowKey combination, a context, like the tables, cannot contain duplicate partitionkey/rowkey combinations.
Possible causes of such a collison are:
Retrieving an entity, modifying it, and then retrieving it again using the same context.
Adding an entity to the context, and then retrieving one with the same keys.
In both cases, the context the encounters it's already tracking a different object which does however have the same keys. This is not something the context can sort out by itself, hence the exception.
Hope this helps. If you could give a little more information, that would be helpful.