I am "connecting" CosmosDB to an Azure Function by change feed binding. I wonder if there is a way to trigger change feed only when certain property has some specific value.
For instance, a new user is inserted in CosmosDB. Then, run the Azure Function only when the user has user.email != null.
I could filter this out in the Azure Function of course. Just concern about the pricing filtering out potentially thousands of events I don't need.
No this is not currently possible.
There is a 5 year old request on the Feedback site and a response saying the "feature is now planned" but it is unclear when that response was posted as there is no date on it.
For the time being at least you need to filter out any documents not matching your criteria within the function itself rather than being able to get this done server side by CosmosDB when sending the batch of changes.
Related
I am trying to get the UID within an onWrite cloud function of any authenticated user who deletes a document in firestore (not the real time database which doesn't have this issue). The reason is... I am trying to create a log of all actions performed on documents in a collection. I have to use cloud functions as the client could hypothetically create/edit/delete a document and then prevent the corresponding log entry from being sent.
I have seen in other stackoverflow questions like:
Firestore - Cloud Functions - Get uid
Getting the user id from a Firestore Trigger in Cloud Functions for Firebase?
That firestore will not include any auth data for firestore in the onWrite function, and that the accepted workaround is to have fields like updated_by, created_by, created_at, updated_at in the document being created/updated which are verified using firebase permissions. This is great for documents being inserted or updated, but deleted documents in onWrite cloud functions only have change.before data, and no change.after data, meaning you have no way to see who deleted the document, and at best who updated the document last before deletion.
I am in the middle of trying out some work arounds as follows (but they have serious detractors):
Sending an update to a document right before it is to be deleted. Issues -> Might have timing issue, debounce issues, requires messy permissions to ensure that a document is only deleted if it has the proceeding update.
Updating it with a field that tags it for deletion and watching for this tag in a cloud function that then does the deleting. Issues -> leads to a very noticeable lag before the item is deleted.
Does anyone have a better way of doing something like this? Thanks!
Don't delete the document at all. Just add a field called "deleted", set it to true, and add another field with the UID of the user that deleted it. Use these new fields in your queries to decide if you want to deal with deleted documents or not for any given query.
Use a different document in a separate collection that records deletions. Query that collection whenever you need to know if a document has been deleted. Or create a different record in a different database that marks the deletion.
There are really no other options. Either use the existing document or create a new document to record the change. Existing documents are the only things that can be queried - you can't query data that doesn't exist in Firestore.
I have a Customer container with items representing a single customer in SQL API (DocumentDB) in CosmosDB. I also have a Gremlin API (GraphDB) with the customers' shoppingcart data. Both these data are temporary/transient. The customer can choose clear shopping cart which will delete the temporary customer and the shoppingcart data.
Currently I make separate calls, one to the SQL API (DocumentDB) and Gremlin API (GraphDB) which works but I want to do both as a transaction (ACID principle). To delete a customer, I call the Gremblin API and delete the shoppingcart data, then call the SQL API to delete the customer. But if deleting the customer with the SQL API (second step) fails, I want to roll back the changes done in the first call which will roll back the shoppingcart data which were deleted. In the T-SQL world, this is done with a commit and rollback.
How can I achieve distributed transaction coordination around the delete operations of the customer and shoppingcart data?
Since you don't have transactions in Cosmos DB across different collections (only within the partition of one container), this won't be directly possible.
Next best thing could be to use the Change Feed. It gets triggered whenever an item gets changed or inserted. But: It does not get triggered on deletes. So you need another little workaround of "soft deletes". Bascially you create a flag to that document ("to-be-deleted" etc.) and set its TTL to something very soon. This does trigger then change feed and you can from there delete the item in the other collection.
Is all that better than what you currently have? Honestly, not really if you ask me.
//Update: To add to the point regarding commit/rollback: This also does not exist in Cosmos DB. One possible workaround for this that comes to mind:
Update elements in collection shopping cart. Set a flag to-be-deleted to true and set the TTL for those elements to something like now() + 5 minutes
Delete the element in customer collection. If this works, all good.
If deletion failed, update the shoppingcart again. Remove the to-be-deleted flag and remove the TTL so Comsos DB won't automatically delete it.
Of course, you also need to update any queries you run against your shoppingcart to exclude any elements with the deletion flag in place.
I'm looking to assign a unique Id to each event that is processed via Azure Stream Analytics (IoT Hub input source).
It appears there is no way to just assign a new random Guid in ASA (like in SQL) but I did come across the following MSDN article which mentions being able to use the GetMetadataPropertyValue function to achieve what I want with the EventId property:
Creates a unique id (Guid) for an input event, which can be useful for
primary key purposes. It is consistent (not random) i.e., Stream
Analytics will produce the same id for an event, if you go back in
time and re-read the same input event.
The issue I have is this function returns NULL each time when I try it in the query builder. Am I missing something or is there another way to achieve a Unique Id for an event?
Unfortunately this property doesn't work when you test it from sample data (we need some additional metadata).
However you should get the GUID value when you run the job on live data from Event Hub or IoT Hub.
Let me know if it works for you! Sorry for the inconvenience.
JS
I am Using Azure Mobile Service Offline Sync with Windows Phone.
First time I am adding data to local database(sqlite) Filtered by userId.
Then for second user I am using PurgeAsync for all tables just to clear first user data and load second user data.
But, For second user request goes with filter with updatedat value.
Which is the timestamp of purge operation performed.
Which make no data download for second user.
I think that you should supply a different string in the PullAsync method for each user and it will work properly.
I solved this problem by passing null value for queryid.
I'm new in windows phone and azure mobile services. I want use in my application azure offline data, but I have one problem with PullAsync(). I don't want pull all data to the local database. I want filter data.
I try use PullAsync() like this:
await App.rateTable.PullAsync(App.rateTable.Where(a => a.user_id == userId));
It's not working and I get bad request message.
user_id is not primary key.
Is there any way to filter data before pulling all to local database?
More information can be found on this msdn forum thread.
The following will retrieve all records the first time, and after that only updated records:
await App.rateTable.PullAsync("userId",
App.rateTable.Where(a => a.user_id == userId));
The first parameter is the queryKey. This is used to track your requests to the server. Note that the queryKey is restricted to 25 alphanumeric characters, and excludes hyphens, whitespace, etc. From the linked thread: the query key should just be a descriptor of the query.