I have a combined index that crawls data from Azure SQL and Blob and are mapped based a common column.
The mapped blob document is optional.
When there is no blob document the search indexer indexes the respective SQL row setting the content propertyas null and if the document is available the content property shows correct data.
I have enabled BLOB deletion tracking and the issue comes when a blob document is deleted. The deletion policy triggers and also removes the mapped SQL row values from index.
I was expecting that the content property will be set to null, but the deletion policy also removes the mapped SQL row values from index.
What am I doing wrong? Kindly help.
Thanks a lot in advance..
BR
The behavior you are seeing is by default. SQL Deletion Policy deletes full index documents based on their document Key, not specific fields. The policy doesn't have a way to know the index has values from other sources too. If there is a combined index and you would like the behavior you are looking for, you may try instead using a Logic App SQL trigger to update only specific fields with Add, update or delete documents, instead of the Deletion Policy.
Related
I have created a azure search service and trying to import data using azure SQL data source and I have scheduled it to refresh the data. Data is getting refreshed properly in data source as I can verify it through indexer and index. However, in index it's getting added every time with initial document but I want only newly/updated documents in index for example if initial no. of document in index was 150 then after refresh it's increases to 156 but I want only 6 document their after refresh.
I tried both option high watermark and soft delete.
Azure Search is designed to incrementally add new documents automatically. So you could delete documents first and then upload documents.
However, delete document you need to specify field in document, currently there's no way to delete all the documents from an index. As you suspected deleting and re-creating the index is the way to go. Also you could vote up this feedback.
If you still want the feature that remove the initial document you could add an item to our userVoice page.
I index the view from my database. When I add an entry to the table, an entry is also added to the view. Index indexes new data. But when I delete this entry from the table in the index, it remains. How to set up a soft delete?
I read the documentation and it says it is necessary to add a field. Should I add it myself?
The only way to remove deleted documents from your index, is by using the SQL Integrated Change Detection feature or by setting up a Data Deletion Detection Policy (Soft Delete).
If you choose the second option, then you have to create a soft-delete column in your view.
Then tell your datasource that it should track the soft-delete field:
Is it possible to update a column using Query in azure table storage ? The table has already 100 records and now I have added a new column. For the previous 100 records , the newly added column value is "null" as shown in the image. I just want to set a default value for all the 100 records. I need to know how to update the table.
There are several ways to edit columns in Azure storage, you can use the Update Entity functionality to do via Web requests, I'd recommend checking the documentation, Then you can also use Storage Explorer to modify single columns if you have enough permissions.
I added a new field. This field is present in pretty much all the data rows that Azure search will be returned and needs to be indexed.
Changed the VIEW that returns rows to AzureSearch to return a new row ("name")
Added a new field to the Index ("name")
Verified via Search Explorer that "name" field is returned in results for that index. It is currently always null (as expected) in the results.
Now how do I trigger a full reindex so the "name" field will contain values for ALL the rows, and not just for the NEW rows going forward.
(of course, I could delete the index and recreate a new one and repoint my code to the new index, but it seems needlessly brute force!)
Right, reset the indexer's change tracking state using Reset Indexer API. You can also reset the indexer in Azure Portal or using .NET client SDK.
How can I update the exiting data source which is create from azure SQL view.
What are steps I need to follow once when dealing with azure search for Database view.
My Data source& index should be updated once my view gets updates.
Assuming that by "updating the view" you mean adding more columns to the SELECT clause:
You don't need to update the datasource as long as you didn't change the name of the view or the credentials.
You will need to add newly added fields to your index definition. This doesn't happen automatically. Once you add new fields to your index schema, indexer will start populating those fields for newly indexed rows. To populate documents that have already been indexed, reset the indexer (in the portal or using the Reset Indexer API).