I am using Azure search and would like to make sure I can recover from a self inflicted disaster before I push more docs in there. How do I backup my index?
Is creating Azure Search replicas equivalent to making a backup?
How would one restore that?
Thanks
Microsoft have released a console app on github that can be used to backup and restore Azure Search Indexes - its not perfect, but I use it almost daily for backup and restores from prod to CI/QC/Dev instances
https://learn.microsoft.com/en-us/samples/azure-samples/azure-search-dotnet-samples/azure-search-backup-restore-index/
Right now you can't do that from the API or the portal, just save a copy of the JSON schema to a .js file, for example. See the Get Index API.
Normally you don't need to touch the index very often, only add, update or remove documents.
You would need to use an indexer from an external source to push the data into Search and be able to create backups at the same time.
If its an AzureSQL database, this may do it for you automatically, depending on your subscription
Create a table with the same fields in the Azure Search Index and add a deleted flag and a last update date, then import all of your data into the database. Set the date flag to the time that you imported the data.
At the top of the azure search bar, there is an option to 'Import Data'. This will allow you to connect the data source, this way you can create an index which will look at the last modified data and deleted flag when you create the connection.
The wizard will take you through all of the options
From there, just update the SQL table with your changes and the indexer will automatically push them to Azure search.
Thank you for an answer about https://learn.microsoft.com/en-us/rest/api/searchservice/Get-Index
Sometimes Azure Search index it's an only source to restore data.
For example in Microsoft QnA maker - if you will delete azure web app or azure app service- you no longer can even export Knowledge base from QnA maker.
To somehow restore data from QnA maker- I used Azure Search index.
Related
I setup a demo instance of Azure Search with the web front end app.
One thing I have noticed is that even after I remove a document from Blob storage and the indexer runs again, the deleted document and its contents are still stored in the index. How can I remove the document’s contents from the index without deleting and recreating the index?
Here is the link to my GitHub repository for the template for this environment… https://github.com/jcbendernh/Azure-Search-Ignite-2018-Demo
Any insight that you can provide is extremely appreciated.
In order to get a document to be removed from your index by the indexer when it is no longer in the data source, you need to define a data deletion detection policy in your indexer.
There are two different approaches:
1. By defining a column that defines which fields are supposed to be deleted from your data source (SoftDeleteColumnDeletionDetectionPolicy)
2. Or by using the new native soft delete support in blob storage (NativeBlobSoftDeleteDeletionDetectionPolicy)
Both of these approaches are documented at https://learn.microsoft.com/en-us/azure/search/search-howto-indexing-azure-blob-storage#incremental-indexing-and-deletion-detection
Thanks,
-Luis Cabrera (Azure Search PM)
Is there a way to use Azure Search against Azure File Shares. I only see blob storage as an option. We have on-prem servers that sync files to Azure File Shares and would like to search inside those files in a web application.
At this moment, there's no way unless you manually query and push file content to your Azure Cognitive Search index. In the future, there's a hope you'll be able to trigger an Azure Function using this type of binding, which will make your life easier. You can follow / vote up for this feature in the following link:
https://github.com/Azure/azure-webjobs-sdk-extensions/issues/14
Per UserVoice Page for Azure Search: https://feedback.azure.com/forums/263029-azure-search/suggestions/14274261-indexer-for-azure-file-shares#{toggle_previous_statuses}, Azure File Indexer is available in private preview (in fact this has been in this stage for almost 2 years now :)).
Search team would like to reach out to them in case you're interested.
I want to check how was my data 2 days ago and to do that I though of exporting a back up that Azure take every hour. But I cannot find it.
I've found this page on Microsoft that talks about exporting a DB but it is for the current data.
Azure SQL Database automatically creates database backups and writes them to read-access geo-redundant storage (RA-GRS) for geo-redundancy. Frequency depends on what kind of load the service is dealing with, duration backups are kept depends on the tier you've purchased.
To recover to a point in time using the Azure portal, open the page for your database and click Restore on the toolbar.
You cannot overwrite an existing database during restore, so you need to create a new instance for the point-in-time restore, and you'll pay for that. You can also do the restores programmatically using PowerShell.
Further details here.
I suggest to execute a restore, with a specific time, on a temporary DB then export it to a bacpac file. You can use Azure CLI for that:
az sql db restore with -time option for restoring
az sql db export for exporting
az sql db delete for deleting the temporary database
https://learn.microsoft.com/en-us/cli/azure/sql/db
The same can be also achieved using Azure Portal.
I have an index in Azure Search lets says called Hotels.
I have a hotels table in Azure SQL that has the same schema that is a copy of the hotels index found in Azure Search.
I push from my back-end to Azure SQL table and Azure Search at create/update/delete.
In a scenario my data was pushed to Azure SQL but failed to be pushed to Azure Search is it possible to have my Azure SQL Hotels table be an indexer, such that the indexer could sync data to my Azure Search index (hotels) that failed to be pushed from my backend?
Yes, you can both mix push and pull as well as have multiple pull indexers targeting the same index. We see this done often when part of the data is in one data source and part in another, where the index is the point where they converge, coordinated by their key.
The pattern you're describing is not as common, but generally speaking it should work. You'd have to account for cases where your write conflicts with an indexer write, and make sure the writes you do as they happen ultimately win. Also if you go down this path make sure to configure a change detection (and deletion detection if you delete rows) policy so we index from SQL incrementally and don't ready everything on every run.
An alternative approach if you're worried about missing writes is to push all your writes into a queue, and then pull from the queue and into Azure Search. That way you have a single stream of writes instead of two.
I was deleting a single record in the Classic Azure Portal in Mobile Services Browse. The entire contents of the table has now disappeared. Please help!!! How do I restore the contents. The table and scripts are still there.
I have gone into the SQL database tab and clicked restore from point in time, this creates a new DB but does not give me an option to restore to Azure mobile services.
Restoring a database always creates a new database on the same server as the original database, so the restored database must be given a new name. Once complete, the restored database is a normal fully accessible online database charged at normal rates based on its service tier and performance level. If you are restoring the database for recovery purposes you can treat the restored database as a replacement for the original database, or use it to retrieve data from and then update the original database. Please check this article for details.
So if you are going to use the new database, you can re-configure your mobile service to use the new database as following snapshot shows(click the mobile service, go to configure tab then click change database):