Moving from AWS Glue to Azure Purview and i am confused about something
Its is possible to query Azure purview data catalog/assets in the same way we can query from AWS Glue data catalog using AWS Athena?
Unfortunately, you cannot query data from Azure Purview.
The Purview search experience is powered by a managed search index. After a data source is registered with Purview, its metadata is indexed by the search service to allow easy discovery. The index provides search relevance capabilities and completes search requests by querying millions of metadata assets. Search helps you to discover, understand, and use the data to get the most value out of it.
The search experience in Purview is a three stage process:
The search box shows the history containing recently used keywords
and assets.
When you begin typing the keystrokes, the search suggests
the matching keywords and assets.
The search result page is shown with assets matching the keyword entered.
For more details, refer to Understand search features in Azure Purview.
Related
My website is hosted on Azure, i want to implement Azure search indexing, but there are some limitations of showing data to anyone. So when i retrieve data from a table, i will check user id and all details from different userrole table and pick some data on the basis of userid,
Can i get data from different database tables using Azure search indexing? currently i am getting onlye one table data on one index?
i have to implement Azure index search, or please suggest me any other workaround for my problem.
for example
i have a table of "users","userroles", "projects" and "tasks".
i want to show tasks of projects related to user. foreign key will be used. now if i create azure index, it will only run my query on tasks table, it will not check the tasks details from projects,users, etc tables. so my question is how i can create such type of index or query in Azure search, where i use different tables to get relevant and correct data in my search.
You can add those roles on your Azure Search index, specifying which roles have access, and use Odata to filter the results.
https://.search.windows.net/indexes//docs?search=&$filter=Administrator%20eq%20true
You can learn more about filters on this documentation.
With the recent preview release of 'Data discovery & classification' for Azure SQL databases, has anybody found where this data is stored and if it can be queried directly from the Azure database? I know for on-premise databases if you right click on a database and choose 'Tasks - Classify Data...' anything you enter into that interface is stored as extended properties on the 'table/column'. However, after entering the same data via the interface in the Azure portal, there are no extended property values that I can find in my Azure SQL database. I would really like to be able to query this classification data directly so I can incorporate other metadata about the column such as data type, sample value, collation etc.
For Azure SQL DB, this metadata is stored in new attributes that have been introduced into the SQL Engine to support tagging column sensitivity, which are currently not exposed. We plan to expose them via REST/Powershell/T-SQL as the feature continues rolling out.
Please follow our announcements and the online feature documentation for updates.
Thanks,
Gilad (MSFT)
Data in sql azure, we have an existing webapi expose the data with odata. Issue is client want to make a call with filters with substring query on a few columns, which making performance really slow. we are debating at this point whether to use full text search index or use the azure search service, thoughts please?
Some of the considerations and tradeoffs between hosting search in Azure Search vs. using SQL Server FTS are captured here.
As pointed out above, Azure Search can index in-database data - see Connecting Azure SQL Database to Azure Search using indexers.
You can point Azure Search at your AzureSQL database and it will index it without you having to write code, but Azure Search is a service you have to pay for on hourly basis and you can learn more information about it here.
Azure Search is recommended for performing searches on various sources and application
Azure search can be used instead of Full-Text search, but If you need to join search results with other tables, then Full-Text Search is recommended.
Hope this helps.
I'm trying to find the best solution for storing dynamic spatial data. I wonder if any of Microsoft's Azure solutions could work. Azure Table Storage would let me create a lot of custom and dynamic structures stored on fast SSD disks.
Because of data's dynamic nature, common indexing seems useless. I would also like to create a lot of table-like structures so the whole architecture cannot be static. Using Azure Table Storage I would dynamically create a table based on country, city, etc sorted by latitude or longitude.
I would appreciate any clue.
Azure Table Storage has mostly been replaced by Azure Cosmos DB.
At the time of writing the Table Storage page even says:
The content in this article applies to the original basic Azure Table storage. However, there is now a premium offering for Azure Table storage in public preview that offers throughput-optimized tables, global distribution, and automatic secondary indexes. To learn more and try out the new premium experience, please check out Azure Cosmos DB: Table API.
You can use Cosmos DB via the Table API, but you'll probably find the Document DB API to be more powerful.
Documents are "schema-free". You can just throw your documents in to a collection, and then you can query against them.
You can create documents which have geo-spatial properties which are indexed automatically.
Then you can perform geo-spatial queries against those properties.
For example you might give each of your documents a point, and then create a query to select all documents that are inside of a polygon.
Or maybe you want to find out how far away each document is from a given point.
I am new to Azure Search and I have just seen this tutorial https://azure.microsoft.com/en-us/documentation/articles/search-howto-dotnet-sdk/ on how to create/delete an index, upload and search for documents. However, I am wondering what type of database is behind the Azure Search functionality. In the given example I couldn't see it specified. Am I right if I assume it is implicitly DocumentDb?
At the same time, how could I specify the type of another database inside the code? How could I possibly use a Sql Server database? Thank you!
However, I am wondering what type of database is behind the Azure
Search functionality.
Azure Search is offered to you as a service. The team hasn't made the underlying storage mechanism public so it's not possible to know what kind of database are they using to store the data. However you interact with the service in form of JSON records. Each document in your index is sent/retrieved (and possibly saved) in form of JSON.
At the same time, how could I specify the type of another database
inside the code? How could I possibly use a Sql Server database?
Short answer, you can't. Because it is a service, you can't specify the service to index any data source. However what you could do is ask search service to populate its database (read index) through multiple sources - SQL Databases, DocumentDB Collections and Blob Containers (currently in preview). This is achieved through something called Data Sources and Indexers. Once configured properly, Azure Search Service will constantly update the index data with the latest data in the specified data source.