With the recent preview release of 'Data discovery & classification' for Azure SQL databases, has anybody found where this data is stored and if it can be queried directly from the Azure database? I know for on-premise databases if you right click on a database and choose 'Tasks - Classify Data...' anything you enter into that interface is stored as extended properties on the 'table/column'. However, after entering the same data via the interface in the Azure portal, there are no extended property values that I can find in my Azure SQL database. I would really like to be able to query this classification data directly so I can incorporate other metadata about the column such as data type, sample value, collation etc.
For Azure SQL DB, this metadata is stored in new attributes that have been introduced into the SQL Engine to support tagging column sensitivity, which are currently not exposed. We plan to expose them via REST/Powershell/T-SQL as the feature continues rolling out.
Please follow our announcements and the online feature documentation for updates.
Thanks,
Gilad (MSFT)
Related
Microsoft Cosmos DB includes DocumentDB API, Table API and others. I have about ~ 10 TB of data and would like to have a fast key-value lookup (very little updating and writing, mostly are reading). Add a link for Microsoft Cosmos DB:
https://learn.microsoft.com/en-us/azure/cosmos-db/
So how should I choose between DocumentDB API and Table API?
Or when should I choose DocumentDB API? When should I choose Table API?
Is it a good practice to use DcoumentDB API to store 10 TB of data?
The Azure Cosmos DB Table API was introduced to make Cosmos DB and its advanced indexing, geo-distribution, etc. features available to the Azure Table storage community. The idea is that someone using Azure Table storage who needs more advanced features only offered by Cosmos DB can literally just change their connection string and their existing code will work with Cosmos DB.
But if you are a greenfield customer then I would recommend using SQL API (formerly called Document DB API) which is a super set of Table API. We are constantly investing in providing more advanced features and capabilities to SQL API where as for Table API we are just looking to maintain compatibility with Azure Table storage's API which hasn't changed in many years.
How much data you have doesn't have any affect on what API you choose. They both have the same multi-model infrastructure and can handle the same sizes of data, query loads, distribution, etc.
So how should I choose between DocumentDB API and Table API?
Choosing between DocumentDB API and Table API will primarily depend on the kind of data that you're going to store. DocumentDB API provides a schema-less JSON database engine with SQL querying capabilities whereas Table API provides a key-value storage database service. Since you mentioned that your data is key-value based, recommended is that you use Table API.
Or when should I choose DocumentDB API? When should I choose Table API?
Same as above.
Is it a good practice to use DcoumentDB API to store 10 TB of data?
Both Document DB API and Table API are designed to store huge amounts of data.
However you may want to look into Azure Table Storage as well. Cosmos DB lets you fine tune the throughput that you need and robust indexing/querying support and that comes at a price. Azure Tables on the other hand comes with fixed throughput and limited indexing/querying support and is extremely cheap compared to Cosmos DB.
You may find this link helpful to explore more about Cosmos DB: https://learn.microsoft.com/en-us/azure/cosmos-db/introduction.
Please don't flag this as off-topic.
It might help for you to know in advance: if you are considering the document interface, then in fact there is a case-insensitivity that can affect how DataContract classes (and I believe all others) are transformed to and from Cosmos.
In the linked discussion below, you will see that there is a case insensitivity in Newtonsoft.Json that can have effects on your handling of objects that you pass or get directly from the API. Not that Cosmos has ANY flaws, and in fact it is totally excellent. But with a document API, you might (like me) start to simply pass DataContract objects into Cosmos (which is obviously not wrong, and in fact very much expected from the object API), but there are some serializer and naming strategy handler options that you are probably better of at least being aware of up front.
So just to add a note for you to be aware of this behavior with an object interface. The discussion is here on GitHub:
https://github.com/JamesNK/Newtonsoft.Json/issues/815
I am associating an Azure SQL DB Table to my Azure Search using an Indexer. I am setting this all up using Azure's website: https://portal.azure.com
When I try and create the Indexer in Azure Search, I get the warning about "Consider enabling integrated change tracking on your database." However, I have enabled integrated change tracking on my database and table.
I have successfully setup several tables this way, in the same database, and they're working just fine with Azure Search. However, this table has a schema other than [dbo], and the others with change tracking were [dbo]. The same SQL user is being used for all the tables, and it has been granted the change tracking permission to this table, too.
Is there a problem with the Azure website where I cannot do this via the UI? Can this be done otherwise? Is there a permission issue with my DB's schema? Something else?
Because of this warning, I have not actually created this Azure Search Index.
Any help is appreciated!
It's a limitation of Azure Search portal - it doesn't support enabling integrated change tracking for non-default schemas. The workaround is to create the indexer programmatically, using REST API or .NET SDK. For a walkthrough, see https://learn.microsoft.com/azure/search/search-howto-connecting-azure-sql-database-to-azure-search-using-indexers.
I am new to Azure Search and I have just seen this tutorial https://azure.microsoft.com/en-us/documentation/articles/search-howto-dotnet-sdk/ on how to create/delete an index, upload and search for documents. However, I am wondering what type of database is behind the Azure Search functionality. In the given example I couldn't see it specified. Am I right if I assume it is implicitly DocumentDb?
At the same time, how could I specify the type of another database inside the code? How could I possibly use a Sql Server database? Thank you!
However, I am wondering what type of database is behind the Azure
Search functionality.
Azure Search is offered to you as a service. The team hasn't made the underlying storage mechanism public so it's not possible to know what kind of database are they using to store the data. However you interact with the service in form of JSON records. Each document in your index is sent/retrieved (and possibly saved) in form of JSON.
At the same time, how could I specify the type of another database
inside the code? How could I possibly use a Sql Server database?
Short answer, you can't. Because it is a service, you can't specify the service to index any data source. However what you could do is ask search service to populate its database (read index) through multiple sources - SQL Databases, DocumentDB Collections and Blob Containers (currently in preview). This is achieved through something called Data Sources and Indexers. Once configured properly, Azure Search Service will constantly update the index data with the latest data in the specified data source.
I've deployed the Sitecore on to Azure CD by using Sitecore Azure 3.0.0.
However, I'm not getting any analytics data until I manually update the "analytics" connection string to sql azure.
If anyone has already configured above, could you help me with queries below:
Is manual connection string the best solution? Or, am I missing any configuration setting for sitecore azure deploy.
Is it possible to Sync SQL Azure analytics to on-premise analytics db? We need this for disaster recovery i.e. to deploy all web,core,analytics to different data centre in event of disaster.
Does DMS slow down the performance of Sitecore CD?
Thanks.
You can add your "analytics" connection string to the connection string patch file defined in your Sitecore Azure config. Do this via the following steps:
Navigate to /sitecore/system/modules/Azure/[Environment]/[Region]/[Farm]/[Role]/[Deployment]
In the deployment item (e.g. Staging, Production), you should see a field named "Connection Strings Patch".
Scroll down in that field until you see the connection strings for the "core", "master", and "web" databases.
Add a connection string element for your "analytics" database. Be sure to use the connection string for the deployment item you're editing, i.e. use your Analytics staging connection string for the Staging item, production connection string for the Production item.
It is not recommended to use Azure SQL Data Sync for backup/disaster recovery (this recommendation is not specific to Sitecore). It is recommended to use a combination of Azure SQL database copying and then Azure SQL database export.
Also, Azure SQL Data Sync has limitations regarding the database schemas supported. SQL Data Sync is unable to synchronize any table that does not have a Primary Key (the Sitecore Analytics database has a few tables without primary keys).
Also, SQL Data Sync synchronizes only data but not stored procedures and triggers (the Sitecore Analytics database does have stored procedures).
Lastly, as your Analytics database grows, a sync operation is likely to take a significant amount of time to complete, whereas a copy operation will still take some time but likely not as much and will place less of a burden on your SQL server.
This MSDN article provides and overview of the copy/export process: http://msdn.microsoft.com/en-US/library/hh852669.aspx#adr3
This MSDN article provides details on how to copy Azure SQL databases: http://msdn.microsoft.com/library/ff951631.aspx
Yes, Sitecore content delivery server performance is impacted when DMS is enabled. To what extent largely depends on how you're using DMS (e.g. personalization, MV testing, engagement plans) and the amount of traffic your server receives.
Is it possible to add a spatial index to Windows Azure Mobile Services?
I added a free Mobile Service free 20mb database, then a table, then a geography type column.
I couldnt add an index to geography column using the 'management portal.' But I connecting remotely using sql server management studio express i managed to write a TSQL to apply a spatial index. It said it succeeded but I couldnt confirm it as the portal doesnt reflect it.
I'm now thinking
1) is it possible to add spatial index to azure sql databases?
2) how do I add the geography type to my Xamarin project for easy read/write using azure component?
// update
I found a 'set index' button on azure dashboard when viewing the table columns but it threw an error:
Column 'Location' in table 'table.place' is of a type that is invalid for use as a key column in an index or statistics
Does this mean Azure database dont support spatial indices?
Spatial indexes are supported in SQL Database (with some restrictions), see: http://msdn.microsoft.com/en-us/library/windowsazure/ff759530.aspx
That said, spatial types are not directly supported with Window Azure Mobile Services; however, you can use converter classes to support additional types that are not handled by the client SDKS. See Carlos Figueira's blog for details.