Azure Data Catalog Backup - azure-data-catalog

Since ADC is provided by MS as SaaS to customers, is MS taking backups of the dataset and business glossary? If yes, how often and how can a customer get access to the backups for recovery purposes?

Unfortunately, there is no explicit backup/restore feature available for catalogs.
I would suggest you to vote up an idea submitted by another Azure customer.
https://feedback.azure.com/forums/906052-data-catalog/suggestions/33125845-azure-data-catalog-backup-feature
All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.
The closest way to achieve this with current functionality is to use the Azure Data Catalog REST API to extract all assets and persist them locally (and re import them manually later).
There is a sample application available that demonstrates this technique: Data Catalog Import/Export sample tool.

Related

Get Azure DevOps Metrics into Elatic Search Database using its REST API?

I would like to analyze some metrics from a Team in my Azure DevOps Organization using Kibana along with some other data. Therefore I need to import all commits into the elastic search database. However, according to elastic’s integration page https://www.elastic.co/de/integrations?solution=all-solutions they only support Acquisition of data from Azure’s infrastructure portfolio not their DevOps product.
This is why I thought about gathering the data using DevOps’s REST-API which I would call on a daily basis.
My Question: Is there a more optimal way? Has someone already done something similar?
Thank you in advance.
You may check Content Sources:
Workplace Search can ingest data from many different content sources.
A content source is usually a third-party service like GitHub, Google
Drive, or Dropbox. You can also build your own connectors using
Custom API sources, which allows you to create unique content
repositories on the platform and send any data to Workplace Search via
uniquely identifiable endpoints.
Since Workplace Search doesn't support DevOps, you could check Connecting custom sources. And work with Commits - Get Commits API to retrieve git commits for a project:
GET https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repositoryId}/commits?api-version=6.0

Does MS use the data on Azure for internal machine learning?

My organization wants to use Microsoft Text Analytics API for sentiment analysis. But my employer concern is that MS will be using that data for the live training of their sentiment engine. Is this the case?
See the Microsoft Trust Center for your answer.
How we manage your data
With Microsoft, you are the owner of your customer data.
Microsoft will use your customer data only to provide the services we
have agreed upon, and for purposes that are compatible with providing
those services. We do not share your data with our
advertiser-supported services, nor do we mine it for marketing or
advertising. If you leave the service, we take the necessary steps to
ensure the continued ownership of your data.

Microsoft Azure realtime charts for externals

since a couple of weeks I'm working with Microsoft Azure and I wonder if there is a possibility to create realtime charts in my Web App for external customers.
I know Microsoft provides two different services called 'Power BI', which supports realtime charts and 'Power BI embedded'. But my problem is that, as far as I know, Power BI is only intended for internal users and Power BI embedded, which is inteded for charts e.g. in Web Apps for external customers, only provides reports which are not realtime.
Am I missing something or is it currently not possible to provide realtime charts inside web apps with the given services of Azure? If yes, what would be alternatives to achieve my goal?
Thank you very much in advance.
Kind regards,
Felix
I would look at Power BI Embedded, with the data source using a Direct Query connection to Azure SQL Database or Azure SQL Datawarehouse. Every user action in the report (filtering, drilling etc) will generate a query against the database.
That Power BI Embedded architecture is explained on this page:
https://learn.microsoft.com/en-us/azure/power-bi-embedded/power-bi-embedded-what-is-power-bi-embedded
Direct Query is explained on this page:
https://powerbi.microsoft.com/en-us/documentation/powerbi-azure-sql-database-with-direct-connect/
1) Consider that Real Time is like in an IOT scenario where you see your graphics on your dashboard moving in Real Time and not after a refresh. So in this context you should consider using Azure Stream Analytics Jobs. It's get an input from a blob storage, an event Hub, ..., and then in output you can use your power BI account to write in real time events ingested from Azure Streaming Analytics. Very powerfull! you use SQL for querying the input, the only thing to be aware is the thumbling time window that is somehow new to the SQL language.
2) Letting your customer access to the dashboard I would suggest you to pubblicate your dashboard for free access, and then secure your dashboard inside a web app on which you can apply a security pattern. You can also invite people outside of your organisation via email. Which is faster than the previous solution, but people accessing to your report must have a power BI Pro license. You can use the free trial for 60 days.
Hope that helps!
Cheers!

Will the unified API return information from Azure?

I've been researching the new unified API for office365 (graph.microsoft.com). Currently one can authenticate to it using an organisationalID and it will return information from emails-from-exchange/OneDrive4Business/Skype4Business(future)/calendar-from-exchange/exchange-contacts/etc...
The ability to surface such a plethora of information from one endpoint strikes me as being hugely valuable. In the future I'd really like to see information relating to Azure be surfaced in here also. For example, I'd love to use graph.microsoft.com to access lists of:
Azure storage accounts
Azure SQL DB servers and databases
Azure SQL DB firewall rules
Azure ML experiments
Countless other things
Are such things on the roadmap?
That is a great question, and it's great to see such enthusiasm around a single endpoint surfacing this information. We're not sure when Azure resource providers would be a part of the unified API, but it would be great to have them. There are a number of other teams who are very interested, and in general we've been pretty focused on user or person centric developer experiences here as you should be able to see from Yina Arena's //Build or Ignite presentations.

What are current and relevant Azure projects to learn from?

There are many Azure projects on codeplex and scattered over MSDN. Some of them are so old that they use the code-based version of the storage client, and not the compiled version.
Which projects are current and relevant, and which are outdated?
Getting Started with Azure
http://blog.syntaxc4.net/post/2010/12/30/Essential-Resources-for-Getting-Started-with-Windows-Azure.aspx
Current and by MSFT:
MSDN Samples
Part 1 Demonstrates COMET via Silverlight and WCF among other samples
Part 2 Includes HTTP compression sample
How to... In Azure
Some entries contain code samples
http://msdn.microsoft.com/library/gg432998.aspx
FabrikamShipping
Includes AD FS Federation, Facebook integration, and provisioning automation
http://blogs.msdn.com/b/vbertocci/archive/2011/03/14/fun-with-fabrikamshipping-saas-ii-creating-an-enterprise-edition-instance.aspx
Azure SDK
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=7a1089b6-4050-4307-86c4-9dadaa5ed018
Azure Training Kit
Located at C:\WAPTK\Default.htm
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=413E88F8-5966-4A83-B309-53B7B77EDF78
Patterns and Practices
Cloud Guidance (Greenfield and Brownfield scenarios)
Tailspin Toys Hands On Lab
Note there is a part 1 and part 2 to this
http://wag.codeplex.com/
Developing applications for the cloud
http://msdn.microsoft.com/en-us/library/ff966499.aspx
All In One Samples
Has some Azure samples for oData and more. See both the 2008 and 2010 samples since each has different code.
http://1code.codeplex.com/
WCF Azure Samples
(looks up to date, but is located on "archive.msdn" which is disconcerting )
http://archive.msdn.microsoft.com/wcfazure
http://code.msdn.microsoft.com/windowsazure/
3rd Party up to date
Encryption for Azure Table
Easily and transparently encrypt Table data
http://azuretableencrypt.codeplex.com/
Lokad.Cloud
An ORM for the cloud aka O/C mapper
http://code.google.com/p/lokad-cloud/
Facebook
http://facebooksdk.codeplex.com/
http://azuretoolkit.codeplex.com/
Azure Storage
http://azurestorageexplorer.codeplex.com/ (Oct of '10)
Azure Accelerators
http://azureaccelerators.codeplex.com/
Aside from makerofthings7's thorough answer, let me add one more:
David Pallman's 'Azure Storage Samples' - these are complete samples with two implementations each: .NET Storage Client Library and REST. This was published in February, 2011.
A project that probably contain all the providers you will ever need; Membership, Role, Session, and Profile providers. Stores in the Azure Table Storage and uses the Azure Queue http://azureproviders.codeplex.com

Resources