So first of all I'd like to say I'm no DBA nor coder, I'm just a regular IT person that works as support for network and infrastructure, however, I like to get familiar with technologies in general and understand the basics of it, let's say how they work, implemented with no additional specific details.
I've been reading about Azure Storage Accounts in regards to tables. As IT, I had to implement simple file shares via SMB 3.0 in order to have them mapped on our network, I've come across other options such as blobs, tables and queues. I've read about them however I'm trying to get the main functionality of tables for a coder.
Correct me if I am wrong, when you code an app with a database, you can put the database on same/different server, and that can be on premise or on the cloud and you kind of link both together.
And as far as Im concerned and what I was able to find out investigating on the web, these tables are NoSQL and no constraints, you create the tables and data through Visual Studio thanks to an API, then that information is reflect on your storage.
How is this is useful when using it for the app you're developing?
I've been reading about Azure Storage Accounts in regards to tables. As IT, I had to implement simple file shares via SMB 3.0 in order to have them mapped on our network, I've come across other options such as blobs, tables and queues. I've read about them however I'm trying to get the main functionality of tables for a coder.
And as far as Im concerned and what I was able to find out investigating on the web, these tables are NoSQL and no constraints, you create the tables and data through Visual Studio thanks to an API, then that information is reflect on your storage.
Azure Storage Accounts is a "box" to keep your Blobs, Tables, Queues, Files organised from the management point of view and for the access control. Each storage type is good for it's specific tasks.
If the world would have just one super storage which will solve all our possible cases for storing, querying and managing the data then there would not be such variety of different databases, storage types etc. available.
If you need to share the files as a "network folder" - try Azure Files.
If your coders need a database storage, then the first question would be what are the requirements to the database do they have? What is the purpose of that database would be, etc. Azure, particularly, has a lot of different database solutions, and again, each of them good for some specific task, and can be not a good choice for other tasks.
As to Azure Tables, from the official docs:
Azure Table storage is a service that stores structured NoSQL data in the cloud, providing a key/attribute store with a schemaless design.
So, if your coders do need to store such data, then yes, that would be one of the possible choices.
Correct me if I am wrong, when you code an app with a database, you can put the database on same/different server, and that can be on premise or on the cloud and you kind of link both together.
Correct. But also you can have your own server with the database which you need to manage yourself, or you can choose some cloud service which will provide the database for you but will keep the underlying server and other maintenance activity managed for you, so you no need to worry/spend your time on that.
How is this is useful when using it for the app you're developing?
It is important to understand what your requirements are for data storage in order to pick a proper one. This question perhaps should be addressed not to you, but to your coders, who are building the app and can consolidate their requirements to the database store. Usually, they will tell you exactly what they need, and you may give them some ideas or advice of the alternatives, if any (That may be a similar solution with extra functionality or the way how the data is stored or processed, or have more built in integrations that may be important for you, or a decision whether keep own installation or use cloud managed service)
For your further possible question about When should I use a NoSQL database instead of a relational database? Is it okay to use both on the same site? see this thread
Update based on further questions:
If I develop an application with a database whose tables are on Azure, can I call let's say functions or data from it to my main application that is hosted on premise? What's the benefit of doing that versus hosting the tables on premise other than it's largely scalable and highly available?
Perhaps you need to better understand the relationship between App (Application) and DB (Database). The Database is a standalone system, which store the data, reply to the incoming queries (receive request, process it, return the result). In overall to the DB is not important who is requesting the data. It is a "passive" system. (There are some cases when DB can trigger further processes in data processing pipelines, but that is beyond this scope).
The App in opposite is an active system in App<->DB relationship. (Also leave behind more advanced designs where App is not just a 1 system). App receive requests, process them (may do external requests to other "services" if that is necessary), give a response (with or without data) to the requester. In App<->DB relationship the external requests is what happening. At some point App need some data from the DB, so App make a request to the DB, obtain the response and continue its own logic.
Where App server and DB server are placed is not that important (for simplicity). The important part is whether DB server is accessable for the requests. DB can be on-prem with public static IP address, it can be in cloud on your own server which has public static IP address (sometimes that is archived in different ways but we skip that for simplicity), that can be a Database as a Service cloud solution, where you do not need to have a server and configure the database, but have a url endpoint which you need to use to query the DB.
I appreciate the answer, and I pretty much agree with what you're saying.
But my questions goes beyond what the requirements are for the developers.
I'll modify the question. If I develop an application with a database whose tables are on Azure, can I call let's say functions or data from it to my main application that is hosted on premise? What's the benefit of doing that versus hosting the tables on premise other than it's largely scalable and highly available?
Azure Storage Tables are the "Notepad" of NoSQL Databases. If you want quick and easy key/value pairs, tables is the way to go. If you are looking for the "Word" of NoSQL in Azure then Cosmos DB is where it's at. Cosmos DB offers global distrobution, better features and better SLA (see comparison). Tables are cheaper too.
Azure also supports MySQL, PostGreSQL, MariaDB and MSSQL as PaaS offerings if you wish to use a traditional database.
My organization wants to use Microsoft Text Analytics API for sentiment analysis. But my employer concern is that MS will be using that data for the live training of their sentiment engine. Is this the case?
See the Microsoft Trust Center for your answer.
How we manage your data
With Microsoft, you are the owner of your customer data.
Microsoft will use your customer data only to provide the services we
have agreed upon, and for purposes that are compatible with providing
those services. We do not share your data with our
advertiser-supported services, nor do we mine it for marketing or
advertising. If you leave the service, we take the necessary steps to
ensure the continued ownership of your data.
I am creating an azure based application that must be pci compliant. There is an understanding within my company that to meet this compliancy any personally identifiable information (PII) should be stored encrypted.
I have a number of questions.
Is it true that pci compliance means encrypting PII within the data store?
What are my options with this on Azure?
I would like to be storing data in documentdb as this would be the closest match to the format of the data within the application. Most of the data is document based and json. Would this meet the PCI compliance standards?
Does it make a difference if the data store that contains payment and card info is different to that containing the PII?
The question regarding what PCI compliance requires is best directed to your organization's compliance officer. They are the one that will ultimately have to "sign off" on your solution so they control the specifications you're working towards.
As for what your options are, mfanto pointed out the SQL support for the new tiers. There's also Azure Storage which now has encryption extensions. Document DB doesn't have anything yet to my knowledge. And if you're running your own database, Windows VMs have had support for bitlocker drive encryption on data drives for some time now.
While the sample uses local files, it should be noted that Azure Encryption Extensions supports streams as well for all upload/download methods - and nothing is ever written to disk (streams are encrypted/decrypted on the fly).
UploadFromStreamEncrypted(...)
DownloadToStreamEncrypted(...)
https://github.com/stefangordon/azure-encryption-extensions/blob/master/AzureEncryptionExtensionsTests/FunctionalTests.cs#L107
Cosmos DB (formerly DocumentDB) now supports encryption-at-rest. It is enabled by default in every region. There is no impact to cost or performance SLA. Note: The local emulator does not support encryption-at-rest (though the emulator is only for dev/test purposes).
As far as compliance goes, you'll need to talk with a compliance/legal expert for that.
For more info on Cosmos DB encryption-at-rest, see this post.
I've been researching the new unified API for office365 (graph.microsoft.com). Currently one can authenticate to it using an organisationalID and it will return information from emails-from-exchange/OneDrive4Business/Skype4Business(future)/calendar-from-exchange/exchange-contacts/etc...
The ability to surface such a plethora of information from one endpoint strikes me as being hugely valuable. In the future I'd really like to see information relating to Azure be surfaced in here also. For example, I'd love to use graph.microsoft.com to access lists of:
Azure storage accounts
Azure SQL DB servers and databases
Azure SQL DB firewall rules
Azure ML experiments
Countless other things
Are such things on the roadmap?
That is a great question, and it's great to see such enthusiasm around a single endpoint surfacing this information. We're not sure when Azure resource providers would be a part of the unified API, but it would be great to have them. There are a number of other teams who are very interested, and in general we've been pretty focused on user or person centric developer experiences here as you should be able to see from Yina Arena's //Build or Ignite presentations.
I'm quite new in Development for Azure, I have a asp.net mvc 4 application in a Azure Cloud Service.
I have a application that has a considering quantity of transaction providing by API and I need to implement some applications loggings for improve the daily diagnostic, I'm looking for a tutorial that store those into a Blob Storage instead of SQL Database, but without relevant success.
Blob Storage sounds good because I don't need to increase substantially my database that also has all the business data and could compromise a business resource (Database) becoming slower because of log transactions.
If I decide to go to storage in SQL database I'm thinking in use Log4Net.
What you guys suggest and send me tutorial that I can follow, please.
Thank you.
Sorry our logging guidance is a little hard to find - something that we are currently working on resolving - but for now please take a look at the following resources:
Client logging overview - Essentially all client library operations are output using System.Diagnostics, so you intercept and write to text / xml file just using a standard TraceListener.
Analytics and Server logs - We have extensive service side logging capabilities as well - which troubleshooting distributed apps much simpler.
Let me know if you have any questions.
Jason