I have a large Azure Sql Database. I need to provide a sandbox to a team that is a copy of the database, but allows them to create sql objects. The data in the sandbox needs to be up to date with production. I used elastic queries, but the performance is not ideal. I've looked at data sync, but the company requires AD authentication. Restoring production periodically as the sandbox is not ideal as the team does not want to lose their work. Any suggestions? I'm sure I must be overlooking something.
I would first make a copy the production database, then create a "From the Hub" sync group.
1. Copy Database
You can easily create a copy of an Azure SQL database by going to the database blade and clicking "Copy" in the header. From there it will ask you the new database name and target server. You can put it on the same server or create a new server, that is up to you.
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-copy
Once you've done that, you now have a "sandbox" database you control which would be an exact copy of production.
2. Sync Group
After that, you can sync specific tables from production to the sandbox by creating a Azure SQL "Sync Group".
You want to initiate this from your production database since that is the source (or hub) database, so go to the database blade of your production database and choose "Sync to other databases".
Click on "New Sync Group". From there it will ask you for a sync group name which could be something like "SyncSandbox".
Select your member database(s), this would be your sandbox database, so choose "Use Existing Database" and select your sandbox database.
Choose your sync direction. This is important, since you only want to sync from production to the sandbox, select "From the Hub".
Finally you can configure the sync group. On the Tables page, select a database from the list of sync group members and select Refresh schema. Once you're done, select Save. You can also go into the properties and select the sync frequency if you want it automatic.
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-get-started-sql-data-sync
The only one thing worry me that you mentioned your team want to keep their work. I dont know how it would be possible imagine you copy database and your team created new customer with new id which is lets say 31 and then same thing will happens on production so how to resolve those conflicts. If to omit this then I would recommend you to do following.
Setup database replication
Create job Logic App or Azure function which will execute command on that replica
CREATE DATABASE Database2 AS COPY OF Database1;
I am not sure but you probably will need to run command make this database writable since if you copy replica it will be read only.
Run script to replace all sensitive data.
But keep in mind that you will have down time so probably better would be to do this job every morning so when team starts they will have fresh data
More options how to COPY
Related
I have a backup file that came from Server A and I copied that .bak files into my local and setup that DB into my Sql Server Management Studio. Now After setting it up I deployed it in Azure Sql Database. But now there were change in the Data in Server A because it's still being used, so I need to get all those changes to the Azure SQL Database that I just deployed. How am I going to do that?
Note: I'm using Azure for my server and I have a local copy of Server A database. So basically in terms of data and structure my local and the previous Server A db is the same. But after a few days Server A data is now updated and my local DB is still the same as when I just backup the db in Server A.
How can I update the DB in Azure to take all the changes in Server A and deploy it in Azure?
You've got a few choices. It's just about migrating data. It's also a question of which data you're going to migrate. Let's say it's a neat, complete replacement. Then, I'd suggest looking at the bacpac mechanism. That's a way to export a database, it's structure and data, then import it into a new location. This is one mechanism of moving to Azure.
If you can't simply replace everything, you need to look at other options. First, there's SSIS. You can build a pipeline to move the data you need. There's also export and import through sqlcmd, which can connect to Azure SQL Database. You can also look to a third party tool like Redgate SQL Data Compare as a way to pick and choose the data that gets moved. There are a whole bunch of other possible Extract/Transform/Load (ETL) tools out there that can help.
Do you want to sync schema changes as well as Data change or just Data? If it is just Data then the best service to be used would be Azure Data Migration Service, where this service can help you copy the delta with respect to Data to Azure incrementally, both is online and offline manner and you can also decide on the schedule.
My requirements are as below :
Move 3 SAP local databases to 3 Azure SQL DB.
Then Sync daily transactions or data to azure every night. If transactions of local DB are already exists in azure, update process will do on these transactions if not insert process will do.
Local systems will not stop after moving to azure. They will still goes about 6 months.
Note :
We are not compatible with Azure Data Sync process because of it's
limitations - only support 500 tables, can't sync no primary keys
table, no views and no procedure. It also increase database size on
both(local and azure).
Azure Data Factory Pipeline can fulfill my requirements but I have
to create pipeline and procedure manually for each table. (SAP has
over 2000 tables, not good for me)
We don't use azure VM and Manage Instance
Can you guide me the best solution to move and sync? I am new to azure.
Thanks all.
Since you mentioned that ADF basically meets your needs, I will try to start from ADF. Actually,you don't need to manually create each table one by one.The creation could be done in the ADF sdk or powershell script or REST api. Please refer to the official document:https://learn.microsoft.com/en-us/azure/data-factory/
So,if you could get the list of SAP table names(i found this thread:https://answers.sap.com/questions/7375575/how-to-get-all-the-table-names.html) ,you could loop the list and execute the codes to create pipelines in the batch.Only table name property need to be set.
I have two subscriptions on Azure, for the ease of argument, lets call them subscription1 and subscription2.
I have a SQL database of size 30 GB on subscription 1, and I want to move it to subscription2.
One way is to take a backup of the database (using export option on azure portal), move the bacpac file to subscription 2 using e.g storage explorer, and importing thus bacpac file to the destination server on subscription2. However, the backup and restoration process take a lot of time when the database size is too large.
So, I came up with using Transact-SQL approach, as given in this article
Using SSMS, I use the following command in the master database of destination server (server2) to copy the database from source server (server1 in subscription1) to destination server (server2 in subscription 2)
CREATE DATABASE Database2 AS COPY OF server1.Database1;
Both the servers have same login credentials so this process works, as given in the article above, and I have tested it for very small database (for testing purposes)
My question is, when I will be copying my database that is of size 30 GB as mentioned above using the Transact SQL approach, will it be using my internet bandwith, or the bandwidth of azure data centers?, just like when we use copy option from the portal
The T-SQL command you reference will not route through your computer. It goes from server 1 to server 2. It is conceptually equivalent to doing the T-SQL command from the Azure portal.
Going one level deeper, both the portal and the T-SQL DDL command in SQL Azure are routed through a control plane REST API in the region. You can see the documentation for the one in question here:
https://learn.microsoft.com/en-us/rest/api/sql/databases/createorupdate
The control plane will then initiate a background operation to create a continuous copy of your current database into a new copy on the second server. When done, it will sever the continuous copy operation and you will have a second database ready for your use.
Even one level deeper - the logic to do all of this seeding is more or less identical to the logic required internally for SQL Azure to create a new replica of your database. In premium/large reservation sizes, you will have N copies locally of each database for high availability and performance, and if one replica node were to go bad (hardware failure, etc), the system would internally need to create a new continuous copy of the remaining replicas to rebuild the right number of backing copies for your database.
One last detail - unlike traditional SQL Server commands, this is asynchronous. So, once you submit the command the rest happens in the background. So, as #Nick.McDermaid mentions, you can turn off your local machine once the command starts and check on it later without worrying about the transaction being aborted and rolled back. The only additional issue is that you need to check on the completion of the command which can take time based on the size of the database, the size of the reservation you use (which governs IOPS), etc.
I have created a Mobile App and Database in Microsoft Azure. Now I am trying to create Easy Tables from Mobile App but I am getting error message "You need database to use East Tables. Click here to create one.
Even though I have existing database Easy Tables doesn't list it.
Below is the screenshot.
I have mapped Data Connection with mobile app, below is the screenshot
It would be great if anyone can help, I am new to azure.
Your connection string must be created with name: MS_TableConnectionString
Just because you have a database doesn't mean it is linked. Click on Data Connections, then Add, then add your existing SQL database.
Note that Easy Tables won't recognize the existing tables unless you add them through Easy Tables. There are notes around the format of Id (it needs to be
a string) and other fields.
When you created your Database Server did you check "Allow azure services to access server"? That could be why you cannot see the Database listed.
I was deleting a single record in the Classic Azure Portal in Mobile Services Browse. The entire contents of the table has now disappeared. Please help!!! How do I restore the contents. The table and scripts are still there.
I have gone into the SQL database tab and clicked restore from point in time, this creates a new DB but does not give me an option to restore to Azure mobile services.
Restoring a database always creates a new database on the same server as the original database, so the restored database must be given a new name. Once complete, the restored database is a normal fully accessible online database charged at normal rates based on its service tier and performance level. If you are restoring the database for recovery purposes you can treat the restored database as a replacement for the original database, or use it to retrieve data from and then update the original database. Please check this article for details.
So if you are going to use the new database, you can re-configure your mobile service to use the new database as following snapshot shows(click the mobile service, go to configure tab then click change database):