I am trying to perform a replication using Azure data sync between an Azure sql database and an Azure SQL Data Warehouse. I have a table with a primary key, identity (1,1). When i try to perform the sync, I receive an error saying something related to a syntax error around "PRIMARY". Is this issue related to how pk's are managed in azure data warehouses or, simply said, this kind of replication is not supported?
SQL Data Sync synchronizes data between SQL Azure Databases and SQL Server instances only. SQL Azure Data Warehouse is not supported.
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-sync-data
Hope this helps.
Regards,
Alberto Morillo
Related
I am thinking about using Snowflake as data warehouse. My databases are in Azure SQl Database and I would like to know what tools I need for etl my data from Azure SQL Database to Snowflake.
I think Snowpark could work for data transformations, but I wonder what other code tool could I use.
Also, I wonder if I use azure blob storage as staging area or snowflake has its own.
Thanks
You can use HEVO data a third-party tool where you can directly migrate data from Microsoft SQL Server to Snowflake.
STEPS TO BE FOLLOWED
Make a connection to your Microsoft SQL Server database.
Choose a replication mode.
Create a Snowflake Data Warehouse configuration.
Alternatively, You can use SnowSQL to Connect Microsoft SQL Server to Snowflake where you export data from SQL Server to SSMS, upload the same to either Azure storage or S3, and move the data from Storage to Snowflake.
REFERENCES:
Microsoft SQL Server to Snowflake
How to move the data from Azure Blob Storage to Snowflake
I am followin this tutorial to move data from SQL to Azure Synapse https://learn.microsoft.com/en-us/azure/data-factory/load-azure-sql-data-warehouse?tabs=data-factory
However, once I get to step 5c I cannot select a a Database name, Do I have to create an Azure Synapse Database first to copy data over there? I though that is what this tutorial will do?
I have a SQL database and I want to move the data into Azure Synapse.
Thanks
Yes, in Azure data factory your source and sink needs to be already present w.r.t database scenarios.
So it is expected that you already have an Azure SQL database and Azure SQL datawarehouse in place before proceeding with copy activity
How to move data from Azure Managed Instance to Azure SQL Database by using SSIS package with Change data capture feature in it?
Please help me with links or documents.
Good news, CDC was implemented for Managed Instance and Azure SQL Database as per yesterday:
"Change data capture (CDC) records insert, update, and delete activity that applies to tables in SQL Server, Azure SQL Managed Instance or Azure SQL Databases [...]"
You can see some common scenarios in this video.
what is the best way to limit latency for SQL Azure in global applications?
My Application uses SQL Azure and would like to know based on the network location of users if its possible to connect SQL Azure near to users.
So Logically would need to have SQL Azure database with global replication but not geo-replication as each copy would serve as Master and not secondary.
Thank you in advance.
You may want to try CosmosDB to distribute data globally and obtain low latency as explained on this article and this documentation.
For replicating data using SQL Data Sync with Azure SQL Database, take in consideration paired regions which may reduce latency. With SQL Data Sync a hub database can be defined and many member database on another region, and data can be synched on both ways between the hub and any member database.
Currently, we are mirroring our local SQL server Database with Azure SQL Server Database. For this, we are using Azure Data Management gateway but the problem is we are not able to handle the update or delete scenario. Update and Delete are not reflecting in Azure SQL Server database(Mirror).
Thanks.
What type of activity are you using in your pipeline? I'm assuming a simple copy activity?
My suggestion would be to have a copy activity that lands a clone of the on premises data in the Azure SQL DB first, maybe in a staging schema set of table or something. Then have a second downstream activity that performs a stored procedure execution activity. You can code a MERGE statement or whatever in the procedure to output the data into a separate table.
It sounds like you almost want a secondary node for your SQL Server in Azure. Maybe just use an availability group?! SQL 2014 or higher required on prem though.
You can also take a look at Transactional Replication to Azure SQL database, with transactional replication, Azure SQL database acts as a subscriber, update/delete changes you made in on-premises SQL Server database will be reflected in Azure SQL database.