Does anyone know when Azure's Migration Service is going to be compatible with migrating Cassandra data over to Cosmos DB? I heard the team might be working on it a while ago and I'm wondering if there have been any updates as to when it will be available/if it's still happening?
Based on this official document,you could find two options to copy data from existing Cassandra workloads to Azure Cosmos DB.
1.Using cqlsh COPY command
2.Using Spark
However,the data migration tool is still not support Cassandra API so far.You could submit feedback here to push the progress of whatever you want.
Related
I migrated from sql server to Azure db.
While migrating I found a job named syspolicy_purge_history, do I also need to implement it on Azure db?
As I found the syspolicy_purge_history job is targeting msdb, but I could not found msdb on Azure. A guidance will be highly appreciated...
You can implement syspolicy_purge_history job in case of azure database and msdb not available for Azure SQL Data Warehouse in your case its applicable and supported all version of msdb database
Syspolicy purge history is the default job that deletes data that is older than the number of days specified in the HistoryRetentionInDays property of Policy Management. This prevents unnecessary space consumption in the MSDB system database.
For your Reference:
sp_syspolicy_purge_history (Transact-SQL)
Migrate SQL Server to Azure SQL Database
We are planning to migrate our self hosted MongoDB database to Azure CosmosDB with Mongo API. We have 186 GB of data. Server-less CosmosDB is our plan. But as we dig in to the documentation we find that, Azure Data Migration Service (DMS) don't support migration to a server-less CosmosDB.
So our plan is to create a provisioned Service of CosmosDB then migrate our data there and from that provisioned service we will migrate to a server-less CosmosDB and then finally we will delete the provisioned CosmosDB service.
But how can we achieve the second stage of our migration.? is there any particular service provided by azure for that?
And we are good if we can migrate in Online mode. Because our service can't bear a large downtime. We know that the first stage of migration (ie. From Native Mongo server to provisioned CosmosDB via DMS) can be done in Online way. But is it possible to parallelly do the online migration from Provisioned CosmosDB to a server-less CosmosDB?
If Online migration isn't possible we are OK with the offline mode as well, unless it don't requires a large downtime of the application. Is there any estimation on the time for migration?
Please shed some light to these concerns. It will be so much helpful for us to do the task. CosmosDB is a great service provided by Azure. We can't wait to see our database there.
There is a way to do an offline migration using Spark. You can learn more by reading this article.
That said you can't use serverless anyway because it has a maximum storage capacity of 50 GB.
Update: Clarified data is not 186 GB for a container so serverless is ok.
I have an Oracle DB with data that I need to load and transform into an Azure SQL Database. I have no control over either the DB nor the application that updates its data.
I'm looking at Azure Data Factory, but I really need data changes in Oracle to be reflected as near to real-time as possible.
I would appreciate any suggestions / insights.
Is ADF the correct tool for the job? If so, what is a good approach to use? If not suitable, what should I consider using instead?
For real-time you don't really want an ELT/ETL tool like ADF. Consider a replication agent like Attunity or (gulp at the licensing costs) GoldenGate.
I don't think Data Factory is not good for you. Yes you can copy data from Oracle to Azure SQL database with it. But like #Thiago Custodio said, we need need to do it to each table you have. That's too complicated.
Just reference: Copy data from and to Oracle by using Azure Data Factory.
As you said, you really need data changes in Oracle to be reflected as near to real-time as possible.
The migration/copy time must be very short. Then the data in Oracle and Azure SQL database could be same before the Oracle data changed next time. I searched a lot and didn't find any real-time copy tools. Actually, I think you want the copy could be something like 'data sync'.
I found this link Sync Oracle Database with SQL Azure, hope it could give some good ideas for you.
About the data migration or copy, You can using bellow ways:
SQL Server Migration Assistant for Oracle (OracleToSQL)
Azure Database Migration Service (DMS)
Reference tutorial:
Migrating Oracle Databases to SQL Server (OracleToSQL): SQL Server Migration Assistant (SSMA) for Oracle is a comprehensive environment that helps you quickly migrate Oracle databases to Azure SQL database.
How to migrate Oracle to Azure SQL Database with minimum downtime:
Hope this helps.
For the record, we went with a product named QLik Replicate (aka Attunity) and it is working very well!
I have asked similar question but I would like to ask question if I can use Microsoft Azure to achieve my goal.
Is streaming input from external database (postgresql) supported in Apache Spark?
I have a database deployed on Microsoft Azure Postgresql. I have a table which I want to stream access from . Using Kafka connect , it seems that I could stream access the table, however, looking on online document , I could not find database(postgresql) as a datasource .
Does azure databricks suport stream reading postgresql table ? Or is it better to use
azure HDInsight with kafka and spark ?
I appreciate if I could get some help.
Best Regards,
Yu Watanabe
Unfortunately, Azure Databricks does not support stream reading of Azure postgresql database.
Azure HDInsight with Kafka and Spark will be the right choice for your requirement.
Managed Kafka and integration with other HDInsight offerings that can be used to make a complete data platform.
Azure also offers a range of other managed services needed in a data platform such as SQL Server, Postgre, Redis and Azure IoT Event Hub.
As per my research, I have found a third-party tool name "Panoply" which integrate Databricks and PostgreSQL using Panoply.
Hope this helps.
Scenario: I have a server farm to deploy my web site. I want to try to add one server from Azure platform. Every host has a replication from one database server. I have tried to add the replication database but I got this message error:
Microsoft SQL Server Management Studio is unable to access replication
components because replication is not installed on this instance of
SQL Server. For information about installing replication, see the
topic Installing Replication in SQL Server Books Online. (New
Subscription Wizard)
After this message I suppose that something is missing in the sqlserver database Azure side.
I have googled something to solve my problem but without luck.
I have found one tool (Azure SQL Data Sync) and tried something to solve the problem but it didn't work.
From the Azure documentation, I have found that until V12 of the azure database replication wasn't available.
But what about v12?
Questions:Am I missing something with the version of the database? Should I change the database plan in Azure subscription?
Should I use some external tool that allow me to maintain sync my database server with the replication?
Replication is not yet available according to the MSDN even on V12. You can use https://azure.microsoft.com/en-us/documentation/articles/data-factory-introduction/ and https://azure.microsoft.com/en-us/documentation/articles/sql-database-get-started-sql-data-sync/. You can write your own sync logic with change tracking, Service broker and linked servers / safe CLR but better use the existing tools.
Maybe Azure Data Factory Service could be a solution for you?
https://azure.microsoft.com/en-us/documentation/articles/data-factory-introduction/