I am using Microsoft Azure Storage Explorer to move patient resource data from MsSQL to Azure cosmos DB of FHIR server. I have installed the FHIR server using the below github link.
https://github.com/Microsoft/fhir-server/blob/master/docs/DefaultDeployment.md
I am able to move MsSQL server data inside FHIR cosmos db server but data format is not matching with FHIR server apps.
Example :- I have patient data with sql server side and we want to move all data with FHIR cosmos db ("resource type ":"Patient")and query on it. FHIR sever apps/services is not able to map with MsSQL server data.
Are there any Azure functions that can be run so that the bulk
ingestion of data into the FHIR server ?(Posting data with Postman is one way which is not feasible for bulk data)
Thanks in Advance.
#Vinayaka you are on a right track with Azure function.
In a nutshell, it is a simple post/put requests of FHIR resources from MS SQL into FHIR Server endpoints.
One approach could be a simple Azure function or a console app, that loops through FHIR json resources and post them in async way.
My humble advice: bump up capacity/throughput of FHIR server before running your ingestion process/load, and downgrade it as needed once ingestion of FHIR resources is completed.
You could also reuse Microsoft FhirImporter function for your case.
You could build a Data Factory pipeline to load the data to CosmosDB in the correct format. You may need to do some transformation to get the data into the format that FHIR expects.
Related
I'm trying to transfer all of my data from SQL server to azure api for fhir.(Managed Server)
Any idea what should be the starting point for this task. I have thousands of patients in the SQL server.
I need to transfer all of my current data into azure cosmos db. Also, our EMR stores all the data into SQL Server. So once I'm done with this bulk transfer any idea how can I transfer this new data into azure cosmos db every day.
I will really appreciate your help on this. Please let me know if you have any questions.
I have tried to convert my data into FHIR resources. I can insert them manuall into FHIR server. But I cannot do this for all the data.
If you're looking to transfer this data every day, keeping both in sync, you probably need to look at some Middleware to handle this for you, such as Mirth Connect.
A Mirth channel could look for new records in SQL Server, transform the data into a bundle of FHIR resources, then either pass the bundle on to an API of your own which handles the FHIR server update, or run the update itself inside the channel.
The downside with this approach is that the FHIR transformer plugin for Mirth requires a commercial license. You could also use the open source version of Mirth, transform to HL7, then run the HL7 files through the open source Microsoft FHIR Converter to produce FHIR resources, then run the insert/update yourself. (More work on your end.)
I am thinking about using Snowflake as data warehouse. My databases are in Azure SQl Database and I would like to know what tools I need for etl my data from Azure SQL Database to Snowflake.
I think Snowpark could work for data transformations, but I wonder what other code tool could I use.
Also, I wonder if I use azure blob storage as staging area or snowflake has its own.
Thanks
You can use HEVO data a third-party tool where you can directly migrate data from Microsoft SQL Server to Snowflake.
STEPS TO BE FOLLOWED
Make a connection to your Microsoft SQL Server database.
Choose a replication mode.
Create a Snowflake Data Warehouse configuration.
Alternatively, You can use SnowSQL to Connect Microsoft SQL Server to Snowflake where you export data from SQL Server to SSMS, upload the same to either Azure storage or S3, and move the data from Storage to Snowflake.
REFERENCES:
Microsoft SQL Server to Snowflake
How to move the data from Azure Blob Storage to Snowflake
Does anybody know if there is a way you can import data into an Azure SQL Database, using an GraphQL API? Or if you could create a connection within an iPaas system to send data to the Azure SQL database?
Since you want to exchange data between azure sql database and GraphQl.
Directus can be used which is an open-source Data Platform that enables anyone to access and manage the database content.
Directus will give us a GraphQl API to be used with the data .
Refer the following article to setup the azure sql database and directus .
I am trying to find an API or service to fetch the metadata of tables in Azure SQL database. However, I can't find anything. I only have an API that gets metadata of a database.
There are no Azure ARM APIs for reading and writing from a database.
To read the metadata you must connect to the database with a SQL Server client and issue metadata queries, like
select *
from sys.tables
etc. You can easily do this with PowerShell, SQLCMD, or mssql-cli.
In the Azure environment, I have an Azure SQL Db and a CosmosDb Graph. Using an Azure Data Factory, I
need to insert/update data from the Sql db to the GraphDb.
My thinking is that I need to first transform the data to json and from there insert it into the GraphDb.
Is this the way to go? Are there any other ways?
Thank you.
1.Based on the ADF copy activity connector and the thread: How can we create Azure's Data Factory pipeline with Cosoms DB (with Graph API) as data sink ? mentioned by #silent,Cosmos db graph api connector is not supported in ADF so far. You could vote up this feature in this feedback link which is updated at April 12, 2019.
2.Cosmos db migration tool isn't a supported import tool for Gremlin API accounts at this time. Please see this link:https://learn.microsoft.com/en-us/azure/cosmos-db/import-data
3.You could get an idea of graph bulk executor .NET library now.This is the sample application:git clone https://github.com/Azure-Samples/azure-cosmosdb-graph-bulkexecutor-dotnet-getting-started.gi