We have encountered a critical error today - we are no longer able to create bacpac files of our live Azure production databases. Everything was working up until now, and suddenly we've started encountering the following error:
Error encountered during the service operation. Could not extract package from specified database. Error SQL70015: Deprecated feature 'String literals as column aliases' is not supported on SQL Azure.
We have a complex database schema which has been deployed live to Azure for over a year. We are relying on daily bacpacs are our only backup strategy - need help to figure out how to resume making bacpacs.
well I fell your pain... the answer here is: Replace your schema...There's no other way...
Instead of 'Column Name' use [Column Name]... instead of Select CryptColumnA 'Column A' from myTable use Select CryptColumnA as [Column A] from myTable and so forth...
We have filed a support ticket with Microsoft and the issue was acknowledged as a bug. We only had a problem with bacpac export, not import - and apparently it was due to a SQL Azure change which hardened some of the export validations.
To make the long story short, the issue has been fixed by Microsoft and we are no longer experiencing the problem - and that is without any schema changes on our end.
Related
Validation Error
I've got this weird issue where validation fails on 'additional columns' for my data sink to Azure SQL coming from a blob storage source in the Azure Data Factory GUI. No matter how many times we recreate the dataset (or specify another dataset, new) we can't get past this validation issue.
The irony of this is we deploy these pipelines from code and when we run them, we get no errors at all. This issue we have had just made life really difficult developing pipelines further as we have to do everything by code. We cant use the pipepline publish option.
Here are some screen grabs for you of the pipeline so you can see the flow.
Pipeline
Inside copyCustomer.
Source
Mapping
Sink
Any ideas on how to fix this validation would be greatly appreciated.
For what it's worth, we have recreated the dataset multiple times (clone and new) to avoid any issue with the dataset model not being the latest as per what's documented here https://learn.microsoft.com/en-us/azure/data-factory/copy-activity-overview#add-additional-columns-during-copy
Sometimes by setting the table in sink to autocreate has shown the validation to be 'fixed' but then when we go to publish it errors out again.
When your Azure SQL dataset was created long time before and is still utilizing an outdated dataset model that Additional Columns do not support, this is expected behavior.
As per official Microsoft documentation
To resolve this issue, you can just follow the error message to create a new Azure SQL dataset and use this as copy sink.
I followed error message and created new data set and it is working fine for me.
Source:
Mapping:
Sink:
Output:
I suspect here, your dataset of Sink type is incorrect. I reproduced,
same at my end. Its working fine. Kindly make sure you create a sink dataset type with Azure SQL database type connector only.
Please check below screenshots from my implementation.
If still it helps, feel free to share your sink dataset type connector details along with screenshots.
I have been trying to set up a snowpipe to ingest data from blob storage in Azure into snowflake, following this guide, I think I have done everything correctly although I am new to azure and snowflake so may have missed something obvious. Everything seems to have been set up correctly on both sides, but whenever I check the pipe status using SELECT SYSTEM$PIPE_STATUS('azure_pipe');, I get the following:
{"executionState":"RUNNING","pendingFileCount":0,"notificationChannelName":"https://snowflakedata.queue.core.windows.net/snowflakequeue","numOutstandingMessagesOnChannel":2,"lastReceivedMessageTimestamp":"2022-02-18T13:25:12.107Z","channelErrorMessage":"downloadAttributes error:Queue not found for channel Name=https://snowflakedata2.queue.core.windows.net/snowflakequeue, AccountId=6713, NotificationChannelID=2045, IntegrationID=1784764","lastErrorRecordTimestamp":"2022-02-18T17:32:47.854Z"}
I'm not sure what I have done wrong, the snowflake app has the queue contributor role in azure and I'm fairly sure I set everything else up correctly. If anyone could point me in the right direction as to how to troubleshoot this that would be really helpful!
I had the same issue as you did just this week when trying to create a Snowpipe for Azure. Using SELECT SYSTEM$PIPE_STATUS('azure_pipe'); gave the exact same error message as you have shown above. Thankfully, Snowflake Support has provided me with the answer and an explanation.
Answer:
Drop all of the objects relating to the Snowpipe (integrations, pipe, stage, etc). Then recreate them in the exact order and specification as shown in this documentation.
Explanation:
The issue for me was caused because I kept using create or replace on the objects when I was modifying them (eg changing the comment on a pipe). This re-created the object and broke the links between the objects in the Snowpipe and prevented the Snowpipe from working as intended. Dropping and starting again solved it for me.
My azure search indexer which reads from an sql table with Change Tracking, is failing with the following error
"Unable to cast object of type 'Newtonsoft.Json.Linq.JObject' to type 'System.String'."
If I reset the indexer, it'll start working fine. What is the root cause of the problem here?
We’ve identified an issue in SQL indexers that use SQL integrated change detection that affects a very small number of customers. We’re working on a fix, which will probably be deployed in production next week. We'll also improve our telemetry so that we'll be able to identify this class of issues proactively.
The workaround you’ve already used (resetting the indexer) is the best workaround for this issue.
Sorry for the trouble!
I have collected a bunch of data using my locally developed website. Now I need to analyze the data, but it seems like I cannot locate the .mdf file for the LocalDB database my website uses.
Looking at the data connection, it says myusername\localdb#abunchofnonsense.mydomainname.Models.UserDBContext.dbo. And the the connection string is data source=(localdb)\v11.0; initial catalog=mydomain.models.userdbcontext; integrated security=true.
Also, I'm using ASP.NET MVC, Visual Studio 2013, and Entity Framework if it helps.
It's probably quite confusing what I'm trying to do here. I collected some data and need to run logistic regression with it. Now the question is how can I connect Excel to this LocalDB so I can export the data I have collected?
You can import your LocalDB tables and data directly into Excel by the option Get External Data > From Other Sources under the Data tab in Microsoft Excel (2013 to be precise) - as the following screenshot shows:
A bit late, but perhaps someone can use the answer as I found this thread when running into a similar problem:
The problem is, that the localdb uses a different provider than the "normal" SQL-Connection. You need to use the "SQL Server Native Client" (in your case version 11) to connect. The provider should be installed with the localdb - if not you can find it here
To use in Excel, just choose
"From Other Sources"
- "From Data Connection Wizard"
- "Other Advanced"
- Choose your provider
- Enter the rest of your connection details
(not tested, but I see no reason why it shouldn't work): For existing connections you can edit the connection string and add/change "Provider=SQLNCLI11.1" (of course considering your installed version)
Source: I had the same problem in Installshield where you have to change the provider manually, so I just tried in Excel.
After installing a custom CLR object Sql Server Developer Tools (SSDT) VS2012 will not allow an update. The error is "Source schema drift detected. Press Compare to refresh. After refresh same thing happens.
Tried
In settings, I set the object to just Stored Procedures.
Settings ->General -> Block on possible data loss -> tried both on and off.
This sort of loop can also be caused by a referenced SSDT project failing to build. The referenced project may be missing, unloaded, or have an error which prevents the compare from completing.
This is not an answer but a clue to deal with this problem.
I was to update a colum from varchar[200] to varchar[MAX] and got this problem as well. So I logged in the server and tried to update the database manually via SQL Management Studio which was installed there, and I got this error:
"Saving changes is not permitted. The changes you have made require the folloing tables to be drpped and re-created. You have either made changes to a table that can't be re-created or enable the option Prevent saving changes that require the table to be re-created."
Seems that re-creating table is something so dangerous that "block/unblock on possible data lose" cannot handle. So I think only if we can walk around this LOCAL warning, could we update the database REMOTELY.
But, why [200] to [max] leads to re-creating table? It does not make any sense. I tried [200] to [1000], and it did not work as well. This might be the key to this problem.
And, if you do the same update in Server explorer in VS, instead of SQL Management Studio, it works. Again, why?
This can happen when a db user "changes".
The following rather scary forum page recounts issues where foreign hackers were trying to brute-force access to the "sa" db user, with each attempt changing the sa-user's date timestamp (which is seen as a schema drift):
https://social.msdn.microsoft.com/Forums/sqlserver/en-US/5c22a7b4-7a82-4717-a118-2475bc62705b/schema-compareupdate-error-target-schema-drift-detected?forum=ssdt
Here is also mentioned that you can query the sa-user a few times, to see if this is happening to you:
SELECT * FROM sys.server_principals WHERE principal_id=1
I am currently experiencing the same issue (that the sa-user is being modified; I know nothing about hackers yet) and am yet to find a solution.
Edit - I turned on logging in Windows Firewall via properties > logging, and we setup a blocking rule on port 3071, which had a lot of unexplained traffic. Then the problem went away.
I tried running VS as an administrator, it worked.