Starting July 1st, 2021, SQL Server Agent fails when using sp_send_dbmail to execute a stored procedure containing a linked server. If I remove the linked server portion of the code from the stored procedure the job runs successfully.
The job simply executes the following SQL statement:
EXECUTE msdb.dbo.sp_send_dbmail
#recipients = 'My.Name#MyEmailAddress',
#subject = 'Daily Security Check',
#query = 'EXECUTE MyDatabase.sec.pr_LinkedServerQuery',
#attach_query_result_as_file = 1
I can successfully run this code manually from a query window while logged on as the Agent. But if I R-click the job and select “Start job at step…” the agent job fails. Also, if I edit the job by removing the dbmail procedure and simply run EXECUTE MyDatabase.sec.pr_LinkedServerQuery the agent job is successful. It is only when nested inside the dbmail procedure that it fails. This job has previously run successfully for over a year.
I am using SQL Server 2017.
The logs show an error about sp_send_dbmail but searches on this error don’t seem to reflect the issue I am having.
Executed as user: HCI\AgentName. Microsoft (R) SQL Server Execute Package Utility Version 14.0.3381.3 for 64-bit Copyright (C) 2017 Microsoft. All rights reserved. Started: 2:19:52 PM Progress: 2021-07-06 14:19:52.96 Source: {457716A2-22BF-47F5-A08E-8A48A218911F} Executing query "DECLARE #Guid UNIQUEIDENTIFIER EXECUTE msdb..sp...".: 100% complete End Progress Error: 2021-07-06 14:19:53.11 Code: 0xC002F210 Source: Execute T-SQL Statement Task Execute SQL Task Description: Executing the query "EXECUTE msdb.dbo.sp_send_dbmail #recipients = '..." failed with the following error: "Failed to initialize sqlcmd library with error number -2147467259.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly. End Error Warning: 2021-07-06 14:19:53.11 Code: 0x80019002 Source: Subplan_1 Description: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors. End Warning Progress: 2021-07-06 14:19:53.16 Source: Notify Operator Task Executing query "EXECUTE msdb.dbo.sp_notify_operator #name=N'Andy H...".: 100% complete End Progress DTExec: The package execution returned DTSER_FAILURE (1). Started: 2:19:52 PM Finished: 2:19:54 PM Elapsed: 1.719 seconds. The package execution failed. The step failed
This issue has bested me so far. Thank you for any insight into slaying this beast!
Thank you Ramya Makam for your article.
After I ran the profiler I got a real error:
Linked servers cannot be used under impersonation without a mapping for the impersonated login.
With that it was easy to edit Linked Servers and I am back up and running!
Related
I have an Azure Function App and Azure SQL DB.
In my Azure Function I query data (Using EF Core 3) and the following code:
var stuffIWant = dbContext.UsageCounts
.Where(a=> a.Elo > 0 && a.Elo < 1300)
.GroupBy(a => a.TheHash)
.Select(b => new stuff { Hash = (long)b.Key, Count = b.Count() })
.OrderByDescending(a => a.Count).Take(10).ToList();
I am getting a very high failure rate with an error that looks like:
[Error] Executed 'FunctionName' (Failed, Id=456456-0040-4349-81e3-54646546, Duration=30220ms)The wait operation timed out.
The exception:
[Error] Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
When I execute the function it sometimes (rarely) works fine and is able to make 8 queries like this (with the bounds for Elo changing) and each takes <200ms to complete.
I can also run this in a Sandbox project on my local machine, connecting to the same Azure DB using EF with the same model and can run it hundreds of times without ever timing out, each query taking <200ms to complete.
When the Azure function does work it always goes through each of the 8 queries and returns the data, when it doesn't work it always fails at the first query.
I added a "test" query to my function:
var test = dbContext.UsageCounts.Where(a => a.Elo > 2200).Take(10).ToList();
This happens before my failing query, and always succeeds.
Why is my function timing out most of the time when this query is nowhere near the execution time limit?
My database is not set to auto-pause.
My compute and IO utilization is under 20%
To resolve this [Error] Microsoft.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding., you can try following ways:
In Tools->Options->Query Execution->SQL Server->General->Execution time-out. Set the time-out to more than 30 seconds or 0 to wait for the indefinite time.
Check connection string, e.g. connectionString="Data Source=.;Initial Catalog=LMS;User ID=test;Password=test#123"
You can refer to The wait operation timed out establishing a connection to SQL Server FROM Azure Function App, Azure Function Execution Timeout Expired and Troubleshooting connectivity issues and other errors with Azure SQL Database and Azure SQL Managed Instance
An error has started popping up in my Azure Data Factory Pipeline. I have a few Azure Function steps in the pipeline, but for some reason, one of the Azure Function steps has started returning an error. In Azure Data Factory, the error is a 3608 code after running for 1 minute 40 seconds:
Failure type: User configuration issue
Details: Call to provided Azure function 'CollateSheetsHTTPTrigger' failed with status-'InternalServerError' and message - 'Invoking Azure function failed with HttpStatusCode - InternalServerError.'.
However, in a prior run sub-pipeline, this Azure Function ran successfully on the same data (parameters and worksheet are on the only difference). The subsequent 3 runs of pipelines fail immediately (after 2 seconds) at the first Azure Function (a different AZ function now) step in each, with the same 3608 error code but different details:
Call to provided Azure function '???????????????' failed with status-'NotFound'
and message - '<html> <head><title>404 Not Found</title></head> <body
bgcolor="white"> <center><h1>404 Not Found</h1></center> <hr><center>nginx</center>
</body> </html> '.
Now it gets even stranger. After these 3 failed pipelines, the next pipeline which is pretty much the same as the previous 4 except for a few parameters, runs successfully, even though it has the same 2 AZ functions that failed before. And then the next 2 pretty similar pipelines also run successfully.
I then went and looked at the monitoring page for the 2 Azure Functions:
The first AZ function that failed, had 2 errors even though it only failed once in AZ Data Factory... the timing is slightly different for the 2 errors but they could only come from the first failed pipeline, so why does it say there are 2 errors? Then if you look at the actual error, all it says is "The operation timed out". The function was not running for more than 150 seconds so this is strange. Additionally, I have a bunch of error catching code and nothing comes up there.
The other failed AZ function steps from the other function do not show up on the monitoring page, it seems as if the first error crashed the AZ function app and then it eventually restarted?
I'm sorry I can't help but I did have a similar problem with an Azure function that executes a SOAP-call to a webservice every minute. Since 4 days this also fails with a timeout. If I run the function within my debugger it runs without problems. But the Azure Function fails every time, after 20 sec.
I'll follow this question and hope someone else can help...
An Azure Support Engineer identified the issue, it was due to a change to the azure-function-host library. The relevant issue is here https://protect-eu.mimecast.com/s/-CT5C3QxrTmREBwhgPxLm?domain=github.com and was fixed last week
I have created 2 pipeline in Azure Datafactory. We have a custom activity created to run a python script inside the pipeline.When the pipeline is executed manually it successfully run for n number of time.But i have created a scheduled trigger of an interval of 15 minutes in order to run the 2 pipelines.The first execution successfully runs but in the next interval i am getting the error "Operation on target PyScript failed: Hit unexpected exception and execution failed." we are blocked wiht this.any input on this would be really helpful.
from ADF troubleshooting guide, it states...
Custom Activity :
The following table applies to Azure Batch.
Error code: 2500
Message: Hit unexpected exception and execution failed.
Cause: Can't launch command, or the program returned an error code.
Recommendation: Ensure that the executable file exists. If the program started, make sure stdout.txt and stderr.txt were uploaded to the storage account. It's a good practice to emit copious logs in your code for debugging.
Related helpful doc: Tutorial: Run Python scripts through Azure Data Factory using Azure Batch
Hope this helps.
If you are still blocked, please share failed pipeline run ID & failed activity run ID, for further analysis.
I'm trying to create a data flow task which brings data from Excel File to SQL table. I'm doing it in SQL Server Data Tools for Visual Studio 2012 as an SSIS package.
However, when I run the task I receive below error:
[Excel Source [52]] Error: SSIS Error Code
DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The
AcquireConnection method call to the connection manager "Excel
Connection Manager" failed with error code 0xC0209303. There may be
error messages posted before this with more information on why the
AcquireConnection method call failed.
Would you be able help please?
Thank you
Complete Error message:
SSIS package "C:\Users\adm.turpan\Documents\Visual Studio
2012\Projects\Integration Services Project1\Integration Services
Project1\project.dtsx" starting. Information: 0x4004300A at Excel as
source, SSIS.Pipeline: Validation phase is beginning. Error:
0xC020801C at Excel as source, Excel Source [52]: SSIS Error Code
DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The
AcquireConnection method call to the connection manager "Excel
Connection Manager" failed with error code 0xC0209303. There may be
error messages posted before this with more information on why the
AcquireConnection method call failed. Error: 0xC0047017 at Excel as
source, SSIS.Pipeline: Excel Source failed validation and returned
error code 0xC020801C. Error: 0xC004700C at Excel as source,
SSIS.Pipeline: One or more component failed validation. Error:
0xC0024107 at Excel as source: There were errors during task
validation. SSIS package "C:\Users\adm.turpan\Documents\Visual Studio
2012\Projects\Integration Services Project1\Integration Services
Project1\project.dtsx" finished: Success.
mehtat_90 was right. you need to set run64BitRuntime to False. See below where to find it:
Project -> Properities ->
for whom, the above answers didn't work, also try this. Some times when the file is open, it creates a temp/hidden file in the same location and it is only visible to the one who opened it. So place a break point right before the load and check the value for "FileName" which is being passed. The temp/hidden files would be in thumb.db for excel and different for different file formats. Hope this helps for someone.
You should run sql job in 32 bit therefore you should select 32-bit runtime checkbox in the configuration of SSIS Job.
http://microsoft-ssis.blogspot.com/2012/09/swtich-package-from-64bit-to-32bit.html#:~:text=Right%20Click%20the%20SSIS%20project,and%20select%20false%20under%20Run64bitRuntime.&text=Edit%20your%20job%20and%20then,%22Use%2032%20bit%20runtime%22
I am using memsql community version and I am able to execute some queries but getting error for few queries like
select * from policy;
Error code:
ERROR 1705 (HY000): The query cannot be executed.
MemSQL code generation has failed: Failed to delete .fin file.
This likely indicates a permissions problem or some other issue with your installation ([install dir]/plancache may not be writable by memsqld for example). Did you install and start MemSQL via Ops?
There will probably be a more detailed error message inside of
[install dir]/tracelogs/memsql.log