I cant seem to run my Synapse Notebook in a pipeline, however if i run it by itself it works fine.
The error i get is simply : "Operation on target LoadFactSalesDoc_without_DimResource failed: MessageQueueFullException: The message queue is full or is completed and cannot accept more items."
Anyone had this error before?
I tried running the notebook in a different pipeline, I also tried allocating more ressources to the notebook. It always has the same error.
Related
I have a Synapse Pipeline which runs a notebook containing unit tests before executing the business job (another notebook). The unit test notebook references the functions using the mssparkutils.notebook.run() command, and works fine when I run the notebook on its own in Synapse Studio. However, when I trigger the notebook in a pipeline, it fails with the error:
{
"errorCode": "6002",
"message": "NameError: name 'get_latest_report_per_user' is not defined",
"failureType": "UserError",
"target": "Run Tests",
"details": []
}
get_latest_report_per_user is defined in the referenced notebook. The reference works fine when run on its own outside of the pipeline.
The above snippet is from the unit test notebook I am running, and get_latest_report_per_user is defined in the "dependency" notebook.
I tried using the magic %run command instead of mssparkutils.notebook.run() to no avail.
Why is this only failing when executed as part of a pipeline?
I've been having similar issues and some of them were resolved by "Publishing" the Notebooks. You could also try the "Enable unpublished notebook reference" option under Notebook properties, but I can't recall if this option is respected by Notebooks which are invoked by a Synapse Pipeline.
We have encounter the issue in our UAT environment however it was working fine in DEV environment and as per the advise we have execute the pipeline multiple time but still isuee is there.
Error code -1000
Failure type - System error
Detail- Fail to complete sub job b.This could be a transient issue and you may re-run the job.if it fails again continuously contact customer support
I was trying to execute a pipeline in Debug mode. After clicking on Debug, notification says the pipeline started running but there is no response there after and the output screen also shows no entry of the pipeline being executed. Anyone have faced this issue before ?
Deploying a Linux VM in azure with custom script extension. It runs bunch of script in sequence till end, which I can see in the custom log. Even though script execution is finished deployment is in progress. Other deployment is getting delayed because of this. Any solution.
I am currently getting the following error on VSTS when trying to do publish a release to a Azure:
The release definition cannot be saved because the environment 'App-Service-Template' references a service endpoint that is in dirty state. Update the endpoint(s) and retry the operation. Details: 'System.Collections.Generic.List`1[System.String]'
I have tried the following troubleshooting steps and still get the same error:
I have recreated the service end point in VSTS and it fails
I have recreated the Resource group in Azure that the service point connects to and have tried to connect the end point to a empty resource group and it fails
I followed the manual steps of creating the end point connection, i then can verify the status of the connection which passes. I then try to publish the release to Azure and get the above error message.
Lastly i have tried all the MS VSTS recommended troubleshooting with still no luck https://learn.microsoft.com/en-us/vsts/pipelines/release/azure-rm-endpoint?view=vsts
I am all out of ideas. Any Help would be appreciated.
Cheers
What ultimately fixed this issue for us was abandoning and deleting the failed releases, and then a new build and release triggered from that new build.
(one weird quirk we encountered was that the release sat in the last stage for over 8 minutes: 'wait for console output from agent'... when I left the release and came back it said successful with the last stage only taking 27 seconds)
We had the exact same error. I also ran through the points on your list with no luck, before arriving at the solution with a new build and release.
I have managed to resolve the issue. It was caused by a corrupt resource group in Azure. Deleted the resource group and created it with a different name and this worked. No idea why this happened as there were no logs and resources where running in the group, just couldn't deploy new or change existing ones.