I'm working on Azure WebJob.
I started by creating a console application in Visual Studio and I published the application as a webJob in portal.azure from VisualStudio.
the WebJob is Triggered Manualy from its Webhook with username and password https://{MyWebAPP}.scm.azurewebsites.net/api/triggeredwebjobs/{MyWebJob}/run?arguments=1 2 3 from a second program.
this WebJob is verry simple. It only displays a the arguments 1,2 and 3.
when I run the program from CommandeLine like so dotnet MyProject.dll
1 2 3 it works well.
but when I run it from webHook it does not read arguments.
here is my main script :
class Program
{
static void Main(string[] args)
{
Console.WriteLine("PARAMS Passed : " + string.Join(",", args));
}
}
This is the log in the WebJob when I run from WebHook by Post request : [06/09/2018 15:19:37 > 33a9f2: INFO] PARAMS Passed :
and this is the console when I run it from commande Line : [06/09/2018 15:19:37 > 33a9f2: INFO] PARAMS Passed : 1,2,3
Can some One Help PLEASE.
Tha,ks From All.
This comes down to a VS publishing bug. The problem is that it auto-generates a run.cmd that has:
dotnet foo.dll
When it really should have:
dotnet foo.dll %*
So that the arguments get flowed into your console app.
I'll report the issue, but for now you can work around as follows:
Explicitly create a run.cmd at the root of your console app (i.e. next to program.cs). Make it contain the correct line above with %*. And obviously, use your actual dll name instead of foo.dll :)
Add it to your project, and set Copy to Output Directory to Copy Always (default is Do not copy).
That will cause your run.cmd to get deployed, and VS won't auto-generate the faulty one.
Related
I have Azure Function app that I was able to debug previously but now it would not trigger when I start it in Visual Studio debug mode. Function looks like this:
public static void RunDownloadTask([TimerTrigger("0 */30 * * * *")] TimerInfo timerInfo, TextWriter log)
{...}
When I started in debug in Visual studio I used to get output like the following, but now I only get the first two lines of message in console. The rest "The next 5 occurrences ..." does not show up, and the function never gets triggered. Has anyone experienced something like this and know how to fix it? Looks like the instances I deployed to Azure App Service continue to work, but I don't know why I can't run this in debug mode locally anymore.
Found the following functions:
ChaseFTPDownloader.Functions.RunDownloadTask
The next 5 occurrences of the schedule will be:
10/5/2020 4:30:00 PM
10/5/2020 5:00:00 PM
...
I can reproduce your problem:
This is the problem of the settings of the Visual Studio.
Solution:
Just right click your functionapp in VS and add this command to the properties of your project:
start --verbose
After that you will see the log like before:(this change is updated in recent months.)
I think your function should start. Why you don't see it triggered is because the cron expression of your timetrigger.(It has an interval of 30 minutes and will not trigger the first time.)
Please do a test and let me know whether this can help you.
There are many tutorials using the following code to create a Webjobs via the WebJob SDK 3.0 library. Specifically 'TimerTrigger'
public void DoSomethingUseful([TimerTrigger("0 */1 * * * *", RunOnStartup = false)] TimerInfo timerInfo, TextWriter log)
{
// Act on the DI-ed class:
string thing = _usefulRepository.GetFoo();
Console.WriteLine($"{DateTime.Now} - {thing}");
}
The above example should run this method as a webjob every 1 minute. However this doesn't work.
I have managed to get the webjob to work when including a setting.job file.
setting.job: { "schedule": "0 */1 * * * *" }
My question is what is the different between these two?
Update:
Please go to the azure webjobs log, then you can see it actually runs as per the timerTrigger defined by SDK(even though the Schedule is n/a, and settings.job is blank, it does not matter):
In short, When using webjob sdk 3.x, you can use TimerTrigger attribute to run the function as per the time you defined. Without using webjobs SDK(like use .zip file or publish a console project from visual studio), you can use setting.job to defined timer instead of TimerTrigger attribute.
1.When you're using webjobs SDK 3.x for timer trigger, you should add this line of code: config.AddTimers(); .
Here are my code using webjobs SDK 3.x(it's a .net core 2.2 console project created in visual studio):
The packages with latest version: Microsoft.Azure.WebJobs / Microsoft.Azure.WebJobs.Extensions / Microsoft.Extensions.Logging.Console
The code in Program.cs:
class Program
{
static void Main(string[] args)
{
var builder = new HostBuilder()
.ConfigureWebJobs(config =>
{
config.AddTimers();
config.AddAzureStorageCoreServices();
})
.ConfigureLogging((context, b) =>
{
b.AddConsole();
}
)
.Build();
builder.Run();
}
}
Then create a new file, like SayHelloWebJob.cs, and code in it:
public class SayHelloWebJob
{
public void ProcessCollateFiles([TimerTrigger("0 */1 * * * *", RunOnStartup = false)]TimerInfo timerInfo,TextWriter writer)
{
writer.WriteLine("hi, it is a testing running");
Console.WriteLine("test");
}
}
Note that in the appsettings.json file, add your storage connection string, like below:
{
"AzureWebJobsStorage": "DefaultEndpointsProtocol=https;AccountName=xxx;AccountKey=xxx;EndpointSuffix=core.windows.net"
}
Then run the project, you can see the function is triggered as per 1 minute:
2.For settings.job, eg. if you're just creating a console project, and does not use the webjobs sdk. Since you're not using webjobs sdk, you cannot use the timerTrigger attribute. At this moment, you can include the settings.job file(in it's property, set "Copy to Output Directory" as "copy if newer") in this project and configure the scheduled timer like you did in your post. After publish as webjob(from visual studio, when publish, select "Webjob run mode" as "run on demand"), it can run as per the schedule you defined in settings.job.
I have been struggling with the same problem. Here is my understanding after longer research.
There are two kind of Webjobs:
triggered
continuous
Triggered one has to be triggered manually or can be triggered by App Service as per CRON expression schedule provided in setting.job. These jobs are not present in memory once not running.
Continuous one runs always, so the process exists in memory all the time. You can schedule it using Webjobs SKD TimeTrigger attribute.
You will also notice difference between these two Webjob types in the Dashboard.
For triggered Webjobs you will see on top level jobs runs then functions invoked and eventually invocation details.
For continuous Webjobs this will be functions invoked and eventually invocation details. Job runs are missing as this is just one long running job.
Check App Service / Process explorer under Kudu w3wp process to see Webjobs processes running.
Note that continuous and triggered Webjobs have to be started in different way in Main method where you provide the configuration. All comes configured when Webjob of specific type is added via Visual Studio.
This is based on WebJobs 2.x.
My recommendation is
for periodical (e.g. once per few hours, days) jobs use triggered
ones, job when not running will not consume resources,
for more frequent jobs use continuous ones with TimeTrigger
attribute, it will consume resources all the time but will not need
extra time for start-up.
I am trying to run a Console Application in Azure Batch. I have created the Client Application having responsibility of creating and running Job & Tasks in the Azure Batch so far. My ClientApplication successfully creates a Job, a Task within the Job but when the application code uploaded on the AzureStorage get executed in Azure the Pool, the Task fails and exits with an Exit code -532462766. I could not find any information for this exit code. Please help me to fix this.
Here is my code for creating the task and it's configuration
private static async Task<List<CloudTask>> AddTasksAsync(BatchClient batchClient, string jobId)
{
Console.WriteLine("Adding {0} tasks to job [{1}]...", 1, jobId);
List<CloudTask> tasks = new List<CloudTask>();
string taskId = "topNtask";
string taskCommandLine = String.Format("cmd /c %AZ_BATCH_NODE_SHARED_DIR%\\SelfHostedWebAPI.exe {0} {1} {2}", 11109, "MSS_F_MST_______", "Normal");
//string taskCommandLine = String.Format("cmd /c %AZ_BATCH_NODE_SHARED_DIR%\\SelfHostedWebAPI.exe");
CloudTask task = new CloudTask(taskId, taskCommandLine);
tasks.Add(task);
await batchClient.JobOperations.AddTaskAsync(jobId, tasks);
return tasks;
}
And the task failure report from my Azure Portal
Corresponding hex value for -532462766 is 0xE0434352; more information about this error code/exception can be found on this SO post. Common culprits are missing assemblies (see the Assembly Binding Log Viewer). Additionally, the Windows Event Viewer might be able to provide you with more information prior to having to resort to WinDbg and SOS.
Finally I got the solution of this problem and to save one's precious time here is the solution.
We need to upload each and every .dll files and dependencies of our code that run in Azure Pool to the linked AzureStorage account. In addition, when the pool is being created and it's StartTask is initialized, a ResourceFile with SAS(Shared Access Signature, for the target .dll that it points) for every .dll file or any other type, should be given to it as a parameter.
I am running an azure webjobs SDK console application (continuous) with the recommended setup:
public static void ProcessQueueMessage([QueueTrigger("logqueue")] string logMessage, TextWriter logger)
The azure queue I am running against has ~6000 messages in it and I am running the web-job locally, as a console application.
The problem I'm having is that the processing randomly stops after processing between zero and ~30 messages. The console stays open, but no more console messages are displayed.
For example, it might just process 2 messages:
Executing: 'Functions.ProcessQueueMessage' - Reason: 'New queue message detected on 'QueueName'.'
Executed: 'Functions.ProcessQueueMessage' (Succeeded)
Executing: 'Functions.ProcessQueueMessage' - Reason: 'New queue message detected on 'QueueName'.'
Executed: 'Functions.ProcessQueueMessage' (Succeeded)
And then, nothing. There doesn't seem to be anything wrong with my internet connection and I can't trace the issues down to any particular messages.
Has anyone else had issues with this SDK?
Update:
I made sure that I was using the right versions of all of the dependencies by removing the nuget packages and then re-running install-package Microsoft.Axure.Webjobs. I am now using webjobs version 1.1.0 which has pulled in version 4.3 of azure storage.
As recommended by Matthew, I have pulled down the source code for azure webjobs to determine where the process is freezing up. Once the freez-up occurs, I pause execution and checked the running threads for what I believe is the culprit within Microsoft.Azure.WebJobs.Host.CompositeTraceWriter
protected virtual void InvokeTextWriter(TraceEvent traceEvent)
{
if (_innerTextWriter != null)
{
string message = traceEvent.Message;
if (!string.IsNullOrEmpty(message) &&
message.EndsWith("\r\n", StringComparison.OrdinalIgnoreCase))
{
// remove any terminating return+line feed, since we're
// calling WriteLine below
message = message.Substring(0, message.Length - 2);
}
_innerTextWriter.WriteLine(message);
if (traceEvent.Exception != null)
{
_innerTextWriter.WriteLine(traceEvent.Exception.ToDetails());
}
}
}
The line it freezes on is line 66 : _innerTextWriter.WriteLine(message);
_innerTextWriter is an instance of System.IO.TextWriter.SyncTextWriter
Is it possible there is some deadlock issue with this class or the way it is being used?
Some notes:
I am running in the debugger, so in this case I believe the textwriter is forwarding to the console internally
I have my batchsize set to 1 via config.Queues.BatchSize = 1;, not sure if that could matter
I'm currently working on setting up an environment on another computer so that I can see if it is reproducible somewhere other than this machine (surface book).
Update
The issue was me not understanding how the new windows 10 command prompt works. Any time you click on the command window, it goes into "select" mode which completely pauses execution of the process.
Basically: https://superuser.com/questions/419717/windows-command-prompt-freezing-randomly?newreg=ece53f5584254346be68f85d1fd2f18d
You can tell it is in this state because it will prefix the window title with the word "Select":
You have to press enter or click again to get it going once again.
So, two final comments:
1) What an incredibly confusing and un-intuitive behavior for a command window!
2) I hope some admin will come take pity on the shame I have brought upon myself and my family by deleting this question.
To get rid of this strange behavior, you can disable QuickEdit mode:
Strange. When it is in this stuck state, can you try adding a new queue message to the queue and see if that triggers? Are you sure your function isn't hanging internally? What version of the SDK are you using? You might also try upgrading to v1.1.0 which we just released last week. If there are really a bunch of messages in the queue waiting to be processed, I can't think of anything that would cause this. The queue listener in the SDK should chug along, reading batches of messages in parallel and dispatching them to your function. Have you changed any of the JobHostConfiguration.Queues configuration knobs? You haven't force updated the version of the Azure SDK have you to something higher than the WebJobs SDK supports?
Another option if you can't figure this out might be to clone the SDK, build it and debug it locally. The repo is here. The main queue processing loop is here.
I have:
Created a manifest for msdeploy to:
Stop, Uninstall, Copy over, Install, and Start a Windows service.
Created a package from the manifest
Executed msdeploy against the package against a remote server.
Problem: It executes the entire manifest twice.
Tried: I have tinkered with the waitInterval and waitAttempts thinking it was timing out and starting over, but that hasn't helper.
Question: What might be making it execute twice?
The Manifest:
<sitemanifest>
<runCommand path="net stop TestSvc"
waitInterval="240000"
waitAttempts="1"/>
<runCommand
path="C:\Windows\Microsoft.NET\Framework\v4.0.30319\installutil.exe /u
C:\msdeploy\TestSvc\TestSvc\bin\Debug\TestSvc.exe"
waitInterval="240000"
waitAttempts="1"/>
<dirPath path="C:\msdeploy\TestSvc\TestSvc\bin\Debug" />
<runCommand
path="C:\Windows\Microsoft.NET\Framework\v4.0.30319\installutil.exe
C:\msdeploy\TestSvc\TestSvc\bin\Debug\TestSvc.exe"
waitInterval="240000"
waitAttempts="1"/>
<runCommand path="net start TestSvc"
waitInterval="240000"
waitAttempts="1"/>
</sitemanifest>
The command issued to package it:
"C:\Program Files\IIS\Microsoft Web Deploy V2\msdeploy"
-verb:sync
-source:manifest=c:\msdeploy\custom.xml
-dest:package=c:\msdeploy\package.zip
The command issued to execute it:
"C:\Program Files\IIS\Microsoft Web Deploy V2\msdeploy"
-verb:sync
-source:package=c:\msdeploy\package.zip
-dest:auto,computername=<computerNameHere>
I am running as a domain user who has administrative access on the box. I have also tried passing credentials - it is not a permissions issue, the commands are succeeding, just executing twice.
Edit:
I enabled -verbose and found some interesting lines in the log:
Verbose: Performing synchronization pass #1.
...
Verbose: Source filePath (C:\msdeploy\MyTestWindowsService\MyTestWindowsService\bin\Debug\MyTestWindowsService.exe) does not match destination (C:\msdeploy\MyTestWindowsService\MyTestWindowsService\bin\Debug\MyTestWindowsService.exe) differing in attributes (lastWriteTime['11/08/2011 23:40:30','11/08/2011 23:39:52']). Update pending.
Verbose: Source filePath (C:\msdeploy\MyTestWindowsService\MyTestWindowsService\bin\Debug\MyTestWindowsService.pdb) does not match destination (C:\msdeploy\MyTestWindowsService\MyTestWindowsService\bin\Debug\MyTestWindowsService.pdb) differing in attributes (lastWriteTime['11/08/2011 23:40:30','11/08/2011 23:39:52']). Update pending.
After these lines, files aren't copied the first time, but are copied the second time
...
Verbose: The dependency check 'DependencyCheckInUse' found no issues.
Verbose: Received response from agent (HTTP status 'OK').
Verbose: The current synchronization pass is missing stream content for 2 objects.
Verbose: Performing synchronization pass #2.
...
High Level
Normally I deploy a freshly built package with newer bits than are on the server.
During pass two, it duplicates everything that was done in pass one.
In pass 1, it will:
Stop, Uninstall, (delete some log files created by the service install), Install, and Start a Windows service
In pass 2, it will:
Stop, Uninstall, Copy files over, Install, and Start a Windows service.
I have no idea why it doesn't copy over the files in pass 1, or why pass 2 is triggered.
If I redeploy the same package instead of deploying fresh bits, it will run all the steps in pass 1, and not run pass 2. Probably because the files have the same time stamp.
There is not enough information in the question to really reproduce the problem to give a specific answer... but there are several things to check/change/try to make this work:
runCommand needs specific privileges
waitInterval="240000" and waitAttempt="1" (double quotes instead of single quotes)
permissions for the deployment service / deployment agent regarding directories etc. on the target machine
use tempAgent feature
work through the troubleshooting section esp. the logs and try the -whatif and -verbose options
EDIT - after the addition of -verboseoutput:
I see these possibilities:
Time
Both machines have a difference in time (either one of them is just a bit off or some timezone issue...)
Filesystem
If one of the filesystems is FAT this could lead to problems (timestamp resolution...)
EDIT 2 - as per comments:
In my last EDIT I wrote about timestamp because my suspicion is that something goes wrong when these are compared... that can be for example differring clocks between both machines (even a difference of 30 sec can have an impact) and/or some timezone issues...
I wrote about filesystem esp. FAT since the timestamp resolution of FAT is someabout 2 seconds while NTFS has much higher resolution, again this could have an impact when comparing timestamps...
From what you describe I would suggest the following workarounds:
use preSync and postSync for the Service handling parts (i.e. preSync for stop + uninstall and postSync for install + start) and do only the pure sync in the manifest or commandline
OR
use a script for the runCommand parts
EDIT 3 - as per comment from Merlyn Morgan-Graham the result for future reference:
When using the runCommand provider, use batch files. For some reason this made it stop running two passes.
The problem with this solution is that one can't specify the installation directory of the service via a SetParameters.xml file (same for dontUseCommandExe / preSync / postSync regarding SetParameters.xml).
EDIT 4 - as per comment from Merlyn Morgan-Graham:
The timeout params apply to whether to kill that specific command, not to the closing of the Windows Service itself... in this case it seems that the Windows Service takes rather long to stop and thus only the runCommands get executed without the copy/sync and a new try for the whole run is initiated...
I had the same problem, but I don't make package.zip file.
I perform synchronization directly in one step.
The preSync/postSync solution helped me a lot and there is no need to use manifest files.
You can try the following command in your case:
"C:\Program Files\IIS\Microsoft Web Deploy V2\msdeploy"
-verb:sync
-preSync:runCommand="net stop TestSv && C:\Windows\Microsoft.NET\Framework\v4.0.30319\installutil.exe /u
C:\msdeploy\TestSvc\TestSvc\bin\Debug\TestSvc.exe",waitInterval=240000,waitAttempts=1
-source:dirPath="C:\msdeploy\TestSvc\TestSvc\bin\Debug"
-dest:auto,computername=<computerNameHere>
-postSync:runCommand="C:\Windows\Microsoft.NET\Framework\v4.0.30319\installutil.exe
C:\msdeploy\TestSvc\TestSvc\bin\Debug\TestSvc.exe && net start TestSvc",waitInterval=240000,waitAttempts=1
"-verb:sync" parameter means you synchronize data between a source and a destination. In your case your case, first time you perform synchronization between the "C:\msdeploy\TestSvc\TestSvc\bin\Debug" folder and the "package.zip". Plus, you are using manifest file, so when you perform second synchronization between the "package.zip" and the destination "computername", msbuild uses previously provided manifest twice for the destination and for the source, so each manifest operation runs twice.
I used the && trick to perform several commands in one command line.
Also, in my case, I had to add timeout operation to be sure the service were completely stopped ("ping -n 30 127.0.0.1 > nul").