A workflow is started upon instantiation of the the entity Hazaa. It waits for a while and then creates a new instance of Hazaa. After that, it's put to sleep as successful.
I'd expect it to fire perpetually creating a bunch of Hazaas. However, I only get 15 new ones before the procreations cease. Together with the original one that I create manually to set off the workflow-flow, there's 16 instances in total. I've tested with longer delays (up to several hours) but the behavior is consistent.
That's for CRM On-line. On premise, the behavior is similar but limited to 8 instances in grand total.
According to the harvest of links I've found, there's a setting in CRM to control the number of iterations. The problem is that my solution will be mainly deployed for on-line customers so unless I own the cloud, that's a show stopper.
I understand it's CRM protecting against the recurrence. What can I do about it?
The best solution I can think of at the moment is to set up a super workflow, firing the sub workflow 16 times. Then I'd need to have a super super workflow etc. Not a braggable in my view.
A CorrelationToken contains a counter and a one-hour "self-destruct" timer.
When the first workflow runs, a new CorrelationToken is created. The counter is set to 1 and the timer is set to one hour.
When the second workflow is started from the first workflow (even indirectly, such as in your case), this same CorrelationToken is used if its self-destruct timer has not already expired. If it has, a new CorrelationToken is created. If it hasn't, it increments the counter and resets the timer. Lather, rinse, repeat.
The second (and subsequent) workflows will only execute if the counter is 8 or less (On-Premise) or 16 or less (CRM Online)
What this really means is that in practice, if your child workflows are executing sooner than one hour apart, the CorrelationToken never gets a chance to expire, which means eventually the counter increments past the limit. It does not mean that you can execute up to 8 (or 16) of these workflows every hour.
It sounds like you already figured most of this out, but I wanted to give other readers background. So, to answer your question: if your design includes looping workflows that are executed sooner than one hour apart, you will need to consider an alternate design. It will definitely involve an external process or service.
If I'm understanding you correctly, it sounds like you're creating an infinite loop, which is why the CRM kills workflows like these, since otherwise they'll never end. On what condition would you stop making more Hazaa records? You could add a number field and increment that field on each new Hazaa and when it reaches a certain number stop the workflow.
Related
Premise: I have a calendar-like system that allows the creation/deletion of 'events' at a scheduled time in the future. The end goal is to perform an action (send message/reminder) prior to & at the start of the event. I've done a bit of searching & have narrowed down to what seems to be my two most viable choices
Unix Cron Jobs
Bree
I'm not quite sure which will best suit my end goal though, and additionally, it feels like there must be some additional established ways to do things like this that I just don't have proper knowledge of, or that I'm entirely skipping over.
My questions:
If, theoretically, the system were to be handling an arbitrarily large amount of 'events', all for arbitrary times in the future, which of these options is more practical system-resource-wise? Is my concern in this regard even valid?
Is there any foreseeable problem with filling up a crontab with a large volume of jobs - or, in bree's case, scheduling a large amount of jobs?
Is there a better idea I've just completely missed so far?
This mainly stems from bree's use of node 'worker threads'. I'm very unfamiliar with this concept
and concerned that since a 'worker thread' is spawned per every job, I could very quickly tie up all of my available threads and grind... something, to a halt. This, however, sounds somewhat silly & possibly wrong(possibly indicative of my complete lack of knowledge here), & thus, my question.
Thanks, Stark.
For a calendar-like system, it seems you could query your database to find all events occuring in the next hour, then create a setTimeout() for each one of those. Then, an hour later, do the same thing again. Then, upon any server restart, do the same thing again. You don't really need to worry about events that aren't imminent. They can just sit in the database until shortly before their time. You will just need an efficient way to query the database to find events that are imminent and user a timer for them.
WorkerThreads are fairly heavy weight items in nodejs as they create a whole separate heap and a whole new instance of a V8 interpreter. You would definitely not want a separate WorkerThread for each event.
I should add that timers in nodejs are very lightweight items and it is not problem to have lots of them. They are just stored in a sorted linked list and only the insertion of a new timer takes a little bit more time (to do an insertion sort as it is added to the list) as the list gets longer. There is no continuous run-time overhead because there are lots of timers. The event loop, then just checks the first item in the linked list to see if it's time yet for the next timer to fire. If so, it removes it from the head of the list and calls its callback. If not, it goes about the rest of the event loop work items and will check the first item in the list again the next through the event loop.
The title isn't accurate because based on what I have found in my research there doesn't seem to be a way to make a function atomic in nodejs, but I will lay out my problem to see if you people can come up with something that I have not been able to think about.
I am trying to setup a scheduler where I can set my appointment time slots say 1 hr long each and when someone makes an appointment I want to make sure that the time slot is not taken before scheduling it.
So for example I decide that I will be working from 9 am to 2 pm with a time slot of one hour. Then my schedule would be 9-10, 10-11, 11-12, 12-1, 1-2.
An appointment will come in with a start time of 11 and end time of 12. I need to make sure that slot isn't already taken.
I am using mongodb with nodejs and restify.
I understand that in my appointments collection I can set an index on a combination of values like start time and end time, as discussed here Creating Multifield Indexes in Mongoose / MongoDB.
But if I decide to change my time slot from 1 hour to say 1.5 hours then I will have scheduling conflicts as the start time and end time of entries in the database will not match up with the new interval
Currently I have a function which checks to make sure that the new appointment will not conflict but I am not sure if it will work out well when I have multiple requests coming in. This is a nodejs and restify app so basically an api with a mongodb that it talks to, to handle appointments.
I am running it with multiple workers, so I am worried that at a certain point two requests will come in at the same time, handled by two different workers for the same time slot. When my conflict checking function executes it will return saying that the slot is open for both of them since no appointment has been made yet and then there will be a scheduling conflict.
Any ideas on how to combat this, or is there something in the way javascript executes so that I shouldn't have to worry about it this? All input will be appreciated
Thanks!
I ended up using https://github.com/Automattic/kue, to queue my requests and added another endpoint where you can check the status of your request. So when you want to make an appointment your request ends up in the job queue, and you can then periodically check the status of your request. This way only one appointment request gets processed at a time so no concurrency issues.
I'm trying to implement an auto order cancel feature in my app. So i'm thinking of adding setTimeouts on Node which will cancel the user's order on a given time.
I tried adding the timer in the app but there's too much constraints.
Will multiple setTimeouts slow down the performance of our server?
Use Agenda instead of setTimeouts.
Agenda uses a MongoDB database to persist scheduled tasks(and the parameters needed for the task) so that even if the server goes down, the tasks will still run at the specified time or intervals.
References :
https://thecodebarbarian.com/node.js-task-scheduling-with-agenda-and-mongodb
https://medium.com/hacktive-devs/nodejs-scheduling-tasks-agenda-js-4b6824f9457e
Will multiple setTimeouts slow down the performance of our server?
No, it won't slow it down any more so than the CPU time used when each timer runs.
The timer design in node.js is specifically built to manage large numbers of timers well. There should be no issue with having lots of timers (tens of thousands would be fine). There's a sorted list of timers and it only uses an actual OS level timer or the "next" timer event to fire. When that fires, it grabs the next event in the list and sets an OS level timer for that one. When a new timer is created, it is inserted into the sorted list and if it's not now the first timer in the list, it will just wait its turn until it is the first one in the list.
That said, you may not actually "need" a separate timer for each order. Since you don't need millisecond or even minute level accuracy, you could maintain a list of unfinished orders with a timestamp for when they were last modified and then you could have a single interval timer that runs every several minutes that just checks which orders have exceeded your inactive time and should be cancelled. If the order list was sorted by its timestamp, you'd just check a few orders from the end until you found ones that no longer need to be cancelled.
I have a long-running SharePoint timer job and I would like to display it's progress in central administration (so I'm using SPJobDefinition.UpdateProgress(int percentage) method).
Let's say I have 50 000 elements on a list that i want to update in a foreach loop. If I place something like job.UpdateProgress((int) itemNo / itemCount) in a loop, would it send a web request to SharePoint server each time method is called (50 000 times), or only if the percentage actually changed (no more than 100 times)?
I don't want any noticeable performance degradation because of this and I suppose that more requests might slow down the job.
Also, what tool is good to see the requests and responses (Fiddler? Or something else would be better for Sharepoint?).
(in SP2010) Every time you call job.UpdateProgress, the SPJobDefinition class will send an SPRequest. SPJobDefinition does not internally track its' percent complete, so it has no way of knowing whether your new int is an update or not unless it contacts the server, so it just contacts the server. So yes, calling this 50000 times may slow down your code significantly.
The easiest way to figure out stuff like this (since the online MSDN documentation can be very sparse at times) is to use a .Net reflector on the Microsoft.SharePoint.dll. Personally, I tend to use ilSpy.
At SharePoint Saturday in Lisle, IL this weekend, Robert Bogue said there's a difference between active and running workflows. I've looked on the web, but can someone clarify?
If I can have up to millions of active workflows on the server, why can I only have 15 or so running workflows per server?
Yes, there is a difference:
"Running" Workflows are all which currently are doing something (i.e. executing an activity).
"Active" Workflows are simply all which are "running" but currently are not doing anything - e.g. waiting for OnItemChanged or DelayActivity.
The key to understand this is WorkflowEventDeliveryThrottle (here for SP2007, because the documentation for 2010 doesn't exist). The standard value for this is property is 15. That means that there are only 15 concurrent workflow which can run at the same time. After this limit is reached the workflows get queued to the OWSTimer which executes the workflows after some arbitrary time (I think the workflow timer job is set to every 5 minutes).
This Throttle can be changed by using stsadm (AFAIK Powershell doesn't work - you can change the property via code of course setting SPWebService.WorkflowEventDeliveryThrottle):
stsadm -o setproperty -pn workflow-eventdelivery-throttle -pv "20"
Now the maximum number of "running" workflows (better would be "maximum number of workflow events that can be processes simultaneously") would be 20. See some other SO post where someone plays with the parameter.
There is a nice technical blog post to understand Workflow Event Processing: About the “workflow-eventdelivery-throttle” parameter.
Similar to the throttle is the WorkflowEventDeliveryBatchSize which denotes the maximum number of workflow events that are processed in a batch.
Summary:
You can have thousands of active workflows, e.g. all waiting for the workflow item to be changed. They are not running, not finished - simply active.
There is a limited number of workflow events that can be processed at the same time (you called it "running" workflows)
You could also have thousands of running workflows, e.g. all of them might get triggered by a delay activity set to 5 minutes, but only a limited number of them is running simultaneously, the rest of them gets queued for later execution.