This question already has answers here:
Run a cron job only once after a set time
(2 answers)
Firebase Functions, run after 15 minutes of user inactivity
(1 answer)
Run a Cron Job every 30mins after onCreate Firestore event
(3 answers)
How to run a background Cloud Function and schedule a change in database?
(1 answer)
Is it ok to use setTimeout in Cloud Functions?
(1 answer)
Closed 10 months ago.
I've searched the internet but haven't found anything that works for me.
I describe what I need.
I have a function in firebase that is executed every time I add a new node to my Realtime Database
exports.makeMagic = functions.database.ref("/users/{uid}")
.onCreate((snap, context) => {
const iterate = snap.val();
if(iterate1.loops <= 5){
// run 5 minutes later
myFunction(iterate1,iterate2,iterate3);
}
});
What I need is for myFunction to make a change to the database. But it must be strictly 5 minutes after the node is created. Not before and not after. I also don't want a cron job that is always running checking.
I have also read about using settimeout
But this would block my "makeMagic" function, until the 5 minutes were up. And I think maximum can last 1 minute, I'm not sure about that.
Any help would be nice to keep trying
Thank you so much!
You can use Cloud Tasks, as explained in this article.
You will need to write another Cloud Function, of type HTTPS, which implements the desired business logic.
Related
Let's say, I have 1000 documents in a Firestore collection.
How do I execute the same 1 Cloud Function but 10 times in parallel to process 100 documents each, say every 5 minutes?
I am aware I can use a Scheduler for the "every 5 minutes" part. The objective here is to distribute the load using multiple executions of the same function in parallel to handle the tasks. When the collection grows, I would like to add more instances. For example, let's say 1 execution per 100 documents.
I don't mind having another (or more) function to handle the distribution itself, and I don't mind the number of executions. I just don't want to loop through a large collection and process the tasks in a single function execution.
The numbers given above are examples. I am also open to using other services within GCP.
If you wanna execute the Cloud Function every time some changes occur in the Firestore documents, then you can use Cloud Firestore Trigger in Cloud Functions. The Cloud Function basically waits for changes, triggers when an event occurs and performs its tasks. You can go through these documents on Firestore triggers: Google Cloud Firestore Trigger, Cloud Firestore Triggers.
In case you are concerned that Cloud Function will not be able to process the requests parallely, then you should check out this document. Cloud Functions handle incoming requests by assigning it to an instance, in case the volume of requests increases, the Cloud Functions will start new instances to handle the requests.
Let's assume you have a function that, when called, process the single document and does anything you need with it. Let's call that function doSomething and let's assume it takes the document's path as parameter.
Then, you can create a function that will be scheduled every 5 minutes. In this function, you'll retrieve all the documents, holding them in an array (let's call it documents) and do something like:
const doSomething = httpsCallable(functions, 'doSomething');
let calls = [];
documents.map((document) => {
calls.push(
doSomething({path: document.path})
);
});
await Promise.all(calls);
This will create an array of calls, then it will fire all the calls at once, obtaining parallel executions of the same function.
I have an application that it's gathering data from another application, both in NodeJS.
I was wondering, how can I trigger sending the data to a third application on certain conditions? For example, every 10 mins if there's data in a bucket or when I have 20 elements to send?
And if the call on the third parties fails, how can I repeat it after 10-15 mins?
EDIT:
The behaviour should be something like:
if you have 1 data posted (axios.post) AND [10 mins passed OR other 10 data posted] SUBMIT to App n.3
What can help me doing so? Can I keep the value saved until those requirements are satisfied?
Thank you <3
You can use packages like node-schedule which is popular to schedule tasks. When callback runs check if there is enough data(posts) to send.
I have two collections named graphData and users. 'users' collection has documents that store the user information. 'graphData' has 10 documents initially that has some dummy data regarding the number of users in the last 10 days.
Now, I want a query to fire automatically after 24 hours that finds the number of users at that time and adds it to graphData and deletes one document also (using TTL index, I figured that out)
Please tell if there is some way to do this. Thanks
I am using MERN stack (MongoDB, Express, ReactJS and NodeJS)
To add onto the excellent answer from #Ifaruki, you can use a module like node-cron, this will ensure that you code is not dependent on the OS or cloud platform you are running.
And scheduling something to run on a specific time is as simple as:
var cron = require('node-cron');
cron.schedule('* * * * *', () => {
console.log('running a task every minute');
});
However do note that this will need to have your node+express app running all the time, which I assume you are running based on the stack you have mentioned.
P.S. If you plan to run it the app all the time, do take a look at forever or pm2 node modules.
If this is not the case stick to the accepted answer.
You could use linux cron jobs to execute an specific file every 24 hours.
You need to connect with SSH to your server, lets say putty, then type in
crontab -e
Then add following
5 0 * * * /usr/local/bin/node /home/somepath/example/script.js
This crontab means, execute this script every 5 mins after midnight.
For testing you can go with
* * * * * /usr/local/bin/node /home/somepath/example/script.js
This cron job means: Execute this script at every min, with it you can first test if you did everything right to see if it works
If you are using heroku you might read this question: Using Heroku Scheduler with Node.js
This question already has answers here:
Schedule query in BigQuery
(3 answers)
Closed 4 years ago.
What I want to do is run a daily query and insert that daily data into another table.
I am a beginner to big query and the jobs part confuses me a bit since it cannot be done through the GUI.
I read here that it should be done programmatically.
https://cloud.google.com/bigquery/docs/jobs-overview
The problem here is that this all looks like one time jobs which cannot be automatically rescheduled. Is there any way to do this in nodejs or do I need to setup a chronjob or something to use for scheduling?
Does anybody have a good example or some pointers to get me started correctly.
Thanks
You can schedule a job in node with something like this:
node-cron
For this the node server will have to be running all time so that it can execute the job at the required time.
Alternatively you can create a cron job that calls a script which would perform the job.
We created a class that inherit from DefaultOrchardCommandHandler to add new command line. Our process take 8 to 12 minutes depending of the environment. We are suspecting that we hit a timeout at 10 minutes because:
if we split the job in 3 parts, everything runs fine
when we check the log, the last operation before the error is around 9 minutes 40 seconds
Is there a way to change a setting to increase the timeout from 10 minutes to 15 minutes for orchard.exe? We are using orchard 1.5.1 .
Thank you, Have a great day!
Sebastien
Error
The transaction associated with the current connection has completed but has not been disposed. The transaction must be disposed before the connection can be used to execute SQL statements.
Ok, just found out that as soon that I override the parameter maxTimeout with a default value of 10 minutes in C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Config\machine.config, .NET start respecting the timeout value in the transactionScope constructor in C#.
Thank you all!