How to call external function in background tasks in arangodb - arangodb

I have a functionality fully working and I want to call this every 30 min from background task. But It is not calling and throwing error as 'undefined'.
app.js
function hourly() { require("console"); console.log('I am running');}
controller.get('/testOnce', function(req, res) {
var tasks = require("org/arangodb/tasks");
tasks.register({
id : "Test",
name : "Testing background task",
period : 5,
command : "hourly()"
});
});
I tried defining hourly in a separate js and then calling that with 'require' But this throws cannot locate module 'myjob'
myjob.js
function hourly() { require("console"); console.log('I am running');
app.js
controller.get('/testOnce', function(req, res) {
var tasks = require("org/arangodb/tasks");
tasks.register({
id : "Test",
name : "Testing background task",
period : 5,
command : "var job = require('myjob');"
});
});

The contents of the command attribute cannot refer to variables defined in other scopes. For example, in the app.js variant you're using a variable named hourly which may not be present anymore when the command gets executed.
In the simple case of just logging something, the app.js variant could be made working if its command parameter is changed to the following (which won't require any variables):
var tasks = require("org/arangodb/tasks");
tasks.register({
period : 5,
command : "require('console').log('I am running from inline command');"
});
The variant that defines the job function in a separate file (named myjob.js) can be made working by making the function available via an export of that module:
function hourly() {
require("console").log('I am running from myjob.js');
}
exports.hourly = hourly;
This is because a require() will only expose what the module exported. In the above case the module will expose a function named hourly, which can now be invoked from a background task as follows:
var tasks = require("org/arangodb/tasks");
tasks.register({
period : 5,
command : "require('myjob').hourly();"
});
Please note that in order for this to work, the file myjob.js needs to be located in the module search path. IIRC that is js/node by default. Also note that this directory already includes the bundled modules and may change on ArangoDB updates.
If the regular command is to be executed from within a Foxx route, then using Foxx queues might also be an option as they should allow putting the script with the job function inside the application directory. However, I have not tried this yet.

The "Foxx way" of solving this would be using the queue and a script-based job (introduced in 2.6).
I've covered this in the last Foxx webinar and am working on a blog post and cookbook recipe.
The problem with doing it this way is that Foxx jobs can not be periodic in 2.6. The feature is planned for 2.7 but with 2.6 only just having been released, you probably won't be able to use it any time soon.
Personally I would recommend using an external scheduler and invoking the Foxx script from there (via the foxx-manager CLI or the HTTP API).

Related

Managing multiple processes within a NodeJS application

Let's say I'd want to control (start/stop) several other NodeJS scripts from within one "main" NodeJS app. However, not necessarly NodeJS Scripts exclusively, but also simple bash scripts.
I'm looking into the following solution, using execa
// simplified
const managedProcesses = [];
async function Start (pkg) {
const runningProcess = await execa(pkg.run_command, {
cwd : pkg.path
});
return runningProcess;
}
async function Stop (pkg) {
// somehow stop the child process
return
}
const someProcess = await Start({
run_command : 'node app.js',
path : './path/to/my/script/'
});
// Keep Reference of process
managedProcesses.push(someProcess);
I first thought pm2 would be a viable solution, but I guess this would only fit for NodeJS-only scripts.
What Problems could I run into using the approach above ?
Should I consider moving forward with this idea ?
For node.js subprocesses there is the cluster module and I strongly recommend using this. For general subprocesses (e.g. bash scripts as you mentioned) you have to use child_process (-> execa). Communication between processes may then be accomplished via grpc. Your approach is fine, so you can consider moving forward with it.
I decided to go full with pm2 for the time being, as they have an excellent programmatic API - also (which I only just learned about) you can specify different interpreters to run your script. So not only node apps are possible but also bash, python, php and so on - which is exactly what I am looking for.

Azure function run code on startup for Node

I am developing Chatbot using Azure functions. I want to load the some of the conversations for Chatbot from a file. I am looking for a way to load these conversation data before the function app starts with some function callback. Is there a way load the conversation data only once when the function app is started?
This question is actually a duplicate of Azure Function run code on startup. But this question is asked for C# and I wanted a way to do the same thing in NodeJS
After like a week of messing around I got a working solution.
First some context:
The question at hand, running custom code # App Start for Node JS Azure Functions.
The issue is currently being discussed here and has been open for almost 5 years, and doesn't seem to be going anywhere.
As of now there is an Azure Functions "warmup" trigger feature, found here AZ Funcs Warm Up Trigger. However this trigger only runs on-scale. So the first, initial instance of your App won't run the "warmup" code.
Solution:
I created a start.js file and put the following code in there
const ErrorHandler = require('./Classes/ErrorHandler');
const Validator = require('./Classes/Validator');
const delay = require('delay');
let flag = false;
module.exports = async () =>
{
console.log('Initializing Globals')
global.ErrorHandler = ErrorHandler;
global.Validator = Validator;
//this is just to test if it will work with async funcs
const wait = await delay(5000)
//add additional logic...
//await db.connect(); etc // initialize a db connection
console.log('Done Waiting')
}
To run this code I just have to do
require('../start')();
in any of my functions. Just one function is fine. Since all of the function dependencies are loaded when you deploy your code, as long as this line is in one of the functions, start.js will run and initialize all of your global/singleton variables or whatever else you want it to do on func start. I made a literal function called "startWarmUp" and it is just a timer triggered function that runs once a day.
My use case is that almost every function relies on ErrorHandler and Validator class. And though generally making something a global variable is bad practice, in this case I didn't see any harm in making these 2 classes global so they're available in all of the functions.
Side Note: when developing locally you will have to include that function in your func start --functions <function requiring start.js> <other funcs> in order to have that startup code actually run.
Additionally there is a feature request for this functionality that can voted on open here: Azure Feedback
I have a similar use case that I am also stuck on.
Based on this resource I have found a good way to approach the structure of my code. It is simple enough: you just need to run your initialization code before you declare your module.exports.
https://github.com/rcarmo/azure-functions-bot/blob/master/bot/index.js
I also read this thread, but it does not look like there is a recommended solution.
https://github.com/Azure/azure-functions-host/issues/586
However, in my case I have an additional complication in that I need to use promises as I am waiting on external services to come back. These promises run within bot.initialise(). Initialise() only seems to run when the first call to the bot occurs. Which would be fine, but as it is running a promise, my code doesn't block - which means that when it calls 'listener(req, context.res)' it doesn't yet exist.
The next thing I will try is to restructure my code so that bot.initialise returns a promise, but the code would be much simpler if there was a initialisation webhook that guaranteed that the code within it was executed at startup before everything else.
Has anyone found a good workaround?
My code looks something like this:
var listener = null;
if (process.env.FUNCTIONS_EXTENSION_VERSION) {
// If we are inside Azure Functions, export the standard handler.
listener = bot.initialise(true);
module.exports = function (context, req) {
context.log("Passing body", req.body);
listener(req, context.res);
}
} else {
// Local server for testing
listener = bot.initialise(false);
}
You can use global variable to load data before function execution.
var data = [1, 2, 3];
module.exports = function (context, req) {
context.log(data[0]);
context.done();
};
data variable initialized only once and will be used within function calls.

Setup Heroku Scheduler job to email all users (Meteor/MongoDB)

Does anyone know if it's possible to make a Heroku Scheduler job that would send an email to all of my users once per day? I'm using Meteor and MongoDB.
I can see that the Heroku Scheduler can run a command such as "node somefile.js" but I can't seem to figure out how to make a connection to the mongodb in a file like this. Can I somehow tap into the DB without involving Meteor in this?
Any help would be appreciated!
I eventually found a package to do so: synced-cron. Basically, you need to setup a method in which use the package to fire a recurring job.
The package website also has a sample code:
SyncedCron.add({
name: 'Crunch some important numbers for the marketing department',
schedule: function(parser) {
// parser is a later.parse object
return parser.text('every 2 hours');
},
job: function() {
var numbersCrunched = CrushSomeNumbers();
return numbersCrunched;
}
});
Here you just need to replace the code in the job function to send out the email.
The job supports schedules like "every 5 minutes", "at 5:00pm", etc. The package relies on the text parser in Later.js to parse the schedule. You can refer to the Later.js doc.
Two different options.
The first is to use Heroku's scheduler,
In which you create a text file in your bin directory:
#! /app/.heroku/node/bin/node
var test = require('./jobToDo') //put your job in this file (jobToDo.js)
Now you don't have to put the job in another .js file, but it makes it easier to work with, rather than coding in a plain text file. (put again that is up to you)
The first line #! /app/.heroku/node/bin/node may be different for you depending on how your configuration is set up, depending on your OS and node/npm set up.
The second option is a cron style library. This will allow you to decide when you want your code to run.
This is pretty easy, and for me the preferred method.
var CronJob = require('cron').CronJob;
var fn = function(){
// Do Something
}
var job = new CronJob({
cronTime: "00 00 02 * * 1-5",
onTick: fn,
start: true,
timeZone: 'America/Los_Angeles'
});
You can look at documentation on github

Execute node command from UI

I am not very much familiar with nodejs but, I need some guidance in my task. Any help would be appreciated.
I have nodejs file which runs from command line.
filename arguments and that do some operation whatever arguments I have passed.
Now, I have html page and different options to select different operation. Based on selection, I can pass my parameters to any file. that can be any local node js file which calls my another nodejs file internally. Is that possible ? I am not sure about what would be my approach !
I always have to run different command from terminal to execute different task. so, my goal is to reduce that overhead. I can select options from UI and do operations through nodejs file.
I was bored so I decided to try to answer this even though I'm not totally sure it's what you're asking. If you mean you just need to run a node script from a node web app and you normally run that script from the terminal, just require your script and run it programmatically.
Let's pretend this script you run looks like this:
// myscript.js
var task = process.argv[2];
if (!task) {
console.log('Please provide a task.');
return;
}
switch (task.toLowerCase()) {
case 'task1':
console.log('Performed Task 1');
break;
case 'task2':
console.log('Performed Task 2');
break;
default:
console.log('Unrecognized task.');
break;
}
With that you'd normally do something like:
$ node myscript task1
Instead you could modify the script to look like this:
// Define our task logic as functions attached to exports.
// This allows our script to be required by other node apps.
exports.task1 = function () {
console.log('Performed Task 1');
};
exports.task2 = function () {
console.log('Performed Task 2');
};
// If process.argv has more than 2 items then we know
// this is running from the terminal and the third item
// is the task we want to run :)
if (process.argv.length > 2) {
var task = process.argv[2];
if (!task) {
console.error('Please provide a task.');
return;
}
// Check the 3rd command line argument. If it matches a
// task name, invoke the related task function.
if (exports.hasOwnProperty(task)) {
exports[task]();
} else {
console.error('Unrecognized task.');
}
}
Now you can run it from the terminal the same way:
$ node myscript task1
Or you can require it from an application, including a web application:
// app.js
var taskScript = require('./myscript.js');
taskScript.task1();
taskScript.task2();
Click the animated gif for a larger smoother version. Just remember that if a user invokes your task script from your web app via a button or something, the script will be running on the web server and not the user's local machine. That should be obvious but I thought I'd remind you anyway :)
EDIT
I already did the video so I'm not going to redo it, but I just discovered module.parent. The parent property is only populated if your script was loaded from another script via require. This is a better way to test if your script is being run directly from the terminal or not. The way I did it might have problems if you pass an argument in when you start your app.js file, such as --debug. It would try to run a task called "--debug" and then print out "Unrecognized task." to the console when you start your app.
I suggest changing this:
if (process.argv.length > 2) {
To this:
if (!module.parent) {
Reference: Can I know, in node.js, if my script is being run directly or being loaded by another script?

using streamlinejs with nodejs express framework

I am new to the 'nodejs' world.So wanting to explore the various technologies,frameworks involved i am building a simple user posts system(users posting something everybody else seeing the posts) backed by redis.I am using express framework which is recommended by most tutorials.But i have some difficulty in gettting data from the redis server i need to do 3 queries from the redis server to display the posts.In which case have to use neested callback after each redis call.So i wanted to use streamline.js to simplify the callbacks.But i am unable to get it to work even after i used npm install streamline -g and require('streamline').register(); before calling
var keys=['comments','timestamp','id'];
var posts=[];
for(var key in keys){
var post=client.sort("posts",'by','nosort',"get","POST:*->"+keys[key],_);
posts.push(post);
}
i get the error ReferenceError: _ is not defined.
Please point me in the right direction or point to any resources i might have missed.
The require('streamline').register() call should be in the file that starts your application (with a .js extension). The streamline code should be in another file with a ._js extension, which is required by the main script.
Streamline only allows you to have async calls (calls with _ argument) at the top level in a main script. Here, your streamline code is in a module required by the main script. So you need to put it inside a function. Something like:
exports.myFunction = function(_) {
var keys=['comments','timestamp','id'];
var posts=[];
for(var key in keys){
var post=client.sort("posts",'by','nosort',"get","POST:*->"+keys[key],_);
posts.push(post);
}
}
This is because require is synchronous. So you cannot put asynchronous code at the top level of a script which is required by another script.

Resources