ArangoDB with custom JS - node.js

Hello is there any way to use JS environment built in ArangoDB to execute custom JS? I'd like to set up path to my JS files which would be executed instead of foxx application files.

Via GitHub: https://github.com/arangodb/arangodb/issues/1723#issuecomment-183289699
You are correct that modules are cached independently of the routing
cache. Clearing the module cache (or preventing a module from being
cached) is currently not supported.
The actions mechanism is really only intended as an internal API and
only supported for backwards compatibility with early ArangoDB
versions and some edge cases.
As you may have noticed while digging through the ArangoDB source
code, Foxx provides a per-service module cache which is cleared
whenever a Foxx service is reloaded. I would strongly encourage you to
see whether Foxx fits your use case before continuing to dig into the
actions mechanism.
It's actually possible to create a Foxx service with just two files (a
manifest and a controller file) and without using repositories or
models (you can just use the same APIs available in actions).
You just need a controller file like this (e.g. ctrl.js):
'use strict';
const Foxx = require('org/arangodb/foxx');
const ctrl = new Foxx.Controller(applicationContext);
ctrl.get('/', function (req, res) {
res.send('Hello World');
});
with a manifest.json like this:
{
"name": "my-foxx",
"version": "0.0.0",
"controllers": "ctrl.js",
"defaultDocument": "",
"engines": {"arangodb": "^2.8.0"}
}
You can then mount the service (upload a zip bundle) at a path like
/db and access it:
curl http://localhost:8529/_db/_system/db
The upcoming 3.0 release will remove a lot of the existing conceptual
overhead of Foxx which will hopefully make it even easier to get
started with it.

Yes, this can be done with User Actions. Foxx was created as a more comfortable alternative and is likely a better choice for non-trivial applications. The documentation can be intimidating but Foxx services can actually be very lightweight and simple (see my other answer). If you really don't want to use Foxx for this, here's how to do it manually:
First create a virtual module in the _modules system collection:
var db = require('org/arangodb').db;
db._modules.save({
path: '/db:/ownTest',
content: `
exports.do = function (req, res, options, next) {
res.body = 'test';
res.responseCode = 200;
res.contentType = 'text/plain';
};
`
});
Then create a route that uses it:
db._routing.save({
url: '/ourtest',
action: {
controller: 'db://ownTest'
}
});
Finally, tell ArangoDB to update its routing cache so it notices the new route:
require('internal').reloadRouting();
If you install your JavaScript module to the js/common/ or the js/server/ directory you can use the module name (e.g. myOnDiskModule) instead of the virtual module name "db://owntest" in the controller.
For smaller modules you can just define the function inline using callback instead of controller:
db._routing.save({
url: '/hello/echo',
action: {
callback: `
function (req, res) {
res.statusCode = 200;
res.body = require('js-yaml').safeDump({
Hello: 'World',
are: 'you here?'
});
}
`
}
});
Remember to always update the routing cache after changes to the routing collection:
require('internal').reloadRouting();
Note: the callback implementation in 2.8 has a bug that will be fixed in 2.8.3. If you want to apply the fix manually, it's in commit b714dc5.

Related

Register new route at runtime in NodeJs/ExpressJs

I want to extend this open topic: Add Routes at Runtime (ExpressJs) which sadly didn't help me enough.
I'm working on an application that allows the creation of different API's that runs on NodeJs. The UI looks like this:
As you can see, this piece of code contains two endpoints (GET, POST) and as soon as I press "Save", it creates a .js file located in a path where the Nodejs application is looking for its endpoints (e.g: myProject\dynamicRoutes\rule_test.js).
The problem that I have is that being that the Nodejs server is running while I'm developing the code, I'm not able to invoke these new endpoints unless I restart the server once again (and ExpressJs detects the file).
Is there a way to register new routes while the
NodeJs (ExpressJs) is running?
I tried to do the following things with no luck:
app.js
This works if the server is restarted. I tried to include this library (express-dynamic-router, but not working at runtime.)
//this is dynamic routing function
function handleDynamicRoutes(req,res,next) {
var path = req.path; //http://localhost:8080/api/rule_test
//LoadModules(path)
var controllerPath = path.replace("/api/", "./dynamicRoutes/");
var dynamicController = require(controllerPath);
dynamicRouter.index(dynamicController[req.method]).register(app);
dynamicController[req.method] = function(req, res) {
//invocation
}
next();
}
app.all('*', handleDynamicRoutes);
Finally, I readed this article (#NodeJS / #ExpressJS: Adding routes dynamically at runtime), but I couldn't figure out how this can help me.
I believe that this could be possible somehow, but I feel a bit lost. Anyone knows how can I achieve this? I'm getting a CANNOT GET error, after each file creation.
Disclaimer: please know that it is considered as bad design in terms of stability and security to allow the user or even administrator to inject executable code via web forms. Treat this thread as academic discussion and don't use this code in production!
Look at this simple example which adds new route in runtime:
app.get('/subpage', (req, res) => res.send('Hello subpage'))
So basically new route is being registered when app.get is called, no need to walk through routes directory.
All you need to do is simply load your newly created module and pass your app to module.exports function to register new routes. I guess this one-liner should work just fine (not tested):
require('path/to/new/module')(app)
Is req.params enough for you?
app.get('/basebath/:path, (req,res) => {
const content = require('content/' + req.params.path);
res.send(content);
});
So the user can enter whatever after /basepath, for example
http://www.mywebsite.com/basepath/bergur
The router would then try to get the file content/bergur.js
and send it's contents.

Add MDC in Nodejs with log4j without requiring external libraries

I am facing an issue in nodejs where I have to print the username in the logs in the Metadata. Each request is having its own username and nodejs being async its overriding the username property that i have created for the singleton log.
I have read about continous-local-storage, but I am looking for a simpler solution without requiring any external libraries.
Is it advisable to create a seperate log instance per each request?
P.S.: I am very new to Node js!!
just an update, I figured out how to solve this but had to rely on the library continous-local-storage.
var myses= require('continuation-local-storage').createNamespace('myses');
app.use(function (req, res, next) {
myses.bindEmitter(req);
myses.bindEmitter(res);
myses.run(function () {
myses.set('someid','somevalue');
next();
});
});
and in some other file, like custom logger,
var thisSession = require('continuation-local-storage').getNamespace('myses');
thisSession .get('someid');

Sails with AWS XRAY

How in the world is one supposed to install AWS XRAY with Sails?
I'm attempting to translate the installation instructions to Sails' preferred ways of using Express middleware, but I'm falling flat on my face.
Most people will instantly start with "use config/http.js" to configure middleware. Well, that doesn't work in my case, because my API is consumed exclusively with Sails.io (sockets), so the http middleware config is never used.
So now, the logical step is to use policies. Well, if you've read the XRAY instructions, you know that they are trying to capture ALL requests to the app, which requires "start" and "stop" function calls, before and after routes have been configured. So, policies don't work.
So, my next step was to attempt it in the app.js, and the config/bootstrap.js files, to no avail, probably because I can't easily get the Express instance Sails is using. So, is it even possible with Sails' current config options? Anyone have any clue how to accomplish this?
To anyone that should stumble upon this, attempting to integrate AWS X-Ray into Sails.js:
I finally got it working, by building a project hook for it. If someone is ambitious enough, they are more then welcome to make it an installable hook.
IMPORTANT NOTES
The hook is designed to only run when the environment variable AWS_XRAY === 'yes'. This is a safety trap, to prevent local and CI machines from running XRAY.
The hook taps into the "before" part of the route setup. What this means is: "before routes are instantiated, use this middleware".
This code is setup to ignore the route "/_ping" (for X-Ray, it'll let the request complete as normal), which is used
for ELB health checks. These do not need to be logged on X-Ray, they
are just a waste of money. I HIGHLY recommend you read through this
code, and adjust as needed. Especially the req.headers.host and
req.connection "fixes". This was the only way I could get X-Ray to
work, without changing the repo's code (still can't find the Github
repo for it).
The req.connection.encrypted injection is just to have X-Ray report the URL as https. It's not important, unless you want your
traces to reflect the correct URL.
Because we use CloudFlare, there are additional catches to collect the end-user's IP address for requests. This should have no affect if you don't use CF, and should not require any modification. But, I have to ask, WHY aren't use using CF?
This has only gotten me so far, and I can only see basic data about
requests in the X-Ray console. I can't yet see database queries, or
other services that are in use.
RESULTS MAY VARY
Don't forget!
npm i aws-xray-sdk --save.
To install and run the X-Ray Daemon
This is the code I put together api/hooks/setup-aws-xray.js:
var AWSXRay = require('aws-xray-sdk');
module.exports = function setupAwsXray(sails){
var setupXray = false;
function injectXrayIfRequested(req, res, next){
if (
setupXray
&& !req.segment
&& req.path !== '/_ping'
) {
req.headers.host = (sails.config.environment === 'production')
? 'myapp.com'
: 'dev.myapp.com';
req.connection = {
remoteAddress: req.headers['http_cf_connecting_ip']
|| req.headers['HTTP_CF_CONNECTING_IP']
|| req.headers['X-Real-IP']
|| req.ip,
encrypted: true
};
AWSXRay.express.openSegment()(req, res, next); // not a mistake
} else {
next();
}
}
// This just allows us to get a handle on req.segment.
// This is important if you want to add annotations / metadata.
// Despite AWS's documentation, you DO NOT need to close segments
// when using manual mode and express.openSegment, it will
// do this for you automatically.
AWSXRay.enableManualMode();
return {
configure: function(){
if (process.env.AWS_XRAY && process.env.AWS_XRAY === 'yes') {
setupXray = true;
AWSXRay.setDefaultName('myapp_' + sails.config.environment);
}
},
routes: {
before: {
'/*': injectXrayIfRequested
}
}
};
};

Access Previously-Defined Middleware

Is there a way to access or delete middleware in connect or express that you already defined on the same instance? I have noticed that under koa you can do this, but we are not going to use koa yet because it is so new, so I am trying to do the same thing in express. I also noticed that it is possible with connect, with somewhat more complicated output, but connect does not have all the features I want, even with middleware.
var express = require('express');
var connect = require('connect');
var koa = require('koa');
var server1 = express();
var server2 = connect();
var server3 = koa();
server1.use(function express(req, res, next) {
console.log('Hello from express!');
});
server2.use(function connect(req, res, next) {
console.log('Hello from connect!');
});
server3.use(function* koa(next) {
console.log('Hello from koa!');
});
console.log(server1.middleware);
// logs 'undefined'
console.log(server2.middleware);
// logs 'undefined'
console.log(server2.stack);
logs [ { route: '', handle: [Function: connect] } ]
console.log(server3.middleware);
// logs [ [Function: koa] ]
koa's docs say that it added some sugar to its middleware, but never explicitly mentions any sugar, and in particular does not mention this behavior.
So is this possible in express? If it is not possible with the vanilla version, how hard would it be to implement? I would like to avoid modifying the library itself. Also, what are the drawbacks for doing this, in any of the 3 libraries?
EDIT:
My use case is that I am essentially re-engineering gulp-webserver, with some improvements, as that plugin, and all others like it, are blacklisted. gulp is a task runner, that has the concept of "file objects", and it is possible to access their contents and path, so I basically want to serve each file statically when the user goes to a corresponding URL in the browser. The trouble is watching, as I need to ensure that the user gets the new file, and not the old version. If I just add an app.use each time, the server would see the file as it is originally, and never get to the middleware with the new version.
I don't want to restart the server every time a file changes, though I will if I can find no better way, so it seems I need to either modify the original middleware on the fly (not a good idea), delete it, or add it to the beginning instead of the end. Either way, I first need to know where it "lives".
You might be able to find what your looking for in server1._router.stack, but it's not clear what exactly you're trying to do (what do you mean "access"?).
In any case, it's not a good idea to do any of these, since that relies strictly on implementation, and not on specification / API. As a result any and all assumptions made regarding the inner implementation of a library is eventually bound to break. You will eventually have to either rewrite your code (and "reverse engineer" the library again to do so), or lock yourself to a specific library version which will result in stale code, with potential bugs and vulnerabilities and no new features / improvements.

express view cache acting funny

I'm running into some funny stuff with the view cache in express/Jade. The controller fetches an article from MongoDB via Mongoose and hands it to the res.render function. However, after running for a couple of minutes Express starts serving the same compiled template for all requests to that route. This even happens to shared .jade includes that are used in various templates.
The database is fetching the correct articles and it doesn't matter if I pass some random strings to the template, I always get the same output.
This is the controller function:
exports.show = function(req, res) {
var articleId;
articleId = req.params.id;
Article.findOne({
_id: articleId
}).populate('author').exec(function(err, article) {
if (err) {
console.log(err);
} else {
res.render('articles/show', {
article: article,
articleId: article.id
});
}
});
};
And that's the route:
app.get('/articles/:id', articles.show);
The same things happen whether I'm running in production or development mode.
Has anyone run into this kind of toruble with Express/Jade?
Edit:
Notice that express sets view cache enabled for production:
see express docs
view cache Enables view template compilation caching, enabled in
production by default
Try adding this line in your app config section:
app.disable('view cache');
Also, try adding cache-control headers
res.setHeader('Cache-Control', 'no-cache');
res.render('articles/show', {
...
From w3.org docs:
Cahce-Control
The Cache-Control general-header field is used to specify directives
that MUST be obeyed by all caching mechanisms along the
request/response chain. The directives specify behavior intended to
prevent caches from adversely interfering with the request or
response. These directives typically override the default caching
algorithms. Cache directives are unidirectional in that the presence
of a directive in a request does not imply that the same directive is
to be given in the response.
If you need a more advanced control, consider other fields like max-age, this question is also a good resource, you'll see that different browsers may implement this rfc slightly different.
TL;DR: try
let articleId;
instead of
var articleId;
I'm just another newbro to Node.js, but I've just solved the same issue by substituting "var" keyword for "let". The thing is that "var" creates a variable scoped by a function, while "let" – a scoped to the current block one. It is re-created every time the block is executed, which is important due to asynchronous nature of Node.js.

Resources