express view cache acting funny - node.js

I'm running into some funny stuff with the view cache in express/Jade. The controller fetches an article from MongoDB via Mongoose and hands it to the res.render function. However, after running for a couple of minutes Express starts serving the same compiled template for all requests to that route. This even happens to shared .jade includes that are used in various templates.
The database is fetching the correct articles and it doesn't matter if I pass some random strings to the template, I always get the same output.
This is the controller function:
exports.show = function(req, res) {
var articleId;
articleId = req.params.id;
Article.findOne({
_id: articleId
}).populate('author').exec(function(err, article) {
if (err) {
console.log(err);
} else {
res.render('articles/show', {
article: article,
articleId: article.id
});
}
});
};
And that's the route:
app.get('/articles/:id', articles.show);
The same things happen whether I'm running in production or development mode.
Has anyone run into this kind of toruble with Express/Jade?

Edit:
Notice that express sets view cache enabled for production:
see express docs
view cache Enables view template compilation caching, enabled in
production by default
Try adding this line in your app config section:
app.disable('view cache');
Also, try adding cache-control headers
res.setHeader('Cache-Control', 'no-cache');
res.render('articles/show', {
...
From w3.org docs:
Cahce-Control
The Cache-Control general-header field is used to specify directives
that MUST be obeyed by all caching mechanisms along the
request/response chain. The directives specify behavior intended to
prevent caches from adversely interfering with the request or
response. These directives typically override the default caching
algorithms. Cache directives are unidirectional in that the presence
of a directive in a request does not imply that the same directive is
to be given in the response.
If you need a more advanced control, consider other fields like max-age, this question is also a good resource, you'll see that different browsers may implement this rfc slightly different.

TL;DR: try
let articleId;
instead of
var articleId;
I'm just another newbro to Node.js, but I've just solved the same issue by substituting "var" keyword for "let". The thing is that "var" creates a variable scoped by a function, while "let" – a scoped to the current block one. It is re-created every time the block is executed, which is important due to asynchronous nature of Node.js.

Related

What does this immutable property in options object of express.static used for?

I am a newbie to Node and Express, trying to explore things. I was going through the Express' documentation at this link: https://expressjs.com/en/4x/api.html#express.static
Here, The description of the immutable property passed in options says:
Enable or disable the immutable directive in the Cache-Control response header. If enabled, the maxAge option should also be specified to enable caching. The immutable directive will prevent supported clients from making conditional requests during the life of the maxAge option to check if the file has changed.
I am unable to understand this. I've understood how maxAge is used but not able to get this immutable.
When and How is this property used?
Thanks in advance for any help you are able to provide
It's used for caching resources. Immutable here implies that it won't change(mutate). If you're delivering static assets through your express routes, the browser can use it's cache instead of verifying the asset's validity when the page refreshes. The maxAge directive here sets the duration for which the asset is immutable.
This simply means less server requests when you navigate or refresh pages.
You can use it in your routes by setting the response header before sending the response or you could create a middleware to plug-in between your routes.
router.get('/assets/*', function (req, res, next) {
res.setHeader('Cache-Control', 'max-age=36000,immutable');
res.send({});
}

At what point are request and response objects populated in express app

I’m always coding backend api’s and I don’t really get how express does its bidding with my code. I know what the request and response objects offer, I just don’t understand how they come to be.
This simplified code for instance:
exports.getBlurts = function() {
return function(req, res) {
// build query…
qry.exec(function(err, results) {
res.json(results);
}
});
}
}
Then I’d call in one of my routes:
app.get('/getblurts/, middleware.requireUser, routes.api.blurtapi.getBlurts());
I get that the function is called upon the route request. It’s very abstract to me though and I don’t understand the when, where, or how as it pertains to the req\res params being injected.
For instance. I use a CMS that modifies the request object by adding a user property, which is then available globally on all requests made whether ajax or otherwise, making it easy at all times to determine if a user is logged in.
Are the req and res objects just pre-cooked by express but allow freedom for them to be modified to your needs? When are they actually 'built'
At its heart express is actually using node's default http-module and passing the express-application as a callback to the http.createServer-function. The request and response objects are populated at that point, i.e. from node itself for every incoming connection. See the nodeJS documentation for more details regarding node's http-module and what req/res are.
You might want to check out express' source code which shows how the express application is passed as a callback to http.createServer.
https://github.com/expressjs/express/blob/master/lib/request.js and https://github.com/expressjs/express/blob/master/lib/response.js show how node's request/response are extended by express specific functions.

Sails with AWS XRAY

How in the world is one supposed to install AWS XRAY with Sails?
I'm attempting to translate the installation instructions to Sails' preferred ways of using Express middleware, but I'm falling flat on my face.
Most people will instantly start with "use config/http.js" to configure middleware. Well, that doesn't work in my case, because my API is consumed exclusively with Sails.io (sockets), so the http middleware config is never used.
So now, the logical step is to use policies. Well, if you've read the XRAY instructions, you know that they are trying to capture ALL requests to the app, which requires "start" and "stop" function calls, before and after routes have been configured. So, policies don't work.
So, my next step was to attempt it in the app.js, and the config/bootstrap.js files, to no avail, probably because I can't easily get the Express instance Sails is using. So, is it even possible with Sails' current config options? Anyone have any clue how to accomplish this?
To anyone that should stumble upon this, attempting to integrate AWS X-Ray into Sails.js:
I finally got it working, by building a project hook for it. If someone is ambitious enough, they are more then welcome to make it an installable hook.
IMPORTANT NOTES
The hook is designed to only run when the environment variable AWS_XRAY === 'yes'. This is a safety trap, to prevent local and CI machines from running XRAY.
The hook taps into the "before" part of the route setup. What this means is: "before routes are instantiated, use this middleware".
This code is setup to ignore the route "/_ping" (for X-Ray, it'll let the request complete as normal), which is used
for ELB health checks. These do not need to be logged on X-Ray, they
are just a waste of money. I HIGHLY recommend you read through this
code, and adjust as needed. Especially the req.headers.host and
req.connection "fixes". This was the only way I could get X-Ray to
work, without changing the repo's code (still can't find the Github
repo for it).
The req.connection.encrypted injection is just to have X-Ray report the URL as https. It's not important, unless you want your
traces to reflect the correct URL.
Because we use CloudFlare, there are additional catches to collect the end-user's IP address for requests. This should have no affect if you don't use CF, and should not require any modification. But, I have to ask, WHY aren't use using CF?
This has only gotten me so far, and I can only see basic data about
requests in the X-Ray console. I can't yet see database queries, or
other services that are in use.
RESULTS MAY VARY
Don't forget!
npm i aws-xray-sdk --save.
To install and run the X-Ray Daemon
This is the code I put together api/hooks/setup-aws-xray.js:
var AWSXRay = require('aws-xray-sdk');
module.exports = function setupAwsXray(sails){
var setupXray = false;
function injectXrayIfRequested(req, res, next){
if (
setupXray
&& !req.segment
&& req.path !== '/_ping'
) {
req.headers.host = (sails.config.environment === 'production')
? 'myapp.com'
: 'dev.myapp.com';
req.connection = {
remoteAddress: req.headers['http_cf_connecting_ip']
|| req.headers['HTTP_CF_CONNECTING_IP']
|| req.headers['X-Real-IP']
|| req.ip,
encrypted: true
};
AWSXRay.express.openSegment()(req, res, next); // not a mistake
} else {
next();
}
}
// This just allows us to get a handle on req.segment.
// This is important if you want to add annotations / metadata.
// Despite AWS's documentation, you DO NOT need to close segments
// when using manual mode and express.openSegment, it will
// do this for you automatically.
AWSXRay.enableManualMode();
return {
configure: function(){
if (process.env.AWS_XRAY && process.env.AWS_XRAY === 'yes') {
setupXray = true;
AWSXRay.setDefaultName('myapp_' + sails.config.environment);
}
},
routes: {
before: {
'/*': injectXrayIfRequested
}
}
};
};

Using socket.io with sails js

While there used to be very good documentation for using sockets, thanks to Irl Nathon's Sails Cast series. Things have changed in v0.11, with the sails team wrapping and burying the socket.io routines.
The sails site e.g. SailsSocket is maddeningly concise, saying what to do, but not how or where to do it, or if I need to npm or bower something. This has been particularly frustrating trying to use the sails.config.sockets talked about on the sails site. Which I cannot even find in my v0.11 directories.
First, I would like to know how and where to create my own response to a io.socket.get or .post or whatever. Right now when I do a get with something like:
`io.socket.request({
method: 'get',
url: '/sites/2',
params: {},
headers: {}
},function serverResponded(body, JWR){console.log("Body: ", JSON.stringify(body,null, 4)); console.log(' JWR: ', JWR.body)});'
I get back:
undefined
VM1149:7 "Not implemented in core yet"
VM1149:7 JWR: Not implemented in core yet
I can see the sites being called in the sails console, but nothing comes across.
I believe it is because I have defined my own routes and have my own find: function in my site controller and I manually need to push something into the server side socket. But I am confused as to how I am to call a whole page with HTTP and just the tables with socket.io in the same controller routine.
Where do I write my own low level socket.io routines that can be called from a web page?
Do I still do it in the app.js file?
Sails Cast showed it being done there, but again things have changed.
Sails "virtual requests" (what they call these socket.io-based HTTP-ish request) are generally used to retrieve or post JSON data to the server. Additionally, if a client-side script makes a virtual request, the server may add or remove the requesting socket to/from rooms.
Note that using a "virtual method" will ultimately run the same controller action, but will set req.isSocket = true.
This example is a view that renders a view for HTML-wanting requests but returns JSON data for socket-based requests:
...
// 'get /sites/:id': 'SomeController.showSite' (should be put in your `routes.js`)
showSite: function(req, res) {
// load something from the database
Site.findOne(req.param('id')).exec(function(err, site) {
// handler errors (same for HTTP or sockets)
if (err) return res.serverError();
if (!site) return res.notFound();
if (req.isSocket) return res.json(site); // render JSON response for our `site` object
else return res.view('sites/show', {site: site}); // render an HTML view
});
}
As for low-level socket.io, sails provides the global variable io (from sails.io.js), which is an instance of SailsSocket. It allows you to make HTTP-ish "virtual requests". More info here (although it seems you have already read all there is to read about SailsSocket :). You can access the underlying socket.io client with io.socket._raw.
// do this in the browser.
// sails.io.js should be included in layout.ejs by default.
io.socket.get('/site/2', console.log); // "virtual request"
// neat little trick ^^^^^^^^^^^ for testing :)
var rawIO = io.socket._raw;
rawIO.emit('some:event', "using native socket.io");
Hope this helps!

Access Previously-Defined Middleware

Is there a way to access or delete middleware in connect or express that you already defined on the same instance? I have noticed that under koa you can do this, but we are not going to use koa yet because it is so new, so I am trying to do the same thing in express. I also noticed that it is possible with connect, with somewhat more complicated output, but connect does not have all the features I want, even with middleware.
var express = require('express');
var connect = require('connect');
var koa = require('koa');
var server1 = express();
var server2 = connect();
var server3 = koa();
server1.use(function express(req, res, next) {
console.log('Hello from express!');
});
server2.use(function connect(req, res, next) {
console.log('Hello from connect!');
});
server3.use(function* koa(next) {
console.log('Hello from koa!');
});
console.log(server1.middleware);
// logs 'undefined'
console.log(server2.middleware);
// logs 'undefined'
console.log(server2.stack);
logs [ { route: '', handle: [Function: connect] } ]
console.log(server3.middleware);
// logs [ [Function: koa] ]
koa's docs say that it added some sugar to its middleware, but never explicitly mentions any sugar, and in particular does not mention this behavior.
So is this possible in express? If it is not possible with the vanilla version, how hard would it be to implement? I would like to avoid modifying the library itself. Also, what are the drawbacks for doing this, in any of the 3 libraries?
EDIT:
My use case is that I am essentially re-engineering gulp-webserver, with some improvements, as that plugin, and all others like it, are blacklisted. gulp is a task runner, that has the concept of "file objects", and it is possible to access their contents and path, so I basically want to serve each file statically when the user goes to a corresponding URL in the browser. The trouble is watching, as I need to ensure that the user gets the new file, and not the old version. If I just add an app.use each time, the server would see the file as it is originally, and never get to the middleware with the new version.
I don't want to restart the server every time a file changes, though I will if I can find no better way, so it seems I need to either modify the original middleware on the fly (not a good idea), delete it, or add it to the beginning instead of the end. Either way, I first need to know where it "lives".
You might be able to find what your looking for in server1._router.stack, but it's not clear what exactly you're trying to do (what do you mean "access"?).
In any case, it's not a good idea to do any of these, since that relies strictly on implementation, and not on specification / API. As a result any and all assumptions made regarding the inner implementation of a library is eventually bound to break. You will eventually have to either rewrite your code (and "reverse engineer" the library again to do so), or lock yourself to a specific library version which will result in stale code, with potential bugs and vulnerabilities and no new features / improvements.

Resources