Debug Node.js & Express App - Intermittently using 100% CPU - node.js

I'm developing an app using NGinx + Node.js + Express + Firebase that simply takes input from a mobile app and stores it to Firebase, optionally uploading files to S3.
In its simplest terms, the "create" function does this
Validates input
Formats the input Checks if there is a file uploaded
(via the multer plugin) and stores it
If there was a file, upload
to Amazon S3 and delete the source file (it's important to note I was
encountering this issue before the inclusion of S3).
Create the item
by pushing into the items reference on Firebase
Create the item for
the user by pushing into the user_items reference on Firebase.
There are a few other functions that I have implemented as an API.
My trouble is coming from an intermittent spike in CPU usage, which is causing the nginx server to report a gateway timeout from the Node.js application.
Sometimes the server will fall over when performing authentication against a MongoDB instance, other times it will fall over when I'm recieving the input from the Mobile app. There doesn't seem to be any consistency between when it falls over. Sometimes it works fine for 15+ various requests (upload/login/list, etc), but sometimes it will fall over after just one request.
I have added error checking in the form of:
process.on('uncaughtException', function(err) {
console.error(err.stack);
});
Which will throw errors if I mistype a variable for example, but when the server crashes there are no exceptions thrown. Similarly checking my logs shows me nothing. I've tried profiling the application but the output doesn't make any sense at all to me. It doesn't point to a function or plugin in particular.
I appreciate this is a long winded problem but I'd really appreciate it if you could point me in a direction for debugging this issue, it's causing me such a headache!

This may be a bug in the Firebase library. What version are you using?
I've been having a very similar issue that has had me frustrated for days. Node.js + Express + Firebase on Heroku. Process will run for a seemingly random time then I start getting timeout errors from Heroku without the process ever actually crashing or showing an error. Higher load doesn't seem to make it happen sooner.
I just updated from Firebase 1.0.14 to latest 1.0.19 and I think it may have fixed the problem for me. Process has been up for 2 hours now where it would only last for 5-30 min previously. More testing to do, but thought I'd share my in-progress results in case they were helpful.

It seems the answer was to do with the fact that my Express app was reusing one Firebase connection for every request, and for some reason this was causing the server to lock up.
My solution was to create some basic middleware that provides a new reference to the Firebase on each API request, see below:
var Middleware = {
/*
* Initialise Firebase Refs per connection
*/
initFireBase: function(req, res, next) {
console.log('Intialising Firebase for user');
// We need a authToken
var authToken = req.param('authToken');
// Validate the auth token
if(!authToken || authToken.length === 0) {
return res.send(500, {code: 'INVALID_TOKEN', message: 'You must supply an authToken to this method.'});
}
else {
// Attempt to parse the auth token
try {
var decodedToken = JWTSimple.decode(authToken, serverToken);
}
catch(e) {
return res.send(500, {code: 'INVALID_TOKEN', message: 'Supplied token was not recognised.'});
}
// Bail out if the token is invalid
if(!decodedToken) {
return res.send(500, {code: 'INVALID_TOKEN', message: 'Supplied token was not recognised.'});
}
// Otherwise send the decoded token with the request
else {
req.auth = decodedToken.d;
}
}
// Create a root reference
var rootRef = new Firebase('my firebase url');
// Apply the references to each request
req.refs = {
root: rootRef,
user: rootRef.child('users'),
inbox: rootRef.child('inbox')
};
// Carry on to the calling function
next();
}
};
I then simply call this middleware on my routes:
/*
* Create a post
*/
router.all('/createPost', Middleware.initFireBase, function(req, res) {
var refs = req.refs;
refs.inbox.push({}) // etc
....
This middleware will soon be extended to provide Firebase.auth() on the connection to ensure that any API call made with a valid authToken would be signed to the user on Firebase's side. However for development this is acceptable.
Hopefully this helps someone.

Related

google cloud functions release resources without respond request

I want to release the resources associated with a node js request without sending any kind of response to the client.
This might sound weird but my goal is very simple, the last few days my servers have been targeted by hackers... i'm trying to improve the defenses and if i identify a malicious request i could just DROP IT without sending any response i would make the attacker wait for connection timeout and it would give a little more advantage.
i tried:
exports.test = (req, res) => {
res.end();
};
but this case the server sends an empty response which isn't my goal since i want make client wait forever
also tried:
exports.test = (req, res) => {
res.socket.destroy();
};
which on google cloud functions throws an exception
does anyone know if on GCF if i simple return the function it will be released or the connection will be hang on?
exports.test = (req, res) => {
return; //will google release all resources or connection and socket will be kept until timeout?
};
Cloud Functions does not enable what you're trying to do. The only way it will keep the connection open is if your function times out with no response. You can't instruct it to keep the connection open while also terminating the function. Or, to put it another way, you're going to have to pay the usual Cloud Functions rate for execution-seconds in order to keep that connection open.

How can I log the express endpoint hit if the server crashes during an asynchronous callback?

I have an express service where a lot of my endpoints look like the following.
var express = require('express');
var router = express.Router();
router.get('/updateThisValue', (req, res) => {
let value = req['query'].value
try {
database.update(value, function() { // callback function on update
console.log(undefinedValue.undefined) // crash the server
})
} catch (err) {
// Log the error and other stuff
}
})
Looking at this code, if a database update is called, something will crash in the callback function and since there is no try-catch, the whole server should crash as well.
Right now, I am okay with the server crashing. The problem is that I would like to be able to log some debugging information in the callback before it crashes, namely the express endpoint hit and the parameters passed to it.
One solution is that I could just log every single endpoint hit, but the only problem with that is that the server starts to have very large logs. So we don't log endpoints hit by default. The frustration however is that recently an endpoint hit the server that did cause it crash and I'm still unsure what the actual endpoint hit and parameters passed to it where.
Basically, I want the server to still crash but at least I would be able to trace the endpoint hit and its parameters
Could this help: https://expressjs.com/en/guide/error-handling.html
Normally you would see the exception in the console or in the logs of your process manager (like pm2).

Send request progress to client side via nodejs and express

I am using this (contentful-export) library in my express app like so
const app = require('express');
...
app.get('/export', (req, rex, next) => {
const contentfulExport = require('contentful-export');
const options = {
...
}
contentfulExport(options).then((result) => {
res.send(result);
});
})
now this does work, but the method takes a bit of time and sends status / progress messages to the node console, but I would like to keep the user updated also.. is there a way I can send the node console progress messages to the client??
This is my first time using node / express any help would be appreciated, I'm not sure if this already has an answer since im not entirely sure what to call it?
Looking of the documentation for contentful-export I don't think this is possible. The way this usually works in Node is that you have an object (contentfulExport in this case), you call a method on this object and the same object is also an EventEmitter. This way you'd get a hook to react to fired events.
// pseudo code
someLibrary.on('someEvent', (event) => { /* do something */ })
someLibrary.doLongRunningTask()
.then(/* ... */)
This is not documented for contentful-export so I assume that there is no way to hook into the log messages that are sent to the console.
Your question has another tricky angle though. In the code you shared you include a single endpoint (/export). If you would like to display updates or show some progress you'd probably need a second endpoint giving information about the progress of your long running task (which you can not access with contentful-export though).
The way this is usually handled is that you kick of a long running task via a certain HTTP endpoint and then use another endpoint that serves infos via polling or or a web socket connection.
Sorry that I can't give a proper solution but due to the limitation of contentful-export I don't think there is a clean/easy way to show progress of the exported data.
Hope that helps. :)

Generated Access Token invalid for Demo Video Application

I am using the sample Video application pulled from GitHub. I am using a node.js server to supply the sample application with the access token. When I use the Twilio Console to generate a video access token and put it in my Node.js server as a literal and return it I am able to run the example application and connect to a room. If I use the sample token generation code in my Node.js server I get 'Invalid Access Token' back in an exception in the onDisconnected method in the Room.Listener.
The following code is what is running in the server to create the access token, I also found a different sample which I tried as well. I have gone back and verified that my data values for the account SID and the API keys are correct. I have a similar method running returning the VoiceGrant access token and that is working, but something about this VideoGrant one is off, I just do not see it.
// ***********************************************************************************
// ***********************************************************************************
// Video Access Token
// ***********************************************************************************
// ***********************************************************************************
var videoCallAccessToken = function(request, response) {
console.log('/twilio/video/accessToken');
var accessToken = makeVideoAccessToken();
response.writeHead(200, {'Content-Type': 'text/plain'});
response.end(accessToken);
console.log(accessToken);
};
app.get('/twilio/video/accessToken', videoCallAccessToken);
var makeVideoAccessToken = function() {
const AccessToken = twilio.AccessToken;
const VideoGrant = AccessToken.VideoGrant;
const grant = new VideoGrant({configurationProfileSid: accountData.videoConfigurationProfileSid});
const accessToken = new AccessToken(accountData.sid, accountData.videoApiSid, accountData.videoApiSecret);
accessToken.identity = 'ABC123';
accessToken.addGrant(grant);
return accessToken.toJwt();
};
FYI...I plan to alter the identity generation, but have not got there yet.
Thanks,
Adding this from my comment as an answer to close this question out, the issue was that the example code was flawed...
Ok, thought I had waited long enough prior to actually sending this, but apparently not. The issue is the example does not work in that the value passed into the VideoGrant constructor needed to have the attribute name quoted, so {configurationProfileSid: accountData.videoConfigurationProfileSid}); needed to be {'configurationProfileSid': accountData.videoConfigurationProfileSid}); Glad I finally found that, wasted a ton of time on it, but at least it is working properly now.

Create google spreadsheet with node js

I want to create a google docs sheet within my alexa skill, that is written in Node.js. I have the enabled the google API, I set the required scope in amazon dev portal, I actually can log into the google account (so the first few lines of the posted code seem to work), and I do not get any error messages. But the sheet is never created.
Now the main question would be whether anyone can see the problem in my code.
But I would also have an additional question I would be very interested in: since I use account linking, I can not try that code in the Alexa test simulator, but have to upload it to Alexa before running it, where I can not get any debug messages. How does one best debug in that way?
if (this.event!== undefined)
{
if (this.event.session.user.accessToken === undefined)
{
this.emit(':tellWithLinkAccountCard','to start using this skill, please use the companion app to authenticate on Google');
return;
}
}
else
{
this.emit(':tellWithLinkAccountCard','to start using this skill, please use the companion app to authenticate on Google');
return;
}
var oauth2Client = new google.auth.OAuth2('***.apps.googleusercontent.com', '***', '***');
oauth2Client.setCredentials({
access_token: this.event.session.user.accessToken,
refresh_token: this.event.session.user.refreshToken
});
var services = google.sheets('v4');
services.spreadsheets.create({
resource : {properties:{title:"MySheet"}},
auth : oauth2Client
}, function(err,response) {
if( err ) {
console.log('Error : unable to create file, ' + err);
return;
} else {
console.dir(response);
}
});
Edit: I tried just the lower part manually, and could create a spreadsheet. So the problem seems indeed to be retrieving the access token with "this.event.session.user.accessToken" .
I find it is much easier to debug issues like this using unit tests. This allows rerunning code locally. I use NPM and Mocha and it makes it easier to debug both custom and smart home skills. There is quite a bit of information available online about how to use NPM and Mocha to test Nodejs code, so I won't repeat that here. For example, refer to the Big Nerd Ranch article. It makes it a bit more complex to setup your project initially, but you'll be glad you did every time you hit a bug.
In this example, I would divide the code in half:
The first half would handle the request coming from Alexa and extract the token.
The second half would use the token to create the Google doc. I would also pass the name of the doc to create.
I would test the 2nd part first, passing in a valid token (for testing only) and a test doc name. When that is working, at least you'd know that the doc creation code was working, and any issues would have to be with the token or how you're getting it.
Once that was working, I would then create a test for the first part.
I would us a hardcoded JSON object to pass in as the 'event', with event.session.user.accesToken set to a the working test token used in the first test:
'use strict';
var token = '<valid token obtained from google account>';
let testEvent = {
'session': {
'user': {
'accessToken': token
}
}
}

Resources