I am using ejs-local to render views for my node.js application. I saw lots of too many open files" exception when rendering views:
Error: EMFILE, too many open files '/home/user/app/views/index.html'
at Object.fs.openSync (fs.js:432:18)
at fs.readFileSync (fs.js:289:15)
at Object.exports.renderFile (/home/wangchen/bweb_auto/node_modules/ejs-locals/node_modules/ejs/lib/ejs.js:313:9)
at View.module.exports [as engine] (/home/user/app/node_modules/ejs-locals/index.js:85:7)
at View.render (/home/user/app/node_modules/express/lib/view.js:76:8)
at Function.app.render (/home/user/app/node_modules/express/lib/application.js:506:10)
at ServerResponse.res.render (/home/user/app/node_modules/express/lib/response.js:759:7)
at /home/user/app/routes/index.js:226:17
at Query._callback (/home/user/app/node_modules/orm/lib/ChainFind.js:190:13)
at Query.Sequence.end (//home/user/appode_modules/mysql/lib/protocol/sequences/Sequence.js:75:24)
Which tear down the entire website. Although seemingly adding ulimit might solve this problem: it current is set to 1024. I am more concerned about possible bug in ejs code/node, that didn't close the file when necessary.
Have you come across similar thing?
Related
Backend Setup:
Node.js/Express server running on port 5000
AWS S3 (for images)
AWS RDS Mysql (for data)
Hi guys,
I'm a frontend guy, so I don't know much about backend, I just started learning.
I have a Node.js/Express server on port 5000 that I use to connect to an S3 bucket and an RDS Mysql database, that is enough for the React frontend I'm developing at the moment.
I use nodemon to deal with disconnects and reconnect automatically. It used to work just fine 24/7, but I must have changed something during the last 3 months and now it crashes several times a day and I have to restart the server manually. It's not a huge deal but I would like to learn how to deal with it.
Here's the error I keep getting:
[nodemon] restarting due to changes...
[nodemon] starting `node server.js`
Listening on port 5000..
Connection Established Successfully
Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:205:27)
--------------------
at Protocol._enqueue (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/mysql/lib/protocol/Protocol.js:144:48)
at Connection.query (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/mysql/lib/Connection.js:198:25)
at /Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/routes/meProfileImageData.js:10:12
at Layer.handle [as handle_request] (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/layer.js:95:5)
at next (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/route.js:137:13)
at Route.dispatch (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/layer.js:95:5)
at /Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/index.js:281:22
at Function.process_params (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/index.js:335:12)
at next (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/index.js:275:10) {
errno: 'ECONNRESET',
code: 'ECONNRESET',
syscall: 'read',
fatal: true
}
Error: Cannot enqueue Query after fatal error.
at Protocol._validateEnqueue (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/mysql/lib/protocol/Protocol.js:212:16)
at Protocol._enqueue (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/mysql/lib/protocol/Protocol.js:138:13)
at Connection.query (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/mysql/lib/Connection.js:198:25)
at /Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/routes/meProfileImageData.js:10:12
at Layer.handle [as handle_request] (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/layer.js:95:5)
at next (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/route.js:137:13)
at Route.dispatch (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/layer.js:95:5)
at /Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/index.js:281:22
at Function.process_params (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/index.js:335:12) {
code: 'PROTOCOL_ENQUEUE_AFTER_FATAL_ERROR',
fatal: false
}
Obviously the second part happens when I try to connect without knowing the server has already crashed.
There are plenty of posts on stackoverflow about similar ECONNRESET problems, so it's very possible that my exact problem has already been replied somewhere else, but to be honest I'm just not good enough at this yet to assimilate the answer by myself, I could use some pointers as to what files the problem seems to be in or whatever you could tell me.
I have seen in several posts that the best way to deal with this kind of problem is to set error listeners, and I'm happy to do just that, but to be honest I don't know where to start because the problem happens when I'm not using the server usually, like at night or when I'm not even in front of the computer. I have no idea how to reproduce the problem on my own, it just happens by itself whenever it happens, so that's why it's so hard for me to know where to start. I looked at the logs on my S3 and RDS Mysql and didn't see anything that seemed directly connected to this.
Could you offer me some pointers based on what you see here? I have some possible ideas:
could it be the router at index.js causing this?
when I look at my RDS Mysql database panel I see at the moment there are 3 active connections, even though I stopped working like 2 hours ago. Could it be that those connections timing out are causing this? (read this in another post)
could it be meProfileImageData.js that should get error listening first?
That's all I have for now, any ideas or pointers are highly welcome, and I'll be glad to run any tests or install any necessary additional software to troubleshoot this if you feel like putting in the time to help me.
Thanks
EDIT 1: had a new error today that seems to point at connections being closed automatically and Node.js not dealing with it before there is an attempt to connect:
Error: read ETIMEDOUT
at TCP.onStreamRead (internal/stream_base_commons.js:205:27)
--------------------
at Protocol._enqueue (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/mysql/lib/protocol/Protocol.js:144:48)
at Connection.query (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/mysql/lib/Connection.js:198:25)
at /Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/routes/meProfileImageData.js:10:12
at Layer.handle [as handle_request] (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/layer.js:95:5)
at next (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/route.js:137:13)
at Route.dispatch (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/route.js:112:3)
at Layer.handle [as handle_request] (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/layer.js:95:5)
at /Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/index.js:281:22
at Function.process_params (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/index.js:335:12)
at next (/Volumes/Content/Desktop/Dropbox/HugoPrivate/React/ladswork-server/node_modules/express/lib/router/index.js:275:10) {
errno: 'ETIMEDOUT',
code: 'ETIMEDOUT',
syscall: 'read',
fatal: true
Greenie here. Trying to start with a new Express project and I already hit a snag. I used the generator to start my app (express --view=handlebars sms_alarm_nodejs) and I got this error:
Error: Failed to lookup view "error" in views directory "c:\Users\joe_t\workspace\sms_alarm_nodejs\views"
application.js:630
at Function.render (c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\application.js:580:17)
at ServerResponse.render (c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\response.js:971:7)
at c:\Users\joe_t\workspace\sms_alarm_nodejs\app.js:43:7
at Layer.handle_error (c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\router\layer.js:71:5)
at trim_prefix (c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\router\index.js:315:13)
at c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\router\index.js:284:7
at Function.process_params (c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\router\index.js:335:12)
at next (c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\router\index.js:275:10)
at Layer.handle_error (c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\router\layer.js:67:12)
at trim_prefix (c:\Users\joe_t\workspace\sms_alarm_nodejs\node_modules\express\lib\router\index.js:315:13)
Am I missing a view reference in app.js? I assumed the app generator would handle this for me to start. And can I use handlebars? The sample I saw used pug.
Fixed it. I had to use --view=hbs as the generator parameter.
I am developing a node.js application on Ubuntu and am trying to programmatically create a directory for my application in the user's home directory.
When I execute the following Javascript in Node:
const fs = require("fs");
fs.mkdirSync("~/mynewdir");
I get the following error:
Error: ENOENT: no such file or directory, mkdir '~/mynewdir'
at Error (native)
at Object.fs.mkdirSync (fs.js:923:18)
at repl:1:4
at sigintHandlersWrap (vm.js:22:35)
at sigintHandlersWrap (vm.js:73:12)
at ContextifyScript.Script.runInThisContext (vm.js:21:12)
at REPLServer.defaultEval (repl.js:346:29)
at bound (domain.js:280:14)
at REPLServer.runBound [as eval] (domain.js:293:12)
at REPLServer.<anonymous> (repl.js:545:10)
Executing:
fs.mkdirSync("/home/dah/mynewdir");
works just fine however, but I want to use the home directory of whomever is executing the script.
Does anyone have any suggestions?
Edit - this question is not a duplicate. In this case, the issue is not finding the home directory (I already have this), but why the fs module won't use it.
You can do it like that:
const homedir = require('os').homedir();
// `homedir()` returns absolute path so we use `join` here
require("fs").mkdir(require('path').join(homedir, 'mynewdir'));
I am using MEAN.JS for developing a web app, but for some reason after I made few changes, well mostly copied and paste new modules into the app I see the error as below:
error: uncaughtException: require(...).invokeRolesPolicies is not a function
I dont know why is this happening, anyone has any clue how to fix this?
This is the whole error message:
TypeError: require(...).invokeRolesPolicies is not a function
at C:\oferdo\oferdo\config\lib\express.js:180:39
at Array.forEach (native)
at Object.module.exports.initModulesServerPolicies (C:\oferdo\oferdo\config\lib\express.js:179:32)
at Object.module.exports.init (C:\oferdo\oferdo\config\lib\express.js:252:8)
at C:\oferdo\oferdo\config\lib\app.js:29:23
at C:\oferdo\oferdo\config\lib\mongoose.js:36:15
at C:\oferdo\oferdo\node_modules\mongoose\lib\connection.js:284:19
at open (C:\oferdo\oferdo\node_modules\mongoose\lib\connection.js:511:17)
at C:\oferdo\oferdo\node_modules\mongoose\lib\connection.js:518:7
at C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\lib\db.js:1504:5
at handleCallback (C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\lib\utils.js:96:12)
at _callback (C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\lib\db.js:1420:5)
at C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\lib\db.js:1463:7
at C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\node_modules\mongodb-core\lib\topologies\server.js:1416:5
at finish (C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\node_modules\mongodb-core\lib\auth\scram.js:157:16)
at handleEnd (C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\node_modules\mongodb-core\lib\auth\scram.js:170:7)
at C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\node_modules\mongodb-core\lib\auth\scram.js:264:17
at commandCallback (C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\node_modules\mongodb-core\lib\topologies\server.js:1194:9)
at Callbacks.emit (C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\node_modules\mongodb-core\lib\topologies\server.js:119:3)
at null.messageHandler (C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\node_modules\mongodb-core\lib\topologies\server.js:358:23)
at Socket. (C:\oferdo\oferdo\node_modules\mongoose\node_modules\mongodb\node_modules\mongodb-core\lib\connection\connection.js:292:22)
at emitOne (events.js:77:13)
at Socket.emit (events.js:169:7)
at readableAddChunk (_stream_readable.js:153:18)
at Socket.Readable.push (_stream_readable.js:111:10)
at TCP.onread (net.js:531:20)
Actually, this is nothing to do with incorrectly named files.
You need to make sure that you have the invokeRolesPolicies function defined within your policies file. The minimum required is:
'use strict';
var acl = require('acl');
acl = new acl(new acl.memoryBackend());
exports.invokeRolesPolicies = function () {};
The reason why is that the express.js lib defined in mean.js assumes this is set, and tries to call it, hence the error.
You'll get this error if you have an incorrectly named file under your module's server > policies directory, so check there for any inconsistently named files.
EDIT
I see now that it's not the naming of the file but if you have a file in the policies directory that doesn't contain this method, it will produce this error. In my case I had temporarily commented out the code in a file in this directory in preparation for removal.
Sometimes, my (large) application gives the following error:
Error: write after end
at ServerResponse.OutgoingMessage.write(_http_outgoing.js:413:15)
at ServerResponse.res.write(/nodeapp/node_modules/express/node_modules/connect/lib/middleware/compress.js:110:17)
at ServerResponse.res.end(/nodeapp/node_modules/express/node_modules/connect/lib/middleware/compress.js:116:14)
at ServerResponse.res.send(/nodeapp/node_modules/express/lib/response.js:153:8)
at fn(/nodeapp/node_modules/express/lib/response.js:797:10)
at/nodeapp/node_modules/vash/build/vash.js:2082:4
at Object.vash.loadFile (/nodeapp/node_modules/vash/build/vash.js:2070:10)
at View.vash.renderFile [as engine] (/nodeapp/node_modules/vash/build/vash.js:2078:8)
at View.render (/nodeapp/node_modules/express/lib/view.js:76:8)
at Function.app.render (/nodeapp/node_modules/express/lib/application.js:504:10)
Is there a way to find out in which line this error happend? Normally the file is mentioned in the stack trace, but not in this case. Is it even from my code or can it also be caused by vash or another module?
I am using vash v0.7.12-1.
I solved it by increasing the stacktrace limit:
Error.stackTraceLimit = 20;