We use an express backend and we recently ran into a strange bug when adding a new middleware to one of our routes. We have been binding the handlers to the route as follows:
app.all('/route', [AsyncFunction1, AsyncFunction2, AsyncFunction3, AsyncFunction4, AsyncFunction5])
I added another Middleware to the route, so now we have:
app.all('/route', [AsyncFunction1, AsyncFunction2, AsyncFunction3, AsyncFunction4, AsyncFunction5, AsyncFunction6])
It worked fine and as expected until we run the containerized version of it (we run containerized versions for production). Suddenly, anything but GET requests dont' work on that route. Hitting it with an OPTIONS request returns "GET, HEAD". Again, it works fine locally.
We tried completely commenting out the new middleware and just have it return next(), but it doesn't work. We've tried removing one of the other, existing middleware, and the endpoint works again. We tried duplicating one of the other, existing middleware with a new name and adding it in place of the new one we made, and we get the same error. We've tried numerous different things, and it seems that the number of middleware is the limiting factor. Everything I've read makes no mention of there being a limit. Even more strangely, it works locally but not when containerized??
We did come up with two fixes. One was to assign the handlers as follows:
app.all('/route', AsyncFunction1)
app.all('/route', AsyncFunction2)
app.all('/route', AsyncFunction3)
.... so on for all 6
This works as expected but is a very low-level change of the sort we'd like to avoid if possible. We ended up moving the logic of the new middleware into one of the existing middlewares. Everything works as expected.
But there's something weird going on here and we don't know what!
The container image uses the same version of node and npm we are running locally. I'm thinking it might be some sort of resource constraint? I don't really know.
Is there any type of limit on the number of handlers added via array?
Related
I'm trying to build a graphql server using nestjs and using vite + swc as the compiler/builder for performance reasons, webpack would take 50-60+ seconds on each rebuild on a big project, SWC/vite seems to cut that down by a factor of 5 at least.
Here's a repository that reproduces the issue with a basic 'health check' endpoint and graphql query.
The main tools concerning this:
"#nestjs/graphql": "10.0.9",
"#nestjs/apollo": "10.0.9",
"typescript": "4.7.4",
"vite-plugin-node": "1.0.0",
"vite": "2.9.13",
"#swc/core": "1.2.207",
"vite-tsconfig-paths": "3.5.0"
Now, I have played around with these fixed versions trying out various combinations of older versions. But I've narrowed down the flaw to be a problem with vite specifically.
There's this github issue opened over a month ago that's probably directly related, with this being merely a symptom of that issue.
If you build the app and serve it, everything works fine, because the production version calls the bootstrap() function which is not handled by the vite development server.
This is also a nestjs-specific problem due to nestjs doing the code-first approach.
I'm trying to patch this issue somehow by attempting three things:
stop the development server from rebuilding on every request
configure the development server to cleanup after itself on every request
configure nestjs's graphql in a way that only builds the schema 1 time, something as simple as:
let built = false;
if(!built) { buildSchema(); built = true; }
I'm counting on that built variable not changing between requests, but if it does, I might find a way to tie it to the start command via a file outside of vite's scope.
Thank you.
I've tried to get the setUserContext function with raven-node in my nodejs app, but I cannot find how to set the user context. Has anyone made it work?
I was able to make it work in the client-side, with "Raven.setUserContext" but not in the nodejs backend :(
User context isn't implemented in raven-node: https://github.com/getsentry/raven-node/issues/134
I'm a contributor to the project, and it's my number one priority to have this done shortly – should be a matter of days.
Edit – we just published raven-node 0.10.0 which adds setUserContext.
I am new to node and what I would call, real server-side programming (vs PHP). I was setting up a user database with MongoDB, Mongoose and a simple mongoose user plugin that came with a schema and password stuff to use. You can add validation to Mongoose for your fields like so
schema.path('email').validate(function (email) {
if (this.skipValidation) return true
return email.trim().length
}, 'Please provide a valid email')
(this is not my code). I noticed though when I passed an invalid or blank email, .trim() failed and the entire server crashed. This is very worrisome to me because things like this don't happen in your good ol' WAMP stack. If you have a bug, 99.9% of the time it's just the browser that is affected.
Now that I am delving into lower level programming, do I have to be paranoid about every incoming variable to a simple function? Is there a tried-and-true error system I should follow?
Just check before using the variable with trim, if it is !null for example:
if(!email) {
return false;
}
And if you want to run your app forever, rather use PM2.
If you are interested in running forever, read this interesting post http://devo.ps/blog/goodbye-node-forever-hello-pm2/
You may consider using forever to keep your node.js program running. Even it crashes, it restarts automatically and the error is logged as well.
Note: Although you could actually catch all exceptions to prevent the node.js from crashing, it is not recommended.
One of our strategies is to make use of Node.js Domain to handle errors - http://nodejs.org/api/domain.html
You should set up a error logging node modules like Winston, once configured produces useful error/exceptions.
Have a look in this answer for how to catch error within your node implementation, though specific to expressjs but relevant.
Once you catch exceptions, it prevents unexpected crashes.
When launched through the intern-runner command, my tests are still hanging--intern-runner never exits to give me a report and I can tell that the proxy server is still running on port 9000. The browser I specified through my config just remains open (and no, I did not set leaveRemoteOpen to true). I added some debug to lib/reporters/webdriver.js, because I saw that's what logged the "Tests complete" message. I could see that the topic.publish('/client/end') code was invoked, but nothing ever responded to this event. Doesn't lib/ClientSuite subscribe to this topic? From that module:
topic.subscribe('/client/end', function (sessionId) {
console.log("subscribed to '/client/end' for session", sessionId);
if (sessionId === remote.session.sessionId) {
clearHandles();
// get about:blank to always collect code coverage data from the page in case it is
// navigated away later by some other process; this happens during self-testing when
// the new Leadfoot library takes over
remote.setHeartbeatInterval(0).get('about:blank').then(lang.hitch(dfd, 'resolve'));
}
})
But nothing ever happens, and I don't see my console.log() output. Sorry if I am bringing up things that are red herrings, but I just wanted to do some initial investigation first.
All I want is for my test to end and my JUnit and LCOV reports generated! :( What could be going wrong?
And note: no error messages are logged to the command terminal from which I invoked intern-runner config=unittest/myInternConfig. No errors (obvious ones at least) appear in terminal where Selenium server is running.
Update 03/15/15: I added this info in my last comment, but maybe comments get lost in the shuffle on Stackoverflow. In our legacy DOH tests, we used Sinon to fake a server so as to not make real I/O requests to the backend server in unittests. I didn't see a problem with keeping this in the Intern tests, but apparently, there is. When I disabled the test modules that just do
var server = sinon.fakeServer.create();
(well, that, in addition to calling server.respondWith() and server.respond())
intern-runner completed, I got my reports, and etc. Then I searched for "intern with sinon" and stumbled upon https://github.com/theintern/intern/issues/47, where jason0x43 linked to his Sinon-with-Intern code at https://github.com/theintern/intern/blob/sinon/sinon.js. So, I found that very helpful--it seems that in my situation, Sinon's FakeXMLHttpRequest was ALSO faking requests to Intern's proxy server, and that was what was hanging the process.
So, after pretty much using jason0x43's sinon.js code to filter out the "real request," I re-enabled the problematic test modules, re-ran, and everything worked beautifully.
Again, no errors or any sort of warnings reported in terminal or browser console--it would be great if there could be some sort of head's up about this pitfall. Even if just in a Readme file.
(I also edited my original post to add this info.) In our legacy DOH tests, we used Sinon to fake a server so as to not make real I/O requests to the backend server in unittests. I didn't see a problem with keeping this in the Intern tests, but apparently, there is. When I disabled the test modules that just do
var server = sinon.fakeServer.create();
(well, that, in addition to calling server.respondWith() and server.respond())
intern-runner completed, I got my reports, and etc. Then I searched for "intern with sinon" and stumbled upon https://github.com/theintern/intern/issues/47, where jason0x43 linked to his Sinon-with-Intern code at https://github.com/theintern/intern/blob/sinon/sinon.js. So, I found that very helpful--it seems that in my situation, Sinon's FakeXMLHttpRequest was ALSO faking requests to Intern's proxy server, and that was what was hanging the process.
So, after pretty much using jason0x43's sinon.js code to filter out the "real request," I re-enabled the problematic test modules, re-ran, and everything worked beautifully.
Again, no errors or any sort of warnings reported in terminal or browser console--it would be great if there could be some sort of head's up about this pitfall. Even if just in a Readme file.
I'm currently working on a project where one of the core Node.js modules (dns) does not behave the way I need it to. I've found a module that seems like it would work as a replacement: https://github.com/tjfontaine/node-dns. However, the code using the DNS module is several layers down from the application code I've written. I'm using the Request module (https://github.com/mikeal/request) which makes HTTP requests and uses several core modules to do so. The Request module does not seem to be using the DNS module directly, but I'm assuming one of those core modules is calling the DNS module.
Is there a way I can tell Node to use https://github.com/tjfontaine/node-dns whenever require('dns') is called?
Yes and you should not,
require.cache is a dangerous thing, extremely. It can cause memory leaks if you do not know what you are doing and cache mismatch which is potentially worse. Most requests to change core modules can also result in unintentional side effects (such as discoverability failures with DNS).
You can create a user-space require with something like : https://github.com/bmeck/node-module-system ; however, this faces the same dangers but is not directly tied to core.
My suggestion would be to wrap your require('dns').resolve with require('async').memoize, but be aware that DNS discoverability may fall over.
For better or worse, I've implemented module white lists before doing something as demonstrated below. In your case, it ought to be possible to explicitly check for dns module name and delegate everything else to original require(). However, this implementation assumes that you have full control of when and how your own code is being executed.
var _require = constructMyOwnRequire(/*intercept 'dns' and require something else*/);
var sandbox = Object.freeze({
/* directly import all other globals like setTimeout, setInterval, etc.. */
require : Object.freeze(_require)
});
try {
vm.runInContext(YOUR_SCRIPT, Object.freeze(vm.createContext(sandbox)));
} catch (exception) {
/* stuff */
}
Not really.
require is a core var which is local to each module, so you can't stub it, because Node will give the untouched require var to the loaded module.
You could run those things using the vm module. However, you would have to write too much code to do a "simple workaround" (give all needed variables to the request module, stub the needed ones to work properly, etc, etc...).