I'm working on a quite large nodejs code base which have been refactored and migrated from legacy to new service version several times and I highly suspect that some code is not used any more.
This dead code is still well tested, but I would like to get rid of it.
I had the idea to run 1 API server using Istanbul, put in in the production pool for some time (few minutes/hours/days) and see what code is actually useful (and identify probable dead code).
According to its documentation, Istanbul cover can handle long-lived processes, so this seems not to be an issue.
My concern is about memory overhead and potential slowness due to the instrumentation of the code, and more globally any thoughts, feedback and recommandation about getting code coverage based on real traffic would be very helpful.
Thanks!
Your best bet to do what you want would be to run your app on
SmartOS, OmniOS or some other illumos/OpenSolaris distro and use DTrace.
See:
http://dtrace.org/blogs/about/
https://en.wikipedia.org/wiki/DTrace
https://wiki.smartos.org/display/DOC/DTrace
Related
I'm working on a program written with libpqxx. It has extensive unit tests except for the libpqxx queries, which is becoming problematic as more and more logic gets pushed to SQL.
The only way I've found to add tests that cover this part is to run them against a test database whose data is constantly setup up for tests and then removed after. The downside of that is that it is reasonably heavy, requiring a container or vm with full postgresql instance and harnasses to bring up and tear down tests.
This seems like it must be a solved problem, but I've not found anything I can just copy and use. It also means our developers have to wait longer for test results, since the tests are heavier, though perhaps there's no way around that.
Is there a standard solution to this problem? My friends who write web frameworks test their database code so easily that I'm hesitant to believe the problem is really roll-your-own here.
I see that no active development is going on from vogels page from a long time i.e, about 5 months https://github.com/ryanfitz/vogels
Are there any better options?
Has anyone faced issues with scalibility or I/O time with it?
I've been using vogels on a project recently. While it seems like it isn't as well maintained as it used to be, it's still quite a nice API wrapper - certainly much nicer than using the SDK directly.
I haven't encountered any performance cost with it - it's really just assembling AWS SDK calls, and doesn't do anything too crazy. The code is simple enough that anything I was unsure about I can dive in and check it out, and the docs are pretty good.
However another option that I've found recently is this library, open sourced by Medium. It's promised based and looks well maintained:
https://github.com/Medium/dynamite
I've been using Vogels for around 6 months now and it's done everything I've needed it to do. The raw dynamoDB api is too low level for what I need. I too noticed that the module wasn't being 'maintained' so I created a fork of the project and republished it to npm:
npm
github
I'm actively working on it to bring it up to modern standards and looking for more contributors to help out.
Are there best practices for testing nodejs application performance?
I'm guessing they could fall under many different types of testing (load, speed, stress). I'd like to do all of the above to find weak areas of my API and/or bits of code that are slow, maybe find some areas that can be done in parallel. Are there packages for this? I've found loadtest which looks promising.
Also, I'm using Heroku for deployment, so if there's any heroku-specific stuff out there, that would also be appreciated. I know they scale automatically with the dynos so that shouldn't be an issue. Anyway, any pointers would be appreciated, I've never had to test much in a node app before besides unit tests.
So I have a Meteor application that is working fine on my local machine, but when I deploy to production is has terrible delays resulting in terrible performance.
Currently it talks to two services - a data processing service and mongodb. As well as the client of course.
I'm using Meteor 7.1.2, so I know it's using capped collections and op-log tailing.
I've optimized my code in every way possible but its still slow in the following ways:
Slow to load
Intermittent (bad) delays when sending requests to the data processing server.
Any suggestions would be VERY appreciated , as i've just about had it with Meteor and thinking about switching to bare-bones express and building on that.
Thanks!
Can you please try to use Kadira?
It will show you what's really happening.
Disclaimer: I maintain Kadira.
I've been developing in Node.js for only 2 weeks and started re-creating a web site previously written in PHP. So far so good, and looks like I can do same thing in Node (with Express) that was done in PHP in same or less time.
I have ran into things you just have to get used to such as using modules, modules not sharing common environment, and getting into a habit of using callbacks for file system and database operations etc.
But is there anything that developer might discover a lot later that is pretty important to development in node? Issues that everyone else developing in Node has but they don't surface until later? Pitfalls? Anything that pros know and noobs don't?
I would appreciate any suggestions and advice.
Here are the things you might not realize until later:
Node will pause execution to run the garbage collector eventually/periodically. Your server will pause for a hiccup when this happens. For most people, this issue is not a significant problem, but it can be a barrier for building near-time systems. See Does Node.js scalability suffer because of garbage collection when under high load?
Node is single process and thus by default will only use 1 CPU. There is built-in clustering support to run multiple processes (typically 1 per CPU), and for the most part the Node community believes this to be a solid approach. You might be surprised by this reality, though.
Stack traces are often lost due to the event queue, so your logging and debugging methodology needs to change significantly
Here are some minor stumbling blocks you may run into for a while (I still bump up against these)
Remembering to do callback(null, value) on a successful callback. Passing null as a first parameter is weird and thus I forget to do it. Instead I accidentally do callback(value), which is interpreted as an error by the caller until I debug into it for a while and slap my forehead.
forgetting to use return when you invoke the callback in a guard clause and don't want a function to continue to execute past that point. Sometimes this results in the callback getting invoked twice, which causes all manner of misbehavior.
Here are some NICE things you might not realize initially
It is much easier in node.js, using one of the awesome flow control libraries, to do complex operations like loading 3 network resources in parallel, then making 2 DB calls in serial, then writing to 2 log files in parallel, then sending an HTTP response. This stuff is trivial and beautiful in node and damn near impossible in many synchronous environments.
ALL of node's modules are new and modern, and for the most part, you can find a beautifully-designed module with a great API to do what you need. Python has great libraries by now, too, but compare Node's cheerio or jsdom module to python's BeautifulSoup and see what I mean. Compare python's requests module to node's superagent.
There's a community benefit that comes from working with a modern platform where people are focused on modern web development. The contrast between the node community and the PHP community cannot be overstated.