I want to know the benefit of pre-JIT compilation (ngen.exe). What is the role of the Native Image Generator (NGen) process and why is it required?
Please provide an example.
For code execution on the .NET platform, the Common Intermediate Language (CIL) representation needs to be translated into machine code. If this happens immediately before execution this is referred to as JIT (Just In Time) compilation. Output of JIT is not persisted so your managed application has to go through JIT for every launch.
Alternatively, you can use pre-compilation to reduce startup overheads related with JIT compilation. NGen performs pre-compilation and keeps the native images in a native image cache. Then applications can run with the native images and may experience faster startup time due to reduced JIT compilation overhead. Initially, NGen was an install-time technology, developers made application installers issue NGen commands to trigger pre-compilation during install time. For more details, check out NGen Revs Up Your Performance with Powerful New Features. This article provides an example application that leverages NGen.
With Windows 8 (.NET 4.5), a new NGen mode: "Auto NGen" has been introduced. Basically, the .NET runtime generates usage logs for managed applications. When the system is idle, an automatic maintenance task runs in the background and generates native images. This way developers no longer have to deal with NGen explicitly. Note that this feature is only enabled for .NET 4.5+ applications that target Window Store or use the GAC. Here's an MSDN page that may be helpful:
Creating Native Images
And this is high-level overview of NGen and related technologies:
Got a need for speed? .NET applications start faster
Lastly, .NET framework libraries themselves use NGen for better performance. When .NET framework is serviced, some of the native images get invalidated. Then NGen needs to run to re-generate the invalid native images. This is done automatically via the .NET Runtime Optimization service which runs during idle time.
When a .NET compiler compiles C# or VB.NET code it half compiles them and creates CIL code. When you run this half-compiled .NET EXE file the JIT runs in the background and compiles the half CIL code in to full machine language. This mode is termed as normal JIT.
You can also go the other way around saying you do not want runtime compilation by running a full compiled EXE file. This compilation is done by using negen.exe. In this scenario the JIT does not participate at runtime. This is termed as pre-JIT mode.
If you want to see how they affect performance you can see this YouTube video which demonstrates normal-JIT and pre-JIT mode of compilation:
Explain JIT, Ngen.exe, Pre-jit, Normal-Jit and Econo-Jit.? (.NET interview questions)
Per MSDN:
The Native Image Generator (Ngen.exe) is a tool that improves the performance of managed applications. Ngen.exe creates native images, which are files containing compiled processor-specific machine code, and installs them into the native image cache on the local computer. The runtime can use native images from the cache instead of using the just-in-time (JIT) compiler to compile the original assembly.
I have used NGEN in the past during installation so that the software would start up faster.
NGen (Native Image Generator) basically compiles .NET byte code (CIL) into native code for the computer it's running on. The benefit is that given that you're not compiling the code to native every time, you run it or need it, but you do it just once, the application starts and run faster. If you want more information there are plenty of resources out there about the benefits of JIT vs. Ahead of Time Compilation (which is what NGen does).
How do I debug a Node.js server application?
Right now I'm mostly using alert debugging with print statements like this:
sys.puts(sys.inspect(someVariable));
There must be a better way to debug. I know that Google Chrome has a command-line debugger. Is this debugger available for Node.js as well?
node-inspector could save the day! Use it from any browser supporting WebSocket. Breakpoints, profiler, livecoding, etc... It is really awesome.
Install it with:
npm install -g node-inspector
Then run:
node-debug app.js
Debugging
Joyent's Guide
Debugger
Node Inspector
Visual Studio Code
Cloud9
Brackets
Profiling
node --prof ./app.js
node --prof-process ./the-generated-log-file
Heapdumps
node-heapdump with Chrome Developer Tools
Flamegraphs
0x
jam3/devtool then Chrome Developer Tools Flame Charts
Dtrace and StackVis — Only supported on SmartOS
clinicjs
Tracing
Interactive Stack Traces with TraceGL
Logging
Libraries that output debugging information
Caterpillar
Tracer
scribbles
Libraries that enhance stack trace information
Longjohn
Benchmarking
Apache Bench: ab -n 100000 -c 1 http://127.0.0.1:9778/
wrk
Other
Trace
Vantage
Bugger
Google Tracing Framework
Paul Irish's Guide
Legacy
These use to work but are no longer maintained or no longer applicable to modern node versions.
https://github.com/bnoordhuis/node-profiler - replaced by built-in debugging
https://github.com/c4milo/node-webkit-agent - replaced by node inspector
https://nodetime.com/ - defunct
The V8 debugger released as part of the Google Chrome Developer Tools can be used to debug Node.js scripts. A detailed explanation of how this works can be found in the Node.js GitHub wiki.
Node has its own built in GUI debugger as of version 6.3 (using Chrome's DevTools)
Simply pass the inspector flag and you'll be provided with a URL to the inspector:
node --inspect server.js
You can also break on the first line by passing --inspect-brk instead.
Node.js version 0.3.4+ has built-in debugging support.
node debug script.js
Manual: http://nodejs.org/api/debugger.html
Visual Studio Code will be my choice for debugging. No overhead of installing any tools or npm install stuff.
Just set the starting point of your app in package.json and VSCode will automatically create a configuration file inside your solution. It's build on Electron, on which editors like Atom are built.
VS Code gives similar debugging experience as you might have
had in other IDEs like VS, Eclipse, etc.
I personally use JetBrains WebStorm as it's the only JavaScript IDE that I've found which is great for both frontend and backend JavaScript.
It works on multiple OS's and has Node.js debugging built-in (as well as a ton of other stuff](http://www.jetbrains.com/webstorm/features/index.html).
My only 'issues'/wishlist items are were:
It seems to be more resource hungry on Mac than Windows It no longer seems an issue in version 6.
It would be nice if it had Snippet support (like those of Sublime Text 2 - i.e. type 'fun' and tap 'tab' to put in a function. See #WickyNilliams comment below - With Live Templates you also have snippet support.
A lot of great answers here, but I'd like to add my view (based on how my approach evolved)
Debug Logs
Let's face it, we all love a good console.log('Uh oh, if you reached here, you better run.') and sometimes that works great, so if you're reticent to move too far away from it at least add some bling to your logs with Visionmedia's debug.
Interactive Debugging
As handy as console logging can be, to debug professionally you need to roll up your sleeves and get stuck in. Set breakpoints, step through your code, inspect scopes and variables to see what's causing that weird behaviour. As others have mentioned, node-inspector really is the bees-knees. It does everything you can do with the built-in debugger, but using that familiar Chrome DevTools interface.
If, like me, you use Webstorm, then here is a handy guide to debugging from there.
Stack Traces
By default, we can't trace a series of operations across different cycles of the event loop (ticks). To get around this have a look at longjohn (but not in production!).
Memory Leaks
With Node.js we can have a server process expected to stay up for considerable time. What do you do if you think it has sprung some nasty leaks? Use heapdump and Chrome DevTools to compare some snapshots and see what's changing.
For some useful articles, check out
RisingStack - Debugging Node.js Applications
Excellent article by David Mark Clements of nearForm
If you feel like watching a video(s) then
Netflix JS Talks - Debugging Node.js in Production
Interesting video from the tracing working group on tracing and debugging node.js
Really informative 15-minute video on node-inspector
Whatever path you choose, just be sure you understand how you are debugging
It is a painful thing
To look at your own trouble and know
That you yourself and no one else has made it
Sophocles, Ajax
Theseus is a project by Adobe research which lets you debug your Node.js code in their Open Source editor Brackets. It has some interesting features like real-time code coverage, retroactive inspection, asynchronous call tree.
Node.js Tools for Visual Studio 2012 or 2013 includes a debugger. The overview here states "Node.js Tools for Visual Studio includes complete support for debugging node apps.". Being new to Node.js, but having a background in .NET, I've found this add in to be a great way to debug Node.js applications.
Visual Studio Code has really nice Node.js debugging support. It is free, open source and cross-platform and runs on Linux, OS X and Windows.
You can even debug grunt and gulp tasks, should you need to...
I wrote a different approach to debug Node.js code which is stable and is extremely simple. It is available at https://github.com/s-a/iron-node.
An opensource cross-platform visual debugger.
Installation:
npm install iron-node -g;
Debug:
iron-node yourscript.js;
I created a neat little tool called pry.js that can help you out.
Put a simple statement somewhere in your code, run your script normally and node will halt the current thread giving you access to all your variables and functions. View/edit/delete them at will!
var pry = require('pryjs')
class FizzBuzz
run: ->
for i in [1..100]
output = ''
eval(pry.it) // magic
output += "Fizz" if i % 3 is 0
output += "Buzz" if i % 5 is 0
console.log output || i
bar: ->
10
fizz = new FizzBuzz()
fizz.run()
If you are using the Atom IDE, you can install the node-debugger package.
Using Chrome Version 67.0.3396.62(+)
Run node app
node --inspect-brk=0.0.0.0:9229 server.js(server js filename)
Browse your app in chrome e.g. "localhost:port"
Open DevTools.
Click the the node icon beside the responsive device icon.
There will be another DevTools window that will pop out specifically for debugging node app.
There is built-in command line debugger client within Node.js. Cloud 9 IDE have also pretty nice (visual) debugger.
I put together a short Node.js debugging primer on using the node-inspector for those who aren't sure where to get started.
Visual Studio Code will work for us in debugging.
Use Webstorm! It's perfect for debugging Node.js applications. It has a built-in debugger. Check out the docs here: https://www.jetbrains.com/help/webstorm/2016.1/running-and-debugging-node-js.html
If you need a powerful logging library for Node.js, Tracer https://github.com/baryon/tracer is a better choice.
It outputs log messages with a timestamp, file name, method name, line number, path or call stack, support color console, and support database, file, stream transport easily. I am the author.
Assuming you have node-inspector installed on your computer (if not, just type 'npm install -g node-inspector') you just have to run:
node-inspector & node --debug-brk scriptFileName.js
And paste the URI from the command line into a WebKit (Chrome / Safari) browser.
Just for completeness:
The PyCharm 3.0 + Node.js Plugin offers an awesome development + run + debug experience.
Start your node process with --inspect flag.
node --inspect index.js
and then Open chrome://inspect in chrome. Click the "Open dedicated DevTools for Node" link or install this chrome extension for easily opening chrome DevTools.
For more info refer to this link
There is the new open-source Nodeclipse project (as a Eclipse plugin or Enide Studio):
Nodeclipse became #1 in Eclipse Top 10 NEW Plugins for 2013. It uses a modified V8 debugger (from Google Chrome Developer Tools for Java).
Nodeclipse is free open-source software released at the start of every month.
There are many possibilities...
node includes a debugging utility
node-inspector
Code editors / IDEs (see debug instructions for one of the following)
Atom,
VSCode
Webstorm
and more
Debug support is often implemented using the v8 Debugging Protocol or the newer Chrome Debugging Protocol.
IntelliJ works wonderfully for Node.js.
In addition, IntelliJ supports 'Code Assistance' well.
The NetBeans IDE has had Node.js support since version 8.1:
<...>
New Feature Highlights
Node.js Application Development
New Node.js project wizard
New Node.js Express wizard
Enhanced JavaScript Editor
New support for running Node.js applications
New support for debugging Node.js applications.
<...>
Additional references:
NetBeans Wiki / NewAndNoteworthyNB81.
Node.js Express App in NetBeans IDE, Geertjan-Oracle.
Use this commands
DEBUG_LEVEL=all node file.js
DEBUG=* node file.js
node file.js --inspect
ndb is an improved debugging experience for Node.js, enabled by Chrome DevTools
https://github.com/GoogleChromeLabs/ndb
node-debug -p 8888 scriptFileName.js
The short story
When compiling a C++ application for the X64 platform with Common Language Runtime support and using the native sqlite library inside it, the application crashes inside sqlite3MemRealloc, attempting to allocate a huge amount of memory (around 5GB).
When the same application is compiled without CLR support, the required functionality works and NO attempt to allocate this amount of memory is made. I put a break-point with a condition to verify this fact.
The database itself is a small 800KB file, and we are attempting to run a simple "select * from XYZ" query. Tried it both with the existing sqlite 3.7.11 that we have in our code base and the latest sqlite 3.7.14.
This problem is consistent. No matter how many times I rebuild the application or play with some settings - with CLR support it crashes, without CLR support it works.
The longer story
I was trying to develop an application that leverages code from an existing code-base written in C++ but also leverages the power of the .NET framework.
I created a C++/CLI application that linked against the existing code (that utilizes sqlite inside it). My code does not use sqlite directly. The existing code that does uses sqlite is a native C++ library that the rest of the code base depends on. So I cannot touch it easily and therefore cannot simply use System.Data.Sqlite.
I isolated this problem by removing all dependencies on the .NET framework and creating a simple application that only utilizes the existing native code without using any .NET framework code, and compiling it twice - with and without CLR support.
Eventually, the problem was solved by compiling SQLITE with the memsys5 memory allocator and configuring it to use it on startup. When it did not depend on the MSVCRT memory allocation but on its own internal allocation strategy, everything worked.
It seems that by some mysterious force, the existance of the CLR inteferes with MSVCRT's memory allocation functions.
More info can be found here.
I want to profile a MonoTouch application in Instruments without an iPad. I am able to profile an app compiled for Simulator but it's no fun without the symbols:
There is no option to generate IPA with dSYM in project build settings—looks like they're only available when compiling for the device.
All I want is a MonoTouch-aware dSYM for Simulator.
Is this possible?
This is unfortunately not possible, because in the simulator we JIT managed code. This means that it is not possible to know before executing the process where methods will be in memory, and Instruments doesn't support the required mechanisms for MonoTouch to inform about JIT'ed methods at runtime [1].
[1] See last comment on the response here: Is it possible to notify DTrace on Mac OS X of dynamically generated code?
Take a look at this - miguel wrote a blog post recently to compile aot for profiling on mac - I think this will give you what you're looking for:
Using Instruments to profile Mac apps built with Mono