I want log every http request being made by a particular node app, and all of its modules. Wrapping requests in a function could work for all non-module code, the disadvantage would obviously be it doesn't include module code, and be cumbersome to do.
This is for apps already in production, only other option I thought of was tcpdump.
Setting NODE_DEBUG=http will make node log out detailed HTTP request information to the console.
Examples:
NODE_DEBUG=http,http2 node index.js
NODE_DEBUG=http,http2 npm start
For more information see:
NODE_DEBUG documentation
This blog post: Debugging tools and practices in node.js.
As of 2022-11-23, this is the list of the available NODE_DEBUG attributes (based on the blog post above and checking the nodejs source code):
child_process
cluster
esm
fs
http
http2
inspect
module
net
policy
repl
source_map
stream
stream_socket
timer
tls
tracing
worker
You can pass multiple modules as a comma-separated list: NODE_DEBUG=http,http2,tls
How to find the module IDs in nodejs source code:
Unfortunately, there isn't a listing of the available debug module IDs in the nodejs documentation.
To find the IDs in the source: search/grep the nodejs .js files for .debuglog( usage:
# Grep inside the nodejs project
$ grep -r -E --color "\.debuglog\('" nodejs/lib
This will return results like:
let debug = require('internal/util/debuglog').debuglog('esm', (fn) => {
let debug = require('internal/util/debuglog').debuglog('http2', (fn) => {
The string passed to .debuglog(...) (e.g. 'esm' and 'http2') is the module id that can be passed to NODE_DEBUG.
The easiest / least intrusive way is with a web proxy. Either an off the shelf one or one you write yourself in node. The machines the apps live on would have to get configured* to send all outbound traffic through the proxy and then the proxy can log the traffic. Details on implementation will vary based on which proxy/ approach you pick.
*Arguably, there are ways to do this such that the machines don't even know they're being proxied, but I've found in practice that's really hard to get right, especially with https traffic
In 2022, you can use grackle_tracking library. It helps you track all traffic, errors and analytics into your database or just to the console https://www.getgrackle.com/analytics_and_tracking
Check HTTP request logger called Morgan here
Related
I am implementing API in NestJs and new to this framework. For one of my API I am getting error as
Status Code: 431 Request Header Fields Too Large
To fix this issue I want to change max header size in NestJs configuration. Where do I change this configuration in NestJs application.
This question is already answered for node here and for nest you can pass the argument as same as for node:
nest start -- --max-http-header-size=80000
As it's explained here
NestJS is a framework for NodeJS, you can pass NodeJS params directly, so I think this could be useful for you NodeJS command line options
And in case you want to set it from the code you can follow the same idea as this post explains something bodyparser is available for both.
I hope this is useful for you.
I could not get this to work by forwarding the node argument the way it's done in the currently accepted answer. Perhaps this is specific to my environment, but I was able to get it working using Nest's exec argument:
nest start --exec "node --max-http-header-size=40960"
The official description of the exec argument:
Binary to run (default: node).
https://docs.nestjs.com/cli/usages#nest-start
The NPM package #microsoft/applicationinsights-we from microsoft when used in the frontend will add headers for tracking calls across different parts of the application (e.g. frontend, backend, services etc.).
I'm using axios in the frontend which out of the box does not work with the package. Neither disableFetchTracking:false nor disableAjaxTracking:false works. I don't want to replace axios with fetch, because axios is more convenient to use and this would be a lot of rewrite too.
What can I do?
The #microsoft/applicationinsights-web package does inject correlation headers into axios calls (by instrumenting XMLHttpRequest). There might be a few causes of the problem in your application:
Another library broke the instrumentation
Something else might be hijacking the XMLHttpRequest object and affecting the AppInsights instrumentation. One such library is pace.js, which overwrites the window.XMLHttpRequest constructor but does not add open, send and abort to it's prototype. AppInsights expects these functions to be present on the XMLHttpRequest's prototype: https://github.com/microsoft/ApplicationInsights-JS/blob/91f08a1171916a1bbf14c03a019ebd26a3a69b86/extensions/applicationinsights-dependencies-js/src/ajax.ts#L330
Here is a working axios + #microsoft/applicationinsights-web example:
Here is the same example, but with pace.js loaded - the Request-Id header is not added:
Either removing pace.js or placing it's script tag / import after the AppInsights initialization code should fix the problem.
Cross-origin requests
Another explanation might be that the frontend app is making cross-origin requests, which are not processed by AppInsights by default - the enableCorsCorrelation: true config setting is needed.
I have Clarion 9 app that I want to be able to communicate with HTTP servers. I come from PHP background. I have 0 idea on what to do.
What I wish to be able to do:
Parse JSON data and convert QUEUE data to JSON [Done]
Have a global variable like 'baseURL' that points to e.g. http://localhost.com [Done]
Call functions such apiConnection.get('/users') would return me the contents of the page. [I'm stuck here]
apiConnection.post('/users', myQueueData) would POST myQueueData contents.
I tried using winhttp.dll by reading it from LibMaker but it didn't read it. Instead, I'm now using wininet.dll which LibMaker successfully created a .lib file for it.
I'm currently using the PROTOTYPE procedures from this code on GitHub https://gist.github.com/ddur/34033ed1392cdce1253c
What I did was include them like:
SimpleApi.clw
PROGRAM
INCLUDE('winInet.equ')
ApiLog QUEUE, PRE(log)
LogTitle STRING(10)
LogMessage STRING(50)
END
MAP
INCLUDE('winInetMap.clw')
END
INCLUDE('equates.clw'),ONCE
INCLUDE('DreamyConnection.inc'),ONCE
ApiConnection DreamyConnection
CODE
IF DreamyConnection.initiateConnection('http://localhost')
ELSE
log:LogTitle = 'Info'
log:LogMessage = 'Failed'
ADD(apiLog)
END
But the buffer that winInet's that uses always returns 0.
I have created a GitHub repository https://github.com/spacemudd/clarion-api with all the code to look at.
I'm really lost in this because I can't find proper documentation of Clarion.
I do not want a paid solution.
It kind of depends which version of Clarion you have.
Starting around v9 they added ClaRunExt which provides this kind of functionality via .NET Interop.
From the help:
Use HTTP or HTTPS to download web pages, or any other type of file. You can also post form data to web servers. Very easy way to send HTTP web requests (and receive responses) to Web Servers, REST Web Services, or standard Web Services, with the most commonly used HTTP verbs; POST, GET, PUT, and DELETE.
Otherwise, search the LibSrc\ directory for "http" and you will get an idea of what is already there. abapi.inc for example, appears to provide a wrapper around wininet.lib.
I'm looking for a way to output Node variables directly into the google chrome browser console. The same way a console.log() works on the client side. Something like this for php. This would greatly speed up development.
NOTE:
Since the old answer (written in september 2014) refers to an older version of node-inspector, my instructions are not relevant anymore in 2017. Also, the documentation has gotten a lot better, so I have updated my original answer:
node-inspector is what you need.
It opens up an instance of Chrome with its developer tools for debugging.
It's also easy to use:
1. Install
$ npm install -g node-inspector
2. Start
$ node-debug app.js
Source: https://github.com/node-inspector/node-inspector
You might want to try NodeMonkey - https://github.com/jwarkentin/node-monkey
I know it's an old question but came on top of my Google search so maybe somebody will find my answer useful.
So you can use node --inspect-brk index.js
Now, all you have to do is basically just type chrome://inspect in your Chrome address bar and click Open dedicated DevTools for Node
In DevTools, now connected to Node, you’ll have all the Chrome DevTools features you’re used to:
Complete breakpoint debugging, stepping w/ blackboxing
Source maps for transpiled code
LiveEdit: JavaScript hot-swap evaluation w/ V8
Console evaluation with ES6 feature/object support and custom object formatting
Sampling JavaScript profiler w/ flamechart
Heap snapshot inspection, heap allocation timeline, allocation profiling
Asynchronous stacks for native promises
Hope that helped.
The closest thing to this I've seen is Node JS console object debug inspector
See this post for usage and potential issues: http://thomashunter.name/blog/nodejs-console-object-debug-inspector/
For users with nodejs on linux via ssh-shell (putty):
Problem with nodejs on linux-ssh-shell is, that you have no browser connected.
I tried all this solutions, but didnt get it to work.
So i worked out a solution with firebase (https://firebase.google.com), because my project uses firebase.
If you are familiar with firebase, than this is a great way. If not, firebase is worth using in combination with nodejs - and its free!
In the server-side-script (started with node) use a own function log():
// server-side:
// using new firebase v3 !
var fbRootRef = firebase.database();
var fbConsoleRef = fbRootRef.ref("/console");
var log = function(args) {
fbConsoleRef.set({'obj': args});
}
// inside your server-code:
log({'key':'value'});
On client-side you create a firebase-reference on this console-object:
// client side:
fbRootRef.child('/console').on('value', function(d) {
var v = d.val();
console.log(v);
});
Now everything logged on server-side with the log() - function is transferred in realtime to the firebase-database and from there triggering the client-console-reference and logged into the browsers console.
If anyone needs help, i will explain in more detail and could give a more extended version of this logging with types (console./log/warn/info), grouping with title-info (i.e. server says: (filename + line).
Setting up firebase for your project is done in max 30 minutes, inserting the console-function in 30 minutes. I think its worth the time!
You can use bonsole, a simple way to log something in browser. Even in Linux, you can go to the LAN's ip to check it.
The most simple way with least dependencies is using a WebSocket connection to send the messages to the browser. Any WebSocket example you can find on the internet will suffice to accomplish this. Everything else requires to be heavily integrated into the host system and wouldn't work if you want to actually run this on a remote server. You can also send commands to the server directly from the browser console this way.
Links:
https://www.npmjs.com/package/websocket
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API/Writing_WebSocket_client_applications
Last night I dump windows 7 and formatted my hard driver to port to a Linux based operating system, Purely for the reasons that I wanted to start working with Node.JS
So I have installed Node.JS and have done a few test stuff, the http server and sockets etc.
What I would like to do is build a HTTP Server that is tightly intergrated with an MVC Framework, but before I get started on all that I need to learn how to build efficiently in Node.
For example within PHP as my framework I would create a bootloading system to load all base classes etc, then i would fire my events system ready to start attaching callbacks.
I would continue to process the request etc until the output is generated which then gets sent of to an output handler that would process headers etc etc
But Node s a totally new environment for this and im wondering on the best practises to build an system in Node.
The information im looking for is more to do with the design structure rather then the actual coding of the application, how to load the lib where to load the libs, etc etc
Any help is appreciated.
So far my WebApplication is coming along nicely, I have built my application pretty traditionally and a little procedural.
What i have started out is creating a directory structure like so:
<root>
startup.js
/public/
favicon.ico
/images/
/stylesheets/
/javascripts/
/system/
init.js
config.js
/libs/
/exceptions/
http.js
server.js
/application/
/views/
/_override/
/errors/
generic.view
/partials/
sidebar.voew
index.view
/controllers/
index.js
/models/
users.js
This directory structure is like most MVC Based Web Applications out there so using this method I feel comfortable.
The startup file is whats executed by node as the entry point, node startup & and looks like so:
/*
* Header of t he file, Copyright etc
*/
var _Intitialize = require("./system/init.js");
//Displays the command line header, title, copyright etc
_Intitialize.DisplayCommandLineHeader();
//Check the enviroment, Permissions, Ports etc
_Intitialize.CheckEnviroment();
//Start the server and listen the port.
_Initialize.StartServer();
the init file is the main work, its what tells all other areas of the system to run, stop etc.
I have a file in libs called serverhandler.js, and this is required into init.js, I then create a server and assign the callback to the ServerHandler.Listener. Who then listens for requests, checks to see if the file exists in public directory, if so it then reads in chunks and sends back.
if no file was found in public it would then create a route with Route.Create("/path?params"); which deters 3 elements, Controller, Method, Params from the uri, and then the controller files are loaded if exists.
I've taken on the approach of throwing error pages like so:
if(!FileSystem.exists(RequiredPath))
{
throw new HTTPExceptions.FileNotFound();
}
Hope this helps some people getting started in Node.
Have a look at
http://dailyjs.com/2010/11/01/node-tutorial/ , it's pretty relevant.
I would suggest looking at the current modules too
https://github.com/joyent/node/wiki/modules
and reading the code of any of the projects in the areas you are interested in, esp. the middleware, routing and module loaders.