In the Node https module docs,
regarding https.request, an example is shown:
const options = {
hostname: 'encrypted.google.com',
port: 443,
path: '/',
method: 'GET',
key: fs.readFileSync('test/fixtures/keys/agent2-key.pem'),
cert: fs.readFileSync('test/fixtures/keys/agent2-cert.pem')
};
options.agent = new https.Agent(options);
const req = https.request(options, (res) => {
// ...
});
This example is slightly ambiguous in my opinion, and I've asked an SO question regarding this ambiguity and following a comment reaffirming the strange wording, opened an issue for this.
Regardless, I am still trying to understand the part that the Agent plays in this scenario, seeing as the https.Agent module does accept TLS connection options:
interface AgentOptions extends http.AgentOptions, tls.ConnectionOptions
The definition of the https.Agent object is:
An Agent object for HTTPS similar to http.Agent.
And the definition for the http.Agent object is:
An Agent is responsible for managing connection persistence and reuse for HTTP clients.
From this I understand that an Agent is 'in charge' of managing a connection - and clearly, the fact that https.Agent exists on top of the 'plain' http.Agent exists would imply that it is 'in charge' of managing an HTTPS connection - hence the TLS configuration options it may receive.
My question is this - does this mean that the Agent in this case has an added responsibility of configuring the network security of the requests? this is a strange API if this is true - I would have expected to see the network connection config on a separate key for the https.request (as is shown in the example after the snippet above). Why overload the same object for another responsibility? Really, why have an https.Agent at all? The http.Agent should control connection pooling and keeping connections alive, while another layer should control configuring the actual requests. The https.Agent object doesn't seem well-defined to me.
Actually the HTTPS doc points to some good resources, it also contains a link to HTTPS module source code, which reveals a lot. But to answer your questions:
does this mean that the Agent in this case has an added responsibility of configuring the network security of the requests
Yes sort of, your HTTPS agent can perform custom security feature, or even use external HTTPS implementations, but for the built-in https.Agent, the native TLS module is used, and the options you passed to constructor is ultimately passed to tls.connect, allowing you to configure custom TLS options.
Why overload the same object for another responsibility? Really, why have an https.Agent at all?
Because HTTPS runs on TLS. Actually https.Agent internally calls and constructs with http.Agent. The extra bits are mainly the use of SSL "session" over TCP "socket", if you search getName in https.Agent source and compare that to http.Agent's, you will find the SSL session cache are based on many more fields, like DHparams, client cert, etc. things that does not exist for HTTP at all.
Related
I'm using the node aws-sdk package and I need to send a custom user agent in the S3 requests in order to identify the process in the console log.
I've seen a method to do this in the Java SDK but I can't see any similar in the node package.
Is there any way to do this easily?
After browsing in the source code I found an undocumented option to set the user agent: customUserAgent
const options = { customUserAgent: 'my-process-name' };
const client = new AWS.S3(options);
You can define an agent in the httpoptions field of the options you send to the constructor as per here:
httpOptions (map) — A set of options to pass to the low-level HTTP request.
Currently supported options are:
proxy [String] — the URL to proxy requests through
agent [http.Agent, https.Agent] — the Agent object to perform HTTP requests with. Used for connection pooling. Defaults to the global agent (http.globalAgent) for non-SSL connections. Note that for SSL connections, a special Agent object is used in order to enable peer certificate verification. This feature is only available in the Node.js environment.
connectTimeout [Integer] — Sets the socket to timeout after failing to establish a connection with the server after connectTimeout milliseconds. This timeout has no effect once a socket connection has been established.
timeout [Integer] — Sets the socket to timeout after timeout milliseconds of inactivity on the socket. Defaults to two minutes (120000).
xhrAsync [Boolean] — Whether the SDK will send asynchronous HTTP requests. Used in the browser environment only. Set to false to send requests synchronously. Defaults to true (async on).
Is that what you're looking for?
I have an endpoint (in any language, let's say Python) that exposes some service as HTTPS using a certificate issued by any widely known and trusted CA, that is
probably included in virtually any browser in the world.
The easiest part is that I can issue TLS requests against this endpoint using Node.js with no further problems.
For security reasons, I would like to check that every time my Node.js issues a TLS request against this HTTPS endpoint, I want to make sure that the certificate being used, is the certificate that I trust, and the one that was requested by my company.
What is the best way to accomplish that?
It sounds like the answer at How to get SSL certificate information using node.js? would be suitable for your needs.
You can use the following code to get your endpoint's certificate then check its fingerprint or hash against what you expect.
var https = require('https');
var options = {
host: 'google.com',
port: 443,
method: 'GET'
};
var req = https.request(options, function(res) {
console.log(res.connection.getPeerCertificate());
});
req.end();
i'm building a web application on NodeJS using express; the session store is a Redis instance which i talk to using connect-redis. the usual bits look like, well, usual:
RedisStore = ( require 'connect-redis' ) express
express_options =
...
'session':
'secret': 'xxxxxxxx'
'store': new RedisStore host: '127.0.0.1', port: 6379, ttl: 2 * weeks
'cookie': maxAge: 2 * weeks
app = express()
# Middleware
...
app.use express.cookieParser 'yyyyyy'
app.use express.session express_options[ 'session' ]
...
this does work well as such. however, i have not demonized Redis yet. after starting the server (but not Redis) and re-issuing an HTTP request from the browser, the application (apparently, naturally) failed to recognize yesterday's session cookie. to be more precise, the point of failure was
request.session.regenerate =>
request.session.user = uid_hint
in a login view, and the message was TypeError: Cannot call method 'regenerate' of undefined. now the question is:
(1) is my impression true that express won't balk at me when i try to use a session middleware that is configured to ask for data on a specific port, and yet that port is not served at all? if so, why is there no error message?
(2) what is a good way to test for that condition? i'd like a helpful message at that point.
(3) given that a DB instance may become unavailable at any one time—especially when it is separated by a network from the app server—what are best practices in such a case? fall back to memory-based sessions? refuse to serve clients?
(4) let us assume we fall back on another session storage mechanism. now all existing sessions have become invalid, right? unless we can decide whether a given signed SID coming in from a client is computationally valid in the absence of an existing record. those sessions will still be devoid of data, so it's not clear how useful that would be. we might as well throw away the old session and start a new one. but how? request.session = new ( require 'express' ).session.Session(), maybe?
Bonus Points (i'm aware some people will scoff at me for asking so many different things, but i think a discussion centered on sessions & cookies should include the below aspect)
thinking it over, i'm somewhat unhappy i'm using Redis at all—not because it's Redis, but because i have yet another DB make in the app. a theoretical alternative to using a session DB could be a reasonably secure way to keep all session data (NOT the user ID data, NO credit card numbers—just general stuff like which page did you come from etc) within the cookie. that way, any one server process can accept a request and has all the session data at hand to respond properly. i'm aware that cookie storage space is limited (like 4kB), but that might prove enough still. any middleware to recommend here? or is the idea dumb / insecure / too 1990?
connect-reddis listens to redis for the error event
./lib/connect-redis.js
self.client.on('error', function () { self.emit('disconnect'); });
So after creating the store, listen to the disconnect event
var store = new RedisStore({
host: 'localhost',
port: 6379,
db: 2,
pass: 'RedisPASS'
});
store.on('disconnect', function(){
console.log('disconnect');
});
I'm writing a Node.js app that has to request some data from one of our internal APIs. The tricky part is that the server I'm requesting data from has certain limitations:
The request must be made on HTTPS protocol (not HTTP)
The request must be made using a LAN IP address, because the domain name will not work internally
The request must appear to be requesting from the external domain name, because that is what the Virtual Host is setup for.
In order to do this, I'm running a bit of code that looks like this:
var headers = {
Host: externalHostname,
Hostname: externalHostname,
};
var options = {
host: InternalIP,
path: path,
method: 'GET',
headers: headers
};
var req = https.request(options, function(res) {
res.setEncoding('utf8');
var data = "";
res.on('data', function(chunk) {
data += chunk;
});
res.on('end', function() {
//Do something with that data
});
res.on('error', function(err) {
console.log("Error during HTTP request");
console.log(err);
});
});
req.end();
Unfortunately, I'm getting a 400 (Your browser sent a request that this server could not understand) error as a response. I've double and triple checked that the hostname, ip address, and path name are all correct (I can test them from within my browser, and all is good).
I did an output of my response variable (res), and am receiving an authorizationError value of UNABLE_TO_VERIFY_LEAF_SIGNATURE. I'm not sure what that is, or if it's my problem, but it's the only useful bit of information I could find.
I put a full output of my response variable here.
Any ideas on what might be causing this?
Update: I figured it out! I was trying to authenticate with the server by passing a ?PHPSESSID=asdad GET variable, but they have that disabled. I was able to make it work by setting PHPSESSID in the Cookie header.
set this process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';
I hit here while debugging UNABLE_TO_VERIFY_LEAF_SIGNATURE error in an external api call from my nodejs server.
This error is hit when there is error during verification of the server certificate. While it is not recommended to disable the security by the following code (which is also available as another answer), it helps to verify if you are chasing the right bug. In other words, if putting this also does not fix it, there is something else wrong with the code.
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';
In my case, there was silly bug & request was going to localhost itself. Even after putting the above, request failed and that helped me uncover the bug.
Having said that, it is not recommended to use this as a solution. Rather figure out how you can provide additional certificates by setting agent:false & ca:[fs.readFileSync('root-cert.pem')] options. https.request documentation provides details. While chasing my bug, I also found few more useful resources:
ssl-tools.net site provides root & intermediate certificates. For example: Baltimore CyberTrust Root used by lives.api.net
ssl-root-cas module claims to provide additional CA certificates as used by popular browsers. I have not verified the claim.
openssl s_client -connect apis.live.net:443 -- prints the certificate chain. you need to replace the last parameter (url & port) with what you are connecting to.
check this out from the tls.js source in the latest node.js (there is much more this is what I think you need)
// AUTHENTICATION MODES
//
// There are several levels of authentication that TLS/SSL supports.
// Read more about this in "man SSL_set_verify".
//
// 1. The server sends a certificate to the client but does not request a
// cert from the client. This is common for most HTTPS servers. The browser
// can verify the identity of the server, but the server does not know who
// the client is. Authenticating the client is usually done over HTTP using
// login boxes and cookies and stuff.
//
// 2. The server sends a cert to the client and requests that the client
// also send it a cert. The client knows who the server is and the server is
// requesting the client also identify themselves. There are several
// outcomes:
//
// A) verifyError returns null meaning the client's certificate is signed
// by one of the server's CAs. The server know's the client idenity now
// and the client is authorized.
//
// B) For some reason the client's certificate is not acceptable -
// verifyError returns a string indicating the problem. The server can
// either (i) reject the client or (ii) allow the client to connect as an
// unauthorized connection.
//
// The mode is controlled by two boolean variables.
//
// requestCert
// If true the server requests a certificate from client connections. For
// the common HTTPS case, users will want this to be false, which is what
// it defaults to.
//
// rejectUnauthorized
// If true clients whose certificates are invalid for any reason will not
// be allowed to make connections. If false, they will simply be marked as
// unauthorized but secure communication will continue. By default this is
// false.
//
set rejectUnauthorized to false in your options and cross your fingers...let me know if the output changes.
Set this process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = '0';
Fixed the UNABLE_TO_VERIFY_LEAF_SIGNATURE problem for superagent.
Try this in command line:
npm config set strict-ssl false
It worked for me on mac.
I am using TLS to create session using node.js library. Node.js does it provide a way to retrieve session id of TLS connection established. It is part of SSL ctx in openssl.
Can it be done without using connect, express or geddy?
Unfortunately I don't think that information is exposed from the SSL context for node connections.
You can access the node object representing the context as follows:
var con = tls.connect(..., ...);
con.pair.credentials.context
Unfortunately, the only methods available on that object are setKey, setCert, addCACert, addCRL, addRootCerts, setCiphers and setOptions.
That said, with a little bit of C++ and SSL know-how and come copy/pasting, you could probably patch node's node_crypto.cc and node_crypto.h files to add that lookup without TOO much work.
You can't get the session_id but you can get the session itself for caching / resume purposes by calling conn.getSession() once the connection is established.