I am using Express with Body Parser. Given a below header key:
X-Master-Key
When I am using the below code snippet, it fails to output the value
req.headers['X-Master-Key'] // Fails
but when the above is changed to, it works
req.headers['x-master-key'] // Works
Further, when I tried to output req.headers, it turns out that Express outputs all the headers in a down-case format.
I started digging down further and tried using the below code, either of these snippets work
req.header('X-Master-Key'); // Works
// -- OR
req.header('x-master-key'); // Works
So what's the issue here? Why does Express changes all header keys to down-case? Moreover, how using req.header() is different from req.headers[]?
The problem arises because in the HTTP protocol, headers are case-insensitive. This means that content-type, Content-Type, and coNTEnt-tYPe all refer to the same header, and the Express framework needs to be able to handle any of them.
The different between req.headers (the object) and req.header (the function) is simply this:
If you want to get a property from a Javascript object, the property name is case-sensitive. So req.headers['content-type'] will work; req.headers['Content-Type'] will not. Why does the lower case version work? Because the Express framework, in an attempt to handle all the different possible cases (remember, HTTP will allow anything), converts everything to lower case.
But the developers of Express recognize that you (the developer) might be looking for Content-Type and you might not remember to convert to lower case, so they provided a function, req.header, which will take care of that for you.
So, in short:
This is recommended:
const myHeader = req.header('Content-Type');
Use whatever case you want - the function will convert it to lower case and look up the value in req.headers.
This is not recommended:
const myHeader = req.headers['Content-Type'];
If you don't use a lower-case header name, you won't get what you expect.
The problem comes down to case-sensitivity.
When you look at the documentation for req.get (which is aliased by req.header), it states:
Returns the specified HTTP request header field (case-insensitive match). The Referrer and Referer fields are interchangeable.
The w3 standard indicates that headers should be case-insensitive:
Each header field consists of a name followed by a colon (":") and the field value. Field names are case-insensitive.
So it would appear that node http module, which express uses, just treats them all as lower-case to "save you steps" according to this github issue
You can see that the express framework req object actually utilizes the node module http:
var accepts = require('accepts');
var deprecate = require('depd')('express');
var isIP = require('net').isIP;
var typeis = require('type-is');
var http = require('http');
var fresh = require('fresh');
var parseRange = require('range-parser');
var parse = require('parseurl');
Furthermore, in the code you can see that the req.header method converts whatever you give it to lower-case:
req.get =
req.header = function header(name) {
if (!name) {
throw new TypeError('name argument is required to req.get');
}
if (typeof name !== 'string') {
throw new TypeError('name must be a string to req.get');
}
var lc = name.toLowerCase();
switch (lc) {
case 'referer':
case 'referrer':
return this.headers.referrer
|| this.headers.referer;
default:
return this.headers[lc];
}
};
Finally, the http module parses headers using the matchKnownFields function which automatically lower-cases any and all headers that aren't "traditional headers", in which case it is case-insensitive.
Here is the responsible snippet, that implements the behavior you are seeing:
if (lowercased) {
return '\u0000' + field;
} else {
return matchKnownFields(field.toLowerCase(), true);
}
Related
The code segment is:
app.get('/api/photo/:S3ObjectKey', photo.get);
And photo.get():
const S3ObjectKey = req.params.S3ObjectKey;
if (!S3ObjectKey) {
console.log("No S3ObjectKey specified");
}
console.log("S3ObjectKey: ", S3ObjectKey);
const canShow = true;
if (canShow) {
const bucket_name = process.env.AWS_S3_BUCKET_NAME;
const data = await S3.getFile(bucket_name, S3ObjectKey);
data.Body.pipe(res);
}
Is there a way to map all the rest of url as one parameter, such that:
a GET request to https://mylovelydomain/api/photo/bucketMyPhoto/2020/07/12/home/table.jpeg would hit the api endpoint and S3ObjectKey has the value bucketMyPhoto/2020/07/12/home/table.jpeg?
Express uses path-to-regex:
Custom Matching Parameters
Parameters can have a custom regexp, which overrides the default match ([^/]+).
On your routing path you can use * to get everything overwriting that default match (note that this code also reatcs to empty requests):
app.get("/api/photos/:S3ObjectKey(*)", /*...*/)
As per express documentation, route path supports regular expression, so should be able to do it. Quoting the example from the documentation.
I'm trying to make a simple HTTP module, similar to express, to help learn how to use HTTP.
When working with express I often use parameters such as:
app.get('/id/:id' (req, res) => {console.log(req.params.id); stuff})
I was wondering-
Is it possible just using HTTP?
If it isn't able to be done with just HTTP, then how would you go about creating it?
Your question is a little bit confusing but I think what you mean is how to implement an HTTP Router in pure javascript instead of relying on a framework like express.
If that's the case I support your initiative 100%! It's really important to understand what is going on behind the scenes.
A good way to really understand a good way to do so would be by reading the source code of a good router that is already out there.
You could study the Express router's source code but I recommend you to go play with find-my-way which is a dedicated router that you can use with HTTP only without any other framework.
Yes but you need to process it manually.
Assuming you're using node's http module, the information you are looking for is in req.url but it includes the whole url path.
For example, you want to parse /id/:id, and the browser is making a request to http://your.server/id/100?something=something. Then the value of req.url will be:
/id/100?something=something
But if you are writing an HTTP module from scratch using the net module then you need to know the format of an HTTP request. Basically an HTTP request looks like a text file with the following format:
GET /id/100?something=something HTTP/1.1
Host: your.server
The header section ends with double newlines. Technically it should be \r\n\r\n but \n\n is also acceptable. You first need to get the url from the first line of the request between the protocol (GET in the example above but can be POST, PUT etc.) and the HTTP version number.
For anyone interested in writing a HTTP server from scratch I always recommend James Marshall's excellent article: https://www.jmarshall.com/easy/http/. It was originally written in the late 90s but to this day I haven't found a clearer summary of the HTTP protocol. I've used it myself to write my first HTTP server.
Now you have to write code to extract the 100 from the string.
If you are doing this manually, not trying to write a framework like Express, you can do it like this:
const matches = req.url.match(/id\/([^\/?]+)/);
const id = matches[1];
If you want to write a library to interpret the /id/:id pattern you can maybe do something like this (note: not an optimized implementation, also it behaves slightly differently to express):
function matchRoute (req, route) {
const [ urlpath, query ] = req.url.split('?'); // separate path and query
const pathParts = urlpath.split('/');
const routeParts = urlpath.split('/');
let params = {};
for (let i=0; i<pathParts.length; i++) {
if (routeParts[i].match(/^:/)) { // if route part starts with ":"
// this is a parameter:
params[routeParts[i].replace(':','')] = pathParts[i];
}
else {
if (routeParts[i] !== urlParts[i]) {
return false; // false means path does not match route
}
}
// if we get here (don't return early) it means
// the route matches the path, copy all found params to req.params:
req.params = params;
return true; // true signifies a match so we should process this route
}
}
// Test:
let req.url = '/id/100';
let result = matchRoute(req, '/id/:id');
console.log(result, req.params); // should print "true, {id:100}"
/*Making http request to the api (Git hub)
create request
parse responce
wrap in a function
*/
var https = require("https");
var username = 'lynndor';
//CREATING AN OBJECT
var options = {
host: 'api.github.com',
path: ' /users/'+ username +'/repos',
method: 'GET'
};
var request = https.request(options, function(responce){
var body = ''
responce.on("data", function(chunk){
body += chunk.toString('utf8')
});
responce.on("end", function(){
console.log("Body", body);
});
});
request.end();
Im trying to create a request to the git hub api, the aim is to get the list repository for the specified you, but i keep getting the above mentioned error, please help
for other situation can be helpful
JavaScript encodeURI() Function
var uri = "my test.asp?name=ståle&car=saab";
var res = encodeURI(uri);
Your "path" variable contains space
path: ' /users/'+ username +'/repos',
Instead it should be
path: '/users/'+ username +'/repos',
Use encodeURIComponent() to encode uri
and decodeURIComponent() to decode uri
Its because there are reserved characters in your uri. You will need to encode uri using inbuilt javascript function encodeURIComponent()
var options = {
host: 'api.github.com',
path: encodeURIComponent('/users/'+ username +'/repos'),
method: 'GET'
};
to decode encoded uri component you can use decodeURIComponent(url)
Typically, you do not want to use encodeURI() directly. Instead, use fixedEncodeURI(). To quote MDN encodeURI() Documentation...
If one wishes to follow the more recent RFC3986 for URLs, which makes square brackets reserved (for IPv6) and thus not encoded when forming something which could be part of a URL (such as a host), the following code snippet may help:
function fixedEncodeURI(str) {
return encodeURI(str).replace(/%5B/g, '[').replace(/%5D/g, ']');
}
There is a similar issue with encodeURIComponent() (source: MDN encodeURIComponent() Documentation), as well as a similar fixedEncodeURIComponent() function. These should be used, rather than the actual encodeURI() or encodeURIComponent() function calls...
To be more stringent in adhering to RFC 3986 (which reserves !, ', (, ), and *), even though these characters have no formalized URI delimiting uses, the following can be safely used:
function fixedEncodeURIComponent(str) {
return encodeURIComponent(str).replace(/[!'()*]/g, function(c) {
return '%' + c.charCodeAt(0).toString(16);
});
}
I was getting this error while trying to hit Elasticsearch's API. For me, it was due to Chinese characters in the Document's Title (in the Request I was sending). Switching to all English characters fixed the issue.
Sometimes browser inspector uses abbreviation of long JSON object.
In my case, the data included unescaped characters such as '…' which should not be in http request.
In node.js(running this in Connect.js), I am able to set ether the Location or Set-Cookie with writehead, but not both at the same time. Currently, the below sets the cooke but the URL does not get redirected to the new location:
function foo(req, res, next) {
var url = /container-templates/;
if (url.test(req.url)) {
console.log(url.test(req.url));
console.log(req.url);
res.writeHead(302, ["Location", 'https://staging.dx.host.com' + req.url],
["Set-Cookie", "fake-token=5b49adaaf7687fa"]);
res.end();
} else { next() }
}
On a side note, I am doing this for the learning experience and do not want to use any pre-written plugins.
Response#writeHead expects the headers to be an Object not a list of array arguments.
The Node HTTP documentation has the following signature defined:
response.writeHead(statusCode, [reasonPhrase], [headers])
If you want to pass in multiple headers your code from above should read:
response.writeHead(302, {
Location: 'https://staging.dx.host.com' + req.url',
'Set-Cookie': 'fake-token=5b49adaaf7687fa'
});
The [] on the reasonPhrase means its optional and infers what you've provided to the function based on the types of the arguments.
Also, you don't need to wrap the key part of an object in quotes unless it contains characters that are invalid for variable names (like the -.) And a single ' will do for all strings -- there is no difference in javascript between ' and ".
So as usual, I tried to find this question on SO but still no luck.
I am aware that the answer is "Yes, you CAN modify the req object" but nothing is said about the req object parameters.
For example the following code will throw an error:
app.get('/search', function(req, res) {
req.param('q') = "something";
});
Error:
ReferenceError: Invalid left-hand side in assignment
I imagine this has something to do with the property not having a 'SET' method or something along those lines.
There are a few scenarios where this could come in handy.
A service that turns quick-links into full blown requests and proxies those out.
Simply modifying the parameters before sending it off to another function that you have no desire to modify.
Onto the question, Is there a way to modify the req object parameters?
Rather than:
req.param('q') = "something";
You'll need to use:
req.params.q = "something";
The first one is trying to set a value on the return value of the param function, not the parameter itself.
It's worth noting the req.param() method retrieves values from req.body, req.params and req.query all at once and in that order, but to set a value you need to specify which of those three it goes in:
req.body.q = "something";
// now: req.param('q') === "something"
req.query.r = "something else";
// now: req.param('r') === "something else"
That said unless you're permanently modifying something submitted from the client it might be better to put it somewhere else so it doesn't get mistaken for input from the client by any third party modules you're using.
Alternative approaches to set params in request (use any):
req.params.model = 'Model';
Or
req.params['model'] = 'Model';
Or
req.body.name = 'Name';
Or
req.body['name'] = 'Name';
Or
req.query.ids = ['id'];
Or
req.query['ids'] = ['id'];
Now get params as following:
var model = req.param('model');
var name = req.param('name');
var ids = req.param('ids');
One can also imagine that the left-hand side is not a variable or property, so it makes no sense to try to assign a value to it. If you want to change a query param you have to modify the property in the req.query object.
req.query.q = 'something';
The same goes for req.body and req.params.