I am sending info from a wi-fi connected arduino device (cc3000) to an AWS server running express/node. I plan to eventually hook the endpoint up to SQS as a producer, but for now I am just trying to get a POST request to work. The problem is, I am on secure shell on my AWS and the body portion is empty as I see post requests stream in. However, the headers are present as I have sent them. My question is, where is the data going? Is this a formatting problem or something wrong with AWS?
Raw body of POST request (I am just using plain/text for now to see if I can get this to work):
POST /postdata/ HTTP/1.1
Host: *******************
User-Agent: Arduino/0.6.0
Accept: plain/text
Content-Length: 36
Content-Type: plain/text
{"temperature_c": "29.80"}
Again, the headers are in the JSON payload, but not the "temperature" portion.
Here is the endpoint:
app.post('/postdata', function (req, res) {
Sensor_data.create(req.body);
console.log(req.body);
});
The headers are present. This is intended to write to a mongoDB. If I do a cURL request from the command line, it writes successfully. For example, this works:
curl -v *************** -d "{temp_f=28.40&relative_humidity=45.40"
So if the headers are there, I know I am successfully writing something to the server. Where the heck is the data? Is this a formatting problem or something wrong with AWS?
## EDIT ########:
I double checked to make sure body-parser was set up. I had installed the middleware in my server.js as so:
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
However, another issue has arisen. I have some code in this route to help me see what is going on by logging to console. I changed the POST request to application/json and know it is parsing because I had a small typo that I had to fix. However, once it started parsing correctly, nothing logged to console and nothing wrote to mongoDB. Any idea what is happening there? If the JSON is in fact being parsed, where is it being received on the server?
This issue was solved by setting app.use(bodyParser.urlencoded({ extended: false })); to be initialized with true and creating the post request with Accept and Content-Type to application/x-www-form-urlencoded. Hope this post helps some people as I have have seen quite a few posts with people struggling with this issue
It isn't clear whether you've set up the appropriate middleware to do the parsing. You'll find documentation here - in the first example you'd need the json parser, and in the second you'd need urlencoded (although there appears to be an extra leading { - a typo, perhaps?).
I am just using plain/text for now to see if I can get this to work
That's not going to work. The json-parsing middleware decides whether to try to parse based on the content-type - if you aren't honest, it doesn't know it should try to parse. Use application/json if you're going to send a JSON body.
Related
The error occurs when my client sends a GET request to the node.js server hosted with AWS lambda api gateway. The server is expected to send back an array of objects with res.json({}). The weird part is that when i test with a response of object or an array of objects with lesser variables in it, it works. I have also tried to JSON.stringify() the array in the server side and JSON.parse() in the client but to no avail. Thanks in advance to everyone helping me and do guide me along as its my first time posting on StackOverflow.
Edit: However i have tried it with a curl command to the endpoint and it returns the json array without any errors.
Alright everyone i found which part of my code caused the bug!
app.use(busboy());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(busboyBodyParser());
As i was playing around with file transfer before with the npm package 'Busboy', imported and used some packages associated with it. However, i forgotten to comment out these codes and thus, i believe it messed up the response body and got the error. Just to be clear this was in the node.js backend.
I have a problem with my backend when I send my data to the API req.body sends me null {}
suddenly I don't understand why I can't recover my data, however on the front side I send the data well ?
if you're using form data then include url encoded parser like this::
app.use(express.urlencoded({extended : true}));
app.use(express.json());
add this in your main server file where you create your server
It may be the order of variables, haven't tried, but that is the only thing that sticks out. When you are sending its {email,name,password2,password} but when you are receiving it its {name,email,password,password2}
Edit: messed up variable orders when replying ironically
I want to see in Node Exprees enviroment all the headers that are sent to the client
But when i see do this:
app.get("/", (req, res) => {
console.log(res.getHeaders());
});
i only see this :
At the time you're looking at the outgoing headers, those are the only ones that have been added so far. The rest will be added by the code that sends the actual response or by other middleware.
If you want to see all the headers that were eventually added to the response before it was sent, you can monitor the finish event and THEN look at the headers:
app.use(function(req, res, next) {
res.on('finish', () => {
console.log(`request url = ${req.originalUrl}`);
console.log(res.getHeaders());
});
next();
});
This will sometimes not include the date, content-type or content-length headers unless they are specifically set by the sending code. This is because if these are not set by the sending code, then the HTTP library adds these headers at a lower level as it is sending the headers and thus res.getHeaders() does not retrieve them and never knows about them.
Edit: I overlooked your first screenshot... Are you using any middleware? It looks like you're using the CORS middleware, at least - which is why you are showing more headers than the defaults..
It looks like Node/Express sends a Content-Length header when it can..
The Date header is mandatory per the HTTP spec, but it looks like you can change that behavior in Node/Express - I'm guessing by default Node/Express sets that value to true.
I did test setting res.sendDate = false and the date header was not sent, so it looks like that header is set by default for you, most likely as the last step in the response?
With res.sendDate = false;
Without setting res.sendDate (aka the default):
All in all, I'm assuming the headers you don't see when you console.log(res.getHeaders()) are set by Node/Express by default..
I wasn't able to find anything in the docs about default response headers (outside of the Date header), but it's possible I overlooked something. The Express docs don't have anything on it as they just use the built in http module from Node.
Need some help debugging the following situation. I am using some javascript code that sends data to a server using XMLHttpRequest()
It ends with req.send of this string "{ "xxxxsortBy": "xxxxxcoolness", "take": 10 }". (This is from the Elm Http.post method in practise.)
I'm running express with
app.use(bodyParser.json({limit: '10mb'}));
app.use(bodyParser.urlencoded({limit: '10mb', extended: true}));
Curl commands seem to work as do some other Post commands from older code, but when I log out req.body all I get is {}.
I would welcome some help trying to debug this situation - i.e. to get req.body to include the content posted. (e.g. curl --data "data=this is a text" http://localhost:5000/tcx does get picked up properly)
Below is part of the req that I receive on the server side.
I am interested in implementing a Content Security Policy (CSP) for my Node.js application. Mozilla's docs are rather helpful but I am stuck at how to enable violation reports. I understand the basic premise of how they work (the browser sends a POST request to the specified URL to notify the website of a violation) but could not figure out where to find the JSON document describing the violation in the HTTP request. Perhaps this would have been obvious to someone more familiar with the HTTP spec.
Looking at the W3C draft for CSP, I established that the JSON is contained in a portion of the HTTP called the "entity body". I still don't know what the purpose of the entity is (the only mildly useful page I could find on the matter was one from the HTTP spec). I am assuming it the body of the request.
Perhaps more importantly, I cannot find any way to retrieve the contents of the entity body. I thought of using req.header('entity-body') but that doesn't work as the entity is not a HTTP header. What is it and how to I fetch it?
(Additionally, I tried finding a tutorial on how to implement CSP violation reporting in Node.js and found nothing. I did find one for PHP but it wasn't particularly helpful, referencing a file_get_contents('php://input') which I don't have anything similar to in Node.js/Express.)
Any assistance would be greatly appreciated.
It turns out I was over-analyzing things. All you need to do is enable the express.bodyParser() middleware for express and then fetch req.body in the POST event handler. This retrieves the body of the HTTP request containing the JSON violation report.
Enable middleware:
var server = express.createServer(
// other middleware here
express.bodyParser()
);
Retrieving violation report:
server.post('/csp/', function(req, res) {
console.log(req.body);
});
I went through some difficulties getting my Express app fronted with Nginx to report csp violations and the two things I learned from the above answer were:
Should be POST method and not GET method
req.body contains the report
But, the above was not sufficient and I kept getting empty req.body and I could not find any other post to describe how to fix it. After some research I came across this post as well as a totally isolated github issue where dougwilson give hints where to put the route that handles the csp report.
The reason the req.body was empty for me was because I placed the csp report route handler after the following configs:
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(cookieParser());
I moved the csp report route handler above these but, I still kept getting empty req.body then I added the following above csp report route handler to get the report in req.body
app.use(bodyParser.json({ type: 'application/csp-report' }));
After adding the above line above csp report request handler, Express understood that it should parse requests that have Content-type as application/csp-report.
Maybe Express by default do not parse application/csp-report, and adding the above resolved the issue for me. I also googled if Express parses application/csp-report by default and I came across this gist claiming that Chrome sends application/csp-report whereas Firefox sends application/json (and I am using Chrome - you can include application/json also if you face issues with FF).
So this is how it looks in my app.js
// without following csp-report don't get parsed.
app.use(bodyParser.json({ type: 'application/csp-report' }));
app.get('/vehicle/cspreport', function(req, res) {
res.status(403);
});
app.post('/vehicle/cspreport', function(req, res) {
console.log('csp report > ' + JSON.stringify(req.body));
});
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));
app.use(cookieParser());
...
The accepted answer posted by OP is from 2011 and I thought of adding an answer to show how I resolved this issue in 2016 with the following versions of Node.js, Express and Nginx
Node: v4.2.4
Express: 4.13.1
Nginx: 1.8.1