I'm trying to migrate one of my systems from RequestJS to the new (currently experimental) fetch feature of Node 17.5+ but I'm having issues with the uploading of files, I'm hoping someone can assist.
At the receiving end I have Multer which is doing a simple upload.single('document') which always returns an empty object, so I'm fairly sure the way I'm sending the data is wrong.
In order to send the request, I'm doing the below (this is not the exact code, it's just to give an idea):
var requestOptions = {
headers: {'Content-Type': 'multipart/form-data'},
formData: {
document: {
value: _bufferData,
options: {
filename: _fileName,
contentType: _mimeType
}
}
}
};
Then I'm obviously sending the request using:
fetch('http://myurl.com/fileUpload', requestOptions).then(....);
I'm guessing new fetch API doesn't process the formData property the same way requestJS does. Does anyone know how I should be doing this?
Thanks.
Update: I've investigated further and the issue appears to be due to the content-disposition header missing the filename. I'm not sure how Multer creates this header but I assume its from the formdata, where the filename is clearly being passed in.
Related
My request body for an endpoint is so long:
1st question: I read that we can use some request.body.js file for storing our request body and then call it where ever we need it. But unfortunately, I could not find any sample framework/tutorial to learn it.
2nd question: in my project the properties of the request body (especially names of properties) are not exactly matching with the response body that is gaven in the Swagger document. What can be the reason? What would be your approach?
I would appreciate it if you could help me to ridd off the question in the best possible way. Thank you!
It's quite straight forward, take a look at this login example:
cy.fixture('users.json').then((userdata) => {
cy.request({
method: 'POST',
url: <auth_url>,
form: true,
body: userdata
});
});
You can export this as cypress function and then have it available in all your test spec files.
users.json file in fixtures folder looks like this:
{
"username": "...",
"password": "..."
}
Hope that answers the first question at least.
I am trying to send a number of data including an image to a node js server so that I can create a new document and store the image in the file system. I know that I should send a form data after appending the image to the backend and in the backend I should use multer package to read the form data and store the image. This I have implemented and works just fine.
The problem I am facing is that I have not been able to find a good way of sending form data alongside additional data, that are required for creating the new document in a single request. For that I first tried
something like this {...data, ...formData} but it didn't work. Then I tried appending all the additional data inside the form data itself.
formData.append("basicDetails", data);
formData.append("picture", picture);
const res = await axios.post(BASE_URL, formData, {
headers: {
'Authorization': `Bearer ${token}`
}
});
But now, since all the data is in form data, reading the data like req.body.basicDetails and trying the create a new document seems to not work. If I am not mistaken I need to use packages like multer to read form data. But is there a way to just use form data for picture and not for other data without making multiple requests?
I was researching a bit and I found this StackOverflow thread talking about formdata.
They were mentioning the headers, so add:
'Content-Type': 'multipart/form-data'
to your headers and try that.
Hope it works then :)
I'm uploading a file to Drive and then trying to update the filename using the id. This is what I'm doing:
const updateFileName = await axios.patch(`https://www.googleapis.com/upload/drive/v3/files/${fileId}?updloadType=media`,
{'name':'MyFile.docx'},
{headers:
{'Content-Type': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
'Authorization': `Bearer ${some_value}`,
},
},)
The problem is, instead of updating the filename, it updates the content of the file and replaces the content with {'name':'MyFile.docx'}. I can't figure out why it's replacing the actual content instead of the filename. The Drive v3 API for updating requires a name/originalName property in the request body, I tried with both. I've also tried replacing name with title. In all cases it overwrites the actual content in the file instead of changing the filename. Can someone please give me an idea about the workaround for this?
The problem is the ?updloadType=media which is telling Google you want to update the media content. Lose that bit, and you're off to the races. Content Type can probably be removed, but if you specify it, it should be application/json
and the URL should be https://www.googleapis.com/drive/v3/files/fileId
I have a POST call to my nodeJS server that searches for some data on the mongo database and returns a CSV file with the requested data. The problem is that the data search and processing exceeds the 2 minute nodeJS default timeout.
On a diferent scenario y used:
res.writeHeader(200,'application/json');
res.write('starting fetch .... ');
to keep alive the request and prevent the timeout from the client by sending some res.write(''); from time to time.
Now Im using res.download() to download the resulting csv file, so not just sending JSON as response.
Tried to use this solution like this:
res.writeHeader(200,'application/json');
res.write('starting fetch .... ');
res.download()
But i get the "headers already sent" error.
Any idea on how to prevent the timeout until the data is processed and the file download is done ?
Thanks in advance
A few small things first of all, i think it's supposed to be res.writeHead() instead of res.writeHeader(). Secondly once you send the headers you can not send them again. You are getting the error because res.download() uses res.sendFile() which re-sends the headers. You can't use res.download() or res.sendFile() or res.send() because all of these three methods implicitly set the headers and send as part of the response. You'll need to use a combination of res.writeHead() ,res.write() and res.end(). I am just GET so i can test it from the browser, you can use post obviously.
router.get('/test', function (req,res,next) {
res.writeHead(200, {'Content-Type': 'application/json',
'Content-Disposition': 'attachment',
'filename':'package.json'
});
fs.readFile("package.json",function (err,data) {
if (err) {
throw err;
}else{
setTimeout(function () {
res.write(data);
res.end()
},200000);
}
});
function keepBusy(){
if(!res.finished){
res.writeProcessing();
setTimeout(keepBusy,1000)
}
}
setTimeout(keepBusy,1000);
});
This works fine for me. res.writeProcessing() sends the 102 status code. It is recommended for requests that last greater than 20 second but this blog post says that
102 has been left out more recent revisions of HTTP specifications
but it works fine for me in most browsers. If you have some control over the client as well there can be a much better and reliable solution.
I've been using Google Translate API for a while now, without any problems.
I recently pushed my app to my new server and even if it has been working perfectly on my local server, the same source code always gives me the "Required parameter: q" as error message.
I'm using NodeJS + ExpressJS + Request to send this request. Here's my test case:
var request = require('request');
request.post({
url: "https://www.googleapis.com/language/translate/v2",
headers: {"X-HTTP-Method-Override": "GET"},
form: {
key: /* My Google API server key */,
target: "en",
q: ["Mon premier essai", "Mon second essai"]
}
}, function(error, response, data) {
if (!error && response.statusCode == 200) {
console.log("everything works fine");
} else {
console.log("something went wrong")
}
});
Running on my local machine gives me "everything works fine", and running it on my server gives me "something went wrong". Digging more into it, I get the error message mentioned above.
As you can see, I'm trying to translate in one request two sentences. It's just a test case, but I really need to use this through POST request instead of doing two GET request.
I have no idea what this is happening, and I double checked my Google settings and I can't find something wrong there.
Also, I'm having no problem using Google Places APi with this same api key on my server.
I'm stuck. Anyone has any idea what's wrong here?
Well I finally found what was wrong: the new version of RequestJS doesn't work as the old one and my server was running 2.16 when my local machine was running 2.14.
The difference is the way the array is sent. I debugged and the old version was sending
key=my_api_key&target=en&q=Mon%20premier%20essai&q=Mon%20second%20essai
When the new version is sending
key=my_api_key&target=en&q[0]=Mon%20premier%20essai&q[1]=Mon%20second%20essai
So I just added 2.14.x instead of 2.x in my package.json file for now, hopefully it will get fixed soon — or maybe it's not a bug? I don't know.
This answer is a little late but to help people out there with this problem. The problem comes from the way the querystring module converts array parameters:
https://github.com/visionmedia/node-querystring
Its function qs.stringify converts fieldnames (q in the given example) that have an array value to the format:
q[0]=..q[1]=...
This is not a bug but an intended functionality. To overcome this problem without reverting to an old version of the request module you need to manually create your post by using the body option instead of the form option. Also you will need to manually add the content-type header with this method:
var request = require('request');
request.request({
url: "https://www.googleapis.com/language/translate/v2",
headers: {
"X-HTTP-Method-Override": "GET",
'content-type':'application/x-www-form-urlencoded; charset=utf-8'
},
body:'key=xxxx&target=en&q=q=Mon%20premier%20essai&q=Mon%20second%20essai'
}, function(error, response, data) {
if (!error && response.statusCode == 200) {
console.log("everything works fine");
} else {
console.log("something went wrong")
}
});
Obviously this is not as clean but you can easily create a utility function that creates the body string from the object the way you want it to.
things that pop into my head:
jquery file version on server and local PC are not the same
file encoding issues (UTF8 on PC ascii on server?)
have you tried testing it with chrome with Developer Tools open, then check "Network Tab" and verify exactly what is being sent to Google.
For me at least, when it works on one machine and not the other, it is usually due to the first 2 options.
Good Luck!