I am new to nodejs and I am trying to call an api server with url parameters stored in a text file.
The text file is a large one (around 20 gb). Now how can I read that file and make urls based on the parameters and call the api server?
The api server doesn't return much information. It just returns OK or nothing. So, the response doesn't matter now. It can be a 'fire and forget' request. If we can handle response, it will be good so that I can re run failed requests (if any) based on response code or response text
Thanks
You want to look into FileSystem.createReadStream http://nodejs.org/api/fs.html#fs_fs_createreadstream_path_options
Related
I want to send to the client in the same request files from the dir and some content from the DB.
DB query -
const derivatives = await Derivative.findAll();
Here is the res -
res.status(200).send({
data: derivatives.map((derivative) => ({
id: derivative.id,
date: derivative.date,
})),
});
Here is the download -
const fileName = derivatives.map((name) => name.wex);
res.status(200).download(__dirname, `/assets/${fileName}`);
How can I add that to my response?
HTTP lets you specify a "content disposition" to indicate whether a response should be treated as a download, but doesn't support sending downloads arbitrarily, it has to be a response to a request. You can't have part of a response be a download and part be not, for a single request.
So if you need a file to be downloaded, and some JSON used to display some UI, you need to handle that in the client somehow. Either:
The client sends a request, and server returns JSON containing a URL for the download as well as the other data you wanted to send, and then the client requests a download of the URL through JavaScript
The client sends two requests, one for the download and one for the other data; this may complicate things on the server if you need to associate the two requests (want to do a database lookup only once for instance), but is simplest on the client.
The client sends a request, and the server returns a response containing the JSON data and the file data, packed in some way (the file data could be inside the JSON but that would be inefficient), and it's unpacked on the client (using JavaScript) and the client then constructs a Blob URL to "download" (in this case the data is already downloaded, so this just entails saving a file)
There are any number of ways you might pack the file and JSON data together, which is what /u/Quentin was alluding to. Sending both as one response may be better for performance, but you probably don't need to.
I am developing an api based on nestjs. I used multer package to upload file. The code sample on nestjs documentation is at the following:
#Post('upload')
#UseInterceptors(FilesInterceptor('files'))
uploadFile(#UploadedFiles() files: Array<Express.Multer.File>) {
console.log(files);
}
But I want to save uploaded file after send mail. If the mail sends successfully then I will save the file. If sending mail process is fail, I ignore the file uploding.
How can I figure out?
You could do this in two ways, One is to create another route like#Post('mail') then depending on the response you receive in your client, let's say it returns an OK you can send another request to upload the files, or you can send both the requests to your API at the same time then cancel uploading the files (supposing this request takes longer to complete) if sending the mail was not successful (for this you need to handle errors that might result in incomplete files in your API, basically your API should know the full size of the files to expect so that you can compare if there was no cancellation).
The other way is using one route to do both of the tasks, in your example add the code for handling sending the mail and based on the condition that it went successfully do or don't proceed with uploading the files.
I have a nodejs app where I am using express for REST APIs.
One of the api is which accepts a SQL query, runs it on a DB and returns the JSON response. Everything was working fine until I tested the api with a long sql query.
Upon debugging , I noticed that the SQL query is trimmed automatically.
Is there a limit on the length of the param that can be passed in the GET URL?
This is what my api looks like
app.get('/v1/runsql/:query', (req, res) => {
let result = runQuery.executeQuery(req.params.query);
..... execute some more code here
})
Node enforces a limit not on the URL itself, but on the overall request headers (including URI) Requested headers + URI can not be more than 80 kb.
Also, it's an incredibly bad idea to expose an API that allows arbitrary SQL queries regardless of whether they're on the URL or not. Most applications spend a lot of effort trying to prevent arbitrary SQL from querying records that shouldn't be exposed, dropping tables, etc. Intentionally exposing an endpoint like this just feels like you're asking for trouble.
The http protocol not limit the length of url, but the browser and server(whatever Node or others) do the limit. If you really want to implement that, you may use a POST method instead of Get
And the http protocol spec set that: The server may return code 414 if the url length is out of limit
I defined my_function inside app.post('/someRoute',my_function) in nodejs which is used for making an http-request(posting data) to another server.
However it seems that my_function will run twice when I do not defined any response to the browser inside my_function.
What will be the reason for this? And is it required to send a response in every request no matter GET or POST in nodejs?
Thanks!
Whenever you do not define a response to a function/route handling get or post requests, the request response cycle will not end and thus the request keeps running until it times out because it took too long to get a response from the server. It is thus important to define a response to every request.
I'm trying to validate a webhook via facebook. So facebook hits my url my-url/facebook/receive within my route in nodejs i'd do res.send(req.query['hub.challenge']); to send an http response.
I'm using KoaJS. From what i understand, Koajs merges the request and response object into ctx but when reading through the docs I can't find anything along the lines of ctx.send or similar to send a http response.
Can anyone give me some direction or links.
Thanks.
To send the body of a response, you can simply do ctx.response.body = 'Hello'. There are many aliases attached to ctx, so you don't necessarily have to reference the response or request yourself. Doing ctx.body = 'Hello' would be the same as the code above.
If you wanted to set headers, you would use the ctx.set() method. For example: ctx.set('Content-Type', 'text/plain').
To access the query parameters, you would use ctx.request.query['some-key'] (or simply the alias ctx.query['some-key']).
All of the different request/response methods are documented pretty well at the Koa website along with a list of aliases attached to ctx. I highly recommend you give it a read.