Using express to send a modified file - node.js

I'd like to deliver a modified version of a file based on the URL route.
app.get('/file/:name/file.cfg', function (req, res) {
res.send(<the file file.cfg piped through some sed command involving req.params.name>)
});
The point being, the response should not be of type text/html, it should be the same MIME type as normal (which may still be wrong, but at least it works).
I am aware of security issues with this approach. The question is about how to do this with express and node.js, I'll be sure to put in lots of code to sanitize input. Better yet, never hit the shell (easy enough to use JS rather than e.g. sed to do the transformation)

I believe the answer is something along these lines:
app.get('/file/:name/file.cfg', function (req, res) {
fs.readFile('../dir/file.cfg', function(err, data) {
if (err) {
res.send(404);
} else {
res.contentType('text/cfg'); // Or some other more appropriate value
transform(data); // use imagination please, replace with custom code
res.send(data)
}
});
});
The cfg file I happen to be working with is (this is dump of node repl):
> express.static.mime.lookup("../kickstart/ks.cfg")
'application/octet-stream'
>
Quite the generic option, I'll say. Anaconda would probably appreciate it.

What's a normal filetype for you?
Set the mimetype using (docs):
app.get('/file/:name/file.cfg', function (req, res) {
res.set('content-type', 'text/plain');
res.send(<the file file.cfg piped through some sed command involving req.params.name>)
});
If you want to detect the file's mime type use node-mime
To send a file from disk use res.sendfile which sets the mimetype based on the extension
res.sendfile(path, [options], [fn]])
Transfer the file at the given path.
Automatically defaults the Content-Type response header field based on the filename's extension. The callback fn(err) is invoked when the transfer is complete or when an error occurs.
app.get('/file/:name/file.cfg', function (req, res) {
var path = './storage/' + req.params.name + '.cfg';
if (!fs.existsSync(path)) res.status(404).send('Not found');
else res.sendfile(path);
});
You can also force the browser to download the file with res.download. express has much more to offer, have a look at the docs.

Related

request.get() - if url not found

So I want to end the request if the user tries to fetch a url which does not work.
This works:
var remote = "https://storage.googleapis.com/ad-system/testfolder/OUTOFAREA.mp3";
var streama = request.get(remote);
however lets say the following mp3 does not exisit
https://storage.googleapis.com/ad-system/testfolder/playme.mp3
When the request.get('https://storage.googleapis.com/ad-system/testfolder/playme.mp3'); tries and fetch the file it will return a 404 error. What I want to do is file not found I want to run res.end();
Any ideas?
you can write like ### request('endpoint url goes here', { json: true }, (err, res, body) => {
if (err) { // keep your business logic here}
}) ######
hope it helps
I doesn't know about this Cloud Storage but even the link isn't accessible so I recommend you check if something is wrong in this service or if you require some special keys or something else to access the content or the service. Then, check your code or use a library that can handle your requirements (if available).
If I found something, I'll let you know :)
EDIT: I can access to https://storage.googleapis.com/ad-system/testfolder/Video/30%20Second%20Timer-0yZcDeVsj_Y.f137.mp4
You could do
app.get("*",(req,res)=> {
res.end()
});
and insert it just before your 404 handler however, I would challenge why that ever might be a good idea. If it's literally so you scan skip checking the return code I really advice you to think twice if thats something you wanna be doing atleast do:
app.get("*",(req,res)=> {
res.status(404)
res.end()
});

Send an image as the body of a request, image recived with a request from outside

Yeah i kinda didn't know how to type the title well...
I've a node server which recives an image via post form. I then want to send this image to Microsoft vision and the same Google service in order to gether information from both, do some stuff, and return a result to the user that has accessed my server.
My problem is: how do i send the actual data?
This is the actual code that cares of that:
const microsofComputerVision = require("microsoft-computer-vision");
module.exports = function(req, res)
{
var file;
if(req.files)
{
file = req.files.file;
// Everything went fine
microsofComputerVision.analyzeImage(
{
"Ocp-Apim-Subscription-Key": vision_key,
"content-type": "multipart/form-data",
"body": file.data.toString(),
"visual-features":"Tags, Faces",
"request-origin":"westcentralus"
}).then((result) =>
{
console.log("A");
res.write(result);
res.end();
}).catch((err)=>
{
console.log(err);
res.writeHead(400, {'Content-Type': 'application/json'});
res.write(JSON.stringify({error: "The request must contain an image"}));
res.end();
});
}
else
{
res.writeHead(400, {'Content-Type': 'application/octet-stream'});
res.write(JSON.stringify({error: "The request must contain an image"}));
res.end();
}
}
If instead of calling "analyzeImage" i do the following
res.set('Content-Type', 'image/jpg')
res.send(file.data);
res.end();
The browser renders the image correctly, which made me think "file.data" contains the actual file (considered it's of type buffer).
But apparently Microsoft does not agree with that, because when i send the request to computer vision i get the following response:
"InvalidImageFormat"
The only examples i found are here, and the "data" that is used in that example comes from a file system read, not stright from a request. But saving the file to load it and then delete it to me looks like an horrible workaround, so i'd rather like to know in what form and how should i work on the "file" that i have to send it correctly for the APIs call.
Edit: if i use file.data (which i thought was the most correct since it would be sending the raw image as the body) i get an error which says that i must use a string or a buffer as content. So apparently that file.data is not a buffer in the way "body" requires O.o i'm not understanding honestly.
Solved, the error was quite stupid. In the "then" part, res.write(result) did not accept result as argument. This happened when i actually used the corret request (file.data which is a buffer). The other errors occurred everytime i tryed using toString() on file.data, in that case the request wasn't accepted.
Solved, the request asked for a buffer, and file.data is indeed a buffer. After chacking file.data type in any possible way i started looking for other problems. The error was much easier and, forgive my being stupid, too stupid to be evident. The result was a json, and res.write didn't accept a json as argument.
This is how I did it with Amazon Recognition Image Classifier, I know its not the same service your using - hoping this helps a little thou:
const imagePath = `./bat.jpg`;
const bitmap = fs.readFileSync(imagePath);
const params = {
Image: { Bytes: bitmap },
MaxLabels: 10,
MinConfidence: 50.0
};
route.post('/', upload.single('image'), (req, res) => {
let params = getImage();
rekognition.detectLabels(params, function(err, data) {
if (err) {
console.log('error');
}else {
console.log(data);
res.json(data);
}
});
});

Node/Express, How do I modify static files but still have access to req.params?

I'm new to node/express, so there's (hopefully) an obvious answer that I'm missing.
There's a middleware for transforming static content: https://www.npmjs.com/package/connect-static-transform/. The transformation function looks like:
transform: function (path, text, send) {
send(text.toUpperCase(), {'Content-Type': 'text/plain'});
}
So, that's great for transforming the content before serving, but it doesn't let me look at query parameters.
This answer shows how to do it Connect or Express middleware to modify the response.body:
function modify(req, res, next){
res.body = res.body + "modified";
next();
}
But I can't figure out how to get it to run with static file content. When I run it res.body is undefined.
Is there some way to get a middleware to run after express.static?
My use case is that I want to serve files from disk making a small substitution of some text based on the value of a query parameter. This would be easy with server-side templating, like Flask. But I want the user to be able to do a simple npm-install and start up a tiny server to do this. Since I'm new to node and express, I wanted to save myself the bother of reading the url, locating the file on disk and reading it. But it's becoming clear that I wasted much more time trying this approach.
The answer appears to be "There is no answer." (As suggested by Pomax in the comment.) This is really annoying. It didn't take me too long to figure out how to serve and transform files myself, but now I'm having to figure out error handling. A million people have already written this code.
You can create middleware that only does transformation of body chunks as they are written with res.write or res.end or whatever.
For example:
const CSRF_RE = /<meta name="csrf-token" content="(.*)"([^>]*)?>/
function transformMiddleware (req, res, next) {
const _write = res.write
res.write = function(chunk, encoding) {
if (chunk.toString().indexOf('<meta name="csrf-token"') === -1) {
_write.call(res, chunk, encoding)
} else {
const newChunk = chunk.toString().replace(CSRF_RE, `<meta name="csrf-token" content="${req.csrfToken()}">`)
_write.call(res, newChunk, encoding)
}
}
next()
}

Getting filetype in an Express route

I have an Express route /doc/:id which serves up the HTML representation of a document, and I want it to serve the EPUB representation when appended with ".epub". Express doesn't separate on a period, however, so if I use /doc/:id.epub sets req.params.id to "id.epub". Is there a way to have the file extension recognised as a separate parameter or do I simply need to use a regex to extract it?
I have looked at res.format, but it seems this is only effective when the Accepted header is set, which it would not be if the URL is simply typed into a browser, as far as I can see.
This works:
app.get('/doc/:filename.:ext', function(req, res) {
...
});
This requires that the part following /doc/ contains at least one period, which may or may not be an issue.
Based on the behavior you are describing, I think you might have your route matching rules out of order.
This works:
app.get('/doc/:id.epub', function(req, res, next){
res.send('id: ' + req.params.id); //match /doc/x.epub (id=x)
});
app.get('/doc/:id', function(req, res, next){
res.send('id: ' + req.params.id); //match doc/x (id=x) | doc/y.html (id=y.html)
});
If /doc:/:id is first you'd get an id of x.epub for /doc/x.epub.

Prompt a csv file to download as pop up using node.js and node-csv-parser (node module)

Recently I have started working with node.js. While going through a requirement in one of my projects I am facing an issue where I should be able to write some data to a csv file dynamically and let it prompt as a popup to download for user (with save and cancel options - as we normally see). After googling for some time I decided to use csv npm module https://github.com/wdavidw/node-csv-parser. I am able to write data into a file and save it using this module. I want to prompt a popup for saving this file with/without saving the file.
my code looks something like this:
// Sample Data
var data = [["id", "subject1", "subject2", "subject3"], ["jack", 85, 90, 68], ["sam", 77, 89, 69]]
// Server Side Code
var csv = require('../../node_modules/csv');
var fs = require('fs');
createCSV = function(data, callback) {
csv().from(data).to(fs.createWriteStream('D:/test.csv')) // writing to a file
}
// Client side call sample
$("#exportToCSV").click(function() {
callToServer.createCSV(data);
return false;
});
This is working good as far as writing the csv file is concerned.
I want to prompt this file immediately to download for users.
If this can be done without saving the file, that will be great.
How can I set content-type and content-disposition as we do in PHP
Any help is greatly appreciated.
-Thanks
Manish Kumar's answer is spot on - just wanted to include a Express 4 syntax variant to accomplish this:
function(req, res) {
var csv = GET_CSV_DATA // Not including for example.
res.setHeader('Content-disposition', 'attachment; filename=testing.csv');
res.set('Content-Type', 'text/csv');
res.status(200).send(csv);
}
I did it something like this :
http.createServer(function(request, response) {
response.setHeader('Content-disposition', 'attachment; filename=testing.csv');
response.writeHead(200, {
'Content-Type': 'text/csv'
});
csv().from(data).to(response)
})
.listen(3000);
Following solution is for Express
Express is evolved, instead of setting attachment and content type header, directly use attachment api http://expressjs.com/4x/api.html#res.attachment
Note: attachment() don't transfer the file, it just sets filename in header.
response.attachment('testing.csv');
csv().from(data).to(response);
Express-csv is a great module for writing csv contents to stream from a node.js server, which will be sent as a response to the client (and downloaded as a file). Very easy to use.
app.get('/', function(req, res) {
res.csv([
["a", "b", "c"]
, ["d", "e", "f"]
]);
});
The docs: https://www.npmjs.com/package/express-csv
When you pass an object, you need to prepend the headers explicitly (if you want them). Here's my my example using npm mysql
router.route('/api/report')
.get(function(req, res) {
query = connection.query('select * from table where table_id=1;', function(err, rows, fields) {
if (err) {
res.send(err);
}
var headers = {};
for (key in rows[0]) {
headers[key] = key;
}
rows.unshift(headers);
res.csv(rows);
});
});
Check out this answer: Nodejs send file in response
Basically, you don't have to save the file to the hard drive. Instead, try sending it directly to the response. If you're using something like Express then it would look something like this:
var csv = require('csv');
req.get('/getCsv', function (req, res) {
csv().from(req.body).to(res);
});

Resources