I am running a node script that reads a .svg font file, passes it as a string variable to Cheerio, modifies it and attempts to write to disk
The problem is that although the script seems to work, the output file is identical to the input file, as if no modification happened.
It looks to me as if the original "svgFont" variable that I pass to cheerio is not modified at all.
So I would need to either pass the modifications back to it, or output from cheerio directly to fs write. But I can't see how to.
const cheerio = require('cheerio');
var fs = require('fs');
// read the svg font
fs.readFile('font.svg', function read(err, data) {
if (err) {
throw err;
}
// convert to string and pass to cheerio
var svgFont = data.toString();
const $ = cheerio.load(svgFont, {
normalizeWhitespace: true,
xmlMode: true
});
// loop all glyph tags
$("glyph").each(function(i,el){
// modify things
});
// Finally write the font with the modified paths
fs.writeFile("outFont.svg", svgFont, function(err) {
if(err) {
throw err;
}
console.log("The file was saved!");
});
});
You could use the fs-cheerio package to write to files. In your code the original variable is not being modified. It is the parsed cheerio representation that gets modified.
The right answer is to pass the cheerio object '$' that contains all the modifications using, in this case, .xml(), as this is a .svg file I am reading. For example:
// Finally write the font with the modified paths
fs.writeFile("outFont.svg", $.xml(), function(err) {
if(err) {
throw err;
}
console.log("The file was saved!");
});
Related
If I have a program as follows to modify a JSON file:
var fs = require('fs');
var dt = require('./dataWrite.json');
console.log("before",dt);
fs.readFile('./data.json', 'utf8', (err, data) => {
if (err) {
throw err;
}
else {
fs.writeFileSync('./dataWrite.json', data);
}
});
console.log("after",dt);
The console for the before and after gives me the same results. The data in the file is modified as expected though. Is there a way to always have the latest state of the file in your program?
Side question: the following code doesn't modify the files at all, I wasn't able to figure why
var fs = require('fs');
var dt = fs.readFileSync('./dataTest.json', 'utf8', function (err, data) {
if (err) {
throw err;
}
});
console.log('before', dt);
fs.readFileSync('./data.json', 'utf8', (err, data) => {
if (err) {
throw err;
}
fs.writeFileSync('./dataTest.json', data);
console.log('data', data);
});
console.log("after", dt);
It's important here to distinguish between synchronous and asynchronous logic.
Since you are using require to read in the json file, the value of the file when the program executes is read in synchronously within dt, and read in once at the beginning of the program only.
When you use the fs.readFile API, you'll notice that it is an asynchronous API and that it requires you to provide a callback to handle the file's data. This means that any execution within it is handled at a later date.
As such, your before and after code will just print the same contents.
If you console.log(dt) after executing fs.writeFileSync you will still see the old value since dt is holding the old value known at the beginning of the program and not the latest value, but if you update the variable to the contents of the file after rereading the file, then you would see the latest contents.
e.g.
...
fs.writeFileSync('./dataWrite.json', data);
dt = fs.readFileSync('./dataWrite.json', 'utf8');
console.log(dt);
...
See fs.readFileSync.
I'm trying to read the file content as string and want to set the variable value equals to it. But when I try to access file_data variable in this case, this gives an empty value if it's used outside the function. However inside the fs.readFile function, it works fine.
var fs = require('fs');
let file_data = '';
fs.readFile('text_file.txt', 'utf8', function(err, data) {
if (err) throw err;
console.log(data); //this works fine here
file_data = data; //setting it here so that I can use it afterwards
});
//the following line gives blank output
console.log(file_data );
I'm a bit new at this, so please point out if there is something I need to read first before using functions like this.
fs.readFile() is an async function so
//the following line gives blank output
console.log(file_data );
Is processed before fs.readFile() responds, so you can:
fs.readFile('text_file.txt', 'utf8', function(err, data) {
if (err) throw err;
console.log(data); //this works fine here
file_data = data; //setting it here so that I can use it afterwards
// CONSOLE LOG HERE!! AND IT WORKS
console.log(file_data );
});
Alternatively, you can use the synchronous version fs.readFileSync()
try {
const data = fs.readFileSync('text_file.txt', 'utf8')
console.log(data)
} catch (err) {
console.error(err)
}
Source
I am using request js to download a file.
function requ(){
const options = {
uri: `api/tasks/${id}/attachments/${attachmentId}`
}
return rp.get(options)
}
My question is:
why piping to "res" like requ().pipe(res) works and returning the result of the request above using "send" like
requ().then((result)=>{
//here result is the file's representing string
res.send(result)
})
don't?
const fs = require('fs');
requ().then((result) => {
//here result is the file's representing string
const path = __dirname + '/tempFiles' + Date.now(); // a temporary file to send it
fs.writeFile(path, result, function(err) {
if(err) throw err;
return res.sendFile(path);
})
});
Read More About fs, link 2
My file was being corrupted because request was converting the response body to utf8. Using:
const options = {
uri: `api/tasks/${id}/attachments/${attachmentId}`,
encoding:null
}
fixed the problem
I'm currently building web using Sails.js and got stuck in retrieving image file from GridFS. I have successfully uploading the file using skipper-gridfs to my mongo gridfs. I have no idea to display the file in the correct way (I'm new in Sails.js and Node system)
Here is my code for retrieving image file from gridfs looks like in FileController.js (I'm using gridfs-stream):
show: function (req, res, next) {
var mongo = require('mongodb');
var Grid = require('gridfs-stream');
var buffer="";
// create or use an existing mongodb-native db instance
var db = new mongo.Db('testDb', new mongo.Server("192.168.0.2", 27017), {safe:true});
var gfs = Grid(db, mongo);
// streaming from gridfs
var readstream = gfs.createReadStream({
filename: 'e1ecfb02-e095-4e2f.png'
});
//check if file exist
gfs.exist({
filename: 'e1ecfb02-e095-4e2f.png'
}, function (err, found) {
if (err) return handleError(err);
found ? console.log('File exists') : console.log('File does not exist');
});
//buffer data
readstream.on("data", function (chunk) {
buffer += chunk;
console.log("adsf", chunk);
});
// dump contents to console when complete
readstream.on("end", function () {
console.log("contents of file:\n\n", buffer);
});
}
When I ran it, the console showed nothing.
There is no error either.
How should I fix this?
Additional Question:
Is it better & easier to store/read file to/from local disk instead of using gridfs?
Am I correct in choosing gridfs-stream to retrieve the file form gridfs?
In the skipper-gridfs codes and there's a 'read' method that accept fd value and returns the required file corresponding to that value. So, you just have to pull that file from mongo by that method and send as a response. It should work file.
download: function (req, res) {
var blobAdapter = require('skipper-gridfs')({
uri: 'mongodb://localhost:27017/mydbname.images'
});
var fd = req.param('fd'); // value of fd comes here from get request
blobAdapter.read(fd, function(error , file) {
if(error) {
res.json(error);
} else {
res.contentType('image/png');
res.send(new Buffer(file));
}
});
}
I hope it helps :)
Additional Questions:
Yes, using gridfs is better both in performance and efficiency. And normally mongodb has a limitation of 16MB probably for binary files, but using gridfs you can store any size file, it breaks them in chunks and stores them.
Retrieving has been shown above.
You can now use skipper-gridfs in sails to manage uploads/downloads.
var blobAdapter = require('skipper-gridfs')({uri: 'mongodb://jimmy#j1mtr0n1xx#mongo.jimmy.com:27017/coolapp.avatar_uploads' });
Upload:
req.file('avatar')
.upload(blobAdapter().receive(), function whenDone(err, uploadedFiles) {
if (err) return res.negotiate(err);
else return res.ok({
files: uploadedFiles,
textParams: req.params.all()
});
});
Download
blobAdapter.read(filename, callback);
Bear in mind the file name will change once you upload it to mongo, you have to use the file name returned in the first response.
Using nodejs and imagemagick am able to re-size an image and send it to the browser with this.
var http = require('http'),
spawn = require('child_process').spawn;
http.createServer(function(req, res) {
var image = 'test.jpg';
var convert = spawn('convert', [image, '-resize', '100x100', '-']);
convert.stdout.pipe(res);
convert.stderr.pipe(process.stderr);
}).listen(8080);
The test image is read from the file-system, I want to alter so that test image is a binary string.
var image = 'some long binray string representing an image.......';
My plan is to store the binary strings in Mongodb and read them of dynamically.
Take a look at the node module node-imagemagick. There is the following example on the module's page to resize and image and write it to a file...
var fs = require('fs');
im.resize({
srcData: fs.readFileSync('kittens.jpg', 'binary'),
width: 256
}, function(err, stdout, stderr){
if (err) throw err
fs.writeFileSync('kittens-resized.jpg', stdout, 'binary');
console.log('resized kittens.jpg to fit within 256x256px')
});
You can alter this code to do the following...
var mime = require('mime') // Get mime type based on file extension. use "npm install mime"
, fs = require('fs')
, util = require('util')
, http = require('http')
, im = require('imagemagick');
http.createServer(function (req, res) {
var filePath = 'test.jpg';
fs.stat(filePath, function (err, stat) {
if (err) { throw err; }
fs.readFile(filePath, 'binary', function (err, data) {
if (err) { throw err; }
im.resize({
srcData: data,
width: 256
}, function (err, stdout, stderr) {
if (err) { throw err; }
res.writeHead(200, {
'Content-Type': mime.lookup(filePath),
'Content-Length': stat.size
});
var readStream = fs.createReadStream(filePath);
return util.pump(readStream, res);
});
});
});
}).listen(8080);
Ps. Haven't run the code above yet. Will try do it shortly, but it should give you an idea of how to asynchronously resize and stream a file.
Since you are using spawn() to invoke the ImageMagick command line convert, the normal approach is to write intermediate files to a temp directory where they will get cleaned up either immediately after use or as a scheduled/cron job.
If you want to avoid writing the file to convert, one option to try is base64 encoding your images and using the inline format. This is similar to how images are encoded in some HTML emails or web pages.
inline:{base64_file|data:base64_data}
Inline images let you read an image defined in a special base64 encoding.
NOTE: There is a limit on the size of command-line options you can pass .. Imagemagick docs suggest 5000 bytes. Base64-encoded strings are larger than the original (Wikipedia suggests a rough guide of 137% larger) which could be very limiting unless you're showing thumbnails.
Another ImageMagick format option is ephemeral:
ephemeral:{image_file}
Read and then Delete this image file.
If you want to avoid the I/O passing altogether, you would need a Node.js module that directly integrates a low-level library like ImageMagick or GD rather than wrapping command line tools.
What have you tried so far? You can use GridFS to store the image data and retrieve as a stream from there.. This in C#..Not sure if this helps..
public static void UploadPhoto(string name)
{
var server = MongoServer.Create("mongodb://localhost:27017");
var database = server.GetDatabase("MyDB");
string fileName = name;
using (var fs = new FileStream(fileName, FileMode.Open))
{
var gridFsInfo = database.GridFS.Upload(fs, fileName);
var fileId = gridFsInfo.Id;
//ShowPhoto(filename);
}
}
public static Stream ShowPhoto(string name)
{
var server = MongoServer.Create("mongodb://localhost:27017");
var database = server.GetDatabase("MyDB");
var file = database.GridFS.FindOne(Query.EQ("filename",name));
var stream = file.OpenRead())
var bytes = new byte[stream.Length];
stream.Read(bytes,0,(int)stream.Length);
return stream;
}
You can now use the stream returned by ShowPhoto.