When I load a markdown file from GitHub I am running into a lot of errors. I think I am not using the right encoding for GitHub files through Octokit. Any suggestions on how to fix my Buffer code in Node.js?
Is base64 and then to ascii correct for Github content? It works fine when loading it directly in my project without GitHub. I have a feeling GitHub stores their files in a different format but can't find docs on it.
const repos = await octokit.repos.getContents({
owner: 'owner-hidden',
repo: 'repo-hidden'
path: '/dinner.md
});
// repo loads with data.content just fine
const bufferedData = Buffer.from(repos.data.content, 'base64').toString('ascii');
const ymlData = YAML.parse(bufferedData); ## issue with reading this
The error is below, but the error doesn't necessarily matter because it works when I load it directly in my project there are no errors.
YAMLException: the stream contains non-printable characters at line 36, column 126:
... auteLed spinach and ratatouille
^
Loading the markdown file directly in my project there was no errors:
const fs = require('fs');
const path2 = require('path');
const file = path2.resolve(__dirname, '/dinner.md');
const content = fs.readFileSync(file);
const bufferedData = Buffer.from(content).toString('ascii');
console.log({bufferedData});
As one of the members of Octokit replied to me on my Github issue, I don't need to encode with ascii, I should be using uft8 as shown here:
- const bufferedData = Buffer.from(repos.data.content, 'base64').toString('ascii')
- const bufferedData = Buffer.from(repos.data.content, 'base64').toString()
buffer.toString() defaults to utf8 which is what I want.
Related
I'm getting a ZIP archive from S3 using the aws s3 node SDK.
In this zip file there is a single .json file where I want to get the contents from. I don't want to save this file to storage, but only get the contents of this zip file.
Example:
File.zip contains a single file:
file.json with contents({"value":"abcd"})
I currently have:
const { S3Client, GetObjectCommand} = require("#aws-sdk/client-s3");
const s3Client = new S3Client({ region: 'eu-central-1'});
const file = await s3Client.send(new GetObjectCommand({Bucket:'MyBucket', Key:'file.zip'}));
file.body now contains a Readable stream with the contents of the zip file. I now want to transfer this Readable stream into {"value":"abcd"}
Is there a library or piece of code that can help me do this and produce the result without having to save the file to disk?
You could use the package archiver or zlib (zlib is integrated in nodejs)
a snippet from part of my project looks like this:
import { unzipSync } from 'zlib';
// Fetch data and get buffer
const res = await fetch('url')
const zipBuffer = await res.arrayBuffer()
// Unzip data and convert to utf8
const unzipedBuffer = await unzipSync(zipBuffer)
const fileData = unzipedBuffer.toString('utf8')
Now fileData is the content of your zipped file as a string, you can use JSON.parse(fileData) to get the content as a json and work with it
I make an API request which returns a binary image. How can I save it to a file like photo.png on my machine? Doing some research, I've tried the following but when I open the image, my machine says it's damaged:
const buffer = new Buffer(imageBinary);
const b64 = buffer.toString("base64");
const path = `temp/${userId}`;
const url = path + "/photo.png";
if (!fs.existsSync(path)) fs.mkdirSync(path);
if (fs.existsSync(url)) fs.unlinkSync(url)
fs.createWriteStream(url).write(b64);
return url;
Edit: Here is the binary data FYI: https://gist.github.com/AskYous/1fd26dc0eb02b4ec1672dcf5c61a34df
You do not need to re-encode the buffer as base64. Just write the binary buffer as is:
fs.createWriteStream(url).write(imageBinary);
I have a file name is momo1.json. I can read them in Nodejs
var fs = require('fs');
var jsonWelcome = fs.readFileSync(WELCOME_JSON, UTF8_FORMAT);
var dataWelcome = JSON.parse(jsonWelcome);
But now i change file name is ももたろう.json. I cant read file anymore, tell my why and solutions i can read file name by japanese charaters
Try to use require() instead:
var dataWelcome = require("./ももたろう.json");
//output the data to see if it works
console.log(JSON.stringify(dataWelcome));
Live Demo
I tried searching for an entire day. Either i am blind or there is no specific answer to my question. How do i do it? I'm developing a steam bot in node js and i want to load accounts - username, pass etc.. of couple of accounts from .ini file. After a long time searching i only found how to read from an .ini file using node-ini module.
var ini = require('node-ini');
ini.parse('./config.ini', function(err,data){
console.log( data.Admin.pass ); //working.
data.Admin.pass = '1136'; //not working how to change value in ini.
data.Admin.H = 'test'; //not working as well.
});
This library (https://github.com/npm/ini) is a good option to do what you are looking to achieve.
Checkout this example from the docs:
var fs = require('fs')
, ini = require('ini')
var config = ini.parse(fs.readFileSync('./config.ini', 'utf-8'))
config.scope = 'local'
config.database.database = 'use_another_database'
config.paths.default.tmpdir = '/tmp'
delete config.paths.default.datadir
config.paths.default.array.push('fourth value')
fs.writeFileSync('./config_modified.ini', ini.stringify(config, { section: 'section' }))
I'm trying to read a json file using node js/ express and deploying it to parseCloud but i keep getting
*Failed to load filters.json with: Could not find file filters.json *
here is my code:
var fs = require('fs');
var obj = JSON.parse(fs.readFileSync('cloud/filters.json', 'utf8'));
or this
var filterJson = require('cloud/filters.json');
thanks
Looks like parse.com doesn't allow .json files. You can save your file as .js and load it as plain text file (doesn't work with require()).
var fs = require('fs');
var parsedObject = JSON.parse(fs.readFileSync('cloud/path/json_file.js'));
It looks ugly, but works for me. :)
Try to add ./
fs.readFileSync('./cloud/filters.json', 'utf8')