today i'd like to translate a text to json data
exemple :
1 kokoa#0368's Discord Profile Avatar kokoa#0000 826
2 Azzky 陈东晓#7475's Discord Profile Avatar Azzky 陈东晓#0000 703
3 StarKleey 帅哥#6163's Discord Profile Avatar StarKleey 帅哥#0000 640
to =>
{
"kokoa#0000": "826",
"Azzky 陈东晓#0000": "703",
"StarKleey 帅哥#0000": "640"
}
for the moment i have this one :
fs.appendFile(`./Text/mess.txt`, `${body.replace(/s Discord Profile Avatar/g, '').split(/[\n \t ' ']/)}`, function (err) {
So the result is this one =>
1,kokoa#0368,,kokoa#0000,826
2,Azzky 陈东晓#7475,,Azzky 陈东晓#0000,703
3,StarKleey 帅哥#6163,,StarKleey 帅哥#0000,640
But i would like delete firsts numbers, delete the duplicate array
and make this one on .json
Someone know how can i do this ? And if i need new method to make this one, it's not a problem.
Thank you.
You can read file line by line and convert each line to a 2-elements array with key and value. For example, first line can be converted to:
['kokoa#0000','826']
As you can se 0 index is a key and 1 index is a value. When you convert all lines to this kind of array you can reduce it to an object and convert it to JSON.
See example method:
function convertLinesToJson(lines) {
const dpaText = "Discord Profile Avatar";
const jsonObject = lines
//only with 'Discord ...' text
.filter(line => line.includes(dpaText))
//get what is after 'Discord ...' text and split into 2-elements array
.map(line => {
const index = line.indexOf(dpaText) + dpaText.length;
const keyValue = line.substr(index).trim().split(/[ ]{4}/);
return keyValue;
})
// convert [key, value] array into an object
.reduce((accumulator, currentValue) => {
accumulator[currentValue[0]] = currentValue[1];
return accumulator;
}, {});
//convert object to JSON
return JSON.stringify(jsonObject);
}
Usage:
let lines = [
"1 kokoa#0368's Discord Profile Avatar kokoa#0000 826",
"2 Azzky 陈东晓#7475's Discord Profile Avatar Azzky 陈东晓#0000 703",
"3 StarKleey 帅哥#6163's Discord Profile Avatar StarKleey 帅哥#0000 640"
];
const json = convertLinesToJson(lines);
console.log(json);
Above example code prints:
{"kokoa#0000":"826","Azzky 陈东晓#0000":"703","StarKleey 帅哥#0000":"640"}
See also:
Read a file one line at a time in node.js?
Related
I am trying to encode a text to Base64 and using NodeJS, and then while getting the data I am decoding it back from Base64. Now I need to change the data into JSON so I could fill the relevant fields but its not converting to JSON.
Here is my code:
fs.readFile('./setting.txt', 'utf8', function(err, data) {
if (err) throw err;
var encodedData = base64.encode(data);
var decoded = base64.decode(encodedData).replace(/\n/g, '').replace(/\s/g, " ");
return res.status(200).send(decoded);
});
In setting.txt I have the following text:
LENGTH=1076
CRC16=28653
OFFSET=37
MEASUREMENT_SAMPLING_RATE=4
MEASUREMENT_RANGE=1
MEASUREMENT_TRACE_LENGTH=16384
MEASUREMENT_PRETRIGGER_LENGTH=0
MEASUREMENT_UNIT=2
MEASUREMENT_OFFSET_REMOVER=1
This decodes the result properly but when I use JSON.parse(JSON.stringify(decoded ) its not converting to JSON.
Can someone help me with it.
Try below snippet
let base64Json= new Buffer(JSON.stringify({}),"base64").toString('base64');
let json = new Buffer(base64Json, 'ascii').toString('ascii');
What does base-64 encoding/decoding have to do with mapping a list of tuples (key/value pairs) like this:
LENGTH=1076
CRC16=28653
OFFSET=37
MEASUREMENT_SAMPLING_RATE=4
MEASUREMENT_RANGE=1
MEASUREMENT_TRACE_LENGTH=16384
MEASUREMENT_PRETRIGGER_LENGTH=0
MEASUREMENT_UNIT=2
MEASUREMENT_OFFSET_REMOVER=1
into JSON?
If you want to "turn it (the above) into JSON", you need to:
Decide on what its JSON representation should be, then
Parse it into its component bits, convert that into an appropriate data struct, and then
use JSON.stringify() to convert it to JSON.
For instance:
function jsonify( document ) {
const tuples = document
.split( /\n|\r\n?/ )
.map( x => x.split( '=', 2) )
.map( ([k,v]) => {
const n = Number(n);
return [ k , n === NaN ? v : n ];
});
const obj = Object.fromEntries(tuples);
const json = JSON.stringify(obj);
return json;
}
im working on a Discord bot and have a reputation system with fs (npm package) and saving peoples reps in a file and doing the file name as they discord id
now im working on a top 10 command and would need some help here, i currently have this as code:
let users = [];
let reps = [];
fs.readdirSync('./data/reps/').forEach(obj => {
users.push(obj.replace('.json', ''))
let file = fs.readFileSync(`./data/reps/${obj}`)
let data = JSON.parse(file)
reps.push(data.reps)
})
let top = [...users, ...reps]
top.sort((a,b) => {a - b})
console.log(top)
the files form the users are like this:
{
"users": [
"437762415275278337"
],
"reps": 1
}
users are the current users that can't rep the persion anymore and don't need to use it in the command
i wan to get the top 10 of reps so that i can get the user id and how many reps they have, how could i do it with the code above?
You could try this
const topTen = fs.readdirSync('./data/reps/').map(obj => {
const file = fs.readFileSync(`./data/reps/${obj}`);
const data = JSON.parse(file);
return { ...data, name: obj.replace('.json', '') };
}).sort((a, b) => a.reps - b.reps).slice(0, 10);
console.log(topTen);
I would change how you push the data
const users = [];
fs.readdirSync('./data/reps/').forEach(obj => {
let file = fs.readFileSync(`./data/reps/${obj}`)
let data = JSON.parse(file)
reps.push({ reps: data.reps, id: obj.replace(".json", "") });
})
That way when you sort the array the id goes along with
//define this after the fs.readdirSync.forEach method
const top = users.sort((a,b)=> a.reps-b.reps).slice(0,10);
If you want an array of top ids
const topIds = top.map(e => e.id);
If you want a quick string of it:
const str = top.map(e => `${e.id}: ${e.reps}`).join("\n");
Also you should probably just have one or two json files, one would be the array of user id's and their reps and then the other could be of user id's and who they can't rep anymore
I have a csv sheet where some of the cells could have either single record or multiple records like an array. A sample cell would look like either ["London"] or ["Cairo", "Montreal", "Paris"] - with the parenthesis included. I'm using sever.js to show these records in a visualization on a webpage.
Everything's fine except the parenthesis and quotes are being shown in the visualization. How do I write a rule/logic in nodeJS such that both parenthesis and quotes are not shown in the final visualization?
There's no point in cleansing the data in the csv file itself because I'm gonna have to use this code for hundreds of csv files, so it's better to write a rule in nodeJS.
The following's the server.js code I've used so far:
/**
* API for getting country Names
*/
app.get('/get-country-names.json', (req, res) => {
william.william().then(response => {
res.json(response);
}).catch(message => res.send("Failed to read file" + message));
});
/**
* API to read file and return response in json
* Takes country id as parameter
*/
app.post('/get-city-sources.json', (req, res) => {
let countryName = req.body.countryName;
let city = req.body.city;
william.william().then(sources => {
let filteredSources = [{ name: city, parent: null, color: 'red' }];
Array.from(new Set(sources.filter(source => source.countryName === countryName && source.city === city).map(src => src.source)))
.forEach(source => {
let sourceArr = source.split(',');
console.log(sourceArr);
sourceArr.forEach(src => filteredSources.push({ name: src, parent: city, color: 'blue' }));
});
res.json(filteredSources);
}).catch(message => res.send("Failed to read file" + message));
});
app.listen(port, () => console.log(`Example app listening at http://localhost:${port}`))
Regular expression should do the trick. I'm not exactly sure where the stripping should happen within your code but this is a sample function that uses regular expression to strip out square brackets and quotes:
const formatCellString = (cell = '') => {
return cell
// strip out the square brackets
.replace(/^\[(.*?)\]$/, "$1")
// strip out the quotes
.replace(/("\b)|(\b")/g, '')
}
formatCellString('["Cairo", "Montreal", "Paris"]'); => // Cairo, Montreal, Paris
formatCellString('["London"]'); => // London
When I generate the CSV file each "item" output comes out with a  symbol. How would I go about removing this with my code. I tried to change it to utf-8 because I read that might be what's causing it. Any ideas? Example:
const products = await page.$$('.item-row');
Promise.all(products.map(async product => {
// Inside of each product find product SKU, it's human-readable name, and it's price
let productId = await product.$eval(".custom-body-copy", el => el.innerText.trim().replace(/,/g,' -').replace('Item ', ''));
let productName = await product.$eval(".body-copy-link", el => el.innerText.trim().replace(/,/g,' -'));
let productPrice = await product.$eval(".product_desc_txt div span", el => el.innerText.trim().replace(/,/g,' -'));
// Format them as a csv line
return productId + ',' + productName + ',' + productPrice + ',';
})).then(lines => {
// Write the lines to a file
fs.writeFileSync("products.csv", lines.join('\n'), 'utf-8');
browser.close();
});
});
There are probably better solutions, but the first thing that comes to mind to to change the string to an array with split() and then .map through and test for the à ascii code and change back to a string with join() like this:
const strToChange = 'My string with an à char';
const charWeDoNotWant = 'Ã'.charCodeAt();
const toFixArr = strToChange.split('');
fixedArr = toFixArr.map(char =>
char.charCodeAt() === charWeDoNotWant ? '' : char
);
const fixedStr = fixedArr.join('');
console.log(`String without Ã: ${fixedStr}`);
When a user send an image via Telegram bot it there any way to get the image URL? or I just need to save the image somewhere?
In the message array you receive you can find the key photo.
There you will find multiple arrays with the following format
"file_id" : "XXXX",
"file_size" : 1107,
"width" : 90,
"height" : 51
From one of those array you need to take the file_id. You can then request the file_path with a simple get get on the url https://api.telegram.org/bot<token>/getFile?file_id=<file_id>
You will receive an array that looks as following
"ok" : true,
"result" : {
"file_id" : "XXXX",
"file_size" : 27935,
"file_path" : "photo\/file_1.jpg"
}
From the result you need the file_path and you then got the image location https://api.telegram.org/file/bot<token>/<file_path>
This is a three steps process
First when the user send an image, your bot get a JSON structure like this:
Array
(
[update_id] => 820488009
[message] => Array
(
[message_id] => 11338
[from] => Array
(
[id] => xxxxxx
[is_bot] =>
[first_name] => ANSB
[language_code] => fr
)
[chat] => Array
(
[id] => 333333333
[first_name] => ANSB
[type] => private
)
[date] => 1606316785
[photo] => Array
(
[0] => Array
(
[file_id] => AgACAgEAAxkBAAIsSl--cvE_bez8g1Kzbk6LsR4JZOJWAALxqDEbw8TxRQpbG7Np1dvbARV2ShcAAwEAAwIAA20AA6SRAAIeBA
[file_unique_id] => AQADARV2ShcAA6SRAAI
[file_size] => 34888
[width] => 320
[height] => 240
)
[1] => Array
(
[file_id] => AgACAgEAAxkBAAIsSl--cvE_bez8g1Kzbk6LsR4JZOJWAALxqDEbw8TxRQpbG7Np1dvbARV2ShcAAwEAAwIAA3gAA6WRAAIeBA
[file_unique_id] => AQADARV2ShcAA6WRAAI
[file_size] => 204583
[width] => 800
[height] => 600
)
[2] => Array
(
[file_id] => AgACAgEAAxkBAAIsSl--cvE_bez8g1Kzbk6LsR4JZOJWAALxqDEbw8TxRQpbG7Np1dvbARV2ShcAAwEAAwIAA3kAA6KRAAIeBA
[file_unique_id] => AQADARV2ShcAA6KRAAI
[file_size] => 372915
[width] => 1280
[height] => 960
)
)
)
)
As you can see, Telegram create low resolution of the image. If the original image is small, you can have only the original. If it's medium you'll get two. Here yu can see I have 3 images (the original is the big one 1280*960).
So you have to check the size of the array of image wih (eg i PHP)
$nbr_image = count($jsondata['message']['photo']);
in order to read the file_id of your choice so the oe of the smallest, the biggest etc... Take care, the id is NOT the file_unique_id but the big one so the file_id.
Notice if the user send in one time more than one picture, you will get one message for each picture. So each message is about ONE picture, in multiple resolutions.
Once your bot has the file_id, you must call Telegram sending the file_id. The call is a basic one with:
https://api.telegram.org/bot<token_of_your_bot>/getFile?file_id=<file_id of the picture>
You get back a JSON with:
{"ok":true,
"result":
{"file_id":"AgACAgEAAxkBAAIsUV--hZoxZ5_ctnfbVa0zFWjRtMYUAALyqDEbw8TxRdkTI6iDNvHUmKQSMAAEAQADAgADeAADsx8EAAEeBA",
"file_unique_id":"AQADmKQSMAAEsx8EAAE",
"file_size":41597,
"file_path":"photos/file_0.jpg"
}
So a copy of the file_id, the weight (notice you don't get back pixel size!) and the path.
After that, just make a call using the path like this:
https://api.telegram.org/file/bot<token_of_your_bot>/<file_path from the JSON>
and you'll get the picture
One point: each time I get a JSON with the picture in more than one resolution, the big one is the last. But I've find nothing in the doc about that fact. So I'm note sure you can't have the big one in index [0]...
I know I'm too late, but I was researching for it too long. Here is the answer:
const TelegramBot = require('node-telegram-bot-api');
let bot = new TelegramBot(token, {polling: true});
And the function to download it requires file id, which comes within the message
var file_id = (msg.photo[msg.photo.length-1].file_id);
var downloadDir = './images';
let something = ''
var https = require('https')
bot.getFileLink(fileId).then( async (fileUri) => {
var base64Img = require('base64-img');
let time = process.hrtime();
let extension = fileUri.split('.').pop();
let newName = `${time[0]}${time[1]}.${extension}`;
let file = fs.createWriteStream(`${downloadDir}/${newName}`);
let request = await https.get(fileUri, (response) => {
response.pipe(file);
});
file.on('finish', () =>{
console.log('msg.text ='/images/'+newName')
})
//
});
};
Main function is the bot.getFileLink(fileId). Hope it will help who will read this :)