I would like to know if there will be a way to transform a csv to the JSON format suitable for the Tabulator library?
The idea would be to have a format as seen on excel :
- the first cell on the top left, empty
- columns A, B, C... AA, AB... according to the number of cells on the longest row
- the line number automatically on the first cell of each line)
I had the idea of doing it directly with loops, but it takes a lot of time I find. I don't see any other way.
Thank you for the help.
Check the following function, I hope this is what you are looking for...
let csvfile = 'title1,title2,title3,title4\n1,2,3,4\n11,22,33,44' //YOUR CSV FILE
let capLetters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' // ALPHABET SET
let finalJson = [];
let headers;
let line =[];
convertCSV2JSON(csvfile)
function convertCSV2JSON(csv) {
line = csv.split("\n"); //PARSE ALL AVAILABLE LINES INTO ARRAY
result = [];
headers = line[0].split(","); //PARSE ALL AVAILABLE STRING NAMES INTO ARRAY AND KEEP ONLY THE FIRST ONE (HEADER)
line.slice(1).forEach(function(item,i){ //RUN EACH ITEM EXCLUDING COLUMN NAMES
var obj = {};
if(line[i] === null || line[i] === undefined) {
}else{
var entries = line[i+1].split(","); // SEPARATE FOUND ENTRIES EXCLUDING COLUMN NAMES (i+1)
for(var j = 0; j < entries.length; j++) { // PARSE ENTRIES
obj[convert2Letters(j)] = entries[j]; // ASSIGN A LETTER AS COLUMN NAME
}
}
finalJson.push(obj);
})
console.log(finalJson);
}
function convert2Letters(iteration) {
let readyLetter = ''
while (iteration >= 0) {
readyLetter += capLetters[iteration % 26]
iteration = Math.floor(iteration / 26) - 1
}
return readyLetter
}
The fuzzy part was at foreach() function, because you cannot initiate index at your preference... slice() did the trick!
Moreover convert2Letters() function takes an array of letters and on each iteration finds the modulus of 26 letters, removing by one shows you the next combination...
Example:
If you have 30 columns it will give 30 % 26 = 4
4 corresponds to capLetters[4] which is 'E'
calculate next: iteration = Math.floor(iteration / 26) - 1 which means on every 26 increment (0,26,52,78...) it will give (-1,0,1,2...) corresponding. So a 30 columns iteration will have 0 as result which corresponds to capLetters[0] = 'A'
Resulting: 30 columns will give letters 'EA'
Related
I'm trying to filter the last column on a worksheet but I can't seem to get the Index of the column. To be clear, I need the index relative to the worksheet, no the range. I used VisibleView to find the Column, but there may be hidden rows, so my plan is to then load that column via getRangeByIndexes but I need the relative columnIndex to the worksheet.
I've tried a bunch of variations of the below, but I either get Object doesn't support 'getColumn' or columnIndex is undefined
Note: In the below example I've hardcoded 7 as that will be the last column relative to the VisibleView (Columns and rows are already hidden), but I'd like this to by dynamic for other functions and just returnthe "last visible column index".
var ws = context.workbook.worksheets.getActiveWorksheet()
var visible_rng = ws.getUsedRange(true).getVisibleView()
visible_rng.load(["columnCount", "columnIndex"])
await context.sync();
console.log('visible_rng.columnIndex')
console.log(visible_rng.getCell(0,7).columnIndex)
console.log(visible_rng.getColumn(7).columnIndex)
Well this method seems a bit hacky, please share if you know a better way! But, first thing I found was that getVisibleView only metions rows in the Description.
Represents the visible rows of the current range.
I decided to try getSpecialCells and was able to load the address property. I then had to use split and get the last column LETTER and convert this to the Index.
I also wanted the columnCount but this wasn't working w/ getSpecialCells so I polled that from getVisibleView and return an Object relating to Visible Views that I can build on the function later if I need more details.
Here it is:
async function Get_Visible_View_Details_Obj(context, ws) {
var visible_rng = ws.getUsedRange(true).getSpecialCells("Visible");
visible_rng.load("address")
var visible_view_rng = ws.getUsedRange(true).getVisibleView()
visible_view_rng.load("columnCount")
await context.sync();
var Filter_Col_Index = visible_rng.address
var Filter_Col_Index = Filter_Col_Index.split(",")
var Filter_Col_Index = Filter_Col_Index[Filter_Col_Index.length - 1]
var Filter_Col_Index = Filter_Col_Index.split("!")[1]
if (Filter_Col_Index.includes(":") == true) {
var Filter_Col_Index = Filter_Col_Index.split(":")[1]
}
var Filter_Col_Index = Get_Alpha_FromString(Filter_Col_Index)
var Filter_Col_Index = Get_Col_Index_From_Letters(Filter_Col_Index)
var Filter_Col_Index_Obj = {
"last_col_ws_index": Filter_Col_Index,
"columnCount": visible_view_rng.columnCount,
}
return Filter_Col_Index_Obj
}
Helper Funcs:
function Get_Alpha_FromString(str) {
return str.replace(/[^a-z]/gi, '');
}
function Get_Col_Index_From_Letters(str) {
str = str.toUpperCase();
let out = 0, len = str.length;
for (pos = 0; pos < len; pos++) {
out += (str.charCodeAt(pos) - 64) * Math.pow(26, len - pos - 1);
}
return out - 1;
}
I want to change each word that matches the synonym list randomly by another synonym or itself (to randomly keep this keyword).
I test if a string (input) contains one element of an array (words). If it's true, I want to randomly replace this with the element of this same list.
var input = "This is an amazing text blob where this word amazing is replaced by a random word from list_of_words. Isn't this amazing!";
words_synonym = ["amazing", "formidable", "great", "smart"];
// first condition --> true if "input" contain one element of "words_synonym"
input = input.toLowerCase();
console.log(words_synonym.some(word => input.includes(word)));
after, I want to replace the "element" that validated the condition with a random element of the same array (words_synonym).
But I can't select this element. I have just true or false
var random_word = words_synonym[Math.floor(Math.random() * (words_synonym.length))]
input = input.replace(element, random_word, 0)
thanks
The way you have it right now, you're checking if any of the synonyms match any of the words (via words_synonym.some(word => input.includes(word))). In order to do what you want, you'll need both the position of the target word and the new word, neither of which you have now. To do this, you'll want to break apart your nested loops.
The code words_synonym.some(word => input.includes(word)) is equivalent to:
let has_synonym = false;
for (word of words_synonym) { // this is a loop
if (input.includes(word)) { // this is also a loop
has_synonym = true;
break;
}
}
console.log(has_synonym);
So to fix your main issue, just replace includes with indexOf.
To handle the case of replacing all of the tokens, I would suggest keeping track of the token you have replaced outside of the loop, otherwise you end up replacing each token many times which may become very expensive. To do this, just keep track of your starting position outside of the loop and increment it with the end index of the replacement word. indexOf already takes a start argument for exactly this use case!
const input = "This is an amazing text blob where this word amazing is replaced by a random word from list_of_words. Isn't this amazing!";
const words_synonym = ["amazing", "formidable", "great", "smart"];
let output = input;
let start = 0; // index of the end of the last replaced token
for (word of words_synonym) {
let index = output.indexOf(word, start);
while (index >= 0) {
const new_word = words_synonym[Math.floor(Math.random() * (words_synonym.length))];
output = output.substr(0, index) + new_word + output.substr(index + word.length, output.length);
start = index + new_word.length + 1; // increment the start
index = output.indexOf(word, start);
}
}
console.log("input: ", input);
console.log("output: ", output);
You can use method find:
words_synonym.find(word => input.includes(word))
Which returns
The value of the first element in the array that satisfies the
provided testing function. Otherwise, undefined is returned.
from docs:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/find
i have modify answer of dantiston and i have include a loop in order to change all the word match "words_synonym".
But there is a problem. The program don't check all the word of "words_synonym" but only the first with indexof.
var input = "This is an amazing text blob where this word amazing is replaced by a random word from list_of_words. Isn't this amazing!";
words_synonym = ["amazing", "formidable", "great", "smart"];
let output = input;
for (word of words_synonym) {
let index = output.indexOf(word);
if (index >= 0) {
console.log(word);
var indexes = [], i = -1;
while ((i = output.indexOf(word, i+1)) != -1){
index=output.indexOf(word, i);
var new_word = words_synonym[Math.floor(Math.random() * (words_synonym.length))];
output = output.substr(0, index) + new_word + output.substr(index + word.length, output.length);
}
}
}
console.log("input: ", input);
console.log("output: ", output);
Some background on what is going on:
We are processing addresses into standardized forms, this is the code to take addresses scored by how many components found and then rescore them using a levenshtein algorithm across similar post codes
The scores are how many components were found in that address divided by the number missed, to return a ratio
The input data, scoreDict, is a dictionary containing arrays of arrays. The first set of arrays is the scores, so there are 12 arrays because there are 12 scores in this file (it adjusts by file). There are then however many addresses fit that score in their own separate arrays stored in that. Don't ask me why I'm doing it that way, my brain is dead
The code correctly goes through each score array and each one is properly filled with the unique elements that make it up. It is not short by any amount, nothing is duplicated, I have checked
When we hit the score that is -1 (this goes to any address where it doesn't fit in some rule so we can't use its post code to find components so no components are found) the loop specifically ONLY DOES EVERY OTHER ADDRESS IN THIS SCORE ARRAY
It doesn't do this to any other score array, I have checked
I have tried changing the number to something else like 99, same issue except one LESS address got rescored, and the rest stayed at the original failing score of 99
I am going insane, can anyone find where in this loop something may be going wrong to cause it to only do every other line. The index counter of line and sc come through in the correct order and do not skip over. I have checked
I am sorry this is not professional, I have been at this one loop for 5 hours
Rescore: function Rescore(scoreDict) {
let tempInc = 0;
//Loop through all scores stored in scoreDict
for (var line in scoreDict) {
let addUpdate = "";
//Loop through each line stored by score
for (var sc in scoreDict[line.toString()]) {
console.log(scoreDict[line.toString()].length);
let possCodes = new Array();
const curLine = scoreDict[line.toString()][sc];
console.log(sc);
const curScore = curLine[1].split(',')[curLine[1].split(',').length-1];
switch (true) {
case curScore == -1:
let postCode = (new RegExp('([A-PR-UWYZ][A-HK-Y]?[0-9][A-Z0-9]?[ ]?[0-9][ABD-HJLNP-UW-Z]{2})', 'i')).exec(curLine[1].replace(/\\n/g, ','));
let areaCode;
//if (curLine.split(',')[curLine.split(',').length-2].includes("REFERENCE")) {
if ((postCode = (new RegExp('(([A-Z][A-Z]?[0-9][A-Z0-9]?(?=[ ]?[0-9][A-Z]{2}))|[0-9]{5})', 'i').exec(postCode))) !== null) {
for (const code in Object.keys(addProper)) {
leven.LoadWords(postCode[0], Object.keys(addProper)[code]);
if (leven.distance < 2) {
//Weight will have adjustment algorithms based on other factors
let weight = 1;
//Add all codes that are close to the same to a temp array
possCodes.push(postCode.input.replace(postCode[0], Object.keys(addProper)[code]).split(',')[0] + "(|W|)" + (leven.distance/weight));
}
}
let highScore = 0;
let candidates = new Array();
//Use the component script from cityprocess to rescore
for (var i=0;i<possCodes.length;i++) {
postValid.add([curLine[1].split(',').slice(0,curLine[1].split(',').length-2) + '(|S|)' + possCodes[i].split("(|W|)")[0]]);
if (postValid.addChunk[0].split('(|S|)')[postValid.addChunk[0].split('(|S|)').length-1] > highScore) {
candidates = new Array();
highScore = postValid.addChunk[0].split('(|S|)')[postValid.addChunk[0].split('(|S|)').length-1];
candidates.push(postValid.addChunk[0]);
} else if (postValid.addChunk[0].split('(|S|)')[postValid.addChunk[0].split('(|S|)').length-1] == highScore) {
candidates.push(postValid.addChunk[0]);
}
}
score.Rescore(curLine, sc, candidates[0]);
}
//} else if (curLine.split(',')[curLine.split(',').length-2].contains("AREA")) {
// leven.LoadWords();
//}
break;
case curScore > 0:
//console.log("That's a pretty good score mate");
break;
}
//console.log(line + ": " + scoreDict[line].length);
}
}
console.log(tempInc)
score.ScoreWrite(score.scoreDict);
}
The issue was that I was calling the loop on the array I was editing, so as each element got removed from the array (rescored and moved into a separate array) it got shorter by that element, resulting in an issue that when the first element was rescored and removed, and then we moved onto the second index which was now the third element, because everything shifted up by 1 index
I fixed it by having it simply enter an empty array for each removed element, so everything kept its index and the array kept its length, and then clear the empty values at a later time in the code
I'm using csv-parser to parse a csv file I have. Then as it's parsing each line, I'm checking each value in the line to get the min and max.
My issue is that my max keeps coming out as 99 in the COL and ROW columns. when their maxs are around 200 for row and 300 for col. I'm not sure whats happening there, as far as I know my method should work.
Here is my relevant code:
var mins = {}
var maxs = {}
function loadCSV(finishedCallback) {
console.log("Loading csv...");
fs.createReadStream(__dirname + '/data.csv')
.pipe(stream)
.on('data', function (data) {
for (var k in data) {
if (data.hasOwnProperty(k)) {
if (mins[k] == undefined)
mins[k] = data[k];
if (maxs[k] == undefined)
maxs[k] = data[k];
if (mins[k] > data[k])
mins[k] = data[k];
if (maxs[k] < data[k])
maxs[k] = data[k];
}
}
}).on('finish', function () {
console.log("Finished reading csv!")
finishedCallback();
console.log("Maxs: " + JSON.stringify(maxs));
console.log("Mins: " + JSON.stringify(mins));
return true
})
}
And here are the first few lines of my CSV: (I'm aware of the duplicate x and y fields, I don't end up using then)
X,Y,X,Y,ROW,COL,EMH,EMV,ASP,DEM,SLOPE
495118.8125,4469900.5,495118.80000,4469901.00000,0,79,15.56310,8.11320,285.65760,318.85077,0.12900
495120.8125,4469900.5,495120.80000,4469901.00000,0,80,15.64225,8.12110,286.47190,318.87570,0.13107
495122.8125,4469900.5,495122.80000,4469901.00000,0,81,15.76020,8.10590,287.70340,318.91686,0.13890
I have a felling I'm probably being stupid and missing something obvious :P
You are comparing strings instead of numbers:
$ node
> '99' > '300'
true
So before doing any comparisons, convert data[k] to a number first:
var value = Number(data[k]);
I'm making an air dictionary and I have a(nother) problem. The main app is ready to go and works perfectly but when I tested it I noticed that it could be better. A bit of context: the language (ancient egyptian) I'm translating from does not use punctuation so a phrase canlooklikethis. Add to that the sheer complexity of the glyph system (6000+ glyphs).
Right know my app works like this :
user choose the glyphs composing his/r word.
app transforms those glyphs to alphanumerical values (A1 - D36 - X1A, etc).
the code compares the code (say : A5AD36) to a list of xml values.
if the word is found (A5AD36 = priestess of Bast), the user gets the translation. if not, s/he gets all the possible words corresponding to the two glyphs (A5A & D36).
If the user knows the string is a word, no problem. But if s/he enters a few words, s/he'll have a few more choices than hoped (exemple : query = A1A5AD36 gets A1 - A5A - D36 - A5AD36).
What I would like to do is this:
query = A1A5AD36 //word/phrase to be translated;
varArray = [A1, A5A, D36] //variables containing the value of the glyphs.
Corresponding possible words from the xml : A1, A5A, D36, A5AD36.
Possible phrases: A1 A5A D36 / A1 A5AD36 / A1A5A D36 / A1A5AD36.
Possible phrases with only legal words: A1 A5A D36 / A1 A5AD36.
I'm not I really clear but to things simple, I'd like to get all the possible phrases containing only legal words and filter out the other ones.
(example with english : TOBREAKFAST. Legal = to break fast / to breakfast. Illegal = tobreak fast.
I've managed to get all the possible words, but not the rest. Right now, when I run my app, I have an array containing A1 - A5A - D36 - A5AD36. But I'm stuck going forward.
Does anyone have an idea ? Thank you :)
function fnSearch(e: Event): void {
var val: int = sp.length; //sp is an array filled with variables containing the code for each used glyph.
for (var i: int = 0; i < val; i++) { //repeat for every glyph use.
var X: String = ""; //variable created to compare with xml dictionary
for (var i2: int = 0; i2 < val; i2++) { // if it's the first time, use the first glyph-code, else the one after last used.
if (X == "") {
X = sp[i];
} else {
X = X + sp[i2 + i];
}
xmlresult = myXML.mot.cd; //xmlresult = alphanumerical codes corresponding to words from XMLList already imported
trad = myXML.mot.td; //same with traductions.
for (var i3: int = 0; i3 < xmlresult.length(); i3++) { //check if element X is in dictionary
var codeElement: XML = xmlresult[i3]; //variable to compare with X
var tradElement: XML = trad[i3]; //variable corresponding to codeElement
if (X == codeElement.toString()) { //if codeElement[i3] is legal, add it to array of legal words.
checkArray.push(codeElement); //checkArray is an array filled with legal words.
}
}
}
}
var iT2: int = 500 //iT2 set to unreachable value for next lines.
for (var iT: int = 0; iT < checkArray.length; iT++) { //check if the word searched by user is in the results.
if (checkArray[iT] == query) {
iT2 = iT
}
}
if (iT2 != 500) { //if complete query is found, put it on top of the array so it appears on top of the results.
var oldFirst: String = checkArray[0];
checkArray[0] = checkArray[iT2];
checkArray[iT2] = oldFirst;
}
results.visible = true; //make result list visible
loadingResults.visible = false; //loading screen
fnPossibleResults(null); //update result list.
}
I end up with an array of variables containing the glyph-codes (sp) and another with all the possible legal words (checkArray). What I don't know how to do is mix those two to make legal phrases that way :
If there was only three glyphs, I could probably find a way, but user can enter 60 glyphs max.