I'm trying to build a db with bluebird and sqlite3 to manage a lot of "ingredients".
So far I've managed to parse a file and extrapolate some data from it using regex.
Every time a line matches the regex I want to search in the database if an element with the same name has already been inserted, if so the element is skipped, otherwise it must be inserted.
The problem is that some elements get inserted more than one time.
The code partially works, and I'm saying it partially works because if I remove the line of code where I check the existence of an element with the same name, the duplicate rows are much more.
Here's the piece of code:
lineReader.eachLine(FILE_NAME, (line, last, cb) => {
let match = regex.exec(line);
if (match != null) {
let matchedName = match[1].trim();
//This function return a Promise for all the rows with corresponding name
ingredientsRepo.getByName(matchedName)
.then((entries) => {
if (entries.length > 0) {
console.log("ALREADY INSERTED INGREDIENT: " + matchedName)
} else {
console.log("ADDING " + matchedName)
ingredientsRepo.create(matchedName)
}
})
}
});
I know I'm missing something about Promises but I can't understand what I'm doing wrong.
Here's the code of Both getByName(name) and create(name):
create(name) {
return this.dao.run(
`INSERT INTO ingredients (name) VALUES (?)`,
[name]
)
}
getByName(name) {
return this.dao.all(
'SELECT * FROM ingredients WHERE name == ?',
[name]
)
}
ingredientsRepo.getByName(matchedName) returns a promise, this means it's asynchronous. I am also guessing that ingredientsRepo.create(matchedName) is asynchronous, because you are inserting something into a DB.
Your loop doesn't wait for these async functions to complete. Hence the loop could already be on the 50th iteration, when the .then(...) is called for the first iteration.
So let's say the first 10 elements have the same name. Well, since the asynchronous functions take some time to complete, the name will not be inserted into the DB until say maybe the 20th iteration of the loop. So for the first 10 elements, the name does not exist within the DB.
It's a timing issue.
Related
so i have the following code
data.push(commands.map(
command => {
if (!command.devOnly) { return command.name; } // first condition
if (command.devOnly && message.author.id != '3251268789058714880') {} // second condition
},
).join('\n'));
if the second condition is true it returns null but then when I run console.log(data) it has a blank line for all the commands where the second condition is true.
Is there a way to stop the second condition from returning anything, and not leaving the blank line
.map() is a 1-for-1 transformation so the output array will have EXACTLY the same number of elements in it as the input array. If you don't return anything, that element in the array will have an undefined value (which is the return value when you don't actively return something).
To transform the array and eliminate some elements, you cannot use .map(). You can either do .filter().map() where you first filter out the items you don't want and then map the others or you can use a regular for loop and just push the items into output array that you want to keep using either a regular for loop iteration or a .reduce() or .forEach() iteration.
One example:
const results = commands.filter(command => !command.devOnly).map(command => command.name);
console.log(results);
const results = [];
for (let command of commands) {
if (!command.devOnly) results.push(command.name);
}
console.log(results);
Note, your second condition doesn't do anything at all in your code example so I wasn't sure how to account for that in these examples.
P.S. I've often wished Javascript had a .filterMap() feature that let you return undefined to leave that value out of the result - otherwise work like .map(). But, it doesn't have that feature built in. You could build your own.
Per your comments, you can filter on two conditions like this:
const results = commands
.filter(command => !command.devOnly || message.author.id === '3251268789058714880')
.map(command => command.name);
console.log(results);
you could use Array.reduce
commands.reduce((prev,command)=>{
if (!command.devOnly)
return [...prev,command.name]
// this if, is useless but shows that you can use more conditions.
if (command.devOnly && message.author.id != '3251268789058714880')
return prev
// add more conditions as you need here
return prev
},[])
another options could be by doing map then filter the undefined values, or use for-loop
like jfriend00 explained in his answer. the filterMap() he wished could be implemented with Array.reduce
I have a JSON file of 2 discord client IDs `{
{
"premium": [
"a random string of numbers that is a client id",
"a random string of numbers that is a client id"
]
}
I have tried to access these client IDs to do things in the program using a for loop + if statement:
for(i in premium.premium){
if(premium.premium[i] === msg.author.id){
//do some stuff
}else{
//do some stuff
When the program is ran, it runs the for loop and goes to the else first and runs the code in there (not supposed to happen), then runs the code in the if twice. But there are only 2 client IDs and the for loop has ran 3 times, and the first time it runs it goes instantly to the else even though the person who sent the message has their client ID in the JSON file.
How can I fix this? Any help is greatly appreciated.
You may want to add a return statement within your for loop. Otherwise, the loop will continue running until a condition has been met, or it has nothing else to loop over. See the documentation on for loops here.
For example, here it is without return statements:
const json = {
"premium": [
"aaa-1",
"bbb-1"
]
}
for (i in json.premium) {
if (json.premium[i] === "aaa-1") {
console.log("this is aaa-1!!!!")
} else {
console.log("this is not what you're looking for-1...")
}
}
And here it is with return statements:
const json = {
"premium": [
"aaa-2",
"bbb-2"
]
}
function loopOverJson() {
for (i in json.premium) {
if (json.premium[i] === "aaa-2") {
console.log("this is aaa-2!!!!")
return
} else {
console.log("this is not what you're looking for-2...")
return
}
}
}
loopOverJson()
Note: without wrapping the above in a function, the console will show: "Syntax Error: Illegal return statement."
for(i in premium.premium){
if(premium.premium[i] === msg.author.id){
//do some stuff
} else{
//do some stuff
}
}
1) It will loop through all your premium.premium entries. If there are 3 entries it will execute three times. You could use a break statement if you want to exit the loop once a match is found.
2) You should check the type of your msg.author.id. Since you are using the strict comparison operator === it will evaluate to false if your msg.author.id is an integer since you are comparing to a string (based on your provided json).
Use implicit casting: if (premium.premium[i] == msg.author.id)
Use explicit casting: if (premium.premium[i] === String(msg.author.id))
The really fun and easy way to solve problems like this is to use the built-in Array methods like map, reduce or filter. Then you don't have to worry about your iterator values.
eg.
const doSomethingAuthorRelated = (el) => console.log(el, 'whoohoo!');
const authors = premiums
.filter((el) => el === msg.author.id)
.map(doSomethingAuthorRelated);
As John Lonowski points out in the comment link, using for ... in for JavaScript arrays is not reliable, because its designed to iterate over Object properties, so you can't be really sure what its iterating on, unless you've clearly defined the data and are working in an environment where you know no other library has mucked with the Array object.
I have two queries:
a) select id from ingredietns where name = my_param;
b) select word_id from synonyms where name = my_param;
Both return 0 or 1 row. I can also add limit 1 if needed (or in knex first()).
I can translate each into knex like this:
knex("ingredients").select('id').where('name', my_param) //do we need first()?
knex("synonyms").select('word_id').where('name', my_param) //do we need first()?
I need function called "ingredientGetOrCreate(my_param)". This function would
a) check if any of above queries return result
b) if any of these return, then return ingredients.id or synonyms.word_id - only one could be returned
c) if record doesn't eixst in any of tables, I need to do knex inesrt aand return newly added id from function
d) later I am not sure I also understand how to call this newly create function.
Function ingredientGetOrCreate would be used later as seperate function or in the following scenario (like "loop") that doesn't work for me either:
knex("products") // for each product
.select("id", "name")
.map(function (row) {
var descriptionSplitByCommas = row.desc.split(",");
Promise.all(descriptionSplitByCommas
.map(function (my_param) {
// here it comes - call method for each param and do insert
ingredientGetOrCreate(my_param)
.then(function (id_of_ingredient) {
knex('ingredients_products').insert({ id_of_ingredient });
});
...
I am stuck with knex and Promise queries because of asynchronouse part. Any clues, please?
I though I can somehow use Promise.all or Promise.some to call both queries.
P.S. This is my first day with nodejs, Promise and knex.
As far as I decode your question, it consists of two parts:
(1) You need to implement upsert logic (get-or-create logic).
(2) Your get part requires to query not a single table, but a pair of tables in specific order. Table names imply that this is some sort of aliasing engine inside of your application.
Let's start with (2). This could definitely be solved with two queries, just like you sense it.
function pick_name (rows)
{
if (! rows.length) return null
return rows[0].name
}
// you can sequence queries
function ingredient_get (name)
{
return knex('ingredients')
.select('id').where('name', name)
.then(pick_name)
.then(name =>
{
if (name) return name
return knex('synonyms')
.select('word_id').where('name', name)
.then(pick_name)
})
}
// or run em parallel
function ingredient_get (name)
{
var q_ingredients = knex('ingredients')
.select('id').where('name', name)
.then(pick_name)
var q_synonyms = knex('synonyms')
.select('word_id').where('name', name)
.then(pick_name)
return Promise.all([ q_ingredients, q_synonyms ])
.then(([name1, name2]) =>
{
return name1 || name2
})
}
Important notions here:
Both forms works well and return first occurence or JS' null.
First form optimizes count of queries to DB.
Second form optimizes answer time.
However, you can go deeper and use more SQL. There's a special tool for such task called COALESCE. You can consult your SQL documentation, here's COLASCE of PostgreSQL 9. The main idea of COALESCE is to return first non-NULL argument or NULL otherwise. So, you can leverage this to optimize both queries and answer time.
function ingredient_get (name)
{
// preparing but not executing both queries
var q_ingredients = knex('ingredients')
.select('id').where('name', name)
var q_synonyms = knex('synonyms')
.select('word_id').where('name', name)
// put them in COALESCE
return knex.raw('SELECT COALESCE(?, ?) AS name', [ q_ingredients, q_synonyms ])
.then(pick_name)
This solution guarantees single query and furthermore DB engine can optimize execution in any way it sees appropriate.
Now let's solve (1): We now got ingredient_get(name) which returns Promise<string | null>. We can use its output to activate create logic or return our value.
function ingredient_get_or_create (name, data)
{
return ingredient_get(name)
.then(name =>
{
if (name) return name
// …do your insert logic here
return knex('ingredients').insert({ name, ...data })
// guarantee homohenic output across get/create calls:
.then(() => name)
})
}
Now ingredient_get_or_create do your desired upsert logic.
UPD1:
We already got ingredient_get_or_create which returns Promise<name> in any scenario (both get or create).
a) If you need to do any specific logic after that you can just use then:
ingredient_get_or_create(…)
.then(() => knex('another_table').insert(…))
.then(/* another logic after all */)
In promise language that means «do that action (then) if previous was OK (ingredient_get_or_create)». In most of the cases that is what you need.
b) To implement for-loop in promises you got multiple different idioms:
// use some form of parallelism
var qs = [ 'name1', 'name2', 'name3' ]
.map(name =>
{
return ingredient_get_or_create(name, data)
})
var q = Promise.all(qs)
Please, note, that this is an agressive parallelism and you'll get maximum of parallel queries as your input array provides.
If it's not desired, you need to limit parallelism or even run tasks sequentially. Bluebird's Promise.map is a way to run map which analogous to example above but with concurrency option available. Consider the docs for details.
There's also Bluebird's Promise.mapSeries which conceptually is an analogue to for-loop but with promises. It's like map which runs sequentially. Look the docs for details.
Promise.mapSeries([ 'name1', 'name2', 'name3' ],
(name) => ingredient_get_or_create(name, data))
.then(/* logic after all mapSeries are OK */)
I believe the last is what you need.
I have an ajax call in my add in which it should create or update the table in excel. If table is already exists, it should remove the rows and add the new results.
When deleting the rows in loop, it is deleting some rows and then I am getting following error:
Debug info: {"code":"InvalidArgument","message":"The argument is invalid or missing or has an incorrect format.","errorLocation":"TableRowCollection.getItemAt"}
My ajax call in my excel web add-in looks like this:
$.ajax({
//....
}).done(function (data) {
Excel.run(function (ctx) {
var odataTable = ctx.workbook.tables.getItemOrNullObject("odataTable");
//rows items are not available at this point, that is why we need to load them and sync the context
odataTable.rows.load();
return ctx.sync().then(function () {
if (odataTable.rows.items == null) {
odataTable.delete();
odataTable = ctx.workbook.tables.add('B2:G2', true);
odataTable.name = "odataTable";
} else {
console.log("Rows items:" + odataTable.rows.items.length);
odataTable.rows.items.forEach(function (item) {
console.log("Removing row item: " + item.values);
item.delete();
});
console.log("rows cleaned");
}
}).then(function () {
//add rows to the table
});
}).then(ctx.sync);
}).catch(errorHandler);
}).fail(function (status) {
showNotification('Error', 'Could not communicate with the server. ' + JSON.stringify(status));
}).always(function () {
$('#refresh-button').prop('disabled', false);
});
The idea of the iterable collections is that they consist of different items. Once you remove something from these items in a not appropriate way, the collection stops being a collection. This is because they are implemented as a linked list, in which every unit knows only the next unit. https://en.wikipedia.org/wiki/Linked_list
In your case, you are deleting wtih a for-each loop. After the first deletion, the collection is broken. Thus, you need another approach.
Another approach:
Start looping with a normal for loop. Reversed.
E.g.:
for i = TotalRows to 1 i--
if row(i) something then delete
This has already been answered but I recently solved this issue myself and came here to see if anyone had posted a question about it.
When you delete a row, Excel will reorder the row index for each row: i.e. when you delete row 1, row 2 becomes row 1 and all other rows get shifted down 1 index. Because these deletions are pushed to a batch to be completed, when the second deletion is executed, your second row has become row one, so it actually skips the second row and executes what you think is your third row.
If you start from the last row and work backwards, this reordering doesn't occur and neither does the error.
For completeness sake the above example would become:
return ctx.sync().then(function () {
if (odataTable.rows.items == null) {
odataTable.delete();
odataTable = ctx.workbook.tables.add('B2:G2', true);
odataTable.name = "odataTable";
} else {
console.log("Rows items:" + odataTable.rows.items.length);
for (let i = odataTable.rows.items.length -1; i >= 0; i--) // reverse loop
odataTable.rows.items[i].delete();
}
console.log("rows cleaned");
}
I'm trying to add a paragraph at the end of the document and escape the possibility of the newly added paragraph to be added inside a list (if the document is ending with a list).
I have the following code:
let paragraph = paragraphs.items[paragraphs.items.length - 1]
let p = paragraph.insertParagraph('', window.Word.InsertLocation.after)
if (paragraph.listItemOrNullObject) {
p.detachFromList()
p.leftIndent = 0
}
The following happens: if there is a ListItem, the code works. If not, it breaks inside the if condition, like I wrote paragraph.listItem.
Shouldn't this be used like this?
EDIT - error thrown:
name:"OfficeExtension.Error"
code:"GeneralException"
message:"GeneralException"
traceMessages:[] 0 items
innerError:null
▶debugInfo:{} 4 keys
code:"GeneralException"
message:"GeneralException"
toString:function (){return JSON.stringify(this)}
errorLocation:"Paragraph.detachFromList"
the issue here is that the *.isNullObject methods/properties does not return a regular js 'null' object, but a NullObject (a special framework type of null).
check out this code i rewrote it i think in a more efficient way. excuse my js, you can port it to ts.
hope this helps.
Word.run(function (context) {
var listI = context.document.body.paragraphs.getLast().listItemOrNullObject;
context.load(listI);
return context.sync()
.then(function () {
if (listI.isNullObject) { // check out how i am validating if its null.
console.log("there is no list at the end")
}
else {
context.document.body.paragraphs.getLast().detachFromList();
context.document.body.paragraphs.getLast().leftIndent = 0;
return context.sync();
}
})
})
listItemOrNullObject will return a null object if it isn't a ListItem. Conceptually you're if is asking "if this is a list item or it isn't a list item" which effectively will also return true.
It is failing here you are attempting to detach from a non-existent list. I would take a look at isListItem. This will tell you specifically if the paragraph is a ListItem so you only execute p.detachFromList() when in fact it is part of a list.