I got confuse, how to accomplished this task let me show code then issue
case 1
let arr1=[1,3]
let arr2=[1,2,3]
compare these 2 array if arr2 is greater then delete 2 from database
case 2
let arr1=[1,2,3]
let arr2=[1,2]
compare these 2 array if arr1 is greater, then insert 3 into database
and return promise reject like resolve can anyone tell me what is best to achieve this.
solution to your problem
Step 1: find the difference in the two arrays lets say arr1 & arr2
Step 2: check if arr2.length is greater than delete the difference from the db
Step 3: if arr1.length is greater than insert difference in db
for Step 1 implement the below "difference" function:
Array.prototype.difference = function(arr) {
return this.filter(function(i) {return arr.indexOf(i) === -1;});
};
[1,2,3,4,5,6].diff( [3,4,5] );// return [1,2,6]
// here you capture the difference among arrays
let diffArray = arr1.difference(arr2);
for step 2 & step 3:
if(arr2.length > arr1.length){
diffArray.forEach((element)=>{
// your db deletion code comes here something like.....db.delete(element);
return new Promise((resolve, reject) => {
// db deletion code
// return resolve(result)....if successfully inserted
// reject(err).........if some error occurs
})
.then(result => result)
.catch(err => err)
});
// similarly
if (arr1.length >arr2.length){
diffArray.forEach((element)=>{
// your db insertion code comes here
return new Promise((resolve, reject) => {
// db insertion code
// return resolve(result)....if successfully inserted
// reject(err).........if some error occurs
})
.then(result => result)
.catch(err => err)
});
}
}
Happy Coding :)
Related
I have a collection of teams containing around 80 000 documents. Every Monday I would like to reset the scores of every team using firebase cloud functions. This is my function:
exports.resetOrgScore = functions.runWith(runtimeOpts).pubsub.schedule("every monday 00:00").timeZone("Europe/Oslo").onRun(async (context) => {
let batch = admin.firestore().batch();
let count = 0;
let overallCount = 0;
const orgDocs = await admin.firestore().collection("teams").get();
orgDocs.forEach(async(doc) => {
batch.update(doc.ref, {score:0.0});
if (++count >= 500 || ++overallCount >= orgDocs.docs.length) {
await batch.commit();
batch = admin.firestore().batch();
count = 0;
}
});
});
I tried running the function in a smaller collection of 10 documents and it's working fine, but when running the function in the "teams" collection it returns "Cannot modify a WriteBatch that has been committed". I tried returning the promise like this(code below) but that doesn't fix the problem. Thanks in advance :)
return await batch.commit().then(function () {
batch = admin.firestore().batch();
count = 0;
return null;
});
There are three problems in your code:
You use async/await with forEach() which is not recommended: The problem is that the callback passed to forEach() is not being awaited, see more explanations here or here.
As detailed in the error you "Cannot modify a WriteBatch that has been committed". With await batch.commit(); batch = admin.firestore().batch(); it's exactly what you are doing.
As important, you don't return the promise returned by the asynchronous methods. See here for more details.
You'll find in the doc (see Node.js tab) a code which allows to delete, by recursively using a batch, all the docs of a collection. It's easy to adapt it to update the docs, as follows. Note that we use a dateUpdated flag to select the docs for each new batch: with the original code, the docs were deleted so no need for a flag...
const runtimeOpts = {
timeoutSeconds: 540,
memory: '1GB',
};
exports.resetOrgScore = functions
.runWith(runtimeOpts)
.pubsub
.schedule("every monday 00:00")
.timeZone("Europe/Oslo")
.onRun((context) => {
return new Promise((resolve, reject) => {
deleteQueryBatch(resolve).catch(reject);
});
});
async function deleteQueryBatch(resolve) {
const db = admin.firestore();
const snapshot = await db
.collection('teams')
.where('dateUpdated', '==', "20210302")
.orderBy('__name__')
.limit(499)
.get();
const batchSize = snapshot.size;
if (batchSize === 0) {
// When there are no documents left, we are done
resolve();
return;
}
// Delete documents in a batch
const batch = db.batch();
snapshot.docs.forEach((doc) => {
batch.update(doc.ref, { score:0.0, dateUpdated: "20210303" });
});
await batch.commit();
// Recurse on the next process tick, to avoid
// exploding the stack.
process.nextTick(() => {
deleteQueryBatch(resolve);
});
}
Note that the above Cloud Function is configured with the maximum value for the time out, i.e. 9 minutes.
If it appears that all your docs cannot be updated within 9 minutes, you will need to find another approach, for example using the Admin SDK from one of your server, or cutting the work into pieces and run the CF several times.
This below code is written in nodejs with mongodb. I am trying to print temp_child_array value in console I am getting blank array. Can any one tell me how to pass fetch data to parent.
let pr = new Promise(function(resolve, reject) {
MasterCodesModel
.getData(key)
.then(data => {
let temp_child_array = [];
let process_child = (key) => {
MasterCodesModel
.getData(key)
.then(data => {
temp_child_array.push(data);
})
}
process_child(key)
console.log("TEMP_CHILD_ARRAY: ", temp_child_array)// Its Empty
});
});
Your inner process_child function has an asynchronous mongoDB task (MasterCodesModel.getData(key).then(...)) which is not done (either rejected or resolved) by the time you're calling console.log. So the result is not there yet and temp_child_array would have its initial value: [].
You should write your process_child function as a Promise which when it's resolved, you have your temp_child_array available with your values.
I'm trying to create a test, to verify that I've put an item in a dynamoDB table. In order to do so, right after I make a call that should put an Item (vehicle) in the database, I am trying to get the vehicle from the DB.
In my test I want to have a maximum number of retries (5). I want this while loop to be block the thread until my query has resolved to give a vehicle, or tried 5 times. Inside my test I have:
let count = 0
let car
while (!car || count < 5) {
setTimeout(async () => {
car = await findVehicle(greenCar.vehicleInfo)
}, 3000)
count++
}
And findVehicle is an asynchronous function that does a get from the dynamoDB table
If you want to wait on each iteration you can do this:
let count = 0;
let car;
while (!car || count < 5) {
await new Promise((resolve) =>
setTimeout(async () => {
car = await findVehicle(greenCar.vehicleInfo);
resolve();
}, 3000));
count++
}
So you are resolving the promise you are awaiting after you get your data. Also your function must be async in order to use await. Hope this helps.
I'm new to Node and am having problems reading from Oracle.
I have the basic examples all set up and can issue basic queries, and process the results etc..
The problem I'm having is that I need to;
Execute one query (Q1)
For each item in the results of Q1 I need to execute a second query (Q2)
I need to combine the results of Q1 and Q2s into an array to return as a promise
I am struggling to find an example where I can perform #2 - call the same query multiple times for each item returned from Q1, using the same connection which was used for Q1.
My code is below - I first perform a read, then iterate through the results storing connection.execute objects which I then run via the Promise.all line - the result of which I just output as I want to get this working before I code the logic to combine the results of Q1 and Q2.
When I run this via mocha, the results of don't contain any data - I see the column headings but no data.
So what am I missing here?
// placeholder for the connection
let conn;
// return case list array
var caseList = [];
var queryList = [];
return new Promise((resolve, reject) => {
// retrieve connection
oracledb.getConnection({
user: dbconfig.user,
password: dbconfig.password,
connectString: dbconfig.connectString
}) // the connection is returned as a promise
.then(connection => {
console.log('Connected to the DB!');
// assign connection
conn = connection;
// execute statement
return connection.execute(
`select caseid, casereference, startdate from caseheader inner join orgobjectlink on caseheader.ownerorgobjectlinkid = orgobjectlink.orgobjectlinkid where orgobjectlink.username = :username`,
[params.username], {
outFormat: oracledb.OBJECT // set the output format to be object
}
);
})
.then(result => {
// iterate around rows
result.rows.forEach(row => {
var caseObj = {
caseID: row.CASEID,
reference: row.CASEREFERENCE,
dateAssigned: moment(row.STARTDATE).format('YYYY-MM-DD'),
username: params.username,
}
caseList.push(caseObj);
console.log(caseObj.caseID)
queryList.push(conn.execute(`select concernroleid, concernrolename from concernrole inner join caseparticipantrole on concernrole.concernroleid = caseparticipantrole.participantroleid where caseparticipantrole.caseid = :caseID and (caseparticipantrole.typecode = 'PRI' or caseparticipantrole.typecode = 'MEM')`,
[caseObj.caseID], {
outFormat: oracledb.OBJECT
}));
});
// build up queries
return Promise.all(queryList).then(results => {
console.log(results);
Promise.resolve(results);
}, err => {
console.log(err);
});
}).then({
if(conn){
console.log("Closing DB connection");
conn.close();
}
}).catch(err => {
console.log('Error', err);
});
});
Promise.all will not work for you as you want to use a single connection and as mentioned previously a connection will only do one thing at a time anyway. To solve this problem using promises, you'd have to build up and unwind a promise chain. I can show you an example, but it's nasty - probably better to just forget I mentioned it.
A better option would be to go into a simple for loop using async/await. I can show you can example of that too but again, I think this is the wrong move. We call this row by row fetching (a.k.a slow by slow).
It's likely the best solution for you will be to take the results from the first query and build up an array. Then execute the second query using one of these options to process the array. https://oracle.github.io/node-oracledb/doc/api.html#sqlwherein
You'll need to include the caseid column in the select clause and perhaps even order by that column so that post-processing of the result set is simplified in Node.js.
This solution has the potential to greatly improve performance and resource utilization, but that has to be balanced against the amount of data you have, the resources, etc. I could probably show you an example of this too, but it will take a bit longer and I'd want to get some more info from you to ensure we're on the right path.
One problem is the Promise.all().then... function doesn't return anything (and doesn't need the additional resolve()). The way to get this sorted is build small, testable, promise returning functions, and test them individually.
Starting simply, write a mocha test to connect to the database...
function connect() {
return oracledb.getConnection({
user: dbconfig.user,
password: dbconfig.password,
connectString: dbconfig.connectString
});
}
Here's one that can run a command on the db. Test this with a simple query that you know will return some results.
function executeCmd(connection, cmd, params) {
return connection.execute(cmd, params, { outFormat: oracledb.OBJECT });
}
With just these two (and one more) we can outline a simple function that does the job: connect to the database, run a select, process each result asynchronously, then disconnect.
function connectAndQuery(username) {
let connection;
return connect().then(result => {
connection = result;
let cmd = `select caseid, casereference, startdate from caseheader inner join orgobjectlink on caseheader.ownerorgobjectlinkid = orgobjectlink.orgobjectlinkid where orgobjectlink.username = :username`;
return executeCmd(connection, cmd, [username]);
}).then(result => {
let promises = result.rows.map(row => processCaseRow(connection, row, username));
return Promise.all(promises);
}).then(result => {
// result should be an array of caseObj's
return connection.close().then(() => result);
});
}
The last thing to build and test is a promise-returning function which processes a row from the main function above.
I had to take some liberty with this, but I think the objective is -- given a row representing a "case" -- build a case object, including a collection of "concernedRoles" that can be queried with the caseID. (that last bit was my idea, but you can build a separate collection if you like)
// return a promise that resolves to an object with the following properties...
// caseID, reference, dateAssigned, username, concernedRoles
// get concernedRoles by querying the db
function processCaseRow(connection, row, username) {
var caseObj = {
caseID: row.CASEID,
reference: row.CASEREFERENCE,
dateAssigned: moment(row.STARTDATE).format('YYYY-MM-DD'),
username: username
}
let cmd = `select concernroleid, concernrolename from concernrole inner join caseparticipantrole on concernrole.concernroleid = caseparticipantrole.participantroleid where caseparticipantrole.caseid = :caseID and (caseparticipantrole.typecode = 'PRI' or caseparticipantrole.typecode = 'MEM')`;
return executeCmd(connection, cmd, row.CASEID).then(result => {
caseObj.concernedRole = result
return caseObj
})
}
This question already has answers here:
Promise Retry Design Patterns
(21 answers)
Closed 4 years ago.
I'm new to nodejs and ES6 and trying to get my head around promises. I have a requirement to retry a dynamodb query function for specific intervals (5 seconds in this case) if the the result is not acceptable! So I have a function like this:
const retryFunc = (ddbParams, numberOfRetry) => {
return new Promise((resolve, reject) => {
return DDBUtils.query(ddbParams).then(result => {
//need to return a specific number of rows from the ddb table
if(numberOfRetry > 0){
if(result.length !== 100){
numberOfRetry = numberOfRetry - 1
setTimeout(() => {
retryFunc(ddbParams, numberOfRetry)
}, 5000)
}
}
resolve(result)
}).catch(error => {
reject(error)
})
})
}
When the dynamodb query returning the acceptable result (100 records) in the first call then the function working fine and returning the result to the caller. But if the function needs to be retried to satisfied the 100 condition then it is not returning the result to the caller when it gets satisfied! Can anybody help me to understand what is happening?
First, avoid the explicit promise construction antipattern - .query already returns a Promise, so there's no need to construct another one. Then, you inside your if(result.length !== 100){, you need to be able to chain together recursive calls of retryFunc; you can't directly return from an (asynchronous, callback-based) setTimeout, as with your current code.
One option would be to create a delay function, which returns a Promise that resolves after the desired amount of time - then, you can use return delay(5000).then(() => retryFunc(ddbParams, numberOfRetry - 1)) to return the recursive call.
const delay = ms => new Promise(res => setTimeout(res, ms));
const retryFunc = (ddbParams, numberOfRetry) => {
return DDBUtils.query(ddbParams).then(result => {
if(numberOfRetry > 0 && result.length !== 100) {
return delay(5000).then(() => retryFunc(ddbParams, numberOfRetry - 1));
}
});
}