The following code takes a good 5 to 10 seconds to resolve (there are 5 API-calls). It's not usable as it is. Is the slow response my fault or is the Coinbase API just slow and large?
app.get('/buys', async (req, res) => {
const buys = await getAllBuys()
res.json(buys)
})
const fetchAllAccounts = () => {
return new Promise(
(resolve, reject) => {
coinbase.getAccounts(null, (err, accounts) => {
if (err) {
reject(err)
}
else{
resolve(accounts)
}
})
}
)
}
const fetchAllBuys = (account) => {
return new Promise(
(resolve, reject) => {
account.getBuys(null, (err, buys) => {
if (err) {
reject(err)
}
else{
resolve(buys)
}
})
}
)
}
const getAllBuys = async () => {
const accounts = await fetchAllAccounts()
let combinedBuys = []
for (let account of accounts) {
const buys = await fetchAllBuys(account)
combinedBuys = [...combinedBuys, ...buys]
}
//console.log(combinedBuys)
return combinedBuys
}
Response is a array with items that include account information like certificates and so on.
Ok, so it was my code. I the code above I wait for each request to finish before firing off a new one. Which results in unnecessary delay. With the Promise.all method we can generate all the promises and resolve them at the same time. Look at the refactored code bellow.
app.get('/buys', async (req, res) => {
console.time('[/buys]');
const promises = await getAllBuys()
const buys = await Promise.all(promises)
res.json(buys)
console.timeEnd('[/buys]');
})
const fetchAllAccounts = () => {
return new Promise(
(resolve, reject) => {
coinbase.getAccounts(null, (err, accounts) => {
if (err) {
reject(err)
}
else{
resolve(accounts)
}
})
}
)
}
const fetchAllBuys = (account) => {
return new Promise(
(resolve, reject) => {
account.getBuys(null, (err, buys) => {
if (err) {
reject(err)
}
else{
resolve(buys)
}
})
}
)
}
const getAllBuys = async () => {
const accounts = await fetchAllAccounts()
let promises = []
for (let account of accounts) {
promises.push(fetchAllBuys(account))
}
return promises
}
From 2,5 / 3 seconds to 0,8 / 1 second.
Related
The following code works great locally, but after deploying it to AWS Lambda and running it my records are not saving to DynamoDB and I'm getting a return of null from Lambda.
I know it's not a permissions issue with the Lambda execution role because I can successfully insert one individual record into DynamoDB from the AWS console.
I think the issue has to do with the .forEach loop and how the aws-sdk works. I'm not sure I'm completely wrapping my head around how to properly use JavaScript promises with Lambda. Any help is greatly appreciated!
module.exports.handler = async event => {
const getItems = () => {... // return items //...}
const addToDb = (items) => {
items.forEach(item => {
var params = {
Item: {"id": {S: item.id}, "title": {S: item.title}},
ReturnConsumedCapacity: "TOTAL",
TableName: "my-table"
};
dynamodb.putItem(params, (err, data) => {
if (err) console.log(err, err.stack);
else console.log(data);
});
});
};
const getItemsPromise = new Promise((resolve) => {
const items = getItems();
const itemsAddedToDb = addToDb(items);
resolve(itemsAddedToDb);
});
return getItemsPromise
.catch(err => console.log(err));
};
This should work!
exports.handler = (event) => {
const getItems = () => {...} // assuming getItems returns promise
const addToDb = (items) => {
asyncForEach(items, async (item) => {
const params = {
Item: {
id: {
S: item.id
},
title: {
S: item.title
}
},
ReturnConsumedCapacity: 'TOTAL',
TableName: 'my-table'
}
await dynamodb.putItem(params, (err, data) => {
if (err) console.log(err, err.stack)
else console.log(data)
})
})
}
const getItemsPromise = new Promise(async (resolve) => { // rule eslintno-async-promise-executor - use then instead
const items = await getItems()
const itemsAddedToDb = await addToDb(items)
resolve(itemsAddedToDb)
})
const asyncForEach = async (array, callback) => {
for (let index = 0; index < array.length; index++) {
await callback(array[index], index, array)
}
}
return getItemsPromise.catch((err) => console.log(err))
}
Notice:
async for export.handler has no use. use async only if function has await inside it.
async await doesn't support for forEach try for loop instead
So I've got a function which is :
module.exports.play = async (client, channel, players) => {
players.push(client.guilds.cache.first().members.cache.array().filter(mem => mem.user.username === "user1").map(mem => mem.user)[0]);
players.push(client.guilds.cache.first().members.cache.array().filter(mem => mem.user.username === "users2").map(mem => mem.user)[0]);
try {
await beginMessage(players, channel);
let playersCards = await beginGame(client, players);
console.log(playersCards);
emitter.on("stop", async () => {
await channel.delete();
console.log("jeu fini");
});
}
catch(e) {
console.error(e);
}
};
and the is that the value of my variable 'playersCards' is print before the value is returned by the async function "beginGame". Here's beginGame's function code :
async function beginGame(client, players) {
let playersCards = [];
fs.readFile("Utils/cartes.json", 'utf8', async (err, data) => {
if(err) console.log(err);
let cartes = JSON.parse(data);
cartes = cartes.cartes;
await asyncForEach(players, async player => {
let playerCards = await distributeCards(player, cartes);
playersCards.push(playerCards);
});
return playersCards;
});
}
Where could the problem be ?
Because beginGame returns a Promise with no value because fs.readFile took a callback which is called else where, so there is nothing to wait for.
async function beginGame(client, players) {
return new Promise(async (resolve, reject) => {
let playersCards = [];
return fs.readFile('Utils/cartes.json', 'utf8', async (err, data) => {
if (err) return reject(err);
let cartes = JSON.parse(data);
cartes = cartes.cartes;
await asyncForEach(players, async player => {
let playerCards = await distributeCards(player, cartes);
playersCards.push(playerCards);
});
return resolve(playersCards);
});
});
}
this should work since the function is wrapped with promise and there is something to wait for.
I am trying to set up a route that sends data from a nested promise to my vue app.
But i'm having trouble with the getting data from the nested promises.
i tried using a callback with no success
app.get('/notification', (req, res) => {
const getData = (data) => {
console.log(data)
}
scheduler(data)
})
const scheduler = (callback) => {
sftp
.connect({ credentials })
.then(() => {
return sftp.list(root);
})
.then(async data =>
{
const filteredFile = data.filter(file => {
let currentDate = moment();
let CurrentTimeMinusFive = moment().subtract(5, "minutes");
let allAccessTimes = file.accessTime;
let parsedAccessTimes = moment(allAccessTimes);
let filteredTime = moment(parsedAccessTimes).isBetween(
CurrentTimeMinusFive,
currentDate
);
return filteredTime;
});
for (const file of filteredFile) {
let name = file.name;
let filteredThing;
await sftp
.get(`Inbound/${name}`)
.then(data => {
csv()
.fromString(data.toString())
.subscribe(function (jsonObj) {
return new Promise(function (resolve, reject) {
filteredThing = new Notification(jsonObj);
filteredThing.save()
.then(result => {
console.log(result);
callback(result) **// THIS IS THE RESULT I NEED IN MY FRONT END**
})
.catch(err => {
console.log(err);
});
resolve();
});
});
});
}
})
When i go to localhost/notification i get:
ReferenceError: data is not defined
Thanks in advance!
I am trying to get the array after it is filled in a function but it is empty when I print it out.
Code:
var usersList = [];
app.post('/submitLogin',function(req,res) {
getUsersGroups();
console.log(usersList);
});
function getUsersGroups() {
const users = new Promise((resolve, reject) => {
dbConnection
.getUsers()
.then(data => {
resolve(data)
})
});
const groups = new Promise((resolve, reject) => {
dbConnection
.getGroups()
.then(data => {
resolve(data)
})
});
Promise.all([users, groups])
.then(data => {
usersList = data[0];
groupsList = data[1];
console.log(usersList)
});
}
However, the console.log in the getUsersGroups(), prints out the filled array but it is empty in the app.post('/submitLogin...')
Why is this happening assuming that getUsersGroups() runs before I try to print out the array?
You are not observing the async execution of your db call.
The console.log(usersList); happens BEFORE the db call, so it is empty.
You must adapt your code like this and post the code AFTER the db execution:
app.post('/submitLogin', function (req, res) {
getUsersGroups().then((data) => {
console.log(data)
res.send(data.usersList)
})
})
function getUsersGroups () {
const users = new Promise((resolve, reject) => {
dbConnection
.getUsers()
.then(data => {
resolve(data)
})
})
const groups = new Promise((resolve, reject) => {
dbConnection
.getGroups()
.then(data => {
resolve(data)
})
})
return Promise.all([users, groups])
.then(data => {
console.log(data[0])
return {
usersList: data[0],
groupsList: data[1]
}
})
}
I strongly suggest to don't change a global variable like usersList (that I removed from my example)
because if you receive two request contemporaneously, the second one could overwrite the data of the first one and
cause many side effect.
app.post('/submitLogin', async function(req,res) {
await getUsersGroups();
console.log(usersList);
});
try this
I have a typescript module.
public multipleQuery(queries: string[]) {
return new Promise(async (resolve, reject) => {
const cPool = new sql.ConnectionPool(this.room.db);
await cPool.connect().then((pool: any) => {
const transaction = new sql.Transaction(pool);
return transaction.begin(async (err: any) => {
const request = new sql.Request(transaction, { stream: true });
try {
queries.forEach(async (q) => {
await request.query(q);
});
transaction.commit((err2: any) => {
pool.close();
if (err2) {
reject(err2);
} else {
resolve(true);
}
});
} catch (err) {
transaction.rollback(() => {
pool.close();
reject(err);
});
}
});
}).catch((err: Error) => {
cPool.close();
reject(err);
});
});
}
queries variable is an array of string, I put inside a lot of sql insert queries.
No matter what I write in queries, I still receive this error, why?
RequestError: Requests can only be made in the LoggedIn state, not the
SentClientRequest state TransactionError: Can't acquire connection for
the request. There is another request in progress.
the solutions is to use async
const async = require("async");
public multipleQuery(queries: string[]) {
return new Promise((resolve, reject) => {
const pool = new sql.ConnectionPool(this.room.db);
return pool.connect().then((p: any) => {
const transaction = new sql.Transaction(p);
return transaction.begin((err: any) => {
const request = new sql.Request(transaction);
if (err) {
reject(err);
}
return async.eachSeries(queries, async (query: any, callback: any) => {
return request.query(query);
}, async (err2: any) => {
if ( err2 ) {
await transaction.rollback(() => {
pool.close();
reject(err2);
});
} else {
await transaction.commit(() => {
pool.close();
resolve(true);
});
}
});
});
});
});
}