I am learning SSO and trying this out without the conventional User class/object. I am new to asynchronous programming and having difficulty in managing the data flow. I am stuck at a point where I have successfully exported a boolean value, but my import (in another module) gets undefined. I suspect it is because import does not wait for the corresponding export statement to execute first. How do I make it and all subsequent code wait?
I don't know what to try in this case.
Module that is exporting usrFlag
const request = require("request");
let usrFlag = false; // assuming user doesn't already exist.
function workDB(usr_id, usr_name, dateTimeStamp) {
//some code excluded - preparing selector query on cloudant db
request(options, function (error, response, body) {
if (error) throw new Error(error);
if (body.docs.length == 0) addUsr(usr_id, usr_name, dateTimeStamp);
else {
xyz(true); //This user already exists in cloudant
console.log('User already exists since', body.docs[0].storageTime);
}
});
}
async function setUsrFlag(val) { usrFlag = val; }
async function xyz(val) {
await setUsrFlag(val);
//module.exports below does not execute until usrFlag has the correct value.
//so value is not exported until usrFlag has been properly set.
console.log(usrFlag);
module.exports.usrFlag = usrFlag;
}
Module that is importing this value
const usrP = require('../config/passport-setup');
const dbProcess = require('../dbOps/dbProcessLogic'); // <-- This is import
router.get('/google/redirect', passport.authenticate('google'), (req, res) => {
dbProcess.workDB(usrP.usrPrf.id, usrP.usrPrf.displayName, new Date());
// Instead of true/false, I see undefined here.
console.log(dbProcess.usrFlag);
});
I expect the require function of import module to wait for export module to send it all the required values. However, I know that is probably not going to happen without me explicitly telling it to do so. My question is, how?
So, I have just modified some of the code, so that I can work on it easily.
Module that is exporting usrFlag
// const request = require("request");
let usrFlag = false; // assuming user doesn't already exist.
function workDB(usr_id, usr_name, dateTimeStamp) {
return new Promise(function (resolve, reject) {
setTimeout(function () {
xyz(true).then(function () {
resolve('done');
})
}, 1000);
});
}
function setUsrFlag(val) { usrFlag = val; }
function xyz(val) {
return new Promise(function (resolve, reject) {
setUsrFlag(val);
module.exports.usrFlag = usrFlag;
resolve('done');
});
}
module.exports = {
usrFlag,
workDB
}
Module that is importing this value
const dbProcess = require('../dbOps/dbProcessLogic'); // <-- This is import
dbProcess.workDB().then(function () {
console.log(dbProcess.usrFlag);
})
Now when you run the second file, you get usrFlag as true.
I have used setTimeout to imitate a request.
Sorry if I butchered up some of your code.
Related
Currently i am trying to create a CSV reader that can handle very large CSV files. I chose for a streaming implementation with the event-stream NPM package.
I have created a function getNextp() that should return a promise and give me the next piece of data every time i call it.
"use strict";
const fs = require('fs');
const es = require('event-stream');
const csv = require('csv-parser');
class CsvFileReader {
constructor(file) {
this.file = file;
this.isStreamReading = false;
this.stream = undefined;
}
getNextP() {
return new Promise( (resolve) => {
if (this.isStreamReading === true) {
this.stream.resume();
} else {
this.isStreamReading = true;
// Start reading the stream.
this.stream = fs.createReadStream(this.file)
.pipe(csv())
.pipe(es.mapSync( (row) => {
this.stream.pause();
resolve(row);
}))
.on('error', (err) => {
console.error('Error while reading file.', err);
})
.on("end", () => {
resolve(undefined);
})
}
});
}
}
I call this then with this code.
const csvFileReader = new CsvFileReader("small.csv");
setInterval( () => {
csvFileReader.getNextP().then( (frame) => {
console.log(frame);
})
}, 1000);
However every time when i try this out i only get the first row and the subsequent rows i do not get. I can not figure out why this it not working. I have tried the same with a good old callback function and then it works without any problem.
Update:
So what i basically want is a function (getNext()) that returns me every time when i call it the next row of the CSV. Some rows can be buffered, but yeah until now i could not figure out how to do this with streams. So if somebody could give me a pointer on how to create a correct getNext() function that would be great.
I would like to ask if somebody understands what is going wrong here, and ask kindly to share his/hers knowledge.
Thank you in advance.
I'm trying to download a bunch of files. Let's say 1.jpg, 2.jpg, 3.jpg and so on. If 1.jpg exist, then I want to try and download 2.jpg. And if that exist I will try the next, and so on.
But the current "getFile" returns a promise, so I can't loop through it. I thought I had solved it by adding await in front of the http.get method. But it looks like it doesn't wait for the callback method to finish. Is there a more elegant way to solve this than to wrap the whole thing in a new async method?
// this returns a promise
var result = getFile(url, fileToDownload);
const getFile = async (url, saveName) => {
try {
const file = fs.createWriteStream(saveName);
const request = await http.get(url, function(response) {
const { statusCode } = response;
if (statusCode === 200) {
response.pipe(file);
return true;
}
else
return false;
});
} catch (e) {
console.log(e);
return false;
}
}
I don't think your getFile method is returning promise and also there is no point of awaiting a callback. You should split functionality in to two parts
- get file - which gets the file
- saving file which saves the file if get file returns something.
try the code like this
const getFile = url => {
return new Promise((resolve, reject) => {
http.get(url, response => {
const {statusCode} = response;
if (statusCode === 200) {
resolve(response);
}
reject(null);
});
});
};
async function save(saveName) {
const result = await getFile(url);
if (result) {
const file = fs.createWriteStream(saveName);
response.pipe(file);
}
}
What you are trying to do is getting / requesting images in some sync fashion.
Possible solutions :
You know the exact number of images you want to get, then go ahead with "request" or "http" module and use promoise chain.
You do not how the exact number of images, but will stop at image no. N-1 if N not found. then go ahed with sync-request module.
your getFile does return a promise, but only because it has async keyword before it, and it's not a kind of promise you want. http.get uses old callback style handling, luckily it's easy to convert it to Promise to suit your needs
const tryToGetFile = (url, saveName) => {
return new Promise((resolve) => {
http.get(url, response => {
if (response.statusCode === 200) {
const stream = fs.createWriteStream(saveName)
response.pipe(stream)
resolve(true);
} else {
// usually it is better to reject promise and propagate errors further
// but the function is called tryToGetFile as it expects that some file will not be available
// and this is not an error. Simply resolve to false
resolve(false);
}
})
})
}
const fileUrls = [
'somesite.file1.jpg',
'somesite.file2.jpg',
'somesite.file3.jpg',
'somesite.file4.jpg',
]
const downloadInSequence = async () => {
// using for..of instead of forEach to be able to pause
// downloadInSequence function execution while getting file
// can also use classic for
for (const fileUrl of fileUrls) {
const success = await tryToGetFile('http://' + fileUrl, fileUrl)
if (!success) {
// file with this name wasn't found
return;
}
}
}
This is a basic setup to show how to wrap http.get in a Promise and run it in sequence. Add error handling wherever you want. Also it's worth noting that it will proceed to the next file as soon as it has received a 200 status code and started downloading it rather than waiting for a full download before proceeding
I would like to have a configuration file with variables set with data I fetch from an API.
I think I must use async and await features to do so, otherwise my variable would stay undefined.
But I don't know how to integrate this and keep the node exports.myVariable = myData available within an async function ?
Below is the code I tried to write to do so (all in the same file) :
const fetchAPI = function(jsonQuery) {
return new Promise(function (resolve, reject) {
var reqOptions = {
headers: apiHeaders,
json:jsonQuery,
}
request.post(apiURL, function (error, res, body) {
if (!error && res.statusCode == 200) {
resolve(body);
} else {
reject(error);
}
});
});
}
var wallsData = {}
const fetchWalls = async function (){
var jsonQuery = [{ "recordType": "page","query": "pageTemplate = 1011"}]
let body = await utils.fetchAPI(jsonQuery)
let pageList = await body[0].dataHashes
for(i=0;i<pageList.length;i++){
var page = pageList[i]
wallsData[page.title.fr] = [page.difficultyList,page.wallType]
}
return wallsData
throw new Error("WOOPS")
}
try{
const wallsData = fetchWalls()
console.log(wallsData)
exports.wallsData = wallsData
}catch(err){
console.log(err)
}
The output of console.log(wallsData) shows Promise { <pending> }, therefore it is not resolved and the configuration file keep being executed without the data in wallsData...
What do I miss ?
Thanks,
Cheers
A promise is a special object that either succeeds with a result or fails with a rejection. The async-await-syntax is syntactic sugar to help to deal with promises.
If you define a function as aync it always will return a promise.
Even a function like that reads like
const foo = async() => {
return "hello";
}
returns a promise of a string, not only a string. And you need to wait until it's been resolved or rejected.
It's analogue to:
const foo = async() => {
return Promise.resolve("Hello");
}
or:
const foo = async() => {
return new Promise(resolve => resolve("Hello"));
}
Your fetchWalls similarly is a promise that will remain pending for a time. You'll have to make sure it either succeeds or fails by setting up the then or catch handlers in your outer scope:
fetchWalls()
.then(console.log)
.catch(console.error);
The outer scope is never async, so you cannot use await there. You can only use await inside other async functions.
I would also not use your try-catch for that outer scope promise handling. I think you are confusing the try-catch approach that is intended to be used within async functions, as there it helps to avoid nesting and reads like synchronous code:
E.g. you could do inside your fetchWalls defintion:
const fetchWalls = async function (){
var jsonQuery = [{ "recordType": "page","query": "pageTemplate = 1011"}]
try {
let body = await utils.fetchAPI(jsonQuery)
} catch(e) {
// e is the reason of the promise rejection if you want to decide what to do based on it. If you would not catch it, the rejection would chain through to the first error handler.
}
...
}
Can you change the statements like,
try{
const wallsData = fetchWalls();
wallsData.then((result) => {
console.log(result);
});
exports.wallsData = wallsData; // when importing in other file this returns as promise and we should use async/await to handle this.
}catch(err){
console.log(err)
}
I want a result like this
var rolecheck = ['289773584216358912','281531832938266625'];
Only fetched from a database, so I can compare it to another array with Id's (and yes it's supposed to be a string)
The purpoose of this is to check, before executing a command, if the user has a specific role with permission for that role. So it needs to be a function able to be called.
I've never worked with NodeJs async functions, so i have no clue how to convert this sql to an array:
The content of the .then is just some code of me trying to find out how it works, so ignore the consolelogs etc. Note: the logs do return the correct roles, but i just need them to return them to use them in my compare function.
sql.all("SELECT roleId FROM roles WHERE punish = 'true' and guildId = '"+guildids+"'").then(row => {
if (row) {
var rolecheck = [];
row.forEach(function(row){
rolecheck.push(row.roleId);
});
console.log(rolecheck);
}
});
returning does not work, so I need a workaround.
Here's where i compare it: (this works fine as long as rolecheck and role.id are defined correctly, which they aren't. It does work when i hardcode the rolecheck array.
member.forEach(function(role){
if(HasRole(rolecheck, role.id)){
console.log('user has role: '+role.name);
return true;
}
});
In case you can use a node version with async/await support, like version 7, here is a way to write promise-based code in a synchronous manner. This makes it simpler to pass around the row value.
async function myFunction () {
try {
let member = ''; // whatever member should be
let row = await sql.all("SELECT roleId FROM roles WHERE punish = 'true' and guildId = '"+guildids+"'");
// now you have the row available, outside of 'then' blocks
var rolecheck = [];
if (row) {
row.forEach(function(row){
rolecheck.push(row.roleId);
});
console.log(rolecheck);
}
member.forEach(function(role){
if (HasRole(rolecheck, role.id)) {
console.log('user has role: '+role.name);
return true;
}
});
} catch (error) {
consol.log(error.stack);
}
}
If sql.all did not return a promise you could instead do something like this which would also work with callback based functions
async function myFunction () {
try {
let row = await runSql();
// everything else same as first example above
}
async function runSql () {
try {
return new Promise(function (resolve, reject) {
sql.all("SELECT roleId FROM roles WHERE punish = 'true' and guildId = '"+guildids+"'")
.then(row => {
if (row) {
var rolecheck = [];
row.forEach(function(row){
rolecheck.push(row.roleId);
});
console.log(rolecheck);
resolve(row);
} else {
reject('Row not found');
}
});
});
} catch (error) {
console.log('erro')
}
};
I am currently developing a bot using wit.ai. I am quite new to node.js. Basically, I am following the guide provided by node-wit lib. I construct my wit object by:
const wit = new Wit({
accessToken: WIT_TOKEN,
actions,
logger: new log.Logger(log.INFO)
});
In my actions, I have something like:
const actions = {
send({sessionId}, {text}) {
//my sending action goes here.
},
firstaction({context, entities,sessionId}) {
var data = async_function();
context.receiver = data;
return context;
}
}
The issue is that whatever comes after async_function will be executed first. I tried to let async_function return a promise. However, this wouldn't work since whatever comes after my first action in node-wit library will be executed first without waiting for the context to return. I don't want to modify the node-wit library.
Any idea that would solve my issue is appreciated!
you can use async library for asynchronous call
https://caolan.github.io/async/docs.html
const async = require('async')
const actions = {
send({sessionId}, {text}) {
//my sending action goes here.
},
firstaction({context, entities,sessionId}) {
async.waterfall([function(callback) {
var d = async_function();
// if err pass it to callback first parameter
// return callback(err)
callback(null,d);
}], function(err, result) {
if(err) {
return err;
}
var data = result;
context.receiver = data;
return context;
})
}
}