NodeJS / request promise : the promise is resolved automatically - node.js

I use the request-promise node module as following :
let rp = require('request-promise');
I have the following promises chain :
getData().then( data => {
return getUser(data);
})
.then( data => {
return getProfiles(data);
}).then( data => {
updateAddresses(data);
});
The updateAddresses is as following :
function updateAddresses(data){
var promises = data.map( (aProfile) => {
var options = {url:'http://example.com', method:'POST'};
return rp(options);
});
// Promise.all(promises).then(...);
}
So I'm preparing a promise (request-promise) for each element of the array.
The problem is that those promises are firing even when I remove Promise.all !
How is that possible ? How can I make the promise not firing ?
Regards.

Your updateAddresses is not returning anything to call then() on and you aren't waiting on what would be coming out of updateAddress.
function updateAddresses(data){
var promises = data.map( (aProfile) => {
var options = {url:'http://example.com', method:'POST'};
return rp(options);
});
//Return something to then()
return Promise.all(promises).then(...);
}
Then wait on it:
getData().then( data => {
return getUser(data);
})
.then( data => {
return getProfiles(data);
}).then( data => {
return updateAddresses(data);
}).then(e => somethingelse);
Per your requirement to store the mapped address Objects:
function updateAddresses(data) {
return data.map((aProfile) => {
return {
url: 'http://example.com',
method: 'POST'
}
});
}
getData()
.then(data => getUser(data))
.then(data => getProfiles(data))
.then((data) => {
let addressOptions = updateAddresses(data);
/*
* You now have an Array of Objects of the form:
* {
* url: "",
* method: "",
* }
*/
return Promise.resolve(addressOptions);
})
.then(addressOptions => Promise.all(addressOptions.map(address => rp(address))))
.then((arrayOfResolvedData) => {
/*
* arrayOfResolvedData is an Array of data per whatever rp() returns
* probably Response Objects
*/
console.log(arrayOfResolvedData);
})

You could do what zero298 suggested in his edit.
the main problem is that you map your data to an actual promise, rather than something that is ready to execute a promise,
compare
Let promises = Options.map(option => execute(option))
Promise.all(promises)
With
Let promiseGenerators =Options.map(option => () => execute(option))
Promise.all(promiseGenerators.map(generator => generator())
The second you call the function that will return the promise, it will start doing it. That's where the problem was, because you called rp(options) too soon.

Related

Looping through a fetch operation in Node JS

I'm trying to write a collection of Json objects into an array whilst looping through a fetch operation in Node JS. I'm sure the issue is something to do with it being an Async operation, but can't figure out how to get round it.
This is my code:
for (const car of carFilter) {
const carJson = fetch(modelUrl + car, settings)
.then(res => res.json())
.then((json.data.trim));
carData.push(carJson);
}
console.log(carData);
All I'm getting from the console.log is:
Promise { <pending> },
Promise { <pending> },... etc
Which I presume means I'm trying to do the console.log before the data has been push into the array. I may be wrong.
Thanks in advance.
You can do something like this:
const promises = [];
for (const car of carFilter) {
const carJson = fetch(modelUrl + car, settings)
promises.push(carJson);
}
Promise.all(promises)
.then((values) => {
console.log(values); // will be in same order they are called
console.log(values[0]); // will be an Array
})
.catch( e => console.log(e));
So, When we call out an async operation, it returns a Promise(in this case). We are pushing all the promises in an array and can use "Promises.all" that can help to wait for all of them to resolve and gives results.
Note: if any of your promise will be rejected, you won't be able to get the subsequent promises resolved or rejected.
Sample one is here:
const promises = [];
for (let i = 0; i < 10; i++) {
const carJson = promiseCall(); //any promise call
promises.push(carJson);
}
Promise.all(promises)
.then((values) => {
console.log(values); // will be in same order they are called
})
.catch( e => console.log(e));
function promiseCall () {
return new Promise((res, rej) => {
setTimeout(()=> {
res(true);
} ,1000);
})
}

Get back empty nodejs + firebase

Guys I need some help with a get return with node and firebase.
I get an empty array.
but in console.log inside foreach prints correct
[{cnpj: '03052164798-000179', Address: 'Av Duke of Caxias 99999', Name: 'Testing', Tel: '999999999'}]
getEmpresas() {
let empresas = []
firebase.firestore().collection('empresa').get().then(snapshot => {
snapshot.docs.forEach(empresa => {
empresas.push(empresa.data())
console.log(empresas)
});
})
return empresas
I think this is a question about async. You're not getting the data you expect in the return because that fires before the async request is made. You'd need something like:
const getEmpresas = async () => {
let empresas = []
const snapshot = await firebase
.firestore()
.collection('empresa')
.get();
snapshot.docs.forEach((empresa) => {
empresas.push(empresa.data());
});
return empresas;
};
As imjared said in his answer, the function returns before .get() gets executed because it is an asynchronous function
You will need to return the chain of firebase functions which will return a Promise
getEmpresas() {
return firebase.firestore().collection('empresa').get().then(snapshot => (
snapshot.docs.map(({ data }) => data())
))
}
Then to access the returned Promise use .then() on the Promise or await inside an async function
getEmpresas.then(empresas => console.log(empresas))
Or inside an async function
const main = async () => {
const empresas = await getEmpresas()
console.log(empresas)
}
main()

insert document into multiple instance of couch DB in Node JS with all success and failure result in any way possible

i have array of db like
const dbArr = ["http://localhost:5984", "http://xyz_couchdb.com:5984"]
data to insert
let data ={
_id: 324567,
name: Harry,
gerder: male
}
here is the logic i am using nano module
return new Promise((resolve, reject) => {
let res = [];
let rej = [];
let counter = 0;
for(let i = 0; i < dbArr.length ; i++){
dbArr[i].insert(data, (err, body) => {
err ? rej.push(err) : res.push(body)
if(counter === obj.dbArray.length -1){
rej.length ? reject(rej) : resolve(res)
}
counter++;
})
}
})
what can be the best possible way to achieve this using promise or async module or anything.
In the following example, we gotta use Array.map to create one promise for each element of dbArr, then we gotta wait all promises to end using Promise.all. The catch is here so we handle the errors.
function getAll(dbArr) {
return Promise.all(dbArr.map(x => x.insert(data)));
}
getAll(dbArr)
.then((rets) => {
// Handle the returns
// They are in an array
})
.catch((err) => {
// Handle the error
});
EDIT :
Ok after checking out the documentation of node-couchdb (the one I suppose you use) - I saw that the .insert() method do not return a Promise but only a callback.
So we gotta transform the method, so it will return a Promise using util.Promisify()
const {
promisify,
} = require('util');
function getAll(dbArr) {
return Promise.all(dbArr.map(x => promisify(x.insert)(data)));
}
getAll(dbArr)
.then((rets) => {
// Handle the returns
// They are in an array
})
.catch((err) => {
// Handle the error
});

Axios.all, how to configure axios wait time to mitigate hung up?

My application uses an internal webservice for fetching data, i have a job which creates approx 500 requests which getsfired async to complete the fetch operation.
I make use of Axios, by creating an array of axios promises and then resolving them using using Axios.all();
It works fine until some 200 requests but post that i get socket hung up, however on the server side i see the requests are being processed.
How to configure axios to set custom time out, or is it a better idea to splice my promises array and then run them as multiple batches ?
Source code
let getAxiosPromiseArray = (urlList) => {
var axiosArrayofPromise = [];
return new Promise ( (resolve, reject) => {
try {
urlList.forEach ( (URL) => {
axiosArrayofPromise.push(axios.get(URL));
});
resolve(axiosArrayofPromise);
}
catch (err) {
reject("There is a problem getting Axios array of promises " + err);
}
})
}
async function processAxiosPromises (PromiseArray) {
try {
var results = []
results = await axios.all(PromiseArray);
return results;
}
catch(err) {
throw("There was a problem resolving promises array (Axios) " + err);
}
}
getallID().then ( (urlList) => {
return getAxiosPromiseArray(urlList);
}).then( (AxiosPromises) => {
return processAxiosPromises(AxiosPromises);
}).then ((resultData) => {
console.log(resultData);
});
Error
There was a problem resolving promises array (Axios) Error: socket hang up
First, that pair of functions getAxiosPromiseArray() and processAxiosPromises() needs fixing.
Your new Promise() construction is unnecessary. You can simply return Promise.all(arrayofPromise) (or axios.all(...) if you must) and do away with the other function.
Renaming the remaining function to something meaningful, you would end up with eg :
let getData = (urlList) => {
return Promise.all(urlList.map(URL => axios.get(URL)))
.catch(error => {
error.message = "There is a problem getting Axios array of promises " + error.message; // augment the error message ...
throw error; // ... and re-throw the errror.
});
};
And call as follows :
getallID().then(getData)
.then(resultData => {
console.log(resultData);
}).catch(error => {
console.error(error);
});
That will put you on solid ground but, on its own, is unlikely to fix a concurrency problem (if that's what it is), for which the simplest approach is to use Bluebird's Promise.map with the concurrency option.
The caller code can remain the same, just change getData(), as follows:
let getData = (urlList) => {
let concurrency = 10; // play with this value to find a reliable concurrency limit
return Promise.map(urlList, URL => axios.get(URL), {'concurrency': concurrency})
.catch(error => {
error.message = "There is a problem getting Axios array of promises " + error.message;
throw error;
});
};
// where `Promise` is Bluebird.
const axios = require('axios');
const axiosThrottle = require('axios-throttle');
//pass axios object and value of the delay between requests in ms
axiosThrottle.init(axios,200)
const options = {
method: 'GET',
};
const urlList = [
'https://jsonplaceholder.typicode.com/todos/1',
'https://jsonplaceholder.typicode.com/todos/2',
'https://jsonplaceholder.typicode.com/todos/3',
'https://jsonplaceholder.typicode.com/todos/4',
'https://jsonplaceholder.typicode.com/todos/5',
'https://jsonplaceholder.typicode.com/todos/6',
'https://jsonplaceholder.typicode.com/todos/7',
'https://jsonplaceholder.typicode.com/todos/8',
'https://jsonplaceholder.typicode.com/todos/9',
'https://jsonplaceholder.typicode.com/todos/10'
];
const promises = [];
const responseInterceptor = response => {
console.log(response.data);
return response;
};
//add interceptor to work with each response seperately when it is resolved
axios.interceptors.response.use(responseInterceptor, error => {
return Promise.reject(error);
});
for (let index = 0; index < urlList.length; index++) {
options.url = urlList[index];
promises.push(axiosThrottle.getRequestPromise(options, index));
}
//run when all promises are resolved
axios.all(promises).then(responses => {
console.log(responses.length);
});
https://github.com/arekgotfryd/axios-throttle

Angular : Receive responses in order with the calls

Hi I am pretty new to Angular and Observables
I am trying to GET Objects by theirs ID through a loop.
But don't receive my Response in Order.
Example
get ID(1)
get ID(2)
get ID(3)
Receive Object ID(2)
Receive Object ID(3)
Receive Object ID(1)
Is it possible to get my Objects back in order ??
Below is where I call multiple times my service function :
conferences-attendance.component.ts
ExportExcelAttendance() {
for (var i = 0; i < this.contactsAttendance.length; i++) {
this.practiceService.GetPracticebyDBID(this.contactsAttendance[i].practiceId)
.subscribe(
(practice: Practice) => {
this.practicesAttendance.push(practice);
if (this.practicesAttendance.length == this.contactsAttendance.length) {
this.ExportExcelAttendance2();
}
},
error => this.errorMessage = <any>error
);
}
}
Here is my function in my service, it where I receive the data (not in order with the calls).
practices.service.ts
GetPracticebyDBID(id: string) {
let params: URLSearchParams = new URLSearchParams();
params.set('thisId', id);
let requestOptions = new RequestOptions();
requestOptions.params = params;
return this.http.get('http://ec2-34-231-196-71.compute-1.amazonaws.com/getpractice', requestOptions)
.map((response: Response) => {
return response.json().obj;
})
.catch((error: Response) => Observable.throw(error.json()));
}
forkJoin gives you a little less code,
const arrayOfFetches = this.contactsAttendance
.map(attendee => this.practiceService.GetPracticebyDBID(attendee.practiceId) );
Observable.forkJoin(...arrayOfFetches)
.subscribe((practices: Practice[]) => {
this.practicesAttendance = practices;
this.ExportExcelAttendance2();
});
Edit
Snap! #Anas beat me to it. Although, I don't think you need the concatAll()
you should use concatAll operator to ensure calling your observables in sequence.
also, you can use completed callback to call ExportExcelAttendance2 instead of checking practicesAttendance length on every response callback.
check the below example:
let contactsAttendanceObservables = this.contactsAttendance
.map((item) => {
return this.practiceService.GetPracticebyDBID(item.practiceId);
});
Observable.of(...contactsAttendanceObservables)
.concatAll()
.subscribe(
(practice: Practice) => {
this.practicesAttendance.push(practice);
},
(err) => {
// handle any errors.
},
() => {
// completed
this.ExportExcelAttendance2();
}
);
if you still want your observables to run in parallel, you can use forkJoin Operator, which will emit the last value of all the passed observables to a one subscriber when all observables are completed.
check the below example:
let contactsAttendanceObservables = this.contactsAttendance
.map((item) => {
return this.practiceService.GetPracticebyDBID(item.practiceId);
});
Observable.forkJoin(...contactsAttendanceObservables)
.subscribe(
(practices: Practice[]) => {
this.practicesAttendance = practices;
this.ExportExcelAttendance2();
}
);
The forkJoin operator is simple to use. It waits until all observables complete, then emit an array with all the items emitted.
ExportExcelAttendance() {
const all = this.contactsAttendance.map(it => this.practiceService.GetPracticebyDBID(it.practiceId));
Rx.Observable.forkJoin(all)
.subscribe(
practicesAttendance => this.ExportExcelAttendance2(practicesAttendance),
error => this.errorMessage = < any > error);
}

Resources