I am working with firestore in NodeJS, and wrote the function below, this function receives the results array as a parameter, which contains the firestore doc id that needs to be updated.
So, for each document, i am calling the set method to add the fileName field to this document. But for some reason, the result only updated when i have a single id in this results array.
Can someone help me?
async function update(results, fileName) {
results.forEach(data => {
let id = firestore.collection('My_collection').doc(data.DOC_ID)
id.set(
{ filename: fileName}, {merge:true}
)
})
}
You can try by adding a promise to your code.
async function update(results, fileName) {
results.forEach(data => {
let id = firestore.collection('My_collection').doc(data.DOC_ID)
await id.set({ filename: fileName}, {merge:true})
})
}
Like shown in the documentation[1]
[1]https://firebase.google.com/docs/firestore/quickstart#add_data
Related
I'm trying to migrate an existing function to use it inside an Apify actor.
Originally, the function loads a given URL, reads its JSON response, and according to some supplied parameters, extracts some data and returns an object with results.
If you ask, it's not scraping anything "final" at this point. Its results are temporary and will be used to create other URLs which will be scraped then (with another crawler) for actual, useful results.
The current function that executes the crawler is something like this:
let url = new URL('/content', someBaseURL);
url.searchParams.set('search', someKeyword);
const reqList = new apify.RequestList({
sources: [ { url: url.toString() } ]
});
await reqList.initialize();
const crawler = new apify.BasicCrawler({
requestList: reqList,
handleRequestFunction: reqHandler
});
// How do I set the inputs for reqHandler() here ?
await crawler.run();
// How do I get the output from reqHandler() here ?
And the reqHandler code is something like this:
async function reqHandler(options) {
const response = await apify.utils.requestAsBrowser({
url: options.request.url
});
// How do I read parameters from the caller here ?
let searchResults = JSON.parse(response.body);
// ... result object creation logic goes here ...
// How do I return a result to the caller here ?
}
I am pretty new to this Apify thing and lost in the documentation.
Thanks for your help.
handleRequestFunction doesn't take any external input or produce any outputs. Simply use it as a closure and capture inputs from the surrounding code or you can wrap it in a different function.
Normally we do it like this:
const context = {}; // put your inputs here
const crawler = new apify.BasicCrawler({
requestList: reqList,
handleRequestFunction: async () => {
// use context here
// output data
await Apify.pushData(results);
}
});
EDIT: I forgot to mention a use-case on how to pass input. You need to do it via the request.userData object when adding to a queue or a list.
// The same userData is available in request list.
await requestQueue.addRequest({
url: 'https://example.com',
userData: { myInput: 'any-data' }
});
// Then in handleRequestFunction
handleRequestFunction: async (( request }) => {
const { myInput } = request.userData;
// ...
}
How do you get the information for the Chrome extension by using async and await?
[chrome.instanceID.getID]
[chrome.storage.sync.get]
We tried this code:
async function test()
{
let _r = await chrome.instanceID.getID();
return _r;
}
let _pc_id = test();
but _pc_id returns a promise. We find no way to get the value in it. How should we do this?
You can get the instanceID like this, but can't store it in a variable to use it out of scope of the promise, AFAIK. You may want to read this: saving data from promise in a variable
If you want to use the returned value of Promise you need to do it in the promise or in .then()s after the promise, at least that is how I do it.
chrome.instanceID.getID example:
chrome.instanceID.getID((instance_id) => {
console.log(instance_id);
// Execute your other related code here
});
or
var promise = chrome.instanceID.getID();
promise.then((instance_id) => {
console.log(instance_id);
// Execute your other related code here
});
or
chrome.instanceID.getID()
.then((instance_id) => {
console.log(instance_id);
// Execute your other related code here
});
chrome.storage.sync.get example:
chrome.storage.sync.get('myKey', function(items) {
var key = items.myKey;
// Below code is an example, change it to your needs
if (key) {
console.log(key)
} else {
key = createKey(); // a custom function that generates keys
chrome.storage.sync.set({myKey: key}, function () {
console.log(key);
});
}
};
**I want to return the data key values but it is not working. Please help me. The main concept of this method is when this is invoke dal is the keyword and it fetches that key values from the dynamodb. For that i used global also but not updating that global variable also.
I tried returning the value from the callback, as well as assigning the result to a local variable inside the function and returning that one, but none of those ways actually return the response (they all return undefined or whatever the initial value of the variable result is).**
function getLocation(){
let a=[];
const documentClient = new AWSS.DynamoDB.DocumentClient({region:"us-east-2"});
const params = {
TableName : "ProductLocation1",
Key :{
Product_Name : 'dal'
}
};
documentClient.get(params,(err,data)=>{
if(err){
console.log('error is',err);
}
console.log('data is : ',data);
global.a=Object.keys(data);
});
return global.a;
}
try {
const result = await documentClient.get(params).promise();
} catch (error) {
// handle error
}
You could get the result using the promise & await rather than the callback function. In this way, you don't need to store them in a local variable.
I am writing to the RealTime Database using a db file. The field generates a unique id key and I need to pass that key back to the parent file.
My db function:
export const doCreateCompany = name =>
db
.ref(`company`)
.push()
.set({ name });
My calling function is as follows:
db.doCreateCompany(companyName).then((e) => {
console.log("Key", e);
I need to write another function immediately following using the unique key. Any help is great. Thanks
If you want your function to still return a promise that resolves when the set() is complete, and also have access to the push ID, try something like this:
export const doCreateCompany = name => {
const ref = db.ref('company').push()
return ref.set({ name }).then(() => {
return Promise.resolve(ref.key)
})
}
Then you can call it like this:
doCreateCompany(name).then(key => {
// key is the push id here
})
You can separate the call to push() from the set(...):
export const doCreateCompany = name =>
var ref = db.ref(`company`).push();
var key = ref.key;
ref.set({ name });
Now you can return the key to whoever called the function.
My removeObjects function has me stummped.The function is suppose to syncronoulsy get a list of objects in an S3 bucket then asyncronously removes the objects. Repeat if the list was truncated, until the there are no more objects to remove. (AWS doesn't provide the total count of objects in the bucket and listObjects pages the results.)
What am I doing wrong / why doesn't my function work? The solution should exploit single thread and async nature of JS. For the bounty I am hoping for an answer specific to the module. The git repo is public if you want to see the entire module.
export function removeObjects(params: IS3NukeRequest): Promise<S3.Types.DeleteObjectsOutput> {
const requests: Array<Promise<S3.Types.DeleteObjectsOutput>> = [];
let isMore;
do {
listObjectsSync(params)
.then((objectList: S3.Types.ListObjectsV2Output) => {
isMore = objectList.ContinuationToken = objectList.IsTruncated ? objectList.NextContinuationToken : null;
requests.push(params.Client.deleteObjects(listObjectsV2Output2deleteObjectsRequest(objectList)).promise());
})
.catch((err: Error) => { Promise.reject(err); });
} while (isMore);
return Promise.all(requests);
}
export async function listObjectsSync(params: IS3NukeRequest): Promise<S3.Types.ListObjectsV2Output> {
try {
return await params.Client.listObjectsV2(s3nukeRequest2listObjectsRequest(params)).promise();
} catch (err) {
return Promise.reject(err);
}
}
Thanks.
The thing is that listObjectsSync function returns a Promise, so you need to treat it as an async function and can't just use a loop with it. What you need to do is to create a chain of promises while your isMore is true, I've done it using a recursive approach (I'm not pro in TS, so please check the code before using it). I also haven't tried the code live, but logically it should work :)
const requests: Array<Promise<S3.Types.DeleteObjectsOutput>> = [];
function recursive(recursiveParams) {
return listObjectsSync(recursiveParams).then((objectList: S3.Types.ListObjectsV2Output) => {
let isMore = objectList.ContinuationToken = objectList.IsTruncated ? objectList.NextContinuationToken : null;
requests.push(params.Client.deleteObjects(listObjectsV2Output2deleteObjectsRequest(objectList)).promise());
if (isMore) {
//do we need to change params here?
return recursive(recursiveParams)
}
//this is not necessary, just to indicate that we get out of the loop
return true;
});
}
return recursive(params).then(() => {
//we will have all requests here
return Promise.all(requests);
});