I am using nightmare.js to get the price for all the products in the database. When I try running just
const data = await monitor.find({})
data.forEach(async (monitor1) => {
const url = monitor1.Link
console.log(url)
})
It works, and console.logs many links, but when I try
data.forEach(async (monitor1) => {
const url = monitor1.Link
console.log(url)
try {
const priceString = await nightmare.goto(monitor1.Link)
.wait(".a-price-whole")
.evaluate(() => document.getElementsByClassName("a-price-whole").innerText)
.end()
console.log(priceString)
} catch (e) {
throw e
}
})
It returns undefined for all the prices. The class is called a-price-whole as shown here
Why is this happening?
Information is less, but I believe you are using same nightmare instance.
Refer this answer
Related
I have difficulties returning a value using a callable firebase function, while the same code works fine when it is a httprequest.
So what I am doing, is getting some user data, then getting some other data (a list of vessels) and then only return the vessels the user has edit rights for. I am very sure the accessibleVessels object holds some json data: I changed the function in a functions.https.onRequest firebase function and it went fine.
exports.getFormData = functions.https.onCall( (data, context) => {
const uid = context.auth?.uid.toString;
try {
const user = admin.firestore().
doc("users/"+uid)
.get().then((doc: any) => {
return doc.data();
});
const vessels = admin.firestore().
collection("vessels").get()
.then(mapSnapshot((doc: any) => doc.data()));
const accessibleVessels = vessels.filter((vessel: any) => {
return user.hasEditRights.some((right: any) => {
return vessel.name === right;
});
});
return accessibleVessels;
} catch (error) {
console.log(error);
return {"error": true};
}
});
When I run this I get:
Data cannot be encoded in JSON. [Function (anonymous)]
Looking in the documentation I understand that I do need to return json or a promise. I read other answers about this (returning a promise) but don't see how this would work in my example: in the examples I find not much is done with the data, it's just returned. I want to put the data in variables instead of chaining everything, so I can combine both. How should I do this?
The easiest way to fix this is using async/await:
exports.getFormData = functions.https.onCall(async(data, context) => { // 👈
const uid = context.auth?.uid.toString;
try {
const user = await admin.firestore() // 👈
.doc("users/"+uid)
.get().then((doc: any) => {
return doc.data();
});
const vessels = await admin.firestore() // 👈
.collection("vessels").get()
.then(mapSnapshot((doc: any) => doc.data()));
const accessibleVessels = vessels.filter((vessel: any) => {
return user.hasEditRights.some((right: any) => {
return vessel.name === right;
});
});
return accessibleVessels;
} catch (error) {
console.log(error);
return {"error": true};
}
});
I also recommend reading the MDN documentation on async/await, the Firebase documentation on sync, async, and promises and Doug's awesome series on JavaScript Promises in Cloud Functions.
I'm new to fetching and posting data using an API, and I can't work out how to do something once my Post has been completed.
I have a function that calls the API with the Post data. I need to set the loading state to false once the Post has been completed. Everything works apart from that, the data gets sent to Mongo, I just need to turn off my loading spinner once it has completed.
How do I do this, please?
This is how I'm trying to do it:
const postData = async () => {
setLoading(true)
await axios.post('/api/addData',form)
.then(response => {
setLoading(false)
})
}
And this is the API bit:
import { connectToDatabase } from "util/mongodb"
export default async (req, res) => {
const { db } = await connectToDatabase()
await db
.collection("posts")
.insertOne(req.body);
}
There is two potential problem in your code, first you're not sending any data back to the front in your backend code. Usually you send back the id of the inserted element (It can be usefull to do some mutation in your front), you'll also need to try catch your call to the db to notify that something went wrong to the front end side :
import { connectToDatabase } from "util/mongodb"
export default async (req, res) => {
try {
const { db } = await connectToDatabase()
const insertedPost = await db
.collection("posts")
.insertOne(req.body);
res.status(201).send(insertedPost.insertedId);
// again it's up to you to know what can be usefull to your front-end to use
// Look at http status code online to know what's the best fit
} catch (err) {
res.status(500).send(err.message);
// send whatever that can be usefull for your front end to handle the error
}
}
In your front-end code you're using await with .then, it's weird usage. You can put your setLoading(false) after the await without the .then but you'll still need to try catch it. What I prefer to do is using the finally block to stop loading, so if my api call fail the loading is still stopped :
const postData = async () => {
setLoading(true)
try {
const response = await axios.post('/api/addData',form)
// do something with response
} catch (err) {
// notify user that something went wrong
} finally {
setLoading(false);
}
}
const postData = () => {
setLoading(true)
axios.post('/api/addData',form)
.then(response => {
// do something with response
})
.catch((err) => {
// notify user that something went wrong
})
.finally(() => {
setLoading(false);
})
}
Building a NodeJS REST API.
Trying to send load data from FireBase collection, then sending it to the user (as API response).
Looks like the problem is that it's not waits for the firebase fetch to resolve, but send back a response without the collection data. (tried to use ASYNC-AWAIT but its not working)
exports.getChatMessages = async (req, res, next) => {
const chatId = req.params.chatId
const getChatData = () => {
db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot((snapshot) => {
snapshot.docs.forEach(msg => {
console.log(msg.data().messageContent)
return {
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
}
})
})
}
try {
const chatData = await getChatData()
console.log(chatData)
res.status(200).json({
message: 'Chat Has Found',
chatData: chatData
})
} catch (err) {
if (!err.statusCode) {
err.statusCode(500)
}
next(err)
}
}
As you can see, I've used 2 console.logs to realize what the problem, Terminal logs looks like:
[] (from console.logs(chatData))
All messages (from console.log(msg.data().messageContent))
Is there any way to block the code unti the firebase data realy fetched?
If I correctly understand, you want to send back an array of all the documents present in the messages subcollection. The following should do the trick.
exports.getChatMessages = async (req, res, next) => {
const chatId = req.params.chatId;
const collectionRef = db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc');
try {
const chatsQuerySnapshot = await collectionRef.get();
const chatData = [];
chatsQuerySnapshot.forEach((msg) => {
console.log(msg.data().messageContent);
chatData.push({
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
});
});
console.log(chatData);
res.status(200).json({
message: 'Chat Has Found',
chatData: chatData,
});
} catch (err) {
if (!err.statusCode) {
err.statusCode(500);
}
next(err);
}
};
The asynchronous get() method returns a QuerySnapshot on which you can call forEach() for enumerating all of the documents in the QuerySnapshot.
You can only await a Promise. Currently, getChatData() does not return a Promise, so awaiting it is pointless. You are trying to await a fixed value, so it resolves immediately and jumps to the next line. console.log(chatData) happens. Then, later, your (snapshot) => callback happens, but too late.
const getChatData = () => new Promise(resolve => { // Return a Promise, so it can be awaited
db.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot(resolve) // Equivalent to .onSnapshot((snapshot) => resolve(snapshot))
})
const snapshot = await getChatData();
console.log(snapshot)
// Put your transform logic out of the function that calls the DB. A function should only do one thing if possible : call or transform, not both.
const chatData = snapshot.map(msg => ({
authorID: msg.data().authorID,
messageContent: msg.data().messageContent,
timeStamp: msg.data().timeStamp,
}));
res.status(200).json({
message: 'Chat Has Found',
chatData
})
Right now, getChatData is this (short version):
const getChatData = () => {
db
.collection('chats')
.doc(chatId)
.collection('messages')
.orderBy('timeStamp', 'asc')
.onSnapshot((snapshot) => {}) // some things inside
}
What that means is that the getChatData function calls some db query, and then returns void (nothing). I bet you'd want to return the db call (hopefully it's a Promise), so that your await does some work for you. Something along the lines of:
const getChatData = async () =>
db
.collection('chats')
// ...
Which is the same as const getChatData = async() => { return db... }
Update: Now that I've reviewed the docs once again, I see that you use onSnapshot, which is meant for updates and can fire multiple times. The first call actually makes a request, but then continues to listen on those updates. Since that seems like a regular request-response, and you want it to happen only once - use .get() docs instead of .onSnapshot(). Otherwise those listeners would stay there and cause troubles. .get() returns a Promise, so the sample fix that I've mentioned above would work perfectly and you don't need to change other pieces of the code.
I have a use case where I need to call an external API, parse the JSON that is returned and populate a form field in a web page all within a Selenium script written using Node JS.
Something like this:
// in Selenium script get the form field
let inputElement = await getElementById(driver, "my-id");
// then call an API including callback function
// in the callback function with the JSON response from the API
const myText = response.data.text;
await inputElement.sendKeys(myText,Key.ENTER);
I actually not even sure where to start with this - because I would be adding asynchronous code (the API call and waiting for the response in the callback) to the existing asynchronous code that is running as part of the Selenium script. And I need to not lose references to the web driver and the input element.
Some advice and recommendations to get me going would be very helpful.
If you are using node's inbuild https module the you can do something like this..
const { Builder, By, Key, until } = require("selenium-webdriver");
const https = require("https");
(async function example() {
let driver = await new Builder().forBrowser("chrome").build();
try {
await driver.get("http://www.google.com/ncr");
await https.get("https://jsonplaceholder.typicode.com/users/1", (resp) => {
let data = "";
resp.on("data", (chunk) => {
data += chunk;
});
resp.on("end", async () => {
// console.log(JSON.parse(data)["name"]);
await driver
.findElement(By.name("q"))
.sendKeys(JSON.parse(data)["name"], Key.RETURN);
});
});
await driver.wait(until.titleContains("- Google Search"), 1000);
} finally {
await driver.quit();
}
})();
Or if you are already using library like axios, then you can do something like this
const { Builder, By, Key, until } = require("selenium-webdriver");
const axios = require("axios");
(async function example() {
let driver = await new Builder().forBrowser("chrome").build();
try {
await driver.get("http://www.google.com/ncr");
const { data } = await axios.get(
"https://jsonplaceholder.typicode.com/users/1"
);
await driver.findElement(By.name("q")).sendKeys(data["name"], Key.RETURN);
await driver.wait(until.titleContains("- Google Search"), 1000);
} finally {
await driver.quit();
}
})();
Hope this is what you are looking for..
I'm working on a node.js backend project that uses a MongoDb database. After I query the database and received the data it will not display the data in my browser using res.send(). I also tried res.json(). However, the data does display on my console,but just will not display in postman or my browser. Is the query data from mongoDB not json or an array? I did a little reading and it states it's a cursor pointing to the data. Can this data not be converted to display in a broswer?
mycode as well as console and browser display
ProductRouter.js
router.get('/', (req, res) => {
const products = allProducts.plantProducts();
setTimeout(() => {
if (products === "400") {
res.status("400").send("Error querying database");
}else{
console.log(products);
res.status("200").send(products);
}
}, 1000);
ProductController.js
async function plantProducts(){
try {
const products = await getProducts();
return products;
} catch(err) {
let code = "400";
return code;
//res.status('400').send(err.message);
}
}
plantProducts();
//Search mongodb for all products.
function getProducts() {
return new Promise((resolve, reject) => {
const products = Product
.find()
resolve(products);
});
}
The problem is that products yields a promise which express' res.json will not automatically await and convert to a json-response. You need await the promise and then send the response. Something like:
router.get('/', async (req, res) => {
const products = await allProducts.plantProducts();
res.json(products);
});