I am new to MongoDB/backend and have spent a couple of hours on this one. Hoping/expecting there is a silly mistake here!
Essentially, in the process of adding new users to the db, I want to check one of the specified values (the "position code(s)") against a list of acceptable codes. I retrieve these acceptable position codes via an async call and then use an underscore method to compare the specified position code(s) against the acceptable ones. If any are found, I throw an error.
Multiple users can be added at a time and each user can have multiple roles. So both of these are array types.
The most recent thing i've tried is simply using the "throw new Error()" methodology since my understanding is that this should get picked up by the Express middleware. But it's not working. The error message I specify in the error object IS getting printed in the console, but in a "UnhandledPromiseRejectionWarning" error. When running this in Postman, Postman just doesn't return anything (basically appears like it's loading and then says could not get a response").
I also tried wrapping this in a try catch block.
I also tried passing the error to a next() method.
router.route('/add').post(async (req, res) => {
let
newAdults = [],
roleCodes = await Role.find()
.then(roles => _.pluck(roles, "code"))
.catch(err => res.status(400).json('Error: ' + err));
req.body.forEach((el, id) => {
let
firstName = el.firstName,
lastName = el.lastName,
emailAddress = el.emailAddress,
username = el.username,
roles = el.roles;
// one thing we need to lookout for is if an invalid role code is being used.
// here we compare role codes associated with the new adult and all approved
// role codes. this will identify invalid role codes
let invalidCodes = _.difference(_.pluck(roles, "code"), roleCodes)
if (invalidCodes.length > 0)
throw new Error(`Role code(s) ${invalidCodes} for ${firstName} ${lastName} are invalid.`)
Ultimately I want to just return the message I have in my error object.
Since you are throwing the error without handling it in your API, it remains in incomplete state and you get could not get a response error in POSTMAN.
You can try this,
Create new error object array and store the error messages in that object rather then throwing the error.
You can return this error object array in response in case this array has values.
Here is the updated snippet:
router.route('/add').post(async (req, res) => {
let
newAdults = [],
roleCodes = await Role.find()
.then(roles => _.pluck(roles, "code"))
.catch(err => res.status(400).json('Error: ' + err));
let errorArray = [];
req.body.forEach((el, id) => {
let
firstName = el.firstName,
lastName = el.lastName,
emailAddress = el.emailAddress,
username = el.username,
roles = el.roles;
// one thing we need to lookout for is if an invalid role code is being used.
// here we compare role codes associated with the new adult and all approved
// role codes. this will identify invalid role codes
let invalidCodes = _.difference(_.pluck(roles, "code"), roleCodes)
if (invalidCodes.length > 0) {
errorArray.push(`Role code(s) ${invalidCodes} for ${firstName} ${lastName} are invalid.`);
}
if (errorArray && errorArray.length) {
res.status(400).json(errorArray)
}
Related
this is the error that I get. I checked multiple times that the paths that I indicate are actually pointing at something in the database. I'm kinda going crazy about why this is not working, so help will be appreciated. (the same error is given by both functions, two times every invocation of the function)
this is my code:
exports.onCreatePost = functions.firestore
.document('/time/{userid}/date/{postId}')
.onCreate (async (snapshot, context) => {
const postCreated = snapshot.data();
const userID = context.params.userid;
const postID = context.params.postId;
//get all the followers who made the post
const userFollowerRef = admin.firestore().collection('time').doc(userID).collection('followers');
const querySnap = await userFollowerRef.get();
//add the post in each follower timeline
querySnap.forEach(doc => {
const followerid = doc.id;
admin.firestore().collection('time').doc(followerid).collection('timelinePosts').doc(postID).set(postCreated);
})
});
//when a post is updated
exports.onUpdatePost = functions.firestore
.document('/time/{userid}/date/{postid}')
.onUpdate(async (change, context) => {
const postUpdated = change.after.data();
const userID = context.params.userid;
const postID = context.params.postId;
//get all the followers who made the post
const userFollowerRef = admin.firestore().collection('time').doc(userID).collection('followers');
const querySnap = await userFollowerRef.get();
//Update the post in each follower timeline
querySnap.forEach(doc => {
const follower = doc.id;
admin.firestore().collection('time').doc(follower).collection('timelinePosts').doc(postID)
.get().then(doc => {
if (doc.exists) {
doc.ref.update(postUpdated);
}
});
});
});
I personally don't know how to log each variable and did not find how to do it online. I'll keep searching but in the mindtime I can share my extensive logs that from my interpretation are not very useful but maybe is just because I'm inexperienced.
this is the error log
enter image description here
In function exports.onUpdatePost ...you're likely trying to access documentPath null (or something alike that). Add logging, this permits to log custom debug information into the log which you've screenshotted. When logging every step of the procedure, it's a whole lot easier to determine what is happening and why - or why not, when skipping something. Alike this you should be able to solve the issue on your own. My functions logging actually utilizes emojis, because UTF-8 is being supported: ✅❌ (visual indicators make the log more readable).
The cause seems to be one of these instructions:
admin.firestore().collection('time') // it is being assumed that it exists
.doc(userID) // argument or return value may be null
.collection('followers') // return value may be null
Or:
admin.firestore().collection('time') // it is being assumed that it exists
.doc(follower) // argument or return value may be null
.collection('timelinePosts') // return value may be null
.doc(postID) // argument or return value may be null
eg. one first has to check if follower != null or empty and if the desired document even exists. The same goes for userID and .doc(userID) (the seemingly "own" timeline).
if (follower != null && follower.length > 0) {
admin.firestore().collection('time').doc(follower).get().then(timeline => {
functions.logger.log('timeline: ' follower + ', ' + timeline.exists);
if (timeline.exists) {
} else {
}
});
}
documentPath == null comes from .doc() + userID, followerid, follower or postID.
Is there a condensed way to do this validation on nodejs
//inbound express request
{ "myVal":"abcdefghij" }
/* first test if value exists,
then if it fails a regex return an error & exit function
if pass then continue
*/
// ... logic to get express.post(...=> {
if (( req.body.myVal == null ){
// no value, send error response and exit function
}else{
const inVar = req.body.myVal;
if ( inVar ){
const regex = /^([a-j0-9]){1,11}$/;
if(regex.test(inVar ) == false){
res.send({ error: "failed regex" });
res.end();
return;
}
... else continue
Here are a few options:
Use express-validator or a similar package to validate your endpoint params and/or body in the endpoint configuration. This nicely separates validation from the logic of the endpoint and is extensible to your other endpoints without having to maintain code in each one.
Check out the package here:
Express-validator
Specifically, what you might want is their custom validator:
Express-validator's custom validation docs
Example usage:
const { body } = require('express-validator');
const MY_REGEX = /^([a-j0-9]){1,11}$/;
app.post('/my-endpoint',
body('myVar').custom(value => {
if(!MY_REGEX.test(value)) Promise.reject("Error: invalid value for 'myVar.'")
}),
(req, res) => {
// Business logic only, no validation
},
);
You could even take the function that's inside .custom(), and put it in another file, import it here, and reuse it elsewhere.
If you want to do the validation in the handler, there are a few ways to make your code more brief/safe/nice.
Set your regexp variable in a different file or as a constant at the top of the file (descriptively) or in a different constants/utils file.
"Error gate" by returning after you find an error. Avoid nested if/else.
Is the error the same for missing value and incorrect value? If so, your regexp will error appropriately for undefined, empty string, and null anyway. consider combining them in the same if statement.
Example:
// imports
// ...
const MY_DESCRIPTIVE_REGEX_NAME = /^([a-j0-9]){1,11}$/;
//...
(req, res) => {
if(!MY_DESCRIPTIVE_REGEX_NAME.test(req.body.myVal) {
res.status(400);
return res.send({ error: "MyVal must be between 1 and 11 characters and contain only numbers and letters between 'a' and 'j'."});
}
// rest of logic, not in an if statement
}
I'm new to node.js and DialogFlow. I am using DynamoDB to store data and I'm creating skills on Google. I'm trying to write a code to retrieve a specific item on that table.
I've got it working to show all items where ID is equal = 1, but how would I make it so I can just get attribute 'name'?
My idea is a user provides an id then the code will retrieve name where id was 1 and store that as a variable and use agent.add('hello $(name)'); to display it back as speech to the user.
function readdata(agent){
let dbread = new aws.DynamoDB.DocumentClient();
const id = agent.parameters.id;
let read = function(){
var parameters = {
TableName:"Dynamodb",
Key:{
"id":id
}
};
dbread.get(parameters, function(err,data){
if(err){
console.log("error",JSON.stringify(data,null,2));
}else{
console.log("success",JSON.stringify(data,null,2));
}
});
agent.add(`hello ${name}`);
};
read();
}
Once you have the data back from the get() call, the data object will contain an Item attribute. The value of this attribute will be another object that contains the attribute/value pairs for this record, or be empty if the record isn't found.
The debugging you have in place that shows JSON.stringify(data) should show this.
Assuming you knew all the fields were there, you could do something like
const name = data.Item.name;
a more robust way using current JavaScript would be to make sure everything was assigned, otherwise return undefined at any point. So something like this would work
const name = data && data.Item && data.Item.name;
However - you will have a problem doing this with Dialogflow
You don't show which Dialogflow library you're using, but most of them require you to return a Promise to indicate that it needs to wait for asynchronous calls (such as the call to DynamoDB) to complete. You're using get() with a callback function instead of a Promise. So you need to do one of the following:
Wrap the call in a Promise
Since get() returns an AWS.Request you can use the promise() method of this to get a Promise that you can return and which has then portions that generate the response - similar to how you're doing your callbacks now.
Under this scheme, your call might look something like this (untested):
return dbread.get(parameters).promise()
.then( data => {
console.log("success",JSON.stringify(data,null,2));
const name = data && data.Item && data.Item.name;
if( name ){
agent.add( `Hello ${name}` );
} else {
agent.add( "I don't know who you are." );
}
})
.catch( err => {
console.log("error",JSON.stringify(data,null,2));
agent.add( "There was an error" );
});
async changeCarOwner(ctx, carNumber, newOwner) {
const carAsBytes = await ctx.stub.getState(carNumber);
if (!carAsBytes || carAsBytes.length === 0) {
throw new Error(`${carNumber} does not exist`);
}
const car = JSON.parse(carAsBytes.toString());
car.owner = newOwner
await ctx.stub.putState(carNumber, Buffer.from(JSON.stringify(car)));
}
}
I keep getting an error: Unexpected end of JSON input. Why? I am trying to update an existing key-value pair in couchDb using the above code.
This error happens at this line:
const car = JSON.parse(carAsBytes.toString());
It is due to the fact that carAsBytes.toString() does not evaluates to a properly formatted JSON string. The code you show seems fine, but the error is coming from elsewhere in your code.
Debugging tip: use the debugger statement to examine variables before the faulty line, simply add a console.log(carAsBytes.toString()) before it.
I've run into an issue with a firebase function, written in TypeScript for the Node.js environment. I have a function with https-endpoint where the client can send data that needs to be stored in the database. In order to know which objects already has been added to the database, it first reads a path ("lookup") that has a simplified registry of the object (lookup/:objectId/true). Then it makes the values that should be updated at the actual object path and updates these in the database.
The function is as follows:
export const scrapeAssignments = functions.https.onCall((data, context) => {
const htmlString = data.htmlString
// const htmlString = fs.readFileSync(testPath.join(__dirname, "./assignmentListExample.html"), { encoding: 'utf8' })
if (!(typeof htmlString === 'string') || htmlString.length === 0) {
throw new functions.https.HttpsError('invalid-argument', 'The function must be called with one argument "htmlString"');
}
const userId = getUserIdFromCallableContext(context)
console.log("userId", userId)
let newAssignments: ScrapedAssignment[] = []
try {
newAssignments = parseAssignment(htmlString)
} catch (e) {
const error = <Error>e
throw new functions.https.HttpsError('not-found', 'parsing error: ' + error.message)
}
return admin.database().ref("lookup").child(userId).child("assignments")
.once("value", lookupSnapshot => {
const oldAssignmentsLookup = lookupSnapshot.val() || {}
const newAssignmentsLookup = makeLookup(newAssignments)
// 1. Create update values for scraped assignment data
let scrapedAssignmentUpdateValues = newAssignments.reduce((prev, current) => {
const prefixed = prefixObject(current.id + "/", current)
return { ...prev, ...prefixed }
}, {})
// 2. Use the diff from the two lookups to find old assignments to delete
const removeAssignmentsValues = {}
Object.keys(oldAssignmentsLookup).forEach(assignmentId => {
if (isUndefined(newAssignmentsLookup[assignmentId]))
removeAssignmentsValues[assignmentId] = null
})
// 3. Add other user values to newly found assignments
Object.keys(newAssignmentsLookup).forEach(assignmentId => {
if (isUndefined(oldAssignmentsLookup[assignmentId])) {
const doneKey = assignmentId + "/done"
scrapedAssignmentUpdateValues[doneKey] = false
}
})
const combinedValues = { ...scrapedAssignmentUpdateValues, ...removeAssignmentsValues }
return admin.database().ref("userAssignments").child(userId).update(combinedValues)
}).catch(reason => {
throw new functions.https.HttpsError('internal', 'Database reason: ' + reason)
})
})
I see that the data is written to the right place and everything seems to go as expected, except that when I call the function from an iOS app, It returns an "Internal"-error
When I check the function logs in the cloud console, I see the following error:
assignment-scrapeAssignments uolk47opctna Unhandled error RangeError:
Maximum call stack size exceeded at Function.mapValues
(/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13395:23)
at encode
(/user_code/node_modules/firebase-functions/lib/providers/https.js:204:18)
at
/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
at
/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
at baseForOwn
(/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
at Function.mapValues
(/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)
at encode
(/user_code/node_modules/firebase-functions/lib/providers/https.js:204:18)
at
/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13400:38
at
/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:4925:15
at baseForOwn
(/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:3010:24)
at Function.mapValues
(/user_code/node_modules/firebase-functions/node_modules/lodash/lodash.js:13399:7)
All I can read out of this, is that it's an "RangeError: Maximum call stack size exceeded"-error, and something happens at Function.mapValues. As I can read from this SO question it seems to be an issue with a reading and writing to the same location at the same time. But I'm quite sure that I'm not doing that here.. Plus everything seems behave like it should, except for the actual error.
When combinedValues is updated, it is an object with ~300 key/value pairs, is that a problem?
Looks like your function run out of memory, every cloud function has allocated memory to its execution.
You can try to increase the stack size from its default value (256MB) up to 2GB by going to:
Functions-> Dashboard, then go to your problematic function and click on the right menu "Detailed usage stats":
then on your function details in google cloud dashboard click edit:
then increase the value of "memory allocated" to 2GB (or lower sufficient value):
note: you should keep in mind that when your data growth you may exceed the highest limit so consider this when querying in the cloud functions