Query is not working with promise for Dynamo DB - node.js

I have a dynamo db table where I was able to insert data using node js via lambda. I am able to query from the console and I am also able to query using the cli. When using query with promise its erroring out with invoke error. Its not throwing any specific errors. IF I remove promise and run I can see that connection is successful to the db. I also tried ExpressionAttributeValues: {
":name": {"S":id}
},
even hard coding the value for id and same issue. What am I doing wrong??
import AWS from "aws-sdk"
const dyanamoDB = new AWS.DynamoDB.DocumentClient()
AWS.config.update({ region: "us-east-1" })
export const checkIFIDExist = async (id) => {
try {
const params = {
ProjectionExpression: "String1, String2",
IndexName: "String2",
KeyConditionExpression: "String2 = :name",
ExpressionAttributeValues: {
":name": id
},
TableName: 'my-table',
}
const data = await dynamoDB.query(params).promise()
console.log("Data:", data)
return "success"
}catch (err) {
throw new Error (`Failed query for ${id} `, err)
}
}
Error:
2022-08-16T20:24:09.210Z c2e0c093-2719-48b8-b0bb-4f38de3ac7b6 ERROR Invoke Error
{
"errorType": "Error",
"errorMessage": "Failed query for OE0K0I ",
"stack": [
"Error: Failed query for OE0K0I ",
" at checkIFStepFunctionIDExists (file:///var/task/src/dynamo-query.js:24:15)",
" at processTicksAndRejections (internal/process/task_queues.js:95:5)",
" at async Runtime.handler (file:///var/task/src/index.js:11:19)"
]
}

I basically deleted and trashed the project created new one and did the same stuff I had mentioned earlier in my post and instead of throwing error after catch statement console log it and I am getting the result I expected. I really couldn't tell what was I doing wrong before. #jarmond the error I posted above, I accidentally included a dependency while changing something and caused the error I provided. Thanks everyone for looking into the issue.

If the promise() function doesn't do what you expect it to do. It's worth noting, that you can actually also do the same thing with the standard Node.js promisify function.
import { DocumentClient } from "aws-sdk/clients/dynamodb";
import { promisify } from "util";
const docClient = new AWS.DynamoDB.DocumentClient()
...
const data = await promisify((cb) => docClient.query(params, cb))();
As #jarmod pointed out, there's probably something else going on though. I added some sidenotes to clarify some things that you may or may not already know.
Some sidenotes
Here are just some remarks which aren't entirely on topic but which can lead to confusion.
// don't do this, it doesn't do what you think it does.
throw new Error(`Failed query for ${id}`, err );
// this prints both a text and an object.
console.error(`Failed query for ${id}`, err);
// this throws just an error with a message
throw new Error(`Failed query for ${id}`);
// there is a 2nd argument which you can pass, which is an "options" parameter, which you can use to send a `cause` along.
throw new Error(`Failed query for ${id}`, { cause: err } );
PS:More details about it can be found it in the MDN documentation.
I haven't seen how you got it working without the promise, but if you did it like this, then it's not what you think.
try {
const params = { ... };
dynamoDB.query(params);
// this point is always reached
return "success"
}catch (err) {
// this point is never reached
}
Instead, without a promise, you would have to use a callback function.
const params = { ... };
dynamoDB.query(params, (err, data) => {
if(err) {
console.error(err);
return;
}
console.log("success", data);
});

Related

Mongoose connection to Mongo DB: Best practice for exception cases

It's not clear how to best trap errors using "mongoose.createConnection"
The Mogoose docs are pretty good at explaining how to trap an initial connection error when using "mongoose.connect" ... excerpt from docs., below:
mongoose.connect('mongodb://localhost:27017/test').
catch(error => handleError(error));
// Or:
try {
await mongoose.connect('mongodb://localhost:27017/test');
} catch (error) {
handleError(error);
}
However, in the case of "mongoose.createConnection" (neeed for more than one DB) ... this strategy does not seem to work. Here is what I've tried
const Db = mongoose.createConnection(dbUrl);
try {
await Db;
} catch (error) {
console.log(Error Connecting to DB");
}
which results in:
"SyntaxError: await is only valid in async functions and the top level bodies of modules"
Any pointers, please?
Thanks,
Tim.
Update following Anshu's post:
After making the changes, per Anshu, I then called the function
connectFunction()
and everything seemed OK, but only until the code reached the following line:
const Data = Db.model(`someCollection`, someSchema);
where it complained that "Db" is not defined. Presumably, because it is only scoped within the function? It feels like a chicken and egg situation. Any further guidance, please?
Update 11/18/22:
After making the following change:
async function connectFunction(){
try {
var db = await mongoose.createConnection(dbUrl);
} catch (error) {
console.log(Error Connecting to DB");
}
... same issue. I'm using VS Code and as a trivial observation, it colorizes variables, etc. In this case the variable "db" is "grayed out", i.e. it has recognized that something is wrong.
Update#2 11/18/22: I was able to make the "db is not defined error" go away once I realized I needed a return in the function. This is the latest code:
async function connectToBrandDb(url) {
try {
var Db = await mongoose.createConnection(url, {
keepAlive: true,
keepAliveInitialDelay: 300000,
serverSelectionTimeoutMS: 5000 });
}
catch (error) {
console.log(`Error Connecting To DB`);
}
return Db;
}
const brandDb = connectToBrandDb(dbUrl);
var Data = brandDb.model(`someCollection`, someSchema);
... and now looking at this from the terminal:
var Data = brandDb.model(`someCollection`, someSchema);
^
TypeError: brandDb.model is not a function
You should use async keyword before functions when you're trying to use await inside them. You cannot use await at the top (outside async function), so that's what the error is telling.
async function connectFunction() {
try {
const db = await mongoose.createConnection(dbUrl);
} catch (error) {
console.log("Error Connecting to DB");
}
}

Struggling with calling a function that uses promises in node.js

I am struggling with some code... The 2 examples below I would think would work the same but the second example throws an error? I am also struggling to figure out the error, it's not bubbling up? Admittedly I am not a seasoned node developer so any guidance would be much appreciated! If it's relevant the create method in the module is calling the sequelize create.
This works
var p1 = deliverabiltyConfigs.create2(cfgObject);
return Promise.all([p1]).then(function([res1]) {
res.json({result: res1})
});
This does not
deliverabiltyConfigs.create2(cfgObject).then(res1 =>{
res.json({result: res1})
})
Here is the function that I am calling in a controller module
exports.create2 = (dConfig) => {
DeliverabilityConfig.create(dConfig)
.then(data => {
return data
})
.catch(err => {
return {
message:
err.message || "Some error occurred while createing this config."
};
});
};
The create2 function always returns null, so neither invocation will work. Promise.all([p1]) hides the problem, returning a promise to perform an array of no promises.
create2(cfgObject).then(res1 =>{ attempts to invoke then() on null, generating a more obvious error. But neither way works.
Fix by deciding which promise syntax you want, using each as follows:
Using original promise syntax....
exports.create2 = dConfig => {
// note the return
return DeliverabilityConfig.create(dConfig)
.catch(err => {
const message = err.message || "Some error occurred while createing this config.";
return { message };
});
};
// caller
deliverabiltyConfigs.create2(cfgObject).then(result =>{
res.json(result);
})
With recent syntactic sugar...
exports.create2 = async (dConfig) => {
try {
// its fine to not await here, since the caller will await
// but just to illustrate how you might perform more async work here...
return await DeliverabilityConfig.create(dConfig);
} catch (err) {
const message = err.message || "Some error occurred while createing this config."
return { message }
}
}
// caller
var result = await deliverabiltyConfigs.create2(cfgObject);
res.json(result);
Use Promise.all() to run >1 promise concurrently. You've only got one promise in the OP, so no reason for it here.

Why does my Node.js AWS Lambda Function Only Work Every Other Time?

I have a Node.js Lambda function that makes a copy of a snapshot. The problem is that it only works every other time it is invoked. I think the problem is something to do with my not handling the asynchronous part of the code properly via async / await. But I can’t figure out what the problem is.
No error is ever caught. I've set the timeout value to 35 seconds which should be plenty. The logged output is identical, whether the snapshot is created or not.
But the snapshot is only created the second time I run the function!?
Can anyone tell me what is going on? Thanks.
Using the latest version of Node.js available in AWS.
var AWS = require("aws-sdk");
AWS.config.update({region: 'eu-west-2'});
const ec2 = new AWS.EC2();
const dst_account = "xxxxxxxxxxxx";
const src_account = "yyyyyyyyyyyy";
async function get_all_snapshots(owner){
console.log("*** get_all_snapshots() entered");
var params = {
Filters: [{
Name: "status", Values: [ "completed" ]
}],
MaxResults: 5, // Minimum value, as just need the last one.
OwnerIds: [ owner ]
}; // end params
try {
var snapshots = await ec2.describeSnapshots(params).promise();
return snapshots;
} catch (err) {
throw Error(err);
}
}
async function copy_snapshot(id){
console.log("*** copy_snapshot() entered", id);
var params = {
Description: "Snapshot created by Lambda Function",
DestinationRegion: "eu-west-2",
SourceRegion: "eu-west-2",
SourceSnapshotId: id
}; // end params
try {
var ss = await ec2.copySnapshot(params).promise();
console.log("*** ec2.copySnapshot() call completed. ss: ", ss); // THIS NEVER APPEARS In The LOGS
return ss;
} catch (err) {
console.log("!!! copy_snapshots() Throwing Error: ", err);
throw Error(err); // TODO: throwing for now, but could this be handled better?
}
}
exports.handler = async (event) => {
return get_all_snapshots(src_account)
.then(snapshots => {
copy_snapshot(snapshots.Snapshots[0].SnapshotId) // *** Add return here to fix ***
.then(r =>{
console.log("**new ss:", r); // THIS NEVER APPEARS IN THE LOGS
})
console.log("*** exports.handler() Ending");
})
.catch((err)=>{
console.log("*** get_all_snapshots() Error: ",err);
}); // End get_snapshots()
};

Unhandled Rejection (FirebaseError): No document to update

I'm still very new to coding so bear with me! I have followed a youtube course to build a note app and get a base to work from, but I'm now getting this error at random times when deleting the notes in firebase, hoping someone might be able to spot what's cooking here!
"Unhandled Rejection (FirebaseError): No document to update: projects/speakle-dc94b/databases/(default)/documents/notes/GdWPrQNxR3Z9TFMWmqOZ"
And it's referencing the node modules like so:
screenshot of the error in chrome
The code I have that interacts with firebase looks like this:
componentDidMount = () => {
firebase
.firestore()
.collection('notes')
.onSnapshot(serverUpdate => {
const notes = serverUpdate.docs.map(_doc => {
const data = _doc.data();
data['id'] = _doc.id;
return data;
});
console.log(notes);
this.setState({ notes: notes });
});
}
selectNote = (note, index) => this.setState({ selectedNoteIndex: index, selectedNote: note });
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.update({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
newNote = async (title) => {
const note = {
title: title,
body: ''
};
const newFromDB = await firebase
.firestore()
.collection('notes')
.add({
title: note.title,
body: note.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
const newID = newFromDB.id;
await this.setState({ notes: [...this.state.notes, note] });
const newNoteIndex = this.state.notes.indexOf(this.state.notes.filter(_note => _note.id === newID)[0]);
this.setState({ selectedNote: this.state.notes[newNoteIndex], selectedNoteIndex: newNoteIndex });
}
deleteNote = async (note) => {
const noteIndex = this.state.notes.indexOf(note);
await this.setState({ notes: this.state.notes.filter(_note => _note !== note) })
if(this.state.selectedNoteIndex === noteIndex) {
this.setState({ selectedNoteIndex: null, selectedNote: null});
} else {
this.state.notes.lenght > 1 ?
this.selectNote(this.state.notes[this.state.selectedNoteIndex - 1], this.state.selectedNoteIndex - 1) :
this.setState({ selectedNoteIndex: null, selectedNote: null });
}
firebase
.firestore()
.collection('notes')
.doc(note.id)
.delete()
.then(function() {
console.log("Document successfully deleted!");
}).catch(function(error) {
console.error("Error removing document: ", error);
});
}
}
It simply means that there is no document of that name to be uploaded.
you could either use set() or add() to add the document because it is not present.
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.update({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
replace the above code with this
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.add({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
or
noteUpdate = (id, noteObj) => {
firebase
.firestore()
.collection('notes')
.doc(id)
.set({
title: noteObj.title,
body: noteObj.body,
timestamp: firebase.firestore.FieldValue.serverTimestamp()
});
}
I was working with something like this only especially in Cloud Functions and while writing an endpoint for doing some operation I came across the below-quoted error.
I was trying to read a document in a collection and if it existed I was trying to write a new doc into another collection. So it was kind of a nested code.
A piece of my code.
const firstDocRef = db.collection('myFirstCollection').doc('myDocName');
const existDoc = firstDocRef.get()
.then((resDoc)=>{
if(resDoc.exists)
{
db.collection('mySecondCollection').add({
.
.
.
.
.
orderCreatedAt:Firestore.FieldValue.serverTimestamp()
})
.then((new_doc)=>{
return res.status(200);
// return 200 ok what we were trying to achieve has completed.
})
.catch(()=>{
console.log("Log, as write failed somehow");
return res.status(500);
// return a 500 internal server error occurred
});
}
else
{
console.log("My first condition wasn't met, so logged it");
return res.end();
// properly terminate the processing of request
}
});
/*.catch((err)=>{
console.log("Our first doc presence check couldn't complete and hence I arrived here, log it");
res.writeHead(500);
return res.end();
// again give back 500 to client
});*/
UnhandledPromiseRejectionWarning: ReferenceError: Firestore is not defined
UnhandledPromiseRejectionWarning: Unhandled promise rejection.
This error originated either by throwing inside of an async function without a catch block
Now I am also new to Firebase but I came across this and somehow solved it.
So I was not getting the above error if I was putting in a catch block in get() document.
Strange huh!
Removed the catch block by commenting it. Got this error.
Now, this is a haywire error, it says the catch is not there, but we did it on purpose.
So I began searching, came across this question here on stack overflow and saw it's still unanswered. Searched and read the documentation myself.
I would like to tell you that this isn't because of any Firestore Security Rules, or anything else. Because I came across some guesses around these notions too while searching for an answer.
The common thing we all are doing here is that we are trying to achieve the ServerTimeStamp at FireStore
I would like to bring your notice to my imports in my node cloud function code.
const functions = require('firebase-functions');
const express = require('express');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
const db = admin.firestore();
So you see, I am using the new way of getting the permission to use Firestore because I am trying to establish a cloud function.
Now this is the documentation reference provided by Google: Click here!
The right syntax proposed by above API reference is
Firestore.FieldValue.serverTimestamp()
It is the culprit, it isn't providing me any timestamp and if there is not a catch block unhandled promise error is occuring and no error is being shown while debugging, it just doesn't work.
I did this, Solution part:
Even after those imports in my Node Program, I imported this:
const {Firestore} = require('#google-cloud/firestore');
Now all I did was that I used the statement in the timestamp field as
Firestore.FieldValue.serverTimestamp()
Exactly as mentioned and even used a catch block just in case any other problem occurs while at production. That is using db constant to do all the Database Transactional Stuff and just for serverTimeStamp I have to bring in new import.
And it worked, I guess require('#google-cloud/firestore') statement imported as {FireStore} brings in all the things that are required for the FieldValue thing to use as a reference.
I hope it helps any new person looking for it and saves a lot of time which I wasted searching for a solution.
I have verified it by running the cloud function on firebase emulator.
You could simply do it like this
You
Get it
If it exsits: you update it
If it doesn't exist, you set it.
const docRef = this.db.collection("users_holdings").doc(userId);
docRef.get().subscribe((doc) => {
if (doc.exists) {
docRef.update({
stockHoldings: stockHoldings,
});
} else {
docRef.set({
stockHoldings: stockHoldings,
});
}
});

headObject never throws an error when object doesn't exist

I am currently trying to check if a file exists, using aws-sdk for Amazon s3 (More precisely, the function headObject).
As I could read literally everywhere, this is the function that is supposed to be used when trying to check if a file exists (In order to then get its URL through getSignedUrl), however I can't bring it to work.
It seems that, no matter what I do, the function s3.headObject tells me that the object exists. I tried checking for existing items, non-existing items, and even in non-existing buckets : All of these got me the exact same output. I tried different ways of calling the function (Async or not, using its callback or not), but no difference.
Here is how I realize my call to the function :
var params = {
Bucket: 'BUCKET NAME',
Key: ""
}
// Some more code to determine file name, confirmed working
params.Key = 'FILE NAME'
try {
s3.headObject(params)
// Using here the file that is supposed to exist
} catch (headErr) {
console.log("An error happened !")
console.log(headErr)
}
I also tried using a callback : However, it seems that said callback was never entered. Here is what my code looked like :
var params = {
Bucket: 'BUCKET NAME',
Key: ""
}
// Some more code to determine file name, confirmed working
params.Key = 'FILE NAME'
s3.headObject(params, function(err: any, data: any) {
console.log("We are in the callback")
if (err) console.log(err, err.code)
else {
// Do things with file
}
})
console.log("We are not in the callback")
With this code, "We are in the callback" never appeared, while "We are not in the callback" was correctly appearing.
No matter what I do, no error is ever caught.
From what I understand from how the function is supposed to work, in case the file doesn't exist, it is supposed to throw an error (Then caught by my catch), thus allowing me not to create false URLs with the getSignedUrl function.
What am I doing wrong here ?
Thank you all for your answers. If you have additional questions, I'll be more than glad to answer the best I can.
This is the right way to check object existence using async/await syntax:
// Returns a promise that resolves to true/false if object exists/doesn't exist
const objectExists = async (bucket, key) => {
try {
await s3.headObject({
Bucket: bucket,
Key: key,
}).promise(); // Note the .promise() here
return true; // headObject didn't throw, object exists
} catch (err) {
if (err.code === 'NotFound') {
return false; // headObject threw with NotFound, object doesn't exist
}
throw err; // Rethrow other errors
}
};
I do try out the syntex but it doesn't work. It is in my lambda function.
params and params2 are predefined set of bucket and key.
var url = s3.getSignedUrl('getObject', params);
const objectExist = async (par) => {
try{
console.log(s3.headObject(par).response); //I honestly couldn't find any
//section in the resoonse,
// that make a DNE file different from a existing file.
const ext = await s3.headObject(par).promise((resolve, reject) =>{
console.log("bbbbbbbbbbb");
if(err) { // it keeps saying the err is not global variable.
//I am wondering why this is not defined.
//Really had no idea of what else I could put as condition.
console.log("aaaa"); //never reach here.
return reject(false);}
return resolve(true);
});
console.log(ext); //always true.
if(!ext){url = url = s3.getSignedUrl('getObject', params2, callback); }
}catch(err){
console.log("reeeeeeeeee"); //when the method failed it execute.
url = s3.getSignedUrl('getObject', params2, callback);
console.log(url); //even though I am sure that params2 are valid key, but url in the log always returned undefined.
}
};
objectExist(params);

Resources