I have written the following code to return a row from a Dynamodb table based on a key called id.
var db = new AWS.DynamoDB();
var params = {
AttributesToGet: [
"dealername"
],
TableName : "syv_dealer",
Key : {
"id" : {
"S" : "1"
}
}
}
db.query(params, function(err, data) {
if (err) {
console.log(err); // an error occurred
} else {
console.log(data); // successful response
}
});
My code to write the data to the table worked fine, but the above code keeps generating the following error: Unexpected key 'Key' found in params
The table exists and there is an id attribute with a value of "1" in the table.
I have looked at all the examples I could find and have changed the params statement a hundred times, but nothing has worked. I am sure it is something simple, but any help would be appreciated.
This is failing because Key is an invalid parameter for submitting a request to that API. Look at the DynamoDB Query syntax to determine what attributes you can set.
The proper way to perform this query would be with a KeyConditionExpression which would look something like:
var params = {
TableName : "syv_dealer",
KeyConditionExpression: "id = :v1",
ExpressionAttributeValues: {
":v1": {"S": "1"}
},
ProjectionExpression: "dealername"
}
Using ProjectionExpression instead of AttributesToGet is also the modern way of specifying attributes.
Thank you, I had changed things so much I hadn't realized I had changed it from a getItem to Query.
The crux of my problem all along was that the function call is asynchronous, so I wasn't being patient and letting the callback execute.
I am new to Node.js, so I am trying to figure out how to check the credentials on a web service call and not have it continue until I have authenticated the requestor. Is there a way on the getItem call to have it be synchronous? I realize I could embed the logic that follows within the getItem callback, but I was really trying to use the same security routine for every endpoint.
Thoughts?
Related
I am setting up my first REST API to query a Postgres database I set up. I have two CRUD methods that currently work that query for all rows in a table and for rows where ID = something, respectively.
The problem I'm having occurs when trying to query when the request parameter is a String. Here is the error I'm getting:
error: invalid input syntax for type integer: "NaN"
Here is how I've set up my GET route and endpoint URL:
const getShowByTitle = (request, response) => {
const title = request.params.title
pool.query('SELECT * FROM show WHERE title = $1', [title], (error, results) => {
if (error) {
throw error
}
response.status(200).json(results.rows)
})
}
app.get('/show/:title', getShowByTitle)
Expected result is that sending a GET request using a show title as a parameter (String) returns a JSON response of just that show.
Any help or direction to some useful resources would be greatly appreciated. Thank you.
There are some issues here, first in SQL the name of the tables should be in plural "shows", second you are making the select without quotes, you need something like:
"SELECT * FROM show WHERE title = '$1'"
Third, since the user can use uppercase and down cases you need a more robust way to search for text using LIKE, ILIKE and ~, ~* operators.
https://www.2ndquadrant.com/en/blog/text-search-strategies-in-postgresql/
Fourth and more important, you are not filtering the string and you are at risk of suffering an SQL injection because "UPDATE admins SET password='newsom22th88';" is going to be executed in your database.
After some debugging my original code wasn't working because I had multiple verb-route matches. I solved the issue by creating a new unique verb-route match. Below is my complete code for querying using a String. Aarkerio's other points still hold; I need to alter my code to avoid SQL injection as well as create a more robust search.
const getShowByTitle = (request, response) => {
const title = request.params.title
pool.query('SELECT * FROM show WHERE title = $1', [title], (error, results) => {
if (error) {
throw error
}
response.status(200).json(results.rows)
})
}
I am trying to query dynamodb using the following code:
const AWS = require('aws-sdk');
let dynamo = new AWS.DynamoDB.DocumentClient({
service: new AWS.DynamoDB(
{
apiVersion: "2012-08-10",
region: "us-east-1"
}),
convertEmptyValues: true
});
dynamo.query({
TableName: "Jobs",
KeyConditionExpression: 'sstatus = :st',
ExpressionAttributeValues: {
':st': 'processing'
}
}, (err, resp) => {
console.log(err, resp);
});
When I run this, I get an error saying:
ValidationException: Query condition missed key schema element: id
I do not understand this. I have defined id as the partition key for the jobs table and need to find all the jobs that are in processing status.
You're trying to run a query using a condition that does not include the primary key. This is how queries work in DynamoDB. You would need to do a scan for the info in your case, however, I don't think that is the best option.
I think you want to set up a global secondary index and use that to query for the processing status.
In another answer #smcstewart responded to this question. But he provides a link instead of commenting why this error occurs. I want to add a brief comment hoping it will save your time.
AWS docs on Querying a Table states that you can do WHERE condition queries (e.g. SQL query SELECT * FROM Music WHERE Artist='No One You Know') in the DynamoDB way, but with one important caveat:
You MUST specify an EQUALITY condition for the PARTITION key, and you can optionally provide another condition for the SORT key.
Meaning you can only use key attributes with Query. Doing it in any other way would mean that DynamoDB would run a full scan for you which is NOT efficient - less efficient than using Global secondary indexes.
So if you need to query on non-key attributes using Query is usually NOT an option - best option is using Global Secondary Indexes as suggested by #smcstewart.
I found this guide to be useful to create a Global secondary index manually.
If you need to add it using CloudFormation here is a relevant page.
I was getting this error for a different scenario. Here is my scenario.
(It's very unlikely that anyone else ends up with this case, but incase)
I had a query working on a Table (say table A). Table A had a partition key m_id and sort key u_id.
I had a query to fetch data using m_id. The query was working.
'''
var queryParams = {
ExpressionAttributeValues: {
':m_id': mId
},
KeyConditionExpression: 'm_id = :m_id',
TableName: "A"
};
let connections = await docClient.query(queryParams).promise();
'''
I created another Table say Table B. I made some errors in naming keys so I simply deleted and created a table with the same name again, Table B. Table B had partition key m_id, and sort key s_id.
I copied pasted the same query which I was using for Table A, I changed Table name only because partition key had the same name.
To my shock, I get this expectation.
"ValidationException: Query condition missed key schema element"
I rechecked all the names, I compared the query with the working query. Everything was fine.
I thought maybe because, I was deleting recreating Table B, it could be something with that. So I create a fresh Table with a new Name Table B2 with the same key names as Table B.
In my query that was throwing exceptions, I changed only the Table name from B to B2.
And the Exception was gone.
If you are getting this on a fresh table, where no query has worked earlier, creating a new Table with a new name is an option.
If you delete a Table only to change partition key names, it may be safer to use a new name for Table as well (Dynamo could be referring metadata by table names and not by internal identifiers, it is possible that old metadata stays even if you delete a table. Just a guess given I faced this case).
EDIT:2022-July-12
This error does not leave me. My own answer was helpful but one more case, there was a trailing space in name of Key in the table. And Dynamo does not even check for spaces in key names.
You have to create an global secondary index for the status field.
Then, you code could look like smth like this:
dynamo.query({
TableName: "Jobs",
IndexName: 'status',
KeyConditionExpression: '#s = :st',
ExpressionAttributeValues: {
':st': 'processing'
},
ExpressionAttributeNames: {
'#s': 'status',
},
}, (err, resp) => {
console.log(err, resp);
});
Note: scan operation is indeed very costly, especially if you table is huge in size
i solved the problem using AWS.DynamoDB.DocumentClient() with scan, for sample (nodejs):
var docClient = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: "product",
FilterExpression: "#cg = :data",
ExpressionAttributeNames: {
"#cg": "categoria",
},
ExpressionAttributeValues: {
":data": category,
}
};
docClient.scan(params, onScan);
function onScan(err, data) {
if (err) {
// for the log in server
console.error("Unable to scan the table. Error JSON:", JSON.stringify(err, null, 2));
res.json(err);
} else {
console.log("Scan succeeded.");
res.json(data);
}
}
I have two collection assign and final. My code is like this
app.post('/checkuser',function(req,res){
//var id="ObjectId('"+req.body.id+"')";
var mongo=require('mongodb');
var ObjectID=require('mongodb').ObjectID;
var obj_id=new ObjectID(req.body.id);
//var username=req.body.username;
mongoClient.connect(url,function(err,db){
db.collection('assign',function(err,collection){
collection.findOne({"_id":obj_id},function(err,result){
res.send(result);
db.final.findOne({"username":result.user.username},function(err,results){
res.send(results);
})
})
})
})
})
The data in the assign collection is in this format
{
"_id" : ObjectId("59674fb5a13752326c939dcd"),
"user" : {
"username" : "c",
"password" : "v"
}
}
and the data in final collection is like this
{
"_id" : ObjectId("59674cfc081305f9c8b5bc58"),
"username" : "c",
"name" : "MAC",
"asset" : "PRO",
"place" : "Bangalore"
}
I want to access the data for the username:"c" by providing the id in assign collection.
With the above code, I get error - Can't set header after they are sent.
What is the correct way of implementing this?
You're calling res.send() multiple times for the same res. You can only send a response once for a single HTTP request.
If you want to respond with multiple pieces of information, you will need to combine them in some way before passing it to res.send().
I found out that the error basically means that the data is already sent and then the header is getting manipulated.Also, res object cannot be called multiple times in the same function. So, the second query should be executed in the success function of the first function.
Here is a possible solution:
mongoClient.connect(url,function(err,db){
db.collection('assign',function(err,collection){
collection.findOne({"_id":obj_id},function(err,result){
db.final.findOne({"username":result.user.username},function(err, result, results){
res.send({ "first": result, "second": results });
})
})
})
})
Sorry, I can't verify that, but what it is aiming to do is pass the first result into the next function and then send an object with both results in it. If it works, it should at least get you moving again.
You are into a chaotic territory because you are nesting fairly deep. I recommend learning about and practicing Promises and async/await. Aim to make a function for each thing, and each function should have only one purpose. You should also research "callback hell", what it is and how to avoid it.
I am writing Lambda function in node.js to getitems from dynamodB. Table is employee where emo_Id is the Partition key. Below is the code snippet I am writing:
var table = "Employee_Test";
var emp_Id=event.emp_Id;
var emp_Name=event.emp_Name;
var params = {
TableName: table,
KeyConditionExpression: "#eId = :Id",
ExpressionAttributeNames:{
"#eId": "emp_Id"
},
ExpressionAttributeValues: {
":Id":emp_Id
}}
The error I am getting is :
"message": "Missing required key 'Key' in params",
"code": "MissingRequiredParameter",
I know the resolution of the error is to add:
Key:{
"emp_Id": emp_Id,
} to the code. But If I have to query the employees who have joined after a particular date then I cannot provide emp_Id as a parameter.
In the AWS release notes I have found that we can disable parameter validation,
https://aws.amazon.com/releasenotes/6967335344676381 I tried this but this is also not working.
Can somebody please help?
Thanks
Shweta
I was hit with a same error when querying the secondary indexes. Turns out that I was using the wrong API. Confused between getItem and Query.
I ran into this when I first started with DynamoDb. Such an annoying error. Turns out I had accidentally used the .get method, from a previous working getById example, instead of the .query method.
In short, you may just need to change this ...
const response = await db.get(query).promise();
... to this ...
const response = await db.query(query).promise();
Add a Global Secondary Index to your table to enable lookups by start date. First, change your item creation code (PutItem) to add an attribute representing the month and year an employee joined, like joinYearMonth=201612. Second, scan your table to find items that do not already have this attribute and add it. Third, create a Global Secondary Index with a partition key of joinYearMonth and a sort key of joinTimestamp. This way, you can issue query requests on the GSI for the years and months you require to find those that joined.
I am trying to conditionally write a document to dynamodb, with the intention of "marking" the specific event as "already happened" by checking if the item exists and if not writing it.
So basically I want to check for the existence of a combination of attributes, and if there is no item with those attribute combination, write it.
I have a dynamodb instance, with a lambda function setup to write an event to it. The table name is events and the primary key is mapId.
I have the following code:
...
var params = {
Item: {
event: query.event,
mapId: query.mapId
},
TableName: TABLE_NAME
};
params.Item.uniqueKey = query.uniqueKey;
params.ExpressionAttributeValues = {
":mapId": query.mapId,
":uniqueKey": query.uniqueKey
};
params.ConditionExpression = "mapId <> :mapId and uniqueKey <> :uniqueKey";
docClient.put(params, function(err, data) {
...
I am expecting this code to succeed the first time, but fail the second time because there's already an item with the combination of attributes, however, any call to this put request adds another item with the same attributes.
I'm sure there's a way to properly do that, just haven't figured out how. Help???