I am using Lambda (nodeJS 4.3) to query my DynamoDB with the following:
var params = {
TableName : "shoes",
KeyConditionExpression: "gender = :gender AND support = :support AND terrain = :terrain",
ExpressionAttributeValues: {
":input": inputGender,
":input": inputSupport,
":input": inputTerrain
}
}
When this runs I am getting an error that I am missing "Query condition missed key schema element: Id". It may just be a fundamental misunderstanding on my part, but if I want to query 2 or more of the fields in the DynamoDB do I need to make them a key or create indexes on all of them??
Thanks in advance.
A query must include the partition key of the table (or a global secondary index). Your table's partition key is id and you didn't include that in the query. Given the query you are trying to run I don't think it makes sense to create a GSI on your table. You will need to perform a full table scan operation instead of a query operation.
Related
I am trying to delete and update records in cosmosDB using my graphql/nodejs code and getting error - "Entity with the specified id does not exist in the system". Here is my code
deleteRecord: async (root, id) => {
const { resource: result } = await container.item(id.id, key).delete();
console.log(`Deleted item with id: ${id}`);
},
Somehow below code is not able to find record, even "container.item(id.id, key).read()" doesn't work.
await container.item(id.id, key)
But if I try to find record using query spec it works
await container.items.query('SELECT * from c where c.id = "'+id+'"' ).fetchNext()
FYI- I am able to fetch all records and create new item, so Connection to DB and reading/writing is not an issue.
What else can it be? Any pointer related to this will be helpful.
Thanks in advance.
It seems you pass the wrong key to item(id,key). According to the Note of this documentation:
In both the "update" and "delete" methods, the item has to be selected
from the database by calling container.item(). The two parameters
passed in are the id of the item and the item's partition key. In this
case, the parition key is the value of the "category" field.
So you need to pass the value of your partition key, not your partition key path.
For example, if you have document like below, and your partition key is '/category', you need to use this code await container.item("xxxxxx", "movie").
{
"id":"xxxxxx",
"category":"movie"
}
I am trying to query dynamodb using the following code:
const AWS = require('aws-sdk');
let dynamo = new AWS.DynamoDB.DocumentClient({
service: new AWS.DynamoDB(
{
apiVersion: "2012-08-10",
region: "us-east-1"
}),
convertEmptyValues: true
});
dynamo.query({
TableName: "Jobs",
KeyConditionExpression: 'sstatus = :st',
ExpressionAttributeValues: {
':st': 'processing'
}
}, (err, resp) => {
console.log(err, resp);
});
When I run this, I get an error saying:
ValidationException: Query condition missed key schema element: id
I do not understand this. I have defined id as the partition key for the jobs table and need to find all the jobs that are in processing status.
You're trying to run a query using a condition that does not include the primary key. This is how queries work in DynamoDB. You would need to do a scan for the info in your case, however, I don't think that is the best option.
I think you want to set up a global secondary index and use that to query for the processing status.
In another answer #smcstewart responded to this question. But he provides a link instead of commenting why this error occurs. I want to add a brief comment hoping it will save your time.
AWS docs on Querying a Table states that you can do WHERE condition queries (e.g. SQL query SELECT * FROM Music WHERE Artist='No One You Know') in the DynamoDB way, but with one important caveat:
You MUST specify an EQUALITY condition for the PARTITION key, and you can optionally provide another condition for the SORT key.
Meaning you can only use key attributes with Query. Doing it in any other way would mean that DynamoDB would run a full scan for you which is NOT efficient - less efficient than using Global secondary indexes.
So if you need to query on non-key attributes using Query is usually NOT an option - best option is using Global Secondary Indexes as suggested by #smcstewart.
I found this guide to be useful to create a Global secondary index manually.
If you need to add it using CloudFormation here is a relevant page.
I was getting this error for a different scenario. Here is my scenario.
(It's very unlikely that anyone else ends up with this case, but incase)
I had a query working on a Table (say table A). Table A had a partition key m_id and sort key u_id.
I had a query to fetch data using m_id. The query was working.
'''
var queryParams = {
ExpressionAttributeValues: {
':m_id': mId
},
KeyConditionExpression: 'm_id = :m_id',
TableName: "A"
};
let connections = await docClient.query(queryParams).promise();
'''
I created another Table say Table B. I made some errors in naming keys so I simply deleted and created a table with the same name again, Table B. Table B had partition key m_id, and sort key s_id.
I copied pasted the same query which I was using for Table A, I changed Table name only because partition key had the same name.
To my shock, I get this expectation.
"ValidationException: Query condition missed key schema element"
I rechecked all the names, I compared the query with the working query. Everything was fine.
I thought maybe because, I was deleting recreating Table B, it could be something with that. So I create a fresh Table with a new Name Table B2 with the same key names as Table B.
In my query that was throwing exceptions, I changed only the Table name from B to B2.
And the Exception was gone.
If you are getting this on a fresh table, where no query has worked earlier, creating a new Table with a new name is an option.
If you delete a Table only to change partition key names, it may be safer to use a new name for Table as well (Dynamo could be referring metadata by table names and not by internal identifiers, it is possible that old metadata stays even if you delete a table. Just a guess given I faced this case).
EDIT:2022-July-12
This error does not leave me. My own answer was helpful but one more case, there was a trailing space in name of Key in the table. And Dynamo does not even check for spaces in key names.
You have to create an global secondary index for the status field.
Then, you code could look like smth like this:
dynamo.query({
TableName: "Jobs",
IndexName: 'status',
KeyConditionExpression: '#s = :st',
ExpressionAttributeValues: {
':st': 'processing'
},
ExpressionAttributeNames: {
'#s': 'status',
},
}, (err, resp) => {
console.log(err, resp);
});
Note: scan operation is indeed very costly, especially if you table is huge in size
i solved the problem using AWS.DynamoDB.DocumentClient() with scan, for sample (nodejs):
var docClient = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: "product",
FilterExpression: "#cg = :data",
ExpressionAttributeNames: {
"#cg": "categoria",
},
ExpressionAttributeValues: {
":data": category,
}
};
docClient.scan(params, onScan);
function onScan(err, data) {
if (err) {
// for the log in server
console.error("Unable to scan the table. Error JSON:", JSON.stringify(err, null, 2));
res.json(err);
} else {
console.log("Scan succeeded.");
res.json(data);
}
}
I am writing Lambda function in node.js to getitems from dynamodB. Table is employee where emo_Id is the Partition key. Below is the code snippet I am writing:
var table = "Employee_Test";
var emp_Id=event.emp_Id;
var emp_Name=event.emp_Name;
var params = {
TableName: table,
KeyConditionExpression: "#eId = :Id",
ExpressionAttributeNames:{
"#eId": "emp_Id"
},
ExpressionAttributeValues: {
":Id":emp_Id
}}
The error I am getting is :
"message": "Missing required key 'Key' in params",
"code": "MissingRequiredParameter",
I know the resolution of the error is to add:
Key:{
"emp_Id": emp_Id,
} to the code. But If I have to query the employees who have joined after a particular date then I cannot provide emp_Id as a parameter.
In the AWS release notes I have found that we can disable parameter validation,
https://aws.amazon.com/releasenotes/6967335344676381 I tried this but this is also not working.
Can somebody please help?
Thanks
Shweta
I was hit with a same error when querying the secondary indexes. Turns out that I was using the wrong API. Confused between getItem and Query.
I ran into this when I first started with DynamoDb. Such an annoying error. Turns out I had accidentally used the .get method, from a previous working getById example, instead of the .query method.
In short, you may just need to change this ...
const response = await db.get(query).promise();
... to this ...
const response = await db.query(query).promise();
Add a Global Secondary Index to your table to enable lookups by start date. First, change your item creation code (PutItem) to add an attribute representing the month and year an employee joined, like joinYearMonth=201612. Second, scan your table to find items that do not already have this attribute and add it. Third, create a Global Secondary Index with a partition key of joinYearMonth and a sort key of joinTimestamp. This way, you can issue query requests on the GSI for the years and months you require to find those that joined.
Using Node.js, I'm querying DynamoDB for objects using a secondary index:
TableName: "Products",
IndexName: 'MerchantAndDateIndex',
KeyConditionExpression: "#creator = :creatorId",
ExpressionAttributeNames:{
"#creator": "createdBy"
},
ExpressionAttributeValues: {
":creatorId": uuid
}
What I want to do is query the "Products" table with the "createdBy" hash key accepting several different possible strings, something like this:
ExpressionAttributeValues: {
":creatorId": ["multiple","valid","uuids","here"]
}
Where every element in my dynamic array is checked against the ID in the table when running a query.
Of course, this specific implementation returns a type mismatch, but I'm not sure how to do this correctly
This is not supported by the DynamoDB API, and it would still count as multiple read requests from DynamoDB's throughput perspective.
If you need to query for multiple values you would need to implement that logic yourself in your application. Basically write a loop over the values to query and issue a query for each. That's what Dynamo would do server-side anyway.
I am trying to get and delete the data from dynamo db using some 2 primary key column, In which one is reserved keyword timestamp.
When I get data using one of the primary key then it is working fine, But when I trying to delete using two primary key field in projection expression it giving me an error like :
ValidationException: Invalid ProjectionExpression: An expression attribute name used in the document path is not defined; attribute name: #P
And this is my params object
{ TableName: 'Log',
ProjectionExpression: 'username, #P',
ExpressionAttributeNames: { '#P': 'timestamp' },
ExpressionAttributeValues: null }
username and timestamp are primary key columns.
This thing is not working in only this table. In some others table I have same situation but it will not throwing me this error. Only in this case I am getting the error.
The DeleteItem API does not allow you to specify a ProjectionExpression. You can get the entire pre-deletion image of the item by selecting ReturnValues = ALL_OLD.