I am creating an app using a GraphQL API in AWS AppSync, with DynamoDB for storage. My DynamoDB table utilises a primary key only (id) and does not use a sort key. I want to ensure that new objects added to the table (with the same primary key / id) do not overwrite existing objects.
My lambda function handler contains the following code:
const params = {
TableName : "myTable",
Item: {
id: "4321",
name: event.arguments.input.name,
description: event.arguments.input.description
},
ConditionalExpression: 'attribute_not_exists(id)',
};
await documentClient.put(params, function(err, data) {
if (err) {
console.log(err);
callback(err);
} else {
callback(null, data)
}
}).promise();
According to multiple sources (1, 2, 3), the conditional expression attribute_not_exists(id) should prevent entries with the same id being added to the table, since this is the primary key and I do not have a sort key.
After adding the first object to the database, I expected that subsequent calls would fail, since I hardcoded the id field to check that objects cannot be overwritten with the same ID. However, I can see that the object is being updated in DynamoDB, as the non-key fields (name and description) are changing.
For additional context, I have been executing this mutation using the 'Run a Query' part of the AppSync console in the AWS management console. Then I check to see if the mutation is successful by looking at my DynamoDB tables in the AWS management console.
Found that this was due to a spelling error - 'ConditionalExpression' should be 'ConditionExpression'.
Related
Is it possible to reference a newly created DynamoDB record in AWS Lambda? For example, retrieving and using the ID of the newly created record. Hoping this is possible without a query to retrieve the new record from DynamoDB.
const docClient = new AWS.DynamoDB.DocumentClient();
... // Omitting the rest of the code in this example
const params = {
TableName : 'ExampleTableName',
Item: {
id: uuid.v1()
}
}
try {
await docClient.put(params).promise();
} catch (err) {
return err;
}
// Reference newly created record to retrieve the ID.
Of course you can achieve it using ReturnValues Paramter.
ReturnValues:- return the item's attribute values in the same operation.
But I am afraid that if you want to achieve your purpose you need to use an alternative API which UpdateItem
From the Docs
Use ReturnValues if you want to get the item attributes as they appear before or after they are updated. For UpdateItem, the valid values are:
NONE - If ReturnValues is not specified, or if its value is NONE, then nothing is returned. (This setting is the default for ReturnValues.)
ALL_OLD - Returns all of the attributes of the item, as they appeared before the UpdateItem operation.
UPDATED_OLD - Returns only the updated attributes, as they appeared before the UpdateItem operation.
ALL_NEW - Returns all of the attributes of the item, as they appear after the UpdateItem operation.
UPDATED_NEW - Returns only the updated attributes, as they appear after the UpdateItem operation.
There is no additional cost associated with requesting a return value aside from the small network and processing overhead of receiving a larger response. No read capacity units are consumed.
The values returned are strongly consistent.
Why you can use returnValue with putItem? Reason -> https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html#API_PutItem_RequestSyntax which wont solve your purpose.
You can store return values in variable and add custom logic to proceed further. :)
I am trying to delete and update records in cosmosDB using my graphql/nodejs code and getting error - "Entity with the specified id does not exist in the system". Here is my code
deleteRecord: async (root, id) => {
const { resource: result } = await container.item(id.id, key).delete();
console.log(`Deleted item with id: ${id}`);
},
Somehow below code is not able to find record, even "container.item(id.id, key).read()" doesn't work.
await container.item(id.id, key)
But if I try to find record using query spec it works
await container.items.query('SELECT * from c where c.id = "'+id+'"' ).fetchNext()
FYI- I am able to fetch all records and create new item, so Connection to DB and reading/writing is not an issue.
What else can it be? Any pointer related to this will be helpful.
Thanks in advance.
It seems you pass the wrong key to item(id,key). According to the Note of this documentation:
In both the "update" and "delete" methods, the item has to be selected
from the database by calling container.item(). The two parameters
passed in are the id of the item and the item's partition key. In this
case, the parition key is the value of the "category" field.
So you need to pass the value of your partition key, not your partition key path.
For example, if you have document like below, and your partition key is '/category', you need to use this code await container.item("xxxxxx", "movie").
{
"id":"xxxxxx",
"category":"movie"
}
I am trying to query dynamodb using the following code:
const AWS = require('aws-sdk');
let dynamo = new AWS.DynamoDB.DocumentClient({
service: new AWS.DynamoDB(
{
apiVersion: "2012-08-10",
region: "us-east-1"
}),
convertEmptyValues: true
});
dynamo.query({
TableName: "Jobs",
KeyConditionExpression: 'sstatus = :st',
ExpressionAttributeValues: {
':st': 'processing'
}
}, (err, resp) => {
console.log(err, resp);
});
When I run this, I get an error saying:
ValidationException: Query condition missed key schema element: id
I do not understand this. I have defined id as the partition key for the jobs table and need to find all the jobs that are in processing status.
You're trying to run a query using a condition that does not include the primary key. This is how queries work in DynamoDB. You would need to do a scan for the info in your case, however, I don't think that is the best option.
I think you want to set up a global secondary index and use that to query for the processing status.
In another answer #smcstewart responded to this question. But he provides a link instead of commenting why this error occurs. I want to add a brief comment hoping it will save your time.
AWS docs on Querying a Table states that you can do WHERE condition queries (e.g. SQL query SELECT * FROM Music WHERE Artist='No One You Know') in the DynamoDB way, but with one important caveat:
You MUST specify an EQUALITY condition for the PARTITION key, and you can optionally provide another condition for the SORT key.
Meaning you can only use key attributes with Query. Doing it in any other way would mean that DynamoDB would run a full scan for you which is NOT efficient - less efficient than using Global secondary indexes.
So if you need to query on non-key attributes using Query is usually NOT an option - best option is using Global Secondary Indexes as suggested by #smcstewart.
I found this guide to be useful to create a Global secondary index manually.
If you need to add it using CloudFormation here is a relevant page.
I was getting this error for a different scenario. Here is my scenario.
(It's very unlikely that anyone else ends up with this case, but incase)
I had a query working on a Table (say table A). Table A had a partition key m_id and sort key u_id.
I had a query to fetch data using m_id. The query was working.
'''
var queryParams = {
ExpressionAttributeValues: {
':m_id': mId
},
KeyConditionExpression: 'm_id = :m_id',
TableName: "A"
};
let connections = await docClient.query(queryParams).promise();
'''
I created another Table say Table B. I made some errors in naming keys so I simply deleted and created a table with the same name again, Table B. Table B had partition key m_id, and sort key s_id.
I copied pasted the same query which I was using for Table A, I changed Table name only because partition key had the same name.
To my shock, I get this expectation.
"ValidationException: Query condition missed key schema element"
I rechecked all the names, I compared the query with the working query. Everything was fine.
I thought maybe because, I was deleting recreating Table B, it could be something with that. So I create a fresh Table with a new Name Table B2 with the same key names as Table B.
In my query that was throwing exceptions, I changed only the Table name from B to B2.
And the Exception was gone.
If you are getting this on a fresh table, where no query has worked earlier, creating a new Table with a new name is an option.
If you delete a Table only to change partition key names, it may be safer to use a new name for Table as well (Dynamo could be referring metadata by table names and not by internal identifiers, it is possible that old metadata stays even if you delete a table. Just a guess given I faced this case).
EDIT:2022-July-12
This error does not leave me. My own answer was helpful but one more case, there was a trailing space in name of Key in the table. And Dynamo does not even check for spaces in key names.
You have to create an global secondary index for the status field.
Then, you code could look like smth like this:
dynamo.query({
TableName: "Jobs",
IndexName: 'status',
KeyConditionExpression: '#s = :st',
ExpressionAttributeValues: {
':st': 'processing'
},
ExpressionAttributeNames: {
'#s': 'status',
},
}, (err, resp) => {
console.log(err, resp);
});
Note: scan operation is indeed very costly, especially if you table is huge in size
i solved the problem using AWS.DynamoDB.DocumentClient() with scan, for sample (nodejs):
var docClient = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: "product",
FilterExpression: "#cg = :data",
ExpressionAttributeNames: {
"#cg": "categoria",
},
ExpressionAttributeValues: {
":data": category,
}
};
docClient.scan(params, onScan);
function onScan(err, data) {
if (err) {
// for the log in server
console.error("Unable to scan the table. Error JSON:", JSON.stringify(err, null, 2));
res.json(err);
} else {
console.log("Scan succeeded.");
res.json(data);
}
}
I am using AWS.DynamoDB.DocumentClient in a nodejs program to fetch items from multiple Dynamodb tables. To make code simple, I choose to use BatchGetItem/BatchGet method.
The challenge is I need to fetch items based on a Global Secondary Index, e.g. name+age, rather than the initial primary key generated when creating the table. I went through BatchGetItem/BatchGet but not see any parameters of using Global Secondary Index.
I ran some testing with the following code
var params = {
RequestItems: {
'Table-1': {
Keys: [
{
name: 'abc',
age: 18,
},
]
}
}
};
var docClient = new AWS.DynamoDB.DocumentClient();
docClient.batchGet(params, function(err, data) {
if (err) console.log(err);
else console.log(data);
});
And got following error.
> ValidationException: The provided key element does not match the
> schema
Does it mean BatchGetItem/BatchGet can't use Global Secondary Index, and I have to read from tables one by one?
I don't believe so. You will likely have to query one-by-one.
INDEXES - The response includes the aggregate ConsumedCapacity for the operation, together with ConsumedCapacity for each table and secondary index that was accessed. Note that some operations, such as GetItem and BatchGetItem , do not access any indexes at all. In these cases, specifying INDEXES will only return ConsumedCapacity information for table(s).
Source: https://docs.aws.amazon.com/cli/latest/reference/dynamodb/batch-get-item.html
I have a table called followProduct in Dynamodb and it has following strucure
id - item id
email - user email
product - product id
Whenever a user follows a product I am making an entry in the table. I am trying to stop duplicate entry and using the following code
let params = {
TableName: "followProduct",
ConditionExpression: "email <> :email AND product <> :pid",
Item: {
email: "a#a.com",
product: req.body.productId,
id: shortid.generate()
},
ExpressionAttributeValues: {
':email': "a#a.com",
":pid": req.body.productId
}
};
createItemInDDB(params).then(() => {
res.status(200).send("Company Added");
}, err => {
console.log(err);
res.sendStatus(500);
});
CreateItemInDDB is just a function that takes params as input and run put function provided by document client. This params is still making a duplicate entry. I want that for every email each product id should be entered only once.
can you describe your table hash-range keys?
Dynamodb can force uniqueness only for hash-range table keys (not for global secondary index keys)
from http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html
To prevent a new item from replacing an existing item, use a conditional expression that contains the attribute_not_exists function with the name of the attribute being used as the partition key for the table. Since every record must contain that attribute, the attribute_not_exists function will only succeed if no matching item exists.
and http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html:
The PutItem operation will overwrite an item with the same key (if it exists). If you want to avoid this, use a condition expression. This will allows the write to proceed only if the item in question does not already have the same key: