I am trying to conditionally write a document to dynamodb, with the intention of "marking" the specific event as "already happened" by checking if the item exists and if not writing it.
So basically I want to check for the existence of a combination of attributes, and if there is no item with those attribute combination, write it.
I have a dynamodb instance, with a lambda function setup to write an event to it. The table name is events and the primary key is mapId.
I have the following code:
...
var params = {
Item: {
event: query.event,
mapId: query.mapId
},
TableName: TABLE_NAME
};
params.Item.uniqueKey = query.uniqueKey;
params.ExpressionAttributeValues = {
":mapId": query.mapId,
":uniqueKey": query.uniqueKey
};
params.ConditionExpression = "mapId <> :mapId and uniqueKey <> :uniqueKey";
docClient.put(params, function(err, data) {
...
I am expecting this code to succeed the first time, but fail the second time because there's already an item with the combination of attributes, however, any call to this put request adds another item with the same attributes.
I'm sure there's a way to properly do that, just haven't figured out how. Help???
Related
Is it possible to reference a newly created DynamoDB record in AWS Lambda? For example, retrieving and using the ID of the newly created record. Hoping this is possible without a query to retrieve the new record from DynamoDB.
const docClient = new AWS.DynamoDB.DocumentClient();
... // Omitting the rest of the code in this example
const params = {
TableName : 'ExampleTableName',
Item: {
id: uuid.v1()
}
}
try {
await docClient.put(params).promise();
} catch (err) {
return err;
}
// Reference newly created record to retrieve the ID.
Of course you can achieve it using ReturnValues Paramter.
ReturnValues:- return the item's attribute values in the same operation.
But I am afraid that if you want to achieve your purpose you need to use an alternative API which UpdateItem
From the Docs
Use ReturnValues if you want to get the item attributes as they appear before or after they are updated. For UpdateItem, the valid values are:
NONE - If ReturnValues is not specified, or if its value is NONE, then nothing is returned. (This setting is the default for ReturnValues.)
ALL_OLD - Returns all of the attributes of the item, as they appeared before the UpdateItem operation.
UPDATED_OLD - Returns only the updated attributes, as they appeared before the UpdateItem operation.
ALL_NEW - Returns all of the attributes of the item, as they appear after the UpdateItem operation.
UPDATED_NEW - Returns only the updated attributes, as they appear after the UpdateItem operation.
There is no additional cost associated with requesting a return value aside from the small network and processing overhead of receiving a larger response. No read capacity units are consumed.
The values returned are strongly consistent.
Why you can use returnValue with putItem? Reason -> https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html#API_PutItem_RequestSyntax which wont solve your purpose.
You can store return values in variable and add custom logic to proceed further. :)
I am creating an app using a GraphQL API in AWS AppSync, with DynamoDB for storage. My DynamoDB table utilises a primary key only (id) and does not use a sort key. I want to ensure that new objects added to the table (with the same primary key / id) do not overwrite existing objects.
My lambda function handler contains the following code:
const params = {
TableName : "myTable",
Item: {
id: "4321",
name: event.arguments.input.name,
description: event.arguments.input.description
},
ConditionalExpression: 'attribute_not_exists(id)',
};
await documentClient.put(params, function(err, data) {
if (err) {
console.log(err);
callback(err);
} else {
callback(null, data)
}
}).promise();
According to multiple sources (1, 2, 3), the conditional expression attribute_not_exists(id) should prevent entries with the same id being added to the table, since this is the primary key and I do not have a sort key.
After adding the first object to the database, I expected that subsequent calls would fail, since I hardcoded the id field to check that objects cannot be overwritten with the same ID. However, I can see that the object is being updated in DynamoDB, as the non-key fields (name and description) are changing.
For additional context, I have been executing this mutation using the 'Run a Query' part of the AppSync console in the AWS management console. Then I check to see if the mutation is successful by looking at my DynamoDB tables in the AWS management console.
Found that this was due to a spelling error - 'ConditionalExpression' should be 'ConditionExpression'.
I am trying to query dynamodb using the following code:
const AWS = require('aws-sdk');
let dynamo = new AWS.DynamoDB.DocumentClient({
service: new AWS.DynamoDB(
{
apiVersion: "2012-08-10",
region: "us-east-1"
}),
convertEmptyValues: true
});
dynamo.query({
TableName: "Jobs",
KeyConditionExpression: 'sstatus = :st',
ExpressionAttributeValues: {
':st': 'processing'
}
}, (err, resp) => {
console.log(err, resp);
});
When I run this, I get an error saying:
ValidationException: Query condition missed key schema element: id
I do not understand this. I have defined id as the partition key for the jobs table and need to find all the jobs that are in processing status.
You're trying to run a query using a condition that does not include the primary key. This is how queries work in DynamoDB. You would need to do a scan for the info in your case, however, I don't think that is the best option.
I think you want to set up a global secondary index and use that to query for the processing status.
In another answer #smcstewart responded to this question. But he provides a link instead of commenting why this error occurs. I want to add a brief comment hoping it will save your time.
AWS docs on Querying a Table states that you can do WHERE condition queries (e.g. SQL query SELECT * FROM Music WHERE Artist='No One You Know') in the DynamoDB way, but with one important caveat:
You MUST specify an EQUALITY condition for the PARTITION key, and you can optionally provide another condition for the SORT key.
Meaning you can only use key attributes with Query. Doing it in any other way would mean that DynamoDB would run a full scan for you which is NOT efficient - less efficient than using Global secondary indexes.
So if you need to query on non-key attributes using Query is usually NOT an option - best option is using Global Secondary Indexes as suggested by #smcstewart.
I found this guide to be useful to create a Global secondary index manually.
If you need to add it using CloudFormation here is a relevant page.
I was getting this error for a different scenario. Here is my scenario.
(It's very unlikely that anyone else ends up with this case, but incase)
I had a query working on a Table (say table A). Table A had a partition key m_id and sort key u_id.
I had a query to fetch data using m_id. The query was working.
'''
var queryParams = {
ExpressionAttributeValues: {
':m_id': mId
},
KeyConditionExpression: 'm_id = :m_id',
TableName: "A"
};
let connections = await docClient.query(queryParams).promise();
'''
I created another Table say Table B. I made some errors in naming keys so I simply deleted and created a table with the same name again, Table B. Table B had partition key m_id, and sort key s_id.
I copied pasted the same query which I was using for Table A, I changed Table name only because partition key had the same name.
To my shock, I get this expectation.
"ValidationException: Query condition missed key schema element"
I rechecked all the names, I compared the query with the working query. Everything was fine.
I thought maybe because, I was deleting recreating Table B, it could be something with that. So I create a fresh Table with a new Name Table B2 with the same key names as Table B.
In my query that was throwing exceptions, I changed only the Table name from B to B2.
And the Exception was gone.
If you are getting this on a fresh table, where no query has worked earlier, creating a new Table with a new name is an option.
If you delete a Table only to change partition key names, it may be safer to use a new name for Table as well (Dynamo could be referring metadata by table names and not by internal identifiers, it is possible that old metadata stays even if you delete a table. Just a guess given I faced this case).
EDIT:2022-July-12
This error does not leave me. My own answer was helpful but one more case, there was a trailing space in name of Key in the table. And Dynamo does not even check for spaces in key names.
You have to create an global secondary index for the status field.
Then, you code could look like smth like this:
dynamo.query({
TableName: "Jobs",
IndexName: 'status',
KeyConditionExpression: '#s = :st',
ExpressionAttributeValues: {
':st': 'processing'
},
ExpressionAttributeNames: {
'#s': 'status',
},
}, (err, resp) => {
console.log(err, resp);
});
Note: scan operation is indeed very costly, especially if you table is huge in size
i solved the problem using AWS.DynamoDB.DocumentClient() with scan, for sample (nodejs):
var docClient = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: "product",
FilterExpression: "#cg = :data",
ExpressionAttributeNames: {
"#cg": "categoria",
},
ExpressionAttributeValues: {
":data": category,
}
};
docClient.scan(params, onScan);
function onScan(err, data) {
if (err) {
// for the log in server
console.error("Unable to scan the table. Error JSON:", JSON.stringify(err, null, 2));
res.json(err);
} else {
console.log("Scan succeeded.");
res.json(data);
}
}
I have a table called followProduct in Dynamodb and it has following strucure
id - item id
email - user email
product - product id
Whenever a user follows a product I am making an entry in the table. I am trying to stop duplicate entry and using the following code
let params = {
TableName: "followProduct",
ConditionExpression: "email <> :email AND product <> :pid",
Item: {
email: "a#a.com",
product: req.body.productId,
id: shortid.generate()
},
ExpressionAttributeValues: {
':email': "a#a.com",
":pid": req.body.productId
}
};
createItemInDDB(params).then(() => {
res.status(200).send("Company Added");
}, err => {
console.log(err);
res.sendStatus(500);
});
CreateItemInDDB is just a function that takes params as input and run put function provided by document client. This params is still making a duplicate entry. I want that for every email each product id should be entered only once.
can you describe your table hash-range keys?
Dynamodb can force uniqueness only for hash-range table keys (not for global secondary index keys)
from http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_PutItem.html
To prevent a new item from replacing an existing item, use a conditional expression that contains the attribute_not_exists function with the name of the attribute being used as the partition key for the table. Since every record must contain that attribute, the attribute_not_exists function will only succeed if no matching item exists.
and http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.ConditionExpressions.html:
The PutItem operation will overwrite an item with the same key (if it exists). If you want to avoid this, use a condition expression. This will allows the write to proceed only if the item in question does not already have the same key:
I have written the following code to return a row from a Dynamodb table based on a key called id.
var db = new AWS.DynamoDB();
var params = {
AttributesToGet: [
"dealername"
],
TableName : "syv_dealer",
Key : {
"id" : {
"S" : "1"
}
}
}
db.query(params, function(err, data) {
if (err) {
console.log(err); // an error occurred
} else {
console.log(data); // successful response
}
});
My code to write the data to the table worked fine, but the above code keeps generating the following error: Unexpected key 'Key' found in params
The table exists and there is an id attribute with a value of "1" in the table.
I have looked at all the examples I could find and have changed the params statement a hundred times, but nothing has worked. I am sure it is something simple, but any help would be appreciated.
This is failing because Key is an invalid parameter for submitting a request to that API. Look at the DynamoDB Query syntax to determine what attributes you can set.
The proper way to perform this query would be with a KeyConditionExpression which would look something like:
var params = {
TableName : "syv_dealer",
KeyConditionExpression: "id = :v1",
ExpressionAttributeValues: {
":v1": {"S": "1"}
},
ProjectionExpression: "dealername"
}
Using ProjectionExpression instead of AttributesToGet is also the modern way of specifying attributes.
Thank you, I had changed things so much I hadn't realized I had changed it from a getItem to Query.
The crux of my problem all along was that the function call is asynchronous, so I wasn't being patient and letting the callback execute.
I am new to Node.js, so I am trying to figure out how to check the credentials on a web service call and not have it continue until I have authenticated the requestor. Is there a way on the getItem call to have it be synchronous? I realize I could embed the logic that follows within the getItem callback, but I was really trying to use the same security routine for every endpoint.
Thoughts?