I am attempting to delete data from a DynamoDB table.
If I delete data using a partition key, it works.
But when I delete multiple rows using any other fields, it fails.
var params = {
TableName: "test",
Key: {
dmac: dmac,
},
ConditionExpression: "dmac= :dmac"
};
docClient.delete( params, (error) => {
if (error) {
console.log( "Delete data fail" );
} else {
console.log( "Delete data Success" );
}
});
Items (or rows) in DynamoDB are uniquely identified by their primary key. A table can have a simple primary key (a partition key) or a composite primary key (a partition key plus a sort key).
To delete an item, you must provide the full primary key (whether it's a simple partition key or composite partition key plus sort key).
So, if you want to delete items that meet a specific condition, for example cars with maxspeed < 120, then issue a query or scan to identify those items, retrieve the primary keys, and then delete the items in a second operation.
To delete a single item, use DeleteItem. To delete multiple items, use BatchWriteItem. Despite the naming of BatchWriteItem, it can be used to put multiple items or to delete multiple items, and you can target one or more DynamoDB tables in the same API call.
Here is an AWS SDK v2 example of deleting multiple items:
const aws = require("aws-sdk");
const ddb = new aws.DynamoDB({ region: "us-east-1" });
(async () => {
const params = {
RequestItems: {
albums: []
}
};
params.RequestItems.albums.push({
DeleteRequest: {
Key: {
pk: { S: "The Who" },
sk: { S: "Tommy" }
}
}
});
params.RequestItems.albums.push({
DeleteRequest: {
Key: {
pk: { S: "The Beatles" },
sk: { S: "Abbey Road" }
}
}
});
await ddb.batchWriteItem(params).promise();
})();
Here is an AWS SDK v3 example of deleting multiple items:
const {
BatchWriteItemCommand,
DynamoDBClient
} = require("#aws-sdk/client-dynamodb");
(async () => {
const client = new DynamoDBClient({ region: "us-east-1" });
const params = {
RequestItems: {
albums: []
}
};
params.RequestItems.albums.push({
DeleteRequest: {
Key: {
pk: { S: "The Who" },
sk: { S: "Tommy" }
}
}
});
params.RequestItems.albums.push({
DeleteRequest: {
Key: {
pk: { S: "The Beatles" },
sk: { S: "Abbey Road" }
}
}
});
await client.send(new BatchWriteItemCommand(params));
})();
in DynamoDB you can only delete an item using its key (that is: the partition key and the sort key, if it is defined on the table). This is, for example, underlined by the fact that the Key attribute is a required attribute in the canonical specification of the delete operation. See: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DeleteItem.html#DDB-DeleteItem-request-Key
This means that if you want to delete an item using other attributes you must first lookup the item by the attributes you do have, extract the key from the returned item, and then delete the item using that key.
The standard solution for "looking up an item by attributes that are not the item's key" is to define a global secondary index (GSI) on the table with those attribute(s) defined as the GSI's key.
Related
The DynamoDB.DocumentClient automatically marshals & unmarshals values between JavaScript types DynamoDB's more descriptive AttributeMap type. However, when working with an Item that has a StringSet attribute, it does not seem to do the conversion automatically.
When adding a StringSet attribute to the table using DocumentClient, I use the createSet(...) method to convert the array to a Set. When retrieving the value back, what is the inverse of createSet(...)? Is the best practice to just access the Set's .values directly? And if so, is that documented somewhere?
Here's sample code adding an Item with a StringSet attribute, then retrieving that item:
const docClient = new DocumentClient();
const TableName = "StringSets-Example";
const PK = "Names";
const Values = ["Peter", "Paul", "Mary"];
const putParams = {
TableName,
Item: {
PK,
names: docClient.createSet(Values)
}
}
await docClient.put(putParams).promise();
// ... some time later, I can retrieve the value with ...
const getParams = {
TableName,
Key: { PK }
}
const result = await docClient.get(getParams).promise();
The result.Item there is a Set object, whereas I would expect it to be the same array I passed into createSet(...).
If interested in seeing this live, this repo has a fully-functioning example. Clone it, npm install, and run index.js and you'll see something like:
$ ./index.js
Running On: darwin 19.6.0
Node version: v12.20.0
AWS SDK version: 2.799.0
-------------------------
Creating table "StringSets-Example"
Waiting for "StringSets-Example" status to be "ACTIVE"
Table status is: CREATING
Table status is: ACTIVE
Put String Set "["Peter, "Paul, "Mary"]" into "StringSets-Example" with key "Names" and attribute "names"
Retrieved Item with key "Names" from "StringSets-Example"
The raw Item: {
PK: 'Names',
names: Set {
wrapperName: 'Set',
values: [ 'Mary', 'Paul', 'Peter' ],
type: 'String'
}
}
The raw Item.names.values: [ 'Mary', 'Paul', 'Peter' ]
-------------------------
Done. To clean up, run:
./src/deleteTable.js
The best solution I have here is to avoid the DocumentClient and the createSet(...) method. Here's a sample using AWS SDK V3:
const key = { PK: `SampleNames`, SK: `SampleNames` };
const names = new Set([`Peter`, `Paul`, `Mary`]);
const item = { ...key, names };
const marshalledItem = marshall(item);
console.log(`Raw item: ${inspect(item)}`)
console.log(`Marshalled item to PUT: ${inspect(marshalledItem, { depth: 4 })}`)
const client = new DynamoDBClient({});
await client.send(new PutItemCommand({
TableName: tableName,
Item: marshalledItem,
}));
const { Item } = await client.send(new GetItemCommand({
TableName: tableName,
Key: marshall(key),
}));
console.log(`Returned item: ${inspect(Item, { depth: 4 })}`);
console.log(`Unmarshalled returned item: ${inspect(unmarshall(Item))}`);
The console output from there is:
Raw item: {
PK: 'SampleNames',
SK: 'SampleNames',
names: Set { 'Peter', 'Paul', 'Mary' }
}
Marshalled item to PUT: {
PK: { S: 'SampleNames' },
SK: { S: 'SampleNames' },
names: { SS: [ 'Peter', 'Paul', 'Mary' ] }
}
Returned item: {
PK: { S: 'SampleNames' },
SK: { S: 'SampleNames' },
names: { SS: [ 'Mary', 'Paul', 'Peter' ] }
}
Unmarshalled returned item: {
PK: 'SampleNames',
SK: 'SampleNames',
names: Set { 'Mary', 'Paul', 'Peter' }
}
... which makes a lot more sense to me. I expect using the marshall/unmarshall methods from AWS SDK V2's Converter module would work similarly.
Is it possible to delete records from dynamodb based on PrimaryKey and ConditionExpression on SortKey?
Following code sample is throwing an exception for me
DeleteVideoCall = async function (pk, sk) {
let params = {
TableName: this._tableName,
Key: {
pk: { S: pk.toString() },
sk: { S: sk.toString() }
},
ConditionExpression: "begins_with(sk,:sk)",
ExpressionAttributeValues: {
":sk" : { S: sk.toString() + "_" }
}
};
return this._ddb
.deleteItem(params)
.promise()
.then((data) => {
console.log(`Video Call '${pk}/${sk}' deleted`);
return null;
})
.catch((error) => {
console.error(
`Error deleting video room '${pk}/${sk}' (${error})`
);
throw error;
});
};
I want to delete all records that begins_with sk and . For example if sk is 560622 then delete all records where sk begins_with 560622
with the code above I get this error:
Error deleting video room '10900/560622'
(ConditionalCheckFailedException: The conditional request failed)
You can't do that. You need whole key to perform a delete. What you can do:
query items (limit retrieved properties to your PK and SK)
use batch-write to remove multiple items, it also accepts delete requests
I've hit a weird problem where dynamodb.getItem returns {} when I specify a parameter that ends with ID:
var AWS = require("aws-sdk");
AWS.config.update({ region: "ap-southeast-2" });
var dynamodb = new AWS.DynamoDB({ apiVersion: '2012-08-10' });
const tableName = "ReportingDB"
async function logReports() {
try {
var params = {
Key: {
"Category": { "S": "Certification" },
"Report": { "S": "By EmployeeID" }
},
TableName: tableName
};
var result = await dynamodb.getItem(params).promise()
console.log(JSON.stringify(result))
} catch (error) {
console.error(error);
}
}
When I change the value "By EmployeeID" to "By Employee Number" it then works, why?
async function logReports() {
try {
var params = {
Key: {
"Category": { "S": "Certification" },
"Report": { "S": "By Employee Number" }
},
TableName: tableName
};
var result = await dynamodb.getItem(params).promise()
console.log(JSON.stringify(result))
} catch (error) {
console.error(error);
}
}
To reproduce, create a DynamoDB table with two columns Category and Report, then add an item with "Certification" and "By EmployeeID/Number" values respectively.
The error handling in DynamoDB is pretty good, typically it will fail with an error message such as Reserved keyword (eg Using a ProjectionExpression with reserved words with Boto3 in DynamoDB) however with "By EmployeeID" it doesn't produce any error, it just doesn't return anything. Could it be a bug or something I don't know about? I couldn't find it documented.
PS I've inserted the values in the dynamo db using the web console.
The documentation for GetItem explains that:
If there is no matching item, GetItem does not return any data and there will be no Item element in the response.
In other words, if there is no item matching the given key, you'll get an empty result - not an error message. I suspect that this is exactly what is happening in your case.
I don't know why it didn't find an item with "By EmployeeID". I don't think it's a bug involving the letters "ID" :-) I suspect that you simply have a bug in the code which inserted this item, perhaps mis-spelling the word "By EmployeeID". Please look again at the code which inserts this item, or use the AWS DynamoDB UI (or do a Scan request) to inspect the contents of your table manually to see what it actually contains.
Struggling to find an example of how to query a table to return rows with ids from a given list.
The query below throws on the inclusion of IN
var params = {
id: '7deb3df9-552b-47a4-aef3-ad601f141d50'
};
var p = {
TableName: 'players',
KeyConditionExpression: 'id IN (:id)',
ExpressionAttributeValues: buildQuery(params)
};
You can't use "IN" operator with KeyConditionExpression, please see details in this SO question
You may want to use batchGetItem instead of query, which is not so efficient though.
Here is how your params could look like:
var params = {
RequestItems: {
'players': {
Keys: [{
id: "7deb3df9-552b-47a4-aef3-ad601f141d50",
rangeKey: "<range key 1>" // <--- if your table has a range key, you must specify its value here
}, {
id: "<ANOTHER ID 2>",
rangeKey: "<range key 2>"
}, {
id: "<ANOTHER ID 3>",
rangeKey: "<range key 3>"
}]
}
}
};
dynamodbDoc.batchGet(params, function(err, data) {
});
I need to update my dynamo db table by using only partition key. But i got validation exeption.
I have created a table with 3 fields.
id (Partition Key)
name (Sort Key)
age
Then i have triyed to update age field using only id.(tryied to modify age 30 to 40) this is my code
var AWS = require("aws-sdk");
AWS.config.update({
region: "us-east-1",
});
var params = {
TableName: 'test',
Key: { id: '100' },
UpdateExpression: 'set #age = :age ',
ConditionExpression: '#age = :testAge',
ExpressionAttributeNames: { '#age': 'age' },
ExpressionAttributeValues: { ':age': '40', ':testAge': '30' }
};
var docClient = new AWS.DynamoDB.DocumentClient();
docClient.update(params, function (err, data) {
if (err) {
console.log(err);
}
else {
console.log(data);
}
});
But i got error like this.
{ [ValidationException: The provided key element does not match the schema]
message: 'The provided key element does not match the schema',
code: 'ValidationException',
time: Thu Nov 17 2016 22:38:01 GMT+0530 (IST),
requestId: '34PNMFM6CEACQIRHTSV77OI0JRVV4KQNSO5AEMVJF66Q9ASUAAJG',
statusCode: 400,
retryable: false,
retryDelay: 0 }
After getting error, i modified my params variable like this
var params = {
TableName: 'test',
Key: { id: '100',name: 'manaf' },
UpdateExpression: 'set #age = :age ',
ConditionExpression: '#age = :testAge',
ExpressionAttributeNames: { '#age': 'age' },
ExpressionAttributeValues: { ':age': '40', ':testAge': '30' }
};
Using this, updation is successfully completed. How to update table using without sort key?
Currently, the DynamoDB update API doesn't have an option to update the item by partition key only. There is no batchUpdateItem API similar to batchWriteItem as well.
So, if the sort key is not available, get all the sort keys of partition key and update each item for the partition and sort key combination.
For the primary key, you must provide all of the attributes. For
example, with a simple primary key, you only need to provide a value
for the partition key. For a composite primary key, you must provide
values for both the partition key and the sort key.
Sample code:-
You may need to change it for your table. The below code uses "Movies" table which has "yearkey" as partition key and "title" as sort key.
The below code updates the "createdate" attribute for the given hash key "2012".
The variable paramsUpdate is formed based on the query operation. Please update it accordingly for your requirement (i.e. table structure). Logic remains same, you just need to change the table name and key values accordingly.
var AWS = require("aws-sdk");
var creds = new AWS.Credentials('akid', 'secret', 'session');
AWS.config.update({
region : "us-west-2",
endpoint : "http://localhost:8000",
credentials : creds
});
var docClient = new AWS.DynamoDB.DocumentClient();
var hashKey = 2012;
var paramsQuery = {
TableName : "Movies",
KeyConditionExpression : 'yearkey = :hkey',
ExpressionAttributeValues : {
':hkey' : hashKey
}
};
function updateItem(paramsUpdate) {
console.log("Updating the item...");
docClient.update(paramsUpdate, function(err, data) {
if (err) {
console.error("Unable to update item. Error JSON:", JSON.stringify(
err, null, 2));
} else {
console.log("UpdateItem succeeded:", JSON.stringify(data));
}
});
}
docClient.query(paramsQuery, function(err, data) {
if (err) {
console.error("Unable to read item. Error JSON:", JSON.stringify(err,
null, 2));
} else {
console.log(data.Count);
var itemIndex = 0;
while (itemIndex < data.Count) {
console.log('Hashkey to be updated ======>',
data.Items[itemIndex].yearkey,
';Title to be updated ========>',
data.Items[itemIndex].title);
var paramsUpdate = {
TableName : "Movies",
Key : {
"yearkey" : data.Items[itemIndex].yearkey,
"title" : data.Items[itemIndex].title
},
UpdateExpression : "set #createdate = :createdate",
ExpressionAttributeNames : {
'#createdate' : 'createdate'
},
ExpressionAttributeValues : {
':createdate' : '2016-11-17'
},
ReturnValues : 'UPDATED_NEW'
};
updateItem(paramsUpdate);
itemIndex++;
}
}
});
In DynamoDB, partition key + sort key is treated as a "composite primary key", which uniquely identifies an item (on the contrary, Dynamo also supports simple primary key, which only contains partition key). So you need both to update an item. This is the reason that you can have two items with the same partition key but different sort key. So if you only provide the partition keys, Dynamo will get confused with which item to update.
For your current table configuration, the only way to update an item given a partition key is to make a query with only partition key to get all the items, and filter out the one with the intended sort key. Then use the combination of partition key and sort key to update this item.