Is it possible to delete records from dynamodb based on PrimaryKey and ConditionExpression on SortKey?
Following code sample is throwing an exception for me
DeleteVideoCall = async function (pk, sk) {
let params = {
TableName: this._tableName,
Key: {
pk: { S: pk.toString() },
sk: { S: sk.toString() }
},
ConditionExpression: "begins_with(sk,:sk)",
ExpressionAttributeValues: {
":sk" : { S: sk.toString() + "_" }
}
};
return this._ddb
.deleteItem(params)
.promise()
.then((data) => {
console.log(`Video Call '${pk}/${sk}' deleted`);
return null;
})
.catch((error) => {
console.error(
`Error deleting video room '${pk}/${sk}' (${error})`
);
throw error;
});
};
I want to delete all records that begins_with sk and . For example if sk is 560622 then delete all records where sk begins_with 560622
with the code above I get this error:
Error deleting video room '10900/560622'
(ConditionalCheckFailedException: The conditional request failed)
You can't do that. You need whole key to perform a delete. What you can do:
query items (limit retrieved properties to your PK and SK)
use batch-write to remove multiple items, it also accepts delete requests
Related
I am attempting to delete data from a DynamoDB table.
If I delete data using a partition key, it works.
But when I delete multiple rows using any other fields, it fails.
var params = {
TableName: "test",
Key: {
dmac: dmac,
},
ConditionExpression: "dmac= :dmac"
};
docClient.delete( params, (error) => {
if (error) {
console.log( "Delete data fail" );
} else {
console.log( "Delete data Success" );
}
});
Items (or rows) in DynamoDB are uniquely identified by their primary key. A table can have a simple primary key (a partition key) or a composite primary key (a partition key plus a sort key).
To delete an item, you must provide the full primary key (whether it's a simple partition key or composite partition key plus sort key).
So, if you want to delete items that meet a specific condition, for example cars with maxspeed < 120, then issue a query or scan to identify those items, retrieve the primary keys, and then delete the items in a second operation.
To delete a single item, use DeleteItem. To delete multiple items, use BatchWriteItem. Despite the naming of BatchWriteItem, it can be used to put multiple items or to delete multiple items, and you can target one or more DynamoDB tables in the same API call.
Here is an AWS SDK v2 example of deleting multiple items:
const aws = require("aws-sdk");
const ddb = new aws.DynamoDB({ region: "us-east-1" });
(async () => {
const params = {
RequestItems: {
albums: []
}
};
params.RequestItems.albums.push({
DeleteRequest: {
Key: {
pk: { S: "The Who" },
sk: { S: "Tommy" }
}
}
});
params.RequestItems.albums.push({
DeleteRequest: {
Key: {
pk: { S: "The Beatles" },
sk: { S: "Abbey Road" }
}
}
});
await ddb.batchWriteItem(params).promise();
})();
Here is an AWS SDK v3 example of deleting multiple items:
const {
BatchWriteItemCommand,
DynamoDBClient
} = require("#aws-sdk/client-dynamodb");
(async () => {
const client = new DynamoDBClient({ region: "us-east-1" });
const params = {
RequestItems: {
albums: []
}
};
params.RequestItems.albums.push({
DeleteRequest: {
Key: {
pk: { S: "The Who" },
sk: { S: "Tommy" }
}
}
});
params.RequestItems.albums.push({
DeleteRequest: {
Key: {
pk: { S: "The Beatles" },
sk: { S: "Abbey Road" }
}
}
});
await client.send(new BatchWriteItemCommand(params));
})();
in DynamoDB you can only delete an item using its key (that is: the partition key and the sort key, if it is defined on the table). This is, for example, underlined by the fact that the Key attribute is a required attribute in the canonical specification of the delete operation. See: https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_DeleteItem.html#DDB-DeleteItem-request-Key
This means that if you want to delete an item using other attributes you must first lookup the item by the attributes you do have, extract the key from the returned item, and then delete the item using that key.
The standard solution for "looking up an item by attributes that are not the item's key" is to define a global secondary index (GSI) on the table with those attribute(s) defined as the GSI's key.
EDIT:
I am aware of setDefaultsOnInsert but I cant seem to figure out how to pass in that option...
activity.notify.forEach( user_id => {
query = { 'notification._id': activity._id, user_id, type: "replied to an idea you're subscribed to" }
update = { $set: { notification: activity }, $addToSet: { user_details: user_data } }
options = { setDefaultsOnInsert: true }
bulk.find( query ).upsert().update( update, options )
} )
Currently building out a notification system.
This is what my code currently looks like:
let bulk = Notification.collection.initializeUnorderedBulkOp()
activity.notify.forEach( user_id => {
query = { 'notification._id': activity._id, user_id, type: "replied to an idea you're subscribed to" }
update = { $set: { notification: activity }, $addToSet: { user_details: user_data } }
bulk.find( query ).upsert().update( update )
} )
bulk.execute()
.then( r => console.log( 'r', r ) )
.catch( err => console.log( 'bulk exec err ', err.writeErrors[ 0 ] ) )
What the code does is that it looks through an array called notify that contains user_ids.
It then looks to see if a similar notification for that said user_id already exists. If it does, update it with $addToSet to the array user_details.
The issue with this method is that it doesn't have the timestamps. Within my model, I have the line: timestamps: true but since I am upserting some documents, they dont go through the normal instantiation of a document with: new Notification()
Is there a way or method to automatically include those or do I have to create some helper functions to generate the createdAt and updatedAt
This didnt work for me
Im trying to get an Item from DynamoDB based on Primary Key but it throws me an exception:
ValidationException: The provided key element does not match the schema
Here is how my table looks:
I'm following a tutorial and here is how I wrote my get:
let params = {
TableName: process.env.CALL_NAVEGATION_HISTORY_TABLE,
Key: {
"Id": requestBody.CallSid
}
}
dynamoDb.get(params, function(err, data) {
if(err){
console.log('Error on dynamodb', err);
callback(null, Helpers.xmlTwimlResponse(twiml));
}
console.log(data);
callback(null, Helpers.xmlTwimlResponse(twiml));
});
What is wrong on my code?
Sometimes the most obvious thing is what we miss right in front of our eyes.
let params = {
TableName: process.env.CALL_NAVEGATION_HISTORY_TABLE,
Key: {
"Id": requestBody.CallSid
}
}
The Key name is case-sensitive. If you change it to 'id' it should work fine.
I am trying to overwrite an item in DynamoDB (that uses a primary key called username) by using put as shown below:
console.log('writing commands',existingCommands,message.username);
var t2 = performance();
var writeParams = {
Item: {
username: message.username,
commands: existingCommands // Sorry for the confusing name, due to deepExtend existingCommands are the new commands
},
TableName: TableName
};
docClient.put(writeParams, function(err, data){
if(err){
console.error('error',err);
} else {
console.log('write result',data);
var t3 = performance();
console.info('delete & write performance',(t3-t2).toFixed(3));
}
// End function
context.done();
});
That works for:
Inserting a new item where the username doesn't exist.
Updating an item that matches the schema of the Item i'm trying to insert, for example, I'm trying to insert that item:
{
"username":"ausin441062133",
"commands": {
"command1":"command",
"command2":"command"
}
}
and if there's an item that matches the schema and the username it'll get overwritten, i.e.
{
"username":"ausin441062133",
"commands": {
"command1":"I will be overwritten",
"command2":"I will be overwritten"
}
}
But when there's an item with the exact username but different schema, it doesn't work, i.e.
{
"username":"ausin441062133",
"commands": {
"command1":"I will NOT be overwritten"
}
}
What command do I need to use to overwrite an existing item if it matches the username?
Eventually as Dmitry suggested update works but it needs some different params as opposed to put here's my code:
// Step 3 write command back
console.log('writing commands',existingCommands,message.username);
var t2 = performance();
var updateParams = {
Key: {
username: message.username
},
UpdateExpression: "set commands = :c",
ExpressionAttributeValues: {
":c":existingCommands
},
ReturnValues: "UPDATED_NEW",
TableName: TableName
};
docClient.update(updateParams, function(err, data){
if(err){
console.error('error',err);
} else {
console.log('write result',data);
var t3 = performance();
console.info('delete & write performance',(t3-t2).toFixed(3));
}
// End function
context.done();
});
Im new to DynamoDB and have a table which is "feeds" and partition key is "id" and i have 3 other attributes which are "category", "description", "pubDate".
I want to query the "category" attribute. But it doesn't work, because i can only query the partition key (hashkey), if im right.
Now my query is that which doesnt work;
let category = event.category;
const params = {
Key: {
"category": {
S: category
}
},
TableName: "feeds"
};
dynamodb.getItem(params, function (err, data) {
if (err) {
console.log(err);
callback(err);
}
else {
console.log(data);
callback(null, data);
}
});
How can i make it work? I tried to write a scan query but i couldn't understand the documentation of AWS good.
Edit: I did make it work with the help of Dunedan. Here is the working code,
var params = {
TableName: 'feeds',
IndexName: 'category-index',
KeyConditionExpression: 'category = :category',
ExpressionAttributeValues: {
':category': 'backup',
}
};
var docClient = new AWS.DynamoDB.DocumentClient();
docClient.query(params, function(err, data) {
if (err) callback(err);
else callback(null, data);
});
If your application will regularly query for the category, you should check out Global Secondary Indexes (GSI), which allow you to generate a projection of your data with another key than the original hash key as the key you can use to query.
Scanning and filtering as you suggested doesn't scale very well, as it fetches all data in the table and just filters the results.