I have the following dynamodb table and want to remove "oldInfo" from the attribute "accounts" using update_expression = f"REMOVE accounts.oldInfo" however that does not work. Does anyone have any ideas or suggestions for this? Note that the "accounts" attribute is a map and so is "oldInfo".
{
"accounts": {
"oldInfo": {
"oldStuff": []
}
"otherInfo": {
"otherStuff": []
}
}
}
This is the code that I am using to remove "oldInfo"
self.clean_up_dynamo_db.updateitem(table_name=self.table_name,
key={
"partitionKey": item["partitionKey"],
"sortKey": item["sortKey"]
},
update_expression=f"REMOVE {self.attribute}")
I didn't find a clear example in the DDB docs, so I replicated your situation in my AWS account.
I created a map attribute named accounts and gave it keys of oldInfo and otherInfo. I executed the following operation, which removed accounts.oldInfo as expected.
const params = {
TableName: "YOUR_TABLE_NAME",
Key: {
PK: "PARTITION_KEY",
SK: "SORT_KEY"
},
UpdateExpression: "REMOVE accounts.otherInfo"
}
await ddbClient.updateItem(params);
Although this snippet is in JavaScript, it confirms that you are going about the operation correctly according to the DDB API.
Perhaps there is something else going on in your code that is preventing the operation from succeeding? Are you certain the variables you are sending update_item are as you expect them?
Related
Good morning.
I'm quite new to NodeJS / sequelize world and I'm currently facing a problem while trying to display a dashboard on screen.
This dashboard has three filters: two dates (period), client name, and employee name. The user can select none, one, two, or all the filters and my database needs to work accordingly.
That being said, my problem is with Sequelize because I don't know how to treat this problem of parameters not being "always" there.
I've seen this question:
Sequelize optional where clause parameters?
but this answer doesn't work anymore. I also tried another way of building the where clause, but I failed on it as well (mainly due to sequelize operators).
The last thing I tried was to make a single query with all parameters included but try to find some value (or flag) that would make sequelize ignore the parameter, for the case when the parameter was no there*, but it looks like Sequelize doesn't have anything like that.
* I've read a question here that has an answer saying that {} would do the trick but I tried that as well but didn't work.
In summary: I need to make a query that can "change" over time, for example:
Foo.findAll({
where: {
id : 1,
}
});
Foo.findAll({
where: {
id {
[Op.in] : [1,2,3,4,5]
},
name: "palmeiira",
}
});
Do you know a way of doing it without the need of using a lot if / switch statements?
I'm currently using Sequelize v. 5.5.1.
Update
I tried doing as suggested by #Anatoly and created a function to build the parameters. It was something like that. (I tried a "smaller" version just to test)
async function test() {
const where = {};
where[Op.and] = [];
where[Op.eq].push({
id: {
[Op.in]: [1,2,3]
}
});
return where;
}
I setted the return value to a const:
const query = await test()
And tried console.log(query)
The result was: { [Symbol(and)]: [ { id: [Object] } ] }, which made me believe that the problem was parsing the Op part so i tried using 'Op.and' and 'Op.in' to avoid that and it solved this problem, but led to another on sequelize that said Invalid value
Do you have any idea where is my error ?
P.S.: #Anatoly very nice idea you gave me on original answer. Thank you very much.
If these three conditions should work together then you can use Op.and with an array of conditions:
const where = {}
if (datesFilter || clientNameFilter || employeenameFilter) {
where[Op.and] = []
if (datesFilter) {
where[Op.and].push({
dateField: {
[Op.between]: [datesFilter.start, datesFilter.finish]
}
})
}
if (clientNameFilter) {
where[Op.and].push({
name: {
[Op.iLike]: `%${clientNameFilter.value}%`
}
})
}
if (employeenameFilter) {
where[Op.and].push({
employeeName: {
[Op.iLike]: `%${employeenameFilter.value}%`
}
})
}
}
const dashboardItems = await DashboardItem.findAll({ where }, {
// some options here
})
If the conditions should work as alternatives then just replace Op.and with Op.or
I try to update an item in dynamodb by adding a condition, without passing the key in the parameters.
And as soon as my condition is true update. Is it possible to do this?
Below an example of an item:
{
"id" : "bcc2f32e-305e-4469-88e2-463724b5c6a9",
"name" : "toto",
"email" : "toto#titi.com"
}
Where email is unique for items.
I tested this code and it works :
const name= "updateName";
const params = {
TableName: MY_TABLE,
Key: {
id
},
UpdateExpression: 'set #name = :name',
ExpressionAttributeNames: { '#name': 'name' },
ExpressionAttributeValues: { ':name': name },
ReturnValues: "ALL_NEW"
}
dynamoDb.update(params, (error, result) => {
if (error) {
res.status(400).json({ error: 'Could not update Item' });
}
res.json(result.Attributes);
})
But i want to do something like this (replace the Key by conditionExpression):
const params = {
TableName: MY_TABLE,
UpdateExpression: 'set #name = :name',
ConditionExpression: '#email = :email',
ExpressionAttributeNames: {
'#name': 'name',
'#email': 'email'
},
ExpressionAttributeValues: {
':name': name,
':email': email
},
ReturnValues: "ALL_NEW"
}
dynamoDb.update(params, (error, result) => {
if (error) {
res.status(400).json({ error: 'Could not update User' });
}
res.json(result.Attributes);
})
But this code doesn't work.
Any ideas?
You cannot update an item in DynamoDB without using the entire primary key (partition key, and sort key if present). This is because you must specify exactly one record for the update. See the documentation here.
If you want to find an item using a field that is not the primary key, then you can search using a scan (potentially slow and expensive) or by using a Global Secondary Index (GSI) on that field. Either of these methods requires that you do a separate request to find the item in question, and then use its primary key to perform the update.
It sounds like you want to do an update that waits for a condition. That's not how DynamoDb works; it cannot wait for anything (except consistency, I suppose, but that's somewhat different). What you can do is make a request with a condition, and if it fails the condition (returning immediately), make the request again later. If you do this you'll need to be careful to backoff appropriately, or you might end up making a lot of requests very quickly.
The key is a required parameter when doing updates; the condition expression can be used in addition to providing the key, but can't be used instead of the key.
Also, I am not sure you fully understand what the conditionExpression is for - its not like the 'where' clause in an SQL update statement (i.e. update mytable set name='test' where email='myemail.com'.
Instead, logically the conditionExpression in an update would be more like:
update mytable set name='test' where key='12345' but only if quantity >0 - for example,
i.e. you are telling dynamodb the exact key of the record you want updated, and once it finds it it uses the condition expression to determine if the update should proceed - i.e. find the record with id=12345, and change the name to 'test', only of the quantity is greater than 0.
It does not use the conditionExpression to find records to update.
I have created a set using dynamoDB document client . I am able to remove items in this set however when i remove to the last element in the set nothing returns until i make a new post. Then all the other data is displayed.
const params = {
TableName: 'beta-user-' + process.env.NODE_ENV,
Key: {
username: request.username
},
UpdateExpression: "DELETE #features :feature",
ExpressionAttributeNames: { "#features" : "features" },
ExpressionAttributeValues: { ":feature": dynamodb.createSet([request.feature]) },
ReturnValues: "NONE"
};
and im calling it like
const dynamoPromise = dynamodb.update(params).promise();
return await dynamoPromise.then(result => { // stuff })
The UpdateExpression i do not think is wrong
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Expressions.UpdateExpressions.html#Expressions.UpdateExpressions.DELETE
I belive the problem is with ExpressionAttributeValues if i remove
dynamodb.createSet I get many validation errors.
When i make a get request to app i get
{
"message": [
{
"username": "x",
"feature": [
"blah",
"test"
]
},
{
"username": "z",
"feature": [
"blah"
]
},
}
I make a delete request and remove the feature test from username x. This works and returns the same response minus the test feature. I make another delete request to remove blah. Blah is removed however when I make a get request I recieve:
{
"message": {}
}
The other data is returned when i make a new post to that specific user.
EDIT:
I think the issue might be due to dynamo not liking an empty set
The issue was with my return statement in my get request. I assumed that once features were deleted the record would be deleted. I was trying to return features on an object that had no features, therefore, it was erroring out and not returning anything.
I need to get record (-s) from DynamoDB via Dynamoose by non-key attribute. For example, I need to get all records from the table someModel where the field my_field is equal to 111. I do it by the next way:
const data = await someModel.query("my_field").eq("111").exec();
And this code stops executing! I mean that following code after that is not called.
If I change code to this:
const data = await someModel.query("my_field").eq("111");
my query is working, but data contains:
{
"options": {
"all": {
"delay": 0,
"max": 1
}
},
"query": {
"hashKey": {
"name": "my_field",
"value": "111"
}
},
"filters": {},
"buildState": false,
"validationError": null,
"notState": false,
"success": true
}
I understand that above code is prepared query parameters for query but how can I apply them and execute query to DynamoDB? Of course the query code is placing inside async function - that is why await is written there.
I use also serverless framework for describing DynamoDB schemes. But all models I write via dynamoose.
Where am I mistaking?
As mentioned in the documentation here, Dynamoose query returns the value in the callback and not as a promise. Therefore, your response is actually returned in a callback that should come inside the exec part of your query. async/await is valid for promises and not callbacks.
someModel.query("my_field").eq("111").exec((err, data) => {
// Do whatever you wish to with your data
});
I'm trying to retrieve all items from a DynamoDB table that match a FilterExpression, and although all of the items are scanned and half do match, the expected items aren't returned.
I have the following in an AWS Lambda function running on Node.js 6.10:
var AWS = require("aws-sdk"),
documentClient = new AWS.DynamoDB.DocumentClient();
function fetchQuotes(category) {
let params = {
"TableName": "quotient-quotes",
"FilterExpression": "category = :cat",
"ExpressionAttributeValues": {":cat": {"S": category}}
};
console.log(`params=${JSON.stringify(params)}`);
documentClient.scan(params, function(err, data) {
if (err) {
console.error(JSON.stringify(err));
} else {
console.log(JSON.stringify(data));
}
});
}
There are 10 items in the table, one of which is:
{
"category": "ChuckNorris",
"quote": "Chuck Norris does not sleep. He waits.",
"uuid": "844a0af7-71e9-41b0-9ca7-d090bb71fdb8"
}
When testing with category "ChuckNorris", the log shows:
params={"TableName":"quotient-quotes","FilterExpression":"category = :cat","ExpressionAttributeValues":{":cat":{"S":"ChuckNorris"}}}
{"Items":[],"Count":0,"ScannedCount":10}
The scan call returns all 10 items when I only specify TableName:
params={"TableName":"quotient-quotes"}
{"Items":[<snip>,{"category":"ChuckNorris","uuid":"844a0af7-71e9-41b0-9ca7-d090bb71fdb8","CamelCase":"thevalue","quote":"Chuck Norris does not sleep. He waits."},<snip>],"Count":10,"ScannedCount":10}
You do not need to specify the type ("S") in your ExpressionAttributeValues because you are using the DynamoDB DocumentClient. Per the documentation:
The document client simplifies working with items in Amazon DynamoDB by abstracting away the notion of attribute values. This abstraction annotates native JavaScript types supplied as input parameters, as well as converts annotated response data to native JavaScript types.
It's only when you're using the raw DynamoDB object via new AWS.DynamoDB() that you need to specify the attribute types (i.e., the simple objects keyed on "S", "N", and so on).
With DocumentClient, you should be able to use params like this:
const params = {
TableName: 'quotient-quotes',
FilterExpression: '#cat = :cat',
ExpressionAttributeNames: {
'#cat': 'category',
},
ExpressionAttributeValues: {
':cat': category,
},
};
Note that I also moved the field name into an ExpressionAttributeNames value just for consistency and safety. It's a good practice because certain field names may break your requests if you do not.
I was looking for a solution that combined KeyConditionExpression with FilterExpression and eventually I worked this out.
Where aws is the uuid. Id is an assigned unique number preceded with the text 'form' so I can tell I have form data, optinSite is so I can find enquiries from a particular site. Other data is stored, this is all I need to get the packet.
Maybe this can be of help to you:
let optinSite = 'https://theDomainIWantedTFilterFor.com/';
let aws = 'eu-west-4:EXAMPLE-aaa1-4bd8-9ean-1768882l1f90';
let item = {
TableName: 'Table',
KeyConditionExpression: "aws = :Aw and begins_with(Id, :form)",
FilterExpression: "optinSite = :Os",
ExpressionAttributeValues: {
":Aw" : { S: aws },
":form" : { S: 'form' },
":Os" : { S: optinSite }
}
};