I tried to create GlobalSecondaryIndexes with serverless and query for it later. I get an
Query condition missed key schema element: id
error doing this, which makes sense but my plan is to query for all elements at a specific day.
var params = {
TableName: process.env.DYNAMODB_TABLE_LIGHTHOUSE,
KeyConditionExpression: 'fetchDate = :fetchDate',
ExpressionAttributeValues: {
':fetchDate': { S: '2019-05-13' }
}
};
const dbResult = await dynamoDb.query(params).promise()
console.log(dbResult)
This is the serverless part for the dynamodb. Probably here is something missing?
DynamoDbTableExpenses:
Type: 'AWS::DynamoDB::Table'
Properties:
AttributeDefinitions:
-
AttributeName: id
AttributeType: S
-
AttributeName: fetchDate
AttributeType: S
KeySchema:
-
AttributeName: id
KeyType: HASH
GlobalSecondaryIndexes:
- IndexName: fetchDateIndex
KeySchema:
- AttributeName: fetchDate
KeyType: HASH
Projection:
ProjectionType: ALL
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:provider.environment.DYNAMODB_TABLE_LIGHTHOUSE}
In the web frontend it looks like this:
You need to add the IndexName attribute. Try:
var params = {
TableName: process.env.DYNAMODB_TABLE_LIGHTHOUSE,
IndexName: 'fetchDateIndex',
KeyConditionExpression: 'fetchDate = :fetchDate',
ExpressionAttributeValues: {
':fetchDate': '2019-05-13'
}
};
See API Query
The Query operation finds items based on primary key values. You can
query any table or secondary index that has a composite primary key (a
partition key and a sort key).
Related
So I was stuck for a while now with dynamo db's batchGet operation.
Here is my table definition (in serverless)
Resources:
MyTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: MyTable
TimeToLiveSpecification:
AttributeName: ttl
Enabled: true
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: myId
AttributeType: S
- AttributeName: otherId
AttributeType: S
KeySchema:
- AttributeName: myId
KeyType: HASH
GlobalSecondaryIndexes:
- IndexName: otherIdIndex
KeySchema:
- AttributeName: myId
KeyType: HASH
- AttributeName: otherId
KeyType: RANGE
Projection:
ProjectionType: ALL
here is how I do batchGet in my function.
await getClient().batchGet({
RequestItems: {
'MyTable': {
Keys: [
{'myId': { 'S': 'z12345' }},
{'myId': { 'S': 'z12346' }},
],
ProjectionExpression: 'myId, otherId',
},
},
}).promise();
however, for some reason, I only get an exception The provided key element does not match the schema Did I missed something?
Thanks in advance!
Edit
Searching deeper, I stumbled upon this answer from other post (https://stackoverflow.com/a/32238597/969645). I just realized that I was using the higher-level of the SDK so I have to change my code like this (without defining the type):
await getClient().batchGet({
RequestItems: {
'MyTable': {
Keys: [
{'myId': 'z12345'},
{'myId': 'z12346'},
],
ProjectionExpression: 'myId, otherId',
},
},
}).promise();
Now the The provided key element does not match the schema error is gone but I still end up with Internal server error which unfortunately doesn't provide any clue in the logs. Any ideas?
BatchGet - gets the items using the primary key. In your case the PrimaryKey is NOT the partition key but both partition and sort key.
Pk = partition key AND sort key
Your code should be
await getClient().batchGet({
RequestItems: {
'MyTable': {
Keys: [
{'myId': 'z12345', 'otherId': 'whatever value'},
{'myId': 'z12346', 'otherId': 'whatever value'},
],
ProjectionExpression: 'myId, otherId',
},
},
}).promise();
Important to understand is that the UNIQUE IDENTIFIER (aka SQL id) is defined in dynamo by using both partition + sort key
I am new to aws dynamodb and hit the query as shown below
let params = {
TableName: tableName
};
if (from && to) {
params.FilterExpression = 'createdOn >= :from and createdOn <= :to';
params.ExpressionAttributeValues = { ":from": from, ":to": to }
}
return new Promise((resolve, reject) => {
docClient.scan(params, (err, data) => {
if (err) {
//do stuff
}else{//do stuff}
});
});
and here is my db schema
const ApplicationDataSchema = {
TableName: "data1", //config.get('aws_tables.APPLICATIONS_DATA'),
KeySchema: [
{ AttributeName: "id", KeyType: "HASH" },
{ AttributeName: 'createdOn', KeyType: 'RANGE' }
],
AttributeDefinitions: [
{ AttributeName: "id", AttributeType: "S" },
{ AttributeName: 'createdOn', AttributeType: 'N' }
],
ProvisionedThroughput: {
ReadCapacityUnits: 10,
WriteCapacityUnits: 10
}
}
is there any need to change in schema I am not getting the correct result.
createdOn field contains epoc time eg:"1548659664131"
If you want to use a filter expression with 2 dates, you're probably looking for the BETWEEN statement.
'#createdOn between :from and :to'
I suggest that you take a look at this guide on AWS for using dynamodb with javascript.
I am looking to fetch the chat history. Now each chat message is saved as an item in the dynamodb and I want to fetch the last say 10 msgs. I am facing issue in that.
Please see,
I have defined my table as:
chatTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: Chats
KeySchema:
- AttributeName: groupId
KeyType: HASH
- AttributeName: timestamp
KeyType: RANGE
GlobalSecondaryIndexes:
- IndexName: timestampIndex
KeySchema:
- AttributeName: groupId
KeyType: HASH
- AttributeName: timestamp
KeyType: RANGE
Projection:
ProjectionType: ALL
ProvisionedThroughput:
ReadCapacityUnits: 10
WriteCapacityUnits: 5
AttributeDefinitions:
- AttributeName: groupId
AttributeType: S
- AttributeName: timestamp
AttributeType: N
ProvisionedThroughput:
ReadCapacityUnits: 10
WriteCapacityUnits: 5
And I am trying to query to fetch chat history of one of the group as:
let getChatParams = {
TableName: "Chats",
IndexName: "timestampIndex",
Limit: 10,
ScanIndexForward: true,
KeyConditionExpression: "groupId = :groupId",
ExpressionAttributeValues: { ":groupId": groupId }
};
let groupChat = await docClient.query(getChatParams).promise();
But I am getting an error,
ValidationException: Query condition missed key schema element
Can you please help what i have to do to make this query run properly.
Thanks.
In your KeyConditionExpression you are specifying an attribute called groupId, but your partition key is called id. The Query wont work unless you specify id.
I'm completely new to DynamoDB, and I can't find out how query works.
So, in my case, I have a table(collection/etc) Users with this fields and AWS types in brackets:
id [type String], name [type String], email[type String], ref_code[type Map], register_date[type String]
And my table Indexes are
KeySchema: [
{ AttributeName: 'id', KeyType: 'HASH' },
{ AttributeName: 'register_date', KeyType: 'RANGE' },
],
AttributeDefinitions: [
{ AttributeName: 'id', AttributeType: 'S' },
{ AttributeName: 'register_date', AttributeType: 'S' },
],
I've read documentation here, here, here and a lot of other info too, but still can't understand how can I query user by his/her name.
So, in MySQL world if I have primary index on field Id, I still can query user data by his/her name, like this SELECT * FROM users WHERE name = '<name>';
But, this behaviour isn't work with DynamoDB.
I've tried to query like this:
var params = {
TableName: 'Users',
IndexName: 'name',
KeyConditionExpression: '#u_name = :name',
ProjectionExpression: '#u_name, lives, fb_account',
ExpressionAttributeNames: {
'#u_name': 'name',
},
ExpressionAttributeValues: {
':name': 'Mykhaylo',
},
};
And a lot of other options, but nothing worked out.
So, my question is How to make query in AWS DynamoDB?
EDIT
If I need to set more than 5 global second indexes, how should I perform my query? Is it possible at all?
You cannot directly query on the field which is not hash key. You have to either use scan with filter on name like
var params = {
TableName: 'Users',
FilterExpression: '#u_name = :value',
ExpressionAttributeNames: {
'#u_name': 'name'
},
ExpressionAttributeValues: {
':value': 'Mykhaylo'
},
};
docClient.scan(params, function(err, data) {
////////
});
or you need to create a gsi with name as hash key. Then you can use Query to get the result according to name
You can run sql queries against dynamodb by doing the following:
open ubuntu bash
pip install dql
dql -r eu-west-2
scan * from table_name;
I am trying, and failing, to delete a record with a condition. I keep getting this error: The provided key element does not match the schema.
This is my definition in the yml:
resources:
Resources:
vuelosTable:
Type: 'AWS::DynamoDB::Table'
DeletionPolicy: Delete
Properties:
AttributeDefinitions:
-
AttributeName: id
AttributeType: S
-
AttributeName: vuelta
AttributeType: S
KeySchema:
-
AttributeName: id
KeyType: HASH
-
AttributeName: vuelta
KeyType: RANGE
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:provider.environment.DYNAMODB_TABLE}
These are the params when trying to delete:
params = {
RequestItems: {
[process.env.DYNAMODB_TABLE]: [{
DeleteRequest: {
Key: {
"vuelta": "2017-09-09"
}
}
}]
}
};
I know it's something I am not getting, but I don't know what. Any ideas?
Your table key is both id and vuelta but you're only providing vuelta in the delete request. Modify the key in your delete request so it contains both the id and vuelta.
Also, depending on your client library may need to specify
Key: {
id: { S: "some value" },
vuelta: { S: "some value" }
}