Query AWS DynamoDB - node.js

I'm completely new to DynamoDB, and I can't find out how query works.
So, in my case, I have a table(collection/etc) Users with this fields and AWS types in brackets:
id [type String], name [type String], email[type String], ref_code[type Map], register_date[type String]
And my table Indexes are
KeySchema: [
{ AttributeName: 'id', KeyType: 'HASH' },
{ AttributeName: 'register_date', KeyType: 'RANGE' },
],
AttributeDefinitions: [
{ AttributeName: 'id', AttributeType: 'S' },
{ AttributeName: 'register_date', AttributeType: 'S' },
],
I've read documentation here, here, here and a lot of other info too, but still can't understand how can I query user by his/her name.
So, in MySQL world if I have primary index on field Id, I still can query user data by his/her name, like this SELECT * FROM users WHERE name = '<name>';
But, this behaviour isn't work with DynamoDB.
I've tried to query like this:
var params = {
TableName: 'Users',
IndexName: 'name',
KeyConditionExpression: '#u_name = :name',
ProjectionExpression: '#u_name, lives, fb_account',
ExpressionAttributeNames: {
'#u_name': 'name',
},
ExpressionAttributeValues: {
':name': 'Mykhaylo',
},
};
And a lot of other options, but nothing worked out.
So, my question is How to make query in AWS DynamoDB?
EDIT
If I need to set more than 5 global second indexes, how should I perform my query? Is it possible at all?

You cannot directly query on the field which is not hash key. You have to either use scan with filter on name like
var params = {
TableName: 'Users',
FilterExpression: '#u_name = :value',
ExpressionAttributeNames: {
'#u_name': 'name'
},
ExpressionAttributeValues: {
':value': 'Mykhaylo'
},
};
docClient.scan(params, function(err, data) {
////////
});
or you need to create a gsi with name as hash key. Then you can use Query to get the result according to name

You can run sql queries against dynamodb by doing the following:
open ubuntu bash
pip install dql
dql -r eu-west-2
scan * from table_name;

Related

DynamoDB - Delete item - The provided key element does not match the schema

I've gone through many answers on stack overflow for similar questions. But the problem is a bit different here. I know that I should be sending the key to delete the operation (pk or pk/sk). But in my case, I only have one key in schema (verified by describe-table)
{ .....
"TableName": "**********",
"KeySchema": [
{
"AttributeName": "user_id",
"KeyType": "HASH"
}
],
....}
The table has been create using following sam code
DynamoDbTable:
Type: AWS::DynamoDB::Table
Properties:
AttributeDefinitions:
- AttributeName: "user_id"
AttributeType: "S"
KeySchema:
- AttributeName: "user_id"
KeyType: "HASH"
ProvisionedThroughput:
ReadCapacityUnits: 5
WriteCapacityUnits: 5
SSESpecification:
SSEEnabled: True
TableName: !Ref TableName
Here are the params that I'm creating for the delete call
var docClient = new AWS.DynamoDB.DocumentClient({
apiVersion: '2012-08-10',
region: process.env.AWS_REGION
});
const deleteParams = {
TableName: process.env.TABLE_NAME,
Key: {
user_id: connectionData.Items[0].user_id
}
};
try {
let response = await docClient.delete(deleteParams).promise();
console.log('deleted');
} catch (err) {
return {
statusCode: 500,
body: 'Failed to disconnect: ' + JSON.stringify(err)
};
}
Any help will be much appreciated

DynamoDB - how to read a record by primary key values?

I'm trying to get an item from a DynamoDB table, but this error "ValidationException: The provided key element does not match the schema" keeps happening.
These are the parameters i created the table:
TableName : "Users-test",
KeySchema: [
{ AttributeName: "id", KeyType: "HASH"}, //Partition key
{ AttributeName: "email", KeyType: "RANGE" } //Sort key
],
AttributeDefinitions: [
{ AttributeName: "id", AttributeType: "S" },
{ AttributeName: "email", AttributeType: "S" }
]
And i'm trying to access the data like this:
const docClient = new AWS.DynamoDB.DocumentClient();
const params = {
TableName: 'Users-test',
Key:{
id: "someID",
email: "example#example.com"
}
};
docClient.get(params, function...
I know the record exists on the database and i can filter by id using the aws console, but actually using code i cannot get a record using the primary key, i tried to remove the "email" field from params variable, but it returned the same error.
You can this,
const config = {
region: 'eu-central-1'
};
const docClient = new AWS.DynamoDB.DocumentClient(config);
const params = {
TableName: 'Users-test',
Key:{
"someId",
"example#example.com"
}
};
docClient.get(params, function...
Also, you can make your function async
const docClient = new AWS.DynamoDB.DocumentClient(config);
const params = {
TableName: 'Users-test',
Key:{
"someId",
"example#example.com"
}
};
const result = await docClient.get(params).promise();

DynamoDB BatchGet always gives "The provided key element does not match the schema"

So I was stuck for a while now with dynamo db's batchGet operation.
Here is my table definition (in serverless)
Resources:
MyTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
TableName: MyTable
TimeToLiveSpecification:
AttributeName: ttl
Enabled: true
BillingMode: PAY_PER_REQUEST
AttributeDefinitions:
- AttributeName: myId
AttributeType: S
- AttributeName: otherId
AttributeType: S
KeySchema:
- AttributeName: myId
KeyType: HASH
GlobalSecondaryIndexes:
- IndexName: otherIdIndex
KeySchema:
- AttributeName: myId
KeyType: HASH
- AttributeName: otherId
KeyType: RANGE
Projection:
ProjectionType: ALL
here is how I do batchGet in my function.
await getClient().batchGet({
RequestItems: {
'MyTable': {
Keys: [
{'myId': { 'S': 'z12345' }},
{'myId': { 'S': 'z12346' }},
],
ProjectionExpression: 'myId, otherId',
},
},
}).promise();
however, for some reason, I only get an exception The provided key element does not match the schema Did I missed something?
Thanks in advance!
Edit
Searching deeper, I stumbled upon this answer from other post (https://stackoverflow.com/a/32238597/969645). I just realized that I was using the higher-level of the SDK so I have to change my code like this (without defining the type):
await getClient().batchGet({
RequestItems: {
'MyTable': {
Keys: [
{'myId': 'z12345'},
{'myId': 'z12346'},
],
ProjectionExpression: 'myId, otherId',
},
},
}).promise();
Now the The provided key element does not match the schema error is gone but I still end up with Internal server error which unfortunately doesn't provide any clue in the logs. Any ideas?
BatchGet - gets the items using the primary key. In your case the PrimaryKey is NOT the partition key but both partition and sort key.
Pk = partition key AND sort key
Your code should be
await getClient().batchGet({
RequestItems: {
'MyTable': {
Keys: [
{'myId': 'z12345', 'otherId': 'whatever value'},
{'myId': 'z12346', 'otherId': 'whatever value'},
],
ProjectionExpression: 'myId, otherId',
},
},
}).promise();
Important to understand is that the UNIQUE IDENTIFIER (aka SQL id) is defined in dynamo by using both partition + sort key

Serverless and GlobalSecondaryIndexes - Query condition missed key schema element with dynamodb

I tried to create GlobalSecondaryIndexes with serverless and query for it later. I get an
Query condition missed key schema element: id
error doing this, which makes sense but my plan is to query for all elements at a specific day.
var params = {
TableName: process.env.DYNAMODB_TABLE_LIGHTHOUSE,
KeyConditionExpression: 'fetchDate = :fetchDate',
ExpressionAttributeValues: {
':fetchDate': { S: '2019-05-13' }
}
};
const dbResult = await dynamoDb.query(params).promise()
console.log(dbResult)
This is the serverless part for the dynamodb. Probably here is something missing?
DynamoDbTableExpenses:
Type: 'AWS::DynamoDB::Table'
Properties:
AttributeDefinitions:
-
AttributeName: id
AttributeType: S
-
AttributeName: fetchDate
AttributeType: S
KeySchema:
-
AttributeName: id
KeyType: HASH
GlobalSecondaryIndexes:
- IndexName: fetchDateIndex
KeySchema:
- AttributeName: fetchDate
KeyType: HASH
Projection:
ProjectionType: ALL
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
TableName: ${self:provider.environment.DYNAMODB_TABLE_LIGHTHOUSE}
In the web frontend it looks like this:
You need to add the IndexName attribute. Try:
var params = {
TableName: process.env.DYNAMODB_TABLE_LIGHTHOUSE,
IndexName: 'fetchDateIndex',
KeyConditionExpression: 'fetchDate = :fetchDate',
ExpressionAttributeValues: {
':fetchDate': '2019-05-13'
}
};
See API Query
The Query operation finds items based on primary key values. You can
query any table or secondary index that has a composite primary key (a
partition key and a sort key).

Date range query not works in dynamodb

I am new to aws dynamodb and hit the query as shown below
let params = {
TableName: tableName
};
if (from && to) {
params.FilterExpression = 'createdOn >= :from and createdOn <= :to';
params.ExpressionAttributeValues = { ":from": from, ":to": to }
}
return new Promise((resolve, reject) => {
docClient.scan(params, (err, data) => {
if (err) {
//do stuff
}else{//do stuff}
});
});
and here is my db schema
const ApplicationDataSchema = {
TableName: "data1", //config.get('aws_tables.APPLICATIONS_DATA'),
KeySchema: [
{ AttributeName: "id", KeyType: "HASH" },
{ AttributeName: 'createdOn', KeyType: 'RANGE' }
],
AttributeDefinitions: [
{ AttributeName: "id", AttributeType: "S" },
{ AttributeName: 'createdOn', AttributeType: 'N' }
],
ProvisionedThroughput: {
ReadCapacityUnits: 10,
WriteCapacityUnits: 10
}
}
is there any need to change in schema I am not getting the correct result.
createdOn field contains epoc time eg:"1548659664131"
If you want to use a filter expression with 2 dates, you're probably looking for the BETWEEN statement.
'#createdOn between :from and :to'
I suggest that you take a look at this guide on AWS for using dynamodb with javascript.

Resources