I am learning DynamoDB and AWS serverless stack. I see that a lot of tutorials suggest using AWS.DynamoDB.DocumentClient. For example, to create an item:
const dynamodb = new AWS.DynamoDB.DocumentClient();
and then
try {
await dynamodb
.put({
TableName: process.env.DISHES_TABLE_NAME,
Item: dish,
})
.promise();
} catch (error) {
console.error(error);
throw new createError.InternalServerError(error);
}
But the doc says that put
Creates a new item, or replaces an old item with a new item by
delegating to AWS.DynamoDB.putItem()
I am confused why not use AWS.DynamoDB.putItem in the first place or when to use which one.
Thank you!
From the documentation the DocumentClient is just an abstract class to make it easier to implement.
The document client simplifies working with items in Amazon DynamoDB by abstracting away the notion of attribute values. This abstraction annotates native JavaScript types supplied as input parameters, as well as converts annotated response data to native JavaScript types.
You're free to choose whichever method you want, however by using the DocumentClient class you would be having less control over the processing or manipulation of your data.
Related
I'm using Drivine, with Neo4j. The example shows how to inject a query stored in a separate file. How can I simply inline a query as a string?
The idea behind having a separate file is so that queries can be profiled, tested and have a life-cyle in their own right. You don't have to use this if it doesn't suit you.
The example also shows how to create inline queries:
async countAllVertices(): Promise<number> {
const results = await this.persistenceManager.query<any>(
new QuerySpecification(`match (n) return count(n) as count`)
);
return results[0].count;
}
For simple queries that don't need to be profiled or heavily tested, inline is easier.
I recently started using Node.js + Express.js (generated with pug) + pg-promise for handling db.
My first target is to obtain data from Postgres (already set up) and display it pretty using render and pug. Let's say it is user list from Users table.
On this restful tutorial I have learned how to get data and return it as JSON - it worked.
Based on Mozilla's tutorial I seperated my code:
routes/users.js: where for '/' I call user_controller.user_list method (using router.get)
controllers/userController.js I have exported user_list where I would like to ask model for data and call render if I have results
queries.js which is kinda my model? But I'm not sure. It has API: connection to db with promises and one function for every query I am going to use in Controllers. I believe I should have like one Model file per table (or any logical entity) but where to store pgp connections?
This file is based on first tutorial I mentioned
// queries.js (connectionString is set properly to my postgres)
var pgp = require('pg-promise')(options);
var db = pgp(connectionString);
function getUsers(req, res, next) {
db.any('SELECT (user_id, username) FROM public.users ORDER BY user_id ASC LIMIT 1000')
.then(function (data) {
res.json({ data: data });
})
.catch(function (err) {
return next(err);
});
}
module.exports = {
getUsers: getUsers
};
Here starts my problem as most tutorials uses mongoose which is very model-db-schema-friendly and what I have is simple 'SELECT ...' string I pass to pg-promise's any() function.
Therefore I have no model class like User.
In userControllers.js I don't know how to call getUsers() to handle its data. Returning JS object from getUsers() would be nice.
Also: where should I call render? In controller or only in
db.any(...).then(function (data) { <--here--> })
Before, I also tried to embed whole Postgres handling into Controller but from db.any() I got this array for handling:
[{ row: '(1,John)' },{ row: '(2,Amy)' },{ row: '(50,Peter)' } ]
Didn't know how go from there as I probably lost my API functionality as well ;-)
I am browsing through multiple tutorials how to handle MVC but usually they handle MongoDB and
satisfy readers with res.send() not render().
I am not sure that I understand what your question is exactly about, but since I do not have enough reputation to comment, I'll do my best to help you with your interrogations. :)
First, regarding the queries.js file, it is IMO not exactly a model, but rather a DAO (Data Access Object) file. DAO comes between you Model (which is actually you database) and your Controller layers. There usually is a DAO file per object (User, Pet, whatever you want) in your data model.
When the data model is rather complex, it can be useful to use an Object Relational Mapping (ORM) such as Mongoose to map your database and execute complexe processes on your objects. In such a case, you might need a specific file per object so as to describe your model and store your queries. But since you don't need an ORM, you DAO can directly interact with your database. That is why you do not have a User.js file.
Regarding the way the db object should be used, I think you should refer directly to pg-promise documentation on the matter.
IMPORTANT: For any given connection, you should only create a single
Database object in a separate module, to be shared in your application
(see the code example below). If instead you keep creating the
Database object dynamically, your application will suffer from loss in
performance, and will be getting a warning in a development
environment (when NODE_ENV = development)
As a matter of fact, a db object in pg-promise sort of represents the database itself and is actually designed for the simultaneous use of several databases, which does not seem to be your case for the moment.
Finally, when it comes to the render function, I believe it should be in the controller, as your DAO is not supposed to know how the data it has gathered is going to be used.
Modularity is always a time-saving choice on the long-term.
Furthermore, note that you might later need a Business Layer between your DAO and your controller, in order to preprocess and postprocess data you are going to persist or to display. In such a case, if you need for instance to ask for data from your database, you will need to render data after it is processed by the Business layer. If the render is made in the DAO layer, it will not be possible.
In the link I provided earlier to pg-promise's db object connection, you will also find documentation on the any() method. You might already have looked it up.
It specifically states that it returns
A promise object that represents the query result:
When no rows are returned, it resolves with an empty array.
When 1 or more rows are returned, it resolves with the array of rows.
so your returned data is a JS Array. If you want to make it a JS object, just use
JSON.stringify(yourArray) to process your data before rendering it in your controller.
But I wonder if Pug is not able to use your data directly.
Also, if you cannot get any data out of your DAO, maybe you should check that your data object is not empty, as such a case is tolerated by the any() method. If you expect your query to always return something, you might want to consider using the many() or the one() methods.
I hope this helps you.
I tracked down this package. Generally it's pretty nice. But it seems to lack support for Projection Expressions. What is your tool of choice for dynamodb in node/typescript?
I'm not fan of the data mappers listed here because they tend to wrap the table data, or are abandoned as projects.
If typescript is an option, we use https://github.com/shiftcode/dynamo-easy. Which also does not support Projection Expression, but the underlying params can always be accessed and manipulated, so adding some non-supported feature is easy.
import { DynamoStore } from '#shiftcoders/dynamo-easy'
const queryRequest = new DynamoStore(PersonModel)
.query()
.wherePartitionKey('2018-01')
.whereSortKey().beginsWith('a')
.limit(1)
const queryParams = queryRequest.params
queryParams.ProjectionExpression = 'projectionExpression'
// also add expression attribute names if required
queryParams.ExpressionAttributeNames = {'#someExpressionAttributeName': 'someExpressionAttributeName'}
// you can also use new DynamoDB().query(queryParams), but we just use the preconfigured wrapped client
queryRequest.dynamoDBWrapper.makeRequest('query', queryParams)
.then(r => console.log('first found item with projection expression:', r))
full disclosure: I am one of the authors of this library
We use dynogels, it is maintained until to date.
https://github.com/clarkie/dynogels
If you need a GUI to construct your query, try using the "DynamoDB Visual Query Builder" I've built: https://dynobase.dev/dynamodb-query-builder/
This is more of a call for help with an exception when calling insertEntity().
I'm using Nodejs on Azure and editing in Monaco, and I've NPM-installed the latest version of azure storage.
The exception I encounter is: (full stack trace at the bottom)
Unaught exception: Error: Parameter entityDescriptor.PartitionKey for function entityOperation should be an object at ArgumentValidator._.extend.object
I'm basically taking my object to save, and creating 2 new properties: PartitionKey and RowKey. I give them string values. I'm following the examples. I'm not using entityGenerator, as the samples here don't, whereas the examples on the Azure Node developers portal do. I wouldn't mind using entityGenerator on the storage-specific properties if required, but the samples in the node azure github repo seem to suggest that you can use simple strings. The entityGenerator looks a bit ugly and cumbersome, honestly, as you'd have to code extra around the entity when you bring it back.
How can I adjust my code to solve this problem and call insertEntity() with success?
exports.saveTally = function(tally, callback) {
var tableSvc = getAzureTableService();
tableSvc.createTableIfNotExists("tally", function(error, result, response) {
if (!error) {
tally.PartitionKey="tally";
tally.RowKey = tally.id;
tableSvc.insertEntity("tally", tally, function(error, result, response) {
if (error) {
console.log("*Error saving tally " + error.toString());
}
else {
callback(tally.id);
}
});
}
});}
The location of the Azure storage client library has changed to https://github.com/Azure/azure-storage-node. The samples you’re using are from the old location and from an older version of the library which is why they’re not working. You’ll find updated samples and code at the new location.
In the newer version, an Edm type must be specified for each table entity. This is because type is stored in the storage service and we want to make sure that we are storing what you intend. Each table entity is an object with the form {_:value, $:Edm.Type}.
Entity generator is a convenience feature which makes it simpler to construct table entity objects. We return entities in the form just mentioned and using this convenience feature will not change that in any way.
Maybe the question does not apply to dynamoDB due to it not being Relational Db.
However, I'm looking for a good object mapper which can be used in nodejs and aws sdk to map existing model classes to dynamoDB tables. Does anyone have experience with this issue/question, or have you used such a module/library?
If you are looking for schema:
https://github.com/clarkie/dynogels (well supported forked from vogels which has been abandoned)
https://github.com/automategreen/dynamoose (inspired by Mongoose)
If you are looking for something to throw javascript objects (even circular graphs) to:
https://github.com/aaaristo/dyngodb (alpha)
https://github.com/aaaristo/angular-gson-express-dyngodb
dyngodb has experimental support for full-text search, and transactions too.
Both are based on aws-sdk.
Also worth considering is simple marshallers, which just translate between the dynamoDB format and regular js objects or JSON.
DynamoDb-Data-Types
https://github.com/kayomarz/dynamodb-data-types
https://www.npmjs.com/package/dynamodb-data-types
"This utility helps represent AWS DynamoDb data types. It maps (marshalls) JavaScript data into the format required by DynamoDb."
dynamoDb-marshaler
https://github.com/CascadeEnergy/dynamoDb-marshaler
https://www.npmjs.com/package/dynamodb-marshaler
"Translates sane javascript objects (and JSON) into DynamoDb format and vice versa." [does not support B type.]
Update 2016-06:
Just discovered that the AWS SDK now does this for you. Their documentation is only partially converted so I guess this is a recent addition. Read about it here.
But these marshallers are still useful because there are circumstances where you can't use the new document client, eg. when processing a dynamoDB stream.
You could also try: https://dynamoosejs.com/. It is inspired by mongoose again.
If you are using Typescript, dynamo-easy might be a good option. Just add some decorators to your model and start using it.
import { Model, PartitionKey, DynamoStore } from '#shiftcoders/dynamo-easy'
#Model()
export class Person {
#PartitionKey()
id: string
name: string
yearOfBirth: number
}
const personStore = new DynamoStore(Person)
personStore
.scan()
.whereAttribute('yearOfBirth').equals(1958)
.exec()
.then(res => console.log('ALL items with yearOfBirth == 1958', res))
It uses the AWS DynamoDB sdk but takes care of the mapping between JS and DynamoDB types and provides a simple to use fluent API.
full disclosure: I am one of the authors of this library
After looking over all the posts I landed on https://github.com/awspilot/dynamodb-oop
It doesn't hide the API but instead just wraps it in a nice, fluent way with promises even and you inject your version of the aws-sdk. It's similar to dynamodb-data-types but also wraps the methods too (not just the data types).
Extra bonus, the same author has https://github.com/awspilot/dynamodb-sql Didn't use the sql wrapper but I can see how some people may prefer that.
Dynamoose is obviously inspired by mongoose and is a good choice if you have a well-defined schema and/or want to be abstracted away from the DynamoDB details.
Have you seen dynasaur? It seems to be the type of thing you're looking for, but I have not used it myself. There's also dynamodb-data-types which is not an ORM, but makes it easy to convert to/from standard JavaScript objects.