I am using Serverless NodeJs with PostgreSQL and want to start tracing Sequelize queries with AWS X-ray. The documentation only shows how we can trace the pg clients but doesn't state how we can do the same for Sequelize ORM. Can anyone help with this?
Your question is answered here on our github repo. The X-Ray Node SDK doesn't currently support tracing of sequelize queries. We have added it to our backlog. Please stay tuned.
Thanks!
This only capture db calls. This is how you can do it. dialect need to be specific
const Sequelize = require('sequelize');
const AWSXRay = require('aws-xray-sdk');
AWSXRay.captureHTTPsGlobal(require('https'));
const sequelize = () => {
return new Sequelize(database, username, password, {
host: host,
dialect: 'mysql',
dialectModule: AWSXRay.captureMySQL(require('mysql2')),
});
};
Related
I installed redis and mongodb in VM, I'm trying to speed the response time of my mongoose find() requests .. I used lean() and it's working good .. then I found redis and I installed it and I followed this tutorial to set up redis in my controllers :
https://epsagon.com/development/using-redis-to-optimize-mongodb-queries/
I created the cache.js in /services with connection details and then in my controller code I imported :
const { clearKey } = require("../services/cache");
And in my find() I added .cache() :
await Book.find({ author: req.query.author }).cache();
I want to know how the app is knowing the .cache() function is my redis server because I only imported "clearKey " and I didn't use it ? And the performance is not speeded so I don't know if the Redis set up is correct and working or not. How to check that ?
Thanks
You added the cache function to mongoose.Query.prototype.cache, it means that you extended mongoose.Query with cache function and it was automatically added to all Mongoose models.
Regarding your second question, you need to add to your Redis:
const client = redis.createClient({
host: keys.redisHost, // make sure its your redis host
port: keys.redisPort, // make sure its your redis port
retry_strategy: () => 1000
});
When your query is running over MongoDB you should see a log:
console.log("Response from MongoDB");
And when your query is running over Redis you should see a different log:
console.log("Response from Redis");
You can see more the this github file.
Well I recently started using knex for my database connectivity for my nodejs project. Here's the following code:
var knex = require('knex')({
client: 'mysql',
connection: {
'host':'localhost',
'port':'3306',
'user':'root',
'password':'',
'database':'db_sample'
}
});
So what I need is, I want to add an access key as well in my Database configuration. I just wanted to know if its possible to do it. If yes, please do tell me how. Thank you
Is it possible to use Sequelize with Redshift? If not, what are the alternatives? I need an ORM for Node.js with built-in support for transactions, hence Sails.js is not an option. I've also looked at Bookshelf, but could not find any support for Redshift either.
I've been able to get Sequelize to at least connect to Redshift (and make a simple SELECT query) with these options:
var Sequelize = require('sequelize');
Sequelize.HSTORE.types.postgres.oids.push('dummy'); // avoid auto-detection and typarray/pg_type error
module.exports = new Sequelize(process.env.REDSHIFT_DATABASE, process.env.REDSHIFT_USER, process.env.REDSHIFT_PASSWORD, {
host: process.env.REDSHIFT_HOST,
port: process.env.REDSHIFT_PORT,
dialect: 'postgres',
pool: false,
keepDefaultTimezone: true, // avoid SET TIMEZONE
databaseVersion: '8.0.2' // avoid SHOW SERVER_VERSION
});
Sequelize is not compatible with Redshift. Though Redshift is written on top of Postgres, it is a columnar DB and major core functions are rewritten.
While trying to connect to it gives an error 'Set Time Zone is not supported'
The following thread shows a few people overriding the time zone error but facing other issues subsequently. 'Using Node 'pg' library to connect to Amazon Redshift
if Redshift is the mandatory you may use the node-jdbc package to connect with Redshift
https://github.com/CraZySacX/node-jdbc
of if ORM is mandatory, you should may try moving your data store to pure Postgres
Redshift is based on top of postgres 8.0.2 (http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres-sql.html), so both postgres and sequelize should be able to connect to it.
I don't have any personal experience with it, but the redshift documentation suggests that you can connect to it using regular JDBC / ODBC drivers, so I would be surprised if the node drivers don't work
At this point in time I dont think Sequelize is the best option to connect to Redshift. I recommend that you use node-postgres module directly.
That being said it might be the case that you might already have a project that uses Sequelize and want to reuse it in those cases you can do the following.
const Sequelize = require('sequelize');
// sequelize config options documented at
// https://sequelize.org/master/class/lib/sequelize.js~Sequelize.html#instance-constructor-constructor
const config = {
dialect: 'postgres',
host: 'localhost',
port: 5439,
database: '',
username: '',
password: '',
dialectOptions: {
// either set to ssl to true or use the config options documented at
// https://nodejs.org/api/tls.html#tls_new_tls_tlssocket_socket_options
ssl: true
},
standardConformingStrings: false,
clientMinMessages: false
};
const redshift = new Sequelize(config);
const queryOpts = {type: sequelize.QueryTypes.SELECT, raw:true};
redshift.query('SELECT 1', queryOpts)
.then(console.log)
.catch(console.log);
This will generate the following deprecation warning as Redshift is based on postgres 8.x so consider using pg module directly.
(node:5177) [SEQUELIZE0006] DeprecationWarning: This database engine version is not supported, please update your database server. More information https://github.com/sequelize/sequelize/blob/master/ENGINE.md
I'm using express with node-mysql to implement an MVC app. I want to ask the best way to get connection of mysql in each models file.
My models file is in directory models.
In app.js of express, I create a connection like :
var db = mysql.createConnection({
host: config.dbHost,
user: config.dbUser,
password: config.dbPassword,
database: dbName
});
I can use :
app.use(function(req,res,next){
req.db = db;
next();
});
then in modules,models get connection by call req.db.query().. but I think it's not good for performance.
What should I do ?
Edit:
if I have to access different database in each request, meaning create many connections. What 's the best way to do it ?
I have been looking around for simple database abstraction implementation, then i found great article http://howtonode.org/express-mongodb, which old but I still like the idea.
Well maybe the construction, could take some kind of object literal with database settings.
So the main idea is that there could be different implementations of UserService-s, but locate in different directories and require only the one that's needed.
/data-layer/mongodb/user-service.js
/post-service.js
/comment-service.js
/data-layer/couchdb/user-service.js
/post-service.js
/comment-service.js
When the Database is needed, I wil get it with var UserService = require(__dirname + '/data-layer/mongodb/user-service).UserService(db); where var db = "open db object"
Would this be the correct way to do it or is there any better solutions ?
There are a few solutions, available via NPM :
Node-DBI : "Node-DBI is a SQL database abstraction layer library, strongly inspired by the PHP Zend Framework Zend_Db API. It provides unified functions to work with multiple database engines, through Adapters classes. At this time, supported engines are mysql, mysql-libmysqlclient and sqlite3". Looks like the developpment has been paused.
Accessor : "A database wrapper, provide easy access to databases." Supports only MySQL and MongoDB at the moment.
Activerecord : "An ORM written in Coffeescript that supports multiple database systems (SQL, NoSQL, and even REST), as well as ID generation middleware. It is fully extendable to add new database systems and plugins."
Update:
Since I've posted this answer, I have abandoned mongoose for official MongoDB NodeJS Drivers as it is fairely intuitive and more loyal to the concept of NoSQL databases.
Original Answer:
I though it might be time to update the answer of an old question:
If you want to use MongoDB as your document-oriented database, mongoose is a good choice and easy to use (example from official site):
var mongoose = require('mongoose');
mongoose.connect('mongodb://localhost/test');
var Cat = mongoose.model('Cat', { name: String });
var kitty = new Cat({ name: 'Zildjian' });
kitty.save(function (err) {
if (err) // ...
console.log('meow');
});
For a rather modern approach, Mongorito is a good ODM which uses ES6 generators instead of callbacks.
As of 06.2015 I reckon that the best ORM for SQL databases with Node.js/io.js is Sequelize supporting the following databases:
PostgreSQL
MySQL
MariaDB
SQLite
MSSQL
The setup is fairly easy:
var sequelize = new Sequelize('database', 'username', 'password', {
host: 'localhost',
dialect: 'mysql'
});
// Or you can simply use a connection uri
var sequelize = new Sequelize('postgres://user:pass#example.com:5432/dbname');
It also provides transactions, migrations and many other goodies.