I'm trying to create a Node (and Express) based app which is functionally similar to a WebAPI app using Mongo as the data store. I'm trying to use TypeScript (and also VS 2013).
I've gotten it to work, and now I'm trying to clean it up.
First, Express 4.1.1 is available, but no typings for it are.
Second, What's the proper way to access Mongo - both making a connection, and querying a collection - such that it's async the way Node would like it to be (so I'm not blocking, etc)? Should I be using q/Promises?
Third, What's the proper way (in this setup) to be accessing a method in another file? I've wrestled for awhile with import/export/require/module/class to get what I think seems ok, but what a pain to get there. Here's what I have:
app.ts with some imports, etc and a reference
/// <reference path='./scripts/typings/node/node.d.ts' />
import express = require('express');
import http = require('http');
import path = require('path');
import badgeApi = require('./routes/api/Badge');
var app = express();
...
new badgeApi.Badge();
app.get('/badge', badgeApi.Badge.ListAll);
...
And then, some generic DB code:
/// <reference path='../../scripts/typings/mongodb/mongodb.d.ts' />
import util = require('util');
import mongodb = require('mongodb');
export var ConnectDataStore: (dbname: string, dbuser: string, dbpass: string) => mongodb.Db;
ConnectDataStore = (dbname: string, dbuser: string = 'xxxx', dbpass: string = 'xxxx') => {
var baseMongoConnect: string = 'xxxx';
var mongoConnect = util.format(baseMongoConnect, dbuser, dbpass, dbname);
var DB: mongodb.Db;
mongodb.MongoClient.connect(mongoConnect, (err, db) => {
if (!err) {
DB = db;
return DB;
}
else {
throw err;
}
});
}
And then Badge.ts
import util = require('util');
import express = require('express');
import mongodb = require('mongodb');
import datastore = require('./DataStore');
export class Badge {
private static DB: mongodb.Db = null;
constructor() {
Badge.DB = datastore.ConnectDataStore('credential', 'xxxx', 'xxxx')
}
public static ListAll(req: express.Request, res: express.Response): void {
Badge.DB.collection('badge', (collErr, coll) => {
coll.find().toArray((arrayErr, badges) => {
if (badges.length > 0) {
res.jsonp(200, badges);
}
else {
res.send(200, 'No Badges');
}
});
});
}
}
Now I know this doesn't exactly work - ConnectDataStore is wrong (it doesn't return a Db as it's supposed to). I was starting to workaround that by having it return a promise when I noticed I was using express 3.5.2, not 4.1.1 and it all finally got to me - this stuff is not very solid.
Of course, I'm probably missing something, but I don't know what!
Any direction around this soup would be appreciated!
Until there's definition file for Express 4+, I'd suggest you use JavaScript for the core of the application (as it shouldn't need to change much) and use TypeScript for things like application logic (controllers, etc.). In a Node.JS, it's easy to mix and match.
I'd use a promise to handle the MongoDB connection or make the connection early enough that the application hasn't actually started serving web pages until the MongoDB connection has been successfully established.
The 3.0 typings for Express may work with the 4+ version in many places, as the signatures are often the same.
Related
After reading through the mLab -> Atlas migration plan a few times, I decided I'd try a different way. My coding background is mainly asm on mcs51 so I'm something of a n00b in the node.js/mongo/heroku world. I barely understood half of the migration process.
So I wrote a small test app following this blog entry and then used what I'd learned to modify my actual app to talk to Atlas directly. I exported the collections from the old db to JSON, then imported them into the Atlas version to recreate the database. Everything appears to be working correctly; I don't see any data going into the old db and it looks like the new Atlas db is getting all the action.
But I'm leery of deleting the mLab add-on from Heroku until I've verified that it's truly not needed any more, because I'm pretty sure that I won't be able to recreate it if it turns out I've missed something.
So my question is, how can I ensure I'm no longer using the mLab add-on? I don't really understand what it was doing for me in the first place so I'm not sure how to verify I'm not using it any more.
Here are the relevant code snippets I'm using to access the Atlas db...
function myEncode(str) { // https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent
return encodeURIComponent(str).replace(/[!'()*]/g, function(c) {
return '%' + c.charCodeAt(0).toString(16);
});
}
const ATLASURI = process.env.ATLASURI;
const ATLASDB = process.env.ATLASDB;
const ATLASUSER = process.env.ATLASUSER;
const ATLASPW = myEncode(process.env.ATLASPW); // wrapper needed to handle strong paswords...
const dbURL = "mongodb+srv://"+ATLASUSER+":"+ATLASPW+"#"+ATLASURI+"/"+ATLASDB+"?retryWrites=true&w=majority";
var GoogleStrategy = require('passport-google-oauth20').Strategy;
const {MongoClient} = require('mongodb');
const client = new MongoClient(dbURL, { useNewUrlParser: true, useUnifiedTopology: true });
var store = new MongoDBStore({uri: dbURL,collection: 'Sessions'});
var db = undefined;
client.connect(async function(err) {
if(err) {console.log("Error:\n"+String(err));}
db = await client.db(ATLASDB);
console.log("Connected to db!");
banner();
});
This is a little hard articulate so I hope my title isn't too terrible.
I have a frontend/backend React/Node.js(REST API) Web app that I want to add Redis support to for storing retrieving app global settings and per-user specific settings (like language preference, last login, etc... simple stuff) So I was considering adding a /settings branch to my backend REST API to push/pull this information from a redis instance.
This is where my Node.js inexperience comes through. I'm looking at using the ioredis client and it seems too easy. If I have a couple of helpers (more than one .js which will call upon redis) will constructing the client as a const in each be safe to do? Or is there another recommended approach to reusing a single instance of it be the way to go?
Here's a sample of what I'm thinking of doing. Imagine if I had 3 helper modules that require access to the redis client. Should I declare them as const in each? Or centralize them in a single helper module, and get the client from it? Is there a dis-advantage to doing either?
const config = require('config.json');
const redis_url = config.redis_url;
//redis setup
const Redis = require('ioredis');
const redis = new Redis(redis_url);
module.exports = {
test
}
async function test(id) {
redis.get(id, function (err, result) {
if (err) {
console.error(err);
throw(err);
} else {
return result;
}
});
Thank you.
If no redis conflicts...
If the different "helper" modules you are referring to have no conflicts when interacting with redis, such as overwriting / using the same redis keys, then I can't see any reason not to use the same redis instance (as outlined by garlicman) and export this to the different modules in which it is used.
Otherwise use separate redis databases...
If you do require separate redis database connections, redis ships with 16 databases so you can specify which to connect to when creating a new instance - see below:
const redis = new Redis({ // SET UP CONFIG FOR CONNECTION TO REDIS
port: 6379, // Redis port
host: 127.0.0.1, // Redis host
family: 4, // 4 (IPv4) or 6 (IPv6)
db: 10, // Redis database to connect to
});
Normally what I would do (in Java say) is implement any explicit class with singleton access the hold the connection and any connection error/reconnect handling.
All modules in Node.js are already singletons I believe, but what I will probably go with will be a client class to hold it and my own access related methods. Something like:
const config = require('config.json');
const Redis = require("ioredis");
class Redis {
constructor(){
client = new Redis(config.redis_url);
}
get(key) {
return this.client.get(key);
}
set(key, value, ttl) {
let rp;
if (ttl === 0) {
rp = this.client.set(key, value);
}
else {
rp = this.client.set(key, value)
.then(function(res) {
this.client.expire(key, ttl);
});
}
return rp;
}
}
module.exports = new Redis;
I'll probably include a data_init() method to check and preload an initial key/value structure on first connect.
I'm completely new to node and it's frameworks Koa and express. I've a mongoose model called Drawing and a router module for that.
Problem is with express routers I was able the get the data from database using Drawing.find method but with Koa, control is not even going into Drawing.find. And I'm not able to get the data at all. Please find the following related code and help me understand the things better.
This is my router module
import * as Router from "koa-router";
import Drawing from "../../models/drawing";
function getRoutesForDrawing(): Router {
console.log("Inside getRoutes for drawing");
let route = new Router();
route.get("/drawing", function(context,next) {
console.log("Inside /drawing");
Drawing.find(function(err,drawings) {
console.log("Not gettig executed");
context.body = "Welcome";
});
//context.body = "Welcome";
});
}
export default getRoutesForDrawing();
And the model is
import mongoose = require("mongoose");
export interface IDrawing extends mongoose.Document {
drawingId:Number,
drawingName:String,
updatedOn:Date,
updatedBy:Number
};
export const DrawingSchema = new mongoose.Schema({
drawingId:Number,
drawingName:String,
updatedOn:Date,
updatedBy:Number
});
const Drawing = mongoose.model<IDrawing>('Drawing', DrawingSchema);
export default Drawing;
As you can see in my router module, the control is actually coming for /drawing and it's printing in console "Inside /drawing" but then control isn't coming to Drawing.find. I'm getting difficulty in understanding this.
It's a little bit hard to figure out what's going on because it looks like you have problems all over the place. Let me point out the things that stand out:
getRoutesForDrawing is declared to return a router and doesn't return anything
Koa routes are not like express. In particular they are not callback based. They take either generator functions (Koa 1.x) or async functions (Koa 2.x). You seem to expect that it's wanting a callback function which won't work. Assuming koa 2.x, its router.get('/drawing', async(context) => {...});
Assuming koa 2.x, you need to await the result of the mongoose methods, e.g. context.body = await Drawing.find({})
I started a new project from scratch with ExpressJS.
Everything works fine but now I begin to have a dozen of 'app.get(....)' function and I need to give the project a structure.
What I have in mind is quite simple, it should have a folder named 'routes' containing a file such as 'module1.js', with all of the app.get related to that module. (like I've seen in many examples)
The issue is how to tell Express to route 'http://url/module1/' to that route file and how to pass it a param variable, containing for instance the mongodb connection.
what I tried is :
var params = {
db: myMongoConnection
};
var mod1 = require('routes/module1');
app.use('/module1', mod1);
but now I still miss the 'params'.
If I try to pass it as an argument to the require method i get an error saying it needs middleware.
Another issue is related to the fact that the myMongoConnection is valid in the connection callback, so I think i need to require and use the route.js inside the MongoClient connect callback.
Any idea?
thanks a lot
For custom modules, create a folder, call it modules
In its index.js, expose the modules that you need.
Something like,
var mods = [
'mod1',
'mod2',
];
function init() {
var expose = {};
var params = {
db: myMongoConnection
};
mods.forEach(mods, function (mod) {
expose[mod] = require('./' + mod)(params);
});
return expose;
}
// export init
module.exports = init;
In mod1.js, wrap the params
module.exports = function(params) {
// all your functions here will have access to params.
}
Then in, server/app.js, require this and set it in the app.
app.set('mods', require('path-to/modules'));
Now, you can access all your modules, using app.get('mods').moduleName.methodname
Coming from a non Node background, my first instinct is to define my service as such
MyService.js
module.exports = new function(dbConnection)
{
// service uses the db
}
Now, I want one open db connection per request, so I define in middleware:
res.locals.db = openDbConnection();
And in some consuming Express api code:
api.js
var MyService = require(./services/MyService')
...
router.get('/foo/:id?', function (req, res) {
var service = new MyService(res.locals.db);
});
Now, being that Node's preferred method of dependency injection is via the require(...) statement, it seems that I shouldn't be using the constructor of MyService for injection of the db.
So let's say I want to have
var db = require('db');
at the top of MyService and then use somehow like db.current....but how would I tie the db to the current res.locals object now that db is a module itself? What's a recommended way of handling this kind of thin in Node?
Updated Answer: 05/02/15
If you want to attach a DB connection to each request object, then use that connection in your service, the connection will have to be passed to myService some how. The example below shows one way of doing that. If we try to use db.current or something to that effect, we'll be storing state in our DB module. In my experience, that will lead to trouble.
Alternatively, I lay out the approach I've used (and still use) in this previous answer. What this means for this example is the following:
// api.js
var MyService = require(./services/MyService')
...
router.get('/foo/:id?', function (req, res) {
MyService.performTask(req.params.id);
});
// MyService.js
var db = require('db');
module.exports = {
performTask: function(id)
{
var connection = db.getOpenConnection();
// Do whatever you want with the connection.
}
}
With this approach, we've decoupled the DB module from the api/app/router modules and only the module that actually uses it will know it exists.
Previous Answer: 05/01/15
What you're talking about could be done using an express middleware. Here's what it might look like:
var db = require('db');
// Attach a DB connection to each request coming in
router.use(req, res, next){
req.locals.db = db.getOpenConnection();
next();
}
// Later on..
router.get('/foo/:id?', function (req, res) {
// We should now have something attached to res.locals.db!
var service = new MyService(res.locals.db);
});
I personally have never seen something like new MyService before in express applications. That doesn't mean it can't be done, but you might consider an approach like this
// router.js
var MyService = require('MyService');
router.get('/foo/:id?', function (req, res) {
MyService.foo(res.locals.db);
});
// MyService.js
module.exports.foo(connection){
// I have a connection!
}