I am new to MongoDB. Now I am using Mongoose to take care of the database in my express.js app. My question is that I don't know when should I close the connection? Since someone says better to close it, while someone else says to leave it open? And what is the difference between db.close() and mongoose.disconnect()? What is your experience of using these?
Thanks!
Open a connection and share that between your routes/services. You can initialize the db and all related schema before you start express.
import mongoose from 'mongoose';
// define models
import User from './user';
import Order from './order';
// connect to db
const connect = async () => {
await mongoose.connect(process.env.DATABASE_URL);
return mongoose.connection;
};
const models = { User, Order };
export { connect };
export default models;
Then connect before you startup Express:
import models, { connect } from './models';
...
// connect to the db
const connection = await connect();
// now pass in the models to your routes
request('./routes/users')(models);
request('./routes/orders')(models);
// or use middleware so you can access in routes like req.models.user.find({})
app.use((req, res, next) => {
req.models = models;
next();
})
app.listen(process.env.PORT, () =>
console.log(`Example app listening on port ${process.env.PORT}!`),
);
You can add a cleanup handler to listen to process.exit and close the connection there. Roughly something like:
const cleanUp = (eventType) => {
connection.close(() => {
console.info('closed');
});
};
[`exit`, `SIGINT`, `SIGUSR1`, `SIGUSR2`, `uncaughtException`, `SIGTERM`].forEach((eventType) => {
process.on(eventType, cleanUp.bind(null, eventType));
})
Related
I don't know how to establish connection to the mongo db for my node JS server in AWS Lambda using serverless on AWS. I've mentioned my question in the handler function below.
The code looks something like this:
import express from "express";
import mongoose from "mongoose";
import dotenv from "dotenv";
import cookieParser from "cookie-parser";
import serverless from "serverless-http";
const PORT = 1234;
dotenv.config();
mongoose.connect(
process.env.MONGO_URL,
() => {
console.log("connected to db");
},
(err) => {
console.log({
error: `Error connecting to db : ${err}`,
});
}
);
const app = express();
app.use(cookieParser());
app.use(express.json());
// this part has various routes
app.use("/api/auth", authRoutes);
app.use((err, req, res, next) => {
const status = err.status || 500;
const message = err.message || "Something went wrong";
return res.status(status).json({
success: false,
status,
message,
});
});
app.listen(PORT, () => {
console.log(`Server listening on port ${PORT}`);
});
export const handler = () => {
// how to connect to mongodb here?
return serverless(app);
};
Here handler is the AWS lambda's handler function. For each http request I'm reading/writing data from/to my DB in some way. After checking the cloudwatch logs, it was clear that the requests sent to the server result in timeout because the connection to mongodb hasn't been established. So how exactly do I use mongoose.connect here?
I tried doing this:
export const handler = () => {
mongoose.connect(
process.env.MONGO_URL,
() => {
console.log("connected to db");
}
);
return serverless(app);
};
But it didn't work possibly because it's asynchronous. So I'm not sure how to make this connection here.
EDIT:
One thing that I realised was that the database server's network access list had only my IP because that's how I set it up initially.
So I changed it to anywhere for now and made the following minor changes:
const connect_to_db = () => {
mongoose
.connect(process.env.MONGO_URL)
.then(() => {
console.log("Connected to DB");
})
.catch((err) => {
throw err;
});
};
app.listen(PORT, () => {
connect_to_db();
console.log(`Server listening on port ${PORT}`);
});
Now I can see "Connected to DB" in the logs but the requests sent still times out after 15 seconds (the timeout limit set currently).
My logs:
What am I doing wrong?
So I did some more digging and asked this around the community. Few things that made me understand what I was doing wrong:
It appeared I wasn't connecting the db and my app response
together. My app was handling the request fine, my db was connecting
fine. But there was nothing tying them together. It's supposed to be simple:
Requests comes in > App waits until db connection has been established > App handles request > App returns response.
Second, calling app.listen was another problem in my code. Calling listen keeps the process open for incoming requests and it ultimately is killed by Lambda on timeout.
In a serverless environment, you don't start a process that listens for requests but, instead, the listening part is done by AWS API Gateway (which I have used to have my Lambda handle http requests) and it knows to send request information to Lambda handler for processing and returning a response. The handler function is designed to be run on each request and return a response.
So I tried adding await mongoose.connect(process.env.MONGO_URL); to all my methods before doing any operation on the database and it started sending responses as expected. This was getting repetitive so I created a simple middleware and this helped me avoid lot of repetitive code.
app.use(async (req, res, next) => {
try {
await mongoose.connect(process.env.MONGO_URL);
console.log("CONNECTED TO DB SUCCESSFULLY");
next();
} catch (err) {
next(err);
}
});
Another important, but small change. I was assigning lambda's handler incorrectly.
Instead of this:
export const handler = () => {
return serverless(app);
};
I did this:
export const handler = serverless(app);
That's it I suppose, these changes fixed my express server on Lambda. If anything I've said is wrong in any way just let me know.
I'm learning socket.io and typescript. I'm currently building an app where I want to emit event on databases changes to the client.For the backend, I'm using Express, typescript and nodeJS
I've looked on a lot of posts but I don't understand how I can access my io instance to be able to emit my event from my controller functions. Also most examples are on javascript which is more confusing.
Currently in my app.ts I have the following configuration. How can I move my addOnlineHandler, logOutHandler and sendNotificationHandler in my controller functions to be able to emit event on database change directly from createOrder function of the controller ?
I've tried importing all orderHandler function in the controller but it's not working. I'm unable to access the io instance to emit events.
app.ts
import express from "express";
import dotenv from "dotenv";
import userRoute from './routes/user'
import mongoose from "mongoose";
import { createServer } from "http";
import { Server,Socket } from "socket.io";
import {addOnlineShopHandler,logOutHandler, sendNotificationHandler} from "./socket/orderHandler";
dotenv.config();
const app = express();
const httpServer = createServer(app);
const io = new Server(httpServer, {
cors:{
origin:"*"
}
});
const onConnection = (socket:Socket) => {
addOnlineShopHandler(socket)
logOutHandler(socket)
sendNotificationHandler(socket)
}
io.on("connection",onConnection)
httpServer.listen(process.env.PORT, () => {
console.log('Backend server is running')
});
const connectToDatabase = async (dbConnectionString:string) => {
await mongoose.connect(dbConnectionString)
console.log('Connection to DB sucess')
}
connectToDatabase(`${process.env.MONGO_URI}`).catch(err => console.log(err))
app.use('/api/users',userRoute)
My controller file where I need to be able to emit event when order is created.
orderController.ts
import { RequestHandler } from "express";
export const createOrder:RequestHandler = async (req, res) => {
// I want to use socket.io in this function
// emit event when order created
}
My orderHandler file.
orderHandler.ts
import { Server,Socket } from "socket.io";
interface IOnlineShop {
shopId:string,
socketId:string
}
let onlineShop:IOnlineShop[] = []
export const addOnlineShopHandler = (socket:Socket) => {
console.log('someone is in')
const addNewShop = (shopId:string, socketId:string) => {
!onlineShop.some((currentShop) => currentShop.shopId === shopId) && onlineShop.push({shopId,socketId})
}
const newShopConnected = (shopId:string) => {
addNewShop(shopId,socket.id)
}
socket.on('newShopConnected',newShopConnected)
}
export const logOutHandler = (socket:Socket) => {
const removeShop = (socketId:string) => {
onlineShop = onlineShop.filter((currentShop)=>{currentShop.socketId !== socketId})
console.log('someone is out')
}
socket.on('disconnect',removeShop)
}
export const sendNotificationHandler = (socket:Socket) =>{
const getUser = (shopId:string) =>{
return onlineShop.find((currentShop) => currentShop.shopId === shopId)}
const sendNotification = (data:any) =>{
const receiver = getUser('randomshopId')
console.log(receiver)
}
socket.on("sendNotif",sendNotification)
}
I've looked theses posts and still can't get around the solution, maybe using typescript is what is confusing me. Any help or explanation of the workflow would be appreciated. how to use socket.io in controller file Use socket.io in controllers
Socket.IO + Express framework | Emit/Fire From Express Controller & Router
Thank you,
You can attach io to the app object:
app.set('io', io);
You can then access it from within controllers with req.app.get('io')
I searched a lot but nothing about how to set a firebase cloud function using Typeorm and express.
I'm trying to build an api server with Express + Apollo server + TypeORM, but to connect to the DB with TypeORM, I need to do something like
createConnection().then(async connection => {
// init server here
app = express()
...
})
But to deploy to firebase, I have to do
export const server = functions.https.onRequest(app)
How can I combine these?
Maybe I could do
export const server = functions.https.onRequest(async (req, res) => {
createConnection().then(async connection => {
// init server here
app = express()
...
})
})
But since it's Cloud Function, so every time a request come, a new instance is created, which mean a new connection is created. Is this OK? Wouldn't there be any speed issues?
UPDATE
import { connect } from './config';
const app = express()
export const server = functions.https.onRequest(async (req, res) => {
const connection = await connect();
return app(req, res)
})
Assuming that you have a ./config file where you establish a DB connection. Then, you can import it in your index.ts and use it as follows:
import * as functions from 'firebase-functions';
import { connect } from './config';
export const server = functions.https.onRequest(async (request, response) => {
// Establish connection to DB
const connection = await connect();
const usersRepository = connection.getRepository(Users);
// Count the users
const count = await usersRepository.count();
// Get all the users
const allUsers = await usersRepository.find();
// SQL Query to fetch the users who are older than 21
const query = await usersRepository.query('SELECT name FROM users WHERE AGE > 21');
response.send(allUsers);
});
If you still have doubts as to how to configure the config file, I believe that using this sample code along with the typeORM configuration files may be helpful.
I also recommend that you have a look at the Firebase documentation for examples about using Express.js in Cloud Functions for Firebase.
Update: To add Express + TypeORM to your Cloud Function for Firebase, you can do the following:
import * as functions from 'firebase-functions';
import {Request, Response} from "express";
import { connect } from './config';
import * as express from 'express';
// Initializing Express server and defined PORT
const PORT = 3000;
const app = express();
// Register '/' endpoint
app.get('/', (request, response, next) => {
response.send('Hello World from Cloud Firestore for Firebase');
});
// Register 'users' endpoint
app.get('/users', async function(request: Request, response: Response) {
// Establish connection to DB
const connection = await connect();
const usersRepository = connection.getRepository(Users);
// Get all the users
const allUsers = await usersRepository.find();
response.json(allUsers);
});
app.listen(PORT, () => {
console.log('Server is running on PORT', PORT);
});
// Exports the Express app as a single HTTP function
exports.app = functions.https.onRequest(app);
Please see the Firebase documentation and TypeORM repository for adding Express to the application.
I have deployed an application on Zeit Now using ExpressJS. The application makes a connection to MongoDB using Mongoose. However, the state of the connection, which I obtain using mongoose.connection.readyState is displayed as 2, which denotes 'connecting'.
I tried running the application locally and it works fine, where I am able to write to the database.
const connectionURL = "mongodb+srv://MONGODB_USERNAME:MONGOD_BPASSWORD#cluster1-23abc.mongodb.net/DATABASE_NAME?retryWrites=true"
expressApp.listen(3000, function() {
console.log("Listening to port 3000");
});
mongoose
.connect(
connectionURL, { useNewUrlParser: true }
)
.then(function() {
console.log("db connected!");
});
expressApp.get("/", function(req, res) {
res.write(`Connection State: ${mongoose.connection.readyState}\n`);
res.end();
});
I would expect mongoose.connection.readyState to be 1, which denotes 'connected'.
However, mongoose.connection.readyState is stuck at 2, which denotes 'connecting'.
Also, now logs does not show any errors.
You'll want to cache your MongoDB connection so you won't have to make a new connection on each lamda call.
You can make an lib folder and empty mongoose.js file (lib/mongoose.js) as I did and place this code inside:
`import mongoose from 'mongoose';
let cachedDb = null;
console.log('outside-cachedDB:', cachedDb);
async function connectToDatabase(uri) {
if (cachedDb) {
console.log('=> using cached database instance');
return cachedDb;
}
// If no connection is cached, create a new one
const db = await mongoose.connect(uri, { useNewUrlParser: true });
console.log('New MongoDB Connected');
// Cache the database connection and return the connection
cachedDb = db;
return db;
}
export default async () => {
await connectToDatabase(process.env.MONGODB_URI);
};`
Then call this custom mongoose function in any lamda that needs a mongoDB connection:
`import express from "express";
import mongoose from "../lib/mongoose";
const app = express();
// #route Get api/connect
// #desc Tests post route
// #access Public
app.get("*", async (req, res) => {
await mongoose();
// Code to query your DB or whatever here
});
export default app;`
You don't have to use express of course, but I personally haven't moved on to newer solutions. One of these days I'll learn micro.js.
I am using express and connect via the npm package "mongodb": "^3.0.10" to a mongodb.
My app.ts looks like this:
const app = express();
let server: http.Server;
(async () => {
app.use(bodyParser.json());
let db: Db;
try {
const host = config.get<string>("api.dbConfig.host");
console.log(host);
const dbName = config.get<string>("api.dbConfig.dbName");
const port = config.get<string>("api.dbConfig.port");
const connectionString = "mongodb://" + host + ":" + port;
const mongoClient = await MongoClient.connect(connectionString);
db = mongoClient.db(dbName);
await db.collection("users").createIndex({email: 1}, {unique: true});
}
catch (err) {
console.log(err);
console.log("can not connect to MongoDB");
}
const userRepo = new UserRepository(db);
// Routes
app.use("/v1/users", userRoutes(userRepo));
server = http.createServer(app);
server.listen(3000);
server.on("listening", () => {
console.log("listening");
});
})();
module.exports = app;
For testing i use jest and supertest. The tests run successfully, but they never end, because there are still connections to mongodb.
The tests look something like this:
describe("user routes", function () {
it("should return all users", async () => {
const response = await agent(app).get("/v1/users/");
expect(response.status).to.be.equal(200);
expect(response.body).to.be.an("array");
expect(response.body).to.have.lengthOf(2);
});
I understand, that the mongodb driver uses connection pooling and the way i pass the db- (or collection-) object to my user repository, makes it impossible to close the connections manually in a test scenario.
I guess a need a better way a pass the db connection to my user repository, but i can not think of a better, or more decoupled way at the moment.
Try await mongoClient.close() after your tests are done. See MongoDB docs. As far as I know, Jest supports before() and after() hooks, and I imagine before() and after() hooks support async/await like Mocha's do.