I have a small personal project of Flutter application with Node.js express backend and a Postgresql database.
For the moment my database is hosted on my pc locally but having an ubuntu server I would like my database on it.
So I created a Docker container with my Postgresql database in it.
However I'm a bit stuck now I don't know how to create a database instance on my remote server and make it communicate with my application...
Here is my ormconfig.ts file i suppose i have to change here...
import { join } from "path";
import {ConnectionOptions } from "typeorm";
import { PostEntity } from "./database/entity/post.entity";
import { UserEntity } from "./database/entity/user.entity";
const connectionOptions : ConnectionOptions = {
type: "postgres",
host: "localhost",
port: 5432
username:"postgres",
password:"pg",
database:"test",
entities: [UserEntity,PostEntity],
synchronize:true,
dropSchema:false,
migrationsRun:true,
logging:false,
logger:"debug",
migrations:[join(__dirname,"src/migration/**/*.ts")],
};
export = connectionOptions;
Thanks a lot !
Unsure of your network setup with your Ubuntu server, but realistically it should be something like:
import { join } from "path";
import {ConnectionOptions } from "typeorm";
import { PostEntity } from "./database/entity/post.entity";
import { UserEntity } from "./database/entity/user.entity";
const connectionOptions : ConnectionOptions = {
type: "postgres",
host: UBUNTU_SERVER_ADDRESS,
port: POSTGRES_DOCKER_PORT
username:"postgres",
password:"pg",
database:"test",
entities: [UserEntity,PostEntity],
synchronize:true,
dropSchema:false,
migrationsRun:true,
logging:false,
logger:"debug",
migrations:[join(__dirname,"src/migration/**/*.ts")],
};
export = connectionOptions;
You'll need to make sure that the Postgres Docker instance has opened ports to connect to. E.g.:
docker run -d -p 5432:5432 ...other-args postgres:latest
Make sure your Ubuntu server has correctly configured firewall and network settings to allow remote access on port 5432.
Related
I used CDK with the ApplicationLoadBalancedFargateService construct to deploy the ECS Fargate service with Load Balancing. It's the WebSocket API written on Node.js with Socket.io.
It works fine locally, but in the AWS it's constantly being killed by the health check, with an error:
service CoreStack-CoreServiceA69F11F4-9ciFOoAL5oj9 (port 5000) is unhealthy in target-group CoreS-CoreS-FMLY84WTNYVN due to (reason Health checks failed with these codes: [404]).
The code of the stack:
import { aws_ecs_patterns, Stack, StackProps } from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Repository } from 'aws-cdk-lib/aws-ecr';
import { ContainerImage } from 'aws-cdk-lib/aws-ecs';
import { HostedZone } from 'aws-cdk-lib/aws-route53';
import { ApplicationProtocol } from 'aws-cdk-lib/aws-elasticloadbalancingv2';
import ec2 = require('aws-cdk-lib/aws-ec2');
import ecs = require('aws-cdk-lib/aws-ecs');
const DOMAIN_NAME = 'newordergame.com';
const SUBDOMAIN = 'core';
const coreServiceDomain = SUBDOMAIN + '.' + DOMAIN_NAME;
export class CoreStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const vpc = new ec2.Vpc(this, 'CoreServiceVpc', { maxAzs: 2 });
const cluster = new ecs.Cluster(this, 'Cluster', { vpc });
const repository = Repository.fromRepositoryName(
this,
'CoreServiceRepository',
'core-service'
);
const image = ContainerImage.fromEcrRepository(repository, 'latest');
const zone = HostedZone.fromLookup(this, 'NewOrderGameZone', {
domainName: DOMAIN_NAME
});
// Instantiate Fargate Service with just cluster and image
new aws_ecs_patterns.ApplicationLoadBalancedFargateService(
this,
'CoreService',
{
cluster,
taskImageOptions: {
image: image,
containerName: 'core',
containerPort: 5000,
enableLogging: true
},
protocol: ApplicationProtocol.HTTPS,
domainName: coreServiceDomain,
domainZone: zone
}
);
}
}
The Dockerfile:
FROM node:16-alpine
RUN mkdir -p /usr/app
WORKDIR /usr/app
COPY package.json ./
RUN npm install
COPY webpack.config.js tsconfig.json ./
COPY src ./src
RUN npm run webpack:build
EXPOSE 5000
CMD [ "node", "build/main.js" ]
The application also listens to the port 5000.
It crashes every 3 mins.
Does anyone have a clue about how to fix it?
Thank you!
Your health check URL is returning a 404 status code. You need to configure the health check with a path that will return 200 status code.
ELB health check works only with HTTP or HTTPS, it does not work with WebSocket including Socket.io
So, the best workaround I found is to make the service listen to the HTTP port along with WebSocket and answer it with status 200.
With this approach, the health check will actually check the status of the service.
Am trying to connect redis sentinel instance from nodeJS using ioredis. Am not able to connect redis sentinel instance despite trying multiple available options. We have not configured sentinel password. But, able to connect same redis sentinel instance from .net core using StackExchange.Redis. Please find below nodeJS code,
import { AppModule } from './app.module';
import IORedis from 'ioredis';
async function bootstrap() {
const app = await NestFactory.create(AppModule);
const ioredis = new IORedis({
sentinels: [
{ host: 'sentinel-host-1' },
{ host: 'sentinel-host-2' },
{ host: 'sentinel-host-3' },
],
name: 'mastername',
password: 'password',
showFriendlyErrorStack: true,
});
try {
ioredis.set('foo', 'bar');
} catch (exception) {
console.log(exception);
}
await app.listen(3000);
}
bootstrap();
Error we got is,
[ioredis] Unhandled error event: Error: connect ETIMEDOUT
node_modules\ioredis\built\redis\index.js:317:37)
at Object.onceWrapper (node:events:475:28)
at Socket.emit (node:events:369:20)
at Socket._onTimeout (node:net:481:8)
at listOnTimeout (node:internal/timers:557:17)
at processTimers (node:internal/timers:500:7)
Connection String used from .net core is below,
Redis_Configuration = "host-1,host-2,host-3,serviceName=mastername,password=password,abortConnect=False,connectTimeout=1000,responseTimeout=1000";
Answering this for the benefit of others. Everything is fine, but this nodeJS package is resolving redis instances into private IPs which i cannot access from my local. So, had to put it over subnet group and make it work. However, FYI - .net core package does not resolve into private IPs, hence i was able to access instances from my local itself.
"The arguments passed to the constructor are different from the ones you use to connect to a single node"
Try to replace password with sentinelPassword.
I am developing an application that relies completely on Socket.io. As we all know NodeJS by default runs only on one core. Now I would like to scale it across multiple cores. I am finding it difficult to make socketio work with PM2 Cluster Mode. Any sample code would help.
I am using Artillery to test. And when the app runs on single core I get the response while It runs in cluster the response would be NaN
When Ran Without Cluster
PM2 docs say
Be sure your application is stateless meaning that no local data is
stored in the process, for example sessions/websocket connections,
session-memory and related. Use Redis, Mongo or other databases to
share states between processes.
Socket.io is not stateless.
Kubernetes implementation get around the statefull issues by routing based on source IP to a specific instance. This is still not 100% since some sources may present more than one IP address. I know this is not PM2, but gives you an idea of the complexity.
NESTjs SERVER
I use Socket server 2.4.1 so then i get the compatible redis adapter that is 5.4.0
I need to extend nest's adepter class "ioAdapter" that class only works for normal ws connections not our pm2 clusters
import { IoAdapter } from '#nestjs/platform-socket.io';
import * as redisIOAdapter from 'socket.io-redis';
import { config } from './config';
export class RedisIoAdapter extends IoAdapter {
createIOServer(port: number, options?: any): any {
const server = super.createIOServer(port, options);
const redisAdapter = redisIOAdapter({
host: config.server.redisUrl,
port: config.server.redisPort,
});
server.adapter(redisAdapter);
return server;
}
}
That is actually nestjs implementation
Now i need to tell nest im using that implementetion so i go to main.ts
import { NestFactory } from '#nestjs/core';
import { AppModule } from './app.module';
import { config } from './config';
import { RedisIoAdapter } from './socket-io.adapter';
import { EventEmitter } from 'events';
async function bootstrap() {
EventEmitter.defaultMaxListeners = 15;
const app = await NestFactory.create(AppModule);
app.enableCors();
app.useWebSocketAdapter(new RedisIoAdapter(app));
await app.listen(config.server.port);
}
bootstrap();
I have a lot of events for this one so i had to up my max event count
now for every gateway you got, you need to use a different connection strategy, so instead of using polling you need to go to websocket directly
...
#WebSocketGateway({ transports: ['websocket'] })
export class AppGateway implements OnGatewayConnection, OnGatewayDisconnect {
...
or if you are using namespaces
...
#WebSocketGateway({ transports: ['websocket'], namespace: 'user' })
export class UsersGateway {
...
last step is to install the redis database on your AWS instance and that is another thing; and also install pm2
nest build
pm2 i -g pm2
pm2 start dist/main.js -i 4
CLIENT
const config: SocketIoConfig = {
url: environment.server.admin_url, //http:localhost:3000
options: {
transports: ['websocket'],
},
};
You can now test your websocket server using FireCamp
Try using this lib:
https://github.com/luoyjx/socket.io-redis-stateless
It makes socket io stateless through redis.
You need to setup Redis with your Node server. Here is how I managed to get cluster mode to work with Socket.io
First install Redis. If you are using Ubuntu, follow this link: https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-redis-on-ubuntu-18-04
Then:
npm i socket.io-redis
Now place Redis in your Node server
const redisAdapter = require('socket.io-redis')
global.io = require('socket.io')(server, { transports: [ 'websocket' ]})
io.adapter(redisAdapter({ host: 'localhost', port: 6379 }))
That was all I had to do to get PM2 cluster mode to work with socket.io in my server.
I've got a node app built on Hapi using MongoDB and mongoose. Locally, I can use the app without issue. I can connect to the db, add data, find it, etc.
I've created an Ubuntu 14.04 x64 droplet on Digital Ocean.
I can ssh into my droplet and verify that my db is there with the correct name. I'm using dokku-alt to deploy and I have linked the db name to the app using dokku's mongodb:link appName mydb
I was having issues once I deployed the app where it would hang and eventually timeout. After a lot of debugging and commenting out code I found that any time I try to hit mongo like this the app will hang:
var User = request.server.plugins.db.User;
User
.findOne({ id: request.auth.credentials.profile.raw.id })
.exec(function(err, user){
// do something
});
Without this, the app loads fine, albeit without data. So my thought is that mongoose is never properly connecting.
I'm using grunt-shell-spawn to run a script which checks if mongo is already running, if not it starts it up. I'm not 100% certain that this is needed on the droplet, but I was having issues locally where mongo was already running... script:
/startMongoIfNotRunning.sh
if pgrep mongod; then
echo running;
else
mongod --quiet --dbpath db/;
fi
exit 0;
/Gruntfile.js
shell: {
make_dir: {
command: 'mkdir -p db'
},
mongodb: {
command: './startMongoIfNotRunning.sh',
options: {
stdin: false,
}
}
},
And here's how I'm defining the database location:
/index.js
server.register([
{ register: require('./app/db'), options: { url: process.env.MONGODB_URL || 'mongodb://localhost:27017/mydb' } },
....
/app/db/index.js
var mongoose = require('mongoose');
var _ = require('lodash-node');
var models = require('require-all')(__dirname + '/models');
exports.register = function(plugin, options, next) {
mongoose.connect(options.url, function() {
next();
});
var db = mongoose.connection;
plugin.expose('connection', db);
_.forIn(models, function(value, key) {
plugin.expose(key, value);
});
};
exports.register.attributes = {
name: 'db'
};
My app is looking for db files in db/. Could it be that dokku's mongodb:link appName mydb linked it to the wrong location? Perhaps process.env.MONGODB_URL is not being set correctly? I really don't know where to go from here.
It turns out the solution to my problem was adding an entry to the hosts file of my droplet to point to the mongo db url:
127.0.0.1 mongodb.myurl.com
For some reason, linking the db to my app with Dokku didn't add this bit. I would have thought that it was automatic. The app container's host file did get a mongodb entry when i linked the db to the app.
I get this error: a hook ('orm') failed to load when trying to lift a sails app using mongoDB. This my connections.js file:
module.exports.connections = {
mongodb: {
adapter : 'sails-mongo',
host : 'localhost',
port : 27017,
database : 'mydb1'
}
};
And this is my models.js file:
module.exports.models = {
connection: 'mongodb'
};
And this is my local.js file:
module.exports = {
connections: {
mongodb: {
host : 'localhost',
port : 27017,
database : 'mydb1'
}
}
}
Sails v 0.10.1
Any idea why this could be happening?
Thanks
All you need to do is start your mongodb.
Navigate to : path_to_your_mongodb_installation_folder\MongoDB\bin\mongod.exe
example : C:\Program Files\MongoDB 2.6 Standard\bin
Let us know if this helps!
Add
models: {
connect: 'your_mongod_db_adapter_name'
}
in both env/development and env/production fixed the problem for me
For those if it didn't work in heroku after deploying, try to change the node js, sails, sails-mongo versions and check.
It worked for me after changing the versions to the below:
1.nodejs - 14.15.1
2.sails - ^1.5.2
3.sails-mongo - ^2.0.0