node.js and Redis on Heroku for IODocs - node.js

I'm trying to get IODocs running on Heroku. It requires node.js and Redis. Admittedly, I'm new to all of these technologies. Nonetheless, I've managed to get it running locally. However, I receive the following error when deploying to Heroku.
2011-12-01T11:55:18+00:00 app[web.1]: Redis To Go - port: 9030 hostname: dogfish.redistogo.com
2011-12-01T11:55:18+00:00 app[web.1]: Express server listening on port 9694
2011-12-01T11:55:19+00:00 heroku[web.1]: State changed from starting to up
2011-12-01T11:55:21+00:00 app[web.1]: ^
2011-12-01T11:55:21+00:00 app[web.1]: Error: Redis connection to localhost:6379 failed - ECONNREFUSED, Connection refused
2011-12-01T11:55:21+00:00 app[web.1]: at Socket.<anonymous> (/app/node_modules/redis/index.js:123:28)
2011-12-01T11:55:21+00:00 app[web.1]: at Socket.emit (events.js:64:17)
2011-12-01T11:55:21+00:00 app[web.1]: at Array.<anonymous> (net.js:828:27)
2011-12-01T11:55:21+00:00 app[web.1]: at EventEmitter._tickCallback (node.js:126:26)
2011-12-01T11:55:23+00:00 heroku[web.1]: State changed from up to crashed
The only time I received a similar warning on my local mating was when Redis was not running. From what I can tell the Redis add-on is enabled for my app and running:
$ heroku config --long
NODE_ENV => production
PATH => bin:node_modules/.bin:/usr/local/bin:/usr/bin:/bin
REDISTOGO_URL => redis://redistogo:52847221366cb677460c306e4f482c5b#dogfish.redistogo.com:9030/
I've also tried some configuration suggestions. Neither seem to work.
// redis connection in app.js
var db;
if (process.env.REDISTOGO_URL) {
var rtg = require("url").parse(process.env.REDISTOGO_URL);
// tried this line as well... gave a different error on .connect();
// db = require('redis-url').connect(process.env.REDISTOGO_URL);
db = redis.createClient(rtg.port, rtg.hostname);
db.auth(rtg.auth.split(":")[1]);
// debug
sys.puts('Redis To Go - port: ' + rtg.port + ' hostname: ' + rtg.hostname);
} else {
db = redis.createClient(config.redis.port, config.redis.host);
db.auth(config.redis.password);
}
From the difference in my Redis To Go debug line and Error, I'm sure this is a configuration issue. But don't know how to fix it. Any help is greatly appreciated.

According to this line:
2011-12-01T11:55:21+00:00 app[web.1]: Error: Redis connection to localhost:6379 failed - ECONNREFUSED, Connection refused
You are trying to connect to localhost:6379, but the redis server is running at redis://redistogo:52847221366cb677460c306e4f482c5b#dogfish.redistogo.com:9030/. Can you try connecting to that URL manually and see if that works?

This indeed had to do with the configuration for Redis on Heroku. There were additional lines that required updates in I/O Docs app.js.
In the end, I piggy-backed the global config object at the top (~ line 60) after sniffing out the production (Heroku) environment.
if (process.env.REDISTOGOURL) {
// use production (Heroku) redis configuration
// overwrite config to keep it simple
var rtg = require(‘url’).parse(process.env.REDISTOGOURL);
config.redis.port = rtg.port;
config.redis.host = rtg.hostname;
config.redis.password = rtg.auth.split(“:”)[1];
}
I created a blog post for installing, configuring, and deploying I/O Docs that includes this as well as other changes that were required to run I/O Docs. I recommend you review it if you're interested in this project.
Thanks to Jan Jongboom and Kirsten Jones for helping me get started. In addition, I believe the project has been updated on GitHub to include Heroku configuration out of the box. However, I've yet to test it.

I actually have a blog post about how to get IODocs working on Heroku. It's got the config changes needed to get the REDIS working on Heroku with IODocs.
http://www.princesspolymath.com/princess_polymath/?p=489
Here's the code changes needed:
Add the following block under “var db;” to app.'s:
if (process.env.REDISTOGO_URL) {
var rtg = require("url").parse(process.env.REDISTOGO_URL);
db = require("redis").createClient(rtg.port, rtg.hostname);
db.auth(rtg.auth.split(":")[1]);
} else {
db = redis.createClient(config.redis.port, config.redis.host);
db.auth(config.redis.password);
}
And then this in the Load API Configs section, after reading the config file:
var app = module.exports = express.createServer();
var hostname, port, password
if (process.env.REDISTOGO_URL) {
var rtg = require("url").parse(process.env.REDISTOGO_URL);
hostname = rtg.hostname;
port = rtg.port;
password = rtg.auth.split(":")[1];
} else {
hostname = config.redis.host;
port = config.redis.port;
password = config.redis.password;
}

Recently, a cleaner way would be to use the redis-url module which handles the configuration.
I'm personnaly using Express with Redis (via the Redis To Go addon) as a sessionStore, and it works well on Heroku.
Exemple :
const express = require('express')
, redis = process.env.REDISTOGO_URL
? require('redis-url').connect(process.env.REDISTOGO_URL)
: require('redis').createClient()
, RedisStore = require('connect-redis')(express)
, sessionStore = new RedisStore({ client: redis })
, app = express.createServer();
[...]
app.configure(function() {
this
.use(express.session({
secret: 'mySecretHash',
store: sessionStore // Set redis as the sessionStore for Express
}));
});

Related

Heroku Node.js RedisCloud Redis::CannotConnectError on localhost instead of REDISCLOUD_URL

When i try to connect my Nodsjs application to RedisCloud on Heroku I am getting the following error
Redis::CannotConnectError: Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED)
I have even tried to directly set the redis URL and port in the code to test it out as well. But still, it tried to connect to the localhost on Heroku instead of the RedisCloud URL.
const {Queue} = require('bullmq');
const Redis = require('ioredis');
const conn = new Redis(
'redis://rediscloud:mueSEJFadzE9eVcjFei44444RIkNO#redis-15725.c9.us-east-1-4.ec2.cloud.redislabs.com:15725'
// Redis Server Connection Configuration
console.log('\n==================================================\n');
console.log(conn.options, process.env.REDISCLOUD_URL);
const defaultQueue = () => {
// Initialize queue instance, by passing the queue-name & redis connection
const queue = new Queue('default', {conn});
return queue;
};
module.exports = defaultQueue;
Complete Dump of the Logs https://pastebin.com/N9awJYL9
set REDISCLOUD_URL on .env file as follows
REDISCLOUD_URL =redis://rediscloud:password#hostname:port
import * as Redis from 'ioredis';
export const redis = new Redis(process.env.REDISCLOUD_URL);
I just had a hard time trying to find out how to connect the solution below worked for me.
Edit----
Although I had been passed the parameters to connect to the Redis cloud, it connected to the local Redis installed in my machine. Sorry for that!
I will leave my answer here, just in case anyone need to connect to local Redis.
let express = require('express');
var redis = require('ioredis');
pwd = 'your_pwd'
url = 'rediss://host'
port = '1234'
redisConfig = `${url}${pwd}${port}`
client = redis.createClient({ url: redisConfig })
client.on('connect', function() {
console.log('-->> CONNECTED');
});
client.on("error", function(error) {
console.error('ERRO DO REDIS', error);
});
Just wanted to post my case in case someone has the same problem like me.
In my situation I was trying to use Redis with Bull, so i need it the url/port,host data to make this happened.
Here is the info:
https://devcenter.heroku.com/articles/node-redis-workers
but basically you can start your worker like this:
let REDIS_URL = process.env.REDISCLOUD_URL || 'redis://127.0.0.1:6379';
//Once you got Redis info ready, create your task queue
const queue = new Queue('new-queue', REDIS_URL);
In the case you are using local, meaning 'redis://127.0.0.1:6379' remember to run redis-server:
https://redis.io/docs/getting-started/

Unable to retrieve data from PG DB (in Azure) using Sequelize

I am unable to retrieve data from a PG DB resource hosted in Azure. I am using Sequelize and Node.
I am able to connect to the DB hosted in Azure using the terminal and a GUI, I can create a new DB with a table and some prepopulated fields to do a proof of concept.
However, when I try to connect in my local and get the data, I get an empty array response ([ ]). If I hit the same endpoint in production, I get a 502 (after a while) with the following message displayed on the client:
Server Error.
There was an unexpected error in the request processing.
Some code below (it works with my local db configured the same way):
This is my DB config:
'use strict';
var Sequelize = require('sequelize');
var cfg = require('../config');
var sequelize = new Sequelize(cfg.db, cfg.username, cfg.password, {
define: {
timestamps: false
},
host: cfg.host,
dialect: 'postgres',
port: 5432
});
And this is my router code:
'use strict';
const express = require('express');
const router = express.Router();
var User = require('../../models/users-model');
router.get('/', (req, res) => {
User.findAll().then(user => {
res.json(user);
});
});
module.exports = router;
Both in local and prod I expect to get the JSON response with an array of User objects.
In my local, as explained, I get an empty array.
In production, as mentioned as well, it seems to timeout and finally I get a 502 err response.
Any help is much appreciated!
Update!: I managed to activate the app logs on Azure (it took me a bit to find it as I'm quite new to the platform!) and got this now when I hit the endpoint in prod:
2019-08-12T12:52:06.355595892Z Unhandled rejection SequelizeConnectionRefusedError: connect ECONNREFUSED 127.0.0.1:5432
2019-08-12T12:52:06.355632393Z at connection.connect.err (/usr/src/app/server/node_modules/sequelize/lib/dialects/postgres/connection-manager.js:170:24)
2019-08-12T12:52:06.355637793Z at Connection.connectingErrorHandler (/usr/src/app/server/node_modules/pg/lib/client.js:191:14)
2019-08-12T12:52:06.355641493Z at emitOne (events.js:116:13)
2019-08-12T12:52:06.355645293Z at Connection.emit (events.js:211:7)
2019-08-12T12:52:06.355648693Z at Socket.reportStreamError (/usr/src/app/server/node_modules/pg/lib/connection.js:72:10)
2019-08-12T12:52:06.355652093Z at emitOne (events.js:116:13)
2019-08-12T12:52:06.355655393Z at Socket.emit (events.js:211:7)
2019-08-12T12:52:06.355658393Z at emitErrorNT (internal/streams/destroy.js:64:8)
2019-08-12T12:52:06.355661493Z at _combinedTickCallback (internal/process/next_tick.js:138:11)
2019-08-12T12:52:06.355664693Z at process._tickCallback (internal/process/next_tick.js:180:9)
After hours and hours, I have hardcoded the data rather than getting that dynamically from my config files, probably I did not set up my Dockerfile properly and was not setting the ENV variable correctly.
Now I attacked the PROD DB from my local and it seems to work! Would really appreciate if someone can affirm my problem lies at the configuration level and the NODE_ENV env node var.
Dockerfile
# Node server serving Angular App
FROM node:8.11-alpine as node-server
WORKDIR /usr/src/app
COPY /server /usr/src/app/server
WORKDIR /usr/src/app/server
ENV NODE_ENV=prod
RUN npm install --production --silent
EXPOSE 80 443
CMD ["node", "index.js"]
Then in /config/index.js I have:
var env = process.env.NODE_ENV || 'global'
, cfg = require('./config.' + env);
module.exports = cfg;
So I understand that by setting the NODE_ENV to prod in Docker, when starting the Node app in Azure it should get the config.prod.js file rather than the config.global.js file, right?
You can see how I implement this on the db.js file on the question.

Error while testing NodeJS and MongoDB stack using Mocha and Chai

Right now, I'm running Mocha tests and am getting the following error:
Error: connect ECONNREFUSED 127.0.0.1:27017
at Object.exports._errnoException (util.js:873:11)
at exports._exceptionWithHostPort (util.js:896:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1077:14)
I assume it's because I am unable to connect to port 27017 because I did not include:
var express = require('express')
var app = express()
However, what is particularly confusing to me is how I connect by test to MongoDB so I can create fake records for testing and then destroy them. If anyone can show me (with an example please) how to do it, that would be awesome!
Thanks again.
The error is coming may be the mongo server is not running or from more than one server trying to listen on same port. Also for test environment only can create different folder or use different port. So that can delete the folder once test case is over
In server.js
if(process.env === 'test')
{
mongoport = 57017;
}
else
{
mongoport = 27017;
}
mongoUrl = "mongodb://localhost:"+mongoport+"/student"
// use the mongodb url
In test.js
//on start of test case
var fs = require('fs-extra');
fs.removeSync("test/db/");
fs.ensureDirSync("test/db/");
//ur test case definition

Why won't my app establish websocket connection on Heroku?

I am trying to deploy my nodejs app on heroku. I cannot get the websocket connection to establish. I have read everything I could find, tried every example and none of them seem to work for me.
I when I try to open my page "heroku open", nodejs is correctly giving me the html file. However, the websocket 'ws' connection never establishes.
My server.js has these configurations:
var pport = process.env.PORT || 5000;
server.listen(pport, function(err) {
if(!err) { console.log("Listening on port " + pport); }
});
Side Note
When I check "heroku logs", I find that the port my app is running on is a random 5 digit number. ( like 28896, 53365, etc.) It never actually runs on the second half || 5000.
But the thing is, in order for my game.html file to establish a websocket connection, it needs to know what port.
I have tried the following client configurations, none have worked:
1)
var host = location.origin.replace(/^http/, 'ws');
this.connection = new WebSocket(host);
2)
var host = location.origin.replace(/^http/, 'ws');
host = host + ":5000";
this.connection = new WebSocket(host);
3)
this.connection = new WebSocket('ws://infinite-earth-7708.herokuapp.com/');
I have also done what their website said, and attempted to use the following after deploying my app:
heroku config:add NODE_ENV=production
Please advise
Well I figured it out. Here is what you should know:
I did not change my server configurations from my original post.
My client configurations looked like this:
var host = location.origin.replace(/^http/, 'ws');
this.connection = new WebSocket(host);
But here is the kicker.
On the terminal I used the following command:
heroku labs:enable websockets
And voila, it works! I hope this helps someone.

Proxy server with Node.js on Heroku

I'm trying to build a proxy server on with Node.js on Heroku using http-proxy.
Everything works fine locally, but I'm having some troubles on Heroku.
var http = require('http');
var httpProxy = require('http-proxy');
settings = {
"localhost": process.env.LOCALHOST,
"devices": process.env.DEVICES_URI
}
var options = { router: { } }
options.router[settings.localhost + '/devices'] = settings.devices + '/devices';
var port = process.env.PORT || 8000;
var server = httpProxy.createServer(options).listen(port);
As you can see the in the example I set a routing object. What I say is this:
when a request matches '/devices' then route the request to the the device service.
(identified by the DEVICES_URI environmental var)
In development I set
LOCALHOST = 'localhost'
DEVICES_URI = 'http://localhost:3000'
This means that all requests going to localhost:8000/devices are proxied to
localhost:3000/devices which is what I want. All works perfectly.
The problem is in production. It gives me a timeout error repeated multiple times
for every request.
2012-08-23T20:18:20+00:00 heroku[router]: Error H12 (Request timeout) -> GET lelylan-api.herokuapp.com/devices dyno=web.1 queue= wait= service=30000ms status=503 bytes=0
In the production the environment vars are configured to the app names.
LOCALHOST = 'lelylan-api.herokuapp.com'
DEVICES_URI = 'lelylan-devices.herokuapp.com/'
I guess I'm wrong is some configurations, but after the whole day I'm still
not able to figure it out.
Update
I've continued with my tests and I've seen that the proxy is not able to reach the proxied service which totally stops me.
In development I set:
LOCALHOST = 'localhost'
DEVICES_URI = 'lelylan-devices.herokuapp.com/'
If I call http://lelylan-devices.herokuapp.com/devices everything works fine.
If I call localhost:8000/devices (which points to http://lelylan-devices.herokuapp.com/devices) Heroku tells me there is no such an app. I guess the problem is somehow in the routing system.
Here you can access at the source code.
Here the configuration vars for Heroku.
NODE_ENV => production
LOCALHOST => lelylan-api.herokuapp.com
DEVICES_URI => lelylan-devices.herokuapp.com
TYPES_URI => lelylan-types.herokuapp.com
LOCATIONS_URI => lelylan-locations.herokuapp.com
I finally made it work using a slightly modified version proxy-by-url. The final code looks something like this and works fine.
var httpProxy = require('http-proxy');
var port = process.env.PORT || 8000;
var routing = {
'/devices': { port: process.env.DEVICES_PORT || 80, host: process.env.DEVICES_URI }
}
var server = httpProxy.createServer(
require('./lib/uri-middleware')(routing)
).listen(port);
One note to remember. The plugin sets the header HOST to the destination application uri. If you do not do so, Heroku will not recognize the app and will not find it, as its internal routing system seems to be based on the HOST header.
I use http-proxy successfully on Heroku. The first thing I noticed in your log err is the url it is GETting:
GET lelylan-api.herokuapp.com/tdevices
There is a typo... instead of '/devices' it shows '/tdevices'. Before continuing can you confirm this is the actual log message?

Resources