Managed DigitalOcean Redis instance giving Redis AbortError - node.js

I setup managed redis and managed postgres on digital ocean. Digital ocean gave me a .crt file, I don't know what to do with this, so didn't do anything with it. Can this be the root of the problem below:
Or do I have to allow docker container to reach outside of the container on the rediss protocol?
I dockerized a node app and then put this container onto my droplet. I have my droplet and managed redis and postgres in same region (SFO2). It connects to redis using this url:
url: 'rediss://default:REMOVED_THIS_PASSWORD#my-new-app-sfo2-do-user-5053627-0.db.ondigitalocean.com:25061/0',
I then did ran my docker container with docker run.
It then gives me error:
node_redis: WARNING: You passed "rediss" as protocol instead of the "redis" protocol!
events.js:186
throw er; // Unhandled 'error' event
^
AbortError: Connection forcefully ended and command aborted. It might have been processed.
at RedisClient.flush_and_error (/opt/apps/mynewapp/node_modules/redis/index.js:362:23)
at RedisClient.end (/opt/apps/mynewapp/node_modules/redis/lib/extendedApi.js:52:14)
at RedisClient.onPreConnectionEnd (/opt/apps/mynewapp/node_modules/machinepack-redis/machines/get-connection.js:157:14)
at RedisClient.emit (events.js:209:13)
at RedisClient.connection_gone (/opt/apps/mynewapp/node_modules/redis/index.js:590:14)
at Socket.<anonymous> (/opt/apps/mynewapp/node_modules/redis/index.js:293:14)
at Object.onceWrapper (events.js:298:28)
at Socket.emit (events.js:214:15)
at endReadableNT (_stream_readable.js:1178:12)
at processTicksAndRejections (internal/process/task_queues.js:80:21)
Emitted 'error' event on RedisClient instance at:
at /opt/apps/mynewapp/node_modules/redis/index.js:310:22
at Object.callbackOrEmit [as callback_or_emit] (/opt/apps/mynewapp/node_modules/redis/lib/utils.js:89:9)
at Command.callback (/opt/apps/mynewapp/node_modules/redis/lib/individualCommands.js:199:15)
at RedisClient.flush_and_error (/opt/apps/mynewapp/node_modules/redis/index.js:374:29)
at RedisClient.end (/opt/apps/mynewapp/node_modules/redis/lib/extendedApi.js:52:14)
[... lines matching original stack trace ...]
at processTicksAndRejections (internal/process/task_queues.js:80:21) {
code: 'NR_CLOSED',
command: 'AUTH',
args: [ 'REMOVED_I_DONT_KNOW_IF_THIS_IS_SENSITIVE' ]

The redis protocol is different from rediss because the latter uses TLS connection. DigitalOcean Managed Redis requires the connections to be made over TLS, so you have to use rediss. However, I couldn't find any info about the TLS certificate provided by DigitalOcean to connect to the Managed Redis service.
Based on your error message, I presumed you're using this redis package. If that's the case, you can pass empty TLS object option in the connection string like so:
const Redis = require('redis')
const host = 'db-redis.db.ondigitalocean.com'
const port = '25061'
const username = 'user'
const password = 'secret'
const url = `${username}:${password}#${host}:${port}`
const client = Redis.createClient(url, {tls: {}})
Further reading/source:
SSL connections arrive for Redis on Compose
Connecting to IBM Cloud Databases for Redis from Node.js

I solved this. Below are snippets from config/env/production.js
Sockets
For sockets, to enable rediss you have to pass in all options through adapterOptions like this:
sockets: {
onlyAllowOrigins: ['https://my-website.com'],
// pass in as adapterOptions so it gets through to redis-adapter
// as i need it "rediss" but this url is not supported i get an error.
// so i need to pass in `tls` empty object. and i see he moves things into
// `adapterOptions` here here - https://github.com/balderdashy/sails-hook-sockets/blob/master/lib/configure.js#L128
adapterOptions: {
user: 'username',
pass: 'password',
host: 'host',
port: 9999,
db: 2, // pick a number
tls: {},
},
adapter: '#sailshq/socket.io-redis',
},
Session
For session, pass tls: {} empty object to config:
session: {
pass: 'password',
host: 'host',
port: 9999,
db: 1, // pick a number not used by sockets
tls: {},
cookie: {
secure: true,
maxAge: 24 * 60 * 60 * 1000, // 24 hours
},
},

Related

Connection timed out while connecting to AWS DocumentDB outside the VPC

I'm trying create a very simple node app that can use DocumentDB. I'm not using Cloud9 neither Lambda, I'm coding locally. I was following this link https://docs.aws.amazon.com/documentdb/latest/developerguide/connect-from-outside-a-vpc.html and this link https://docs.aws.amazon.com/documentdb/latest/developerguide/connect-ec2.html
I created a poorly secured EC2 instance with the following inbound rules
port range
protocol
source
security group
22
TCP
0.0.0.0/0
demoEC2
This demoEC2 security group has the following inbound rules
type
protocol
port range
source
SSH
TCP
22
0.0.0.0/0
Then I created a DocumentDB cluster with 1 instance available that belongs to a security group that has the following inbound rules
type
protocol
port range
source
custom tcp
TCP
27017
demoEC2
After that, I open my terminal and created a tunnel:
ssh -i "mykeypair.pem" -L 27017:<CLUSTER ENDPOINT>:27017 ec2-user#<EC2 PUBLIC IPV4 DNS> -N
And, to test if my tunnel is working, I connect using mongoshell:
> mongo "mongodb://<MASTER USERNAME>:<MASTER PASSWORD>#localhost:27017/<DATABASE>" --tls --tlsAllowInvalidHostnames --tlsCAFile rds-combined-ca-bundle.pem
MongoDB shell version v4.2.13
connecting to: mongodb://localhost:27017/<DATABASE>?compressors=disabled&gssapiServiceName=mongodb
2021-07-29T10:10:59.309+0200 W NETWORK [js] The server certificate does not match the host name. Hostname: localhost does not match docdb-2021-07-27-10-32-49.ctuxybn342pe.eu-central-1.docdb.amazonaws.com docdb-2021-07-27-10-32-49.cluster-ctuxybn342pe.eu-central-1.docdb.amazonaws.com docdb-2021-07-27-10-32-49.cluster-ro-ctuxybn342pe.eu-central-1.docdb.amazonaws.com , Subject Name: C=US,ST=Washington,L=Seattle,O=Amazon.com,OU=RDS,CN=docdb-2021-07-27-10-32-49.ctuxybn342pe.eu-central-1.docdb.amazonaws.com
Implicit session: session { "id" : UUID("63340995-54ad-471b-aa8d-85763f3c7281") }
MongoDB server version: 4.0.0
WARNING: shell and server versions do not match
Warning: Non-Genuine MongoDB Detected
This server or service appears to be an emulation of MongoDB rather than an official MongoDB product.
Some documented MongoDB features may work differently, be entirely missing or incomplete, or have unexpected performance characteristics.
To learn more please visit: https://dochub.mongodb.org/core/non-genuine-mongodb-server-warning.
rs0:PRIMARY>
However, when I try to connect in my node app:
const mongoose = require('mongoose');
const fs = require('fs');
const path = require('path');
const username = ...
const password = ...
const database = ...
const connstring = `mongodb://${username}:${password}#localhost:27017/${database}?tls=true&replicaSet=rs0&readPreference=secondaryPreferred`;
const certFile = path.resolve(__dirname, './rds-combined-ca-bundle.pem');
const certFileBuf = fs.readFileSync(certFile); //I tried this one in tlsCAFile option as well
mongoose.connect(connstring,
{
tlsCAFile: certFile,
useNewUrlParser: true,
tlsAllowInvalidHostnames: true,
}
).then(() => console.log('Connection to DB successful'))
.catch((err) => console.error(err, 'Error'));
I get a connection timeout error after a while:
> > node .\index.js
(node:12388) [MONGODB DRIVER] Warning: Current Server Discovery and Monitoring engine is deprecated, and will be removed in a future version. To use the new Server Discover and Monitoring engine, pass option { useUnifiedTopology: true } to the MongoClient constructor.
MongoNetworkError: failed to connect to server [<CLUSTER ENDPOINT WITHOUT HAVING .cluster->:27017] on first connect [MongoNetworkTimeoutError: connection timed out
at connectionFailureError (D:\projects\documentdb-connect\node_modules\mongoose\node_modules\mongodb\lib\core\connection\connect.js:345:14)
at TLSSocket.<anonymous> (D:\projects\documentdb-connect\node_modules\mongoose\node_modules\mongodb\lib\core\connection\connect.js:313:16)
at Object.onceWrapper (events.js:421:28)
at TLSSocket.emit (events.js:315:20)
at TLSSocket.Socket._onTimeout (net.js:481:8)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7)]
at Pool.<anonymous> (D:\projects\documentdb-connect\node_modules\mongoose\node_modules\mongodb\lib\core\topologies\server.js:441:11)
at Pool.emit (events.js:315:20)
at D:\projects\documentdb-connect\node_modules\mongoose\node_modules\mongodb\lib\core\connection\pool.js:564:14
at D:\projects\documentdb-connect\node_modules\mongoose\node_modules\mongodb\lib\core\connection\pool.js:1013:9
at D:\projects\documentdb-connect\node_modules\mongoose\node_modules\mongodb\lib\core\connection\connect.js:32:7
at callback (D:\projects\documentdb-connect\node_modules\mongoose\node_modules\mongodb\lib\core\connection\connect.js:283:5)
at TLSSocket.<anonymous> (D:\projects\documentdb-connect\node_modules\mongoose\node_modules\mongodb\lib\core\connection\connect.js:313:7)
at Object.onceWrapper (events.js:421:28)
at TLSSocket.emit (events.js:315:20)
at TLSSocket.Socket._onTimeout (net.js:481:8)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7) Error
Since I could connect using mongoshell, I think the tunnel is working and I can even do some inserts on it, but why Mongoose can't connect? I tried also using the MongoClient (const MongoClient = require('mongodb').MongoClient and MongoClient.connect(same everything)) but it didn't worked, I'm still getting the same timeout error.
Turns out all I need to do is to pass the username and password through the options, not in the connection string:
const connstring = `mongodb://localhost:27017/${database}`;
const certFile = path.resolve(__dirname, './rds-combined-ca-bundle.pem');
const certFileBuf = fs.readFileSync(certFile);
mongoose.connect(connstring,
{
tls: true,
tlsCAFile: certFile,
useNewUrlParser: true,
tlsAllowInvalidHostnames: true,
auth: {
username,
password
}
}
)

Redis sentinel connection is timing out from nodeJS

Am trying to connect redis sentinel instance from nodeJS using ioredis. Am not able to connect redis sentinel instance despite trying multiple available options. We have not configured sentinel password. But, able to connect same redis sentinel instance from .net core using StackExchange.Redis. Please find below nodeJS code,
import { AppModule } from './app.module';
import IORedis from 'ioredis';
async function bootstrap() {
const app = await NestFactory.create(AppModule);
const ioredis = new IORedis({
sentinels: [
{ host: 'sentinel-host-1' },
{ host: 'sentinel-host-2' },
{ host: 'sentinel-host-3' },
],
name: 'mastername',
password: 'password',
showFriendlyErrorStack: true,
});
try {
ioredis.set('foo', 'bar');
} catch (exception) {
console.log(exception);
}
await app.listen(3000);
}
bootstrap();
Error we got is,
[ioredis] Unhandled error event: Error: connect ETIMEDOUT
node_modules\ioredis\built\redis\index.js:317:37)
at Object.onceWrapper (node:events:475:28)
at Socket.emit (node:events:369:20)
at Socket._onTimeout (node:net:481:8)
at listOnTimeout (node:internal/timers:557:17)
at processTimers (node:internal/timers:500:7)
Connection String used from .net core is below,
Redis_Configuration = "host-1,host-2,host-3,serviceName=mastername,password=password,abortConnect=False,connectTimeout=1000,responseTimeout=1000";
Answering this for the benefit of others. Everything is fine, but this nodeJS package is resolving redis instances into private IPs which i cannot access from my local. So, had to put it over subnet group and make it work. However, FYI - .net core package does not resolve into private IPs, hence i was able to access instances from my local itself.
"The arguments passed to the constructor are different from the ones you use to connect to a single node"
Try to replace password with sentinelPassword.

connect ECONNREFUSED 127.0.0.1:587 (nodemailer + mailtrap)

I've been trying to get the basic combination of nodemailer, using a mailtrap.io account to work, and striking out.
Here is my app.js:
const nodemailer = require('nodemailer');
let transport = nodemailer.createTransport({
host: "smtp.mailtrap.io",
port: 2525,
// secure: true,
auth: {
user: "myusername",
pass: "mypassword"
},
debug: true,
logger: true
});
let scrapeEmailMessage = {
//from: 'myemail#gmail.com',
to: 'myemail#gmail.com',
subject: 'Hello World',
text: 'hello world'
};
let mailTransporter = nodemailer.createTransport(transport);
mailTransporter.sendMail(scrapeEmailMessage, function(err, data) {
if(err) {
console.log(err);
} else {
console.log('Email sent successfully');
}
});
And here is the error output I'm getting:
[2020-11-10 14:32:20] DEBUG Creating transport: nodemailer (6.4.15; +https://nodemailer.com/; SMTP/6.4.15[client:6.4.15])
[2020-11-10 14:32:20] DEBUG Creating transport: nodemailer (6.4.15; +https://nodemailer.com/; SMTP/6.4.15[client:6.4.15])
[2020-11-10 14:32:20] DEBUG Sending mail using SMTP/6.4.15[client:6.4.15]
[2020-11-10 14:32:20] DEBUG [YlvPyvxQxE] Resolved localhost as 127.0.0.1 [cache miss]
[2020-11-10 14:32:22] ERROR [YlvPyvxQxE] connect ECONNREFUSED 127.0.0.1:587
[2020-11-10 14:32:22] DEBUG [YlvPyvxQxE] Closing connection to the server using "destroy"
[2020-11-10 14:32:22] ERROR Send Error: connect ECONNREFUSED 127.0.0.1:587
Error: connect ECONNREFUSED 127.0.0.1:587
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1129:14) {
errno: 'ECONNREFUSED',
code: 'ESOCKET',
syscall: 'connect',
address: '127.0.0.1',
port: 587,
command: 'CONN'
}
I've tried a number of things to fix it, and continually get this same error message:
Turning on the "less secure apps" options in Gmail and using the above code to try to connect to smtp.gmail.com instead.
Manually editing the port numbers in the code above.
Toggling secure mode on and off in the code above.
Adding Windows Defender Firewall exceptions on ports 587, 465, as well as a couple others just for fun to see if they would work.
Disabling Windows Defender Firewall completely.
Running the code on other computers on my home network.
Checking "netstat -an" to see if ports 587 or 465 are listed. They are not, but is this port open all the time or only opened when needed? Could this be the issue?
Anyone have any ideas on what might be going on here? I'm just a normal guy using his home internet connection to write a small program to send himself an email alert once a day. Could my ISP be blocking this or something?
EDIT- Adding a couple new things I've tried that haven't worked:
Transport option "ignoreTLS: true/false" (tried both)
Transport option "requireTLS: true/false" (tried both)
You are creating two transporter.
Instead of this :
let mailTransporter = nodemailer.createTransport(transport);
and
mailTransporter.sendMail()
Do this :
transport.sendMail()
modify;
let transport = nodemailer.createTransport({
to be;
let transport = {
and remove the closing parentheses; }); to be };
I got this error, and it's because I was using port 25 instead of port 2525. It says 25 is an option but seems to not work for me.

Random SSL handshake error when connecting to ElastiCache with ioRedis

I am attempting to connect to an ElastiCache cluster that is encrypted in transit from a node script using ioRedis. Sometimes my script works, other times I get Error: 140736319218624:error:140940E5:SSL routines:ssl3_read_bytes:ssl handshake failure:../deps/openssl/openssl/ssl/s3_pkt.c:1216:
Here is all of my code:
var Redis = require('ioredis');
var nodes = [{
host: 'clustercfg.name.xxxxxx.region.cache.amazonaws.com',
port: '6379',
}];
var cluster = new Redis.Cluster(nodes,{
redisOptions: {
tls: {}
}});
cluster.set('aws', 'test');
cluster.get('aws', function (err, res) {
console.log(res);
if (err) {
console.error(err)
}
cluster.disconnect()
});
I believe the ssl handshake error is a side-effect of a race-condition bug in ioredis.
I have been banging my head over the same issue the last several days (ioredis version 4.0.0). I just couldn't reliably connect ioredis to our elasticache cluster. I would see the same intermittent error.
Error: 140618195700616:error:140940E5:SSL routines:ssl3_read_bytes:ssl
handshake failure:../deps/openssl/openssl/ssl/s3_pkt.c:1216:
You can view ioredis debug output by setting "DEBUG=ioredis:*" in your node environment. Once I did this I could see that when the error occurred it was accompanied by several messages similar to the following:
2018-10-06T18:24:38.287Z ioredis:cluster:connectionPool Disconnect
xxx.usw2.cache.amazonaws.com:6379 because the node does not hold any
slot
I tried node-redis and redis-clustr it works fine with elasticache.

MongoDB Auth Fails to find username on Bitnami MEAN Stack Image

Trying to run a web app (MEAN) on Amazon EC2 Instance but am encountering the following problem. Can anyone help me with this?
node app.js The Server has started on 9091
/opt/bitnami/apps/YelpCamp/node_modules/mongodb-core/lib/auth/scram.js:128
username = username.replace('=', "=3D").replace(',', '=2C');
^
TypeError: Cannot read property 'replace' of undefined
at executeScram (/opt/bitnami/apps/SomeApp/node_modules/mongodb-core/lib/auth/scram.js:128:24)
at /opt/bitnami/apps/SomeApp/node_modules/mongodb-core/lib/auth/scram.js:277:7
at _combinedTickCallback (internal/process/next_tick.js:73:7)
at process._tickCallback (internal/process/next_tick.js:104:9)
Mongoose can do auth in 2 ways:
1, Connection string:
mongoose.connect('mongodb://username:password#host:port(usually 27017)/db')
Where username and password are the respective username and password for that specific db, host would be the host where your db is hosted (so localhost or some domain/IP), port is the port mongo listens on, and db is the name of the db you want to connect to
2, Using options. From the docs:
var options = {
useMongoClinet: true,
auth: {authdb: 'admin'},
user: 'myUsername',
pass: 'myPassword',
}
mongoose.connect(uri, options);
I also faced the 'username undefined' error in the first approach, but I succeeded in the second approach.
[Reference] https://github.com/Automattic/mongoose/issues/4891

Resources