I'm not able to connect the cloudrun service to cloudsql.
I am using Sequelize and here is my connection section!
let sequelize = new (<any> Sequelize) (
DATABASE_DATABASE,
DATABASE_USERNAME,
DATABASE_PASSWORD,
{
dialect: 'postgres'
dialectOptions: {
socketPath: `/cloudsql/${DATABASE_HOST}`,
supportBigNumbers: true,
bigNumberStrings: true
},
host: `/cloudsql/${DATABASE_HOST}`,
port: DATABASE_PORT,
logging: false,
},
);
PS: Apparently everything is configured correctly, that is,
The cloudsql service is connected to the specific cloudrun service,
they are in the same region, the AMIs are released ...
My unsuccessful attempts were:
The code snippet above returns: Error: connect ENOENT / cloudsql / <instancename ...>,
If I change the host to 127.0.0.1 = connection refused 127.0.0.1:5432,
Putting the native property in the connection: true = Error: Connection not found
direct connection attempt by PG = Error: connect ENOENT / cloudsql / <instancename ...>
I created another project, other permissions and the error continues
I changed the zone and the error continues
Another attempt was:
I tried to create VPC without a server for private IP connection enabled in cloudsql, but I get timeout in cloudrun
What I had left was to enable the cloudsql public network to access 0.0.0.0/0 and the application is working fine.
I'm out of ideas and need help connecting using /cloudsql/
Cloud Run connects to Cloud SQL over Unix sockets.
You'll want to use the instance connection name in the socket path instead of the host IP address. This will follow the format project-name:region-name:instance-name. You might also need to append the suffix s.PGSQL.5432. The full socket path should look like "/cloudsql/project-name:region-name:instance-name.s.PGSQL.5432" If you're using the socket path, you can remove the host and port connection arguments
Related
I am having trouble connecting to aws via python. I have a macOSX operating system.
What i did:
I created an ec2 instance and chose an operating system (ubuntu) and downloaded postgresssql in the remote server. Then i created securitygroups where i added the following configuration:
type:
ssh
protocol:
tcp
port: 22
source: custom, 0.0.0.0/0
Then i added another rule:
postgresql
TCP
5432
custom
my_computer_ip_address 71.???.??.???/32
where i added question marks just to hide the address. but its in that format.
Now, aws had me create a .pem file in order to query from the database. I downloaded this pem file into a secret location.
When i go to my local machine, go to my terminal and type:
ssh -i "timescale.pem" ubuntu#ec2-??-???-??-???.compute-1.amazonaws.com"
i am able to connect. I also went to my dbeaver and created a new connection and set up a connection where i am using an ssh tunnel and a public key to read the 'timescale.pem' file i created. I then go to main and type my username and password:
username: postgres
database: example
password: mycustompassword
and i am able to connect with no issues.
Now, when I go to python with psycopg2 library, i am just unable to connect at all. I have gone through all the examples here in stackoverflow and none of them have helped me. Heres what i am using to connect to aws from python:
path_to_secret_key = os.path.expanduser("~/timescale.pem")
conn = psycopg2.connect(
dbname='example',
user='postgres',
password='pass123',
host='ec2-??-??-??-???.compute-1.amazonaws.com',
port='22',
sslmode='verify-full',
sslrootcert=path_to_secret_key)
I then get this error:
connection to server at "ec2-34-???-??-???.compute-1.amazonaws.com" (34.201.??.???), port 22 failed: could not read root certificate file "/Users/me/timescale.pem": no certificate or crl found
Ok...then i switched ports and added '5432' and get this warning:
connection to server at "ec2-??-???-??-???.compute-1.amazonaws.com" (34.???.??.212), port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
When i ssh into my terminal and type: netstat -nl |grep 5432 i get the following:
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN
unix 2 [ ACC ] STREAM LISTENING 812450 /var/run/postgresql/.s.PGSQL.5432
Can someone please help? Thanks
The .pem file is for connecting to the EC2 instance over SSH, on port 22. PostgreSQL is running on port 5432. You don't use the .pem file for database connections, only for ssh connections. You need to change your Python script to use port='5432', to connect directly to the PostgreSQL service running on the EC2 instance.
This seems to work now:
from sshtunnel import SSHTunnelForwarder
mypkey = paramiko.Ed25519Key.from_private_key_file(path_to_secret_key)
tunnel = SSHTunnelForwarder(
('remote_public_port', 22),
ssh_username='ubuntu',
ssh_pkey=mypkey,
remote_bind_address=('localhost', 5432))
tunnel.start()
conn = psycopg2.connect(
dbname='example_db',
user='postgres',
password='secret123',
host='127.0.0.1',
port=tunnel.local_bind_port)
getting the following error while connecting to AWS DocumentDB from node.js
connection error: { [MongoNetworkError: connection 1 to
docdb-2019-01-28-06-57-37.cluster-cqy6h2ypc0dj.us-east-1.docdb.amazonaws.com:27017
timed out] name: 'MongoNetworkError', errorLabels: [
'TransientTransactionError' ] }
here is my node js file
app.js
var mongoose = require('mongoose');
mongoose.connect('mongodb://abhishek:abhishek#docdb-2019-01-28-06-57-37.cluster-cqy6h2ypc0dj.us-east-1.docdb.amazonaws.com:27017/?ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0', {
useNewUrlParser: true
});
var db = mongoose.connection;
db.on('error', console.error.bind(console, 'connection error:'));
db.once('open', function() {
console.log("connected...");
});
By default aws documentdb is designed to connect only from same VPC.
So to connect nodejs application from an ec2 in same vpc. You need to have the pem file as by default SSL is enabled while db instance is created.
step-1 : $ wget https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem in required directory
step-2 : Change the mongoose connection with options pointing to pem file
mongoose.connect(database.url, {
useNewUrlParser: true,
ssl: true,
sslValidate: false,
sslCA: fs.readFileSync('./rds-combined-ca-bundle.pem')})
.then(() => console.log('Connection to DB successful'))
.catch((err) => console.error(err,'Error'));
Here am using mongoose 5.4.0
To connnect from outside the VPC, please try to follow the below doc from aws:
https://docs.aws.amazon.com/documentdb/latest/developerguide/connect-from-outside-a-vpc.html
Personally I tried only to connect from VPC and it worked fine.
Update =====:>
To connect from Robo 3T outside VPC please follow the link -
AWS DocumentDB with Robo 3T (Robomongo)
to use AWS DocumentDB outside VPC for example your development server EC2 or from the local machine will get a connection error unless you use ssh tunneling or port forwarding
and about tunneling it simple
use this command in your local
ssh -i "ec2Access.pem" -L 27017:sample-cluster.node.us-east-1.docdb.amazonaws.com:27017 ubuntu#EC2-Host -N
in application configuration use
{
uri: 'mongodb://:#127.0.0.1:27017/Db',
useNewUrlParser: true,
useUnifiedTopology:true,
directConnection: true
}
just make sure you can connect from this tunneling ec2 and database
and if you decide to use port forwarding
steps
0- in ec2 security grou[p add inbound role with custom TCP and port 27017 All traffic
1- go to your ec2 instance and install Haproxy
$ sudo apt install haproxy
2- edit Haproxy configuration
$ sudo nano haproxy.cfg
3- in end off file add
listen mongo
bind 0.0.0.0:27017
timeout connect 10s
timeout client 1m
timeout server 1m
mode TCP
server AWSmongo <database-host-url>:27017
4- now restart HaProxy
$ sudo service HaPoxy restart
5- now you can access your database using
{uri: 'mongodb://<database-user>:<database-pass>#<EC2-IP>:27017/<db>'}
I'm attempting to retrieve a User model from a Node js 8.12.0 API, using knex and bookshelf ORM. Database is Postgres 10.4.
The API works fine locally, but hosted on ElasticBeanstalk EC2 and RDS, I get error:
Unhandled rejection TimeoutError: Knex: Timeout acquiring a
connection. The pool is probably full. Are you missing a
.transacting(trx) call?
I'm able to connect and make queries to the RDS instance separately via connection string / password (it prompts for pw after I enter this):
psql -h myinstance.zmsnsdbakdha.us-east-1.rds.amazonaws.com -d mydb -U myuser
Security Groups:
The EC2 security group (set up by EB) is sg-0fa31004bd2b763ce, and RDS has an inbound security rule for PostgreSQL / TCP / port 5432 / for the matching source (sg-0fa31004bd2b763ce)— so it doesn't seem like the security group is a problem
RDS was created in a VPC, but the VPC's security rules are open too:
- security groups attached (multiple)
- name: mysgname
- group ID: sg-05d003b66fe1a4a94
- Inbound rules:
- All Traffic (0.0.0.0/0)
- HTTP (80) for TCP (0.0.0.0/0)
- SSH (22) for TCP (0.0.0.0/0)
- PostgreSQL (5432) for TCP (0.0.0.0/0)
Publicly accessible: Yes
users controller:
router.get('/users', function(req, res) {
new User.User({'id': 1})
.fetch({withRelated: ['addresses']})
.then((user) => {
res.send(user);
});
});
Knexfile:
production: {
client: 'pg',
version: '7.2',
connection: {
host: process.env.PG_HOST || 'localhost',
port: process.env.PG_PORT || '5432',
user: process.env.PG_USER || 'myuser',
password: process.env.PG_PASSWORD || '',
database: process.env.PG_DB || 'mydb',
charset: 'utf8',
},
pool: {
min: 2,
max: 20
},
},
Firstly, why is this happening only on AWS hosted environment and not locally. Secondly, how can I fix this issue? Should I increase max for pools?
You need to check your Network Access Control List (NACL) in your VPC and make sure your INBOUND and OUTBOUND are configured correctly. Security Groups are at the Instance level of security and the NACL is security at the Subnet level.
Most of the time when you are experiencing a Timeout error connecting to something in a custom VPC it will be a configuration problem with a Security Group or a NACL or Both.
I had a working code which was running in heroku instance. I have migrated to EBS and get stuck at this error for hours.
Heroku was setting NODE_ENV=production by default and i have corresponding configurations in my node.
But EBS does not set NODE_ENV=production by default so my code was breaking.
I have an Electron App which connects to an instance of a sql server(Microsoft SQL Server 2012) on my local machine using npm module mssql.The app works fine when connected to the internet, however I get the following error when offline.
"Failed to connect to DELL:undefined - getaddrinfo ENOENT DELL:48988"
following is the config:
var dbconfig = {
server: 'DELL\\INSTANCENAME',
database: 'myDB',
user: 'username',
password: 'password'
port: 1433
};
I have also made sure of the following:
1. TCP/IP protocol is enabled
2. SQL Server INSTANCE/Server Agent and Server Browser is enabled
Since what I'm building is a desktop app, I do not want any dependency with internet connection. Not sure what I'm missing.... please suggest.
I am trying to log into PostgreSQL on my EC2 server via SSH using Navicat.
I get the following error message:
"80070007: SSH Tunnel: Socket error on connecting. WSAGetLastError return 10061($274D)"
On the server, the "role" postgres already exists, and there is already a database called postgres. I have assigned a password to postgres (using ALTER NAME command via Putty).
The SSH settings I am using in Navicat are:
Port: 5432
User Name: [admin user name]
Authentication Method: Public Key
The Connection settings are:
Host Name: localhost
Port: 3306
Initial Database: postgres
User Name: postgres
Password: [password]
When I connect to the MySQL server on the same machine, the settings are exactly the same except for:
SSH to Port 22
User Name (for connection): root (with corresponding password)
I have tried the SSH to port 22, in which case the error message is:
"could not connect to server: Connection refused (0x0000274D/10061)
Is the server running on host "localhost" and accepting TCP/IP connections on port 60122?
received invalid response to SSL negotiation:4"
Any ideas on what settings I need to change to get this to work?
Your config seems to be very wrong.
ssh port should be not 5432, but 22 (ssh default).
postgresql port should be not 3306 (this is actually MySQL), but 5432 (postgres default)
To verify your setup, try ssh-ing into your EC2 instance manually.
After you ssh in, check if you can execute "telnet localhost 5432".
If you see an error immediately, that would mean that postgres server is not running.
If you see nothing - this is good sign and means that Postgres is running.
You can quit from this by Ctrl-], q, Enter.
Note that EC2 instances may require you to use ssh public key authentication (not a password). In this case, you will have to find option in Navicat to provide such a key.