curl (56) Recv failure: Connection reset by peer with a simple docker run - linux

Yesterday this image was working fine.
But from today i have always the same error.
Here the command :
sudo docker run --name matter -d --publish 8065:8065 --add-host dockerhost:127.0.0.1 mattermost/mattermost-preview
When i curl :
curl: (56) Recv failure: Connexion ré-initialisée par le correspondant
English : curl (56) Recv failure: Connection reset by peer with a
simple docker run
Sometimes when i wait i can access to my localhost.
When i i do this :
sudo netstat -tulpn | grep LISTEN
I have this :
tcp6 0 0 :::8065 :::*
LISTEN 25772/docker-proxy
I don't understand this error :/
I am on Linux Ubuntu 18

The server takes a while to get ready to accept requests.
If you do
sudo docker logs matter
you can see the output of the server. Wait until you see a message like this
{"level":"info","ts":1624436113.1993275,"caller":"app/server.go:1226","msg":"Server
is listening on [::]:8065","address":"[::]:8065"}
before you start making requests.

Related

Errors seen when setting up logspout in Hyperledger fabric 2.2

Following steps described here to setup logspout:
https://hyperledger-fabric.readthedocs.io/en/release-2.2/deploy_chaincode.html
Running this produces below errors:
./monitordocker.sh net_test
Starting monitoring on all containers on the network net_test
xxxx
docker: Error response from daemon: network net_test not found.
curl: (7) Failed to connect to 127.0.0.1 port 8000: Connection refused
xxx#xxxx:/home/fabric/fabric-samples/test-network#
xxx#xxxx:/home/fabric/fabric-samples/test-network# ./monitordocker.sh
Starting monitoring on all containers on the network basicnetwork_basic
xxxx
docker: Error response from daemon: network basicnetwork_basic not found.
curl: (7) Failed to connect to 127.0.0.1 port 8000: Connection refused
xxx#xxxx:/home/fabric/fabric-samples/test-network#
xxxx#xxxx:/home/fabric/fabric-samples/test-network# ./monitordocker.sh net_basic
Starting monitoring on all containers on the network net_basic
xxxx
docker: Error response from daemon: network net_basic not found.
curl: (7) Failed to connect to 127.0.0.1 port 8000: Connection refused
Few questions:
there is no process running in default port 8000. So connection refused error is expected. Do we need to use any other port ?
what is the name of the network to be given when running monitordocker.sh ?
Any other troubleshooting info is appreciated.
Ok, found the issue. The network name is fabric_test. So I issued command like
./monitordocker.sh fabric_test
This resolved the problem.

How to access a port exposed from a docker container?

$ docker container ls --format "table {{.ID}}\t{{.Names}}\t{{.Ports}}" -a
CONTAINER ID NAMES PORTS
ae87d83af7d3 hopeful_engelbart
d13e260c4dec unruffled_bouman
db2c482de210 jenkinsci 0.0.0.0:8080->8080/tcp, 50000/tcp
cd201cbd413e xyz 0.0.0.0:5000->5000/tcp
c64c32ac68b8 pqr
$ docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ae87d83af7d3 442c97a73937 "/bin/bash" 11 minutes ago Exited (0) 9 minutes ago hopeful_engelbart
d13e260c4dec 442c97a73937 "/bin/bash" 27 minutes ago Exited (0) 24 minutes ago unruffled_bouman
db2c482de210 jenkins/jenkins:lts "/sbin/tini -- /usr/…" 3 days ago Up 41 minutes 0.0.0.0:8080->8080/tcp, 50000/tcp jenkinsci
cd201cbd413e 442c97a73937 "bash" 3 days ago Up 7 minutes 0.0.0.0:5000->5000/tcp xyz
c64c32ac68b8 442c97a73937 "bash" 3 days ago Exited (0) 2 days ago pqr
Above outputs show that the port 5000 has been exposed (I hope).
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' xyz
172.17.0.3
Now when I do from the host machine:
wget -c 172.17.0.3:5000
--2019-12-30 16:26:44-- http://172.17.0.3:5000/
Connecting to 172.17.0.3:5000... failed: Connection refused.
What is the way to access that port since it is exposed and the container is running?
$ wget -c localhost:5000
--2019-12-30 16:41:57-- http://localhost:5000/
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:5000... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.
--2019-12-30 16:41:58-- (try: 2) http://localhost:5000/
Connecting to localhost (localhost)|127.0.0.1|:5000... connected.
HTTP request sent, awaiting response... Read error (Connection reset by peer) in headers.
Retrying.
First of all, check if you have an application, listening to the port inside your container. Just try to connect to it from your container:
docker exec xyz wget 127.0.0.1:5000
If it'll work then you have a problem with port exposing, otherwise, there is no web server running inside your container.
And the error you get
Read error (Connection reset by peer) in headers.
seems to point to the problem in your web server rather then connectivity issues.
I had this problem running one Quarkus application with Docker.
I found this topic on forum.dockers talking about a similar issue.
In this topic is said:
I was using flask, by default it binds to localhost & 5000, so you have to specify:
app.run(host=“0.0.0.0”)
So, in my case I guess the problem was on my application and not on Docker network.
I added this property on the Java command to start my application on my Dockerfile and everything worked fine:
-Dquarkus.http.host=0.0.0.0

mongodb service cannot start

mongodb service cannot start with systemctl start mongodb.service, it would ask for a password. After then when I try mongo command, it throws :
MongoDB shell version v3.6.2
connecting to: mongodb://127.0.0.1:27017
2018-03-18T16:05:39.307+0700 W NETWORK [thread1] Failed to connect to 127.0.0.1:27017, in(checking socket for error after poll), reason: Connection refused
2018-03-18T16:05:39.307+0700 E QUERY [thread1] Error: couldn't connect to server 127.0.0.1:27017, connection attempt failed :
connect#src/mongo/shell/mongo.js:251:13
#(connect):1:6
exception: connect failed
I've been through all google page 1 solution but did nothing, also I try another query, but still again dont work for me, uninstall & install again also didnt work. My current OS distribution is Linux Manjaro.
Any help is appreciated.
Try this:
Type "Mongod" in the terminal to start MongoDB
and then open another terminal and type mongo to start the shell.
This works fine for me.
The connection refused error sounds like a firewall issue.
I would check the appropriate logs for the real reason. In Linux check /var/log/system or /bar/log/messages or other files in that location.
If it turns out firewall issue or /etc/hosts blocking, the allow/open the appropriate port in.
Update:
Opening firewall (iptables for MongoDB port). Type sudo before the following commands and put of where the request is coming from.
iptables -A INPUT -s <ip-address> -p tcp --destination-port 27017 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A OUTPUT -d <ip-address> -p tcp --source-port 27017 -m state --state ESTABLISHED -j ACCEPT
**Background **
https://docs.mongodb.com/manual/tutorial/configure-linux-iptables-firewall/
Update 2: Add this additional rule to your firewall and type "sudo" in front of it : Outbound traffic must be accepted for the loopback (127.0.0.1) as well. credits go to: iptables blocking local connection to mongodb
iptables -A OUTPUT -o lo -j ACCEPT

What is etcd looking for in 127.0.0.1:4001?

I'm trying to set up a test cluster using etcd 2.3.7 installed from CentOS RPM on CentOS 7.1. On the Loader 1 I executed:
etcdctl member add loader2 http://10.11.51.231:2380
And received response which confirmed the operation completed successfully.
Similarly:
etcdctl member add loader3 http://10.11.51.231:2380
with all default settings, and here's what I see:
Loader 1 10.11.51.166
systemctl status etcd -ln1
etcd.service - Etcd Server
Loaded: loaded (/usr/lib/systemd/system/etcd.service; disabled)
Active: active (running) since Sun 2017-02-19 14:33:18 IST; 28min ago
Main PID: 19009 (etcd)
CGroup: /system.slice/etcd.service
└─19009 /usr/bin/etcd --name=default --data-dir=/var/lib/etcd/default.etcd --listen-client-urls=http://localhost:2379
Feb 19 15:02:03 loader3 etcd[19009]: cannot get the version of member a4803061db803edc (Get http://10.11.51.166:2380/version: dial tcp 10.11.51.166:2380: getsockopt: connection refused)
Tried to see cluster health:
etcdctl --debug cluster-health
Cluster-Endpoints: http://127.0.0.1:4001, http://127.0.0.1:2379
cURL Command: curl -X GET http://127.0.0.1:4001/v2/members
cURL Command: curl -X GET http://127.0.0.1:2379/v2/members
member ce2a822cea30bfca is unhealthy: got unhealthy result from http://localhost:2379
member da05b63349d818dc is unreachable: no available published client urls
cluster is unhealthy
Note how this ignores the two nodes added previously, but sends requests to random port on localhost...
Loader 2 10.11.51.174
At first this machine started OK, but after I saw there was something wrong with Loader 1, I tried adding Loader 1 as a member from this machine, and now I see the same picture on this machine too. I.e. it tries to query this 4001 port, where nobody responds. On all machines:
netstat -tupln | grep etcd
tcp 0 0 127.0.0.1:7001 0.0.0.0:* LISTEN 4507/etcd
tcp 0 0 127.0.0.1:2379 0.0.0.0:* LISTEN 4507/etcd
tcp 0 0 127.0.0.1:2380 0.0.0.0:* LISTEN 4507/etcd
Nobody listens on 4001...
Loader 3 10.11.51.231
On this loader I didn't try to add new members. So it looks like this:
etcdctl --debug cluster-health
Cluster-Endpoints: http://127.0.0.1:4001, http://127.0.0.1:2379
cURL Command: curl -X GET http://127.0.0.1:4001/v2/members
cURL Command: curl -X GET http://127.0.0.1:2379/v2/members
member ce2a822cea30bfca is healthy: got healthy result from http://localhost:2379
cluster is healthy
In other words it still sends requests to random port, but this time it isn't bothered by the fact that nobody replied...
Below is the contents of the configuration files:
cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
User=etcd
# set GOMAXPROCS to number of processors
ExecStart=/bin/bash -c "GOMAXPROCS=$(nproc) /usr/bin/etcd --name=\"${ETCD_NAME}\" --data-dir=\"${ETCD_DATA_DIR}\" --listen-client-urls=\"${ETCD_LISTEN_CLIENT_URLS}\""
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
And:
cat /etc/etcd/etcd.conf
# [member]
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
#ETCD_SNAPSHOT_COUNT="10000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_LISTEN_PEER_URLS="http://localhost:2380"
ETCD_LISTEN_CLIENT_URLS="http://localhost:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
#ETCD_CORS=""
#
#[cluster]
#ETCD_INITIAL_ADVERTISE_PEER_URLS="http://localhost:2380"
# if you use different ETCD_NAME (e.g. test), set ETCD_INITIAL_CLUSTER value for this name, i.e. "test=http://..."
#ETCD_INITIAL_CLUSTER="default=http://localhost:2380"
#ETCD_INITIAL_CLUSTER_STATE="new"
#ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_SRV=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_STRICT_RECONFIG_CHECK="false"
#
#[proxy]
#ETCD_PROXY="off"
#ETCD_PROXY_FAILURE_WAIT="5000"
#ETCD_PROXY_REFRESH_INTERVAL="30000"
#ETCD_PROXY_DIAL_TIMEOUT="1000"
#ETCD_PROXY_WRITE_TIMEOUT="5000"
#ETCD_PROXY_READ_TIMEOUT="0"
#
#[security]
#ETCD_CERT_FILE=""
#ETCD_KEY_FILE=""
#ETCD_CLIENT_CERT_AUTH="false"
#ETCD_TRUSTED_CA_FILE=""
#ETCD_PEER_CERT_FILE=""
#ETCD_PEER_KEY_FILE=""
#ETCD_PEER_CLIENT_CERT_AUTH="false"
#ETCD_PEER_TRUSTED_CA_FILE=""
#
#[logging]
#ETCD_DEBUG="false"
# examples for -log-package-levels etcdserver=WARNING,security=DEBUG
#ETCD_LOG_PACKAGE_LEVELS=""
#
#[profiling]
#ETCD_ENABLE_PPROF="false"
So... what is going on? The error messages given by etcd are the typical mindless nonsense produced by Go built-ins. The HTTP server that etcd uses is again, the Go built-in junk, that produces non-standard and absolutely worthless replies. So I cannot understand what was (if at all) misconfigured / missing.

TCP listening socket is not created

I'm developing a Qt application and experience rather weird network issue.
Let me show how it looks from end-user perspective.
First I start up my server and verify that it's listening on a target port:
[user#host server]$ sudo netstat -anp | grep 30004
tcp 0 0 0.0.0.0:30004 0.0.0.0:* LISTEN 11113/./server
Then I connect to the server with telnet:
[user#host server]$ telnet localhost 30004
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connection closed by foreign host.
Netstat displays that connection is now established. Nothing fancy so far:
[user#host server]$ sudo netstat -anp | grep 30004
tcp 0 0 0.0.0.0:30004 0.0.0.0:* LISTEN 11113/./server
tcp 0 0 127.0.0.1:30004 127.0.0.1:34608 ESTABLISHED 11113/./server
tcp 0 0 127.0.0.1:34608 127.0.0.1:30004 ESTABLISHED 12657/telnet
Then the server drops the connection based on application-specific timeout. It is set to 10 seconds at the moment:
[user#host server]$ sudo netstat -anp | grep 30004
tcp 0 0 0.0.0.0:30004 0.0.0.0:* LISTEN 11113/./server
tcp 0 0 127.0.0.1:30004 127.0.0.1:34608 TIME_WAIT -
I then shut down the server and verify that the listenning socket is destroyed:
[user#host server]$ sudo netstat -anp | grep 30004
tcp 0 0 127.0.0.1:30004 127.0.0.1:34608 TIME_WAIT -
Finally I start up the server again, but the listening port doesn't show up anymore:
[user#host server]$ sudo netstat -anp | grep 30004
tcp 0 0 127.0.0.1:30004 127.0.0.1:34608 TIME_WAIT -
As a result client cannot connect to the server:
[user#host server]$ telnet localhost 30004
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
What am I doing wrong here? I'm inclined to think that this is a configuration issue, not a bug in the application.
This scenario seems to work on my laptop's Ubuntu. The aforementioned output was produced on linux box as well.
UPDATE: One more thing that is different in these two environemnt is qt version. On my notebook I have 4.8.6, on linux box it's 4.6.2. Not sure if it matters.
Apparently there was an issue with versions of qt libraries. We upgraded it to latest 4.x.x and now the problem seems to be resolved.

Resources