I am having a problem with socket.io (websocket) when running in EC2. I don't have any http proxy or loadbalancer installed infront of the node instance. Same code works fine when running in local environment but keep reconnecting in EC2. Also even in the EC2 instance xhr-polling works fine.
Following is socket.io debug log output when transport is set to websocket and xhr-polling.
debug - discarding transport
debug - authorized
info - handshake authorized aoiP_6qFnqiqEC3r2-0N
debug - setting request GET /socket.io/1/websocket/aoiP_6qFnqiqEC3r2-0N
debug - set heartbeat interval for client aoiP_6qFnqiqEC3r2-0N
debug - client authorized for
debug - websocket writing 1::
warn - websocket parser error: reserved fields must be empty
info - transport end (undefined)
debug - set close timeout for client NxkBCtJqwOscfHzE0xba
debug - cleared close timeout for client NxkBCtJqwOscfHzE0xba
debug - cleared heartbeat interval for client NxkBCtJqwOscfHzE0xba
Related
What version of gRPC and what language are you using?
#grpc/grpc-js - 1.5.10
What operating system (Linux, Windows,...) and version?
server running in a docker container on azure cloud
What did you do?
I have created a grpc server with SSL. It is a test server, where I use self signed certificates for server. The connection between server and client works fine. But I enabled the debug and trace (tcp, http) logs on the server. I keep getting handshake failed error.
I0427 12:07:40.319067700 18 tcp_server_custom.cc:224] SERVER_CONNECT: 0x7f06409cf3a0 accepted connection: ipv4:10.92.0.9:52824
I0427 12:07:40.319239300 18 tcp_custom.cc:353] Creating TCP endpoint 0x7f0640c78430
I0427 12:07:40.319432800 18 tcp_custom.cc:174] TCP:0x7f0640c78430 read_allocation_done: "No Error"
I0427 12:07:40.319503900 18 tcp_custom.cc:191] Initiating read on 0x7f0640c78430: error="No Error"
I0427 12:07:40.331081600 18 tcp_custom.cc:127] TCP:0x7f0640afea60 call_cb 0x7f0641ed57e0 0x7f0640848b90:0x7f0641ed5610
I0427 12:07:40.331206000 18 tcp_custom.cc:131] read: error={"created":"#1651061260.331064200","description":"EOF","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":106}
D0427 12:07:40.331327300 18 security_handshaker.cc:176] Security handshake failed: {"created":"#1651061260.331311100","description":"Handshake read failed","file":"../deps/grpc/src/core/lib/security/transport/security_handshaker.cc","file_line":357,"referenced_errors":[{"created":"#1651061260.331064200","description":"EOF","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":106}]}
I0427 12:07:40.331412400 18 tcp_custom.cc:287] TCP 0x7f0640afea60 shutdown why={"created":"#1651061260.331311100","description":"Handshake read failed","file":"../deps/grpc/src/core/lib/security/transport/security_handshaker.cc","file_line":357,"referenced_errors":[{"created":"#1651061260.331064200","description":"EOF","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":106}]}
D0427 12:07:40.331443800 18 chttp2_server.cc:122] Handshaking failed: {"created":"#1651061260.331311100","description":"Handshake read failed","file":"../deps/grpc/src/core/lib/security/transport/security_handshaker.cc","file_line":357,"referenced_errors":[{"created":"#1651061260.331064200","description":"EOF","file":"../deps/grpc/src/core/lib/iomgr/tcp_uv.cc","file_line":106}]}
### Anything else we should know about your project / environment?
I have an envoy proxy also running for the grpc server to make grpc-web requests.
Node version: node:14-alpine
I just installed IoTEdge on Raspberry strech following this:Azure/InternetofThings/IoTEdge/Install or uninstall the Azure IoT Edge runtime
However I get these errors below.
3 weeks I installed another one and it worked perfectly with the same instructions.
pi#raspberrypi:/etc/iotedge $ sudo iotedge check --verbose
Configuration checks
--------------------
√ config.yaml is well-formed - OK
√ config.yaml has well-formed connection string - OK
√ container engine is installed and functional - OK
√ config.yaml has correct hostname - OK
× config.yaml has correct URIs for daemon mgmt endpoint - Error
One or more errors occurred. (Permission denied /var/run/iotedge/mgmt.sock)
caused by: docker returned exit code: 1, stderr = One or more errors occurred. (Permission denied /var/run/iotedge/mgmt.sock)
√ latest security daemon - OK
√ host time is close to real time - OK
√ container time is close to host time - OK
‼ DNS server - Warning
Container engine is not configured with DNS server setting, which may impact connectivity to IoT Hub.
Please see https://aka.ms/iotedge-prod-checklist-dns for best practices.
You can ignore this warning if you are setting DNS server per module in the Edge deployment.
caused by: Could not open container engine config file /etc/docker/daemon.json
caused by: No such file or directory (os error 2)
‼ production readiness: certificates - Warning
The Edge device is using self-signed automatically-generated development certificates.
They will expire in 89 days (at 2021-02-22 07:24:52 UTC) causing module-to-module and downstream device communication to fail on an active deployment.
After the certs have expired, restarting the IoT Edge daemon will trigger it to generate new development certs.
Please consider using production certificates instead. See https://aka.ms/iotedge-prod-checklist-certs for best practices.
√ production readiness: container engine - OK
‼ production readiness: logs policy - Warning
Container engine is not configured to rotate module logs which may cause it run out of disk space.
Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
You can ignore this warning if you are setting log policy per module in the Edge deployment.
caused by: Could not open container engine config file /etc/docker/daemon.json
caused by: No such file or directory (os error 2)
‼ production readiness: Edge Agent's storage directory is persisted on the host filesystem - Warning
The edgeAgent module is not configured to persist its /tmp/edgeAgent directory on the host filesystem.
Data might be lost if the module is deleted or updated.
Please see https://aka.ms/iotedge-storage-host for best practices.
× production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error
Could not check current state of edgeHub container
caused by: docker returned exit code: 1, stderr = Error: No such object: edgeHub
Connectivity checks
-------------------
√ host can connect to and perform TLS handshake with IoT Hub AMQP port - OK
√ host can connect to and perform TLS handshake with IoT Hub HTTPS / WebSockets port - OK
√ host can connect to and perform TLS handshake with IoT Hub MQTT port - OK
√ container on the default network can connect to IoT Hub AMQP port - OK
√ container on the default network can connect to IoT Hub HTTPS / WebSockets port - OK
√ container on the default network can connect to IoT Hub MQTT port - OK
√ container on the IoT Edge module network can connect to IoT Hub AMQP port - OK
√ container on the IoT Edge module network can connect to IoT Hub HTTPS / WebSockets port - OK
√ container on the IoT Edge module network can connect to IoT Hub MQTT port - OK
17 check(s) succeeded.
4 check(s) raised warnings.
2 check(s) raised errors.
iotedge list
pi#raspberrypi:~ $ sudo iotedge list
A module runtime error occurred
caused by: Could not list modules
caused by: connection error: Connection reset by peer (os error 104)
I have a Graphql server with subscriptions enabled, when accessed directly from my client it works great. Now what I need to do is reroute the connection through my backend (an express server).
I have the following configuration on my server:
app.use(
"/graphql",
createProxyMiddleware({
target: `http://${GQL_URL}`,
changeOrigin: true,
ws: true,
})
);
This eventually works, however, the first connection when the server starts-up, which I can see in my console the message [HPM] Upgrading to WebSocket takes over five minute to complete.
By the time it manages to upgrade the connection, the client has already disconnected and retried multiple times and I get "instantly" all the stacked pending retries:
[server] [HPM] Upgrading to WebSocket
[server] [HPM] Client disconnected
[server] [HPM] Upgrading to WebSocket
[server] [HPM] Client disconnected
[server] [HPM] Upgrading to WebSocket
[server] [HPM] Upgrading to WebSocket
[server] [HPM] Client disconnected
[server] [HPM] Client disconnected
[server] [HPM] Upgrading to WebSocket
[server] [HPM] Client disconnected
[server] [HPM] Upgrading to WebSocket
[server] [HPM] Client disconnected
[server] [HPM] Upgrading to WebSocket
After that, the connections to the server work properly.
In principle this is not a major problem, as when I deploy, the application will not work for a few minutes and then start working, is not ideal but not all bad.
However, the major problem is when I am developing. Whenever I make any change to the servers code and it reloads I have to wait those few minutes to properly checkout if my application is working.
How can I fix this issue? Am I missing something?
I am running a Node.js SSH server that spawns a child process to exec code (using require('child_process').spawn) after successful authentication.
The client server connections works fine on port 22 and connection is kept alive successfully through spawned process.
I am trying to setup up now with HAProxy 1.6, to forward port 22 to a non-privileged port on which the SSH server is listening.
However, when the child process is spawned the server either errors Error: write EPIPE or Error: read ECONNRESET.
This suggests to me there is an issue with prematurely closed stream or connection between the client -> HAProxy -> server?
I am looking at websocket configurations and ssh configurations for HAProxy and various keep alive options. However I cannot get the connection to work.
My configuration:
global
daemon
maxconn 10000
log 127.0.0.1 local0
defaults
log global
option tcplog
option logasap
timeout connect 500s
timeout client 5000s
timeout server 2h
timeout server-fin 5000s
timeout client-fin 5000s
timeout tunnel 1h
option tcpka
frontend sshd
bind *:22
default_backend ssh
timeout client 2h
backend ssh
mode tcp
server ssh2server 127.0.0.1:5000 check port 5000
Any pointers or help would be awesome. Thanks in advance.
EDIT
Runing haproxy in debug mode I have
00000000:sshd.accept(0004)=0005 from [my ip]
00000000:ssh.srvcls[0005:0006]
00000000:ssh.clicls[0005:0006]
00000000:ssh.closed[0005:0006].
On the tcplog
Oct 15 15:15:38 localhost haproxy[16036]: 128.277.13.23:51146 [15/Oct/2016:15:15:38.804] sshd ssh/ssh2server 1/0/+0 +0 -- 1/1/1/1/0 0/0
My app use Node.js and Socket.io. It works fine but when I disconnect from the internet, Socket.io falls back to xhr but just on Firefox, On chrome it works fine.
This my debug messages:
info - socket.io started
Express server listening on port 3000 development
GET /connexion 200 38ms - 882
POST /connexion 302 5ms - 64
GET /jeu 200 8ms - 583
debug - client authorized
info - handshake authorized UTHMxCJEKx-DC3sSf57M
debug - setting request GET /socket.io/1/xhr-polling/UTHMxCJEKx-DC3sSf57M?t=1365791206086
debug - setting poll timeout
debug - client authorized for
debug - clearing poll timeout
debug - xhr-polling writing 1::
debug - set close timeout for client UTHMxCJEKx-DC3sSf57M
debug - xhr-polling received data packet 1::/jeu/Jeu1
s:B2ZSTca4rYyle+ooJCY5/Z+d.xuW1XrVrU+IG0UpLvs5q88hm6Nr83SOwr7fpOjfPhbM
B2ZSTca4rYyle+ooJCY5/Z+d
debug - client authorized for /jeu/Jeu1
debug - setting request GET /socket.io/1/xhr-polling/UTHMxCJEKx-DC3sSf57M?t=1365791206116
debug - setting poll timeout
debug - clearing poll timeout
debug - xhr-polling writing 1::/jeu/Jeu1
debug - set close timeout for client UTHMxCJEKx-DC3sSf57M
debug - discarding transport
debug - cleared close timeout for client UTHMxCJEKx-DC3sSf57M
newPlayer: hamou92
debug - broadcasting packet
debug - setting request GET /socket.io/1/xhr-polling/UTHMxCJEKx-DC3sSf57M?t=1365791206203
debug - setting poll timeout
debug - discarding transport
debug - cleared close timeout for client UTHMxCJEKx-DC3sSf57M
This is what Firebug shows me:
Firefox ne peut établir de connexion avec le serveur à l'adresse ws://localhost:3000/socket.io/1/websocket/_uQklvPYeYn-_BNg-WDa.
this.websocket = new Socket(this.prepareUrl() + query); socket.io.js (ligne 2371)