Receiving: 'Auth error:Error: read ECONNRESET' when connecting to google cloud platform behind a proxy - node.js

I've been working on a Node.js project that accesses the google cloud api. When I'm in an external network the request works fine and I'm receiving the expected answer. Yet when I'm behind my cooperate proxy, which is necessary to access our smpt server, I receive the following error:
Auth error:Error: read ECONNRESET
To get through the cooperate proxy I'm using cntlm and I've set it as the environment proxy as mentioned in this article.
Furthermore I've enabled GRPC verbosity and GRPC handshake trace as seen in the code below.
process.env.GRPC_VERBOSITY = 'DEBUG'
process.env.GRPC_TRACE = 'handshaker'
process.env.HTTP_PROXY = 'http://127.0.0.1:3128'
process.env.http_proxy = 'http://127.0.0.1:3128'
process.env.https_proxy = 'http://127.0.0.1:3128'
process.env.HTTPS_PROXY = 'http://127.0.0.1:3128'
process.env.PROXY = 'http://127.0.0.1:3128'
process.env.proxy = 'http://127.0.0.1:3128'
// Imports the Google Cloud client library
const language = require('#google-cloud/language');
// Instantiates a client
const client = new language.LanguageServiceClient({
projectId:'CNL-Showcase',
keyFilename: 'path/to/file/creds.json'
});
const document = {
content: request,
type: 'PLAIN_TEXT',
language: 'DE',
encodingType: 'UTF-8'
};
console.log('sending to Google Api');
client
.analyzeEntities({ document: document })
.then((results: any) => {
...
}).catch((err: any) => {
console.error('ERROR:', err);
res.send(err);
});
Note: The path to my creds file is changed on purpose, as mentioned above if I'm not behind a proxy it works just fine.
The console output looks as follows:
D0809 09:49:45.850000000 3460 dns_resolver.cc:339] Using native dns resolver
sending to Google Api
D0809 09:49:46.039000000 3460 dns_resolver.cc:280] Start resolving.
I0809 09:49:46.069000000 3460 handshaker.cc:141] handshake_manager
000000000423BE30: adding handshaker http_connect [0000000
00286BDD0] at index 0
I0809 09:49:46.073000000 3460 handshaker.cc:141] handshake_manager
000000000423BE30: adding handshaker security [00000000028
6AE00] at index 1
I0809 09:49:46.076000000 3460 handshaker.cc:212] handshake_manager
000000000423BE30: error="No Error" shutdown=0 index=0, ar
gs={endpoint=0000000002868FA0, args=00000000028617C0 {size=9:
grpc.primary_user_agent=grpc-node/1.13.1, grpc.client_channel_f
actory=000007FEEA85EBA0, grpc.channel_credentials=000000000423C960,
grpc.server_uri=dns:///language.googleapis.com:443, grpc.
http_connect_server=language.googleapis.com:443,
grpc.default_authority=language.googleapis.com:443, grpc.http2_scheme=https,
grpc.security_connector=00000000028F54E0,
grpc.subchannel_address=ipv4:127.0.0.1:3128}, read_buffer=0000000002876860
(length
=0), exit_early=0}
I0809 09:49:46.081000000 3460 handshaker.cc:253] handshake_manager
000000000423BE30: calling handshaker http_connect [000000
000286BDD0] at index 0
I0809 09:49:46.083000000 3460 http_connect_handshaker.cc:300] Connecting to
server language.googleapis.com:443 via HTTP prox
y ipv4:127.0.0.1:3128
I0809 09:49:46.211000000 3460 handshaker.cc:212] handshake_manager
000000000423BE30: error="No Error" shutdown=0 index=1, ar
gs={endpoint=0000000002868FA0, args=00000000028617C0 {size=9:
grpc.primary_user_agent=grpc-node/1.13.1, grpc.client_channel_f
actory=000007FEEA85EBA0, grpc.channel_credentials=000000000423C960,
grpc.server_uri=dns:///language.googleapis.com:443, grpc.
http_connect_server=language.googleapis.com:443,
grpc.default_authority=language.googleapis.com:443, grpc.http2_scheme=https,
grpc.security_connector=00000000028F54E0,
grpc.subchannel_address=ipv4:127.0.0.1:3128}, read_buffer=0000000002876860
(length
=0), exit_early=0}
I0809 09:49:46.211000000 3460 handshaker.cc:253] handshake_manager
000000000423BE30: calling handshaker security [0000000002
86AE00] at index 1
I0809 09:49:46.303000000 3460 handshaker.cc:212] handshake_manager
000000000423BE30: error="No Error" shutdown=0 index=2, ar
gs={endpoint=000000000287A7F0, args=0000000002862720 {size=10:
grpc.primary_user_agent=grpc-node/1.13.1, grpc.client_channel_
factory=000007FEEA85EBA0, grpc.channel_credentials=000000000423C960,
grpc.server_uri=dns:///language.googleapis.com:443, grpc
.http_connect_server=language.googleapis.com:443,
grpc.default_authority=language.googleapis.com:443, grpc.http2_scheme=https
, grpc.security_connector=00000000028F54E0,
grpc.subchannel_address=ipv4:127.0.0.1:3128,
grpc.auth_context=000000000285DD60},
read_buffer=0000000002876860 (length=0), exit_early=0}
I0809 09:49:46.304000000 3460 handshaker.cc:240] handshake_manager
000000000423BE30: handshaking complete -- scheduling on_h
andshake_done with error="No Error"
I0809 09:49:46.305000000 3460 subchannel.cc:608] New connected subchannel at
0000000002867790 for subchannel 00000000028A027
0
Auth error:Error: read ECONNRESET
Auth error:Error: read ECONNRESET
Auth error:Error: read ECONNRESET
Auth error:Error: read ECONNRESET
Auth error:Error: read ECONNRESET
Auth error:Error: read ECONNRESET
I cannot identify what causes the issue as the handshake seems to work just fine.

After extensive search I found a solution:
Install a Proxyfier that routes the request from the library on the local machine over the proxy.
For Windows I found two apps:
http://www.proxycap.com/
https://www.proxifier.com/
I hope this helps someone trying to get PubSub running on proxy networks.

Related

urllib3 proxy exception handling

Errors
Urllib3 Exception Given 8 Errors. I need To Handle This Errors.
Error No 1
it given OSError. why oserror given?
Error No 2
it given ssl certificate error, but i pass the ssl certificate to proxymanager(below my code)?
Error No 3
New connection stablish error? but my internet conectiomn working fine?why stablish error given?
More Errors Below
1 ('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 VirusFound'))HTTPSConnectionPool(host='api.ipify.org', port=443)
2 (SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1131)HTTPSConnectionPool(host='api.ipify.org', port=443)/
3 (NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f1aa0334cd0>: Failed to establish a new connection: [Errno 110] Connection timed out'))HTTPSConnectionPool(host='api.ipify.org', port=443)
4 ('Cannot connect to proxy.', OSError('Tunnel connection failed: 307 Temporary Redirect'))HTTPSConnectionPool(host='api.ipify.org', port=443)
5 (EOF occurred in violation of protocol (_ssl.c:1131)HTTPSConnectionPool(host='api.ipify.org', port=443)
6 ('Cannot connect to proxy.', ConnectionResetError(104, 'Connection reset by peer'))HTTPSConnectionPool(host='api.ipify.org', port=443)
7 ('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f1aa20a5430>: Failed to establish a new connection: [Errno 110] Connection timed out'))HTTPSConnectionPool(host='api.ipify.org', port=443)
8 ('Your proxy appears to only use HTTP and not HTTPS, try changing your proxy URL to be HTTP. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#https-proxy-error-http-proxy', SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1131)')))HTTPSConnectionPool(host='api.ipify.org', port=443)
i have a many proxies. i send url api.ipify.org to request after some proxies given above 8 errors? how to handle this 8 errors?i need to except it. how to do it?
My Code is Below
global gcontext
gcontext = ssl.SSLContext()
def check_url_thread(in_proxy,urls):
global checked_url
headers = {
'User-Agent': random.choice(var.ua),
'Accept': 'text/html',
'Accept-Encoding': 'gzip;q=0,deflate;q=0',
'Connection': 'Keep-Alive',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Upgrade-Insecure-Requests': '1',
}
proxy = urllib3.ProxyManager('http://'+in_proxy,proxy_ssl_context = gcontext,headers =headers)
sys.stdout.write('| Cheking : '+urls+' | '+in_proxy+' | ')
try:
res = proxy.request('GET', url)
#checked_url = False
sys.stdout.write('\n| > '+'Success My proxy : '+str(in_proxy)+'| response : '+str(res.data.decode('utf-8'))+ ' | '+str(res.status))
except urllib3.exceptions.NewConnectionError as e:
sys.stdout.write('\n| > '+'Error New Connection Error : '+str(e))
except urllib3.exceptions.ProxySchemeUnknown as e:
sys.stdout.write('\n| > '+'Error proxy scheme unknown : '+str(e.scheme))
except urllib3.exceptions.ProxyError as e:
sys.stdout.write('\n| > '+'Error proxy error : '+str(e))
except urllib3.exceptions.MaxRetryError as e:
sys.stdout.write('\n| > '+'Error max retry error : '+str(e.reason)+str(e.pool)+str(e.url))
except urllib3.exceptions.TimeoutError as e:
sys.stdout.write('\n| > '+'Error Timeout Error : '+str(e))
except Exception as e:
sys.stdout.write('\n| > '+str(in_proxy)+str(e))
proxy.clear()
in_file = open('proxy.txt',"r")
for i in in_file:
lista_proxy_data.append(i.split("/n")[0])
in_file.close()
url = 'https://api.ipify.org'
for in_proxy in lista_proxy_data: // lista_proxy_data is proxy list array
try:
th = threading.Thread(target=check_url_thread,args=(in_proxy,url))
th.setDaemon(True)
th.start()
time.sleep(1)
except:
pass

How could I handle the Err of TLS certificate with fabric-sdk-node

I'm trying to build a web front end of a fabric-network, and after I completed the registerAdmin and registerUser, I got err when I was tring to run my js code.
root#oyu-virtual-machine:~/hyperledger-fabric/test/webapp# node get.js
Load privateKey and signedCert
Get History
Assigning transaction_id: 35e9ed932366df66448d789fbf5989e6ba31be555f96eaca3197475a1602749c
E0429 15:09:28.130483373 4413 ssl_transport_security.cc:1245] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
returned from gethistory
Gethistory result count = 1
error from gethistory = Error: 14 UNAVAILABLE: failed to connect to all addresses
at Object.exports.createStatusError (/root/hyperledger-fabric/test/webapp/node_modules/fabric-client/node_modules/grpc/src/common.js:91:15)
at Object.onReceiveStatus (/root/hyperledger-fabric/test/webapp/node_modules/fabric-client/node_modules/grpc/src/client_interceptors.js:1209:28)
at InterceptingListener._callNext (/root/hyperledger-fabric/test/webapp/node_modules/fabric-client/node_modules/grpc/src/client_interceptors.js:568:42)
at InterceptingListener.onReceiveStatus (/root/hyperledger-fabric/test/webapp/node_modules/fabric-client/node_modules/grpc/src/client_interceptors.js:618:8)
at callback (/root/hyperledger-fabric/test/webapp/node_modules/fabric-client/node_modules/grpc/src/client_interceptors.js:847:24) {
code: 14,
metadata: Metadata { _internal_repr: {}, flags: 0 },
details: 'failed to connect to all addresses'
}
Response is Error: 14 UNAVAILABLE: failed to connect to all addresses
In my opinion, the most important message is
E0429 15:09:28.130483373 4413 ssl_transport_security.cc:1245] Handshake failed with fatal error SSL_ERROR_SSL: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed.
It's not generated by my code, but system generated it.
I do hope someone could help me. Thanks in advance.

How can I connect a Wordpress service to a MariaDB service through Consul Connect?

I'm having a serious problem with connecting to MariaDB through Consul Connect.
I'm using Nomad to create the services with the proxies, with the following job definition:
job "wordpress" {
type = "service"
datacenters = ["dc1"]
group "server" {
network {
mode = "bridge"
port "http" {
static = 8080
to = 80
}
}
task "server" {
driver = "docker"
config {
image = "wordpress"
}
env {
WORDPRESS_DB_HOST = "${NOMAD_UPSTREAM_ADDR_database}"
WORDPRESS_DB_USER = "exampleuser"
WORDPRESS_DB_PASSWORD = "examplepass"
WORDPRESS_DB_NAME = "exampledb"
}
resources {
cpu = 100
memory = 64
network {
mbits = 10
}
}
}
service {
name = "wordpress"
tags = ["production", "wordpress"]
port = "http"
connect {
sidecar_service {
proxy {
upstreams {
destination_name = "database"
local_bind_port = 3306
}
}
}
}
}
}
group "database" {
network {
mode = "bridge"
port "db" {
to = 3306
}
}
task "database" {
driver = "docker"
config {
image = "mariadb"
}
env {
MYSQL_RANDOM_ROOT_PASSWORD = "yes"
MYSQL_INITDB_SKIP_TZINFO = "yes"
MYSQL_DATABASE = "exampledb"
MYSQL_USER = "exampleuser"
MYSQL_PASSWORD = "examplepass"
}
resources {
cpu = 100
memory = 128
network {
mbits = 10
}
}
}
service {
name = "database"
tags = ["production", "mariadb"]
port = "db"
connect {
sidecar_service {}
}
}
}
}
However, it seems that the server can't reach the database.
MySQL Connection Error: (2006) MySQL server has gone away
[25-Aug-2020 10:46:53 UTC] PHP Warning: mysqli::__construct(): Error while reading greeting packet. PID=187 in Standard input code on line 22
[25-Aug-2020 10:46:53 UTC] PHP Warning: mysqli::__construct(): (HY000/2006): MySQL server has gone away in Standard input code on line 22
MySQL Connection Error: (2006) MySQL server has gone away
WARNING: unable to establish a database connection to '127.0.0.1:3306'
continuing anyways (which might have unexpected results)
And the logs of the server and database proxies shows that some sort of TLS issue is happening, but I've got no clue how to solve this problem.
Server Proxy Logs
[2020-08-25 12:20:35.841][18][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:344] [C1229] Creating connection to cluster database.default.dc1.internal.0198bec5-d0b4-332c-973e-372808379192.consul
[2020-08-25 12:20:35.841][18][debug][pool] [source/common/tcp/conn_pool.cc:82] creating a new connection
[2020-08-25 12:20:35.841][18][debug][pool] [source/common/tcp/conn_pool.cc:362] [C1230] connecting
[2020-08-25 12:20:35.841][18][debug][connection] [source/common/network/connection_impl.cc:704] [C1230] connecting to 172.29.168.233:29307
[2020-08-25 12:20:35.841][18][debug][connection] [source/common/network/connection_impl.cc:713] [C1230] connection in progress
[2020-08-25 12:20:35.841][18][debug][pool] [source/common/tcp/conn_pool.cc:108] queueing request due to no available connections
[2020-08-25 12:20:35.841][18][debug][main] [source/server/connection_handler_impl.cc:280] [C1229] new connection
[2020-08-25 12:20:35.841][18][trace][connection] [source/common/network/connection_impl.cc:458] [C1229] socket event: 2
[2020-08-25 12:20:35.841][18][trace][connection] [source/common/network/connection_impl.cc:543] [C1229] write ready
[2020-08-25 12:20:35.841][18][trace][connection] [source/common/network/connection_impl.cc:458] [C1230] socket event: 2
[2020-08-25 12:20:35.841][18][trace][connection] [source/common/network/connection_impl.cc:543] [C1230] write ready
[2020-08-25 12:20:35.841][18][debug][connection] [source/common/network/connection_impl.cc:552] [C1230] connected
[2020-08-25 12:20:35.841][18][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:168] [C1230] handshake error: 2
[2020-08-25 12:20:35.842][18][trace][connection] [source/common/network/connection_impl.cc:458] [C1230] socket event: 3
[2020-08-25 12:20:35.842][18][trace][connection] [source/common/network/connection_impl.cc:543] [C1230] write ready
[2020-08-25 12:20:35.842][18][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:168] [C1230] handshake error: 5
[2020-08-25 12:20:35.842][18][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:201] [C1230]
[2020-08-25 12:20:35.842][18][debug][connection] [source/common/network/connection_impl.cc:190] [C1230] closing socket: 0
[2020-08-25 12:20:35.842][18][debug][pool] [source/common/tcp/conn_pool.cc:123] [C1230] client disconnected
Database Proxy Logs
[2020-08-25 12:26:07.093][15][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:201] [C927] new tcp proxy session
[2020-08-25 12:26:07.093][15][trace][connection] [source/common/network/connection_impl.cc:290] [C927] readDisable: enabled=true disable=true
[2020-08-25 12:26:07.093][15][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:344] [C927] Creating connection to cluster local_app
[2020-08-25 12:26:07.093][15][debug][pool] [source/common/tcp/conn_pool.cc:82] creating a new connection
[2020-08-25 12:26:07.093][15][debug][pool] [source/common/tcp/conn_pool.cc:362] [C928] connecting
[2020-08-25 12:26:07.093][15][debug][connection] [source/common/network/connection_impl.cc:704] [C928] connecting to 127.0.0.1:26344
[2020-08-25 12:26:07.093][15][debug][connection] [source/common/network/connection_impl.cc:713] [C928] connection in progress
[2020-08-25 12:26:07.093][15][debug][pool] [source/common/tcp/conn_pool.cc:108] queueing request due to no available connections
[2020-08-25 12:26:07.093][15][debug][main] [source/server/connection_handler_impl.cc:280] [C927] new connection
[2020-08-25 12:26:07.093][15][trace][connection] [source/common/network/connection_impl.cc:458] [C927] socket event: 2
[2020-08-25 12:26:07.093][15][trace][connection] [source/common/network/connection_impl.cc:543] [C927] write ready
[2020-08-25 12:26:07.093][15][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:168] [C927] handshake error: 5
[2020-08-25 12:26:07.093][15][debug][connection] [source/extensions/transport_sockets/tls/ssl_socket.cc:201] [C927]
[2020-08-25 12:26:07.093][15][debug][connection] [source/common/network/connection_impl.cc:190] [C927] closing socket: 0
[2020-08-25 12:26:07.093][15][debug][pool] [source/common/tcp/conn_pool.cc:204] canceling pending request
[2020-08-25 12:26:07.093][15][debug][pool] [source/common/tcp/conn_pool.cc:212] canceling pending connection
[2020-08-25 12:26:07.093][15][debug][connection] [source/common/network/connection_impl.cc:101] [C928] closing data_to_write=0 type=1
[2020-08-25 12:26:07.093][15][debug][connection] [source/common/network/connection_impl.cc:190] [C928] closing socket: 1
[2020-08-25 12:26:07.093][15][debug][pool] [source/common/tcp/conn_pool.cc:123] [C928] client disconnected
[2020-08-25 12:26:07.093][15][trace][main] [source/common/event/dispatcher_impl.cc:158] item added to deferred deletion list (size=1)
[2020-08-25 12:26:07.093][15][debug][main] [source/server/connection_handler_impl.cc:80] [C927] adding to cleanup list
[2020-08-25 12:26:07.093][15][trace][main] [source/common/event/dispatcher_impl.cc:158] item added to deferred deletion list (size=2)
[2020-08-25 12:26:07.093][15][trace][main] [source/common/event/dispatcher_impl.cc:76] clearing deferred deletion list (size=2)
[2020-08-25 12:26:07.093][15][debug][pool] [source/common/tcp/conn_pool.cc:236] [C928] connection destroyed

autobahn.twisted.websocket opening handshake error 400

Problem: when Bitfinex websocket API connection established application gets errors:
2018-02-07T18:51:52+0200 connecting once using transport type "websocket" over endpoint "tcp"
2018-02-07T18:51:52+0200 Starting factory <autobahn.twisted.websocket.WampWebSocketClientFactory object at 0x106708550>
2018-02-07T18:51:52+0200 failing WebSocket opening handshake ('WebSocket connection upgrade failed (400 - BadRequest)')
2018-02-07T18:51:52+0200 dropping connection to peer tcp4:104.16.173.181:443 with abort=True: WebSocket connection upgrade failed (400 - BadRequest)
2018-02-07T18:51:52+0200 component failed: ConnectionAborted: Connection was aborted locally, using.
2018-02-07T18:51:52+0200 Connection failed: ConnectionAborted: Connection was aborted locally, using.
2018-02-07T18:51:52+0200 Stopping factory <autobahn.twisted.websocket.WampWebSocketClientFactory object at 0x106708550>
Code below:
from autobahn.twisted.component import Component, run
import json
cmp_Bitfinex = Component(
transports=[
{
u'type': u'websocket',
u'url': u'wss://api.bitfinex.com/ws',
u'endpoint': {
u'type': u'tcp',
u'host': 'api.bitfinex.com',
u'port': 443,
},
u'options': {
u"open_handshake_timeout": 100,
}
}
],
realm=u"realm1",
)
#cmp_Bitfinex.on_join
def joined(self):
def onTicker(*args):
print("Ticker event received:", args)
try:
yield from self.subscribe(onTicker, 'ticker', options=json.dumps({
"event": "subscribe",
"channel": "ticker",
"pair": "BTCUSD",
"prec": "P0",
"freq": "F0"
}))
except Exception as e:
print("Could not subscribe to topic:", e)
#cmp_Bitfinex.on_connect
def connected(session, details):
print('Connected: {} \r\n {}'.format(session, details))
def main():
run([cmp_Bitfinex])
if __name__ == "__main__":
main()
I understand, that problem can be in data that application sends. But, I can't understand what exactly wrong in my code.
App uses python 3.6, latest autobahn and latest Twisted
AFAICT autobahn components are for programming with WAMP.
I think you should be using their websocket programming. Check this example

Node rdkafka consumer stops fetching messages

Consumer stops fetching after getting following error
{ Error: Local: Broker transport failure at Error (native) origin: 'local', message: 'broker transport failure', code: -1, errno: -1, stack: 'Error: Local: Broker transport failure\n at Error (native)' }
All kafka servers are healthy even then I am getting the transport failure .
After this error consumers stops fetching further messages

Resources