Hello I am trying to create a testbucket on a Ceph Raspberry Pi cluster(local) and I get the following error message:
OS:Debian Jessie
Ceph: v12.2.12 Luminous
s3cmd:2.0.2
[ceph_deploy.rgw][INFO ] The Ceph Object Gateway (RGW) is now running on host admin and default port 7480
./s3cmd --debug mb s3://testbucket
Debug Message:
DEBUG: Unicodising 'mb' using UTF-8
DEBUG: Unicodising 's3://testbucket' using UTF-8
DEBUG: Command: mb
DEBUG: CreateRequest: resource[uri]=/
DEBUG: Using signature v2
DEBUG: SignHeaders: u'PUT\n\n\n\nx-amz-date:Wed, 15 Jan 2020 02:28:25 +0000\n/testbucket/'
DEBUG: Processing request, please wait...
DEBUG: get_hostname(testbucket): 192.168.178.50:7480
DEBUG: ConnMan.get(): creating new connection: http://192.168.178.50:7480
DEBUG: non-proxied HTTPConnection(192.168.178.50, 7480)
DEBUG: Response:
DEBUG: Unicodising './s3cmd' using UTF-8
DEBUG: Unicodising '--debug' using UTF-8
DEBUG: Unicodising 'mb' using UTF-8
DEBUG: Unicodising 's3://testbucket' using UTF-8
Invoked as: ./s3cmd --debug mb s3://testbucket
Problem: error: [Errno 111] Connection refused
S3cmd: 2.0.2
python: 2.7.17 (default, Oct 19 2019, 23:36:22)
[GCC 9.2.1 20190909]
environment LANG=en_GB.UTF-8
Traceback (most recent call last):
File "./s3cmd", line 3092, in <module>
rc = main()
File "./s3cmd", line 3001, in main
rc = cmd_func(args)
File "./s3cmd", line 237, in cmd_bucket_create
response = s3.bucket_create(uri.bucket(), cfg.bucket_location)
File "/home/cephuser/s3cmd-2.0.2/S3/S3.py", line 398, in bucket_create
response = self.send_request(request)
File "/home/cephuser/s3cmd-2.0.2/S3/S3.py", line 1258, in send_request
conn = ConnMan.get(self.get_hostname(resource['bucket']))
File "/home/cephuser/s3cmd-2.0.2/S3/ConnMan.py", line 253, in get
conn.c.connect()
File "/usr/lib/python2.7/httplib.py", line 831, in connect
self.timeout, self.source_address)
File "/usr/lib/python2.7/socket.py", line 575, in create_connection
raise err
error: [Errno 111] Connection refused
Does anyone know about the error ?
Solution:
Ceph-rgw Service stop automatically after installation
After my gateway was permanently available I could create a testbucket with s3cmd
Related
I am setting up the SSL layer on RabbitMQ on both server and clients. But the clients are failing when creating the connection to the server. At this point I am running the RabbitMQ server on a docker locally and the client locally using a conda environment.
Once the RabbitMQ server is up I see that the secure connection is accepting incoming connections:
test-rabbitmq-1 | 2023-01-20 08:22:01.692731+00:00 [info] <0.726.0> started TCP listener on [::]:5672
test-rabbitmq-1 | 2023-01-20 08:22:01.694836+00:00 [info] <0.746.0> started TLS (SSL) listener on [::]:7575
But the client refuses to connect with:
(rabbitmq-test) ➜ RabbitMQ-TSL ✗ python3 test.py
Enter PEM pass phrase: ********
INFO:pika.adapters.utils.connection_workflow:Pika version 1.3.1 connecting to ('127.0.0.1', 7575)
INFO:pika.adapters.utils.io_services_utils:Socket connected: <socket.socket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 43142), raddr=('127.0.0.1', 7575)>
ERROR:pika.adapters.utils.io_services_utils:SSL do_handshake failed: error=SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); <ssl.SSLSocket fd=6, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 43142), raddr=('127.0.0.1', 7575)>
Traceback (most recent call last):
File "/.../.local/lib/python3.10/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake
self._sock.do_handshake()
File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)
ERROR:pika.adapters.utils.connection_workflow:Attempt to create the streaming transport failed: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); 'localhost'/(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 7575)); ssl=True
ERROR:pika.adapters.utils.connection_workflow:AMQPConnector - reporting failure: AMQPConnectorTransportSetupError: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)')
ERROR:pika.adapters.utils.connection_workflow:AMQP connection workflow failed: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - AMQPConnectorTransportSetupError: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); first exception - None.
ERROR:pika.adapters.utils.connection_workflow:AMQPConnectionWorkflow - reporting failure: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - AMQPConnectorTransportSetupError: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); first exception - None
ERROR:pika.adapters.blocking_connection:Connection workflow failed: AMQPConnectionWorkflowFailed: 1 exceptions in all; last exception - AMQPConnectorTransportSetupError: SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)'); first exception - None
ERROR:pika.adapters.blocking_connection:Error in _create_connection().
Traceback (most recent call last):
File "/.../.local/lib/python3.10/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
File "/.../.local/lib/python3.10/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake
self._sock.do_handshake()
File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)
Traceback (most recent call last):
File "/.../test.py", line 16, in <module>
with pika.BlockingConnection(conn_params) as conn:
File "/.../.local/lib/python3.10/site-packages/pika/adapters/blocking_connection.py", line 360, in __init__
self._impl = self._create_connection(parameters, _impl_class)
File "/.../.local/lib/python3.10/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
raise self._reap_last_connection_workflow_error(error)
File "/.../.local/lib/python3.10/site-packages/pika/adapters/utils/io_services_utils.py", line 636, in _do_ssl_handshake
self._sock.do_handshake()
File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake
self._sslobj.do_handshake()
ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:997)
Any idea of what I am not setting properly on pika or at the RabbitMQ server?
On the server side I set the SSL layer at rabbitmq.conf as:
# Enable AMQPS
listeners.ssl.default = 7575
ssl_options.cacertfile = /etc/rabbitmq/cer/ca_certificate.pem
ssl_options.certfile = /etc/rabbitmq/cer/server_certificate.pem
ssl_options.keyfile = /etc/rabbitmq/cer/server_key.pem
ssl_options.verify = verify_peer
ssl_options.fail_if_no_peer_cert = true
# Enable HTTPS
management.listener.port = 15671
management.listener.ssl = true
management.listener.ssl_opts.cacertfile = /etc/rabbitmq/cer/ca_certificate.pem
management.listener.ssl_opts.certfile = /etc/rabbitmq/cer/server_certificate.pem
management.listener.ssl_opts.keyfile = /etc/rabbitmq/cer/server_key.pem
The docker compose file contains:
version: '3.8'
services:
rabbitmq:
image: rabbitmq:3-management
hostname: rabbitmq-server
volumes:
- ./rabbitmq-config/rabbitmq-cert:/etc/rabbitmq/cer
- ./rabbitmq-config/rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf
- ./rabbitmq/data:/var/lib/rabbitmq/mnesia/rabbit#my-rabbit
- ./rabbitmq/logs:/var/log/rabbitmq/log
ports:
- 5672:5672
- 7575:7575
- 15672:15672
Then, and to simply, I am using the demo client from RabbitMQ's guide as:
import logging
import pika
import ssl
from pika.credentials import ExternalCredentials
logging.basicConfig(level=logging.INFO)
context = ssl.create_default_context(
cafile = '/rabbitmq-config/rabbitmq-cert/ca_certificate.pem'
)
context.load_cert_chain(
'/rabbitmq-config/rabbitmq-cert/client_certificate.pem',
'/rabbitmq-config/rabbitmq-cert/client_key.pem'
)
ssl_options = pika.SSLOptions(context, "localhost")
conn_params = pika.ConnectionParameters(
port = 7575,
ssl_options = ssl_options,
credentials = ExternalCredentials()
)
with pika.BlockingConnection(conn_params) as conn:
ch = conn.channel()
ch.queue_declare("foobar")
ch.basic_publish("", "foobar", "Hello, world!")
print(ch.basic_get("foobar"))
Your Python code is set up to do X509 certificate authentication (you're not using username/password but are using ExternalCredentials). However, you have not configured RabbitMQ to accept X509 certificates for authentication (docs).
The Pika docs need to be updated, so I opened this issue - https://github.com/pika/pika/issues/1413
You would probably get a hint of this if you look at the RabbitMQ log file at the time your Python client tries to connect.
In order to enable X509 certificate authentication, do the following:
rabbitmq-plugins enable rabbitmq_auth_mechanism_ssl (docs)
Edit your rabbitmq.conf file and add the following section:
auth_mechanisms.1 = PLAIN
auth_mechanisms.1 = AMQPLAIN
auth_mechanisms.1 = EXTERNAL
Restart RabbitMQ
Add a password-less user that exactly matches the CN= value in your client certificate: rabbitmqctl add_user 'foobar, O=baz'. You can also get this value by attempting to connect after making the above changes. The failed auth attempt will be logged.
If you continue to have problems, I suggest asking on the mailing list as this is not a good forum for detailed analysis.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
I have installed elastic search through docker-compose on a machine. The version of elastic search in the yaml file is 1.11.0 and pyelasticsearch 7.6.0. I am trying to get all the indices from the elastic search server. The connection to the server is successful, I can see the message "Connection to ES Server successful", but I cannot get the indices. the line "es.indices.get_alias("*")" fails
try:
es = Elasticsearch("https://admin:admin#localhost:9200",
use_ssl = False,
ca_certs=False,
verify_certs=False)
print("Connection to ES Server successful")
except:
print("Unable to connect to server")
exit(1)
# get all wazuh indexes
lsindex = []
for i in es.indices.get_alias("*"):
if i.startswith(ALERTS_PREFIX):
lsindex.append(i)
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 246, in perform_request
method, url, body, retries=Retry(False), headers=request_headers, **kw
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 344, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.7/site-packages/urllib3/packages/six.py", line 686, in reraise
raise value
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
conn.connect()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 301, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.7/site-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f6255479110>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./mcode.py", line 185, in <module>
aut_for_tool()
File "./mcode.py", line 78, in aut_for_tool
for i in es.indices.get("*"):
File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
File "/usr/local/lib/python3.7/site-packages/elasticsearch/client/indices.py", line 193, in get
"GET", _make_path(index), params=params, headers=headers
File "/usr/local/lib/python3.7/site-packages/elasticsearch/transport.py", line 390, in perform_request
raise e
File "/usr/local/lib/python3.7/site-packages/elasticsearch/transport.py", line 365, in perform_request
timeout=timeout,
File "/usr/local/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 258, in perform_request
raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6255479110>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6255479110>: Failed to establish a new connection: [Errno 111] Connection refused)
This is my code example:
#!/usr/bin/env python
import sys
try:
from elasticsearch import Elasticsearch
except Exception as e:
print("No module 'elasticsearch' found.")
sys.exit()
try:
es = Elasticsearch("https://admin:admin#localhost:9200",
use_ssl = True,
ca_certs=True,
verify_certs=False)
print("Connection to ES Server successful")
except:
print("Unable to connect to server")
exit(1)
# get all wazuh indexes
lsindex = []
for i in es.indices.get("*"):
print(str(i))
And it produces the following output:
# python3 test.py
.opendistro_security
security-auditlog-2021.02.19
wazuh-alerts-4.x-2021.02.19
Can I ask why are you getting all Wazuh indices in your script? Understanding your use case could help me to point you to a better solution. If it's something related to reporting, Opendistro for elasticsearch has a module for it.
Regards,
Alberto R
I am really stuck in this one. I am trying to run in local mode a Spark Submit Operator in Airflow. And while the access right to the needed folder have read and execute right for any user, I have the following error. Any help would be great please.
[2020-12-08 11:00:54,456] {base_hook.py:89} INFO - Using connection to: id: spark_local. Host: local[*], Port: None, Schema: None, Login: None, Password: None, extra: XXXXXXXX
[2020-12-08 11:00:54,458] {spark_submit_hook.py:325} INFO - Spark-Submit cmd: /opt/spark/bin/ --master local[*] --name airflow-spark --queue root.default --deploy-mode client /home/ubuntu/market_risk/utils/etl.py
[2020-12-08 11:00:54,463] {taskinstance.py:1150} ERROR - [Errno 13] Permission denied: '/opt/spark/bin/'
Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 984, in _run_raw_task
result = task_copy.execute(context=context)
File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/contrib/operators/spark_submit_operator.py", line 187, in execute
self._hook.submit(self._application)
File "/home/ubuntu/.local/lib/python3.6/site-packages/airflow/contrib/hooks/spark_submit_hook.py", line 395, in submit
**kwargs)
File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: '/opt/spark/bin/'
My setting for Spark connection in Airflow are as follow:
In airflow UI, I have not set any environment variables.
I am using BrowserMobProxy to examine the traffic generated by my selenium webdriver. After restarting my PC I receive
ConnectionRefusedError: [WinError 10061] No connection could be made
because the target machine actively refused
message. It looks like it was not stopped correctly, by can it effect my system even after restart? Any advice?
Here is my code:
from browsermobproxy import Server
from selenium import webdriver
class ProxyManager:
browsermobproxy_path = "D:/browsermob-proxy-2.1.4/bin/browsermob-proxy.bat"
def __init__(self):
self.server = Server(ProxyManager.browsermobproxy_path)
self.client = None #will be initialized manually for better control
def start_server(self):
self.server.start()
return self.server
def start_client(self):
#disable certificate check
self.client = self.server.create_proxy(params={"trustALLServers":"true"})
return self.client
if "__main__" == __name__:
proxy = ProxyManager()
server = proxy.start_server()
client = proxy.start_client()
and here is my bmp.log:
[INFO 2020-09-04T01:02:13,322 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4
[INFO 2020-09-04T01:02:13,434 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT
[INFO 2020-09-04T01:02:13,710 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null}
[WARN 2020-09-04T01:02:14,170 org.eclipse.jetty.util.log] (main) FAILED SelectChannelConnector#0.0.0.0:8080: java.net.BindException: Address already in use: bind
[WARN 2020-09-04T01:02:14,172 org.eclipse.jetty.util.log] (main) FAILED org.eclipse.jetty.server.Server#56f0cc85: java.net.BindException: Address already in use: bind
[ERROR 2020-09-04T01:02:14,218 net.lightbody.bmp.proxy.Main] (main) Failed to start Jetty server. Aborting. java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_261]
at sun.nio.ch.Net.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.Net.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source) ~[?:1.8.0_261]
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:162) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:356) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:240) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.Server.doStart(Server.java:269) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]
at net.lightbody.bmp.proxy.Main.main(Main.java:64) [browsermob-dist-2.1.4.jar:?]
my full error message:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
raise err
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
chunked=chunked,
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 966, in send
self.connect()
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 187, in connect
conn = self._new_conn()
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\adapters.py", line 449, in send
timeout=timeout
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=8866): Max retries exceeded with url: http://localhost:8080/proxy?trustALLServers=true (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "<input>", line 23, in start_client
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\browsermobproxy\server.py", line 40, in create_proxy
client = Client(self.url[7:], params)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\browsermobproxy\client.py", line 31, in __init__
resp = requests.post('%s/proxy' % self.host + urlparams)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\adapters.py", line 510, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPConnectionPool(host='127.0.0.1', port=8866): Max retries exceeded with url: http://localhost:8080/proxy?trustALLServers=true (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')))
Well, I had found that if the script will halt at some point without executing server.stop() it will mess with the system proxy settings by not clearing the changes it had made.
I was able to configure the following setup,
[]
I am able to add a task in a job to run the testapi.py file which takes a simple string as input and send it to the flask app running inside the docker container.
But,
The task execution is throwing an error
> Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 355, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/local/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 183, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=6789): Max retries exceeded with url: /upload (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "preload_testapi.py", line 37, in <module>
r = requests.post(url, json=got_json)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 116, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=6789): Max retries exceeded with url: /upload (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused',))
Things i have done
I made sure that the start task(which runs the flask app) is
executed by sudo user created by me (not _azbatch user), tasks
inside the jobs are also executed by the same user.
I tested the same command executed by the task by doing ssh into the
same node with the user login created by me and the command runs
fine.
when I run the command from _azbatch it throws the same error.
It is not a flask port issue.
#fpark my start task execution is done properly, during node creation I download an image from Azure container registry, download 2 files from azure storage blob as resource files, start task is to run one of these files(shell script) which makes a container, run a flask app inside it from that image. added tasks are to hit the running flask app using the 2nd file(testapi.py)with different parameters for each task.
You can hit the flask API using a task. Follow this way:
Give a job that has two tasks to your compute node where task 1 runs the flask API and task 2 hits the API. And also enable parallel task execution for your nodes before trying this.
Hope this helps (～￣▽￣)～