I am using BrowserMobProxy to examine the traffic generated by my selenium webdriver. After restarting my PC I receive
ConnectionRefusedError: [WinError 10061] No connection could be made
because the target machine actively refused
message. It looks like it was not stopped correctly, by can it effect my system even after restart? Any advice?
Here is my code:
from browsermobproxy import Server
from selenium import webdriver
class ProxyManager:
browsermobproxy_path = "D:/browsermob-proxy-2.1.4/bin/browsermob-proxy.bat"
def __init__(self):
self.server = Server(ProxyManager.browsermobproxy_path)
self.client = None #will be initialized manually for better control
def start_server(self):
self.server.start()
return self.server
def start_client(self):
#disable certificate check
self.client = self.server.create_proxy(params={"trustALLServers":"true"})
return self.client
if "__main__" == __name__:
proxy = ProxyManager()
server = proxy.start_server()
client = proxy.start_client()
and here is my bmp.log:
[INFO 2020-09-04T01:02:13,322 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4
[INFO 2020-09-04T01:02:13,434 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT
[INFO 2020-09-04T01:02:13,710 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null}
[WARN 2020-09-04T01:02:14,170 org.eclipse.jetty.util.log] (main) FAILED SelectChannelConnector#0.0.0.0:8080: java.net.BindException: Address already in use: bind
[WARN 2020-09-04T01:02:14,172 org.eclipse.jetty.util.log] (main) FAILED org.eclipse.jetty.server.Server#56f0cc85: java.net.BindException: Address already in use: bind
[ERROR 2020-09-04T01:02:14,218 net.lightbody.bmp.proxy.Main] (main) Failed to start Jetty server. Aborting. java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_261]
at sun.nio.ch.Net.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.Net.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source) ~[?:1.8.0_261]
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:162) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:356) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:240) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.Server.doStart(Server.java:269) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]
at net.lightbody.bmp.proxy.Main.main(Main.java:64) [browsermob-dist-2.1.4.jar:?]
my full error message:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
raise err
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
chunked=chunked,
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 966, in send
self.connect()
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 187, in connect
conn = self._new_conn()
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\adapters.py", line 449, in send
timeout=timeout
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=8866): Max retries exceeded with url: http://localhost:8080/proxy?trustALLServers=true (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "<input>", line 23, in start_client
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\browsermobproxy\server.py", line 40, in create_proxy
client = Client(self.url[7:], params)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\browsermobproxy\client.py", line 31, in __init__
resp = requests.post('%s/proxy' % self.host + urlparams)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\adapters.py", line 510, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPConnectionPool(host='127.0.0.1', port=8866): Max retries exceeded with url: http://localhost:8080/proxy?trustALLServers=true (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')))
Well, I had found that if the script will halt at some point without executing server.stop() it will mess with the system proxy settings by not clearing the changes it had made.
Related
I used nginx to build mlflow server with its proxy_pass and integrated simple HTTP auth in nginx. However, when I ran the experiment for a while, the mlflow client met this exception. And I have no idea how to fix it.
Here is the error messages:
Traceback (most recent call last):
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/connection.py", line 159, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
TimeoutError: [Errno 60] Operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen
chunked=chunked)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/connectionpool.py", line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/connection.py", line 181, in connect
conn = self._new_conn()
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/connection.py", line 168, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x1280a8438>: Failed to establish a new connection: [Errno 60] Operation timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host=<host_ip>, port=<port>): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=<exp_name> (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1280a8438>: Failed to establish a new connection: [Errno 60] Operation timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "tmp_experiment_entry.py", line 4, in <module>
mlflow.set_experiment(<exp_name>)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mlflow/tracking/fluent.py", line 47, in set_experiment
experiment = client.get_experiment_by_name(experiment_name)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mlflow/tracking/client.py", line 151, in get_experiment_by_name
return self._tracking_client.get_experiment_by_name(name)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mlflow/tracking/_tracking_service/client.py", line 114, in get_experiment_by_name
return self.store.get_experiment_by_name(name)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mlflow/store/tracking/rest_store.py", line 219, in get_experiment_by_name
response_proto = self._call_endpoint(GetExperimentByName, req_body)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mlflow/store/tracking/rest_store.py", line 32, in _call_endpoint
return call_endpoint(self.get_host_creds(), endpoint, method, json_body, response_proto)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mlflow/utils/rest_utils.py", line 133, in call_endpoint
host_creds=host_creds, endpoint=endpoint, method=method, params=json_body)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mlflow/utils/rest_utils.py", line 70, in http_request
url=url, headers=headers, verify=verify, **kwargs)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mlflow/utils/rest_utils.py", line 51, in request_with_ratelimit_retries
response = requests.request(**kwargs)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host=<host_ip>, port=<port>): Max retries exceeded with url: /api/2.0/mlflow/experiments/get-by-name?experiment_name=<exp_name> (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x1280a8438>: Failed to establish a new connection: [Errno 60] Operation timed out',))
In the client, I use mlflow log by the following format and log_params, log_metrics in main function
with mlflow.start_run():
main(params)
My task is to control the Cisco ptz camera. However, I can’t even get a token. I successfully connect to the camera, but I get an error when I execute GetProfiles (). Through ONVIF Device Manager, everything works fine and I can see tokens and other information about my camera
Python 3.6.8 (default, Oct 7 2019, 12:59:55)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from onvif import ONVIFCamera
>>> mycam = ONVIFCamera("***.***.***.***", 15080, "admin", "********")
>>> media = mycam.create_media_service()
>>> poflist = media.GetProfiles()
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 83, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 357, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.6/http/client.py", line 1254, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1300, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1249, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1036, in _send_output
self.send(msg)
File "/usr/lib/python3.6/http/client.py", line 974, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 166, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f125b8bf7f0>: Failed to establish a new connection: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='192.168.1.203', port=80): Max retries exceeded with url: /onvif/device_service (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f125b8bf7f0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/onvif/client.py", line 23, in wrapped
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/onvif/client.py", line 153, in wrapped
return call(params, callback)
File "/usr/local/lib/python3.6/dist-packages/onvif/client.py", line 140, in call
ret = func(**params)
File "/usr/local/lib/python3.6/dist-packages/zeep/proxy.py", line 45, in __call__
kwargs,
File "/usr/local/lib/python3.6/dist-packages/zeep/wsdl/bindings/soap.py", line 122, in send
response = client.transport.post_xml(options["address"], envelope, http_headers)
File "/usr/local/lib/python3.6/dist-packages/zeep/transports.py", line 95, in post_xml
return self.post(address, message, headers)
File "/usr/local/lib/python3.6/dist-packages/zeep/transports.py", line 62, in post
address, data=message, headers=headers, timeout=self.operation_timeout
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 567, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 630, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='192.168.1.203', port=80): Max retries exceeded with url: /onvif/device_service (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f125b8bf7f0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/dist-packages/onvif/client.py", line 26, in wrapped
raise ONVIFError(err)
onvif.exceptions.ONVIFError: Unknown error: HTTPConnectionPool(host='192.168.1.203', port=80): Max retries exceeded with url: /onvif/device_service (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f125b8bf7f0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
I really appreciate any help you can provide.
Are you sure that you are pointing to the correct entry for the device_service? In some infrequent cases, it may be that the ONVIF interface runs on a port different from 80, where the webinterface is running.
In this case, usually a camera responds to the Probe messages by listing the XAddr for the device service with a port different form 80.
I suggest you to use wireshark and to sniff the traffic between ONVIF Device Manager and the camera and to check the correct URI used by ONVIF Device Manager to post the requests.
I was able to configure the following setup,
[]
I am able to add a task in a job to run the testapi.py file which takes a simple string as input and send it to the flask app running inside the docker container.
But,
The task execution is throwing an error
> Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 355, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/local/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 183, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=6789): Max retries exceeded with url: /upload (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "preload_testapi.py", line 37, in <module>
r = requests.post(url, json=got_json)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 116, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=6789): Max retries exceeded with url: /upload (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused',))
Things i have done
I made sure that the start task(which runs the flask app) is
executed by sudo user created by me (not _azbatch user), tasks
inside the jobs are also executed by the same user.
I tested the same command executed by the task by doing ssh into the
same node with the user login created by me and the command runs
fine.
when I run the command from _azbatch it throws the same error.
It is not a flask port issue.
#fpark my start task execution is done properly, during node creation I download an image from Azure container registry, download 2 files from azure storage blob as resource files, start task is to run one of these files(shell script) which makes a container, run a flask app inside it from that image. added tasks are to hit the running flask app using the 2nd file(testapi.py)with different parameters for each task.
You can hit the flask API using a task. Follow this way:
Give a job that has two tasks to your compute node where task 1 runs the flask API and task 2 hits the API. And also enable parallel task execution for your nodes before trying this.
Hope this helps (~ ̄▽ ̄)~
Recently I have come back to some code of mine, that used to work perfectly fine from the weather-api module (https://pypi.org/project/weather-api/). However now it just spits out a long error of which I'm not sure what to do with.
I have traced the error back to the weather.py, and tried artificially slowing down the request rate with time.sleep(), however to no avail.
from weather import Weather , Unit
weather = Weather(unit=Unit.CELSIUS)
location = weather.lookup_by_location('London')
This gives the error:
Traceback (most recent call last):
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\util\connection.py", line 60, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "D:\Program Files\Python\lib\socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 357, in _make_request
conn.request(method, url, **httplib_request_kw)
File "D:\Program Files\Python\lib\http\client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "D:\Program Files\Python\lib\http\client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "D:\Program Files\Python\lib\http\client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "D:\Program Files\Python\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "D:\Program Files\Python\lib\http\client.py", line 964, in send
self.connect()
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connection.py", line 166, in connect
conn = self._new_conn()
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x0398B1B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\adapters.py", line 440, in send
timeout=timeout
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\util\retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='query.yahooapis.com', port=80): Max retries exceeded with url: /v1/public/yql?q=select*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text='London')%20and%20u='c'%20&format=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0398B1B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#14>", line 1, in <module>
location = weather.lookup_by_location('London')
File "D:\Program Files\Python\lib\site-packages\weather\weather.py", line 27, in lookup_by_location
self.URL, location, self.unit)
File "D:\Program Files\Python\lib\site-packages\weather\weather.py", line 38, in _call
def _call(self, url):
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='query.yahooapis.com', port=80): Max retries exceeded with url: /v1/public/yql?q=select*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text='London')%20and%20u='c'%20&format=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0398B1B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))
Now trying to deconstruct the error I get to D:\Program Files\Python\Lib\site-packages\weather\weather.py at Ln: 38 (in the _call() function) which looks like:
def _call(self, url):
req = requests.get(url)
print('here') # my addition to the code. This is never reached.
if self.log:
self.logger.info("Request URL: %s" % req.url)
self.logger.info("Status Code: %s" % req.status_code)
self.logger.info("JSON Response: %s" % req.content)
if not req.ok:
req.raise_for_status()
try:
results = req.json()
if self.log:
self.logger.info(results)
if int(results['query']['count']) > 0:
wo = WeatherObject(results['query']['results']['channel'])
return wo
else:
if self.log:
self.logger.warn("No results found: %s " % results)
return
except Exception as e:
self.logger.warn(e)
self.logger.warn(req.content)
sys.exit(0)
I don't know why the requests module would cause this error, does anyone know a solution?
Expected outcome: a class object which should contain data from yahoo weather that can be read as a string with location.text.
Actual outcome: an error :/
Yahoo's weather API is deprecated! Who would've thought.
(Meaning trying to run requests.get(url) on the Yahoo url that no longer works didn't yield the correct results)
I have received a certificate.p12 with username and password.
While I am able to use Rest Client for post requests after i install this certificate in my system.
How can i use this certificate to authenticate post requests on Rest API using Python requests method ?
I am using below code but it is not working.
import requests
headers = {'Content-Type': 'application/json'}
payload = {'folder': '/Trial/trial_dir'}
response = requests.post('https://<IP>:8080/siteapi/availabletests', params=payload, headers=headers, verify='C:\\Users\\ukhare\\Desktop\\sigos\\cert.p12', auth=('trial_test','trialtest'))
And getting below error:
Traceback (most recent call last):
File "D:\m\Python34\lib\site-packages\urllib3\connection.py", line 171, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "D:\m\Python34\lib\site-packages\urllib3\util\connection.py", line 79, in create_connection
raise err
File "D:\m\Python34\lib\site-packages\urllib3\util\connection.py", line 69, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\m\Python34\lib\site-packages\urllib3\connectionpool.py", line 600, in urlopen
chunked=chunked)
File "D:\m\Python34\lib\site-packages\urllib3\connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "D:\m\Python34\lib\site-packages\urllib3\connectionpool.py", line 849, in _validate_conn
conn.connect()
File "D:\m\Python34\lib\site-packages\urllib3\connection.py", line 314, in connect
conn = self._new_conn()
File "D:\m\Python34\lib\site-packages\urllib3\connection.py", line 180, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x036B0230>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\m\Python34\lib\site-packages\requests\adapters.py", line 445, in send
timeout=timeout
File "D:\m\Python34\lib\site-packages\urllib3\connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "D:\m\Python34\lib\site-packages\urllib3\util\retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='IP', port=8080): Max retries exceeded with url: /siteapi/availabletests?folder=%2FTrial%2Ftrial_dir (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x036B0230>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\m\Python34\lib\site-packages\requests\api.py", line 112, in post
return request('post', url, data=data, json=json, **kwargs)
File "D:\m\Python34\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "D:\m\Python34\lib\site-packages\requests\sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "D:\m\Python34\lib\site-packages\requests\sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "D:\m\Python34\lib\site-packages\requests\adapters.py", line 513, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='IP', port=8080): Max retries exceeded with url: /siteapi/availabletests?folder=%2FTrial%2Ftrial_dir (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x036B0230>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',))
Ref: https://github.com/m-click/requests_pkcs12
I was using payload
data={"folder": "/Trial/trial_dir"}
which is a dictionary while it should be a proper string of dictionary
data='{"folder": "/Trial/trial_dir"}'
So the following are the findings for successful post requests with python :-
Header parameter should be a dictionary.
e.g.
headers={'Content-Type': 'application/json'}
Data parameter should be a dictionary in string format.
e.g.
data='{"folder": "/Trial/trial_dir"}'
Verify should be set to False : verify=False as to ignore verifying the SSL certificate.
Below is the status and contents received from request I made:
>>> import json
>>> from requests_pkcs12 import get,post
>>> url = 'https://IP:8080/siteapi/availabletests'
>>> pkcs12_filename = 'C:\\Users\\ukhare\\Desktop\\tests\\trial_tata.p12'
>>> pkcs12_password = 'trialtest'
>>> response = post(url, data='{"folder": "/Trial/trial_dir"}', headers={'Content-Type': 'application/json'}, verify=False, pkcs12_filename=pkcs12_filename,pkcs12_password=pkcs12_password)
D:\m\Python34\lib\site-packages\urllib3\connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
>>> print(response.status_code)
200
>>> print(json.dumps(json.loads(response.content.decode("utf-8")), indent=4, separators=(',', ': '), sort_keys=True))
{
"availableTests": [
"/Trial/trial_test/HTTP_Download"
],
"serviceError": null