How to use .p12 certificate to authenticate rest api - python-3.x

I have received a certificate.p12 with username and password.
While I am able to use Rest Client for post requests after i install this certificate in my system.
How can i use this certificate to authenticate post requests on Rest API using Python requests method ?
I am using below code but it is not working.
import requests
headers = {'Content-Type': 'application/json'}
payload = {'folder': '/Trial/trial_dir'}
response = requests.post('https://<IP>:8080/siteapi/availabletests', params=payload, headers=headers, verify='C:\\Users\\ukhare\\Desktop\\sigos\\cert.p12', auth=('trial_test','trialtest'))
And getting below error:
Traceback (most recent call last):
File "D:\m\Python34\lib\site-packages\urllib3\connection.py", line 171, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "D:\m\Python34\lib\site-packages\urllib3\util\connection.py", line 79, in create_connection
raise err
File "D:\m\Python34\lib\site-packages\urllib3\util\connection.py", line 69, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\m\Python34\lib\site-packages\urllib3\connectionpool.py", line 600, in urlopen
chunked=chunked)
File "D:\m\Python34\lib\site-packages\urllib3\connectionpool.py", line 343, in _make_request
self._validate_conn(conn)
File "D:\m\Python34\lib\site-packages\urllib3\connectionpool.py", line 849, in _validate_conn
conn.connect()
File "D:\m\Python34\lib\site-packages\urllib3\connection.py", line 314, in connect
conn = self._new_conn()
File "D:\m\Python34\lib\site-packages\urllib3\connection.py", line 180, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x036B0230>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\m\Python34\lib\site-packages\requests\adapters.py", line 445, in send
timeout=timeout
File "D:\m\Python34\lib\site-packages\urllib3\connectionpool.py", line 638, in urlopen
_stacktrace=sys.exc_info()[2])
File "D:\m\Python34\lib\site-packages\urllib3\util\retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='IP', port=8080): Max retries exceeded with url: /siteapi/availabletests?folder=%2FTrial%2Ftrial_dir (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x036B0230>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\m\Python34\lib\site-packages\requests\api.py", line 112, in post
return request('post', url, data=data, json=json, **kwargs)
File "D:\m\Python34\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "D:\m\Python34\lib\site-packages\requests\sessions.py", line 512, in request
resp = self.send(prep, **send_kwargs)
File "D:\m\Python34\lib\site-packages\requests\sessions.py", line 622, in send
r = adapter.send(request, **kwargs)
File "D:\m\Python34\lib\site-packages\requests\adapters.py", line 513, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='IP', port=8080): Max retries exceeded with url: /siteapi/availabletests?folder=%2FTrial%2Ftrial_dir (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x036B0230>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it',))

Ref: https://github.com/m-click/requests_pkcs12
I was using payload
data={"folder": "/Trial/trial_dir"}
which is a dictionary while it should be a proper string of dictionary
data='{"folder": "/Trial/trial_dir"}'
So the following are the findings for successful post requests with python :-
Header parameter should be a dictionary.
e.g.
headers={'Content-Type': 'application/json'}
Data parameter should be a dictionary in string format.
e.g.
data='{"folder": "/Trial/trial_dir"}'
Verify should be set to False : verify=False as to ignore verifying the SSL certificate.
Below is the status and contents received from request I made:
>>> import json
>>> from requests_pkcs12 import get,post
>>> url = 'https://IP:8080/siteapi/availabletests'
>>> pkcs12_filename = 'C:\\Users\\ukhare\\Desktop\\tests\\trial_tata.p12'
>>> pkcs12_password = 'trialtest'
>>> response = post(url, data='{"folder": "/Trial/trial_dir"}', headers={'Content-Type': 'application/json'}, verify=False, pkcs12_filename=pkcs12_filename,pkcs12_password=pkcs12_password)
D:\m\Python34\lib\site-packages\urllib3\connectionpool.py:857: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
>>> print(response.status_code)
200
>>> print(json.dumps(json.loads(response.content.decode("utf-8")), indent=4, separators=(',', ': '), sort_keys=True))
{
"availableTests": [
"/Trial/trial_test/HTTP_Download"
],
"serviceError": null

Related

BrowserMobProxy: ConnectionRefusedError: [WinError 10061] after system restart

I am using BrowserMobProxy to examine the traffic generated by my selenium webdriver. After restarting my PC I receive
ConnectionRefusedError: [WinError 10061] No connection could be made
because the target machine actively refused
message. It looks like it was not stopped correctly, by can it effect my system even after restart? Any advice?
Here is my code:
from browsermobproxy import Server
from selenium import webdriver
class ProxyManager:
browsermobproxy_path = "D:/browsermob-proxy-2.1.4/bin/browsermob-proxy.bat"
def __init__(self):
self.server = Server(ProxyManager.browsermobproxy_path)
self.client = None #will be initialized manually for better control
def start_server(self):
self.server.start()
return self.server
def start_client(self):
#disable certificate check
self.client = self.server.create_proxy(params={"trustALLServers":"true"})
return self.client
if "__main__" == __name__:
proxy = ProxyManager()
server = proxy.start_server()
client = proxy.start_client()
and here is my bmp.log:
[INFO 2020-09-04T01:02:13,322 net.lightbody.bmp.proxy.Main] (main) Starting BrowserMob Proxy version 2.1.4
[INFO 2020-09-04T01:02:13,434 org.eclipse.jetty.util.log] (main) jetty-7.x.y-SNAPSHOT
[INFO 2020-09-04T01:02:13,710 org.eclipse.jetty.util.log] (main) started o.e.j.s.ServletContextHandler{/,null}
[WARN 2020-09-04T01:02:14,170 org.eclipse.jetty.util.log] (main) FAILED SelectChannelConnector#0.0.0.0:8080: java.net.BindException: Address already in use: bind
[WARN 2020-09-04T01:02:14,172 org.eclipse.jetty.util.log] (main) FAILED org.eclipse.jetty.server.Server#56f0cc85: java.net.BindException: Address already in use: bind
[ERROR 2020-09-04T01:02:14,218 net.lightbody.bmp.proxy.Main] (main) Failed to start Jetty server. Aborting. java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method) ~[?:1.8.0_261]
at sun.nio.ch.Net.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.Net.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source) ~[?:1.8.0_261]
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source) ~[?:1.8.0_261]
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:162) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.AbstractConnector.doStart(AbstractConnector.java:356) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:240) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.server.Server.doStart(Server.java:269) ~[browsermob-dist-2.1.4.jar:?]
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:55) ~[browsermob-dist-2.1.4.jar:?]
at net.lightbody.bmp.proxy.Main.main(Main.java:64) [browsermob-dist-2.1.4.jar:?]
my full error message:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
raise err
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 677, in urlopen
chunked=chunked,
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 392, in _make_request
conn.request(method, url, **httplib_request_kw)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1252, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1298, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1247, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "C:\Users\Kosh\AppData\Local\Programs\Python\Python37\lib\http\client.py", line 966, in send
self.connect()
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 187, in connect
conn = self._new_conn()
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connection.py", line 172, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\adapters.py", line 449, in send
timeout=timeout
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\connectionpool.py", line 727, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\urllib3\util\retry.py", line 439, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=8866): Max retries exceeded with url: http://localhost:8080/proxy?trustALLServers=true (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "<input>", line 23, in start_client
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\browsermobproxy\server.py", line 40, in create_proxy
client = Client(self.url[7:], params)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\browsermobproxy\client.py", line 31, in __init__
resp = requests.post('%s/proxy' % self.host + urlparams)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Kosh\PycharmProjects\CSSSR_Selenium\venv\lib\site-packages\requests\adapters.py", line 510, in send
raise ProxyError(e, request=request)
requests.exceptions.ProxyError: HTTPConnectionPool(host='127.0.0.1', port=8866): Max retries exceeded with url: http://localhost:8080/proxy?trustALLServers=true (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.HTTPConnection object at 0x00000273DE63C5C8>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')))
Well, I had found that if the script will halt at some point without executing server.stop() it will mess with the system proxy settings by not clearing the changes it had made.

How to use Python requests in Google Cloud Firestore Triggers

We are working on a mobile app where we are using firebase, we are using firestore as our database and maintaining data in collections here the thing we have to post the data to salesforce whenever there is any change or update in any of the records in order to do this we have created firestore triggers in the cloud using python 3.7 using this reference Click Here
, the cloud function is Trigger type as Cloud Firestore (Beta) and Event type as providers/cloud.firestore/eventTypes/document.write using {wildcard} with the path as Shipments/{ShipmentId} and it is triggered whenever there is a change in Shipments Collections modified my code in main.py file as
import requests
def TriggerShipmentChange(event, context):
resource_string = context.resource
print("function trigger by change to",{resource_string})
data = event
if data["oldValue"] == {} and data["updateMask"] == {}:
print("New shipment record $$$ Created $$$", data["value"])
elif data["updateMask"] == {} and data["value"] == {}:
print("Shipment record is $$$ deleted $$$", data["oldValue"])
elif data["oldValue"] != {} and data["value"] != {} and data["updateMask"] != {}:
print("Shipment record is $$$ updated $$$ and updated fields", data["updateMask"]["fieldPaths"])
print("updated values", data["value"])
else:
print("Function Response", data)
print("update details in backend")
URL ='http://*****.herokuapp.com'
result = requests.get(URL)
print("response", result.status_code, result.content)
and updated requirements.txt with
requests==2.22.0
and changed Function to execute to TriggerShipmentChange
I am able to execute these triggers and get the data of the updated record and hence planning to post them to salesforce here am using an external service to post data but first, in order to check connection I just made a get request which is throwing an error. Here I need to make a post request to an external server which is hosted in Heroku. I'll attach my logs
Function execution took 101 ms, finished with status: 'crash'
Traceback (most recent call last):
File "/env/local/lib/python3.7/site-packages/urllib3/connection.py", line 157, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw
File "/env/local/lib/python3.7/site-packages/urllib3/util/connection.py", line 61, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/opt/python3.7/lib/python3.7/socket.py", line 748, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
chunked=chunked,
File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 387, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/opt/python3.7/lib/python3.7/http/client.py", line 1244, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/opt/python3.7/lib/python3.7/http/client.py", line 1290, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/opt/python3.7/lib/python3.7/http/client.py", line 1239, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/python3.7/lib/python3.7/http/client.py", line 1026, in _send_output
self.send(msg)
File "/opt/python3.7/lib/python3.7/http/client.py", line 966, in send
self.connect()
File "/env/local/lib/python3.7/site-packages/urllib3/connection.py", line 184, in connect
conn = self._new_conn()
File "/env/local/lib/python3.7/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7eef49645410>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/env/local/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/env/local/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/env/local/lib/python3.7/site-packages/urllib3/util/retry.py", line 436, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='nltmsfirst.herokuapp.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eef49645410>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 383, in run_background_function
_function_handler.invoke_user_function(event_object)
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function
return call_user_function(request_or_event)
File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 214, in call_user_function
event_context.Context(**request_or_event.context))
File "/user_code/main.py", line 35, in TriggerShipmentChange
result = requests.get(URL)
File "/env/local/lib/python3.7/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/env/local/lib/python3.7/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/env/local/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/env/local/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/env/local/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='nltmsfirst.herokuapp.com', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7eef49645410>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
E
updated values {'createTime': '2020-01-28T09:56:01.307748Z', 'fields': {'ShipmentId': {'stringValue': '1234'}, 'Status': {'stringValue': 'Tendered'}}, 'name': 'projects/tablet-app-b9154/databases/(default)/documents/App/Dnow/Shipments/SR#1234', 'updateTime': '2020-01-29T08:24:48.233535Z'}
Shipment record is $$$ updated $$$ and updated fields ['Status']
function trigger by change to {'projects/tablet-app-b9154/databases/(default)/documents/App/Dnow/Shipments/SR#1234'}
Function execution started D
I don't want to create a new HTTP function now, I just want to make the above code work is there any way to do that, if not how can I update this updated record data to another server?

"GetProfiles" does not work with my ONVIF camera

My task is to control the Cisco ptz camera. However, I can’t even get a token. I successfully connect to the camera, but I get an error when I execute GetProfiles (). Through ONVIF Device Manager, everything works fine and I can see tokens and other information about my camera
Python 3.6.8 (default, Oct 7 2019, 12:59:55)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from onvif import ONVIFCamera
>>> mycam = ONVIFCamera("***.***.***.***", 15080, "admin", "********")
>>> media = mycam.create_media_service()
>>> poflist = media.GetProfiles()
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 83, in create_connection
raise err
File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 357, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.6/http/client.py", line 1254, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1300, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1249, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/lib/python3.6/http/client.py", line 1036, in _send_output
self.send(msg)
File "/usr/lib/python3.6/http/client.py", line 974, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 166, in connect
conn = self._new_conn()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f125b8bf7f0>: Failed to establish a new connection: [Errno 110] Connection timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 398, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='192.168.1.203', port=80): Max retries exceeded with url: /onvif/device_service (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f125b8bf7f0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/onvif/client.py", line 23, in wrapped
return func(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/onvif/client.py", line 153, in wrapped
return call(params, callback)
File "/usr/local/lib/python3.6/dist-packages/onvif/client.py", line 140, in call
ret = func(**params)
File "/usr/local/lib/python3.6/dist-packages/zeep/proxy.py", line 45, in __call__
kwargs,
File "/usr/local/lib/python3.6/dist-packages/zeep/wsdl/bindings/soap.py", line 122, in send
response = client.transport.post_xml(options["address"], envelope, http_headers)
File "/usr/local/lib/python3.6/dist-packages/zeep/transports.py", line 95, in post_xml
return self.post(address, message, headers)
File "/usr/local/lib/python3.6/dist-packages/zeep/transports.py", line 62, in post
address, data=message, headers=headers, timeout=self.operation_timeout
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 567, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 520, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 630, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='192.168.1.203', port=80): Max retries exceeded with url: /onvif/device_service (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f125b8bf7f0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/dist-packages/onvif/client.py", line 26, in wrapped
raise ONVIFError(err)
onvif.exceptions.ONVIFError: Unknown error: HTTPConnectionPool(host='192.168.1.203', port=80): Max retries exceeded with url: /onvif/device_service (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f125b8bf7f0>: Failed to establish a new connection: [Errno 110] Connection timed out',))
I really appreciate any help you can provide.
Are you sure that you are pointing to the correct entry for the device_service? In some infrequent cases, it may be that the ONVIF interface runs on a port different from 80, where the webinterface is running.
In this case, usually a camera responds to the Probe messages by listing the XAddr for the device service with a port different form 80.
I suggest you to use wireshark and to sniff the traffic between ONVIF Device Manager and the camera and to check the correct URI used by ONVIF Device Manager to post the requests.

how to hit a flask app running in azure batch node using a task?

I was able to configure the following setup,
[]
I am able to add a task in a job to run the testapi.py file which takes a simple string as input and send it to the flask app running inside the docker container.
But,
The task execution is throwing an error
> Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
(self._dns_host, self.port), self.timeout, **extra_kw)
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 80, in create_connection
raise err
File "/usr/local/lib/python3.6/site-packages/urllib3/util/connection.py", line 70, in create_connection
sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 355, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.6/http/client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.6/http/client.py", line 1026, in _send_output
self.send(msg)
File "/usr/local/lib/python3.6/http/client.py", line 964, in send
self.connect()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 183, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python3.6/site-packages/urllib3/util/retry.py", line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=6789): Max retries exceeded with url: /upload (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "preload_testapi.py", line 37, in <module>
r = requests.post(url, json=got_json)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 116, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/api.py", line 60, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=6789): Max retries exceeded with url: /upload (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fe6ff71e630>: Failed to establish a new connection: [Errno 111] Connection refused',))
Things i have done
I made sure that the start task(which runs the flask app) is
executed by sudo user created by me (not _azbatch user), tasks
inside the jobs are also executed by the same user.
I tested the same command executed by the task by doing ssh into the
same node with the user login created by me and the command runs
fine.
when I run the command from _azbatch it throws the same error.
It is not a flask port issue.
#fpark my start task execution is done properly, during node creation I download an image from Azure container registry, download 2 files from azure storage blob as resource files, start task is to run one of these files(shell script) which makes a container, run a flask app inside it from that image. added tasks are to hit the running flask app using the 2nd file(testapi.py)with different parameters for each task.
You can hit the flask API using a task. Follow this way:
Give a job that has two tasks to your compute node where task 1 runs the flask API and task 2 hits the API. And also enable parallel task execution for your nodes before trying this.
Hope this helps (~ ̄▽ ̄)~

Python Weather-API Module - Errno 11001: getaddrinfo failed error

Recently I have come back to some code of mine, that used to work perfectly fine from the weather-api module (https://pypi.org/project/weather-api/). However now it just spits out a long error of which I'm not sure what to do with.
I have traced the error back to the weather.py, and tried artificially slowing down the request rate with time.sleep(), however to no avail.
from weather import Weather , Unit
weather = Weather(unit=Unit.CELSIUS)
location = weather.lookup_by_location('London')
This gives the error:
Traceback (most recent call last):
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connection.py", line 141, in _new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\util\connection.py", line 60, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "D:\Program Files\Python\lib\socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 357, in _make_request
conn.request(method, url, **httplib_request_kw)
File "D:\Program Files\Python\lib\http\client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "D:\Program Files\Python\lib\http\client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "D:\Program Files\Python\lib\http\client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "D:\Program Files\Python\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "D:\Program Files\Python\lib\http\client.py", line 964, in send
self.connect()
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connection.py", line 166, in connect
conn = self._new_conn()
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connection.py", line 150, in _new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x0398B1B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\adapters.py", line 440, in send
timeout=timeout
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\urllib3\util\retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='query.yahooapis.com', port=80): Max retries exceeded with url: /v1/public/yql?q=select*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text='London')%20and%20u='c'%20&format=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0398B1B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<pyshell#14>", line 1, in <module>
location = weather.lookup_by_location('London')
File "D:\Program Files\Python\lib\site-packages\weather\weather.py", line 27, in lookup_by_location
self.URL, location, self.unit)
File "D:\Program Files\Python\lib\site-packages\weather\weather.py", line 38, in _call
def _call(self, url):
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "C:\Users\Me\AppData\Roaming\Python\Python36\site-packages\requests\adapters.py", line 508, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='query.yahooapis.com', port=80): Max retries exceeded with url: /v1/public/yql?q=select*%20from%20weather.forecast%20where%20woeid%20in%20(select%20woeid%20from%20geo.places(1)%20where%20text='London')%20and%20u='c'%20&format=json (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0398B1B0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed',))
Now trying to deconstruct the error I get to D:\Program Files\Python\Lib\site-packages\weather\weather.py at Ln: 38 (in the _call() function) which looks like:
def _call(self, url):
req = requests.get(url)
print('here') # my addition to the code. This is never reached.
if self.log:
self.logger.info("Request URL: %s" % req.url)
self.logger.info("Status Code: %s" % req.status_code)
self.logger.info("JSON Response: %s" % req.content)
if not req.ok:
req.raise_for_status()
try:
results = req.json()
if self.log:
self.logger.info(results)
if int(results['query']['count']) > 0:
wo = WeatherObject(results['query']['results']['channel'])
return wo
else:
if self.log:
self.logger.warn("No results found: %s " % results)
return
except Exception as e:
self.logger.warn(e)
self.logger.warn(req.content)
sys.exit(0)
I don't know why the requests module would cause this error, does anyone know a solution?
Expected outcome: a class object which should contain data from yahoo weather that can be read as a string with location.text.
Actual outcome: an error :/
Yahoo's weather API is deprecated! Who would've thought.
(Meaning trying to run requests.get(url) on the Yahoo url that no longer works didn't yield the correct results)

Resources