Python (Pyppeteer): [WinError 10054] An existing connection was forcibly closed by the remote host - python-3.x

I am trying to serve some HTML through a local server setup using Python and then use the pyppeteer module to take a screenshot of the web page.
This is the Python code to take the screenshot.
from pyppeteer import launch
async def main():
browser = await launch(headless=True)
page = await browser.newPage()
arg = sys.argv[1]
await page.goto("http://localhost:8888/map.html"), {'waitUntil': 'networkidle2'})
await page.pdf({'path': 'screenshot.pdf', 'landscape' : True, 'format': 'A4'});
await browser.close()
asyncio.get_event_loop().run_until_complete(main())
Here is map.html code that I am trying to serve over the Python server. It reads in a json file with some function settings and calls flex_map which has some D3 Javascript code.
<html>
<head>
<meta charset="utf-8">
<script src="https://d3js.org/d3.v5.min.js"></script>
<script src="/js/legend.js"></script>
<script src="/js/flex_map.js"></script>
</head>
<body>
<script>
d3.json("/settings/map_settings.json").then(function(mapSettings) {
flex_map(mapSettings.geoJsonFile,
mapSettings.dataFile,
mapSettings.fillVariable,
mapSettings.geoVariable,
mapSettings.geoJsonGeoVariable,
mapSettings.legend_title,
mapSettings.legendTranslateX,
mapSettings.legendTranslateY);
});
</script>
</body>
</html>
When I submit this code with python take_screenshot.py after creating the local server, I get the following error:
Serving HTTP on 0.0.0.0 port 8888 (http://0.0.0.0:8888/) ...
127.0.0.1 - - [23/Apr/2020 18:34:02] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [23/Apr/2020 18:34:04] "GET /map.html HTTP/1.1" 304 -
127.0.0.1 - - [23/Apr/2020 18:34:04] "GET /settings/map_settings.json HTTP/1.1" 304 -
127.0.0.1 - - [23/Apr/2020 18:34:04] "GET /data/temp_data.csv HTTP/1.1" 304 -
127.0.0.1 - - [23/Apr/2020 18:35:08] "GET /map.html HTTP/1.1" 200 -
127.0.0.1 - - [23/Apr/2020 18:35:08] "GET /js/legend.js HTTP/1.1" 200 -
127.0.0.1 - - [23/Apr/2020 18:35:08] "GET /js/flex_map.js HTTP/1.1" 200 -
127.0.0.1 - - [23/Apr/2020 18:35:08] "GET /settings/map_settings.json HTTP/1.1" 200 -
127.0.0.1 - - [23/Apr/2020 18:35:09] "GET /data/temp_data.csv HTTP/1.1" 200 -
127.0.0.1 - - [23/Apr/2020 18:35:09] "GET /geoJSON/us_counties.json HTTP/1.1" 200 -
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 52861)
Traceback (most recent call last):
File "C:\Anaconda\lib\socketserver.py", line 650, in process_request_thread
self.finish_request(request, client_address)
File "C:\Anaconda\lib\socketserver.py", line 360, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "C:\Anaconda\lib\http\server.py", line 646, in __init__
super().__init__(*args, **kwargs)
File "C:\Anaconda\lib\socketserver.py", line 720, in __init__
self.handle()
File "C:\Anaconda\lib\http\server.py", line 426, in handle
self.handle_one_request()
File "C:\Anaconda\lib\http\server.py", line 394, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "C:\Anaconda\lib\socket.py", line 589, in readinto
return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
----------------------------------------
I am not really sure where the source of this issue could be. One strange thing is if I change the HTML file to
<html>
<head>
<meta charset="utf-8">
<script src="https://d3js.org/d3.v5.min.js"></script>
<script src="/js/legend.js"></script>
<script src="/js/flex_map.js"></script>
</head>
<body>
<script>
flex_map(args go here...);
</script>
</body>
</html>
the code runs fine and takes a screenshot, so it might have something to do with loading in the json file but I am not sure how to resolve that. Also opening the map.html code on Chrome through a localhost works fine.

Related

Gunicorn access log format not apply

I'm using gunicorn to run a fastapi script, the access log file were created using the gunicorn.conf.py with accesslog yet it will not apply the access_log_format. I tried this apply this example from the github and is still not working
My gunicorn.conf.py
accesslog = '/home/ossbod/chunhueitest/supervisor_log/accesslog.log'
loglevel = 'info'
access_log_format = '%(h)s %(l)s %(t)s "%(r)s" %(s)s %(q)s %(b)s "%(f)s" "%(a)s" %(M)s'
The result I got
<IP>:54668 - "GET /docs HTTP/1.1" 200
<IP>:54668 - "GET /openapi.json HTTP/1.1" 200
<IP>:54668 - "POST /api/v1/add_user HTTP/1.1" 201
How can i get the format to apply to the log?

Django+Vue:static file not found

When I use Django+Vue to build a web appliction, It occoured that the staic files always not found though I had placed all the files correctly.
the logs from server like this:
WARNING Not Found: /static/js/app.4c2224dc.js
WARNING Not Found: /static/css/app.a5d43e07.css
WARNING Not Found: /static/css/chunk-vendors.2ccfa1b8.css
WARNING Not Found: /static/js/chunk-vendors.c2488b8d.js
WARNING "GET /static/js/app.4c2224dc.js HTTP/1.1" 404 179
WARNING "GET /static/css/app.a5d43e07.css HTTP/1.1" 404 179
WARNING "GET /static/css/chunk-vendors.2ccfa1b8.css HTTP/1.1" 404 179
WARNING "GET /static/js/chunk-vendors.c2488b8d.js HTTP/1.1" 404 179
WARNING Not Found: /login
WARNING "GET /login HTTP/1.1" 404 179
WARNING Not Found: //performanceProject
WARNING "GET //performanceProject HTTP/1.1" 404 179
WARNING Not Found: /performanceProject
WARNING "GET /performanceProject HTTP/1.1" 404 179
I solved the problem by https://www.cnpython.com/qa/1400788.
1、pip3 install whitenoise
2、modify settings.py add 'whitenoise.middleware.WhiteNoiseMiddleware' to MIDDLEWARE, example below:
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'whitenoise.middleware.WhiteNoiseMiddleware', # Add WhiteNoise here
...
]
# when DEBUG=true
# STATICFILES_DIRS = [
# os.path.join(BASE_DIR, 'dist/static')
# ]
# when debug=False
STATIC_ROOT = os.path.join(BASE_DIR, 'dist/static')
# always need to be set ignore debug=False or True
STATICFILES_STORAGE = 'whitenoise.storage.CompressedStaticFilesStorage'
3、python3 manage.py collectstatic
4、python3 manage.py runserver 0.0.0.0:XXXX

I am unable to successfully use my css file in my django project

[03/May/2021 11:44:59] "GET /add_food/ HTTP/1.1" 200 4583
[03/May/2021 11:44:59] "GET /static/css/base.css HTTP/1.1" 404 1763
[03/May/2021 11:45:04] "GET / HTTP/1.1" 200 3812
[03/May/2021 11:45:04] "GET /static/css/base.css HTTP/1.1" 404 1763
[03/May/2021 11:46:24] "GET / HTTP/1.1" 200 3812
[03/May/2021 11:46:24] "GET /static/css/base.css HTTP/1.1" 404 1763
[03/May/2021 11:48:07] "GET / HTTP/1.1" 200 3812
[03/May/2021 11:48:07] "GET /static/css/base.css HTTP/1.1" 404 1763
This is the error I am facing on running the server and reloading the localhost.
the part where I have added my static file is as follows
<link href="{% static 'css/base.css' %}" rel="stylesheet">
where i have defined static in settings.py is as follows:
STATIC_URL = '/static/'
STATICFILES_DIRS = [os.path.join(BASE_DIR, 'static')]
STATIC_ROOT = os.path.join(BASE_DIR, 'static_saved')
my project structure is linked here
what am i doing wrong?

Trying to connect through the web socket and it always get the HTTP/1.1 405 Method Not Allowed error

I'm trying to make an HTTPS proxy server, but I cant make a connection to any server.
Edit: this is the part of the code that, after a client connect to my server, I get the message and try to send to the web. This is the message of a FireFox client trying to connect to Google:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock = socket.create_connection((google.com, 443))
ssl_sock = ssl.wrap_socket(sock)
fullData=b''
ssl_sock.send(b'CONNECT www.google.com:443 HTTP/1.1\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:67.0) Gecko/20100101 Firefox/67.0\r\nProxy-Connection: keep-alive\r\nConnection: keep-alive\r\nHost: www.google.com:443\r\n\r\n')
while 1:
# receive data from web server
data = ssl_sock.recv(4026)
print(data)
if (len(data) > 0):
fullData+=data
else:
break
clientSock.send(fullData)
Google should got me a ok message but its getting me an error
HTTP/1.1 405 Method Not Allowed
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Content-Length: 1592
Date: Fri, 24 May 2019 05:28:17 GMT
Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
Connection: close
<!DOCTYPE html>
<html lang=en>
<meta charset=utf-8>
<meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">
<title>Error 405 (Method Not Allowed)!!1</title>
<style>
*{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30px 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}#media screen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/branding/googlelogo/1x/googlelogo_color_150x54dp.png) no-repeat;margin-left:-5px}#media only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) 0}}#media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/branding/googlelogo/2x/googlelogo_color_150x54dp.png) no-repeat;-webkit-background-size:100% 100%}}#logo{display:inline-block;height:54px;width:150px}
</style>
<a href=//www.google.com/><span id=logo aria-label=Google></span></a>
<p><b>405.</b> <ins>That\xe2\x80\x99s an error.</ins>
<p>The request method <code>CONNECT</code> is inappropriate for the URL <code>/</code>. <ins>That\xe2\x80\x99s all we know.</ins>

salt-api (cherrypy) returns HTTP/1.1 500 Internal Server Error when trying to log in

salt-api returns HTTP/1.1 500 Internal Server Error when trying to log in:
curl -si localhost:8000/login -H "Accept: application/json" -d username='<notshown>' -d password='<notshown>' -d eauth='pam'
HTTP/1.1 500 Internal Server Error
Content-Length: 1607
Access-Control-Expose-Headers: GET, POST
Vary: Accept-Encoding
Server: CherryPy/3.2.2
Allow: GET, HEAD, POST
Access-Control-Allow-Credentials: true
Date: Thu, 14 Apr 2016 13:23:49 GMT
Access-Control-Allow-Origin: *
X-Auth-Token: 6a529d945c6654848c531337c1a1193f8635b482
Content-Type: text/html;charset=utf-8
Set-Cookie: session_id=6a529d945c6654848c531337c1a1193f8635b482; expires=Thu, 14 Apr 2016 23:23:49 GMT; Path=/
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>
<title>500 Internal Server Error</title>
<style type="text/css">
#powered_by {
margin-top: 20px;
border-top: 2px solid black;
font-style: italic;
}
#traceback {
color: red;
}
</style>
</head>
<body>
<h2>500 Internal Server Error</h2>
<p>Configuration for external_auth could not be read.</p>
<pre id="traceback">Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cherrypy/_cprequest.py", line 656, in respond
response.body = self.handler()
File "/usr/lib/python2.6/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/salt/netapi/rest_cherrypy/app.py", line 506, in hypermedia_handler
ret = cherrypy.serving.request._hypermedia_inner_handler(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
return self.callable(*self.args, **self.kwargs)
File "/usr/lib/python2.6/site-packages/salt/netapi/rest_cherrypy/app.py", line 1525, in POST
'Configuration for external_auth could not be read.')
HTTPError: (500, 'Configuration for external_auth could not be read.')
</pre>
<div id="powered_by">
<span>Powered by CherryPy 3.2.2</span>
</div>
</body>
</html>
When you enter the wrong credentials, the error is different so it definitely knows that the credentials entered before are correct:
curl -si localhost:8000/login -H "Accept: application/json" -d username='<notshown>' -d password='<notshown>' -d eauth='pam'
HTTP/1.1 401 Unauthorized
Content-Length: 1586
Access-Control-Expose-Headers: GET, POST
Vary: Accept-Encoding
Server: CherryPy/3.2.2
Allow: GET, HEAD, POST
Access-Control-Allow-Credentials: true
Date: Thu, 14 Apr 2016 13:23:34 GMT
Access-Control-Allow-Origin: *
Content-Type: text/html;charset=utf-8
Set-Cookie: session_id=b71f4afcc8f1caf3e0f5f33d0542c77bc1b9875a; expires=Thu, 14 Apr 2016 23:23:34 GMT; Path=/
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>
<title>401 Unauthorized</title>
<style type="text/css">
#powered_by {
margin-top: 20px;
border-top: 2px solid black;
font-style: italic;
}
#traceback {
color: red;
}
</style>
</head>
<body>
<h2>401 Unauthorized</h2>
<p>Could not authenticate using provided credentials</p>
<pre id="traceback">Traceback (most recent call last):
File "/usr/lib/python2.6/site-packages/cherrypy/_cprequest.py", line 656, in respond
response.body = self.handler()
File "/usr/lib/python2.6/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
self.body = self.oldhandler(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/salt/netapi/rest_cherrypy/app.py", line 506, in hypermedia_handler
ret = cherrypy.serving.request._hypermedia_inner_handler(*args, **kwargs)
File "/usr/lib/python2.6/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
return self.callable(*self.args, **self.kwargs)
File "/usr/lib/python2.6/site-packages/salt/netapi/rest_cherrypy/app.py", line 1497, in POST
'Could not authenticate using provided credentials')
HTTPError: (401, 'Could not authenticate using provided credentials')
</pre>
<div id="powered_by">
<span>Powered by CherryPy 3.2.2</span>
</div>
</body>
</html>
/etc/salt/master:
rest_cherrypy:
port: 8000
host: 0.0.0.0
disable_ssl: true
debug: true
.....
external_auth:
pam:
<notshownhere>:
-.*
- '#runner'
- '#wheel'
versions used:
salt-api-2015.8.8-2.el6.noarch
salt-master-2015.8.8-2.el6.noarch
salt-minion-2015.8.8-2.el6.noarch
salt-2015.8.8-2.el6.noarch
Any idea why this is happening?
I had a typo in my master configuration:
I had:
external_auth:
pam:
<notshownhere>:
-.*
- '#runner'
- '#wheel'
Shoud be:
external_auth:
pam:
<notshownhere>:
- .*
- '#runner'
- '#wheel'

Resources