Python3 + selenium + PhantomJS UnicodeDecodeError - python-3.x

ENV
PhantomJS2.1.1;Windows10;selenium-3.8.0;python3
CODE
browser = webdriver.PhantomJS() //throw a UnicodeDecodeError
Problem
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbd in position
2: invalid start byte
Details
Traceback (most recent call last):
File "d:/devtools/phantomjs-2.1.1-windows/bin/IndustryLeaderSpider.py",
line 20, in
browser = webdriver.PhantomJS('D:/phantomjs.exe')
File "D:\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\phantomjs\webdriver.py",
line 58, in init
desired_capabilities=desired_capabilities)
File "D:\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 154, in init
self.start_session(desired_capabilities, browser_profile)
File "D:\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 243, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "D:\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py",
line 309, in execute
response = self.command_executor.execute(driver_command, params)
File "D:\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\remote_connection.py",
line 460, in execute
return self._request(command_info[0], url, body=data)
File "D:\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\remote_connection.py",
line 522, in _request
resp = opener.open(request, timeout=self._timeout)
File "D:\Continuum\Anaconda3\lib\urllib\request.py", line 526, in open
response = self._open(req, data)
File "D:\Continuum\Anaconda3\lib\urllib\request.py", line 544, in _open
'_open', req)
File "D:\Continuum\Anaconda3\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "D:\Continuum\Anaconda3\lib\urllib\request.py", line 806, in
meth(r, proxy, type))
File "D:\Continuum\Anaconda3\lib\urllib\request.py", line 814, in proxy_open
if req.host and proxy_bypass(req.host):
File "D:\Continuum\Anaconda3\lib\urllib\request.py", line 2739, in proxy_bypass
return proxy_bypass_registry(host)
File "D:\Continuum\Anaconda3\lib\urllib\request.py", line 2706, in proxy_bypass_registry
fqdn = socket.getfqdn(rawHost)
File "D:\Continuum\Anaconda3\lib\socket.py", line 673, in getfqdn
hostname, aliases, ipaddrs = gethostbyaddr(name)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xbd in position 2: invalid start byte
Do someone have this problem? How to fix it?

Your main error seems to be :
File "D:\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 460, in execute return self._request(command_info[0], url, body=data) File "D:\Continuum\Anaconda3\lib\site-packages\selenium\webdriver\remote\remote_connection.py", line 522, in _request resp = opener.open(request, timeout=self._timeout) File "D:\Continuum\Anaconda3\lib\urllib\request.py", line 526, in open response = self._open(req, data)
Try the following steps :
Run CCleaner tool to wipe off all the OS chores.
Take a System Reboot.
While you initiate PhantomJS() provide the absolute path of the phantomjs binary as follows :
driver = webdriver.PhantomJS(executable_path=r'C:\path\to\phantomjs.exe')
Ensure that there is no unnecessary imports in your script. Example : urllib.request
Execute your Test.

Related

zipfile.BadZipFile: Bad magic number for central directory

I am trying to read an excel file in the backened (flask) sent from the frontend (angularjs). However, I am getting the error
File "/var/task/pandas/util/_decorators.py", line 296, in wrapper
return func(*args, **kwargs)
File "/var/task/pandas/io/excel/_base.py", line 304, in read_excel
io = ExcelFile(io, engine=engine)
File "/var/task/pandas/io/excel/_base.py", line 867, in __init__
self._reader = self._engines[engine](self._io)
File "/var/task/pandas/io/excel/_openpyxl.py", line 480, in __init__
super().__init__(filepath_or_buffer)
File "/var/task/pandas/io/excel/_base.py", line 351, in __init__
self.book = self.load_workbook(filepath_or_buffer)
File "/var/task/pandas/io/excel/_openpyxl.py", line 491, in load_workbook
return load_workbook(
File "/mnt/accesspoint/openpyxl/reader/excel.py", line 315, in load_workbook
reader = ExcelReader(filename, read_only, keep_vba,
File "/mnt/accesspoint/openpyxl/reader/excel.py", line 124, in __init__
self.archive = _validate_archive(fn)
File "/mnt/accesspoint/openpyxl/reader/excel.py", line 96, in _validate_archive
archive = ZipFile(filename, 'r')
File "/var/lang/lib/python3.8/zipfile.py", line 1269, in __init__
self._RealGetContents()
File "/var/lang/lib/python3.8/zipfile.py", line 1364, in _RealGetContents
raise BadZipFile("Bad magic number for central directory")
zipfile.BadZipFile: Bad magic number for central directory
What is the problem that's causing this?
Angularjs html
<nz-card nzTitle="Upload File" bordered={false} style="flex: 1 1 auto;">
<nz-upload
[nzCustomRequest]="handleUpload"
(nzChange)="handleChange($event)"
>
<p class="ant-upload-drag-icon"><i nz-icon nzType="inbox"></i></p>
<p class="ant-upload-text">
Click or drag CSV/Excel a file to this area to upload
</p>
</nz-upload>
</nz-card>
Angularjs component
handleUpload = (item: any) => {
const formData = new FormData();
formData.append(item.name, item.file);
const headers = new HttpHeaders();
headers.set('Content-Type', 'multipart/form-data');
var upload = this.httpClient.post(this.SERVER_URL, formData,{headers: headers}).subscribe(
(res) => {
console.log("success");
}
Flask python
def upload():
file = request.files['file']
print("request", file)
df = pd.read_excel(file, index_col=0, engine='openpyxl')
print(df)
Logs (before the error message)
request <FileStorage: 'sample_excel.xlsx' ('application/vnd.openxmlformats-officedocument.spreadsheetml.sheet')>
I have also tried reading it using with
with open(file, 'rb') as f
But with that I get an error stating TypeError: expected str, bytes or os.PathLike object, not FileStorage
Any help would be greatly appreciated!!

Unable to read a url using fsspec https filesystem implementation directly

I am using fsspec package to implement a function to read an https file.
_hostname = socket.gethostname()
proxy_auth = aiohttp.BasicAuth(_hostname, pwd)
of = fsspec.filesystem("https", client_kwargs={"trust_env":True, "auth":proxy_auth})
http_urls = ["https://stackoverflow.com/"]
print(of.cat(http_urls))
The above code is not fetching the content of the url. The stack trace for the above code is as follows:
File "C:\Anaconda3\envs\mvision\lib\site-packages\fsspec\asyn.py", line 91, in wrapper
return sync(self.loop, func, *args, **kwargs)
File "C:\Anaconda3\envs\mvision\lib\site-packages\fsspec\asyn.py", line 71, in sync
raise return_result
File "C:\Anaconda3\envs\mvision\lib\site-packages\fsspec\asyn.py", line 25, in _runner
result[0] = await coro
File "C:\Anaconda3\envs\mvision\lib\site-packages\fsspec\asyn.py", line 347, in _cat
raise ex
File "C:\Anaconda3\envs\mvision\lib\site-packages\fsspec\implementations\http.py", line 230, in _cat_file
async with session.get(url, **kw) as r:
File "C:\Anaconda3\envs\mvision\lib\site-packages\aiohttp\client.py", line 1117, in __aenter__
self._resp = await self._coro
File "C:\Anaconda3\envs\mvision\lib\site-packages\aiohttp\client.py", line 521, in _request
req, traces=traces, timeout=real_timeout
File "C:\Anaconda3\envs\mvision\lib\site-packages\aiohttp\connector.py", line 535, in connect
proto = await self._create_connection(req, traces, timeout)
File "C:\Anaconda3\envs\mvision\lib\site-packages\aiohttp\connector.py", line 892, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "C:\Anaconda3\envs\mvision\lib\site-packages\aiohttp\connector.py", line 1011, in _create_direct_connection
raise ClientConnectorError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorError: Cannot connect to host stackoverflow.com:443 ssl:default [getaddrinfo failed]
But if we wrap it with a cache, I am able to get the content.
_hostname = socket.gethostname()
proxy_auth = aiohttp.BasicAuth(_hostname, pwd)
of = fsspec.filesystem("simplecache", target_protocol="https", \
target_options={"client_kwargs":{"trust_env":True, "auth":proxy_auth}},
cache_storage="/tmp/files")
http_urls = ["https://stackoverflow.com/"]
print(of.cat(http_urls))
Why is the first code section not able to fetch the content? Am I doing anything wrong ?

Telegram bot aiogram erros

mistakes
mistakes 2
photo
https://github.com/mahenzon/aiogram-lessons/tree/master/lesson-02
I can't understand why my code isn't working. I took code from here and only change file config.py, change token and MY_ID.
executor.py [ LINE:362 ]# INFO [2021-08-12 14:43:46,938] Bot: lap156 [#lab156_bot]
dispatcher.py [ LINE:360 ]# INFO [2021-08-12 14:43:46,938] Start polling.
base_events.py [ LINE:1738 ]# ERROR [2021-08-12 14:47:59,585] Task exception was never retrieved
future: <Task finished name='Task-24' coro=<Dispatcher._process_polling_updates() done, defined at C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py:409> exception=WrongFileIdentifier('Wrong file identifier/http url specified')>
Traceback (most recent call last):
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 417, in _process_polling_updates
for responses in itertools.chain.from_iterable(await self.process_updates(updates, fast)):
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 238, in process_updates
return await asyncio.gather(*tasks)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\handler.py", line 116, in notify
response = await handler_obj.handler(*args, **partial_data)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 259, in process_update
return await self.message_handlers.notify(update.message)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\handler.py", line 116, in notify
response = await handler_obj.handler(*args, **partial_data)
File "C:\Users\Zver\aiogram-lessons\lesson-02\bot.py", line 55, in process_photo_command
await bot.send_photo(message.from_user.id, CAT_BIG_EYES,
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\bot.py", line 482, in send_photo
result = await self.request(api.Methods.SEND_PHOTO, payload, files)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\base.py", line 208, in request
return await api.make_request(self.session, self.server, self.__token, method, data, files,
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\api.py", line 140, in make_request
return check_result(method, response.content_type, response.status, await response.text())
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\api.py", line 115, in check_result
exceptions.BadRequest.detect(description)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\utils\exceptions.py", line 140, in detect
raise err(cls.text or description)
aiogram.utils.exceptions.WrongFileIdentifier: Wrong file identifier/http url specified
base_events.py [ LINE:1738 ]# ERROR [2021-08-12 14:48:03,624] Task exception was never retrieved
future: <Task finished name='Task-32' coro=<Dispatcher._process_polling_updates() done, defined at C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py:409> exception=WrongFileIdentifier('Wrong file identifier/http url specified')>
Traceback (most recent call last):
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 417, in _process_polling_updates
for responses in itertools.chain.from_iterable(await self.process_updates(updates, fast)):
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 238, in process_updates
return await asyncio.gather(*tasks)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\handler.py", line 116, in notify
response = await handler_obj.handler(*args, **partial_data)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 259, in process_update
return await self.message_handlers.notify(update.message)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\handler.py", line 116, in notify
response = await handler_obj.handler(*args, **partial_data)
File "C:\Users\Zver\aiogram-lessons\lesson-02\bot.py", line 73, in process_note_command
await bot.send_video_note(message.from_user.id, VIDEO_NOTE)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\bot.py", line 963, in send_video_note
result = await self.request(api.Methods.SEND_VIDEO_NOTE, payload, files)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\base.py", line 208, in request
return await api.make_request(self.session, self.server, self.__token, method, data, files,
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\api.py", line 140, in make_request
return check_result(method, response.content_type, response.status, await response.text())
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\api.py", line 115, in check_result
exceptions.BadRequest.detect(description)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\utils\exceptions.py", line 140, in detect
raise err(cls.text or description)
aiogram.utils.exceptions.WrongFileIdentifier: Wrong file identifier/http url specified
base_events.py [ LINE:1738 ]# ERROR [2021-08-12 14:48:06,943] Task exception was never retrieved
future: <Task finished name='Task-37' coro=<Dispatcher._process_polling_updates() done, defined at C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py:409> exception=WrongFileIdentifier('Wrong file identifier/http url specified')>
Traceback (most recent call last):
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 417, in _process_polling_updates
for responses in itertools.chain.from_iterable(await self.process_updates(updates, fast)):
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 238, in process_updates
return await asyncio.gather(*tasks)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\handler.py", line 116, in notify
response = await handler_obj.handler(*args, **partial_data)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 259, in process_update
return await self.message_handlers.notify(update.message)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\handler.py", line 116, in notify
response = await handler_obj.handler(*args, **partial_data)
File "C:\Users\Zver\aiogram-lessons\lesson-02\bot.py", line 55, in process_photo_command
await bot.send_photo(message.from_user.id, CAT_BIG_EYES,
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\bot.py", line 482, in send_photo
result = await self.request(api.Methods.SEND_PHOTO, payload, files)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\base.py", line 208, in request
return await api.make_request(self.session, self.server, self.__token, method, data, files,
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\api.py", line 140, in make_request
return check_result(method, response.content_type, response.status, await response.text())
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\api.py", line 115, in check_result
exceptions.BadRequest.detect(description)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\utils\exceptions.py", line 140, in detect
raise err(cls.text or description)
aiogram.utils.exceptions.WrongFileIdentifier: Wrong file identifier/http url specified
base_events.py [ LINE:1738 ]# ERROR [2021-08-12 14:48:15,298] Task exception was never retrieved
future: <Task finished name='Task-41' coro=<Dispatcher._process_polling_updates() done, defined at C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py:409> exception=WrongFileIdentifier('Wrong file identifier/http url specified')>
Traceback (most recent call last):
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 417, in _process_polling_updates
for responses in itertools.chain.from_iterable(await self.process_updates(updates, fast)):
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 238, in process_updates
return await asyncio.gather(*tasks)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\handler.py", line 116, in notify
response = await handler_obj.handler(*args, **partial_data)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\dispatcher.py", line 259, in process_update
return await self.message_handlers.notify(update.message)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\dispatcher\handler.py", line 116, in notify
response = await handler_obj.handler(*args, **partial_data)
File "C:\Users\Zver\aiogram-lessons\lesson-02\bot.py", line 73, in process_note_command
await bot.send_video_note(message.from_user.id, VIDEO_NOTE)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\bot.py", line 963, in send_video_note
result = await self.request(api.Methods.SEND_VIDEO_NOTE, payload, files)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\base.py", line 208, in request
return await api.make_request(self.session, self.server, self.__token, method, data, files,
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\api.py", line 140, in make_request
return check_result(method, response.content_type, response.status, await response.text())
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\bot\api.py", line 115, in check_result
exceptions.BadRequest.detect(description)
File "C:\Users\Zver\AppData\Local\Programs\Python\Python39\lib\site-packages\aiogram\utils\exceptions.py", line 140, in detect
raise err(cls.text or description)
aiogram.utils.exceptions.WrongFileIdentifier: Wrong file identifier/http url specified
I guess the error it seems that one running terminal of the bot is already running when you run your code. So Terminate all running terminals/processes and try again.

HTTP Error 502: Parent proxy unreachable

I have done proxy settings using CNTLM and also have a script(google.py) to check proxy setting are working or not
import urllib.request
proxy_support = urllib.request.ProxyHandler({"http":"http://localhost:3128"})
opener = urllib.request.build_opener(proxy_support)
urllib.request.install_opener(opener)
html = urllib.request.urlopen("http://www.google.com").read()
print(html)
when i run the script I'm getting following errors
C:\Users\asj5cob\Desktop>py -3.4 google.py
Traceback (most recent call last):
File "google.py", line 5, in <module>
html = urllib.request.urlopen("http://www.google.com").read()
File "C:\Python34\lib\urllib\request.py", line 161, in urlopen
return opener.open(url, data, timeout)
File "C:\Python34\lib\urllib\request.py", line 470, in open
response = meth(req, response)
File "C:\Python34\lib\urllib\request.py", line 580, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python34\lib\urllib\request.py", line 508, in error
return self._call_chain(*args)
File "C:\Python34\lib\urllib\request.py", line 442, in _call_chain
result = func(*args)
File "C:\Python34\lib\urllib\request.py", line 588, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Parent proxy unreacheable
Did you start CNTLM by executing in your administrator cmd window?
net start cntlm

Authorization error from Python twitter tools

Trying to use Python Twitter Tools to search for the tweeets containing a hashtag. (On a raspberry Pi with python3).
from twitter import *
token = "token"
token_key = "token_key"
con_secret = "con_secret"
con_secret_key = "con_secret_key"
t = Twitter(
auth=OAuth(token, token_key, con_secret, con_secret_key))
print(t.search.tweets(q="#test"))
But I always get a Authorization error.
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/twitter/api.py", line 319, in _handle_response
handle = urllib_request.urlopen(req, **kwargs)
File "/usr/lib/python3.4/urllib/request.py", line 153, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.4/urllib/request.py", line 461, in open
response = meth(req, response)
File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib/python3.4/urllib/request.py", line 499, in error
return self._call_chain(*args)
File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
result = func(*args)
File "/usr/lib/python3.4/urllib/request.py", line 579, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 401: Authorization Required
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "twitter-test.py", line 10, in <module>
print(t.search.tweets(q="#test"))
File "/usr/local/lib/python3.4/dist-packages/twitter/api.py", line 312, in __call__
return self._handle_response(req, uri, arg_data, _timeout)
File "/usr/local/lib/python3.4/dist-packages/twitter/api.py", line 345, in _handle_response
raise TwitterHTTPError(e, uri, self.format, arg_data)
twitter.api.TwitterHTTPError: Twitter sent status 401 for URL: 1.1/search/tweets.json using parameters: (oauth_consumer_key=**key**&oauth_nonce=**nonce**&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1438333973&oauth_token=**token**&oauth_version=1.0&q=%23test&oauth_signature=**signature**)
details: {'errors': [{'code': 32, 'message': 'Could not authenticate you.'}]}
I have tried checking my time (and changing the timezone).
I have tried putting in a callback URL into the app settings and regenerating the keys.
Any help appreciated
Thanks
Couldn't fix this so moved to tweepy library which works a treat!

Resources