The program should allow to run several https get requests with one aiohttp.ClientSession as the documentation suggests. It is intended to run a telegram bot.
I was not able to catch the exception with try ... except. Therefore the program hangs when exiting. During extended sessions the error is printed in the command windows (but not in the error log).
SSL error in data received
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000016A581E4400>
transport: <_SelectorSocketTransport fd=644 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
File "C:\Users\annet\Anaconda3\lib\asyncio\sslproto.py", line 526, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "C:\Users\annet\Anaconda3\lib\asyncio\sslproto.py", line 207, in feed_ssldata
self._sslobj.unwrap()
File "C:\Users\annet\Anaconda3\lib\ssl.py", line 767, in unwrap
return self._sslobj.shutdown()
ssl.SSLError: [SSL: KRB5_S_INIT] application data after close notify (_ssl.c:2592)
^C
As the error information is very unspecific I could not really isolate the source and have a short code to reproduce the error.
A sample code is on github under https://github.com/fhag/telegram2.git
In order to run the code you will need an API token from telegram of your own bot.
This error showed up the first time when I upgraded to python 3.7.1.
Python is running on Windows 10.
Related
I wish I could use python to execute the Excel macro, so I tried to use the package xlwings to implement it.
The OS of my laptop is macOS Catalina (ver.: 10.15.7), my compiler is PyCharm (ver.: 2021.2.3), my Python version is 3.8.8, I used Anaconda (Ver.: 22.11.1) as my interpreter, my excel version is 16.66.1 (Microsoft Excel for Mac).
I faced the error "Command Error -1743: The User has declined permission" when I tried to use this package originally, and I solved this issue by installing an old compiler & using the old version of the compiler to run my code. My privacy setting for automation in the app Setting was shown below: (This is NOT the question I want to ask, but I'm not sure if it is also related to the issue I faced, so I still attached it here. I had uninstalled the old version of my compiler already.)
I wish I could implement an existing macro (called Hi) in an existing Excel file (called [StakeOverflow]HelloWorld.xlsm) through Python (represented as MY_PYTHON_FILE.py below), like the snapshot below:
My Excel file and my python code were stored on OneDrive. My macro code was shown below:
Sub Hi()
MsgBox "Good morning!"
End Sub
My python code was shown below:
import xlwings as xw
import time
wb = xw.Book('/Users/<MY NAME>/OneDrive/MY PATH DETAILS/[StakeOverflow]HelloWorld.xlsm')
time.sleep(10)
app = wb.app
macro_vba = app.macro("Hi")
macro_vba()
The code looks really simple, but I still faced the error. My Excel was not opened automatically, and I even could not open the Excel file manually thereafter. The error was shown below:
/Users/.../.conda/envs/Program/bin/python "/Users/.../OneDrive/.../MY_PYTHON_FILE.py"
Traceback (most recent call last):
File "/Users/.../.conda/envs/Program/lib/python3.9/site-packages/xlwings/main.py", line 4914, in open
impl= self.impl(name)
File "/Users/.../.conda/envs/Program/lib/python3.9/site-packages/xlwings/_xlmac.py", line 366, in __call__
raise KeyError(name_or_index)
KeyError: '[stakeoverflow] helloworld.xlsm'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/.../.conda/envs/Program/lib/python3.9/site-packages/aeosa/appscript/reference.py", line 482, in __call___
return self.AS_appdata.target().event (self._code, params, atts, codecs=self.AS_appdata).send(timeout, sendflags)
File "/Users/.../.conda/envs/Program/lib/python3.9/site-packages/aeosa/aem/aemsend.py", line 92, in send
raise EventError(errornum, errormsg, eventresult)
aem.aemsend.EventError: Command failed: Parameter error. (-50)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/.../OneDrive/.../MY_PYTHON_FILE.py", line 18, in <module>
wb = xw.Book('/Users/.../OneDrive/.../[stakeoverflow] helloworld.xlsm')
File "/Users/.../.conda/envs/Program/lib/python3.9/site-packages/xlwings/main.py", line 876, in __init__
impl= app.books.open(
File "/Users/.../.conda/envs/Program/lib/python3.9/site-packages/xlwings/main.py", line 4921, in open
impl = self.impl.open(
File "/Users/.../.conda/envs/Program/lib/python3.9/site-packages/xlwings/ xlmac.py", line 420, in open
self.app.xl.open_workbook(
File "/Users/.../.conda/envs/Program/lib/python3.9/site-packages/aeosa/appscript/reference.py", line 518, in __call__
raise CommandError(self, (args, kargs), e, self.AS_appdata) from e
appscript.reference.CommandError: Command failed:
OSERROR: -50
MESSAGE: Parameter error.
COMMAND: app(pid=1647).open_workbook (workbook_file_name='/users/.../onedrive/.../[stakeoverflow] helloworld.xlsm', update_links=k.do_not_update_links, read_only=None, format=None, password=None, write_reserved_password=None, ignore_read_only_recommended=None, origin=None, delimiter=None, editable=None, notify=None, converter=None, add_to_mru=None, timeout=-1)
Process finished with exit code 1
I tried to use terminal to run my python code, but I could not solve the problem, either.
I tried to open the Excel file manually thereafter, the error was shown below:
Excel cannot open the file ’[StakeOverflow]HelloWorld.xlsm’
because the file format or file extension is not valid. Verify
that the file has not been corrupted and that the file
extension matches the format of the file.
I tried to Google this error, but few solutions was found. It seems that it is related to the issue of external storage location. I tried to create another Excel file with the same name & macro code on my desktop and try again, and the error would disappear. (However, our company stored the files on OneDrive, so I wish I could utilise the file online.)
Just wondering if anyone here faced this situation before?
I found the answer by myself today. The issue is related to the naming issue rather than the permission issue.
As we could see that the error is KeyError: '[stakeoverflow] helloworld.xlsm', which implies that the problem is here. (Maybe because the system could not find the Excel file with this name.)
I tried to change the name from [stakeoverflow] helloworld.xlsm to helloworld.xlsm, and the error was gone. It seems that when xlwings want to open an Excel file, it would check the validity of the file name and change the name into smaller cases. If the file name contains special characters which are not allowed (e.g., "[]"), then the error would occur.
Notice that I could store the Excel file with these special characters in our laptop & OneDrive, but xlwings did not accept them.
Hope it is helpful to those who face this issue when using xlwings!
I am making a python-based mac app that uses discord.py to do stuff with discord. As I knew from previous experience making discord bots, running discord bots requires that you run Install Certificates.command in your version of python. However, if another users uses this app, I don't want to require them to install python. I took a snippet of code from Install Certificates.command, thinking it would put the certificate in the right place on a user's computer. However, a tester got this error running the app on their computer:
Traceback (most recent call last):
File "Interface.py", line 136, in <module>
File "installCerts.py", line 25, in installCerts
FileNotFoundError: [Errno 2] No such file or directory: '/Library/Frameworks/Python.framework/Versions/3.8/etc/openssl'
[2514] Failed to execute script 'Interface' due to unhandled exception: [Errno 2] No such file or directory: '/Library/Frameworks/Python.framework/Versions/3.8/etc/openssl'
[2514] Traceback:
Traceback (most recent call last):
File "Interface.py", line 136, in <module>
File "installCerts.py", line 25, in installCerts
FileNotFoundError: [Errno 2] No such file or directory: '/Library/Frameworks/Python.framework/Versions/3.8/etc/openssl'
It's pretty clear what this error is saying: They don't have python (3.8) installed, so it can't put the ssl certificates anywhere (this is because the app is running in a python 3.8 environment).
By the way, the path mentioned in the error is the directory name of the path given by ssl.get_default_verify_paths().openssl_cafile.
I'm not super well-versed in the finer points of web connections and stuff like that, so I don't know the exact role of these certificates. Here's my question:
Is it possible to get this to work without the user installing python on their computer?
I.e. Can I add the ssl certificates to the app's local python version (as far as I can tell, in my app, python is simply a large bundled exec file)? Is there somewhere deep in the file system where I can put the certificates to let the connection to discord happen? . Pretty much any solution would be appreciated.
Additional Info:
My Code to Install Certificates:
STAT_0o775 = (stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
| stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP
| stat.S_IROTH | stat.S_IXOTH)
openssl_dir, openssl_cafile = os.path.split(
ssl.get_default_verify_paths().openssl_cafile)
os.chdir(openssl_dir) #Error happens here
relpath_to_certifi_cafile = os.path.relpath(certifi.where())
print(" -- removing any existing file or link")
try:
os.remove(openssl_cafile)
except FileNotFoundError:
pass
print(" -- creating symlink to certifi certificate bundle")
os.symlink(relpath_to_certifi_cafile, openssl_cafile)
print(" -- setting permissions")
os.chmod(openssl_cafile, STAT_0o775)
print(" -- update complete")
The error that discord.py throws when the user doesn't have correct certificates installed:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/aiohttp/connector.py", line 969, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs) # type: ignore # noqa
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py", line 1050, in create_connection
transport, protocol = await self._create_connection_transport(
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py", line 1080, in _create_connection_transport
await waiter
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/sslproto.py", line 529, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/sslproto.py", line 189, in feed_ssldata
self._sslobj.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/ssl.py", line 944, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1125)
If you need more info, let me know.
Ok. This was very tough, but I got to an answer after much research. ssl in python is basically just a set of bindings for openSSL. When you do import ssl, it builds an openSSL environment (I don't think I'm using the exact right words here). As you could see, it was defaulting to the openSSL folder in Python because from python's perspective, that is where openSSL keeps its certs. Turns out, ssl.DefaultVerifyPaths objects have other attributes, namely cafile. This was how I made the path to the cert whatever I wanted. You see, when openSSL builds, it looks for an environment variable SSL_CERT_FILE. As long as I set that variable with os.environ before I imported ssl, it would work, because ssl would find the certificate. I simplified installCerts down to the following:
import os
import stat
import certifi
def installCerts():
os.environ['SSL_CERT_FILE'] = certifi.where()
import ssl
# ssl build needs to happen after enviro var is set
STAT_0o775 = (stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR
| stat.S_IRGRP | stat.S_IWGRP | stat.S_IXGRP
| stat.S_IROTH | stat.S_IXOTH)
cafile = ssl.get_default_verify_paths().cafile
os.chmod(cafile, STAT_0o775)
And it seems to work fine on other people's computers now without them needing to install python.
This question helped me:
How to change the 'cafile' argument in the ssl module in Python3?
I am trying to send http2 request using postman. However, when my server receives the request gives error:
handle: <Handle _SelectorSocketTransport._read_ready()>
Traceback (most recent call last):
File "/usr/lib64/python3.6/asyncio/events.py", line 145, in _run
self._callback(*self._args)
File "/usr/lib64/python3.6/asyncio/selector_events.py", line 721, in _read_ready
self._protocol.data_received(data)
File "/home/deesharm/jetconf/jetconf/jetconf/rest_server.py", line 76, in data_received
events = self.conn.receive_data(data)
File "/home/deesharm/jetconf/venv/lib/python3.6/site-packages/h2/connection.py", line 1448, in receive_data
.. versionchanged:: 2.0.0
File "/home/deesharm/jetconf/venv/lib/python3.6/site-packages/h2/frame_buffer.py", line 52, in add_data
raise ProtocolError("Invalid HTTP/2 preamble.")
h2.exceptions.ProtocolError: Invalid HTTP/2 preamble.
Currently, Postman doesn't support HTTP/2.
https://github.com/postmanlabs/postman-app-support/issues/2701
As of late 2022, Postman still does not support HTTP2.
A workaround is to click the "Code" icon (looks like </> ) to generate the cURL command, add the --http2 command line flag, then copy/paste it into the terminal:
Works well with any mac / linux terminal as well as WSL2 on Windows. You can also provide the --verbose flag to cURL to make sure HTTP2 is working as expected.
I created a GUI tool that you can use to send HTTP requests, with support for HTTP/2 and HTTP/3. It has full compatibility with existing Postman collections and environments.
https://github.com/alexandrehtrb/Pororoca
Both the server and my computer have geckodriver 0.26.0, Firefox 71, and Selenium 3.141.0.
My computer has MacOS Mojave with python 3.8 and the server is CentOS 7 with python 3.7. The code runs perfectly on my computer, but it returns errors on the server.
I don't remember how, but I have been getting different errors depending on if I add breakpoints to it, or if I run it in terminal or submit the job in SLURM.
On terminal:
File "Main.py", line 230, in <module>
main()
File "Main.py", line 179, in main
dfs=get_data(stations, inidate, findate)
File "Main.py", line 113, in get_data
list_files=return_list_day(date)
File "Main.py", line 66, in return_list_day
driver.get(webdir)
File "/home/user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "/home/user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Failed to decode response from marionette
or
Submitted on SLURM (and most often error):
Traceback (most recent call last):
File "Main.py", line 230, in <module>
main()
File "Main.py", line 179, in main
dfs=get_data(stations, inidate, findate)
File "Main.py", line 113, in get_data
list_files=return_list_day(date)
File "Main.py", line 66, in return_list_day
driver.get(webdir)
File "/home/user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 333, in get
self.execute(Command.GET, {'url': url})
File "/home/user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/home/user/.local/lib/python3.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: Timeout loading page after 300000ms
I can use Selenium without issues in another unrelated code that finishes perfectly fine. So I don't understand why Selenium is having a hard time here or where it comes from.
The code fails here:
## Commented parts are cuz I tried running with the MOZ_HEADLESS tag on terminal
## Didnt make a difference as far as I could tell.
#options = Options()
#options.headless = True
driver = webdriver.Firefox()#options=options)
driver.get(webdir) ##<-- HERE
## Added these because of other replies to this issue I found
driver.implicitly_wait(7)
time.sleep(3)
.
.
.
driver.close()
That bit of codes retrieves a list from the webpage and is ran inside a for-loop. The error is not in the retrieving part, is in driver.get(webdir). I can't share the website since it's literally like looking into the server of a partner institution. webdir is a directory and I am basically waiting till its contents are loaded so I can retrieve its file names.
I know you can't help much without a website, but do the errors I've shown give any indication as to what the problem might be?. Can the retrieving of a specific website behave differently based on OS? I have googled the errors, found questions here, read through them and applied them to see how it changed, and nothing did or I got a different error (either one of the two above).
I found this that states incompatibility between gecko and Mozilla, but since I can successfully run another (unrelated) code with the exact calling and usage of Selenium (only different URL given) then I don't think that's my issue.
Thanks for any help! Let me know what other information I could give that might help.
Edit:
It is not the same as the question that was linked, since I have given it sleep time and didn't change anything. It has 40 GB of ram allocated so its not dying out of too little memory. Which are the solutions shown in this question.
For anyone with this issue, the server updated Firefox to version 79.0 and now it works without any issues. Nothing else was changed as far as I was let known.
I assume the version change fixed it but I don't know exactly how. It's worth a try if anyone else was experiencing the same as I was with different errors depending on how it was run.
I'm trying to get the largefiles extension working on a mercurial server under Windows Server 2008 / IIS 7.5 with the hgweb.wsgi script.
When I clone a repo with largefiles locally (but using https://domain/, not a file system path) everything gets cloned fine, but when I try it on a different machine I get abort: remotestore: largefile XXXXX is missing
Here's the verbose output:
requesting all changes
adding changesets
adding manifests
adding file changes
added 1 changesets with 177 changes to 177 files
calling hook changegroup.lfiles: <function checkrequireslfiles at 0x0000000002E00358>
updating to branch default
resolving manifests
getting .hglf/path/to.file
...
177 files updated, 0 files merged, 0 files removed, 0 files unresolved
getting changed largefiles
getting path/to.file:c0c81df934cd72ca980dd156984fa15987e3881d
abort: remotestore: largefile c0c81df934cd72ca980dd156984fa15987e3881dis missing
Both machines have the extension working. I've tried disabling the firewall but that didn't help. Do I have to do anything to set up the extension besides adding it to mercurial.ini?
Edit: If I delete the files from the server's AppData\Local\largefiles\ directory, I get the same error when cloning on the server, unless I use a filesystem path to clone, in which case the files are added back to `AppData\Local\largefiles\'
Edit 2: Here's the debug output and traceback:
177 files updated, 0 files merged, 0 files removed, 0 files unresolved
getting changed largefiles
using http://domain
sending capabilities command
getting largefiles: 0/75 lfile (0.00%)
getting path/to.file:64f2c341fb3b1adc7caec0dc9c51a97e51ca6034
sending statlfile command
Traceback (most recent call last):
File "mercurial\dispatch.pyo", line 87, in _runcatch
File "mercurial\dispatch.pyo", line 685, in _dispatch
File "mercurial\dispatch.pyo", line 467, in runcommand
File "mercurial\dispatch.pyo", line 775, in _runcommand
File "mercurial\dispatch.pyo", line 746, in checkargs
File "mercurial\dispatch.pyo", line 682, in <lambda>
File "mercurial\util.pyo", line 463, in check
File "mercurial\commands.pyo", line 1167, in clone
File "mercurial\hg.pyo", line 400, in clone
File "mercurial\extensions.pyo", line 184, in wrap
File "hgext\largefiles\overrides.pyo", line 629, in hgupdate
File "hgext\largefiles\lfcommands.pyo", line 416, in updatelfiles
File "hgext\largefiles\lfcommands.pyo", line 398, in cachelfiles
File "hgext\largefiles\basestore.pyo", line 80, in get
File "hgext\largefiles\remotestore.pyo", line 56, in _getfile
Abort: remotestore: largefile 64f2c341fb3b1adc7caec0dc9c51a97e51ca6034 is missing
The _getfile function throws an exception because the statlfile command returns that the file wasn't found.
I've never used python myself, so I don't know what I'm doing while trying to debug this :D
AFAIK the statlfile command gets executed on the server so I can't debug it from my local machine. I've tried running python -m win32traceutil on the server, but it doesn't show anything. I also tried setting accesslog and errorlog in the server's mercurial config file, but it doesn't generate them.
I run hg through the hgweb.wsgi script, and I have no idea if/how I can get into the python debugger using that, but if I could get the debugger running on the server I could narrow down the problem...
Finally figured it out, the extension tries to write temporary files to %windir%\System32\config\systemprofile\AppData\Local, which was causing permission errors. The call was wrapped in a try-catch block that ended up returning the "file not found" error.
I'm just posting this for anyone else coming into the thread from a search.
There's currently an issue using the largefiles extension in the mercurial python module when hosted via IIS. See this post if you're encountering issues pushing large changesets (or large files) to IIS via TortoiseHg.
The problem ultimlately turns out to be a bug in SSL processing introduced Python 2.7.3 (probably explaining why there are so many unresolve posts of people looking for problems with Mercurial). Rolling back to Python 2.7.2 let me get a little further ahead (blocked at 30Mb pushes instead of 15Mb), but to properly solve the problem I had to install the IISCrypto utility to completely disable transfers over SSLv2.