After upgrading from python 3.8.0 to python 3.9.1, the tremc front-end of transmission bitTorrent client is throwing decodestrings is not an attribute of base64 error whenever i click on a torrent entry to check the details.
My system specs:
OS: Arch linux
kernel: 5.6.11-clear-linux
base64.encodestring() and base64.decodestring(), aliases deprecated since Python 3.1, have been removed.
use base64.encodebytes() and base64.decodebytes()
So i went to the site-packages directory and with ripgrep tried searching for the decodestring string.
rg decodestring
paramiko/py3compat.py
39: decodebytes = base64.decodestring
Upon examining the py3compat.py file,i found this block:
PY2 = sys.version_info[0] < 3
if PY2:
string_types = basestring # NOQA
text_type = unicode # NOQA
bytes_types = str
bytes = str
integer_types = (int, long) # NOQA
long = long # NOQA
input = raw_input # NOQA
decodebytes = base64.decodestring
encodebytes = base64.encodestring
So decodebytes have replaced(aliased) decodestring attribute of base64 for python version >= 3
This must be a new addendum because tremc was working fine in uptil version 3.8.*.
Opened tremc script, found the erring line (line 441), just replaced the attribute decodestring with decodebytes.A quick fix till the next update.
PS: Checked the github repository, and there's a pull request for it in waiting.
If you don't want to wait for the next release and also don't want to hack the way i did, you can get it freshly build from the repository, though that would be not much of a difference than my method
Related
The example is from Chapter 8 of Grokking Deep Reinforcement Learning written by Miguel Morales.
Please click here
The wrappers.Monitor is deprecated after the book is published. The code in question is as below:
env = wrappers.Monitor(
env, mdir, force=True,
mode=monitor_mode,
video_callable=lambda e_idx: record) if monitor_mode else env
I searched the internet and tried 2 methods but failed.
1- gnwrapper.Monitor
I pip install gym-notebook-wrapper and import gnwrapper, then rewrite the code
env = gnwrapper.Monitor(gym.make(env_name),directory="./")
A [WinError 2] The system cannot find the file specified error message is returned.
2- gym.wrappers.RecordVideo
I from gym.wrappers import RecordVideo then rewrite the code
env = RecordVideo(gym.make(env_name), "./")
A AttributeError: 'CartPoleEnv' object has no attribute 'videos' error message is returned.
Is there any way to solve the problem?
After some experiments, the best way to work around the problem is to downgrade the gym version to 0.20.0, in order to preserve the wrappers.Monitor function.
Moreover, the subprocess.Popen and subprocess.check_output does not work as the original author suggested, so I use moviepy (please pip install moviepy if you do not have this libarary and from moviepy.editor import VideoFileClip at the very beginning) to change MP4 to GIF files. The code in get_gif_html is amended as follows.
gif_path = basename + '.gif'
if not os.path.exists(gif_path):
videoClip = VideoFileClip(video_path)
videoClip.write_gif(gif_path, logger=None)
gif = io.open(gif_path, 'r+b').read()
The program now completely works.
Edit (24 Jul 2022):
The following solution is for gym version 0.25.0, as I hope the program works for later versions. Windows 10 and Juptyer Notebook is used for the demonstration.
(1) Maintain the moviepy improvement stated above
(2) import from gym.wrappers.record_video import RecordVideo and substitute
env = wrappers.Monitor(
env, mdir, force=True,
mode=monitor_mode,
video_callable=lambda e_idx: record) if monitor_mode else
with this
if monitor_mode:
env = RecordVideo(env, mdir, episode_trigger=lambda e_idx:record)
env.reset()
env.start_video_recorder()
else:
env = env
RecordVideo function takes (env, video_folder, episode_trigger) arguments, and video_callable and episode_trigger are the same thing.
(3) Loss of env.videos in get_gif_html function in new versions.
In the old setting, after the video is recorded, the MP4 and meta.json files are stored in the following address with a random file name
C:\Users\xxx\AppData\Local\Temp
https://i.stack.imgur.com/Za3Sd.jpg
This env.videos function returns a list of tuples, each tuples consisted of address path of MP4 and meta.json.
I tried to recreate the env.videos by
(a) declare the mdir as global variable
def get_make_env_fn(**kargs):
def make_env_fn(env_name, seed=None, render=None, record=False,
unwrapped=False, monitor_mode=None,
inner_wrappers=None, outer_wrappers=None):
global mdir
mdir = tempfile.mkdtemp()
(b) in the demo_progression and demo_last function, grouping all MP4 and meta_json files, zipping them and use list comprehension to make a new env_videos variable (as opposed to env.videos)
env.close()
mp4_files = ([(mdir + '\\' + file) for file in os.listdir(mdir) if file.endswith('.mp4')])
meta_json_files = ([(mdir + '\\' + file) for file in os.listdir(mdir) if file.endswith('.meta.json')])
env_videos = [(mp4, meta_json) for mp4, meta_json in zip(mp4_files, meta_json_files)]
data = get_gif_html(env_videos=env_videos,
These are all the changes.
Testing the simple example that should execute the external application and catch both inputs there.
# external_application.py
print('First_line:')
x = input()
print('Second_line:')
y = input()
print(f'Done: {x} and {y}')
And the runner:
# runner.py
import subprocess
import pexpect.fdpexpect
p = subprocess.Popen(("python", "external_application.py"), stdin=subprocess.PIPE, stdout=subprocess.PIPE)
session = pexpect.fdpexpect.fdspawn(p.stdout.fileno())
session.expect("First_line:")
p.stdin.write(b'1')
print('first_done')
session.expect("Second_line:")
p.stdin.write(b'2')
print('second_done')
p.stdin.close()
print(f'Result: {p.stdout.read()}')
I can see that output of the print('first_done') and then it locks on the second expect. If we remove the second input and a second expect then everything works correctly till the end.
Running on the windows, python 3.7.9, pexpect 4.8.0
Am I missing some timeout or a flush?
When I check the dump, session matches b'First_line:\r\n', but p.stdin.write doesn't move the pexpect cursor forward to look for "Second_line:", as what happens with send() or sendline().
May I suggest a simpler way, using PopenSpawn? The docs state that it "provides an interface like pexpect.spawn interface (sic) using subprocess.Popen":
NOTE - Tested in Windows 11, using Python 3.9.
import pexpect
from pexpect import popen_spawn
session = pexpect.popen_spawn.PopenSpawn("python external_application.py")
session.expect("First_line:")
session.sendline("1")
print("first_done")
session.expect("Second_line:")
session.sendline("2")
print("second_done")
print(session.read().decode("utf-8"))
Output:
first_done
second_done
Done: 1 and 2
Specs: Python version 3.7.3 (64-bit)
IDE Microsoft Code (latest edition as of 4/23)
Background I am having issues with executing my code with Configparser only on Microsoft code. I installed other IDE's pycharm etc and I do not have any issues. I prefer Microsoft Code interface / debugging so I would prefer to keep using it. I even went an extra step and installed MS-Code on another server and tried to exec my code with the exact same issue. On a side note if you execute the code in debug mode in ms-code it works without a key error.
This issue is really confusing to me and any help would be appreciated
Current structure of Configparser in code
config = configparser.ConfigParser()
config.read('api_config.ini')
#Reads in each of the api URLs to use from the config file
line_item_reporting = config['apis']['line_item_reporting']
line_item_meta = config['apis']['line_item_meta']
package_reporting = config['apis']['package_reporting']
package_meta = config['apis']['package_meta']
io_meta = config['apis']['io_meta']
io_reporting = config['apis']['io_reporting']
Error code
line_item_reporting = config['apis']['line_item_reporting']
File "C:\ProgramData\Anaconda3\lib\configparser.py", line 958, in getitem
raise KeyError(key)
KeyError: 'apis'
api_config.ini
[general]
meta_pull=50
num_loops=10
index_start=0
index_end=20
password=#######
lookback_window=3
[apis]
io_reporting = https://some_url
line_item_reporting = https://some_url
line_item_meta = https://some_url
package_reporting = https://some_url
package_meta = https://some_url
io_meta = https://some_url
I would like to query Windows using a file extension as a parameter (e.g. ".jpg") and be returned the path of whatever app windows has configured as the default application for this file type.
Ideally the solution would look something like this:
from stackoverflow import get_default_windows_app
default_app = get_default_windows_app(".jpg")
print(default_app)
"c:\path\to\default\application\application.exe"
I have been investigating the winreg builtin library which holds the registry infomation for windows but I'm having trouble understanding its structure and the documentation is quite complex.
I'm running Windows 10 and Python 3.6.
Does anyone have any ideas to help?
The registry isn't a simple well-structured database. The Windows
shell executor has some pretty complex logic to it. But for the simple cases, this should do the trick:
import shlex
import winreg
def get_default_windows_app(suffix):
class_root = winreg.QueryValue(winreg.HKEY_CLASSES_ROOT, suffix)
with winreg.OpenKey(winreg.HKEY_CLASSES_ROOT, r'{}\shell\open\command'.format(class_root)) as key:
command = winreg.QueryValueEx(key, '')[0]
return shlex.split(command)[0]
>>> get_default_windows_app('.pptx')
'C:\\Program Files\\Microsoft Office 15\\Root\\Office15\\POWERPNT.EXE'
Though some error handling should definitely be added too.
Added some improvements to the nice code by Hetzroni, in order to handle more cases:
import os
import shlex
import winreg
def get_default_windows_app(ext):
try: # UserChoice\ProgId lookup initial
with winreg.OpenKey(winreg.HKEY_CURRENT_USER, r'SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\FileExts\{}\UserChoice'.format(ext)) as key:
progid = winreg.QueryValueEx(key, 'ProgId')[0]
with winreg.OpenKey(winreg.HKEY_CURRENT_USER, r'SOFTWARE\Classes\{}\shell\open\command'.format(progid)) as key:
path = winreg.QueryValueEx(key, '')[0]
except: # UserChoice\ProgId not found
try:
class_root = winreg.QueryValue(winreg.HKEY_CLASSES_ROOT, ext)
if not class_root: # No reference from ext
class_root = ext # Try direct lookup from ext
with winreg.OpenKey(winreg.HKEY_CLASSES_ROOT, r'{}\shell\open\command'.format(class_root)) as key:
path = winreg.QueryValueEx(key, '')[0]
except: # Ext not found
path = None
# Path clean up, if any
if path: # Path found
path = os.path.expandvars(path) # Expand env vars, e.g. %SystemRoot% for ext .txt
path = shlex.split(path, posix=False)[0] # posix False for Windows operation
path = path.strip('"') # Strip quotes
# Return
return path
I am using Redhawk 1.9. I have created a Redhawk Device from the 1.8.4 IDE.
1.8.4 Redhawk Device (created with all default settings)
C++ implementation
Import existing project into 1.9 IDE
I read the Redhawk 1.9 Release notes on how to convert between 1.8.4 and 1.9 projects
*http://redhawksdr.github.io/Documentation/Release_Notes_1.9.0.pdf*
I import the default 1.8.4 device into the 1.9 IDE. I am able to run and build the 1.8.4 device in the 1.9 IDE. When I try to regenerate the code for 1.8.4 device, the IDE ask me if I want to upgrade to 1.9. The popup says "ConversionTestDevice uses deprecated code generators. Would you like to upgrade this project?". I decided to do an upgrade. I then get the following error message:
/usr/local/redhawk/core/bin/update_project returned with error code 1
Traceback (most recent call last):
File "/usr/local/redhawk/core/bin/update_project", line 222, in ?
if check_bulkio_input(compCpp):
File "/usr/local/redhawk/core/bin/update_project", line 105, in check_bulkio_input
for line in strip_comments(open(filename, 'r')):
File "/usr/local/redhawk/core/bin/update_project", line 86, in strip_comments
ch += safe_next(chars)
File "/usr/local/redhawk/core/bin/update_project", line 56, in safe_next
return next(item)
NameError: global name 'next' is not defined
I would appreciate suggestions on how to convert 1.8.4 device to 1.9 device.
Based on your error message "NameError: global name 'next' is not defined" and the contents of the 1.9.0 release of update_project python script, I am assuming that you are running a version of python less than 2.6. The next function is a python builtin that was introduced in Python 2.6 (http://docs.python.org/2/library/functions.html#next). This is a known bug in the upgrade script as it should be compatible with Python 2.4 as well as Python 2.6 (The default python installations in CentOS 5 and 6 respectively). To fix this, you may modify the update_project script located in $OSSIEHOME/bin/update_project and define the following function:
if not hasattr(__builtins__, 'next'):
# Python 2.4 does not have a free-standing next() function
def next(iterator, default):
"""
Backwards compatibility next() equivalent for Python 2.4.
"""
try:
return iterator.next()
except StopIteration:
return default
You should then remove the previously defined "safe_next" function.
Lastly, you need to replace the two calls to "safe_next" with a call to the newly implement next function and add a second argument of empty string ''
For clarity, a diff of update_project with these changes is below:
## -46,16 +46,16 ## Options:
_bulkio_re = re.compile('BULKIO_data[A-Za-z]+_In_i')
-def safe_next(item):
- """
- Returns the next value of the iterator, or an empty string if the end of
- iteration has been reached. Allows string processing to gracefully handle
- the end of a line without explicit catch statements.
- """
- try:
- return next(item)
- except StopIteration:
- return ''
+if not hasattr(__builtins__, 'next'):
+ # Python 2.4 does not have a free-standing next() function
+ def next(iterator, default):
+ """
+ Backwards compatibility next() equivalent for Python 2.4.
+ """
+ try:
+ return iterator.next()
+ except StopIteration:
+ return default
def strip_comments(source):
"""
## -75,7 +75,7 ## def strip_comments(source):
# Look for end comment token; if the end of the line is reached
# ch will be an empty string
while ch == '*':
- ch = safe_next(chars)
+ ch = next(chars, '')
if ch == '/':
inComment = False
break
## -83,7 +83,7 ## def strip_comments(source):
if ch == '/':
# Read the next character to see if it matches a comment token
# (if it does not, both characters will be added to the output)
- ch += safe_next(chars)
+ ch += next(chars, '')
if ch == '/*':
# Comment, start discarding
inComment = True