I have started converting code which I want run on both posix and windows from os.path to the pathlib.Path module. Specifically, I replaced -
import os
os.path.join('grandparent_dir', 'parent_dir', 'file_name')
with
from pathlib import Path
Path('grandparent_dir', 'parent_dir', 'file_name')
It seems to work but I can't find this way of instantiating the class in the documentation. Is it good practice?
In the documentation on Pure Paths, I have just found an example instantiation -
PurePath('foo', 'some/path', 'bar')
Since pathlib.Path is a subclass of PurePath, it must also be valid pathlib.Path
I'm trying to build some custom Python modules.
Even though I was able to use all modules. When I try to import a module other modules are being imported.
For example:
mod1.py
import os
import sys
def f(x):
return (x**2)
main.py
import mod1
dir(mod1.os)
how can I avoid this behaviour? The user should not be able to access the other modules from mod1. In this example os and sys.
Should I put the import statements inside the function? Are there other ways to prevent such thing?
The behavior you are seeing is perfectly normal.
Python does not hide elements quite so strongly as some other languages.
We're all adults here, and are expected to behave responsibly, without strong enforcement from the language.
Should I put the import statements inside the function?
No, your code is just fine the way it is.
If you plan to write a bunch of python code, you should follow the usual conventions that everyone else does, as shown in your example code. In main.py, if you want to access an os function, you should certainly import it there, rather than borrowing a reference from mod1.os.
I am using the google adwords api. Currenlty my only code is:
from googleads import adwords
adwords_client = adwords.AdWordsClient.LoadFromStorage()
This results in an error displaying Your default encoding, cp1252, is not UTF-8. Please run this script with UTF-8 encoding to avoid errors.
I am using Python 3.6, which should be UTF-8 by default. What is the source of this error/how is it avoided?
It turns out that this is actually a warning emitted by googleads whenever the default encoding returned by locale.getdefaultlocale() is not UTF-8.
If your script runs without issues, I feel that you can safely ignore it. Otherwise it might be worth a try to set a different locale at the beginning of your code:
import locale
locale.setlocale(locale.LC_ALL, NEW_LOCALE)
I take it that you are running Windows, so I'm not sure what the proper locale definitions are. On Linux, you could use en_US.UTF-8, but that's probably not going to work for you.
Try importing the _locale module.
import _locale
_locale._getdefaultlocale = (lambda *args: ['en_US', 'UTF-8'])
in python with getdefaultlocale I can get the locale
>>> import locale
>>> locale.getdefaultlocale()
('es_ES', 'UTF-8')
in nodejs exists some similar?
Unfortunately, it's not as straightforward as it seems. The docs tells the whole story. There's also os-locale that it might be helpful.
But if you happen to be running a simulated browser environment, you can try this:
console.log('navigator.language:', navigator.language);
When the lib click detects that the runtime is python3 but the encoding is ASCII then it ends the python program abruptly:
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Either switch to Python 2 or consult http://click.pocoo.org/python3/ for mitigation steps.
I found the cause of this issue in my case, when I connect to my Linux host from my Mac, the Terminal.app set the SSH session locale to my Mac locale (es_ES.UTF-8) However my Linux host hasn't installed such locale (only en_US.utf-8).
I applied an initial workaround to fix it (but It had many issues, see accepted answer):
import locale, codecs
# locale.getpreferredencoding() == 'ANSI_X3.4-1968'
if codecs.lookup(locale.getpreferredencoding()).name == 'ascii':
os.environ['LANG'] = 'en_US.utf-8'
EDIT: For a better patch see my accepted answer.
All my linux hosts have installed 'en_US.utf-8' as locale (Fedora uses it as default).
My question is: Is there a better (more robust) way to choose/force the locale in a python3 script ? For instance, setting one of the available locales in the system.
Maybe there is a different approach to fix this issue but I didn't find it.
If you have python version >= 3.7, then you should not need to do anything. If you have python 3.6 see the original solution.
EDIT 2017-12-08
I've seen that there is a PEP 538 for py3.7, that will change the entire behavior of python3 encoding management during startup, I think that the new approach will fix the original problem: https://www.python.org/dev/peps/pep-0538/
IMHO the changes targeted to python 3.7 for encoding issues, should have been planed years ago, but better late than never, I guess.
EDIT 2015-09-01
There is an opened issue (enhancement), http://bugs.python.org/issue15216, that will allow to change the encoding in a created (not-used) stream easily (sys.std*). But is targeted to python 3.7 So, we'll have to wait for a while.
Original solution that targets python version 3.6
NOTE: this solution should not be needed for anyone running python version >= 3.7 see PEP 538
Well, my initial workaround had many flaws, I got to pass the click library check about the encoding, but the encoding itself was not fixed, so I get exceptions when the input parameters or output had non-ascii characters.
I had to implement a more complex method, with 3 steps: set locale, correct encoding in std in/out and re-encode the command line parameters, besides I've added a "friendly" exit if the first try to set the locale doesn't work as expected:
def prevent_ascii_env():
"""
To avoid issues reading unicode chars from stdin or writing to stdout, we need to ensure that the
python3 runtime is correctly configured, if not, we try to force to utf-8,
but It isn't possible then we exit with a more friendly message that the original one.
"""
import locale, codecs, os, sys
# locale.getpreferredencoding() == 'ANSI_X3.4-1968'
if codecs.lookup(locale.getpreferredencoding()).name == 'ascii':
os.environ['LANG'] = 'en_US.utf-8'
if codecs.lookup(locale.getpreferredencoding()).name == 'ascii':
print("The current locale is not correctly configured in your system")
print("Please set the LANG env variable to the proper value before to call this script")
sys.exit(-1)
#Once we have the proper locale.getpreferredencoding() We can change current stdin/out streams
_, encoding = locale.getdefaultlocale()
import io
sys.stderr = io.TextIOWrapper(sys.stderr.detach(), encoding=encoding, errors="replace", line_buffering=True)
sys.stdout = io.TextIOWrapper(sys.stdout.detach(), encoding=encoding, errors="replace", line_buffering=True)
sys.stdin = io.TextIOWrapper(sys.stdin.detach(), encoding=encoding, errors="replace", line_buffering=True)
# And finally we need to re-encode the input parameters
for i, p in enumerate(sys.argv):
sys.argv[i] = os.fsencode(p).decode()
This patch solves almost all issues, however it has a caveat, the method shutils.get_terminal_size() raises a ValueError because the sys.__stdout__ has been detached, click lib uses that method to print the help, to fix it I had to apply a monkey-patch on click lib
def wrapper_get_terminal_size():
"""
Replace the original function termui.get_terminal_size (click lib) by a new one
that uses a fallback if ValueError exception has been raised
"""
from click import termui, formatting
old_get_term_size = termui.get_terminal_size
def _wrapped_get_terminal_size():
try:
return old_get_term_size()
except ValueError:
import os
sz = os.get_terminal_size()
return sz.columns, sz.lines
termui.get_terminal_size = _wrapped_get_terminal_size
formatting.get_terminal_size = _wrapped_get_terminal_size
With this changes all my scripts work fine now when the environment has a wrong locale configured but the system supports en_US.utf-8 (It's the Fedora default locale).
If you find any issue on this approach or have a better solution, please add a new answer.
It's an aged thread, however this answer might help other in the future or myself. If it's *nux
env | grep LC_ALL
if it's set, do the follows. That's all of it.
unset LC_ALL
If you are running python 3.6 then you will still get this error. Here is a simple solution that the authors of click recommend:
#!/bin/bash
# before your python code executes set two environment variables
export LANG=en_US.utf8
export LC_ALL=en_US.utf8
NOTE: replace the values with whatever your locale is configured to
NOTE: this solution is even given in the PEP 538 document seen here.
I haven't found this simple method (re-exec script with proper environment before doing anything) so I'll add it for future travellers using old Python version for some reason. Add it bellow imports to be that first :
if os.environ["LC_ALL"] != "C.UTF-8" or os.environ["LANG"] != "C.UTF-8":
os.execve(sys.executable,
[os.path.realpath(__file__)] + sys.argv,
{"LC_ALL": "C.UTF-8", "LANG": "C.UTF-8"})