Set locale for stdout in python3 - python-3.x

I want my code to stop producing errors depending on the locale set on the terminal. For example this code:
import os
print(f"Locale {os.getenv('LC_ALL')}")
foo_bytes = b'\xce\x94, \xd0\x99, \xd7\xa7, \xe2\x80\x8e \xd9\x85, \xe0\xb9\x97, \xe3\x81\x82, \xe5\x8f\xb6, \xe8\x91\x89, and \xeb\xa7\x90.'
print(foo_bytes.decode("utf-8", "replace"))
Will print cleanly as long as my locale is US.UTF-8.
However if I change my locale and run the script I described earlier
export LC_ALL=en_US.iso885915
python3 locale_script.py
It will fail on:
Locale en_US.iso885915
Traceback (most recent call last):
File "locale_script.py", line 10, in <module>
print(foo_bytes.decode("utf-8", "replace"))
File "/usr/lib/python3.6/encodings/iso8859_15.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u0394' in position 0: character maps to <undefined>
This could be avoided if I had set the terminal locale within my script so it will use "utf-8" as I required. I have tried setlocale, but it still end up in the same error.
import locale
locale.setlocale(locale.LC_ALL, "C.UTF-8")
Any advice on what to do? I hope I could avoid to have to re-encode all my strings:
foo = 'Δ, Й, ק, ‎ م, ๗, あ, 叶, 葉, and 말.'
print(foo.encode().decode(sys.stdout.encoding))

Related

UnicodeDecodeError: invalid start byte in METADATA file at path:

I see that several Python-package related files have gibberish at their end.
Due to this, I am unable to do several pip operations (even basic ones like "pip list").
(Usually, I use conda by the way)
For example. When I pressed pip list. I get the following error.
ERROR: Exception:
Traceback (most recent call last):
File "C:\Users\shan_jaffry\Miniconda3\envs\SQL_version\lib\site-packages\pip\_internal\cli\base_command.py", line 173, in _main
status = self.run(options, args)
File "C:\Users\shan_jaffry\Miniconda3\envs\SQL_version\lib\site-packages\pip\_internal\commands\list.py", line 179, in run
self.output_package_listing(packages, options)
File "C:\Users\shan_jaffry\Miniconda3\envs\SQL_version\lib\site-packages\pip\_internal\commands\list.py", line 255, in output_package_listing
data, header = format_for_columns(packages, options)
File "C:\Users\shan_jaffry\Miniconda3\envs\SQL_version\lib\site-packages\pip\_internal\commands\list.py", line 307, in format_for_columns
row = [proj.raw_name, str(proj.version)]
File "C:\Users\shan_jaffry\Miniconda3\envs\SQL_version\lib\site-packages\pip\_internal\metadata\base.py", line 163, in raw_name
return self.metadata.get("Name", self.canonical_name)
File "C:\Users\shan_jaffry\Miniconda3\envs\SQL_version\lib\site-packages\pip\_internal\metadata\pkg_resources.py", line 96, in metadata
return get_metadata(self._dist)
File "C:\Users\shan_jaffry\Miniconda3\envs\SQL_version\lib\site-packages\pip\_internal\utils\packaging.py", line 48, in get_metadata
metadata = dist.get_metadata(metadata_name)
File "C:\Users\shan_jaffry\Miniconda3\envs\SQL_version\lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 1424, in get_metadata
return value.decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 14097: invalid start byte in METADATA file at path: c:\users\shan_jaffry\miniconda3\envs\sql_version\lib\site-packages\hupper-1.10.2.dist-info\METADATA
I went into the file META and found the following gibberish at the end. This (I found) has been done in several other files i.e. end of files are appended with gibberish and the actual thin is removed. Any help?
> 0.1 (2016-10-21)
> ================
> -
> - Initial rele9ýl·øA
I found that the by manually going to the site-packages folder, and removing the two folders, :: hupper and hupper-1.10.2.dist-info and then installing hupper again using "pip install hupper", problem was solved.
The issue was that the hupper package (and hupper-1.10.2.dist-info) were corrupted. Hence uninstall and re-install helped.

ValueError: Unsupported 'device_type'

I am new to Python.
I have this code in Python:
from netmiko import ConnectHandler
`sshCli = ConnectHandler(
device_type='Cisco_ios',
host='192.168.56.101',
port=22,
username='cisco',
password='cisco123!'
)
output = sshCli.send_command("show ip int brief")
print("show ip int brief:\n{}\n".format(output))
I get the following error:
========== RESTART: C:/Users/edanpc/AppData/Local/Programs/Python/Python38-32/Lab2.2.py =========
Traceback (most recent call last):
File "C:/Users/edanpc/AppData/Local/Programs/Python/Python38-32/Lab2.2.py", line 2, in <module>
sshCli = ConnectHandler(
File "C:\Users\edanpc\AppData\Local\Programs\Python\Python38-32\lib\site-packages\netmiko\ssh_dispatcher.py", line 297, in ConnectHandler
raise ValueError(
ValueError: Unsupported 'device_type' currently supported platforms are:
a10
accedian
adtran_os
alcatel_aos
alcatel_sros
apresia_aeos
arista_eos
aruba_os
avaya_ers
avaya_vsp
broadcom_icos
brocade_fastiron
brocade_netiron
brocade_nos
brocade_vdx
brocade_vyos
calix_b6
centec_os
checkpoint_gaia
ciena_saos
cisco_asa
cisco_ios
cisco_nxos
cisco_s300
cisco_tp
cisco_wlc
cisco_xe
cisco_xr
cloudgenix_ion
coriant
dell_dnos9
dell_force10
dell_isilon
dell_os10
dell_os6
dell_os9
dell_powerconnect
dlink_ds
eltex
eltex_esr
endace
enterasys
extreme
extreme_ers
extreme_exos
extreme_netiron
extreme_nos
extreme_slx
extreme_vdx
extreme_vsp
extreme_wing
f5_linux
f5_ltm
f5_tmsh
flexvnf
fortinet
generic
generic_termserver
hp_comware
hp_procurve
huawei
huawei_olt
huawei_smartax
huawei_vrpv8
ipinfusion_ocnos
juniper
juniper_junos
juniper_screenos
keymile
keymile_nos
linux
mellanox
mellanox_mlnxos
mikrotik_routeros
mikrotik_switchos
mrv_lx
mrv_optiswitch
netapp_cdot
netgear_prosafe
netscaler
nokia_sros
oneaccess_oneos
ovs_linux
paloalto_panos
pluribus
quanta_mesh
rad_etx
raisecom_roap
ruckus_fastiron
ruijie_os
sixwind_os
sophos_sfos
ubiquiti_edge
ubiquiti_edgeswitch
ubiquiti_unifiswitch
vyatta_vyos
vyos
watchguard_fireware
yamaha
zte_zxros
>>>
What is wrong with my code ?
This error is specific to netmiko. The error text says the list of supported device_type values. To solve the error, fix the below in your code.
device_type='cisco_ios'
The device type should contain small cisco_ios instead of Cisco_ios.

python : error encoding 'ascii' ONLY running with crontab

My python script working very well when i'm running it in console mode. It's write a html file, convert with pdfkit in pdf and send a mail. All is allright.
But, when this script running from the crontab, there's some encoding error.
if len(html) > 0:
with open(file_html_to_pdf, 'w') as file:
message = '[PDF] >> writing the html file "{0}"'.format(file_html_to_pdf)
logging.info(message)
print(message)
file.write(html)
else:
raise IOError('html file is invalid !')
In the crontab, I've done this :
# VARIABLES FOR PYTHON
PYTHONIOENCODING=utf8
*/5 * * * * /home/users/my-user/my-project/env/bin/python /home/users/my-user/my-project/cronjob.py > /var/log/apps/my-project/cron_error_my-project.log 2>&1
And in the bashrc :
# set locale utf-8 in french.....et voilà
export PYTHONIOENCODING=utf-8
export LC_ALL=fr_FR.utf-8
export LANG="$LC_ALL"
The error message :
UnicodeEncodeError: 'ascii' codec can't encode character '\xf4' in position 37: ordinal not in range(128)
...
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 400: ordinal not in range(128)
...
Message: ('<!> Error => ', UnicodeEncodeError('ascii', '<!DOCTYPE html>\n<head>\n <title>---
Python version :
$ /home/users/my-user/my-project/env/bin/python --version
Python 3.4.2
I don't understand why :(. Can anybody help me?
Thanks
F.

Why can't I decode('utf-16') success in Python3 (even work in Py2)?

In Py2:
(chr(145) + chr(78)).decode('utf-16')
I got u'\u4e91':
But in Py3:
(chr(145) + chr(78)).encode('utf-8').decode('utf-16')
I got an error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-16-le' codec can't decode byte 0x4e in position 2: truncated data
Sometimes, they work in a same way, such as (chr(93) + chr(78)), but sometimes not.
Why? And how can I do this right in Py3?
You have to use latin1 if you want to encode any byte tranparently:
(chr(145) + chr(78)).encode('latin1').decode('utf-16')
#'云'
chr(145) gets encoded with 2 bytes in utf8 (as with all values above 127):
chr(145).encode('utf8')
# b'\xc2\x91'
while it is what you wanted with latin1:
chr(145).encode('latin1')
# b'\x91'

Python SMTP: 'utf-8' codec can't decode byte 0xe7 in position 7: invalid continuation byte

My all code :`
import smtplib
from email.mime.text import MIMEText
smtp_adresi="smtp.gmail.com"
smtp_port=587
user="****#gmail.com"
pass="*****"
gonderilecek_adresler=["****#bilalkocak.net","******#gmail.com"]
konu="Subject"
content="HTML content"
mail=MIMEText(content,"html","UTF-8")
mail["From"]=kullanıcı_adı
mail["Subject"]=konu
mail["To"]=",".join(gonderilecek_adresler)
mail=mail.as_string()
s=smtplib.SMTP(smtp_adresi,smtp_port)
s.starttls()
s.login(user,pass)
s.sendmail(user,gonderilecek_adresler,mail)
Result:
C:\Users\ASUS\AppData\Local\Programs\Python\Python36-32\python.exe
"C:/Users/ASUS/PycharmProjects/Again/SMTP ile Mail/main.py" 'utf-8'
codec can't decode byte 0xe7 in position 7: invalid continuation byte
Process finished with exit code 0
\xe7 is the ç in your name but not encoded in UTF-8 (maybe cp1254, Turkish name?). Save your source file in UTF-8 and try again. It helps to have a reproducible example. Your ****** in the source probably removed the problem.
Note #coding:utf8 at the top of the file declares the encoding of the file, but it is the default in Python 3 so it is not required. Python 2 would need it.

Resources