Observing some odd behavior with SQLite 2.6, where the ROW_NUMBER() throws an error only in Google Colab (Python 3.6.9), whereas the code works fine in my local Python 3.6.9 and Python 3.9.1 instances. Can you help me debug this further?
Code
import sqlite3, sys
try:
print('Py.version : ' + (sys.version))
print('sqlite3.version : ' + (sqlite3.version))
print('sqlite3.sqlite_version : ' + (sqlite3.sqlite_version)+'\n')
conn = sqlite3.connect(':memory:')
conn.execute('''CREATE TABLE team_data(team text, total_goals integer);''')
conn.commit()
conn.execute("INSERT INTO team_data VALUES('Real Madrid', 53);")
conn.execute("INSERT INTO team_data VALUES('Barcelona', 47);")
conn.commit()
sql='''
SELECT
team,
ROW_NUMBER () OVER (
ORDER BY total_goals
) RowNum
FROM
team_data
'''
print('### DB Output ###')
cursor = conn.execute(sql)
for row in cursor:
print(row)
except Exception as e:
print('ERROR : ' + str(e))
finally:
conn.close()
Output
Google Colab (ROW_NUMBER() causes SQL to fail):
Py.version : 3.6.9 (default, Oct 8 2020, 12:12:24) [GCC 8.4.0]
sqlite3.version : 2.6.0
sqlite3.sqlite_version : 3.22.0
### DB Output ###
ERROR : near "(": syntax error
Local Python 3.6.9 (Succeeds):
Py.version : 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 14:00:49) [MSC v.1915 64 bit (AMD64)]
sqlite3.version : 2.6.0
sqlite3.sqlite_version : 3.33.0
### DB Output ###
('Barcelona', 1)
('Real Madrid', 2)
Local Python 3.9.1 (Succeeds):
Py.version : 3.9.1 (default, Dec 11 2020, 09:29:25) [MSC v.1916 64 bit (AMD64)]
sqlite3.version : 2.6.0
sqlite3.sqlite_version : 3.33.0
### DB Output ###
('Barcelona', 1)
('Real Madrid', 2)
Note: Above SQL and code is simplified for error reproduction purposes only
The query in question is a window function and support for that was added in version 3.25. You can check the library (opposed to package) version with sqlite3.sqlite_version or as #forpas shared with the query select sqlite_version().
You can upgrade your sqlite version. Use this code.
!add-apt-repository -y ppa:sergey-dryabzhinsky/packages
!apt update
!apt install sqlite3
# MENU: Runtime > Restart runtime
import sqlite3
sqlite3.sqlite_version # '3.33.0'
Related
I have a problem which I didn't found the solution here.
I am working with SSHClient for connecting to multiple servers.
But, if there is 1 server in the list that I cannot access with my username and password (SSH), it's throwing me an exception.
I've tried to work with try and except but it didn't work.
Here is my original code:
from pssh.clients import ParallelSSHClient
import configparser
res = []
servers = ['test1', 'test2', 'test3', 'test4']
client = ParallelSSHClient(servers, user='test', password='test')
output = client.run_command('service ntpd status')
client.join(output)
for host_out in output:
for line in host_out.stdout:
if 'running' or 'Running' in line:
continue
else:
res.append(host_out.host + ' is not running')
if res:
return res
else:
return "All servers are running"
server test2 isn't accessible with my username so the script is throwing me an exception and failing the script:
pssh.exceptions.AuthenticationError: No authentication methods succeeded
How can I continue the script without the server test2 (if it is not accessible)
try something like this:
from pssh.clients import ParallelSSHClient
from pssh.config import HostConfig
host_config = [
HostConfig(user='user',
password='pwd'),
HostConfig(user='user',
password='pwd'),
HostConfig(user='user',
password='pwd')
]
servers = ['10.97.180.90', '10.97.180.99', "10.97.180.88"]
client = ParallelSSHClient(servers, host_config=host_config, num_retries=1, timeout=3)
output = client.run_command('uname -a', stop_on_errors=False, timeout=3)
for host_output in output:
try:
for line in host_output.stdout:
print(line)
exit_code = host_output.exit_code
except TypeError:
print("timeOut...")
And output looks like this:
Linux hostName-001 3.10.0-957.21.3.el7.x86_64 #1 SMP Fri Jun 14 02:54:29 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux
Linux hostName-003 3.10.0-1160.15.2.el7.x86_64 #1 SMP Thu Jan 21 16:15:07 EST 2021 x86_64 x86_64 x86_64 GNU/Linux
timeOut...
More info can be found here on stackoverflow:
Python parallel-ssh run_command does not timeout when using pssh.clients
I found a strange quirk in the slim version of the python Docker image with regards to date formatting. If you pass it a first-century date, %Y-%m-%d formatting doesn’t yield a zero-padded year-part:
$ docker run -ti python:3.7-slim /bin/bash
root#71f21d562837:/# python
Python 3.7.5 (default, Nov 23 2019, 06:10:46)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import date
>>> d = date(197,1,1)
>>> d.strftime('%Y-%m-%d')
'197-01-01'
But running this on the same python version locally on my macbook does yield 4 digits for the year:
$ python
Python 3.7.5 (default, Nov 1 2019, 02:16:32)
[Clang 11.0.0 (clang-1100.0.33.8)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from datetime import date
>>> d = date(197,1,1)
>>> d.strftime('%Y-%m-%d')
'0197-01-01'
The Python docs suggest that %y should yield no zero padding while %Y should.
Same quirk for version 3.6-slim.
The problem with this is that some systems (like BigQuery) requires the zero padding.
What would be the most elegant/least hacky workaround for this? I'm building a custom image derived from python:3.7-slim. I'm open to using a different image with a small footprint, or making an elegant code change.
You can always use a manual workaround to get identical formatting on all platforms:
from datetime import date
d = date(197,1,1)
dstr = d.strftime('%Y-%m-%d')
dstr = ('0'+dstr if len(dstr.split('-')[0]) == 3 else dstr)
print(dstr)
I have a python script that I'm trying to convert from python 2.7 to python 3.7.
The script includes windows API to read the system registry. In python 2.7 it works correctly. In python 3.7 it does not return the correct result.
I try to run the script python 3 in another PC with python 3. I run the script only in powershell like administrator.
https://learn.microsoft.com/en-us/windows/win32/api/winreg/nf-winreg-regopenkeyexa this is the documentation of RegOpenKeyExA() function.
In python 2.7 I installed "VCForPython27.msi" from: https://download.microsoft.com/download/7/9/6/796EF2E4-801B-4FC4-AB28-B59FBF6D907B/VCForPython27.msi which for windows 3.7 I don't find a updated version.
from ctypes import c_uint, c_char_p, byref, windll
subkey = 'JD'
def RegOpenKeyEx(subkey):
hkey = c_uint(0) ## Initialize to an int
windll.advapi32.RegOpenKeyExA(0x80000002, 'SYSTEM\\CurrentControlSet\\Control\\Lsa\\' + subkey, 0, 0xF003F , byref(hkey))
print(hkey.value)
return hkey.value
In python 2.7 the output is:
656
and the windll.advapi32.RegOpenKeyExA function returns 0 as a return value.
In python 3.7 the output is:
0
and the windll.advapi32.RegOpenKeyExA function returns 2 as a return value
I solved with replace the 6th row with:
windll.advapi32.RegOpenKeyExA(0x80000002, 'SYSTEM\\CurrentControlSet\\Control\\Lsa\\'.encode('ascii') + subkey.encode('ascii'), 0, 0x19 , byref(hkey))
In Python 2.7, strings are byte-strings by default. In Python 3.x, they are unicode by default. I explicitly making string a byte string using .encode('ascii').
according to official document, all str in python3 are in unicode, and actually there in no 'unicode' type in python3.
But there is a strange thing happened when I run jupyter notebook
time = re.findall(r'(\d+/\d+/\d+)', rating_bar.find('span', class_='rating-qualifier').text)[0].split('/')
di['date'] = '/'.join([str.zfill(t,2) for t in time[:2]] + time[2:] )
where rating_bar is a Beautiful Soup node, and jupyter notebook give this error
<ipython-input-9-a5ac4904b840> in parse_page(html)
25 class_='rating-qualifier').text)[
26 0].split('/')
---> 27 di['date'] = '/'.join([str.zfill(t,2) for t in time[:2]] + time[2:] )
28 di['rating'] = float(
29 rating_bar.find('div', class_='i-stars')['title'].split()[0])
TypeError: descriptor 'zfill' requires a 'str' object but received a 'unicode'
it's weird because there is no 'unicode' type in python3. And actually this code run correctly in my terminal.
I'm trying to develop a python module, which I then want to use in Spyder.
Here is how my files are organized in my module :
testing_the_module.py
myModule
-> __init__.py
-> sql_querying.py #contains a function called sql()
testing_the_module.py contains :
import myModule
print(myModule.sql_querying.sql(query = "show tables")) # how this function works is not relevant
__init__.py contains
import myModule.sql_querying
When I use the command line, it works :
> python3 .\testing_the_module.py
[{
'query': 'show tables',
'result': ['table1', 'table2']
}]
It also works if I use the python console :
> python3
Python 3.6.1 |Anaconda 4.4.0 (64-bit)| (default, May 11 2017, 13:25:24) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import myModule
>>> print(myModule.sql_querying.sql(query = "show tables"))
[{
'query': 'show tables',
'result': ['table1', 'table2']
}]
However, when using Spyder, I can't get it to work. Here is what I get when I run (with F9) each of those lines :
import myModule
# no error message
print(myModule.sql_querying.sql(query = "show tables"))
AttributeError: module 'myModule' has no attribute 'sql_querying'
Any idea of why and how to make it work in Spyder ?
Edit to answer comment :
In [665]: sys.path
Out[665]:
['',
'C:\\ProgramData\\Anaconda3\\python36.zip',
'C:\\ProgramData\\Anaconda3\\DLLs',
'C:\\ProgramData\\Anaconda3\\lib',
'C:\\ProgramData\\Anaconda3',
'C:\\ProgramData\\Anaconda3\\lib\\site-packages',
'C:\\ProgramData\\Anaconda3\\lib\\site-packages\\Sphinx-1.5.6-py3.6.egg',
'C:\\ProgramData\\Anaconda3\\lib\\site-packages\\win32',
'C:\\ProgramData\\Anaconda3\\lib\\site-packages\\win32\\lib',
'C:\\ProgramData\\Anaconda3\\lib\\site-packages\\Pythonwin',
'C:\\ProgramData\\Anaconda3\\lib\\site-packages\\setuptools-27.2.0-py3.6.egg',
'C:\\ProgramData\\Anaconda3\\lib\\site-packages\\IPython\\extensions',
'C:\\Users\\fmalaussena\\.ipython']