I am unable to create a database file using python. I am facing this issue only in windows. On my Mac machine i was able to get it working. Here is a test code i tried -
import os, sys, shutil, glob, pandas, sqlite3
db_path = 'E:\\Archive.db'
conn = sqlite3.connect(db_path)
Here are other versions of variable db_path i have tried
db_path = 'E:\Archive.db'
db_path = 'E:/Archive.db'
db_path = 'E://Archive.db'
But i always get following error -
Traceback (most recent call last):
File "test.py", line 4, in <module>
conn = sqlite3.connect(db_path)
sqlite3.OperationalError: unable to open database file
I do have write permissions on E:
I was able to create csv files using python script. Not sure why its stuck with database. It should create the database in E:
Following example taken from here works for me. Try this out. If this doesn't work, try to create a file in your documents folder by changing the path
import sqlite3
from sqlite3 import Error
def create_connection(db_file):
""" create a database connection to a SQLite database """
conn = None
try:
conn = sqlite3.connect(db_file)
print(sqlite3.version)
except Error as e:
print(e)
finally:
if conn:
conn.close()
if __name__ == '__main__':
create_connection(r"E:\Archive.db")
Related
I wrote a small Python script which takes all .jpeg img-files from a given "./Input" folder, creates a thumbnail from these images and saves them in a "./Output" file. It works perfectly fine when executed in an IDEA (In my case PyCharm), but when I created the executable with pyinstaller and tried to execute it, the following error appeared:
Traceback (most recent call last):
File "image_resizer.py", line 3, in <module>
ModuleNotFoundError: No module named 'PIL'
[84693] Failed to execute script 'image_resizer' due to unhandled exception: No module named 'PIL'
[84693] Traceback:
Traceback (most recent call last):
File "image_resizer.py", line 3, in <module>
ModuleNotFoundError: No module named 'PIL'
logout
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
Here is my code:
import shutil
from PIL import Image
import glob, os
import time
output_directory = "./Output"
def photo_picker():
pictures = []
try:
for filename in glob.iglob("./Input/*.jpg"):
pictures.append(filename)
except FileNotFoundError:
print("could not execute")
return pictures
def create_output_folder():
if os.path.exists(output_directory):
shutil.rmtree(output_directory)
try:
os.mkdir(path=output_directory, mode=777)
except FileExistsError:
print("Could not execute")
def crop_image(img, new_name):
size = (2000, 2000)
with Image.open(img) as im:
im.thumbnail(size)
print(im.info)
img_name = f"{new_name}"
im.save(output_directory + "/" + img_name, "JPEG")
def main():
photos = photo_picker()
create_output_folder()
time.sleep(2)
for f in photos:
img_name_splitted = f.split("/")
img_name = img_name_splitted[len(img_name_splitted) - 1]
print(img_name)
crop_image(f, img_name)
return 0
if __name__ == "__main__":
main()
I installed the module Pillow, uninstalled it ,tried to install PIL (which didn't work). But as I mentioned before, when executing it in an IDEA it works.
I use impyla and ibis to connect hive server, but I got the error.
I tried the following code:
from impala.dbapi import connect
impcur = connect(host="kudu3", port=10000, database="yingda_test", password=None, user='admin', kerberos_service_name='None').cursor()
The new error came out:
Traceback (most recent call last):
File "/Users/edy/src/PythonProjects/dt-center-algorithm/test/1.py", line 4, in <module>
impcur = connect(host="kudu3", port=10000, database="yingda_test", password=None, user='admin', kerberos_service_name='None').cursor()
File "/usr/local/conda3/envs/py37/lib/python3.7/site-packages/impala/hiveserver2.py", line 129, in cursor
session = self.service.open_session(user, configuration)
File "/usr/local/conda3/envs/py37/lib/python3.7/site-packages/impala/hiveserver2.py", line 1187, in open_session
resp = self._rpc('OpenSession', req, True)
File "/usr/local/conda3/envs/py37/lib/python3.7/site-packages/impala/hiveserver2.py", line 1080, in _rpc
response = self._execute(func_name, request, retry_on_http_error)
File "/usr/local/conda3/envs/py37/lib/python3.7/site-packages/impala/hiveserver2.py", line 1142, in _execute
.format(self.retries))
impala.error.HiveServer2Error: Failed after retrying 3 times
thrift 0.15.0
thrift-sasl 0.4.3
thriftpy2 0.4.14
pure-sasl 0.6.2
sasl 0.2.1
thrift-sasl 0.4.3
ibis-framework 2.0.0
impyla 0.17.0
python version: 3.7.12 with anaconda
And I have tried ibis-1.3.0 and 2.0 version. Can u guys give some advices? tks a lot
I also met this problem.
My code is:
from impala.dbapi import connect
import psycopg2
conn_hive = connect(host="xxx.xxx.xxx.xxx", port=xxx, user='admin',
password='password', database='xxx', auth_mechanism="PLAIN", timeout=6000)
hive_cursor = conn_hive.cursor()
hive_cursor.execute(query_sql)
data_list = hive_cursor.fetchall()
...get data...
hive_cursor.close()
conn_hive.close()
After my colleagues and I tried, we find it will be successful when we reconnect hive manually.
It means if you want to get different data from the same hive database, you had better close the connection and reconnect hive manually by the following codes:
conn_hive = connect(host="xxx.xxx.xxx.xxx", port=xxx, user='admin',
password='password', database='xxx', auth_mechanism="PLAIN", timeout=6000)
hive_cursor = conn_hive.cursor()
hive_cursor.execute(query_sql)
data_list = hive_cursor.fetchall()
...get data1...
hive_cursor.close()
conn_hive.close()
conn_hive = connect(host="xxx.xxx.xxx.xxx", port=xxx, user='admin',
password='password', database='xxx', auth_mechanism="PLAIN", timeout=6000)
hive_cursor = conn_hive.cursor()
hive_cursor.execute(query_sql)
data_list = hive_cursor.fetchall()
...get data2...
hive_cursor.close()
conn_hive.close()
Finally, my colleagues told me that there was a problem with impala recently.
I keep getting an error with Pandas' 0.25.0 read_sql(). The code below is supposed to establish a couple connections, check if tables need to be created, and then begin extracting the result set from Oracle.
Stack Trace
RESTART: C:\xxxx\xxxx\AppData\Local\Programs\Python\Python36\xxxx\main.py
Creating connections
Attempting to connect
Connection to box_db Successful
Attempting to connect
Connection to JDE Successful
Box table already existed
BoxStaging table already existed
LotStartSiteMap table already existed
Extracting from JDE
Traceback (most recent call last):
File "C:\xxxx\xxxx\AppData\Local\Programs\Python\Python36\xxxx\main.py", line 42, in <module>
main()
File "C:\xxxx\xxxx\AppData\Local\Programs\Python\Python36\xxxx\main.py", line 26, in main
df = data.box_data_extract(queries.BOX_QUERY, jde_conn)
File "C:\xxxx\xxxx\AppData\Local\Programs\Python\Python36\xxxx\data.py", line 46, in box_data_extract
pd.read_sql_query(query, connection)
File "C:\xxxx\xxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\io\sql.py", line 332, in read_sql_query
chunksize=chunksize,
File "C:\xxxx\xxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\io\sql.py", line 1645, in read_query
cursor = self.execute(*args)
File "C:\xxxx\xxxx\AppData\Local\Programs\Python\Python36\lib\site-packages\pandas\io\sql.py", line 1590, in execute
cur = self.con.cursor()
AttributeError: module 'connection' has no attribute 'cursor'
main.py
import connection
import credential
import data
import emailer
import queries
import tables
def main():
# Create connection to box_db file and JDE
#
print("Creating connections")
sqlite_conn = connection.box_db_connect(tables.DB_PATH)
jde_conn = connection.mrap0680_connect(credential.USERNAME, credential.MRAP0680_PASSWORD)
# If new path has been chosen create sqlite db and tables accordingly
#
tables.create_box_table(sqlite_conn)
tables.create_box_staging_table(sqlite_conn)
tables.create_lot_startsite_map_table(sqlite_conn)
# Extract & insert JDE Box Query results into BoxStaging table
#
print("Extracting from JDE")
df = data.box_data_extract(queries.BOX_QUERY, jde_conn) #############LINE CAUSING ERROR
print("Storing in box_db")
data.box_data_insert(df, sqlite_conn)
print("BoxStaging updated")
data.py
# Extract the box data from JDE
#
def box_data_extract(query, connnection):
try:
df = pd.read_sql(query, con=connection)
print("JDE Box Data extracted")
return df
except Exception as e:
print(repr(e))
connection.py
def xxxx_connect(username, password):
# Connect to JDE database
try:
print("Attempting to connect")
conn = cx_Oracle.connect(user=username, password=password, dsn=DSN_TNS)
print("Connection to JDE Successful")
return conn
except Exception as e:
print(repr(e))
I am able to extract the data if running manually.
Please help. Thanks.
I found the error! The issue was in data.py. In the parameter list of data_box_extract, connection is spelled as "connnection". Inside the function, "connection" was being used so a cursor couldn't be instantiated. Not sure why it didn't throw an error about "connection" not being instantiated. Oh well, I'm writing the program in IDLE, maybe it's time to switch to an IDE that will catch those mistakes...
Thanks to all who viewed!
I'm a beginner at python. I'm trying to connect to Mongo DB server from my python flask application. But I couldn't able to run the application as I'm facing the below problem.
from flask import Flask, render_template
from flask_pymongo import PyMongo
from pymongo import MongoClient # Database connector
app=Flask(__name__)
app.config["MONGO_DBNAME"]="connect_to_pymon"
app.config["MONGO_URI"]="mongodb://mongo_test:mongo_test#123#ds129821.mlab.com:29821/connect_to_pymon"
mongo = PyMongo(app)
#app.route("/add")
def add():
user = mongo.db.users
user.insert[{"name":"kishor"}]
return "users added!"
if __name__=='__main__':
app.run(debug=True, port=8080)
This is my source code. When I execute this I'm getting the following error which I couldn't able to trace it.
Traceback (most recent call last):
File "C:/Users/kravi/PycharmProjects/GitUploadTest/mongoTest.py", line 9, in <module>
mongo = PyMongo(app)
File "C:\Kishor\Training\My_Workspace\Python_Basics\TaskCRUD\venv\lib\site-packages\flask_pymongo\__init__.py", line 116, in __init__
self.init_app(app, uri, *args, **kwargs)
File "C:\Kishor\Training\My_Workspace\Python_Basics\TaskCRUD\venv\lib\site-packages\flask_pymongo\__init__.py", line 149, in init_app
parsed_uri = uri_parser.parse_uri(uri)
File "C:\Kishor\Training\My_Workspace\Python_Basics\TaskCRUD\venv\lib\site-packages\pymongo\uri_parser.py", line 379, in parse_uri
user, passwd = parse_userinfo(userinfo)
File "C:\Kishor\Training\My_Workspace\Python_Basics\TaskCRUD\venv\lib\site-packages\pymongo\uri_parser.py", line 97, in parse_userinfo
"RFC 3986, use %s()." % quote_fn)
pymongo.errors.InvalidURI: Username and password must be escaped according to RFC 3986, use urllib.parse.quote_plus().
Process finished with exit code 1
Note: I haven't installed mongoDb in my PC. I'm using mongo db as a service
you can connect to your db using PyMongo as follows:
from pymongo import MongoClient
client = MongoClient("mongodb://mongo_test:mongo_test#123#ds129821.mlab.com:29821/connect_to_pymon")
db = client["dbname"] # connect_to_pymon in your case
I understood that using special characters like "#" in passwords doesn't make sense as we have to configure mongo URI which has a specific format with symbol # in it. I changed the database password and tried, after which it worked.!
Trying Python today for the first time today and got stuck following and example almost immediately. Using Pyhon 3.6 on Windows. Can someone help?
RESTART: C:/Users/tom_/AppData/Local/Programs/Python/Python36-32/Projects/Database/dbexample.py
Traceback (most recent call last):
File "C:/Users/tom_/AppData/Local/Programs/Python/Python36-32/Projects/Database/dbexample.py", line 13, in <module>
enter_data()
File "C:/Users/tom_/AppData/Local/Programs/Python/Python36-32/Projects/Database/dbexample.py", line 11, in enter_data
c.execute("INSERT INTO Example VALUES('Python', 2.7, 'Beginner')")
sqlite3.OperationalError: no such table: Example
Code:
import sqlite3
conn = sqlite3.connect('tutorial.db')
c = conn.cursor()
def create_table():
c.execute("CREATE TABLE Example(Language VARCHAR, Version REAL, Skill TEXT)")
def enter_data():
c.execute("INSERT INTO Example VALUES('Python', 2.7, 'Beginner')")
enter_data()
conn.close()
you need to call create_table() once before you can use enter_data() for a new db.
Once it has been created you will get a sqlite3.OperationalError: table Example already exists if you call it again.