Hello, I need your help for this error
I don’t have experience In Python, I am new on it
I need to export a query from MS Access to excel
I create the connection with my database and the connection seems OK
However, when I call the dataframe from Pandas I have this message:
Empty DataFrame
Columns: [Cofor6, TypePce, Ste, CompteDeNfact]
Index: []
This is my code:
connection_string = (
r"DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};"
r"DBQ=C:\xxx\xx\Python_Projects\xxx\xx\xx\mydb.mdb;"
r"ExtendedAnsiSQL=1;")`
connection_url = sa.engine.URL.create("access+pyodbc",query={"odbc_connect": connection_string})
engine = sa.create_engine(connection_url)
print ('connected')
RAFAELquery= '''SELECT SAP_CONSO.Cofor6,SAP_CONSO.TypePce,SAP_CONSO.Ste,Count(SAP_CONSO.Nfact) AS CompteDeNfact FROM SAP_CONSO GROUP BY SAP_CONSO.Cofor6,SAP_CONSO.TypePce,SAP_CONSO.Ste,SAP_CONSO.SaiParFlux HAVING (((SAP_CONSO.TypePce) Not Like 'y\*') AND ((SAP_CONSO.SaiParFlux) Like '\*E')) ORDER BY SAP_CONSO.TypePce DESC;'''
df=pd.read_sql_query(RAFAELquery,engine)
print(df)
Anyone can help me? Thanks a lot
Related
Hi all I'm new to Dask.
I faced an error when I tried using read_sql_query to get data from Oracle database.
Here is my python script:
con_str = "oracle+cx_oracle://{UserID}:{Password}#{Domain}/?service_name={Servicename}"
sql= "
column_a, column_b
from
database.tablename
where
mydatetime >= to_date('1997-01-01 00:00:00','YYYY-MM-DD HH24:MI:SS')
"
from sqlalchemy.sql import select, text
from dask.dataframe import read_sql_query
sa_query= select(text(sql))
ddf = read_sql_query(sql=sa_query, con=con, index_col="index", head_rows=5)
I refered this post: Reading an SQL query into a Dask DataFrame
Remove "select" string from my query.
And I got an cx_Oracle.DatabaseError with missing expression [SQL: SELECT FROM DUAL WHERE ROWNUM <= 5]
But I don't get it where the query came from.
Seem like it didn't execute the sql code I provided.
I'm not sure which part I did not config right.
*Note: using pandas.read_sql is ok , only fail when using dask.dataframe.read_sql_query
I am trying to replace an existing table in MySQL database. I used the below piece of code to convert the data frame called frame to a database table:
import pandas as pd
import sqlalchemy
from sqlalchemy.types import VARCHAR
database_username = 'root'
database_password = '1234'
database_ip = 'localhost'
database_name = 'my_new_database'
database_connection = sqlalchemy.create_engine('mysql+mysqlconnector://{0}:{1}#{2}/{3}'.format(database_username, database_password, database_ip, database_name),pool_size=3,pool_recycle=3600)
frame.to_sql(schema=database_name,con=database_connection,name='table1',if_exists='replace',chunksize=1000,dtype={'Enrollment No': VARCHAR(frame.index.get_level_values('Enrollment No').str.len().max())})
table1 gets created successfully. But when I rerun the last line of the above code i.e. frame.to_sql(), it throws the below error:
InvalidRequestError: Could not reflect: requested table(s) not available in Engine(mysql+mysqlconnector://root:***#localhost/my_new_database) schema 'my_new_database': (table1)
I want to know why this error is thrown when the table already exists, even though I've used if_exists='replace' and why it works correctly only when creating the table for the first time. What must be done to avoid getting this error?
N.B.: Answers to similar questions only suggest using the table name in lowercase, which I'm following by naming the table as 'table1'.
I'm aware that this one has been asked several times - particularly in this question, but I have not managed to solve my problem. Both snippets below have cx_Oracle and sqlalchemy installed
import cx_Oracle
from sqlachemy import create_engine
I'm trying to connect/write a pandas dataframe to an Oracle Database. I've managed to write to the database using the following code snippet:
ora_con = cx_Oracle.connect("{}/{}#{}".format(schema_name, password, name_of_service))
cur = ora_con.cursor()
statement='CREATE TABLE '+schema_Name+'.History (Name VARCHAR2(15), Entity VARCHAR2(15), Status VARCHAR2(25), Type VARCHAR2(25), Owner VARCHAR2(15), User_521 VARCHAR2(15), Manufacturer VARCHAR2(15), Model VARCHAR2(15), FusInv_Last_inventory DATE, Serial_Number VARCHAR2(25), ID VARCHAR2(15), Version_of_OS VARCHAR2(15), OS VARCHAR2(15), Date_of_Report DATE)'
cur.execute(statement)
That works.
When I try:
con_str = """oracle+cx_oracle://schema_name:password#Host_address:port/?service_name=name_of_service"""
engine = create_engine(con_str, echo=False)
pandasDataframe.to_sql('History', engine , index = False) # Insert the values from the INSERT QUERY into the table 'History'
The pandas .to_sql command fails with a cx_Oracle error of:
cx_Oracle.DatabaseError: ORA-12569: TNS:packet checksum failure
Googling the error indicates a networking error (network is fine) or a listener error (Port number, but that is also fine)
I can connect, write and read the database in SQL Developer.
Any thoughts anyone? Thanks in advance...
Your plain cx_Oracle connect string is different from the one for sqlalchemy. Note that sqlalchemy uses cx_Oracle.makedsn(). So if you have this connect syntax with plain cx_Oracle:
cx_Oracle.connect('myuser/mypassword#myhost:myport/myservice')
you would need this syntax for sqlalchemy:
con_str = 'oracle+cx_oracle://myuser:mypassword#myhost:myport/?service_name=myservice'
I'm trying to retrieve the SQL that makes up a stored query inside an Access database.
I'm using a combination of UcanAccess 4.0.2, and jaydebeapi and the ucanaccess console. The ultimate goal is to be able to do the following from a python script with no user intervention.
When UCanAccess loads, it successfully loads the query:
Please, enter the full path to the access file (.mdb or .accdb): /Users/.../SnohomishRiverEstuaryHydrology_RAW.accdb
Loaded Tables:
Sensor Data, Sensor Details, Site Details
Loaded Queries:
Jeff_Test
Loaded Procedures:
Loaded Indexes:
Primary Key on Sensor Data Columns: (ID)
, Primary Key on Sensor Details Columns: (ID)
, Primary Key on Site Details Columns: (ID)
, Index on Sensor Details Columns: (SiteID)
, Index on Site Details Columns: (SiteID)
UCanAccess>
When I run, from the UCanAccess console a query like
SELECT * FROM JEFF_TEST;
I get the expected results of the query.
I tried things including this monstrous query from inside a python script even using the sysSchema=True option (from here: http://www.sqlquery.com/Microsoft_Access_useful_queries.html):
SELECT DISTINCT MSysObjects.Name,
IIf([Flags]=0,"Select",IIf([Flags]=16,"Crosstab",IIf([Flags]=32,"Delete",IIf
([Flags]=48,"Update",IIf([flags]=64,"Append",IIf([flags]=128,"Union",
[Flags])))))) AS Type
FROM MSysObjects INNER JOIN MSysQueries ON MSysObjects.Id =
MSysQueries.ObjectId;
But get an object not found or insufficient privileges error.
At this point, I've tried mdbtools and can successfully retrieve metadata, and data from access. I just need to get the queries out too.
If anyone can point me in the right direction, I'd appreciate it. Windows is not a viable option.
Cheers, Seth
***********************************
* SOLUTION
***********************************
from jpype import *
startJVM(getDefaultJVMPath(), "-ea", "-Djava.class.path=/Users/seth.urion/local/access/UCanAccess-4.0.2-bin/ucanaccess-4.0.2.jar:/Users/seth.urion/local/access/UCanAccess-4.0.2-bin/lib/commons-lang-2.6.jar:/Users/seth.urion/local/access/UCanAccess-4.0.2-bin/lib/commons-logging-1.1.1.jar:/Users/seth.urion/local/access/UCanAccess-4.0.2-bin/lib/hsqldb.jar:/Users/seth.urion/local/access/UCanAccess-4.0.2-bin/lib/jackcess-2.1.6.jar")
conn = java.sql.DriverManager.getConnection("jdbc:ucanaccess:///Users/seth.urion/PycharmProjects/pyAccess/FE_Hall_2010_2016_SnohomishRiverEstuaryHydrology_RAW.accdb")
for query in conn.getDbIO().getQueries():
print(query.getName())
print(query.toSQLString())
If you can find a satisfactory way to call Java methods from within Python then you could use the Jackcess Query#toSQLString() method to extract the SQL for a saved query. For example, I just got this to work under Jython:
from java.sql import DriverManager
def get_query_sql(conn, query_name):
sql = ''
for query in conn.getDbIO().getQueries():
if query.getName() == query_name:
sql = query.toSQLString()
break
return sql
# usage example
if __name__ == '__main__':
conn = DriverManager.getConnection("jdbc:ucanaccess:///home/gord/UCanAccessTest.accdb")
query_name = 'Jeff_Test'
query_sql = get_query_sql(conn, query_name)
if query_sql == '':
print '(Query not found.)'
else:
print 'SQL for query [%s]:' % (query_name)
print
print query_sql
conn.close()
producing
SQL for query [Jeff_Test]:
SELECT Invoice.InvoiceNumber, Invoice.InvoiceDate
FROM Invoice
WHERE (((Invoice.InvoiceNumber)>1));
working with USDA for product Databse , the Database which in JSON format .However, i have cracked it using json package
USDA food Data json form Cracked into DataFrame:
But, few of the variables in the Data is on Dictionary Form
Am trying to create a DataFrame for variable ' Nutrients' .But getting the below Error
Error while Creating the Database:
Please help me in getting rid of error, below mentioned is the code
nutrients[]
for rec in db:
fnuts = DataFrame(rec['nutrients'])
fnuts['id'] = rec['id']
nutrients.append(fnuts)`
import pandas as pd
nutrients = pd.DataFrame(db1['nutrients'])