Creating a PostgresSQL table using psycopg2 in Python - python-3.x

Question

I am trying to connect to a remote PostgresSQL database using the psycopg2 library in Python. To be clear, I can already do this using psql.exe, but that is not what I want to do here. So far, I have verified that I can connect and use my cursor to perform a simple query on an existing table:
import psycopg2
conn = psycopg2.connect(dbname='mydb', user='postgres', password='mypassword', host='www.mydbserver.com', port='5432', sslmode='require')
cur = conn.cursor()
cur.execute('SELECT * FROM existing_schema.existing_table')
one = cur.fetchone()
print(one)
This essentially, connects to an existing schema and table and selecting everything. I then fetch the first row from cur and print it. Example output: ('090010100001', '09001', None, 'NO', None, 'NO'). Now, I want to create a new table using this same method. I have already created a new schema called test within mydb. My plan is to copy csv data to the table leter, but for now, I just want to create the blank table. Here's what I have tried:
cur.execute("""
CREATE TABLE test.new_table
(
region TEXT,
state TEXT,
tier TEXT,
v_detailed DOUBLE PRECISION,
v_approx DOUBLE PRECISION,
v_unmapped DOUBLE PRECISION,
v_total DOUBLE PRECISION,
a_detailed DOUBLE PRECISION,
a_approx DOUBLE PRECISION,
a_unmapped DOUBLE PRECISION,
a_total DOUBLE PRECISION
)
""")
conn.commit()
When I ran the above in a Jupyter Notebook, I assumed it would be a rather quick process. However, it seems to get stuck and just run and run (the process did not complete after 30 + mins). Eventually, it threw an error: OperationalError: server closed the connection unexpectedly. This probably means the server terminated abnormally before or while processing the request. Should it take that long to run this simple line of code?! (I'm guessing, no). What might I be doing wrong here?

Answer 1

OK, there is an issue with using the .copy_from() method in psycopg2. That was the issue. This is how I overcame it:
conn = psycopg2.connect(dbname='mydb', user='postgres', password='mypassword', host='www.mydbserver.com', port='5432', sslmode='require')
cur = conn.cursor()
cur.execute("""
CREATE TABLE test.new_table
(
region TEXT,
state TEXT,
tier TEXT,
v_detailed DOUBLE PRECISION,
v_approx DOUBLE PRECISION,
v_unmapped DOUBLE PRECISION,
v_total DOUBLE PRECISION,
a_detailed DOUBLE PRECISION,
a_approx DOUBLE PRECISION,
a_unmapped DOUBLE PRECISION,
a_total DOUBLE PRECISION
)
""")
conn.commit()
with open(output_file, 'r') as f:
next(f) # Skip the header row.
#You must set the search_path to the desired schema beforehand
cur.execute('SET search_path TO test, public')
tbl = 'region_report_%s'% (report_type)
cur.copy_from(f, tbl, sep=',')
conn.commit()
conn.close()