Get a table from a website to a CSV file - python-3.x

I need to transform the tables from a website to a CSV file.
I am using Python 3 from a raspberryPi.
Here is the code that I am using:
from urllib.request import urlopen, Request, URLError
from TableParser import TableParser
url_addr ='http://www.tbs-sct.gc.ca/pses-saff/2017-2/results-resultats/bq-pq/12/org-eng.aspx#s1'
req = Request(url_addr)
url = urlopen(req)
tp = TableParser()
tp.feed(url.read())
#Here I want the first table
my_table = tp.get_tables()[0]
filename = 'table_as_csv.csv'
f = open(filename, 'wb')
with f:
writer = csv.writer(f)
for row in table:
writer.writerow(row)
When I try to run it, I get:
ImportError: No module named 'TableParser'.
I am new to Python and I don't know what to do. Any help would be very appreciated.
Thanks!

First off, I think you should familiarise yourself with python and its syntax as well as how it handles errors. Clearly you want to do something quick and be done with it. Here's a simple python tutorial
To answer your question
ImportError exceptions occur when you do not have that particular module installed using pip installation preferrably. Until it is installed your script wont run successfully.
The error means TableParser has not been installed yet and you could do this by
pip3 install TableParser
This assumes you know where the module is and can be installed using pip
I see you are using the module csv but have not imported it as well. You should.

Related

Python Pandas freezing why?

(I'm not very good in English but I will do my best)
Version of pandas = 1.3.4, python = 3.10.6 in all my env conda. And out of conda : python = 3.9.12 and pandas = 1.3.4
I'm working with Pandas since multiple years.
I often do this syntax in my script :
df[df['Column']=='Value']
But now it doesn't work at all and freeze all my scripts with no error at all in prompt.
I'm working on ubuntu 22 with Miniconda and i have multiple env.
The code i'm struggling with :
#!/usr/bin/env python
# coding: utf-8
import pandas as pd
df = pd.read_csv("data/corpus.csv")
print(df.shape)
df = df[df['Title']=='Yes']
print(df['Title'])
And this part of code work
var = df['Title']=='Yes'
But this wont:
df = df[var]
The print of df.shape is working but final print never show up, and prompt never print the end of interpreting the code.
I did try on different envs but that did not change the result of freezing my script.
The csv file is basic, nothing special, i did try with a new one created for testing but no change.
I did try in jupyter-notebook and in the terminal python prompt, no change.
I don't know what to do next, if you need more informations, i'm here to give it to you quickly.
Best regards,
The problem is coming from python=3.10.6. I downgrade to 3.8.16 and it's working well.
Thanks a lot Rainflow for your help.

How can I install parse for python3 if I get importError?

So I'm working in Linux and I need to install parse for python3 but always get the same error: ImportError: No module named parse. I tried it:
from urllib.parse import urlparse
from parser import *
try:
from urllib.parse import urlparse
except ImportError:
from urlparse import urlparse (but as I know its only for python2, I work on python3).
Also tried to do this pip install parse but had no result. Before it I had the next error “NameError: global name 'parse' is not defined”.
Please can you help me, what should I do? I found that some people have the same problem but their resolutions dont help me
urllib is in standard library, no need to install. It works ok for me in python 3.x. Probably you have named your script(the .py file you are running) to urllib. This is a common mistake, rename it to something else then it works.
It could happen even if you have a python file named urllib in your directory... because when you run your script, python will automatically add it's directory to sys.path(where python searched for modules/packages). So it gets reached sooner than the original urllib which is in the standard library.
Search that file in your directory and delete it.

Is it possible to install python libraries with pip programmatically?

Let me explain what I want to do.
The list of libraries I want installed is listed in a .txt file.
My script reads the list from the file sequentially, and if the script isn't installed, it installs it via pip, or if it is already installed, checks the version and updates it if necessary.
I googled it up but didn't find how to do that. Can you offer any help or guidance?
Yes you can. Try this, here is an example of one module which is hard coded
import os
import subprocess
import sys
get_pckg = subprocess.check_output([sys.executable, '-m', 'pip', 'freeze'])
installed_packages = [r.decode().split('==')[0] for r in get_pckg.split()]
required_packeges = ['shopifyAPI'] // Make a change here to fetch from file
for packg in required_packeges:
if packg in installed_packages:
pass
else:
print('installing package')
os.system('pip install ' + packg)
First i will fetch all installed modules and then i will check my required module is installed or not if not then it will install it.
Yes, you can. Python module os does support running script programmatically. Since I don't know how your file structure looks like, I guess you can read the file and run the script sequentially.
import os
os.system("pip install <module>")
Use Following to install lib. programmatically.
import pip
try:
pip.main(["install", "pandas"])
except SystemExit as e:
pass

How to resolve 'No module named 'cPickle'' exception in virtualenv

I'm trying to run a program which connects to all databases(mysql,sqlite) and fetch data from it .
Python version - 3.6.8
Since the code is too long ,i'm showing only particular snippets.
def show_columns_mysql(cursor,tbname):
cursor.execute("""show columns from %s"""%(tbname))
rs=cursor.fetchall()
colname=[]
for i in rs:
colname.append(i[0])
return colname
There is no problem or issue if i exexute the program in normal python environment . When i try to execute this in virtual environment ,it shows me No module named 'cPickle' .
I have tried all the solutions but none solved my problem .
What was the problem ?
There is no cPickle in Python 3. Just import pickle. pickle will automatically use the C accelerator.
Install pickle. Then do:
import pickle as cPickle

Python 3: No module named zlib?

I am trying to run my Flask application with an Apache server using mod_wsgi, and it has been a bumpy road, to say the least.
It has been suggested that I should try to run my app's .wsgi file using Python to make sure it is working.
This is the contents of the file:
#!/usr/bin/python
activate_this = '/var/www/Giveaway/Giveaway/venv/bin/activate_this.py'
with open(activate_this) as f:
code = compile(f.read(), "somefile.py", 'exec')
exec(code)
import sys
import logging
logging.basicConfig(stream=sys.stderr)
sys.path.insert(0,"/var/www/Giveaways/")
from Giveaways import application
application.secret_key = 'Add your secret key'
However, when I run it, I get this error:
ImportError: No module named 'zlib'
And no, I am not using some homebrewed version of Python - I installed Python 3 via apt-get.
Thanks for any help.
does the contents of somefile.py include the gzip package? in which case you may have to install gzip package via pip or similar

Resources