Pandas.ExcelWriter not behave the same after pyinstaller to exe file - python-3.x

I am a absolute beginner of python and working on a small program to collect data and write multiple sheets to Excel using pandas, it worked as expected. However after convert to exe file using pyinstaller, it only write first sheet. No matter in onefolder or onefile mode.
I pasted the minimum code that can recreate the problem.
I know I can just writer.save() once in the end but in real program I writer multiple sheets in for loop and it's a lot more efficient and create much smaller xlsx file than just save once in the end.
PyInstaller: 3.4
Python: 3.7.2
pandas: 0.24.2
Platform: Windows-10-10.0.17763-SP0
I search the problem everywhere, tried to use hiddenimports to add different modules for pyinstaller, like such:
hiddenimports=['pandas', 'pandas.io.excel', 'numpy', 'pandas.compat', 'xlsxwriter', 'openpyxl'],
None of them works.
import pandas as pd
writer = pd.ExcelWriter('xlstest.xlsx')
df1 = pd.DataFrame(data=[[1, 2, 3]], columns=("a", "b", "c"))
df2 = pd.DataFrame(data=[[5, 6, 7]], columns=("d", "e", "f"))
df1.to_excel(writer, sheet_name="sheet1", index=False)
writer.save()
df2.to_excel(writer, sheet_name="sheet2", index=False)
writer.save()

I solved my own problem. I was using Pycharm for the project. It appears that pycharm using a virtual environment while pyinstaller using the local system environment. So I tried to compare the installed packages in both environment and find out what could be the reason. I have same version of pandas and openpyxl installed in both environment, which puzzled me for a while. Tried a lot of things and eventually find out it was XlsxWriter package caused the problem. I have XlsxWriter 1.1.5 installed in local environment but not in pycharm venv. As soon as I pip uninstalled it from local env. My compiled exe file works correctly.
I am still looking into how to correctly using pyinstaller in pycharm venv. Any comment on that is welcomed.

Related

Python Pandas freezing why?

(I'm not very good in English but I will do my best)
Version of pandas = 1.3.4, python = 3.10.6 in all my env conda. And out of conda : python = 3.9.12 and pandas = 1.3.4
I'm working with Pandas since multiple years.
I often do this syntax in my script :
df[df['Column']=='Value']
But now it doesn't work at all and freeze all my scripts with no error at all in prompt.
I'm working on ubuntu 22 with Miniconda and i have multiple env.
The code i'm struggling with :
#!/usr/bin/env python
# coding: utf-8
import pandas as pd
df = pd.read_csv("data/corpus.csv")
print(df.shape)
df = df[df['Title']=='Yes']
print(df['Title'])
And this part of code work
var = df['Title']=='Yes'
But this wont:
df = df[var]
The print of df.shape is working but final print never show up, and prompt never print the end of interpreting the code.
I did try on different envs but that did not change the result of freezing my script.
The csv file is basic, nothing special, i did try with a new one created for testing but no change.
I did try in jupyter-notebook and in the terminal python prompt, no change.
I don't know what to do next, if you need more informations, i'm here to give it to you quickly.
Best regards,
The problem is coming from python=3.10.6. I downgrade to 3.8.16 and it's working well.
Thanks a lot Rainflow for your help.

Issue making .exe with Pyinstaller : compilation is done but the .exe fails

I have a rather large python script that I would like to convert to .exe. I use PyInstaller and I managed to import every library needed so now I don't get any error message. My problem is that at some point, the .exe is stuck, so the conversion with PyInstaller didn't really worked. I suspect the issue is related to the PyAutoGui library, but I had to import it to convert my python script (I used pip install).
Does anyone else got the same issue? Is there something I can do to remove this issue?
I have python 3.9.2 and pyinstaller 4.2 and I am working with windows 10.
The libraries I had to import are : pyautogui, pandas, bs4, pygame, PyQt5, pywin32, winshell, pymysql, sqlalchemy, sqlalchemy.sql.default_comparator, wmi, Pillow, psutil and lxml.
Thanks for your suggestions!
In my experience, PyInstaller has had trouble generating .exe files from python scripts that use graphics libraries such as pygame and PyQt5. Mostly, it fails to generate the .exe and even when it does, the .exe takes a long time to load up and is really slow. If you want an .exe file, I would suggest trying out cx_freeze. To my knowledge, it won't generate .exe in one file like pyinstaller does, but the .exe file it does generate is of much higher quality.
We think we found the issue: pyautogui needs an other package called open-cv to work. Hope it'll help someone someday ;-)

How to actual plot properly via Plotly

I have a problem with the plotly package. To be precise, I'm using auto-py-to-exe in order to have a file .exe which, if the users asks for it, plots a table using the plotly package. My code goes like this:
import pandas as pd
import plotly.graph_objects as go
import numpy
df= pd.read_excel (r'C:\Users\user\Desktop\Program\Data_Studio_Def.xls')
visualize_table= input('visualize table of patologies? (yes/no)')
if visualize_table== yes
fig = go.Figure(...)
fig.show()
Now, when I run the .exe made by auto-py-to-exe on the CMD (OS: Windows 10), I get some errors regarding modules like
Module plotly.validators.table not found
Once I add all such modules via hidden-import, I get the error:
FileNotFoundError No such file or directory : C\Users\user\output...
so it is searching for the plot in a folder rather than actually print it and I don't know how resolve this. Should I provide further informations?
P.S. I found out that the main problem is caused by the plotly package itself: I tried to build the exe via cx_Freeze rather than pyinstaller and the result was:
Module plotly.validators.table has no Attribute CellsValidators
Doing some searching I found out I'm not the only one with the same problem, the "HasNoAttribute", but uninstalling the package to then reinstall it again didn't do anything unfortunately.

Create/write to an excel file in python- Import Error

I am working on Python 3.2.5 version and my requirement is to create and write to an excel file. I tried with xlsxwriter and xlwt both, but ended up in getting ImportError. I can't install these modules.
Any other module which does excel creation please?
I think your python excel lib are not yet contained in python library path.
Window:
C:\Python27\Lib\site-packages\XlsxWriter
In other platform, also you should try to put XlsxWriter library under site-packages. It can solve the ImportError.

Anaconda3 libhdf5.so.9: cannot open shared object file [works fine on py2.7 but not on py3.4]

I just tried to use pd.HDFStore in IPython Notebook with a Python 3 kernel (Anaconda 2&3 on Ubuntu 14.04)
import pandas as pd
store = pd.HDFStore('/home/Jian/Downloads/test.h5')
but it throws the following error
ImportError: HDFStore requires PyTables, "libhdf5.so.9: cannot open shared object file: No such file or directory" problem importing
I initially thought it's because pytables is somehow missing, but when I check $source activate py34 and $conda list, pytables 3.2.0 is already installed under anaconda python3 environment.
Also, if I switch to Python 2, for example, $source activate py27 and start ipython notebook, it works properly and no import error is thrown.
I guess that I must miss something for configuring pytables under anaconda python 3 env, but I cannot figure it out. Any help is highly appreciated.
Update:
I just tried on a fresh install of Anaconda3-2.3.0-Linux-x86_64 from official website and it ends up with the same error. When I try $locate libhdf5.so.9 in command line, nothing shows up.
This is a known issue that we are working on. When it is fixed, conda update --all will update the libraries and fix the issue.

Resources