I have trouble trying to keep the sharing properties of an excel. I tried this :Python and openpyxl is saving my shared workbook as unshared but the part of vout just cancels all the modification I made with the script
To explain the problem :
There's an excel file that is shared in which people can do some modification
Python reads and writes on it
When I save the workbook in the excel file, it automatically either drops the sharing property or when I try to keep it, it just doesn't do any modification
Can someone help me please ?
I'll get a little more precise, as requested.
The sharing mode is the one Microsoft provides. You can see the button below:
Share button Excel
The excel is stored on a server. Several users can write on it at the same time but when I launch my script, it stops automatically the sharing property, so everyone that is writing on it just can't do modification anymore and every modif they did is lost.
First I treated my Excel normally :
DLT=openpyxl.load_workbook(myPath)
ws=DLT['DLT']
...my modifications on ws...
DLT.save()
DLT.close()
But then I tried this (Python and openpyxl is saving my shared workbook as unshared)
DLT=openpyxl.load_workbook(myPath)
ws=DLT['DLT']
zin = zipfile.ZipFile(myPath, 'r')
buffers = []
for item in zin.infolist():
buffers.append((item, zin.read(item.filename)))
zin.close()
...my modif on ws...
DLT.save()
zout = zipfile.ZipFile(myPath, 'w')
for item, buffer in buffers:
zout.writestr(item, buffer)
zout.close()
DLT.close()
The second one just doesn't save my modification on ws.
The thing I would like to do, is not to get rid of the sharing property. I would need to keep it while I write on it. Not sure if it is possible. I have one alternative solution that is to use another file, and just copy/paste by hand the new data from this file to the DLT one.
well... after playing with it back and forth, for some weird reason zipfile.infolist() does contains the sheet data as well, so here's my way to fine tune it, using the shared_pyxl_save example the previous gentleman provided
basically instead of letting the old file overriding the sheet's data, use the old one
def shared_pyxl_save(file_path, workbook):
"""
`file_path`: path to the shared file you want to save
`workbook`: the object returned by openpyxl.load_workbook()
"""
zin = zipfile.ZipFile(file_path, 'r')
buffers = []
for item in zin.infolist():
if "sheet1.xml" not in item.filename:
buffers.append((item, zin.read(item.filename)))
zin.close()
workbook.save(file_path)
""" loop through again to find the sheet1.xmls and put it into buffer, else will show up error"""
zin2 = zipfile.ZipFile(file_path, 'r')
for item in zin2.infolist():
if "sheet1.xml" in item.filename:
buffers.append((item, zin2.read(item.filename)))
zin2.close()
#finally saves the file
zout = zipfile.ZipFile(file_path, 'w')
for item, buffer in buffers:
zout.writestr(item, buffer)
zout.close()
workbook.close()
Related
I'm new to working with python-chess and I was perusing the official documentation. I noticed this very weird thing I just can't make sense of. This is from the documentation:
import chess.pgn
pgn = open("data/pgn/kasparov-deep-blue-1997.pgn")
first_game = chess.pgn.read_game(pgn)
second_game = chess.pgn.read_game(pgn)
So as you can see the exact same function pgn.read_game() results in two different games to show up. I tried with my own pgn file and sure enough first_game == second_game resulted in False. I also tried third_game = chess.pgn.read_game() and sure enough that gave me the (presumably) third game from the pgn file. How is this possible? If I'm using the same function shouldn't it return the same result every time for the same file? Why should the variable name matter(I'm assuming it does) unless programming languages changed overnight or there's a random function built-in somewhere?
The only way that this can be possible is if some data is changing. This could be data that chess.pgn.read_game reads from elsewhere, or could be something to do with the object you're passing in.
In Python, file-like objects store where they are in the file. If they didn't, then this code:
with open("/home/wizzwizz4/Documents/TOPSECRET/diary.txt") as f:
line = f.readline()
while line:
print(line, end="")
line = f.readline()
would just print the first line over and over again. When data's read from a file, Python won't give you that data again unless you specifically ask for it.
There are multiple games in this file, stored one after each other. You're passing in the same file each time, but you're not resetting the read cursor to the beginning of the file (f.seek(0)) or closing and reopening the file, so it's going to read the next data available – i.e., the next game.
I am trying to open a pdf to get the number of pages. I am using PyPDF2.
Here is my code:
def pdfPageReader(file_name):
try:
reader = PyPDF2.PdfReader(file_name, strict=True)
number_of_pages = len(reader.pages)
print(f"{file_name} = {number_of_pages}")
return number_of_pages
except:
return "1"
But then i run into this error:
PdfReadWarning: Xref table not zero-indexed. ID numbers for objects will be corrected. [pdf.py:1736]
I tried to use strict=True and strict=False, When it is True, it displays this message, and nothing, I waited for 30minutes, but nothing happened. When it is False, it just display nothing, and that's it, just do nothing, if I press ctrl+c on the terminal (cmd, windows 10) then it cancel that open and continues (I run this in a batch of pdf files). Only 1 in the batch got this problem.
My questions are, how do I fix this, or how do I skip this, or how can I cancel this and move on with the other pdf files?
If somebody had a similar problem and it even crashed the program with this error message
File "C:\Programy\Anaconda3\lib\site-packages\PyPDF2\pdf.py", line 1604, in getObject
% (indirectReference.idnum, indirectReference.generation, idnum, generation))
PyPDF2.utils.PdfReadError: Expected object ID (14 0) does not match actual (13 0); xref table not zero-indexed.
It helped me to add the strict argument equal to False for my pdf reader
pdf_reader = PdfReader(input_file, strict=False)
For anybody else who may be running into this problem, and found that strict=False didn't help, I was able to solve the problem by just re-saving a new copy of the file in Adobe Acrobat Reader. I just opened the PDF file inside an actual copy of Adobe Acrobat Reader (the plain ol' free version on Windows), did a "Save as...", and gave the file a new name. Then I ran my script again using the newly saved copy of my PDF file.
Apparently, the PDF file I was using, which were generated directly from my scanner, were somehow corrupt, even though I could open and view it just fine in Reader. Making a duplicate copy of the file via re-saving in Acrobat Reader somehow seemed to correct whatever was missing.
I had the same problem and looked for a way to skip it. I am not a programmer but looking at the documentation about warnings there is a piece of code that helps you avoid such hindrance.
Although I wouldn't recomend this as a solution, the piece of code that I used for my purpose is (just copied and pasted it from doc on link)
import sys
if not sys.warnoptions:
import warnings
warnings.simplefilter("ignore")
This happens to me when the file was created in a printer / scanner combo that generates PDFs. I could read in the PDF with only a warning though so I read it in, and then rewrote it as a new file. I could append that new one.
from PyPDF2 import PdfMerger, PdfReader, PdfWriter
reader = PdfReader("scanner_generated.pdf", strict=False)
writer = PdfWriter()
for page in reader.pages:
writer.add_page(page)
with open("fixedPDF.pdf", "wb") as fp:
writer.write(fp)
merger = PdfMerger()
merger.append("fixedPDF.pdf")
I had the exact same problem, and the solutions did help but didn't solve the problem completely, at least the one setting strict=False & resaving the document using Acrobat reader.
Anyway, I still got a stream error, but I was able to fix it after using an PDF online repair. I used sejda.com but please be aware that you are uploading your PDF on some website, so make sure there is nothing sensible in there.
I'm trying to openpyxl.load_workbook xlsx files from compressed zip file, but it doesn't work. The following code fails at openpyxl.load_workbook with "BadZipfile: File is not a zip file"
with zipfile.ZipFile(os.path.join(root, raw)) as z:
for file_info in z.infolist():
wb = openpyxl.load_workbook(z.open(file_info), read_only=True)
There is nothing wrong with the archive and the excel file in it, as if i extract it to disk then the following works:
with open('report.xlsx') as f:
wb = openpyxl.load_workbook(f, read_only=True)
I can go with this solution and temporary extract it somewhere and load xslx, but would like to understand if it possible to load it from zipfile.
The problem is that readonly=True does not do quite what you think it does. According to the docs:
Fortunately, there are two modes that enable you to read and write unlimited amounts of data with (near) constant memory consumption.
While not explicitly stated, I would assume that this involves some equivalent to a memory-mapped file (because of "constant memory consumption") and random access (because of the range of allowed operations).
Either way, setting readonly=True is not an indication of the fact that you only intend to read a workbook (that's all load_workbook can do anyway, you have to overwrite the existing one to make any "changes"). It is an indication of the fact that you want to access the file directly on disk, without loading the entire contents.
It seems pretty clear (and intuitively expected) that ZipFile.open does not provide a random-access file:
Note: The file-like object is read-only and provides the following methods: read(), readline(), readlines(), __iter__(), next().
The fact that seek is not mentioned in this list is quite telling (pun only somewhat intended).
You can get more information about the exception by splitting the offending line into two (a useful general debugging technique for nested function calls):
x = z.open(file_info)
wb = openpyxl.load_workbook(x, readonly=True)
You will notice that the error occurs on the second of those two lines. This is because pretty much all the Microsoft open-document formats are actually just fancy zip files. The problem is most likely that openpyxl cannot open your file in random access mode, not that it's actually an invalid zip file.
Either way, this is a bunch of very educated guesswork that leads to a simple, one-keyword-deletion solution:
TL;DR
Get rid of readonly=True when reading non-random-access data like a compressed zip entry:
wb = openpyxl.load_workbook(z.open(file_info))
Appendix
You should get in the habit of writing minimal programs that demonstrate your issue so that people answering your question can focus on doing their job instead of getting irritated and closing down what would otherwise be a perfectly good question. I liked your question enough to do that for you, so here is a minimal program that demonstrates your issue and requires nothing more than copy-and-paste to run:
import openpyxl, zipfile
from openpyxl.workbook.workbook import Workbook
wb = Workbook()
wb.active['A1'] = 12
wb.active['A2'] = 13
wb.save('report.xlsx')
with zipfile.ZipFile('test.zip', 'w') as z:
z.write('report.xlsx')
with open('report.xlsx') as f:
wb = openpyxl.load_workbook(f, read_only=True)
print(wb.active['A1'].value)
print(wb.active['A2'].value)
with zipfile.ZipFile('test.zip', 'r') as z:
for file_info in z.infolist():
x = z.open(file_info, 'r')
wb = openpyxl.load_workbook(x, readonly=True)
print(wb.active['A1'].value)
print(wb.active['A2'].value)
So I would try ask over in this thread IronPython - Run an Excel Macro but I don't have enough reputation.
So roughly following the code given in the link I created some code which would save a file to a specific location, then open a workbook that exists there, calling the macro's within it, which would perform a small amount of manipulation on the data which I downloaded to .xls, to make it more presentable.
Now I've isolated the problem to this particular part of the code (below).
Spotfire normally is not that informative but it gives me very little to go on here. It seems to be something .NET related but that's about all I can tell.
The Error Message
Traceback (most recent call last): File
"Spotfire.Dxp.Application.ScriptSupport", line unknown, in
ExecuteForDebugging File "", line unknown, in
StandardError: Exception has been thrown by the target of an
invocation.
The Script
from Spotfire.Dxp.Data.Export import DataWriterTypeIdentifiers
from System.IO import File, Directory
import clr
clr.AddReference("Microsoft.Office.Interop.Excel")
import Microsoft.Office.Interop.Excel as Excel
excel = Excel.ApplicationClass()
excel.Visible = True
excel.DisplayAlerts = False
workbook = ex.Workbooks.Open('myfilelocation')
excel.Run('OpenUp')
excel.Run('ActiveWorkbook')
excel.Run('DoStuff')
excel.Quit()
Right, so I'm answering my own question here but I hope it helps somebody. So the above code, as far as I'm aware was perfectly fine but didn't play well with the way my spotfire environment is configured. However, after going for a much more macro heavy approach I was able to find a solution.
On the spotfire end I've two input fields, one which gives the file path, the other the file name. Then there's a button to run the below script. These are joined together in the script but are crucially separate, as the file name needs to be inputted into a separate file, to be called by the main macro so that it can find the file location of the file.
Fundamentally I created three xml's to achieve this solution. The first was the main one which contained all of the main vba stuff. This would look into a folder in another xml which this script populates to find the save location of the file specified in an input field in spotfire. Finding the most recent file using a custom made function, it would run a simple series of conditional colour operations on the cell values in question.
This script populates two xml files and tells the main macro to run, the code in that macro automatically closing excel after it is done.
The third xml is for a series of colour rules, so user can custom define them depending on their dashboard. This gives it some flexibility for customisation. Not necessary but a potential request by some user so I decided to beat them to the punch.
Anyway, code is below.
from Spotfire.Dxp.Data.Export import DataWriterTypeIdentifiers
from System.IO import File, Directory
import clr
clr.AddReference("Microsoft.Office.Interop.Excel")
import Microsoft.Office.Interop.Excel as Excel
from Spotfire.Dxp.Data.Export import *
from Spotfire.Dxp.Application.Visuals import *
from System.IO import *
from System.Diagnostics import Process
# Input field which takes the name of the file you want to save
name = Document.Properties['NameOfDocument']
# Document property that takes the path
location = Document.Properties['FileLocation']
# just to debug to make sure it parses correctly. Declaring this in the script
# parameters section will mean that the escape characters of "\" will render in a
# unusable way whereas doing it here doesn't. Took me a long time to figure that out.
print(location)
# Gives the file the correct extension.
# Couldn't risk leaving it up to the user.
newname = name + ".xls"
#Join the two strings together into a single file path
finalProduct = location + "\\" + newname
#initialises the writer and filtering schema
writer = Document.Data.CreateDataWriter(DataWriterTypeIdentifiers.ExcelXlsDataWriter)
filtering = Document.ActiveFilteringSelectionReference.GetSelection(table).AsIndexSet()
# Writes to file
stream = File.OpenWrite(finalProduct)
# Here I created a seperate xls which would hold the file path. This
# file path would then be used by the invoked macro to find the correct folder.
names = []
for col in table.Columns:
names.append(col.Name)
writer.Write(stream, table, filtering, names)
stream.Close()
# Location of the macro. As this will be stored centrally
# it will not change so it's okay to hardcode it in.
runMacro = "File location\macro name.xls"
# uses System.Diagnostics to run the macro I declared. This will look in the folder I
# declared above in the second xls, auto run a function in vba to find the most
# up to date file
p = Process()
p.StartInfo.FileName = runMacro
p.Start()
Long story short: To run excel macro's from spotfire one solution is to use the system.dianostics method I use above and simply have your macro set to auto run.
I have two problems using openpyxl
The number of rows in the spreadsheet are 1048498. The iteration hogs memory so I put a logic to check for first five empty columns and break from it
Logic 1 works for me and code does not indefinitely iterate over the spreadsheet blank cells. I am using P4Python to delete this read only file after I am done reading it. However, openpyxl is still using that file and there is no method except save to close the archive used internally. Since my file is in read only mode, I cannot save the file. When P4 is trying to delete this file, I get this error - "The process cannot access the file because it is being used by another process."
Help is appreciated :)
If you open the file in read-only mode then it will not hog memory. Cells are created only when read. Memory use has been tested with huge files but if you think this is a bug then please submit a bug report with a sample file.
This looks like an existing issue or intended beahvior with openpyxl. If you have a read only file (P4Python sync operation - p4.run_sync(file_path_to_sync)) and if you are reading it using openpyxl, you will not be able to delete the file (P4Python p4.run_sync(file_path_to_sync + '#0') - Remove from workspace) until you save the file which is not possible (or intended in my case) since it is a read only file.