In my webscraping project I collect different types of data from the internet, which are added to an excel file.
Until recently, my back up method of this excel file consisted of copying it to a different folder, using Shutil. I updated the script in a more proper way so that the filename includes the corresponding year and month too.
Example of original file: data.xlsx
Name of backup file: 2021-03_data.xlsx
No magic whatsoever, but ever since, the script raises an OS Error, stating that the operation is not supported. Interestingly, the file is copied perfectly... But the error bugs me since it breaks the script and does not execute what follows.
This is the code:
#Export to Excel
final_df.to_excel(last_search, index=False)
#Backup
yearBackup = str(datetime.now().year)
monthBackup = str('{:02d}'.format(datetime.now().month)) #2 digits
pathBackup = Path(os.path.dirname(__file__), 'backup' , f'{yearBackup}-{monthBackup}_data.xlsx')
shutil.copy(last_search, pathBackup)
The error message refers to line 146:
shutil.copy(last_search, pathBackup)
This is the error:
Traceback (most recent call last):
File "/run/user/1000/gvfs/smb-share:server=snowserver,share=home/Drive/Drive/Python/Webscraping/projects/online/online-data.py", line 146, in <module>
shutil.copy(last_search, pathBackup)
File "/usr/lib/python3.8/shutil.py", line 416, in copy
copymode(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.8/shutil.py", line 305, in copymode
chmod_func(dst, stat.S_IMODE(st.st_mode))
OSError: [Errno 95] Operation not supported: '/run/user/1000/gvfs/smb-share:server=snowserver,share=home/Drive/Drive/Python/Webscraping/projects/online/backup/2021-03_data.xlsx'
Does anyone know how to solve this?
Related
<<<fixed by getting Tika 1.25 .jar files and changing the TIKA_SERVER_JAR environment variable to file:\\C:\CODING\Apache Tika\tika-server-1.25.jar thanks to all involved>>>
I saw this thread Python Tika error: URLError: <urlopen error unknown url type: c>
This same thing is happening to me right now. And it's frustrating.
I made a post in that thread but I was told to make a new thread by somebody in comments. So here it is.
I Googled and looked through StackOverflow but I found nothing that seems on point.
python 3.9.2 (very recent upgrade (say 03/08/21) from python 3.8.2 or something similar I forget)
tika 1.24 (python package)
tika 1.24.1 (jar file (has been working just fine))
Originally I had set the powershell $env: variable to point to tika.jar file which was working fine until recently. On startup in python tika would make the python interpreter search for the tika REST api to get the latest parser; so I pointed the $env: variable to the tika.jar file so it would think to search...find the tika.jar file on my machine and use that instead.
Now I'm getting this error message below...
2021-03-12 18:14:46,032 [MainThread ] [INFO ] Retrieving
C:\PATH\TO\tika-server-1.24.1.jar to
C:\PATH\TO\Local\Temp\tika-server.jar. Traceback (most
recent call last): File
"C:\CODING\Python392\lib\site-packages\tika\tika.py", line 798, in
getRemoteJar
urlretrieve(urlOrPath, destPath) File "C:\CODING\Python392\lib\urllib\request.py", line 239, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp: File "C:\CODING\Python392\lib\urllib\request.py", line 214, in urlopen
return opener.open(url, data, timeout) File "C:\CODING\Python392\lib\urllib\request.py", line 517, in open
response = self._open(req, data) File "C:\CODING\Python392\lib\urllib\request.py", line 539, in _open
return self._call_chain(self.handle_open, 'unknown', File "C:\CODING\Python392\lib\urllib\request.py", line 494, in _call_chain
result = func(*args) File "C:\CODING\Python392\lib\urllib\request.py", line 1417, in
unknown_open
raise URLError('unknown url type: %s' % type) urllib.error.URLError:
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "c:\PATH\TO#tika test.py", line 75, in
TIKA_FUNCTION(folder) File "c:\PATH\TO#tika test.py", line 38, in TIKA_FUNCTION
rawtext = parser.from_file(filename) File "C:\CODING\Python392\lib\site-packages\tika\parser.py", line 40, in
from_file
output = parse1(service, filename, serverEndpoint, headers=headers, config_path=config_path,
requestOptions=requestOptions) File
"C:\CODING\Python392\lib\site-packages\tika\tika.py", line 336, in
parse1
status, response = callServer('put', serverEndpoint, service, f, File "C:\CODING\Python392\lib\site-packages\tika\tika.py", line 531,
in callServer
serverEndpoint = checkTikaServer(scheme, serverHost, port, tikaServerJar, classpath, config_path) File
"C:\CODING\Python392\lib\site-packages\tika\tika.py", line 592, in
checkTikaServer
getRemoteJar(tikaServerJar, jarPath) File "C:\CODING\Python392\lib\site-packages\tika\tika.py", line 808, in
getRemoteJar
urlretrieve(urlOrPath, destPath) File "C:\CODING\Python392\lib\urllib\request.py", line 239, in urlretrieve
with contextlib.closing(urlopen(url, data)) as fp: File "C:\CODING\Python392\lib\urllib\request.py", line 214, in urlopen
return opener.open(url, data, timeout) File "C:\CODING\Python392\lib\urllib\request.py", line 517, in open
response = self._open(req, data) File "C:\CODING\Python392\lib\urllib\request.py", line 539, in _open
return self._call_chain(self.handle_open, 'unknown', File "C:\CODING\Python392\lib\urllib\request.py", line 494, in _call_chain
result = func(*args) File "C:\CODING\Python392\lib\urllib\request.py", line 1417, in
unknown_open
raise URLError('unknown url type: %s' % type) urllib.error.URLError:
I saw all the comments to the original post. (of the original thread)
I changed the windows defender firewall to accept the tika.jar file as per one of the comments.
Didn't work.
I've been changing the $env: variable to include things like "file:///C:/tika/path.jar" etc (tried both f:/// and f://). Even outright removing the $env: variable so it gets the latest parser info or whatever.
Didn't work.
I'm trying to figure out what all this is about in this thread https://bugs.python.org/issue6193
But this seems to be a little bit of a disparate comparison for my situation because the issue in python.org thread is user asking urllib to open a file in script whereas I am not doing this exactly.
Now it's possible that the fix described in python.org thread is over my head with the file:/// command because I'm not the best coder, I'm not sure I understand it that well.
In the python.org thread discussion everyone is talking like "just do this file:/// thing" but no one is really explaining it in a way I can understand.
There is an inter-thread reference to another link in regards to urllib splittype method, but the link is dead.
But it still seems like a disparate comparison because if urllib just couldn't handle opening the file, the whole process never would have worked to begin with when I originally set it up some months ago.
Which leads me to believe the issue really is that installing python 3.9.2 and uninstalling the 3.8.2 broke the scripts ability to work. But that seems wrong somehow. (And if it's not, how can I fix it? reinstall tika.jar with Maven?)
I suppose it's possible that the default urllib module was updated or something but then that python.org thread is from 2009. It is 2021. In 2020 my program script was working just fine.
My next step will be...to investigate this little...splittype urllib method.
But will it actually yield anything? Not so sure.
Any assistance with reasoning, methods, or code would be appreciated.
Thank you very much in advance.
edit: tika --dump-current-config below
Mar 14, 2021 10:15:23 PM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: J2KImageReader not loaded. JPEG2000 files will not be processed.
See https://pdfbox.apache.org/2.0/dependencies.html#jai-image-io
for optional dependencies.
Mar 14, 2021 10:15:24 PM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: Tesseract OCR is installed and will be automatically applied to image files unless
you've excluded the TesseractOCRParser from the default parser.
Tesseract may dramatically slow down content extraction (TIKA-2359).
As of Tika 1.15 (and prior versions), Tesseract is automatically called.
In future versions of Tika, users may need to turn the TesseractOCRParser on via TikaConfig.
Mar 14, 2021 10:15:24 PM org.apache.tika.config.InitializableProblemHandler$3 handleInitializableProblem
WARNING: org.xerial's sqlite-jdbc is not loaded.
Please provide the jar on your classpath to parse sqlite files.
See tika-parsers/pom.xml for the correct version.
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<properties>
<!--for example: <mimeTypeRepository resource="/org/apache/tika/mime/tika-mimetypes.xml"/>-->
<service-loader dynamic="true" loadErrorHandler="IGNORE"/>
<encodingDetectors>
<encodingDetector class="org.apache.tika.detect.DefaultEncodingDetector"/>
</encodingDetectors>
<translator class="org.apache.tika.language.translate.DefaultTranslator"/>
<detectors>
<detector class="org.apache.tika.detect.DefaultDetector"/>
</detectors>
<parsers>
<parser class="org.apache.tika.parser.DefaultParser"/>
</parsers>
</properties>
tika-config.xml from C:\PATH\TO\tika\parser\pdf
<?xml version="1.0" encoding="UTF-8"?>
<!--Licensed to the Apache Software Foundation (...cutout...)
-<properties>
-<parsers>
-<parser class="org.apache.tika.parser.pdf.PDFParser">
-<params>
<param type="bool" name="sortByPosition">true</param>
</params>
</parser>
</parsers>
</properties>
**So i wish to create a zip file named new.zip using python which also contains a text file called sample.txt. But after running the necessary code on my python editor i get the error as :
Traceback (most recent call last):
File "C:/Users/User/PycharmProjects/PYTHON_CODE/creatingZipFile.py", line 5, in
newZip.write('sample.txt', compress_type=zipfile.ZIP_DEFLATED)
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\zipfile.py", line 1713, in write
zinfo = ZipInfo.from_file(filename, arcname)
File "C:\Users\User\AppData\Local\Programs\Python\Python37-32\lib\zipfile.py", line 506, in from_file
st = os.stat(filename)
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'sample.txt'
My python code is as below :
import os, zipfile
os.chdir("F:\\")
newZip = zipfile.ZipFile('new.zip', 'a')
newZip.write('sample.txt', compress_type=zipfile.ZIP_DEFLATED)
newZip.close()
PLEASE SOMEONE HELP ME REGARDING THIS ISSUE.**
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'sample.txt'
will occur if the file is not present in the specified path, can you check if sample.txt is actually present in the path F:\sample.txt
I am making a program using tkinter that executes some code from different files.
This is the part of my code causing problems:
text=textbox.get('1.0','end-1c')
FYI: textbox is a tkinter.Text object defined by a different file that my code executed
when ever I execute my code, I get the error:
Exception in Tkinter callback
Traceback (most recent call last):
File "/anaconda3/lib/python3.7/tkinter/__init__.py", line 1705, in __call__
return self.func(*args)
File "/Users/iskee/python/programs/atfedit/atf.py", line 98, in save
text=textbox.get('1.0','end-1c')
File "/anaconda3/lib/python3.7/tkinter/__init__.py", line 3246, in get
return self.tk.call(self._w, 'get', index1, index2)
_tkinter.TclError: invalid command name ".!text"
what is going wrong?
That error means that you're trying to access a widget that has been destroyed.
I have an existing LMDB zarr archive (~6GB) saved at path. Now I want to consolidate the metadata to improve read performance.
Here is my script:
store = zarr.LMDBStore(path)
root = zarr.open(store)
zarr.consolidate_metadata(store)
store.close()
I get the following error:
Traceback (most recent call last):
File "zarr_consolidate.py", line 12, in <module>
zarr.consolidate_metadata(store)
File "/local/home/marcel/.virtualenvs/noisegan/local/lib/python3.5/site-packages/zarr/convenience.py", line 1128, in consolidate_metadata
return open_consolidated(store, metadata_key=metadata_key)
File "/local/home/marcel/.virtualenvs/noisegan/local/lib/python3.5/site-packages/zarr/convenience.py", line 1182, in open_consolidated
meta_store = ConsolidatedMetadataStore(store, metadata_key=metadata_key)
File "/local/home/marcel/.virtualenvs/noisegan/local/lib/python3.5/site-packages/zarr/storage.py", line 2455, in __init__
d = store[metadata_key].decode() # pragma: no cover
AttributeError: 'memoryview' object has no attribute 'decode'
I am using zarr 2.3.2 and python 3.5.2. I have another machine running python 3.6.2 where this works. Could it have to do with the python version?
Thanks for the report. Should be fixed with gh-452. Please test it out (if you are able).
If you are able to share a bit more information on why read performance suffers in your case, that would be interesting to learn about. :)
I have a short python script (called VaultTransferScript.py) that should transfer a zip file from one machine to another. The destination machine is a mapped network-attached-storage machine, which I have assigned to be the Z: drive.
The script is:
import shutil
import os
from datetime import datetime
time_stamp = datetime.now().strftime('%Y-%m-%d_%H_%M')
title_str = 'VaultBackup.zip'
name = time_stamp + title_str
shutil.move('C:\\Users\\Hawking\\Desktop\\VaultBackups\\MyBackup.zip',
os.path.join('Z:\\VaultBackups\\'+name))
I can run this script from the notepad++ run facility, using
cmd /C python "$(FULL_CURRENT_PATH)"
But running it in a batch script as:
echo off
C:\Users\Hawking\AppData\Local\Programs\Python\Python37-32\python.exe C:\Users\Hawking\Desktop\VaultBackupTransfer.py
results in this:
C:\Users\Hawking\Desktop>echo off
Traceback (most recent call last):
File "C:\Users\Hawking\AppData\Local\Programs\Python\Python37-32\lib\shutil.py", line 557, in move
os.rename(src, real_dst)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'C:\\Users\\Hawking\\Desktop\\VaultBackups\\MyBackup.zip' -> 'Z:\\VaultBackups\\2018-09-21_14_30VaultBackup.zip'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Hawking\Desktop\VaultBackupTransfer.py", line 7, in <module>
shutil.move('C:\\Users\\Hawking\\Desktop\\VaultBackups\\MyBackup.zip', os.path.join('Z:\\VaultBackups\\'+name))
File "C:\Users\Hawking\AppData\Local\Programs\Python\Python37-32\lib\shutil.py", line 571, in move
copy_function(src, real_dst)
File "C:\Users\Hawking\AppData\Local\Programs\Python\Python37-32\lib\shutil.py", line 257, in copy2
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "C:\Users\Hawking\AppData\Local\Programs\Python\Python37-32\lib\shutil.py", line 121, in copyfile
with open(dst, 'wb') as fdst:
FileNotFoundError: [Errno 2] No such file or directory: 'Z:\\VaultBackups\\2018-09-21_14_30VaultBackup.zip'
WHat is the difference in how I invoke the python script, and why does it error out from the batch script, but not notepad++?
You might be running the Python program with different user permissions in Notepad++ versus the command prompt. Alternatively, another Python VM might be used. Although, nothing in particular makes me think that the later is true.