If I am trying to copy files to a subdir, as:
dirname = os.path.join(sys.argv[1], optdir)
print("dirname: "+dirname)
if not os.path.exists(dirname):
os.makedirs(dirname)
shutil.copy(files, dirname)
shutil.copy is giving error as:
dirname: ./8/opt2
Traceback (most recent call last):
File "/home/rudra/bin/latres.py", line 84, in <module>
shutil.copy(files, dirname)
File "/usr/lib64/python3.5/shutil.py", line 234, in copy
dst = os.path.join(dst, os.path.basename(src))
File "/usr/lib64/python3.5/posixpath.py", line 139, in basename
i = p.rfind(sep) + 1
AttributeError: 'list' object has no attribute 'rfind'
Which is possibly due to dst = os.path.join(dst, os.path.basename(src)) in the error msg, so, it is only getting opt2, and not the ./8 part of the dir name.
in this situation, how can I copy files to a subdir?
files is a list of file names but copy only copes a single file. So put it in a loop:
for fn in files:
shutil.copy(fn, dirname)
Related
I try to read all excel files in a directory to merge them into one df, which seems to work fine (the dataframe is correctly created)
How ever it seems that python is trying perform the loop a second time after all the files have been read, but in different location where the excel files do not exist and I get a traceback for 'file not found' which is obvious cause the files are not saved in this location.
It jumps from 'C:/users/folder/folder' to 'C:/users' and tries to loop again over all files, any idea why this happens?
The code I am using is the following:
import pandas as pd
import xlrd
import os
path = os.getcwd()
files = os.listdir(path)
files_xlsx = [f for f in files if f[-4:] == 'xlsx']
df = pd.DataFrame()
for f in files_xlsx:
data = pd.read_excel(f)
df = df.append(data)
print(df)
Here is the full traceback I get:
C:\ProgramData\Anaconda3\envs\pythonProject\python.exe "C:\Program Files (x86)\JetBrains\PyCharm 2020.2.1\plugins\python\helpers\pydev\pydevd.py" --multiproc --qt-support=auto --client 127.0.0.1 --port 58606 --file "C:/Users/Home/OneDrive/Ops/Media Management/Media Kitchen/Rezepte aktuell vom 10.05.2021/Rezepte Alphabetisch xlsx/XLSX merg.py"
pydev debugger: process 22124 is connecting
Connected to pydev debugger (build 202.6948.78)
C:\ProgramData\Anaconda3\envs\pythonProject\lib\site-packages\openpyxl\worksheet\header_footer.py:48: UserWarning: Cannot parse header or footer so it will be ignored
warn("""Cannot parse header or footer so it will be ignored""")
...
C:\ProgramData\Anaconda3\envs\pythonProject\lib\site-packages\openpyxl\worksheet\header_footer.py:48: UserWarning: Cannot parse header or footer so it will be ignored
warn("""Cannot parse header or footer so it will be ignored""")
Traceback (most recent call last):
File "C:\Program Files (x86)\JetBrains\PyCharm 2020.2.1\plugins\python\helpers\pydev\pydevd.py", line 1448, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "C:\Program Files (x86)\JetBrains\PyCharm 2020.2.1\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "C:/Users/Home/OneDrive/Ops/Media Management/Media Kitchen/Rezepte aktuell vom 10.05.2021/Rezepte Alphabetisch xlsx/XLSX merg.py", line 46, in <module>
data = pd.read_excel(f)
File "C:\ProgramData\Anaconda3\envs\pythonProject\lib\site-packages\pandas\util\_decorators.py", line 299, in wrapper
return func(*args, **kwargs)
File "C:\ProgramData\Anaconda3\envs\pythonProject\lib\site-packages\pandas\io\excel\_base.py", line 336, in read_excel
io = ExcelFile(io, storage_options=storage_options, engine=engine)
File "C:\ProgramData\Anaconda3\envs\pythonProject\lib\site-packages\pandas\io\excel\_base.py", line 1071, in __init__
ext = inspect_excel_format(
File "C:\ProgramData\Anaconda3\envs\pythonProject\lib\site-packages\pandas\io\excel\_base.py", line 949, in inspect_excel_format
with get_handle(
File "C:\ProgramData\Anaconda3\envs\pythonProject\lib\site-packages\pandas\io\common.py", line 651, in get_handle
handle = open(handle, ioargs.mode)
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\$ACHR3-00.00.xlsx'
python-BaseException
I'm trying to go through a list of items and passing each one to a function one by one to create an Excel file with the same name as the argument passed. I am getting the error below which I believe is related to the '/' in the String name. Can anyone advise how I get it to ignore this?
>>> test.createExcel(filename)
Traceback (most recent call last):
File "<pyshell#97>", line 1, in <module>
test.createExcel(filename)
File "C:\Users\danie\OneDrive\JVC\project1.py", line 52, in createExcel
wb2.save(modelname+'.xlsx')
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\openpyxl\workbook\workbook.py", line 392, in save
save_workbook(self, filename)
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\site-packages\openpyxl\writer\excel.py", line 291, in save_workbook
archive = ZipFile(filename, 'w', ZIP_DEFLATED, allowZip64=True)
File "C:\Users\danie\AppData\Local\Programs\Python\Python37\lib\zipfile.py", line 1240, in __init__
self.fp = io.open(file, filemode)
FileNotFoundError: [Errno 2] No such file or directory: '14 A4/32GB BLU.xlsx'
A filename cannot contain any of the following characters: \ / : * ? " < > |
In ur caseļ¼ u could replace ur filename using str.replace('/','-') or any other character u'd like to.
eg:
wb.save(filename.replace('\','-'))
Or using the regular expression to replace it may work well.
I have thousands of files as you can see the year range below. Some of the dates of the files are missing so I want to skip over them. But when I tried the method below, and calling data_in, the variable doesn't exist. Any help would be truly appreciated. I am new to python. Thank you.
path = r'file path here'
DataYears = ['2012','2013','2014', '2015','2016','2017','2018','2019', '2020']
Years = np.float64(DataYears)
NumOfYr = Years.size
DataMonths = ['01','02','03','04','05','06','07','08','09','10','11','12']
daysofmonth=[31,28,31,30,31,30,31,31,30,31,30,31]
for yy in range(NumOfYr):
for mm in range (12):
try:
data_in = pd.read_csv(path+DataYears[yy]+DataMonths[mm]+'/*.dat', skiprows=4, header=None, engine='python')
print('Reached data_in') # EDIT
a=data_in[0] #EDIT
except IOError:
pass
#print("File not accessible")
EDIT: Error added
Traceback (most recent call last):
File "Directory/Documents/test.py", line 23, in <module>
data_in = pd.read_csv(path+'.'+DataYears[yy]+DataMonths[mm]+'/*.cod', skiprows=4, header=None, engine='python')
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 676, in parser_f
return _read(filepath_or_buffer, kwds)
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 448, in _read
parser = TextFileReader(fp_or_buf, **kwds)
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 880, in __init__
self._make_engine(self.engine)
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1126, in _make_engine
self._engine = klass(self.f, **self.options)
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 2269, in __init__
memory_map=self.memory_map,
File "/opt/anaconda3/lib/python3.7/site-packages/pandas/io/common.py", line 431, in get_handle
f = open(path_or_buf, mode, errors="replace", newline="")
FileNotFoundError: [Errno 2] No such file or directory: 'Directory/Documents/201201/*.dat'
You can adapt the code below to get a list of your date folders:
import glob
# Gives you a list of your folders with the different dates
folder_names = glob.glob("Directory/Documents/")
print(folder_names)
Then with the list of folder, you can iterate through there contents. If you just want a list of all .dat files can do something like:
import glob
# Gives you a list of your folders with the different dates
file_names = glob.glob("Directory/Documents/*/*.dat")
print(file_names)
The code above searches the contents of your directories so you bypass your problem with missing dates. The prints are there so you can see the results of glob.glob().
Im trying to copy a file using shutil by reading in a config.dat file.
This file is simply:
/home/admin/Documents/file1
/home/admin/Documents/file2
Code does work but it will copy file1 okay but then misses file2 because it see a \n there, which im guessing is because of the new line.
#!/usr/bin/python3
import shutil
data = open("config.dat")
filelist = data.read()
src = filelist
dest = '/home/admin/Documents/backup/'
shutil.copy(src, dest)
Error code im getting :
Traceback (most recent call last):
File "./testing.py", line 18, in <module>
shutil.copy(src, dest)
File "/usr/lib/python3.4/shutil.py", line 229, in copy
copyfile(src, dst, follow_symlinks=follow_symlinks)
File "/usr/lib/python3.4/shutil.py", line 108, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory:
'/home/admin/Documents/file1\n/home/admin/Documents/file2'
I would like the copy of those files to run based on the files from the config.dat folder but it detects a '\n'. Is there a way to fix this?
thanks for the help
Use split('\n') to get a list of files and the iterate that list. You probably want to throw in the file.strip() to get rid of trailing whitespace and empty lines.
import shutil
dest = '/home/admin/Documents/backup/'
with open('config.dat') as data:
filelist = data.read().split('\n')
for file in filelist:
if file:
shutil.copy(file.strip(), dest)
Or, if you don't need the filelist after this
with open('config.dat') as data:
for file in data:
if file:
shutil.copy(file.strip(), dest)
Hello I can't load a hdf5 file with h5py:
$ python verif.py
Traceback (most recent call last):
File "verif.py", line 4, in <module>
h5f = h5py.File("../DeepFISH-Github_projects/DeepFISH/dataset/'+'LowRes_13434_overlapping_pairs.h5",'r')
File "/home/jeanpat/VirtualEnv/venv3/lib/python3.5/site-packages/h5py/_hl/files.py", line 272, in __init__
fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)
File "/home/jeanpat/VirtualEnv/venv3/lib/python3.5/site-packages/h5py/_hl/files.py", line 92, in make_fid
fid = h5f.open(name, flags, fapl=fapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper (/tmp/pip-at6d2npe-build/h5py/_objects.c:2684)
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper (/tmp/pip-at6d2npe-build/h5py/_objects.c:2642)
File "h5py/h5f.pyx", line 76, in h5py.h5f.open (/tmp/pip-at6d2npe-build/h5py/h5f.c:1930)
OSError: Unable to open file (Unable to open file: name = '../deepfish-github_projects/deepfish/dataset/'+'lowres_13434_overlapping_pairs.h5', errno = 2, error message = 'no such file or directory', flags = 0, o_flags = 0
The string containing the path to the file:
../DeepFISH-Github_projects/DeepFISH/dataset'+'LowRes_13434_overlapping_pairs.h5
seems to be modified by h5py
../deepfish-github_projects/deepfish/dataset/lowres_13434_overlapping_pairs.h5
I could modify the directory name, but it's weird.
In this line
h5f = h5py.File("../DeepFISH-Github_projects/DeepFISH/dataset/'+'LowRes_13434_overlapping_pairs.h5",'r')
you're trying to open a file with a literal '+' in its name. The outer quotes are double quotes, so the single quotes within the string are just part of the name. What you probably wanted to use is:
h5f = h5py.File("../DeepFISH-Github_projects/DeepFISH/dataset/" + "LowRes_13434_overlapping_pairs.h5",'r')
I don't know why the error message is all lower case, maybe the library tries to find the file in a case insensitive way if it doesn't find it by the original name, or the underlying file system is case insensitive and this is just how the OS reports the missing file error.