I wish to exclude any path from further action, if it includes a string.
Code example:
import os
Dirpath = input('What directory path e.g. C:/ ')
FileType = input('What Ext type to search for e.g. txt ')
for root, dirs, files in os.walk(Dirpath):
for file in files:
if file.endswith(FileType):
print(os.path.join(root, file))
I need to ignore any path with contains Dropbox e.g.
c:/Users\ljh36\Dropbox\Shared Folders\walk.tmp
Can any guidance be given please ?
An example in os.walk displays removing a dir named CVS
from the dirs list. You can adapt this to your code.
The forward slash in the input string can be changed to a
backslash by using r just before the string so you do not
need to escape it with an additional backslash.
import os
Dirpath = input(r'What directory path e.g. C:\ ')
FileType = input('What Ext type to search for e.g. txt ')
for root, dirs, files in os.walk(Dirpath):
# Remove 'Dropbox' from the list of dirs to walk.
if 'Dropbox' in dirs:
dirs.remove('Dropbox')
for file in files:
if file.endswith(FileType):
print(os.path.join(root, file))
Related
I am trying to read all the csv files in the particular set of directories. I have a subdirectories that named as report followed by date like 'report2021-12-22-14_15', 'report2022-01-22-11_10'. I am manually trying to join the path as below
root = os.path.join(base, 'report2021-12-22-14_15' , 'report')
Is there any way I can do a wild search like 'report*' to join the directories so that I will not miss any subdirectories. Below is the snippet
from fnmatch import fnmatch
base = '/Users/user/Desktop/report_files/'
root = os.path.join(base, 'report2021-12-22-14_15','report')
pattern = "report.csv"
for path, files in os.walk(root):
for name in files:
if fnmatch(name, pattern):
file_path = os.path.join(path, name)
So, I am trying to output folder & file structure to a txt file, using the below code. But I want to get relative paths outputed instead of absolute paths.
import os
absolute_path = os.path.dirname(__file__)
with open("output.txt", "w", newline='') as a:
for path, subdirs, files in os.walk(absolute_path):
a.write(path + os.linesep)
for filename in files:
a.write('\t%s\n' % filename)
For now this gives me something like this
C:\Users\User\OneDrive\bla\bla
C:\Users\User\OneDrive\bla\bla\folder1
file1.xxx
file2.xxx
file3.xxx
C:\Users\User\OneDrive\bla\bla\folder2
test1.txt
but I want to show only relative paths to where the script ran, not more
.\bla
.\bla\folder1
file1.xxx
file2.xxx
file3.xxx
.\bla\folder2
test1.txt
I have fiddled around a bit, but not getting to the solution, nor finding it here (or maybe I am not searching for the correct thing)
Any help would be appreciated
If you know the path to you current module, and you know the path of each file you find, and all your files will be in subdirectories of the current directory, you can calculate the relative path yourself.
Strings have the .replace(string_to_replace, replacement_value) method which will do this for you.
import os
absolute_path = os.path.dirname(__file__)
with open("output.txt", "w", newline='') as a:
for path, subdirs, files in os.walk(absolute_path):
a.write(path.replace(absolute_path, '.') + os.linesep)
for filename in files:
a.write('\t%s\n' % filename)
Beauty of python is, it os.path module ready for this kind of problems. You can use os.path.relpath() function
import os
absolute_path = os.path.dirname(__file__)
with open("output.txt", "w", newline='') as a:
for path, subdirs, files in os.walk(absolute_path):
a.write('.\\' + os.path.relpath(path + os.linesep, start=absolute_path))
for filename in files:
a.write('\t%s\n' % filename)
Here start parameter is used to current directory from where relative path should be resolved.
I am trying to read folders in one folder, and analyse the daten in these folders. My idea is to write two loop: one to choose the folder, one to analyse daten in the chosen folder. But i don't know how can i write the new path with iterate variable. line3 of this snapshot is my problem.
Thank you for any input.
If you only want to get the folders inside another folder (subfolders) and aren't interested in other files that could be in the root folder you could use:
import os
path = '...'
for file_folder in os.listdir(path):
path1 = os.path.join(path, file_folder)
if os.path.isdir(path1):
print(path1)
# do something
or
import glob
path = '...'
for folder in glob.glob(f'{path}/*/'):
# do something
In line number two & three from the image, you have written:
for file_folder in os.listdir(path):
path1 = 'path' + '\' + str(file_folder)
file_folder is like a temporary variable holding folder name value for a particular iteration.
It should instead be:
for file_folder in os.listdir(path):
path1 = 'path' + '\\' + file_folder
I have one scenario where i have to rename the files in the folder. Please find the scenario,
Example :
Elements(Main Folder)<br/>
2(subfolder-1) <br/>
sample_2_description.txt(filename1)<br/>
sample_2_video.avi(filename2)<br/>
3(subfolder2)
sample_3_tag.jpg(filename1)<br/>
sample_3_analysis.GIF(filename2)<br/>
sample_3_word.docx(filename3)<br/>
I want to modify the names of the files as,
Elements(Main Folder)<br/>
2(subfolder1)<br/>
description.txt(filename1)<br/>
video.avi(filename2)<br/>
3(subfolder2)
tag.jpg(filename1)<br/>
analysis.GIF(filename2)<br/>
word.docx(filename3)<br/>
Could anyone guide on how to write the code?
Recursive directory traversal to rename a file can be based on this answer. All we are required to do is to replace the file name instead of the extension in the accepted answer.
Here is one way - split the file name by _ and use the last index of the split list as the new name
import os
import sys
directory = os.path.dirname(os.path.realpath("/path/to/parent/folder")) #get the directory of your script
for subdir, dirs, files in os.walk(directory):
for filename in files:
subdirectoryPath = os.path.relpath(subdir, directory) #get the path to your subdirectory
filePath = os.path.join(subdirectoryPath, filename) #get the path to your file
newFilePath = filePath.split("_")[-1] #create the new name by splitting the old name by _ and grabbing last index
os.rename(filePath, newFilePath) #rename your file
Hope this helps.
check below code example for the first filename1, replace path with the actual path of the file:
import os
os.rename(r'path\\sample_2_description.txt',r'path\\description.txt')
print("File Renamed!")
I have a folder named "data". It contains subfolders "data_1", "data_2", and "data_3". These subfolders contain some text files. I want to parse through all these subfolders and generate corresponding subfolders with the same name, inside another folder named "processed_data". I want to also generate corresponding files with "processed" as a prefix in the name and want to write all those lines from the original file where "1293" is there in the original files.
I am using the below code but not able to get the required result. Neither the subfolders "data_1", "data_2", and "data_3" nor the files are getting created
import os
folder_name=""
def pre_processor():
data_location="D:\data" # folder containing all the data
for root, dirs, files in os.walk(data_location):
for dir in dirs:
#folder_name=""
folder_name=dir
for filename in files:
with open(os.path.join(root, filename),encoding="utf8",mode="r") as f:
processed_file_name = 'D:\\processed_data\\'+folder_name+'\\'+'processed'+filename
processed_file = open(processed_file_name,"w", encoding="utf8")
for line_number, line in enumerate(f, 1):
if "1293" in line:
processed_file.write(str(line))
processed_file.close()
pre_processor()
You might need to elaborate on the issue you are having; e.g., are the files being created, but empty?
A few things I notice:
1) Your indentation is off (not sure if this is just a copy-paste issue though): the pre_processor function is empty, i.e. you are defining the function at the same level as the declaration, not inside of it.
try this:
import os
folder_name=""
def pre_processor():
data_location="D:\data" # folder containing all the data
for root, dirs, files in os.walk(data_location):
for dir in dirs:
#folder_name=""
folder_name=dir
for filename in files:
with open(os.path.join(root, filename), encoding="utf8",mode="r") as f:
processed_file_name = 'D:\\processed_data\\'+folder_name+'\\'+'processed'+filename
processed_file = open(processed_file_name,"w", encoding="utf8")
for line_number, line in enumerate(f, 1):
if "1293" in line:
processed_file.write(str(line))
processed_file.close()
pre_processor()
2) Check if the processed_data and sub_folders exist; if not, create them first as this will not do so.
Instead of creating the path to the new Folder by hand you could just replace the name of the folder.
Furthermore, you are not creating the subfolders.
This code should work but replace the Linux folder slashes:
import os
folder_name=""
def pre_processor():
data_location="data" # folder containing all the data
for root, dirs, files in os.walk(data_location):
for dir in dirs:
# folder_name=""
folder_name = dir
for filename in files:
joined_path = os.path.join(root, filename)
with open(joined_path, encoding="utf8", mode="r") as f:
processed_folder_name = root.replace("data/", 'processed_data/')
processed_file_name = processed_folder_name+'/processed'+filename
if not os.path.exists(processed_folder_name):
os.makedirs(processed_folder_name)
processed_file = open(processed_file_name, "w", encoding="utf8")
for line in f:
if "1293" in line:
processed_file.write(str(line))
processed_file.close()
pre_processor()