Correct way to use args parser subparser set defaults with imported functions? - python-3.x

I currently have a bunch of stand alone functions that I am trying to implement into a CLI program using argsparse.
The standard way to incorporate functions as command line args seems to be as follows (from the blogs I read):
from consolidate_all_files import consolidate_all_files #my function
import argparse
parser=argparse.ArgumentParser(description="functions for file processing")
subparsers = parser.add_subparsers()
consolidate_parser = subparsers.add_parser("consolidate_files")
consolidate_parser.add_argument('src')
consolidate_parser.add_argument('dest')
consolidate_parser.set_defaults(func=consolidate_all_files)
I don't quite understand how consolidate_all_files will be called with set_defaults.
My consolidate_all_files function looks something like this:
import os
import shutil
def consolidate_all_files(MAIN,DIRS)
for root, subdirs, files in os.walk(DIRS):
'''Basically we move all files from subfolders into one folder '''
print('root', root)
print('subdirs', subdirs)
print('files', files)
for file in files:
path = os.path.join(root, file)
try:
shutil.move(path, MAIN)
except shutil.Error as e:
continue
Most examples I have seen on the web, seem to have their function parameters/arguments revolve around parser args. So would I need to go through each function and change its parameters to something like:
def consolidate_all_files(args)
for root, subdirs, files in os.walk(args.src):
'''Basically we move all files from subfolders into one folder '''
print('root', root)
print('subdirs', subdirs)
print('files', files)
for file in files:
path = os.path.join(root, file)
try:
shutil.move(path, dest)
except shutil.Error as e:
continue
or is there a way to get my original functions to become compatible with subparser's set defaults.
Thank you.

Related

Using Path to check if file exists when running script outside of the directory

So I currently use Path to check if a file exists
from pathlib import Path
if Path("main.conf").is_file():
pass
else:
setup_config()
While this works as long as I'm in the directory where I'm running the script, I'd like to make it work whatever directory I'm at and just run the script. I know it doesn't work because it's expecting the main.conf to be in the directory I'm currently in but how do I tell path to only check the in the folder where the script is located in?
You can resolve the absolute path of the script by using sys.argv[0] and then replace the name of that script with the config file to check, eg:
import sys
import pathlib
path = path.Pathlib(sys.argv[0]).resolve()
if path.with_name('main.conf').is_file():
# ...
else:
# ...
Although it seems like you should probably not worry about that check and structure your setup_config so it takes a filename as an argument, eg:
def setup_config(filename):
# use with to open file here
with open(filename) as fin:
# do whatever for config setup
Then wrap your main in a try/except (which'll also cover file not exists/can't open file for other reasons), eg:
path = pathlib.Path(sys.argv[0]).resolve()
try:
setup_config(path.with_name('main.conf'))
except IOError:
pass

Does the following program access a file in a subfolder of a folder?

using
import sys
folder = sys.argv[1]
for i in folder:
for file in i:
if file == "test.txt":
print (file)
would this access a file in the folder of a subfolder? For Example 1 main folder, with 20 subfolders, and each subfolder has 35 files. I want to pass the folder in commandline and access the first subfolder and the second file in it
Neither. This doesn't look at files or folders.
sys.argv[1] is just a string. i is the characters of that string. for file in i shouldn't work because you cannot iterate a character.
Maybe you want to glob or walk a directory instead?
Here's a short example using the os.walk method.
import os
import sys
input_path = sys.argv[1]
filters = ["test.txt"]
print(f"Searching input path '{input_path}' for matches in {filters}...")
for root, dirs, files in os.walk(input_path):
for file in files:
if file in filters:
print("Found a match!")
match_path = os.path.join(root, file)
print(f"The path is: {match_path}")
If the above file was named file_finder.py, and you wanted to search the directory my_folder, you would call python file_finder.py my_folder from the command line. Note that if my_folder is not in the same directory as file_finder.py, then you have to provide the full path.
No, this won't work, because folder will be a string, so you'll be iterating through the characters of the string. You could use the os module (e.g., the os.listdir() method). I don't know what exactly are you passing to the script, but probably it would be easiest by passing an absolute path. Look at some other methods in the module used for path manipulation.

Moving all files out from a directory and all its subdirectories

I have a small program that moves all files out of a directory and then searches all subdirectories for other files which it also moves out.
import shutil
import os
import ctypes
import sys
copyfrom = r'D:\Downloads\'
copyto = r'D:\Downloads\'
for r, d, f in os.walk(copyfrom):
for file in f:
if os.path.join(r, file) == copyto:
continue
print(os.path.join(r, file))
shutil.move(os.path.join(r, file), os.path.join(copyto, file))
It works right now but will overwrite every file that has a filename of an existing file. For example if i have banana.mp3 and banana.jpeg it will overwrite one of the files. Instead i would like the file with an existing name to be renamed.
You may check whether the file exists using os.path.exists(destination) . But you should make sure that no race condition is going to happen. So you may open the existing file using a command like os.open(), do your work and then close the file.

Message <class WindowError> in reading recursavely a directory

I started to study Python 3 and, for fun, I'm trying to code a small software that, from a directory root, read recursively the filesystem and stores some informations about files in a csv file, but I'm receiving this exception
<class 'WindowsError'>
that I don't understand well. It looks like a problem of import, but I use this code
def buildsinglefile(currentfile, directory):
x = SingleFile(currentfile, directory)
return x
def walkdir(root_path):
# traverse root directory, and list directories as dirs and files as files
readfiles = []
print(root_path)
for root, dirs, files in os.walk(root_path):
path = root.split(os.sep)
for file in files:
print('file>'+file)
row = buildsinglefile(file, path ).tocsv()//here exception
readfiles.add(row)
for directory in dirs:
walkdir(directory)
All that I'm doing is (in Java developer state of mind :) ) in the main file (not inside a class):
parsing filesystem with os.walk
call the function buildsinglefile that creates an Object
put this object in a list
The point 2 is uncorrect, but I don't understand why.
Do you think everything?
Thank you

Python 3.5:Not able to remove non alpha -numeric characters from file_name

i have written a python script to rename all the files present in a folder by removing all the numbers from the file name but this doesn't work .
Note :Same code works fine for python2.7
import os
def rename_files():
#(1) get file names from a folder
file_list = os.listdir(r"D:\prank")
print(file_list)
saved_path = os.getcwd()
print("Current working Directory is " + saved_path)
os.chdir(r"D:\prank")
#(2) for each file ,rename filename
for file_name in file_list:
os.rename(file_name, file_name.translate(None,"0123456789"))
rename_files()
Can anyone tell me how to make it work.Is the translate function which is not working properly
The problem is with os.rename() portion of your code.
os.rename() requires you to give it a full path to the file/folder you want to change it to, while you only gave it the file_name and not the full path.
You have to add the full path to the folders/files directory.
so it should look like this:
def rename_files():
# add the folder path
folder_path = "D:\prank\\"
file_list = os.listdir(r"D:\prank")
print(file_list)
saved_path = os.getcwd()
print("Current working Directory is " + saved_path)
os.chdir(r"D:\prank")
# Concat the folder_path with file_name to create the full path.
for file_name in file_list:
full_path = folder_path + file_name
print (full_path) # See the full path here.
os.rename(full_path, full_path.translate(None, "0123456789"))
look up the documentation for os, heres what ive found on rename:
os.rename(src, dst, *, src_dir_fd=None, dst_dir_fd=None)
Rename the file or directory src to dst. If dst is a directory, OSError will be raised. On Unix, if dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail on some Unix flavors if src and dst are on different filesystems. If successful, the renaming will be an atomic operation (this is a POSIX requirement). On Windows, if dst already exists, OSError will be raised even if it is a file.
This function can support specifying src_dir_fd and/or dst_dir_fd to supply paths relative to directory descriptors.
If you want cross-platform overwriting of the destination, use replace().
New in version 3.3: The src_dir_fd and dst_dir_fd arguments.
heres a link to the documentation, hope this helps, thanks
https://docs.python.org/3/library/os.html
Others have pointed out other issues with your code, but as to your use of translate, in Python 3.x, you need to pass a dictionary mapping ordinals to new values (or None). This code would work:
import string
...
file_name.translate(dict(ord(c), None for c in string.digits))
but this seems easier to understand:
import re
...
re.sub(r'\d', '', file_name)

Resources