I'm having trouble making a subfolder for files to be stored into after choosing the Parent Folder or directory.
This is my current script:
import os, sys, subprocess, shutil, glob, os.path, csv
#User Chooses Folder or File Path Directory
path_new = filedialog.askdirectory()
def saveData():
serial_number = input('Scan Barcode: ')
folder_name = print('Folder Name:', serial_number)
os.chdir(path_new) #Not sure if this is necessary
if not os.path.exists(path_new):
os.makedirs(path_new)
print ('The data has been stored in the following directory:', path_new)
shutil.move('file_directory/TOTAL.csv', 'chosen_directory/%s'% (serial_number))
shutil.move('file_directory/enviro.csv', 'chosen_directory/%s' % (serial_number))
The script runs, but just not how I would like it to. Does anyone have any recommendations or an alternate solution to creating a folder and selecting it as the directory to have subfolders created and have files transferred into them?
I resolved my issue. If anyone else knows of a "cleaner" way of writing this out please update with a better answer. As of now this does exactly what I want.
Code:
import os, sys, shutil. os.path
path = filedialog.askdirectory()
def saveData():
serial_number = input('Scan Barcode: ')
folder_name = print('Folder Name:', serial_number)
path_new = os.path.join(path, '%s' % (serial_number))
if not os.path.exists(path_new):
os.makedirs(path_new)
shutil.move('C:/files/TOTAL.csv', path_new)
shutil.move('C:/files/enviro.csv', path_new)
print('The data has been stored in the following directory:', path_new)
Related
How to make this code works?
There is a zip file with folders and .png files in it. Folder ".\icons_by_year" is empty. I need to get every file one by one without unzipping it and copy to the root of the selected folder (so no extra folders made).
class ArrangerOutZip(Arranger):
def __init__(self):
self.base_source_folder = '\\icons.zip'
self.base_output_folder = ".\\icons_by_year"
def proceed(self):
self.create_and_copy()
def create_and_copy(self):
reg_pattern = re.compile('.+\.\w{1,4}$')
f = open(self.base_source_folder, 'rb')
zfile = zipfile.ZipFile(f)
for cont in zfile.namelist():
if reg_pattern.match(cont):
with zfile.open(cont) as file:
shutil.copyfileobj(file, self.base_output_folder)
zfile.close()
f.close()
arranger = ArrangerOutZip()
arranger.proceed()
shutil.copyfileobj uses file objects for source and destination files. To open the destination you need to construct a file path for it. pathlib is a part of the standard python library and is a nice way to handle file paths. And ZipFile.extract does some of the work of creating intermediate output directories for you (plus sets file metadata) and can be used instead of copyfileobj.
One risk of unzipping files is that they can contain absolute or relative paths outside of the target directory you intend (e.g., "../../badvirus.exe"). extract is a bit too lax about that - putting those files in the root of the target directory - so I wrote a little something to reject the whole zip if you are being messed with.
With a few tweeks to make this a testable program,
from pathlib import Path
import re
import zipfile
#import shutil
#class ArrangerOutZip(Arranger):
class ArrangerOutZip:
def __init__(self, base_source_folder, base_output_folder):
self.base_source_folder = Path(base_source_folder).resolve(strict=True)
self.base_output_folder = Path(base_output_folder).resolve()
def proceed(self):
self.create_and_copy()
def create_and_copy(self):
"""Unzip files matching pattern to base_output_folder, raising
ValueError if any resulting paths are outside of that folder.
Output folder created if it does not exist."""
reg_pattern = re.compile('.+\.\w{1,4}$')
with open(self.base_source_folder, 'rb') as f:
with zipfile.ZipFile(f) as zfile:
wanted_files = [cont for cont in zfile.namelist()
if reg_pattern.match(cont)]
rebased_files = self._rebase_paths(wanted_files,
self.base_output_folder)
for cont, rebased in zip(wanted_files, rebased_files):
print(cont, rebased, rebased.parent)
# option 1: use shutil
#rebased.parent.mkdir(parents=True, exist_ok=True)
#with zfile.open(cont) as file, open(rebased, 'wb') as outfile:
# shutil.copyfileobj(file, outfile)
# option 2: zipfile does the work for you
zfile.extract(cont, self.base_output_folder)
#staticmethod
def _rebase_paths(pathlist, target_dir):
"""Rebase relative file paths to target directory, raising
ValueError if any resulting paths are not within target_dir"""
target = Path(target_dir).resolve()
newpaths = []
for path in pathlist:
newpath = target.joinpath(path).resolve()
newpath.relative_to(target) # raises ValueError if not subpath
newpaths.append(newpath)
return newpaths
#arranger = ArrangerOutZip('\\icons.zip', '.\\icons_by_year')
import sys
try:
arranger = ArrangerOutZip(sys.argv[1], sys.argv[2])
arranger.proceed()
except IndexError:
print("usage: test.py zipfile targetdir")
I'd take a look at the zipfile libraries' getinfo() and also ZipFile.Path() for construction since the constructor class can also use paths that way if you intend to do any creation.
Specifically PathObjects. This is able to do is to construct an object with a path in it, and it appears to be based on pathlib. Assuming you don't need to create zipfiles, you can ignore this ZipFile.Path()
However, that's not exactly what I wanted to point out. Rather consider the following:
zipfile.getinfo()
There is a person who I think is getting at this exact situation here:
https://www.programcreek.com/python/example/104991/zipfile.getinfo
This person seems to be getting a path using getinfo(). It's also clear that NOT every zipfile has the info.
I am running into a problem trying to update an AWS Gamelift script with a python command that zips a directory and uploads it with all its contents as a newer version to AWS Gamelift.
from zipfile import ZipFile
import os
from os.path import basename
import boto3
import sys, getopt
def main(argv):
versInput = sys.argv[1]
#initializes client for updating script in aws gamelift
client = boto3.client('gamelift')
#Where is the directory relative to the script directory. In this case, one folder dir lower and the contents of the RealtimeServer dir
dirName = '../RealtimeServer'
# create a ZipFile object
with ZipFile('RealtimeServer.zip', 'w') as zipObj:
# Iterate over all the files in directory
for folderName, subfolders, filenames in os.walk(dirName):
rootlen = len(dirName) + 1
for filename in filenames:
#create complete filepath of file in directory
filePath = os.path.join(folderName, filename)
# Add file to zip
zipObj.write(filePath, filePath[rootlen:])
response = client.update_script(
ScriptId=SCRIPT_ID_GOES_HERE,
Version=sys.argv[1],
ZipFile=b'--zip-file \"fileb://RealtimeServer.zip\"'
)
if __name__ == "__main__":
main(sys.argv[1])
I plan on using it by giving it a new version number everytime I make changes with:
python updateScript.py "0.1.1"
This is meant to help speed up development. However, I am doing something wrong with the ZipFile parameter of client.update_script()
For context, I can use the AWS CLI directly from the commandline and update a script without a problem by using:
aws gamelift update-script --script-id SCRIPT_STRING_ID_HERE --script-version "0.4.5" --zip-file fileb://RealtimeServer.zip
However, I am not sure what is going on because it fails to unzip the file when I try it:
botocore.errorfactory.InvalidRequestException: An error occurred (InvalidRequestException) when calling the UpdateScript operation: Failed to unzip the zipped file.
UPDATE:
After reading more documentation about the ZipFile parameter:
https://docs.aws.amazon.com/gamelift/latest/apireference/API_UpdateScript.html
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/gamelift.html#GameLift.Client.update_script
I tried sending a base64 encoded version of the zip file. However, that didn't work. I put the following code before the client_update part of the script and used b64EncodedZip as the ZipFile parameter.
with open("RealtimeServer.zip", "rb") as f:
bytes = f.read()
b64EncodedZip = base64.b64encode(bytes)
I was able to get it to work by having some help from a maintainer of boto3 over at https://github.com/boto/boto3/issues/2646
(Thanks #swetashre)
Here is the code and it will only work up to 5mb and requires use of an s3 bucket if you want to upload a zip file any larger than that.
from zipfile import ZipFile
import os
from os.path import basename
import boto3
import sys, getopt
def main(argv):
versInput = sys.argv[1]
#initializes client for updating script in aws gamelift
client = boto3.client('gamelift')
#Where is the directory relative to the script directory. In this case, one folder dir lower and the contents of the RealtimeServer dir
dirName = '../RealtimeServer'
# create a ZipFile object
with ZipFile('RealtimeServer.zip', 'w') as zipObj:
# Iterate over all the files in directory
for folderName, subfolders, filenames in os.walk(dirName):
rootlen = len(dirName) + 1
for filename in filenames:
#create complete filepath of file in directory
filePath = os.path.join(folderName, filename)
# Add file to zip
zipObj.write(filePath, filePath[rootlen:])
with open('RealtimeServer.zip','rb') as f:
contents = f.read()
response = client.update_script(
ScriptId="SCRIPT_ID_GOES_HERE",
Version=sys.argv[1],
ZipFile=contents
)
if __name__ == "__main__":
main(sys.argv[1])
I got the script working but I did it by avoiding the use of boto3. I don't like it but it works.
os.system("aws gamelift update-script --script-id \"SCRIPT_ID_GOES_HERE\" --script-version " + sys.argv[1] + " --zip-file fileb://RealtimeServer.zip")
If anyone knows how to get boto3 to work for updating an AWS Gamelift script then please let me know.
I am trying to move the files from one folder to the other based on the time or date stamp. It's something like I want to keep today file in the same folder and move yesterday file into a different folder.
Currently, I am able to move the files from one folder to other but it's not on date or time-based.
The file name will look something like this.
"output-android_login_scenarios-android-1.43-9859-2019-04-30 11:29:31.542548.html"
-------python
def move(self, srcdir,dstdir):
currentDirectory = os.path.dirname(__file__)
sourceFile = os.path.join(currentDirectory, srcdir)
destFile = os.path.join(currentDirectory, dstdir)
if not os.path.exists(destFile):
os.makedirs(destFile)
source = os.listdir(sourceFile)
try:
for files in source:
shutil.move(sourceFile+'/'+files, destFile)
except:
print("No file are present")
I think I have something that might work for you. I have made some minor tweaks to your "move" function, so I hope you don't mind. This method will also work if you have more than one 'old' file that needs moving.
Let me know if this helps :)
import os
import shutil
import re
from datetime import datetime
sourceDir = 'C:\\{folders in your directory}\\{folder containing the files}'
destDir = 'C:\\{folders in your directory}\\{folder containing the old files}'
files = os.listdir(sourceDir)
list_of_DFs = []
for file in files:
if file.endswith('.html'):
name = file
dateRegex = re.compile(r'\d{4}-\d{2}-\d{2}')
date = dateRegex.findall(file)
df = pd.DataFrame({'Name': name, 'Date': date})
list_of_DFs.append(df)
filesDF = pd.concat(list_of_DFs,ignore_index=True)
today = datetime.today().strftime('%Y-%m-%d')
filesToMove = filesDF[filesDF['Date'] != today]
def move(file, sourceDir, destDir):
sourceFile = os.path.join(sourceDir, file)
if not os.path.exists(destDir):
os.makedirs(destDir)
try:
shutil.move(sourceFile, destDir)
except:
print("No files are present")
for i in range(len(filesToMove)):
file = filesToMove['Name'][i]
move(file,sourceDir,destDir)
I need code that's able to check a folder in the same directory as the python script if it contains either folders or files
this code from How to check to see if a folder contains files using python 3 doesn't work
import os
for dirpath, dirnames, files in os.walk('.'):
if files:
print dirpath, 'Contains files or folders'
if not files:
print dirpath, 'Contains nothing'
The folder I'm checking is DeviceTest
Working
Try this code which uses OSError and aslo os.rmdir never directory which are not empty.So we can use this exception to solve the problem
import os
dir_name = "DeviceTest"
try:
os.rmdir(dir_name)
except OSError as exception_name:
if exception_name.errno == errno.ENOTEMPTY:
print("Directory contains files")
else:
print("Directory don't contain files and is empty")
Simple solution
import os
dir_name = "DeviceTest"
if os.listdir(dir_name) == []:
print("Empty")
for path, dirs, files in os.walk(folder):
if (files) or (dirs):
print("NOT EMPTY: " + str(path))
if (not files) and (not dirs):
print("EMPTY : " + str(path))
I have a script that copies files from a folder to another folder, but I want it to ignore the txt files. I have this code that works, but it does not ignore the txt files. It just copies everything without any warning.
What am I doing wrong?
import sys, os, shutil
script, filetobackup = sys.argv
backupdir = r'*path to backupdir*'
def copytree(filetobackup, backupdir, symlinks=False):
if not os.path.exists(backupdir):
os.makedirs(backupdir)
for item in os.listdir(filetobackup):
s = os.path.join(filetobackup, item)
d = os.path.join(backupdir, item)
if os.path.isdir(s):
shutil.copytree(s, d, symlinks, ignore=ignore_patterns('*.txt'))
else:
if not os.path.exists(d) or os.stat(s).st_mtime - os.stat(d).st_mtime > 1:
shutil.copy2(s, d)
copytree(filetobackup, backupdir)
Edit: Hmm, no one knows?