Scenario:
I want to check whether if a directory contains a certain '.png' image file. If so, this image file along with all the other files (with png extension only) gets stored in a different directory. (The solution I am looking for should work in all OS platforms i.e Windows, Unix, etc.) and in a remote server i.e (FTP etc.)
I have tried the following code below:
import os, sys
import shutil
import pathlib
import glob
def search():
image_file = 'picture.png'
try:
arr = [] #List will be used to append all the files in a particular directory.
directory = pathlib.Path("collection") #checks if the collection directory exists.
files = []
#need to convert the PosixPath (directory) to a string.
[files.extend(glob.glob(str(directory) + "/**/*.png", recursive = True))]
res = [img for img in files if(img in image_file)] #checks if the image is within the list of files i.e 'picture.png' == 'collection\\picture.png'
if str(bool(res)): #If True...proceed
print("Image is available in image upload storage directory")
for file in files:
transfer_file = str(file)
shutil.copy(file, 'PNG_files/') #send all the files to a different directory i.e 'PNG_files' by using the shutil module.
arr.append(transfer_file)
return arr
else:
print("image not found in directory")
except OSError as e:
return e.errno
result = search() #result should return the 'arr' list. This list should contain png images only.
However, during execution, the For loop is not getting executed. Which means:
The image files are not stored in the 'PNG_files' directory.
The images are not getting appended in the 'arr' list.
The code above the For loop worked as expected. Can anyone explain to me what went wrong?
There are several issues:
In this line
res = [img for img in files if(img in image_file)] #checks if the image is within the list of files i.e 'picture.png' == 'collection\\picture.png'
you should check the other way around (as written in the comment): image_file in img, e.g. picture.png in collection/picture.png.
str(directory) + "/**/*.png" is not OS independent. If you need this to work on Windows, too, you should use os.path.join(str(directory), '**', '*.png') instead!
This check is incorrect: if str(bool(res)):. It's actually always true, because bool(res) is either True or False, str(bool(res)) is either "True" or "False", but both are actually True, as neither is an empty string. Correctly: if res:.
And finally, you're missing the creation of the PNG_files directory. You need to either manually create it before running the script, or call os.mkdir().
Related
I'm working on a script that takes a checksum and directory as inputs.
Without too much background, I'm looking for 'malware' (ie. a flag) in a directory of executables. I'm given the SHA512 sum of the 'malware'. I've gotten it to work (I found the flag), but I ran into an issue with the output after generalizing the function for different cryptographic protocols, encodings, and individual files instead of directories:
FileNotFoundError: [Errno 2] No such file or directory : 'lessecho'
There is indeed a file lessecho in the directory, and as it happens, is close to the file that returns the actual flag. Probably a coincidence. Probably.
Below is my Python script:
#!/usr/bin/python3
import hashlib, sys, os
"""
### TO DO ###
Add other encryption techniques
Include file read functionality
"""
def main(to_check = sys.argv[1:]):
dir_to_check = to_check[0]
hash_to_check = to_check[1]
BUF_SIZE = 65536
for f in os.listdir(dir_to_check):
sha256 = hashlib.sha256()
with open(f, 'br') as f: <--- line where the issue occurs
while True:
data = f.read(BUF_SIZE)
if not data:
break
sha256.update(data)
f.close()
if sha256.hexdigest() == hash_to_check:
return f
if __name__ == '__main__':
k = main()
print(k)
Credit to Randall for his answer here
Here are some humble trinkets from my native land in exchange for your wisdom.
Your listdir call is giving you bare filenames (e.g. lessecho), but that is within the dir_to_check directory (which I'll call foo for convenience). To open the file, you need to join those two parts of the path back together, to get a proper path (e.g. foo/lessecho). The os.path.join function does exactly that:
for f in os.listdir(dir_to_check):
sha256 = hashlib.sha256()
with open(os.path.join(dir_to_check, f), 'br') as f: # add os.path.join call here!
...
There are a few other issues in the code, unrelated to your current error. One is that you're using the same variable name f for both the file name (from the loop) and file object (in the with statement). Pick a different name for one of them, since you need both available (because I assume you intend return f to return the filename, not the recently closed file object).
And speaking of the closed file, you're actually closing the file object twice. The first one happens at the end of the with statement (that's why you use with). The second is your manual call to f.close(). You don't need the manual call at all.
How to make this code works?
There is a zip file with folders and .png files in it. Folder ".\icons_by_year" is empty. I need to get every file one by one without unzipping it and copy to the root of the selected folder (so no extra folders made).
class ArrangerOutZip(Arranger):
def __init__(self):
self.base_source_folder = '\\icons.zip'
self.base_output_folder = ".\\icons_by_year"
def proceed(self):
self.create_and_copy()
def create_and_copy(self):
reg_pattern = re.compile('.+\.\w{1,4}$')
f = open(self.base_source_folder, 'rb')
zfile = zipfile.ZipFile(f)
for cont in zfile.namelist():
if reg_pattern.match(cont):
with zfile.open(cont) as file:
shutil.copyfileobj(file, self.base_output_folder)
zfile.close()
f.close()
arranger = ArrangerOutZip()
arranger.proceed()
shutil.copyfileobj uses file objects for source and destination files. To open the destination you need to construct a file path for it. pathlib is a part of the standard python library and is a nice way to handle file paths. And ZipFile.extract does some of the work of creating intermediate output directories for you (plus sets file metadata) and can be used instead of copyfileobj.
One risk of unzipping files is that they can contain absolute or relative paths outside of the target directory you intend (e.g., "../../badvirus.exe"). extract is a bit too lax about that - putting those files in the root of the target directory - so I wrote a little something to reject the whole zip if you are being messed with.
With a few tweeks to make this a testable program,
from pathlib import Path
import re
import zipfile
#import shutil
#class ArrangerOutZip(Arranger):
class ArrangerOutZip:
def __init__(self, base_source_folder, base_output_folder):
self.base_source_folder = Path(base_source_folder).resolve(strict=True)
self.base_output_folder = Path(base_output_folder).resolve()
def proceed(self):
self.create_and_copy()
def create_and_copy(self):
"""Unzip files matching pattern to base_output_folder, raising
ValueError if any resulting paths are outside of that folder.
Output folder created if it does not exist."""
reg_pattern = re.compile('.+\.\w{1,4}$')
with open(self.base_source_folder, 'rb') as f:
with zipfile.ZipFile(f) as zfile:
wanted_files = [cont for cont in zfile.namelist()
if reg_pattern.match(cont)]
rebased_files = self._rebase_paths(wanted_files,
self.base_output_folder)
for cont, rebased in zip(wanted_files, rebased_files):
print(cont, rebased, rebased.parent)
# option 1: use shutil
#rebased.parent.mkdir(parents=True, exist_ok=True)
#with zfile.open(cont) as file, open(rebased, 'wb') as outfile:
# shutil.copyfileobj(file, outfile)
# option 2: zipfile does the work for you
zfile.extract(cont, self.base_output_folder)
#staticmethod
def _rebase_paths(pathlist, target_dir):
"""Rebase relative file paths to target directory, raising
ValueError if any resulting paths are not within target_dir"""
target = Path(target_dir).resolve()
newpaths = []
for path in pathlist:
newpath = target.joinpath(path).resolve()
newpath.relative_to(target) # raises ValueError if not subpath
newpaths.append(newpath)
return newpaths
#arranger = ArrangerOutZip('\\icons.zip', '.\\icons_by_year')
import sys
try:
arranger = ArrangerOutZip(sys.argv[1], sys.argv[2])
arranger.proceed()
except IndexError:
print("usage: test.py zipfile targetdir")
I'd take a look at the zipfile libraries' getinfo() and also ZipFile.Path() for construction since the constructor class can also use paths that way if you intend to do any creation.
Specifically PathObjects. This is able to do is to construct an object with a path in it, and it appears to be based on pathlib. Assuming you don't need to create zipfiles, you can ignore this ZipFile.Path()
However, that's not exactly what I wanted to point out. Rather consider the following:
zipfile.getinfo()
There is a person who I think is getting at this exact situation here:
https://www.programcreek.com/python/example/104991/zipfile.getinfo
This person seems to be getting a path using getinfo(). It's also clear that NOT every zipfile has the info.
I wrote a function that is supposed to look for all the file with the extension chosen, in the selected directory. Actually, it runs but it doesn't return anything.
I am trying to keep things simple/stupid, since I am just at the beginning of my journey in Python
Below, I reported the code.
Thanks for your help!
THIS ONE RUNS, BUT RETURNS AN EMPTY LIST
import fnmatch
import glob
def lookfor(dir, ext):
direct = glob.glob(dir)
files = []
for file in direct:
if fnmatch.fnmatch(file, ext):
files.append(file)
return files
print(lookfor('C:/Users/nameuser/where/folder/', '*.docx'))
THIS ONE WORKS PROPERLY, BUT ONLY FOR .docx FILE, AS WRITTEN INSIDE THE FUNCT
import fnmatch
import glob
def lookfor(dir):
direct = glob.glob(dir)
files = []
for file in direct:
if fnmatch.fnmatch(file, '*.docx'):
files.append(file)
return files
print(lookfor('C:/Users/nameuser/where/folder/*.docx'))
I need code that's able to check a folder in the same directory as the python script if it contains either folders or files
this code from How to check to see if a folder contains files using python 3 doesn't work
import os
for dirpath, dirnames, files in os.walk('.'):
if files:
print dirpath, 'Contains files or folders'
if not files:
print dirpath, 'Contains nothing'
The folder I'm checking is DeviceTest
Working
Try this code which uses OSError and aslo os.rmdir never directory which are not empty.So we can use this exception to solve the problem
import os
dir_name = "DeviceTest"
try:
os.rmdir(dir_name)
except OSError as exception_name:
if exception_name.errno == errno.ENOTEMPTY:
print("Directory contains files")
else:
print("Directory don't contain files and is empty")
Simple solution
import os
dir_name = "DeviceTest"
if os.listdir(dir_name) == []:
print("Empty")
for path, dirs, files in os.walk(folder):
if (files) or (dirs):
print("NOT EMPTY: " + str(path))
if (not files) and (not dirs):
print("EMPTY : " + str(path))
I have 2 questions in trying to retrieve a set of data from a directory and displays it out into the ListWidget.
As I am a linux user, I set my ListWidget to read my directory from Desktop in which insides contains say 5 folders and 5 misc items (.txt, .py etc)
Currently I am trying to make my ListWidget to display just the folders but apparently it does that but it also displays all the items, making it a total of 10 items instead of 5.
I tried looking up on the net but I am unable to find any info. Can someone help me?
Pertaining to Qns 1, I am wondering if it is possible to display the top 3 recent folders in the ListWidget, if a checkbox is being checked?
import glob
import os
def test(object):
testList = QListWidget()
localDir = os.listdir("/u/ykt/Desktop/test")
testList.addItems(localDir)
Maybe you should try "QFileDialog" like the following:
class MyWidget(QDialog):
def __init__(self):
QDialog.__init__(self)
fileNames = QFileDialog.getExistingDirectory(self, "list dir", "C:\\",QFileDialog.ShowDirsOnly)
print fileNames
if __name__ == "__main__":
app = QApplication(sys.argv)
widget = MyWidget()
widget.show()
app.exec_()
2nd question, you could reference to this: enter link description here
I guess that you are expecting that os.listdir() will return only the directory names from the given path. Actually it returns the file names too. If you want to add only directories to the listWidget, do the following:
import os
osp = os.path
def test(object):
testList = QListWidget()
dirPath = "/u/ykt/Desktop/test"
localDir = os.listdir(dirPath)
for dir in lacalDir:
path = osp.join(dirPath, dir)
if osp.isdir(path):
testList.addItem(dir)
This will add only directories to the listWidget ignoring the files.
If you want to get the access time for the files and/or folders, use time following method:
import os.path as osp
accessTime = osp.getatime("path/to/dir") # returns the timestamp
Get access time for all the directories and one which has the greatest value is the latest accessed directory. This way you can get the latest accessed 3 directories.