How can I exclude arbitrary file(s) with arbitrary filename extension from glob
selection?
For example: we need all .pug and .haml files in C:\Users\User1\projectname\src\markup
except index.pug (but index.haml is O'K).
The glob
'C:\\Users\\User1\\projectname\\src\\markup\\!(index).+(pug|haml)'
excludes both index.pug and index.haml - this is not that we want, and also there is no enougnh flexibility.
Let's make clear:
In this question we don't need to receive files by glob - the target is just create single glob selection. So the solutions like globby(['C:\\Users\\User1\\projectname\\src\\markup\\*.+(pug|haml)', '!C:\\Users\\User1\\projectname\\src\\markup\\index.pug']); are
not matches to this question.
If it's impossible - to exclude arbitrary file(s) with arbitrary extension(s) from glob - please say such as.
I tried
minimatch(
'C:\\Users\\bokovgleb\\projectname\\src\\src\\index.pug',
'C:\\Users\\bokovgleb\\projectname\\src\\src\\*(*.pug|!(index.pug))'
)
based on comment to this answer. It returns true...
For example: we need all .pug and .haml files in C:\Users\User1\projectname\src\markup except index.pug (but index.haml is O'K).
I came up with
/markup/!(*index){.pug,.haml,index.haml}
You can test it using globster.xyz
I am using glob.glob() to read all the png files in a directory. This is what I am using now
for filename in glob.glob('D:\test_files\**\*.png',recursive=True):
...
Image files are kept in sub folders in test_files. The code will be used by someone else within another thousands of lines of codes. Hence, the directory should be defined somewhere else in the code as a string. Also I would like to change *.png by using another for loop. So, I need something like
for filename in glob.glob('imagepath'+'extension',recursive=True):
...
where imagepath is defined somewhere else in the code and extension is kept in a list. Can I use the glob.glob() method in this way?
The answer is here. I define the path like path = 'D:\test_files\**' and keep the extensions in a list like imgExtentions =['\*.jpg', '\*.png']. Then simply I use glob:
for extension in imgExtentions:
for filename in glob.glob(path+extension,recursive=True):
I would like to compare two directories, and be able to copy one file from a directory to an other ( for example, copy the file dog.png from the directory 1 to the directory 2 if this one doesn't have this file)
I know that there is the "diff" command diff -rq rep1 rep2 which include recursion, but I don't think that the result is convenient (the way it's presented) to work with it. Plus , it doesn't care about the path as long as there is the same file somewhere in the second directory.
My idea was to list the elements of the rep1 , then do a loop, and if there is an other directory, open it and do another loop, do that until there are only files and then compare and go back to the previous loop. I know that seems like recursion, I wanted to know if it was possible to make it that way ? Or if it's too complex.
Mike C: Welcome to Stack Overflow! I would try posing this Q to another site https://softwareengineering.stackexchange.com/ as it does not quite fit the "norm" for Stack Overflow where usually you ask questions about concrete code.I'm going to try to answer the Q anyways.
Yes you could solve this with recursion. You would make one function or method that takes the full path to a directory as it's input, then that function would list all sub-directories and call itself on each of them.
The fact taht you will be scanning two directories in paralelle will add some complexity, because you have to decide what to do in 3 cases:
Subdir exists only in A
Subdir exists only in B
Subdir exists in both
I would implement this by passing TWO full paths to the function, and simply change behsaviour when one of them was null.
I am exploring the filemanip library to search for markdown files in a path including subfolders
import System.FilePath.Find
find always (fileType ==? RegularFile &&? extension ==? ".md") "a/path/"
is there any way to specify a folder name or pattern into which it should not recurse
Looking at the documentation we can see that find takes as first argument a RecursionPredicate which in turn is just FindClause Bool.
Based on this we can see that we have to pass in a custom RecursionPredicate to find other than always.
One example is to ignore .git directories:
notGit :: FindClause Bool -- or via type alias also a RecursionPredicate
notGit = directory /=? ".git"
We then just use our new recursion predicate with find:
find notGit (fileType ==? RegularFile &&? extension ==? ".md") path
Note also the special combinators for predicates to e.g. compose a notSvn predicate with our notGit predicate via (||?) to get a predicate that enters neither directories.
I using scons for a few days and confused a bit. Why there is no built-in tools for building sources recursively starting from given root? Let me explain:
I have such source disposition:
src
Core
folder1
folder2
subfolder2_1
Std
folder1
..and so on. This tree could be rather deeper.
Now I build this with such construction:
sources = Glob('./builds/Std/*/*.cpp')
sources = sources + Glob('./builds/Std/*.cpp')
sources = sources + Glob('./builds/Std/*/*/*.cpp')
sources = sources + Glob('./builds/Std/*/*/*/*.cpp')
and this looks not so perfect as at can be. Of cause, I can write some python code, but
is there more suitable ways of doing this?
As Torsten already said, there is no "internal" recursive Glob() in SCons. You need to write something yourself. My solution is:
import fnmatch
import os
matches = []
for root, dirnames, filenames in os.walk('src'):
for filename in fnmatch.filter(filenames, '*.c'):
matches.append(Glob(os.path.join(root, filename)[len(root)+1:]))
I want to stress that you need Glob() here (not glob.glob() from python) especially when you use VariantDir(). Also when you use VariantDir() don't forget to convert absolute paths to relative (in the example I achieve this using [len(root)+1:]).
Sure.
You need to write python wrappers to walking through dirs. You can find many recipes on stackoverflow.
Here is my simple function which returns list of subdirs in present dir (and ignore hide dirs starting with '.' - dot)
def getSubdirs(abs_path_dir) :
lst = [ name for name in os.listdir(abs_path_dir) if os.path.isdir(os.path.join(abs_path_dir, name)) and name[0] != '.' ]
lst.sort()
return lst
For example, i've dir modules what containts foo, bar, ice.
corePath = 'abs/path/to/modules'
modules = getSubdirs(corePath)
# modules = [bar, foo, ice]
for module in modules :
sources += Glob(os.path.join(corePath, module, '*.cpp'))
You can improve getSubdirs function adding recurse and walking deeper to subdirs.
The Glob() SCons function doesnt have the ability to go recursive.
It would be much more efficient if you change your Python code to use the list.extend() function, like this:
sources = Glob('./builds/Std/*/*.cpp')
sources.extend(Glob('./builds/Std/*.cpp'))
sources.extend(Glob('./builds/Std/*/*/*.cpp'))
sources.extend(Glob('./builds/Std/*/*/*/*.cpp'))
Instead of trying to go recursive like you are, its quite common to have a SConscript script in each subdirectory and in the root SConstruct call each of them with the SConscript() function. This is called a SCons hierarchical build.
Here is my version of recursive Glob:
from SCons.Environment import Base as BaseEnvironment
def __RGlob(self, root_path, pattern, ondisk=True, source=False, strings=False, exclude=None):
result_nodes = []
paths = [root_path]
while paths:
path = paths.pop()
all_nodes = self.Glob(f'{path}/*', ondisk=ondisk, source=source, exclude=exclude)
paths.extend(entry for entry in all_nodes if entry.isdir() or (entry.srcnode() and entry.srcnode().isdir())) # `srcnode()` must be used because `isdir()` doesn't work for entries in variant dirs which haven't been copied yet.
result_nodes.extend(self.Glob(f'{path}/{pattern}', ondisk=ondisk, source=source, strings=strings, exclude=exclude))
return sorted(result_nodes)
BaseEnvironment.RGlob = __RGlob
It tries imitate the standard Glob as closely as possible. The biggest difference is that beside the pattern is takes a root path as another argument. The pattern is then applied to this root path and every sub-directory in it.
This code adds function RGlob it to the base environment, which means you will be able to call it on every environment that is created after that. The best place to paste this is probably the file site_scons/site_init.py.
I use this:
srcdir = './'
sources = [s for s in glob2.glob(srcdir + '**/*.cpp') if "/." not in s]