I've created a simple test directory:
├── backup.tgz
├── Dummy-dir
│ ├── dummy-file-a.txt
│ ├── dummy-file-b.txt
│ ├── Images
│ │ ├── imgA.jpg
│ │ ├── imgB.jpg
│ ├── Music
│ │ ├── Una Mattina - ludovico Einaudi (Jimmy Sax Impro live).mp3
│ │ └── Worakls - Blue ( Jimmy Sax live).mp3
│ └── Videos
│ ├── IMG_0001.MOV
│ └── IMG_5377.mov
├── Dummy-target
├── excludes.txt
To test, i did rsync of Dummy-dir to Dummy-target.
I ran multiple tests:
rsync -avz --exclude ./Dummy-dir/Images ./Dummy-dir/ ./Dummy-target/
rsync -avz --exclude=./Dummy-dir/Images ./Dummy-dir/ ./Dummy-target/
rsync -avz --exclude-from=./excludes.txt ./Dummy-dir/ ./Dummy-target/
I tested both with relative and full path.
No matter what i try, it doesn't seems to work.
What is going on?
This should exclude Images folder:
rsync -avz --exclude 'Images' ./Dummy-dir/ ./Dummy-target/
should be relative to the source path without the source path
and if you want to exclude multiple files / directories:
rsyc -avz --exlude={'Images','Music','dummy-file-a.txt'} src-directory/ dest-dirctory/
There must be no spaces in the list! It will not work if there are!
Related
I'm new to shell scripting. I'm trying to create a main folder called Analysis. In the Analysis folder I would like four sub-folders named, PhenV1, PhenV2, HypV1, and HypV2. I then want to have each of those four sub-folders have another 2 folders named Genes and Variants and in each of the Genes and Variants folders to have two more folders named CNV and SNV. The directory structure is depicted by the image below. Here is what I've attempted
mkdir -p Analysis/PhenV1/{Genes/{CNV,SNV},Variants/{CNV,SNV},PhenV2/{Genes/{CNV,SNV},Variants/{CNV,SNV},HypV1/{Genes/{CNV,SNV},Variants/{CNV,SNV},HypV2/{Genes/{CNV,SNV},Variants/{CNV,SNV}}
This code only creates the parent folder Analysis, and subdirectories, PhenV1 and {Genes''m
mkdir -p Analysis/{PhenV1,Phenv2,HypV1,HypV2}/{Genes,Variants}/{CNV,SNV}
Creates:
$ tree
.
└── Analysis
├── HypV1
│ ├── Genes
│ │ ├── CNV
│ │ └── SNV
│ └── Variants
│ ├── CNV
│ └── SNV
├── HypV2
│ ├── Genes
│ │ ├── CNV
│ │ └── SNV
│ └── Variants
│ ├── CNV
│ └── SNV
├── PhenV1
│ ├── Genes
│ │ ├── CNV
│ │ └── SNV
│ └── Variants
│ ├── CNV
│ └── SNV
└── Phenv2
├── Genes
│ ├── CNV
│ └── SNV
└── Variants
├── CNV
└── SNV
29 directories, 0 files
Your brackets are not properly balanced. Try:
Analysis/{PhenV1/{Genes/{CNV,SNV},Variants/{CNV,SNV}},PhenV2/{Genes/{CNV,SNV},Variants/{CNV,SNV}},HypV1/{Genes/{CNV,SN},Variants/{CMC,VT}},HypV2/{Genes/{CNV,SNV},Variants/{CNV,SNV}}}
Which can be simplified a bit:
Analysis/{{PhenV1,PhenV2,HypV2}/{Genes,Variants}/{CNV,SNV},HypV1/{Genes/{CNV,SN},Variants/{CMC,VT}}}
I am trying to use autotools in my Yocto project. Working with another user I was able to get bitbake to recognize my autogen.sh, configure.ac and Makefile.am. I am now getting the error
make: *** No rule to make target 'main.c', needed by 'main.o'. Stop.
My tree is as follows:
.
├── files
│ ├── MAIN_Application
│ │ ├── autogen.sh
│ │ ├── configure.ac
│ │ ├── include
│ │ │ ├── main.h
│ │ │ ├── scheduler.h
│ │ │ └── utilities
│ │ │ └── time_conversions.h
│ │ ├── Makefile.am
│ │ ├── project.yml
│ │ └── src
│ │ ├── main.c
│ │ ├── Makefile.am
│ │ ├── scheduler.c
│ │ └── utilities
| | ├── Makefile.am
│ │ └── time_conversions.c
│ └── services
│ └── mainapplication.service
└── mainapplication_0.0.bb
My makefile.am is as follows:
AUTOMAKE_OPTIONS = foreign
CFLAGS = -Wall -pedantic -O2
include_HEADERS = main.h
bin_PROGRAMS = MAIN_Application
MAIN_Application_SOURCES = main.c
I believe I need to add my other source files. What I am not sure is how to do so. Do I add them with MAIN_Application_SOURCES or do I need to add a Makefile.am in each subdirectory?
Edit: Adding link to previous question.
Edit 2: I have added a makefile.am to each directory. In MAIN_Application I have:
AUTOMAKE_OPTIONS = foreign
SUBDIRS = src
In src I have:
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = MAIN_Application
EVCC_Application_SOURCES = main.c scheduler.c time_conversions.c
This gives a new error:
| Making all in src
| /bin/bash: line 20: cd: src: No such file or directory
| make: *** [Makefile:332: all-recursive] Error 1
| ERROR: oe_runmake failed
Edit 3: If I remove my src and include folders, place everything into one folder (top level) it all works. This is ugly and I hate it.
.
├── files
│ ├── MAIN_Application
│ │ ├── autogen.sh
│ │ ├── configure.ac
│ │ ├── main.c
│ │ ├── main.h
│ │ ├── Makefile.am
│ │ ├── project.yml
│ │ ├── scheduler.c
│ │ ├── scheduler.h
│ │ ├── time_conversions.c
│ │ └── time_conversions.h
│ └── services
│ └── mainapplication.service
└── mainapplication_0.0.bb
There has to be a way around this.
Edit 4: I am attempting to move all of my code into a src directory and use SUBDIR in my makefile.am
.
├── files
│ ├── MAIN_Application
│ │ ├── autogen.sh
│ │ ├── configure.ac
│ │ ├── include
│ │ ├── Makefile.am
│ │ ├── project.yml
│ │ ├── src
│ │ │ ├── main.c
│ │ │ ├── main.h
│ │ │ ├── Makefile.am
│ │ │ ├── scheduler.c
│ │ │ ├── scheduler.h
│ │ │ ├── time_conversions.c
│ │ │ └── time_conversions.h
│ │ └── test
│ │ ├── test_scheduler.c
│ │ └── test_time_conversions.c
│ └── services
│ └── mainapplication.service
└── mainapplication_0.0.bb
This leads to the same error of src no such file or directory.
I believe I need to add my other source files. What I am not sure is how to do so.
There is more than one way to do it, but one way or another, yes, you do need to tell Automake about all the sources involved. Depending on how you do it, you might need more than to just designate the sources.
Do I add them with MAIN_Application_SOURCES or do I need to add a Makefile.am in each subdirectory?
For the project as depicted in the question, you can, and probably should, add them to MAIN_Application_SOURCES in the top-level Makefile.am. In that case, you probably do not need any other Makefile.am files. You should provide paths relative to the directory containing the Makefile.am, so something like:
MAIN_Application_SOURCES = \
src/main.c \
src/scheduler.c \
src/utilities/time_conversions.c
(It's not necessary to use the one-per-line format above, but I find that a lot easier to read and maintain than putting multiple filenames on one line.) In this case, you probably also want to add subdir-objects to your AUTOMAKE_OPTIONS:
AUTOMAKE_OPTIONS = foreign subdir-objects
You need more Makefile.am files only if you want more Makefiles to be generated, which ordinarily would be associated with recursive make.
Furthermore, to be useful, a Makefile.am must define at least one target to be built, and I see only one natural target in the whole project (MAIN_Application). You could introduce so-called "convenience libraries" to serve as targets for subdirectory makes, but I do not favor this approach, nor, for that matter, most uses of recursive make.
Additionally, this ...
include_HEADERS = main.h
... does not mean what you probably think it means. It says that file main.h (in, or to be generated in, the current directory) should be installed to $(includedir) by make install. It says nothing whatever about main.h's relationship to other targets. Still working on the assumption of a single top-level Makefile.am, I think what you want here is altogether different:
AM_CPPFLAGS = -I$(srcdir)/include
That will cause what appears to be an appropriate -I option to be provided to the compiler.
Supposing that you do not require support for make dist, and you do not actually want to install your header files as part of make install, you do not need to explicitly designate individual header files to Automake.
The issue is with configure.ac!
This is my new folder structure
.
├── files
│ ├── MAIN_Application
│ │ ├── autogen.sh
│ │ ├── configure.ac
│ │ ├── include
│ │ │ ├── main.h
│ │ │ ├── scheduler.h
│ │ │ └── utilities
│ │ │ └── time_conversions.h
│ │ ├── Makefile.am
│ │ ├── project.yml
│ │ ├── src
│ │ │ ├── main.c
│ │ │ ├── Makefile.am
│ │ │ ├── scheduler.c
│ │ │ └── utilities
│ │ │ └── time_conversions.c
│ │ └── test
│ │ ├── test_scheduler.c
│ │ └── test_time_conversions.c
│ └── services
│ └── mainapplication.service
└── mainapplication_0.0.bb
My top level Makefile.am contains:
AUTOMAKE_OPTIONS = foreign
SUBDIRS = src
My Makefile.am in src contains
AUTOMAKE_OPTIONS = foreign
bin_PROGRAMS = MAIN_Application
MAIN_Application_SOURCES = main.c scheduler.c time_conversions.c
Obviously these did not change from my question above. In my configure.ac I added one item to AC_CONFIG_FILES. This line changed from
AC_CONFIG_FILES([Makefile])
to
AC_CONFIG_FILES([Makefile src/Makefile])
Anytime you use a subdirectory like this you need to tell autotools about it. I solved this by searching for similar projects on GitHub and I found this.
In this project they have a top level makefile, as well as a makefile in each subdirectory containing source files. In their configure.ac they also give a path to each makefile!
I come to you guys because I do not know the way cuz I'm a nob, but I will investigate how to do what you answer me to make it.
I understand I need to create a Script, well, can you help me telling me how it has to be? And I will investigate the rest.
Btw I'm using mac os Catalina, and Win10 on Parallels.
I need to rename 24000 .jpg files they are distributed in more than 2,000 directories, (the parent directory is in desk btw) but I need to add the directory name BEFORE the real name of the files, I want this so they can all stay in one directory and keep order.
This is an extract of the filelist, and here is the entire .txt filelist on MF.
.
├── 1-Architecture and Design
│ ├── 3D Business Building
│ │ ├── shutterstock_71036254.jpg
│ │ ├── shutterstock_71036287.jpg
│ │ ├── shutterstock_71036290.jpg
│ │ └── shutterstock_72979264.jpg
│ ├── 3D Construction & Project
│ │ ├── shutterstock_10472749.jpg
│ │ ├── shutterstock_10486771.jpg
│ │ ├── shutterstock_10518571.jpg
│ │ ├── shutterstock_10574383.jpg
│ ├── 3D Frames Set
│ │ ├── shutterstock_90316177.jpg
│ │ ├── shutterstock_90316195.jpg
│ │ ├── shutterstock_90316198.jpg
│ │ ├── shutterstock_90316201.jpg
│ │ └── shutterstock_90316207.jpg
├── 2-Food and Drink
│ ├── Almonds
│ │ ├── shutterstock_68684467.jpg
│ │ ├── shutterstock_81734134.jpg
│ │ ├── shutterstock_81734977.jpg
│ │ ├── shutterstock_81736846.jpg
│ │ └── shutterstock_83209996.jpg
│ ├── Anise and Cinnamon Spices
│ │ ├── shutterstock_17474206.jpg
│ │ ├── shutterstock_20572118.jpg
│ │ ├── shutterstock_47658130.jpg
│ │ └── shutterstock_65957044.jpg
│ ├── Apple
│ │ ├── shutterstock_29247376.jpg
│ │ ├── shutterstock_29247382.jpg
│ │ ├── shutterstock_32146864.jpg
│ │ └── shutterstock_47037607.jpg
. .
. .
. .
etc
And it has to finish like this:
.
├── Main Directory
├──1-Architecture and Design-3D Business Building-shutterstock_71036254.jpg
├──1-Architecture and Design-3D Construction & Project-shutterstock_10472749.jpg
├──1-Architecture and Design-3D Frames Set-shutterstock_90316177.jpg
├──2-Food and Drink-Almonds-shutterstock_68684467.jpg
├──2-Food and Drink-Anise and Cinnamon Spices-shutterstock_17474206.jpg
├──2-Food and Drink-Apple-shutterstock_29247376.jpg
.
.
.
etc
I need to preserve one symbol like - to separate the directories in the filenames.
And don't care if the - has to be _ or something else, either if before I need to eliminate the spaces, I just need rename the files with the directory of each one.
Thank you so mucho to read me and take care : )
Finally I got it.
I had to use "A Better Finder Rename V.11.11.27", the program allowed me to do exactly what I needed, it let me choose up to 6 level of parental folders to rename my files.
I leave this answer to my question as a solution in case someone needs to do the same.
I am attaching screenshots.
You can see the name before and after
All in just 15 seconds.
os.listdir could retrieve a level one dirs and files as ls
In [66]: os.listdir()
Out[66]: ['build', 'emacs-libvterm']
In [67]: ls
build/ emacs-libvterm/
How could get a recursive tree dirs as
In [68]: !tree
.
├── build
└── emacs-libvterm
├── CMakeLists.txt
├── elisp.c
├── elisp.h
├── emacs-module.h
├── LICENSE
├── README.md
├── utf8.c
├── utf8.h
├── vterm.el
├── vterm-module.c
└── vterm-module.h
2 directories, 11 files
Maybe this will help, if i right understand question.
You can use this example.
My directory structure:
.
├── empty_folder
├── test
│ ├── sub1
│ │ ├── 1.png
│ │ └── 2.png
│ └── sub2
│ ├── 3.png
│ └── 4.png
└── test.py
code in test.py file:
import os
for path, subdirs, files in os.walk('.'):
if not subdirs and not files:
print(path)
else:
for name in files:
print(os.path.join(path, name))
output:
./test.py
./empty_folder
./test/sub2/3.png
./test/sub2/4.png
./test/sub1/2.png
./test/sub1/1.png
After running Doxygen over my code base, it created two new directories:
html and latex.
Both contain documetnation files, as well as lots of these empty folders:
/html
├── d0
│ ├── d00
│ ├── d01
│ ├── d02
│ ├── d03
│ ├── d04
│ ├── d05
│ ├── d06
| ├── ...
├── d1
│ ├── d00
│ ├── d01
│ ├── d02
│ ├── d03
│ ├── d04
│ ├── d05
│ ├── d06
| ├── ...
well.. not all of them are empty.. very few have files inside.
│ ├── d3a
│ ├── d3b
│ │ └── solarpower_8h_source.html
│ ├── d3c
│ ├── d3d
What is all of this? Some kind of directory hash table? why?
I assume it's not Doxygen specific, so any links to the practice would be appreciated.
EDIT:
found it.
http://www.doxygen.nl/manual/config.html#cfg_create_subdirs
This happens when CREATE_SUBDIRS is enabled in doxygen's configuration files. If you have a large project with tens of thousands of output files, this can help to avoid slow directory access on some file systems.