How can I pass each one of my repository files and to do something with them?
For instance, I want to make a script:
#!/bin/bash
cd /myself
#for-loop that will select one by one all the files in /myself
#for each X file I will do this:
tar -cvfz X.tar.gz /myself2
So a for loop in bash is similar to python's model (or maybe the other way around?).
The model goes "for instance in list":
for some_instance in "${MY_ARRAY[#]}"; do
echo "doing something with $some_instance"
done
To get a list of files in a directory, the quick and dirty way is to parse the output of ls and slurp it into an array, a-la array=($(ls))
To quick explain what's going on here to the best of my knowledge, assigning a variable to a space-delimited string surrounded with parens splits the string and turns it into a list.
Downside of parsing ls is that it doesn't take into account files with spaces in their names. For that, I'll leave you with a link to turning a directory's contents into an array, the same place I lovingly :) ripped off the original array=($(ls -d */)) command.
you can use while loop, as it will take care of whole lines that include spaces as well:
#!/bin/bash
cd /myself
ls|while read f
do
tar -cvfz "$f.tar.gz" "$f"
done
you can try this way also.
for i in $(ls /myself/*)
do
tar -cvfz $f.tar.gz /myfile2
done
Related
We have a tool for cutting adaptors https://github.com/vsbuffalo/scythe/blob/master/README.md and we wanted it to be used on all the files in the raw folder and make an output of each file separately as OUT+File Name.
Something is wrong with this script I wrote, because it doesn't take each file separately, and the whole thing doesn't work properly. It's gonna generateing empty file named OUT+files
Expected operation will looks:
take file1, use scythe on it, write output as OUTfile1
take file2 etc.
#!/bin/bash
FILES=/home/dave/raw/*
for f in $FILES
do
echo "Processing the $f file..."
/home/deve/scythe/scythe -a /home/dev/scythe/illumina_adapters.fa -o "OUT"+$f $f
done
Additionally, I noticed (testing for a single file) that the script uses only one core out of 130 available. Is there any way to improve it?
There is no string concatenation operator in shell. Use juxtaposition instead; it's "OUT$f", not "OUT"+$f.
I want to add the ".sbd" after all files ending on ".utf8" in a directory
I do not want to replace the extensions, but really want to add them so the filenames will look like "filename.utf8.sbd"
I think I should adapt the following code, but don't manage to find out exactly how
for f in *.utf8 ; do mv "$f" "$f.sbd" ; done
Can anyone help me? I am very new to the command line
Thanks a bunch!
Your code should work if no file has spaces (or other "special" character) in the name and if the directory is not pathologically big.
In those cases, you can use something like this:
ls|grep '*.utf8$'|while read i; do mv "$i" "$i.sbd"; done
I need to copy all files from
/dirA/[NAME].20151231.txt
to
/dirB/20151231.[NAME].txt
and
/dirC/20151231/[NAME].txt
i.e. I need to copy the files, but change the name.
You can assume that I know the "date" string before hand, so we can assume 20151231 is a supplied argument.
if I have a list of names, I can do something like
for n in $names; do; cp /dirA/$n.$date.txt /dirB/$date.$n.txt; done;
But what if I dont have a list of names? I am looking for an elegant solution as extracting them from dirA sounds a bit cumbersome.
Thanks!
A reasonably reliable way of processing this material is:
date=20151231
cd /dirA || exit 1
mkdir -p "/dirC/$date" || exit 1
for file in *."$date.txt"
do
name="${file%.$date.txt}"
cp "$file" "/dirB/$date.$name.txt"
cp "$file" "/dirC/$date/$name.txt"
done
The cd operation is checked; if it fails, there is no point in continuing. Likewise, the mkdir -p operation ensures that the dated directory under /dirC exists or exits. The relevant error messages were already generated by cd and mkdir.
Using the shell globbing to generate the file names is best; it avoids issues with 'what happens if the file name contains spaces (or newlines, or other unexpected characters)'.
The assignment extracts the '[NAME]' portion of the file name. This is then used to copy the file from /dirA to the relevant locations under /dirB and /dirC. It would be feasible to check that /dirB and /dirC also exist if you thought that was necessary.
Maybe I am just awful at asking questions. What I was looking for was a "sed for file names". And I found the answer -- that's rename.
I want to store output of ls command in my bash script in a variable and use each file name in a loop, but for example one file in the directory has name "Hello world", when I do variable=$(ls) "Hello" and "world" end up as two separate entries, and when I try to do
for i in $variable
do
mv $i ~
done
it shows error that files "Hello" and "world" doesn't exist.
Is there any way I can access all files in current directory and run some command even if the files have space(s) in their names.
If you must, dirfiles=(/path/of/interest/*).
And accept the admonition against parsing the output of ls!
I understand you are new to this and I'd like to help. But it isn't easy for me (us?) to provide you with an answer that would be of much help to you by the way you've stated your question.
Based on what I hear so far, you don't seem to have a basic understanding on how parameter expansions work in the shell. The following two links will be useful to you:
Matching Pathnames, Parameters
Now, if your task at hand is to operate on files meeting certain criteria then find(1) will likely to do the job.
Say it with me: don't parse the output of ls! For more information, see this post on Unix.SE.
A better way of doing this is:
for i in *
do
mv -- "$i" ~
done
or simply
mv -- * ~
I'm familiar with the structure of
for file in foo/folder\ with\ spaces/foo2/*.txt
do
#do some stuff...
done
However, I want to put foo/folder with spaces/foo2/*.txt into a variable and then use it. Something like this:
myDirectory="foo/folder with spaces/foo2/*.txt"
for file in $myDirectory
do
# do some stuff
done
But what I've written here doesn't work, and it won't work even if I do
myDirectory="food/folder\ with\ spaces/foo2/*.txt"
or
for file in "$myDirectory" ...
Any help? is this even possible?
don't parse ls
# your files are expanded here
# note lack of backslashes and location of quotes
myfiles=("food/folder with spaces/foo2/"*.txt)
# iterate over the array with this
for file in "${myfiles[#]}"; do ...
Parsing ls is a bad idea, instead just do the shell globbing outside of the quotes.
You could also do:
$mydir="folder/with spaces"
for file in "$mydir"/*; do
...
done
Also look into how find and xargs works. Many of these sort of problems can be solved using those. Look at the -print0 and -0 options in particular if you want to be safe.
Try using the ls command in the for loop. This works for me:
for file in `ls "$myDirectory"`