related to: Loop through all the files with a specific extension
I want to loop through files that matches a pattern:
for item in ./bob* ; do
echo $item
done
I have a file list like:
bob
bobob
bobob.log
I only want to list files that have no extension:
bob
bobob
what is the best way to archive this? - can I do it in the loop somehow or do I need an if statement within the loop?
In bash you can use features of xtended globbing:
shopt -s extglob
for item in ./bob!(*.*) ; do
echo $item
done
You can put shopt -s extglob in your .bashrc file to enable it.
Recent Bash versions have regular expression support:
for f in *
do
if [[ "$f" =~ .*\..* ]]
then
: ignore
else
echo "$f"
fi
done
Related
I have this script which removes the first periods from a file in order to unhide it. However, all it can do is remove the first period and not the succeeding periods, which makes the file still hidden. What I want to know now is how I can remove more than 1 periods at a time to unhide the file.
#!/bin/bash
period='.'
for i in $#; do
if [[ "$i" == "$period"* ]] ; then
mv "$i" "${i#"$period"}"
else
mv $i .${i}
fi
done
I have some knowledge in using grep and regex so I thought of using + to remove a lot of them at a time but I cant really figure out if it is the correct way to go about it
You can use the bash extended glob +(pattern) to match one or more periods, combined with the ## parameter expansion to remove the longest leading match:
#!/usr/bin/env bash
# Turn on extended globs
shopt -s extglob
name=...foo
printf "%s -> %s\n" "$name" "${name##+(.)}"
Or you can use a regular expression, combining looking for leading periods with capturing the rest of the name:
#!/usr/bin/env bash
# Note the quoted parameters to avoid many issues.
for i in "$#"; do
if [[ "$i" =~ ^\.+(.*) ]]; then
mv "$i" "${BASH_REMATCH[1]}"
else
mv "$i" ".${i}"
fi
done
My pictures are kept in the folder with the picture-date for folder name, for example the original path and file names:
.../Pics/2016_11_13/wedding/DSC0215.jpg
.../Pics/2016_11_13/afterparty/DSC0234.jpg
.../Pics/2016_11_13/afterparty/DSC0322.jpg
How do I rename the pictures into the format below, with continuous sequences and 4-digit padding?
.../Pics/2016_11_13_wedding.0001.jpg
.../Pics/2016_11_13_afterparty.0002.jpg
.../Pics/2016_11_13_afterparty.0003.jpg
I'm using Bash 4.1, so only mv command is available. Here is what I have now but it's not working
#!/bin/bash
p=0
for i in *.jpg;
do
mv "$i" "$dirname.%03d$p.JPG"
((p++))
done
exit 0
Let say you have something like .../Pics/2016_11_13/wedding/XXXXXX.jpg; then go in directory .../Pics/2016_11_13; from there, you should have a bunch of subdirectories like wedding, afterparty, and so on. Launch this script (disclaimer: I didn't test it):
#!/bin/sh
for subdir in *; do # scan directory
[ ! -d "$subdir" ] && continue; # skip non-directory
prognum=0; # progressive number
for file in $(ls "$dir"); do # scan subdirectory
(( prognum=$prognum+1 )) # increment progressive
newname=$(printf %4.4d $prognum) # format it
newname="$subdir.$newname.jpg" # compose the new name
if [ -f "$newname" ]; then # check to not overwrite anything
echo "error: $newname already exist."
exit
fi
# do the job, move or copy
cp "$subdir/$file" "$newname"
done
done
Please note that I skipped the "date" (2016_11_13) part - I am not sure about it. If you have a single date, then it is easy to add these digits in # compose the new name. If you have several dates, then you can add a nested for for scanning the "date" directories. One more reason I skipped this, is to let you develop something by yourself, something you can be proud of...
Using only mv and bash builtins:
#! /bin/bash
shopt -s globstar
cd Pics
p=1
# recursive glob for .jpg files
for i in **/*.jpg
do
# (date)/(event)/(filename).jpg
if [[ $i =~ (.*)/(.*)/(.*).jpg ]]
then
newname=$(printf "%s_%s.%04d.jpg" "${BASH_REMATCH[#]:1:2}" "$p")
echo mv "$i" "$newname"
((p++))
fi
done
globstar is a bash 4.0 feature, and regex matching is available even in OSX's anitque bash.
I have a folder filed with hundreds of text files which I want to run a Linux command called mint. This command outputs a text value which I want stored in unique files, one for each file I have in the folder. Is there a way to run the command using the * character to represent all my input files, while still piping the output to a file that is unique from each other file?
Example:
$ mint * > uniqueFile.krn
With the bugs fixed and caveats closed:
#!/bin/bash
# ^^^^ - bash, not sh, for [[ ]] support
for f in *.krn; do
[[ $f = *.krn ]] && continue # skip files already ending in .krn
mint "$f" >"$f.krn"
done
Or, with a prefix:
for f in *; do
[[ $f = int_* ]] && continue
mint "$f" >"int_$f"
done
You can also avoid recreating hashes that already exist unless the source file changed:
for f in *; do
# don't hash hash files
[[ $f = int_* ]] && continue
# if a non-empty hash file exists, and is newer than our source file, don't hash again
[[ -s "int_$f" && "int_$f" -nt "$f" ]] && continue
# ...if we got through the above conditions, then go ahead with creating a hash
mint "$f" >"int_$f"
done
To explain:
test -s filename is true only if a file by the given name exists and is non-empty
test file1 -nt file2 is true only if both files exist, and file1 is newer than file2.
[[ ]] is a ksh-extended shell syntax derived from that for the test command, adding support for pattern-matching tests (ie. [[ $string = *.txt ]] will be true only if $string expands to a value ending in .txt), and relaxing quoting rules (it's safe to write [[ -s $f ]], but test -s "$f" needs the quotes to work with all possible filenames).
Thanks for all the suggestions! Shiping's solution worked great, I just appended a prefix to the file name. Like so:
$ for file in * ; do mint $file > int_$file ; done
Self-answer moved from question and flagged Community Wiki; see What is the appropriate action when the answer to a question is added to the question itself?
How to write simple shell script to create zip file.
I want to create zip file by collecting files with same string pattern in their file names from a folder.
For example, there may be many files under a folder.
xxxxx_20140502_xxx.txt
xxxxx_20140502_xxx.txt
xxxxx_20140503_xxx.txt
xxxxx_20140503_xxx.txt
xxxxx_20140504_xxx.txt
xxxxx_20140504_xxx.txt
After running the shell script, the result must be following three zip files.
20140502.zip
20140503.zip
20140504.zip
Please give me right direction to create simple shell script to output the result as above.
#!/bin/bash
for file in *_????????_*.csv *_????????_*.txt; do
[ -f "${file}" ] || continue
date=${file#*_} # adjust this and next line depending
date=${date%_*} # on your actual prefix/suffix
echo "${date}"
done | sort -u | while read date; do
zip "${date}.zip" *${date}*
done
Since zip will update the archive, this will do:
shopt -s nullglob
for file in *.{txt,csv}; do [[ $file =~ _([[:digit:]]{8})_ ]] && zip "${BASH_REMATCH[1]}.zip" "$file"; done
The shopt -s nullglob is because you don't want to have unexpanded globs if there are no matching files.
Everything below this line is my old answer...
First, get all the possible dates. Heuristically, this could be the files ending in .txt and .csv that match the regex _[[:digit:]]{8}_:
#!/bin/bash
shopt -s nullglob
declare -A dates=()
for file in *.{csv,txt}; do
[[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=
done
printf "Date found: %s\n" "${!dates[#]}"
This will output to stdout all the dates found in the files. E.g. (I called the previous snipped gorilla and I chmod +x gorilla and touched a few files for demo):
$ ls
banana_20010101_gorilla.csv gorilla_20140502_bonobo.csv
gorilla notthisone_123_lol.txt
gorilla_20140502_banana.txt
$ ./gorilla
Date found: 20140502
Date found: 20010101
Next step, for each date found, get all the files ending in .txt and .csv and zip them in the archive corresponding to the date: appending this to gorilla will do the job:
for date in "${!dates[#]}"; do
zip "$date.zip" *"_${date}_"*.{csv,txt}
done
Full script after removing the flooding part:
#!/bin/bash
shopt -s nullglob
declare -A dates=()
for file in *.{csv,txt}; do
[[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=
done
for date in "${!dates[#]}"; do
zip "$date.zip" *"_${date}_"*.{csv,txt}
done
Edit. I overlooked your requirement with one line command. Then here's the one-liner:
shopt -s nullglob; declare -A dates=(); for file in *.{csv,txt}; do [[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=; done; for date in "${!dates[#]}"; do zip "$date.zip" *"_${date}_"*.{csv,txt}; done
:)
#! /bin/bash
dates=$(ls ?????_[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]_???.{csv,txt} \
| cut -f2 -d_ | sort -u)
for date in $dates ; do
zip $date.zip ?????_"$date"_???.{csv,txt}
done
This is probably quite easy, but I can't figure it out. In a for loop I want to exclude certain files with the prefix zz (e.g. zz131232.JPG) but I don't know how to exclude these files.
for i in *.JPG; do
# do something
done
How do I modify the 'for rule' to exclude files with the prefix zz?
Something like
for i in *.JPG; do
[[ $i != "zz"* ]] && echo "$i"
done
or skip them:
for i in *.JPG; do
[[ $i == "zz"* ]] && continue
# process the other files here
done
If you are dealing with many files you can also use GLOBIGNORE or extended globbing to avoid expanding the files you wish to skip in the first place (which might be faster):
GLOBIGNORE='zz*'
for file in *.JPG; do
do_something_with "${file}"
done
# save and restore GLOBIGNORE if necessary
or
shopt -s extglob # enable extended globbing
for file in !(zz*).JPG; do
do_something_with "${file}"
done
shopt -u extglob # disable extended globbing, if necessary
Note that if there are no .JPG files in the current directory the loop will still be entered and $i be set to the literal *.JPG (in your example), so you either need to check if the file exists inside the loop or use the nullglob option.
for file in *.JPG; do
[ -e "${file}" ] || continue
do_something_with "${file}"
done
or
shopt -s nullglob
for file *.JPG; do
do_something_with "${file}"
done
shopt -u nullglob # if necessary
Try the following in your shell to understand what happens without nullglob:
$ for f in *.doesnotexist; do echo "$f"; done
*.doesnotexist