I wrote a script which removes spaces in a single folder/file name. I want to make it work so that it removes all spaces in folder/files name in the directory the script exists.
MY Script:
#!/bin/bash
var=$(ls | grep " ")
test=$(echo $var | sed 's/ //')
mv "$var" $test
How it worked
Thank you for helping!
Try this
ls | grep " " | while read file_name
do
mv "$file_name" "$(echo $file_name | sed -E 's/ +//g')"
done
sed -E is so that you can use some simple regex, and / +/ so it can work in case of multiple consecutive spaces such as . And /g so it replaces every occurrences such as foo baa .txt .
Something like this might work:
for f in * ; do
if [[ "$f" =~ \ ]] ; then
mv "$f" "${f// /_}"
fi
done
Explanattion:
for f in * ; do
loops over all file names in the directory. It doesn't have the quirks of ls that make parsing the output of ls a bad idea.
if [[ "$f" =~ \ ]] ; then
This is the bash way of pattern matching. The \ is the pattern. You need to escape the space with a backslash, otherwise the shell will not recognize it as a pattern.
mv "$f" "${f// /_}"
${f// /_} is the bash way of pattern-substitution. The // means replace all occurrences. The syntax is ${variable//pattern/replacement} to replace all patterns in the variable with the replacement.
Related
I'm passing two positional args to a script to run, both args are a path, and while in the scenario analyzing the paths, the problem is sometimes there is some path like: m i sc . . . . .. . . it has dots and spaces, and sometimes even we have a backslash in dir names.
It is so tried to get arguments via two procedures, directly and via at sign.
SOURCE_ARG=$1
DESTINATION_ARG=$2
and
ARG_COUNT=0
for POSITIONAL_ARGUMENTS in "${#}"
do
((ARG_COUNT++))
ARGUMENT_ARRAY[$ARG_COUNT]=$POSITIONAL_ARGUMENTS
done
In the loop, I iterate through the result of commands that have forwarded to them.
while IFS= read -r dir
do
echo "${ARGUMENT_ARRAY[1]}"
echo "${dir}"
while IFS= read -r item
do
# do some stuff
done < <(ls -A "$dir"/)
done < <(du -hP "$SOURCE_ARG" | awk '{$1=""; print $0}' | grep -v "^.$" | sed "s/^ //g")
when i use echo "${ARGUMENT_ARRAY[1]}" i get the same path as i need to check but when using loop iteration varible as dir in here ->echo "${dir}" i got all the spaces escaped, since other commands for that path could not do their jobs.
What I'm Asking for is that how can I get the output of $dir within the loop and as like as echo "${ARGUMENT_ARRAY[1]}" that i mentioned above(input with all spaces and backslashes)
Thanks to #Barmar in comments.
The only reason that filenames are without escapes (i.e. you see directories with no special character or special characters have been escaped) is because du is printing the filenames with escapes, so $dir variable would have escaped once and special characters are no longer available for the other loop iteration in my problem.
Now that we know the problem was raised by using du in my script:
while IFS= read -r dir
# do sth
done < <(du -hP "$SOURCE_ARG" | awk '{$1=""; print $0}' | grep -v "^.$" | sed "s/^ //g")
We can change the du to find and the problem is solved:
while IFS= read -r dir
# do sth
done < <(find "$SOURCE_ARG" -type d –)
PS 1:
Another problem raised as I wanted to print the lines to check them if they are ok or not (i.e. while debugging application) was with echo.
So be sure to try printf "%s\n" "$dir" instead of echo, as some versions of echo process escape sequences.
echo "${dir}"
printf "%s\n" "$dir"
PS 2:
Also If a filename has more than one space in a row, The way I used awk, was collapsing them into a single space.
awk '{$1=""; print $0}' | grep -v "^.$" | sed "s/^ //g"
I am currently using the following command:
grep -l -Z -E '.*?FindMyRegex' /home/user/folder/*.csv | xargs -0 -I{} mv {} /home/destination/folder
This works fine. The problem is it uses grep on the entire file.
I would like to use the grep command on the FIRST line of the file only.
I have tried to use head -1 file | at the beginning, but it did not work.
A change I would add to your script is -
for file in *.csv; do
head -1 "$file" | grep -l -Z -E '.*?FindMyRegex' | xargs -0 -I{} mv {} /home/destination/folder;
done
you can maybe try sed '1q' file.csv | grep ... to search the regexp only in the first line.
You don't need grep or find, as long as your files don't have embedded newlines.
I don't know an easy way off the top of my head to get sed to delimit with nulls.
mv $( for f in /home/user/folder/*.csv;
do sed -ns '1 { /yourPattern/F; q; }' $f;
done ) /home/destination/folder/
EDIT
Rewrote with a loop. This will run a separate instance of sed to check each file, but at least it shouldn't read beyond the first line. It will fail syntactically if there are no hits.
You might need -E depending on your regex.
-n says don't print records from the files.
-s says treat each file as a distinct input - this is so the filenames aren't always the first one.
This does require GNU sed for the F.
gawk 'FNR==1{if($0~/PATTERN/)
printf "mv %s %s\n",FILENAME, "/target";nextfile}' /path/*.csv
First of all, in your regex: .*?FindMyRegex the .*? doesn't make any sense, they could be removed.
The above awk (gawk) one-liner will build up mv file target command lines for you. You can check them, if you are satisfied with them, pipe the output to |sh , the commands are gonna be executed.
replace PATTERN by your regex pattern, and /target by the real target dir.
The one-liner is assuming that the filenames don't contain special chars (space i.e.), if it is the case, add "s to the mv cmd.
using GNU awk to find the filenames, pipe the filenames into xargs
gawk -v pattern="myRegex" '
FNR == 1 {if ($0 ~ pattern) printf "%s\0", FILENAME; nextfile}
' *.csv | xargs -0 echo mv -t destination
If it looks OK, remove "echo"
Try this Shellcheck-clean Bash code:
#! /bin/bash
shopt -s nullglob # Globs that match nothing expand to nothing
shopt -s dotglob # Globs match files whose names start with '.'
dest=/home/destination/folder
for file in *.csv ; do
head -n 1 -- "$file" | grep -qE '.*?FindMyRegex' && mv -- "$file" "$dest"
done
shopt -s nullglob prevents an error if there are no .csv files in the directory.
shopt -s dotglob ensures that files whose name starts with '.' are handled.
The -- in the options for head and mv ensures that files whose names begin with - are handled correctly.
The quotes in "$file" and "$dest" ensure that names that contain whitespace (actually $IFS) characters (including newlines) or glob metacharacters are handled correctly.
Note that the .*? in the reqular expression is probably redundant, and may not do what you think it does (grep -E doesn't do non-greedy matching).
I have file names (from image tiles) consisting of two numbers separated by an underscore, e.g.
142_27.jpg
7_39.jpg
1_120.jpg
How can I (in linux) add leading zeros to both of these numbers? What I want is the file names as
142_027.jpg
007_039.jpg
001_120.jpg
You can use a single awk command to format filenames with leading zeroes using a printf:
for f in *.jpg; do
echo mv "$f" $(awk -F '[_.]' '{printf "%03d_%03d.%s", $1, $2, $3}' <<< "$f")
done
This will output:
mv 142_27.jpg 142_027.jpg
mv 1_120.jpg 001_120.jpg
mv 7_39.jpg 007_039.jpg
Once you're satisfied with the output, remove echo before mv command.
With perl based rename command
$ touch 142_27.jpg 7_39.jpg 1_120.jpg
$ rename -n 's/\d+/sprintf "%03d", $&/ge' *.jpg
rename(1_120.jpg, 001_120.jpg)
rename(142_27.jpg, 142_027.jpg)
rename(7_39.jpg, 007_039.jpg)
The -n option is for dry run, remove it for actual renaming
If perl based rename command is not available:
$ for f in *.jpg; do echo mv "$f" "$(echo "$f" | perl -pe 's/\d+/sprintf "%03d", $&/ge')"; done
mv 1_120.jpg 001_120.jpg
mv 142_27.jpg 142_027.jpg
mv 7_39.jpg 007_039.jpg
Change echo mv to just mv once dry run seems okay
You can do it with a little shell-script using sed:
for i in *.jpg;
do
new=`echo "$i" | sed -n -e '{ s/^\([0-9]\{0,3\}\)_\([0-9]\{0,3\}\).jpg/000\1_000\2.jpg/ };
{s/\([0-9]*\)\([0-9]\{3\}\)_\([0-9]*\)\([0-9]\{3\}\).jpg/\2_\4.jpg/p };'`;
mv "$i" "$new";
done;
I first append three leading zeros at the said places by default and afterwards cut off as many digits as necessary brginning at the start at said places so that only 3 digits are left
with bash substitution(a,b)
windows(bash)
for f in *.jpg;do a=${f%_*};b=${f#*_};mv $f $(printf "%03d_%07s" $a $b);done
linux
for f in *.jpg;do a=${f%_*};b=${f#*_};b=${b%.*};mv $f $(printf "%03d_%03d".jpg $a $b);done
I have a bunch of files that all have a name and a serial number and an extension. I want to extract this serial number and extension. They look like this:
photo-123.jpg
photo-456.png
photo-789.bmp
etc.
I want to run a bash script to extract these serial numbers and place them in a file in this way:
123
456
789
etc.
Note that not all the photos have the same extension (bmp, png, jpg) but they all start with photo-.
You can use parameter substitution:
$ ls
photo-123.jpg photo-456.png photo-7832525239.bmp photo-789.bmp
$ for file in *; do
[[ -f "$file" ]] || continue
[[ $file == "num.log" ]] && continue
file=${file%.*} && echo "${file#*-}"
done > num.log
$ ls
num.log photo-123.jpg photo-456.png photo-7832525239.bmp photo-789.bmp
$ cat num.log
123
456
7832525239
789
${parameter#word} removes the shortest match from the start and ${parameter##word} removes the longest match from the start. ${parameter%word} on the contrary will remove shortest match from the end and ${parameter%%word} will remove longest match from the end.
Alternatively, you can read about nullglob instead of checking for existence of file in event there are no files in the directory. (Thanks Adrian Frühwirth for great feedback)
Using BASH regex:
f='photo-123.jpg'
[[ "$f" =~ -([0-9]+)\. ]] && echo "${BASH_REMATCH[1]}"
123
To run it against all the matching files:
for f in *-[0-9]*.*; do
[[ "$f" =~ -([0-9]+)\. ]] && echo "${BASH_REMATCH[1]}"
done
Assuming you just want to keep all of the numbers and you're using bash, here are a couple of things which you may find useful:
danny#machine:~$ file=abc123def.jpg
danny#machine:~$ echo ${file//[^0123456789]/}
123
danny#machine:~$ echo ${file##*.}
jpg
danny#machine:~$ echo ${file//[^0123456789]/}.${file##*.}
123.jpg
You should be able to write your script based on that. Or, just remove the leading "photo-" from $name by using
newname=$(name#photo-}
Those and several others are explained in the bash man page's Parameter Expansion section.
Or maybe with two consecutive awk calls:
ls -1 | awk -F- '{print $2}' | awk -F. '{print $1}'
How about
ls -l | awk {'print $9'} | grep -o -E '[0-9]*'
in the directory where the files reside?
Say I have a directory with a bunch of site names in it.
i.e.
dev.domain.com
dev.domain2.com
dev.domain3.com
How can I rename those to <domain>.com on the linux cli using piping and/or redirection bash?
I get to a point than am stuck.
find . -maxdepth 1 | grep -v "all" | cut --delimiter="." -f3 | awk '{ print $0 }'
Gives me the domain part, but I can't get past that. Not sure awk is the answer either. Any advice is appreciated.
To strip the leading 'dev.' from names it should be like this:
for i in $(find * -maxdepth 1 -type d); do mv $i $(echo $i | sed 's/dev.\(.*\)/\1/'); done
for i in *; do mv $i $( echo $i | sed 's/\([^\.]*\).\([^\.]*\).\([^\.]*\)/\2.\1/' ); done
Explained:
for i in *; do ....; done
do it for every file
echo $i | sed 's/\([^\.]*\).\([^\.]*\).\([^\.]*\)/\2.\1/'
takes three groups of "every character except ." and changes their order
\2.\1 means: print second group, a dot, first group
the $( ... ) takes output of sed and "pastes" it after mv $i and is called "command substitution" http://www.gnu.org/software/bash/manual/bashref.html#Command-Substitution
Try the rename command. It can take a regular expression like this:
rename 's/\.domain.*/.com/' *.com
under the directory you want to work with, try :
ls -dF *|grep "/$"|awk 'BEGIN{FS=OFS="."} {print "mv "$0" "$2,$3}'
will print mv command. if you want to do the rename, add "|sh" and the end:
ls -dF *|grep "/$"|awk 'BEGIN{FS=OFS="."} {print "mv "$0" "$2,$3}'|sh