I have a bash script that will loop through a directory getting each file name. What I would like to do is create a few symbolic links for these files. Except I want to change the link names.
Example 1:
File Name: testFile.so.3.4.5
ln -s testFile.so.3.4.5 testFile.so.3
ln -s testFile.so.3 testFile.so
Example 2:
File Name: testLink.so.4.4
ln -s testLink.so.4.4 testLink.so.4
ln -s testLink.so.4 testLink.so
So I need to transform the file name twice. The first time removing everything except the first number after *.so. The second time removing everything after *.so.
This is what I have so far. I know it's not much:
#! /bin/bash
# clear any info on screen
clear
# greeting
echo "Starting the script!"
# loop through all files in the directory
for f in *
do
echo "Processing: $f"
done
I'm a bit new to bash and file name transformations, so any help or guidance would be appreciated.
Using a combination of bash extended regular expressions and parameter expansion
for file in *.so.*
do
regex='(.*\.so\.[^.]*)\..*'
if [[ $file =~ $regex ]]
then
tempfile="${BASH_REMATCH[1]}"
ln -s "$file" "$tempfile"
ln -s "$tempfile" "${tempfile%.*}"
fi
done
Also more generally, using only parameter expansion:
for f in *.so.*.*
do
if [ -e "$f" ]; then
base=${f%".${f#*.so.*.*}"}
ln -s "$f" "$base"
ln -s "$base" "${base%.*}"
fi
done
or more generally:
files='libfoo.so.1.2.3.4.5 libbar.so libqux.so.1'
for f in $files; do
while test ${f##*.} != so; do
link=${f%.*}
ln -s $f $link
f=$link
done
done
this will create libfoo.so.1.2.3.4 -> libfoo.so.1.2.3.4.5, libfoo.so.1.2.3 -> libfoo.so.1.2.3.4, libfoo.so.1.2 -> libfoo.so.1.2.3, libfoo.so.1 -> libfoo.so.1.2, libfoo.so -> libfoo.so.1, and libqux.so -> libqux.so.1; libbar.so will be ignored.
Related
You will have to forgive me I have very little experience writing Linux Scripts. Ok What I'm trying to do is rename part of a file that has a specified name in, but the problem I'm coming across is I get the error during my For Loop is this 0403-011 The specified substitution is not valid for this command I'm not sure what I'm doing wrong in my for loop, can someone please assist?
#Creates Directory
echo "Name of New Directory"
read newdir
if [[ -n "$newdir" ]]
then
mkdir $newdir
fi
echo $userInput Directory Created
echo
echo "Directory you wish to Copy?"
read copydir
if [[ -n "$copydir" ]]
then
#Copies contents of Specified Directory
cp -R $copydir/!(*.UNC) $newdir;
#Searches through directory
for file in $newdir/$copydir*; do
mv -v -- "$file" "${file/old/new}";
done
fi
Which version of ksh are you using?
"${file//old/new}" and "${file/old/new}" are valid syntaxes in ksh93.
If your env is ksh88 "${file//old/new}" substitution is not supported.
You have to use sed/tr to replace pattern. Here is an example with sed.
mv -v -- "$file" "$(echo ${file}|sed 's/old/new/')"
The offending line:
mv -v -- "$file" "${file/old/new}";
should be:
mv -v -- "$file" "${file//old/new}";
If you want to replace $old with $new (as opposed to the literal string "old" with "new"), write:
mv -v -- "$file" "${file//$old/$new}";
I am trying to write a bash/shell script to zip up a specific folder and ignore certain sub-dirs in that folder.
This is the folder I am trying to zip "sync_test5":
My bash script generates an ignore list (based on) and calls the zip function like this:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR="dir_to_ignore dir2_to_ignore"
ignorelist=""
if [ "$SYNC_WEB_ROOT_IGNORE_DIR" != "" ];
then
for ignoredir in $SYNC_WEB_ROOT_IGNORE_DIR
do
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip"
cd $SYNC_WEB_ROOT_BASE_DIR;
zip -r $FILE $SYNC_WEB_ROOT_BACKUP_DIR -x $ignorelist >/dev/null
echo "Done"
Now this script runs without error, however it is not ignoring/excluding the dirs I've specified.
So, I had the shell script output the command it tried to run, which was:
zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
Now If I run the above command directly in putty like this, it works:
So, why doesn't my shell script exclude working as intended? the command that is being executed is identical (in shell and putty directly).
Because backslash quotings in a variable after word splitting are not evaluated.
If you have a='123\4', echo $a would give
123\4
But if you do it directly like echo 123\4, you'd get
1234
Clearly the arguments you pass with the variable and without the variables are different.
You probably just meant to not quote your argument with backslash:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Btw, what actual works is a non-evaluated glob pattern:
zip -r 12-08-2014_072810.website.zip sync_test5 -x 'sync_test5/dir_to_ignore/***' 'sync_test5/dir2_to_ignore/***'
You can verify this with
echo zip -r 12-08-2014_072810.website.zip sync_test5 -x sync_test5/dir_to_ignore/**\* sync_test5/dir2_to_ignore/**\*
And this is my suggestion:
#!/bin/bash
SYNC_WEB_ROOT_BASE_DIR="/home/www-data/public_html"
SYNC_WEB_ROOT_BACKUP_DIR="sync_test5"
SYNC_WEB_ROOT_IGNORE_DIR=("dir_to_ignore" "dir2_to_ignore")
IGNORE_LIST=()
if [[ -n $SYNC_WEB_ROOT_IGNORE_DIR ]]; then
for IGNORE_DIR in "${SYNC_WEB_ROOT_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/***") ## "$SYNC_WEB_ROOT_BACKUP_DIR/$IGNORE_DIR/*" perhaps is enough?
done
fi
FILE="$SYNC_BACKUP_DIR/$DATETIMENOW.website.zip" ## Where is $SYNC_BACKUP_DIR set?
cd "$SYNC_WEB_ROOT_BASE_DIR";
zip -r "$FILE" "$SYNC_WEB_ROOT_BACKUP_DIR" -x "${IGNORE_LIST[#]}" >/dev/null
echo "Done"
This is what I ended up with:
#!/bin/bash
# This script zips a directory, excluding specified files, types and subdirectories.
# while zipping the directory it excludes hidden directories and certain file types
[[ "`/usr/bin/tty`" == "not a tty" ]] && . ~/.bash_profile
DIRECTORY=$(cd `dirname $0` && pwd)
if [[ -z $1 ]]; then
echo "Usage: managed_directory_compressor /your-directory/ zip-file-name"
else
DIRECTORY_TO_COMPRESS=${1%/}
ZIPPED_FILE="$2.zip"
COMPRESS_IGNORE_FILE=("\.git" "*.zip" "*.csv" "*.json" "gulpfile.js" "*.rb" "*.bak" "*.swp" "*.back" "*.merge" "*.txt" "*.sh" "bower_components" "node_modules")
COMPRESS_IGNORE_DIR=("bower_components" "node_modules")
IGNORE_LIST=("*/\.*" "\.* "\/\.*"")
if [[ -n $COMPRESS_IGNORE_FILE ]]; then
for IGNORE_FILES in "${COMPRESS_IGNORE_FILE[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_FILES/*")
done
for IGNORE_DIR in "${COMPRESS_IGNORE_DIR[#]}"; do
IGNORE_LIST+=("$DIRECTORY_TO_COMPRESS/$IGNORE_DIR/")
done
fi
zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
# echo zip -r "$ZIPPED_FILE" "$DIRECTORY_TO_COMPRESS" -x "${IGNORE_LIST[#]}" # >/dev/null
echo $DIRECTORY_TO_COMPRESS "compressed as" $ZIPPED_FILE.
fi
After a few trial and error, I have managed to fix this problem by changing this line:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/**\*"
to:
ignorelist="$ignorelist $SYNC_WEB_ROOT_BACKUP_DIR/$ignoredir/***"
Not sure why this worked, but it does :)
How to write simple shell script to create zip file.
I want to create zip file by collecting files with same string pattern in their file names from a folder.
For example, there may be many files under a folder.
xxxxx_20140502_xxx.txt
xxxxx_20140502_xxx.txt
xxxxx_20140503_xxx.txt
xxxxx_20140503_xxx.txt
xxxxx_20140504_xxx.txt
xxxxx_20140504_xxx.txt
After running the shell script, the result must be following three zip files.
20140502.zip
20140503.zip
20140504.zip
Please give me right direction to create simple shell script to output the result as above.
#!/bin/bash
for file in *_????????_*.csv *_????????_*.txt; do
[ -f "${file}" ] || continue
date=${file#*_} # adjust this and next line depending
date=${date%_*} # on your actual prefix/suffix
echo "${date}"
done | sort -u | while read date; do
zip "${date}.zip" *${date}*
done
Since zip will update the archive, this will do:
shopt -s nullglob
for file in *.{txt,csv}; do [[ $file =~ _([[:digit:]]{8})_ ]] && zip "${BASH_REMATCH[1]}.zip" "$file"; done
The shopt -s nullglob is because you don't want to have unexpanded globs if there are no matching files.
Everything below this line is my old answer...
First, get all the possible dates. Heuristically, this could be the files ending in .txt and .csv that match the regex _[[:digit:]]{8}_:
#!/bin/bash
shopt -s nullglob
declare -A dates=()
for file in *.{csv,txt}; do
[[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=
done
printf "Date found: %s\n" "${!dates[#]}"
This will output to stdout all the dates found in the files. E.g. (I called the previous snipped gorilla and I chmod +x gorilla and touched a few files for demo):
$ ls
banana_20010101_gorilla.csv gorilla_20140502_bonobo.csv
gorilla notthisone_123_lol.txt
gorilla_20140502_banana.txt
$ ./gorilla
Date found: 20140502
Date found: 20010101
Next step, for each date found, get all the files ending in .txt and .csv and zip them in the archive corresponding to the date: appending this to gorilla will do the job:
for date in "${!dates[#]}"; do
zip "$date.zip" *"_${date}_"*.{csv,txt}
done
Full script after removing the flooding part:
#!/bin/bash
shopt -s nullglob
declare -A dates=()
for file in *.{csv,txt}; do
[[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=
done
for date in "${!dates[#]}"; do
zip "$date.zip" *"_${date}_"*.{csv,txt}
done
Edit. I overlooked your requirement with one line command. Then here's the one-liner:
shopt -s nullglob; declare -A dates=(); for file in *.{csv,txt}; do [[ $file =~ _([[:digit:]]{8})_ ]] && dates[${BASH_REMATCH[1]}]=; done; for date in "${!dates[#]}"; do zip "$date.zip" *"_${date}_"*.{csv,txt}; done
:)
#! /bin/bash
dates=$(ls ?????_[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]_???.{csv,txt} \
| cut -f2 -d_ | sort -u)
for date in $dates ; do
zip $date.zip ?????_"$date"_???.{csv,txt}
done
I have a script that has to process some files (name beginning with AB) in a directory.
The code is :
for file in AB*
do
cp ...
...
done
When there are no *.txt files in the folder the code executes anyway 1 time.
But then there are errors because I try to copy a file that doesn't exist.
How can I make that the do-command doesn't execute when the result of the ls-command is empty?
I already tried using ls, quotes an combinations > nothing gives the result I want.
Maybe you can add a condition before:
if [ $(ls AB* 2>/dev/null) ]; then
for ...
fi
with 2>/dev/null you catch the errors not to be printed.
The other answers are just plain wrong in Bash. Do not use them! Please, always observe this rule:
Each time you use globs in Bash, use them with either shopt -s nullglob or shopt -s failglob.
If you observe this rule, you'll always be safe. In fact, each time you don't observe this rule, God kills a kitten.
shopt -s nullglob: in this case, a non-matching glob expands to nothing. Look:
$ mkdir Test; cd Test
$ shopt -u nullglob # I'm explicitly unsetting nullglob
$ echo *
*
$ for i in *; do echo "$i"; done
*
$ # Dear, God has killed a kitten :(
$ # but it was only for demonstration purposes, I swear!
$ shopt -s nullglob # Now we're going to save lots of kittens
$ echo *
$ for i in *; do echo "$i"; done
$ # Wow! :)
shopt -s failglob: in this case, Bash will raise an explicit error when a glob has no expansions. Look:
$ mkdir Test; cd Test
$ shopt -u nullglob # Unsetting nullglob
$ shopt -s failglob # Setting failglob for the love of kittens
$ echo *
bash: no match: *
$ # cool :) what's the return code of this?
$ echo $?
1
$ # who cares, anyway? and a for loop?
$ for i in *; do echo "$i"; done
bash: no match: *
$ # cool :)
Using either nullglob or failglob, you're sure to not launch random commands with uncontrolled arguments!
Cheers!
You probably need the bash test builtin, often abbreviated as [ , sothing like
if [ -f output.txt ] ; then
beware: spaces are important above.
I have an inotify wait script that will move a file from one location to another whenever it detects that a file has been uploaded to the source directory.
The challenge I am facing is that i need to retain the basename of the file and convert the following extensions: .JPEG, .JPG, .jpeg to .jpg so that the file is renamed with the .jpg extension only.
Currently I have this:
TARGET="/target"
SRC="/source"
( while [ 1 ]
do inotifywait -m -r -e close_write --format %f -q \
$SRC | while read F
do mv "$SRC/$F" $TARGET
done
done ) &
So I need a way to split out and test for those non standard extensions and move the file with the correct extension. All files not having those 4 extensions just get moved as is.
Thanks!
Dave
if [[ "$F" =~ .JPEG\|jpg\|jpeg\|jpg ]];then
echo mv $F ${F%.*}.jpg
fi
Using extglob option with some parameter expansion:
#! /bin/bash
shopt -s extglob
TARGET=/target
SRC=/source
( while : ; do
inotifywait -m -r -r close_write --format %f -q \
$SRC | while read F ; do
basename=${F##*/} # Remove everything before /
ext=${basename##*.} # Remove everything before .
basename=${basename%.$ext} # Remove .$ext at the end
if [[ $ext == #(JPG|JPEG|jpeg) ]] ; then # Match any of the words
ext=jpg
fi
echo mv "$F" "$TARGET/$basename.$ext"
done
done ) &
Try this format. (Updated)
TARGET="/target"
SRC="/source"
(
while :; do
inotifywait -m -r -e close_write --format %f -q "$SRC" | while IFS= read -r F; do
case "$F" in
*.jpg)
echo mv "$SRC/$F" "$TARGET/" ## Move as is.
;;
*.[jJ][pP][eE][gG]|*.[jJ][pP][gG])
echo mv "$SRC/$F" "$TARGET/${F%.*}.jpg" ## Move with new proper extension.
;;
esac
done
done
) &
Remove echo from the mv commands if you find it correct already. Also it's meant for bash but could also be compatible with other shells. If you get an error with the read command try to remove the -r option.