My log file is in the format upd_yyyymmdd_slr.stats. I want to copy all files from a server to my local server between 2 dates, for example.
I want all files having dates in log file as 20151228 and 20160103 means I want to copy the below files from a set of many files.
upd_20151228_slr.stats
upd_20151229_slr.stats
upd_20151230_slr.stats
upd_20151231_slr.stats
upd_20160101_slr.stats
upd_20160102_slr.stats
upd_20160103_slr.stats
I'm using the below command but it is not working
scp server:/reports/logs/upd_20[15-16][12-01][29-03]*slr* ./log_files
Please find me the best way I can do in Linux as well as in shell scripting
Check out this script:
#!/bin/bash
for i in `ls upd_*_slr.stats`; do
# First strip off prefix and suffix strings.
s=`echo $i | sed 's/upd_//g' | sed 's/_slr.stats//g'`
if [[ $s -ge 20151230 && $s -le 20160102 ]]; then # Modify dates as needed
echo $i
cp $i /path/to/dest/dir
fi
done
EDIT:
In case of a remote server one could try this:
#!/bin/bash
for i in `ssh username#remoteaddress ls /path/to/dir/upd_*_slr.stats`; do
i=${i##*/}
# First strip off extra strings leaving only date.
s=`echo $i | sed 's/upd_//g' | sed 's/_slr.stats//g'`
if [[ $s -ge 20151230 && $s -le 20160102 ]]; then # Modify dates as needed
echo $i
scp username#remoteaddress:/path/to/dir/$i /path/to/dest/dir
fi
done
Related
sorry for that odd title. I didn't know how to word it the right way.
I'm trying to write a script to filter my wiki files to those got directories with the same name and the ones without. I'll elaborate further.
here is my file system:
what I need to do is print a list of those files which have directories in their name and another one of those without.
So my ultimate goal is getting:
with dirs:
Docs
Eng
Python
RHEL
To_do_list
articals
without dirs:
orphan.txt
orphan2.txt
orphan3.txt
I managed to get those files with dirs. Here is me code:
getname () {
file=$( basename "$1" )
file2=${file%%.*}
echo $file2
}
for d in Mywiki/* ; do
if [[ -f $d ]]; then
file=$(getname $d)
for x in Mywiki/* ; do
dir=$(getname $x)
if [[ -d $x ]] && [ $dir == $file ]; then
echo $dir
fi
done
fi
done
but stuck with getting those without. if this is the wrong way of doing this please clarify the right one.
any help appreciated. Thanks.
Here's a quick attempt.
for file in Mywiki/*.txt; do
nodir=${file##*/}
test -d "${file%.txt}" && printf "%s\n" "$nodir" >&3 || printf "%s\n" "$nodir"
done >with 3>without
This shamelessly uses standard output for the non-orphans. Maybe more robustly open another separate file descriptor for that.
Also notice how everything needs to be quoted unless you specifically require the shell to do whitespace tokenization and wildcard expansion on the value of a token. Here's the scoop on that.
That may not be the most efficient way of doing it, but you could take all files, remove the extension, and the check if there isn't a directory with that name.
Like this (untested code):
for file in Mywiki/* ; do
if [ -f "$d" ]; then
dirname=$(getname "$d")
if [ ! -d "Mywiki/$dirname" ]; then
echo "$file"
fi
fi
done
To List all the files in current dir
list1=`ls -p | grep -v /`
To List all the files in current dir without extension
list2=`ls -p | grep -v / | sed 's/\.[a-z]*//g'`
To List all the directories in current dir
list3=`ls -d */ | sed -e "s/\///g"`
Now you can get the desired directory listing using intersection of list2 and list3. Intersection of two lists in Bash
My requirement is simple
I have a file with a list of paths provided in a line.
Eg:
/etc/home
/var/www/
I need a command to validate that there are no space characters in between the path like: /var/ www/
How can I achieve this validation?
This should help. It uses the Internal Field Separator.
#!/bin/sh
if [ ! $1 ]; then
echo Filename parameter expected
exit
fi
export IFS=$'\n'
for i in `cat $1`
do
if [ `echo $i | grep ' ' ` ]; then
echo Spaces found in [$i]
else
echo No spaces found in [$i]
fi
done
I have a series of directories that are only different by a numerical tag.
arr=(0 1 2 3)
i=0
while [ $i -le ${arr}]
do
dir="~Documents/seed"
dir+=${arr[i]}
echo $dir #works
cd dir #directory not found
#do other things#
done
Is it possible to do this?
This might be easier:
#!/bin/bash
for d in ~/Dcouments/seed*
do
if [ -d "$d" ]; then
echo $d
fi
done
Note:
You have tarfiles in ~/Documents too (with names that also match the wildcard), so I have added an if statement that checks if it is a directory or a file and only reacts to directories.
So far i have tried below unix script to append the contents of files on the basis of search. Everything is fine. But if i run the same below script multiple times the perl command is keep on appending in the files.
searchArr=(test1 test2 test3)
replaceArr=(somestring1 somestring2 somestring3)
a=`echo ${#searchArr[#]}`
echo $a
for ((i=0;i<$a;i++))
do
searchValue=`echo ${searchArr[i]}`
replaceValue=`echo ${replaceArr[i]}`
for file in `find . -name \* -print`; do
grep "$searchValue" $file &> /dev/null
if [ $? -ne 0 ]; then
echo "Search string not found in $file!"
else
perl -p -i -e "s/$searchValue/$replaceValue/g" $file
echo $file "Replace string Success!"
fi
done
done
Current Output:-
If run more than one times it will keep on appending like that.
For Example if run two times it will append two times
somestring1somestring1test1
somestring2somestring2test2
somestring3somestring3test3
Needed Output:-
No matter how many times it runs
somestring1test1
somestring2test2
somestring3test3
Add word boundaries to your sed search pattern.
Beginning and end of words in sed and grep
Hello from a Linux Bash newbie!
I have a list.txt containing a list of files which I want to copy to a destination($2). These are unique images but some of them have the same filename.
My plan is to loop through each line in the text file, with the copy to the destination occurring when the file is not there, and a mv rename happening when it is present.
The problem I am having is creating the pathname to check the file against. In the code below, I am taking the filename only from the pathname, and I want to add that to the destination ($2) with the "/" in between to check the file against.
When I run the program below I get "Permission Denied" at line 9 which is where I try and create the path.
for line in $(cat list.txt)
do
file=$[ basename $line ]
path=$[ $2$file ]
echo $path
if [ ! -f $path ];
then
echo cp $line $2
else
echo mv $line.DUPLICATE $2
fi
done
I am new to this so appreciate I may be missing something obvious but if anyone can offer any advice it would be much appreciated!
Submitting this since OP is new in BASH scripting no good answer has been posted yet.
DESTINATION="$2"
while read -r line; do
file="${line##*/}"
path="$2/$file"
[[ ! -f $path ]] && cp "$line" "$path" || mv "$line" "$path.DUP"
done < list.txt
Don't have logic for counting duplicates at present to keep things simple. (Which means code will take care of one dup entry) As an alternative you get uniq from list.txt beforehand to avoid the duplicate situation.
#anubhava: Your script looks good. Here is a small addition to it to work with several dupes.
It adds a numer to the $path.DUP name
UniqueMove()
{
COUNT=0
while [ -f "$1" ]
do
(( COUNT++ ))
mv -n "$1" "$2$COUNT"
done
}
while read -r line; do
file="${line##*/}"
path="$2/$file"
[[ ! -f $path ]] && cp "$line" "$path" || UniqueMove "$line" "$path.DUP"
done < list.txt