I have a log file log .txt which is having the data in below format
Name=abc Date=20140710
Name=xyz Date=20140715
Name=pqr Date=20140810 And so on
I am fetching the data based on today's date and appending it to a log file in new line
today=$(date --date "+1 week" +%Y%m%d)
grep $today log.txt $'\r' >> append_file.txt
But when I am running the script, it is giving me exception like
: No such file or directory
Also, in the append_file.txt, it is keeping the data as
log.txt:Name=abc Date=20140710
Ideally it should keep only the data i.e.
Name=abc Date=20140710
Actually, my end point objective is mail the content of append_file.txt and I want the data line wise...... like this
Name=abc Date=20140710
Name=mno Date=20140710
At present, it is mailing the data in single line Name=abc Date=20140710 Name=mno Date=20140710
Any suggestion ?
Your output looks like:
log.txt:Name=abc Date=20140710
Because grep thinks that you're giving more than one file to work with.
The problem is $'\r' in this line:
grep $today log.txt $'\r' >> append_file.txt
Replace it by:
grep $today log.txt >> append_file.txt
or if you need to insert \r at the end of each line:
grep $today log.txt | sed -e 's/$/\r/g' >> append_file.txt
Related
I'm sorry for my poor English, first.
I want to read a file (tel.txt) that contains many tel numbers (a number per line) and use that line to grep command to search about the specific number in the source file (another file)!
I wrote this code :
dir="/home/mujan/Desktop/data/ADSL_CDR_Text_Parts_A"
file="$dir/tel.txt"
datafile="$dir/ADSL_CDR_Like_Tct4_From_960501_to_97501_Part0.txt"
while IFS= read -r line
do
current="$line"
echo `grep -F $current "$datafile" >> output.txt`
done < $file
the tel file sample :
44001547
44001478
55421487
but that code returns nothing!
when I declare 'current' variable with literals it works correctly!
what happened?!
Your grep command is redirected to write its output to a file, so you don't see it on the terminal.
Anyway, you should probably be using the much simpler and faster
grep -Ff "$file" "$datafile"
Add | tee -a output.txt if you want to save the output to a file and see it at the same time.
echo `command` is a buggy and inefficient way to write command. (echo "`command`" would merely be inefficient.) There is no reason to capture standard output into a string just so that you can echo that string to standard output.
Why don't you search for the line var directly? I've done some tests, this script works on my linux (CentOS 7.x) with bash shell:
#!/bin/bash
file="/home/mujan/Desktop/data/ADSL_CDR_Text_Parts_A/tel.txt"
while IFS= read -r line
do
echo `grep "$line" /home/mujan/Desktop/data/ADSL_CDR_Text_Parts_A/ADSL_CDR_Like_Tct4_From_960501_to_97501_Part0.tx >> output.txt`
done < $file
Give it a try... It shows nothing on the screen since you're redirecting the output to the file output.txt so the matching results are saved there.
You should use file descriptors when reading with while loop.instead use for loop to avoid false re-directions
dir="/home/mujan/Desktop/data/ADSL_CDR_Text_Parts_A"
file="$dir/tel.txt"
datafile="$dir/ADSL_CDR_Like_Tct4_From_960501_to_97501_Part0.txt"
for line in `cat $file`
do
current="$line"
echo `grep -F $current "$datafile" >> output.txt`
done
My first input file contains records name abc.txt:
abc#gmail.com
bscd#yahoo.co.in
abcd.21#gmail.com
1234#hotmail.com
My second file contains record name details.txt:
123456^atulsample^1203320
I want my final file having output to be Final.txt:
abc#gmail.com^123456^atulsample^1203320
bscd#yahoo.co.in^123456^atulsample^1203320
abcd.21#gmail.com^123456^atulsample^1203320
I have uses sed command but I am not getting my required output.
Kindly help as I don't have much knowledge in shell scripting.
try something like this;
#!/bin/bash
while read -r line
do
detail="$line"
sed '/^[ \t]*$/d' abc.txt | sed "s/$/^${detail}/" >> Final.txt
done < "details.txt"
this is to delete blank lines;
sed '/^[ \t]*$/d' abc.txt
this is to append from details.txt
sed "s/$/^${detail}/"
I have an executable that takes a file and outputs a line.
I am running a loop over a directory:
for file in $DIRECTORY/*.png
do
./eval $file >> out.txt
done
The output of executable does not contain the name of the file.
I want to append the file name with each output.
EDIT1
Perhaps, I could not explain it correctly
I want the name of the file and the output of the program as well, which is processing the same file, Now I am doing following
for file in $DIRECTORY/*.png
do
echo -n $file >> out.txt
or
printf "%s" "$file" >> out.txt
./eval $file >> out.txt
done
For both new line is inserted
If I understood your question, what you want is:
get the name of the file,
...and the output or the program processing the file (in your case, eval),
...on the same line. And this last part is your problem.
Then I'd suggest composing a single line of text (using echo), comprising:
the name of the file, this is the $file part,
...followed by a separator, you may not need that but it may help further processing of the result. I used ":". You can skip this part if this is not interesting for you,
...followed by the output of the program processing the file: this is the $(...) construct
echo $file ":" $(./eval $file) >> out.txt
...and finally appending this line of text to a file, you got that part right.
please use like this
echo -n `echo ${file}|tr -d '\n'` >> out.txt
OR
newname=`echo ${file}|tr -d '\n'`
echo -n $newname >> out.txt
I have to extract particular date's log from any application log. what we generally do
vi app.log
/date (searching)
.= (position where date found)
G(last line)
first_line,Last_line w filname.log (to generate log file with particular date)
I made small script but I think My logic is not correct:
#!/bin/bash
read -p " enter the App name" : app
read =p " enter the Date in this Format 10 Jan 20:01:00" : date
FILE=/logs/$app.log
# Check if $app exists or not
if test ! -f "$app" /logs
then
echo "Error - $app not found."
else
First_line=$(cat $FILE | grep $date | .=)
Second_line=$(tail -1 $FILE | .=)
vi $FILE | $First_line, $Second_line w $app.txt
fi
There is an error in the line
if test ! -f "$app" /logs
You seperated "$app" and /logs with a space; test -f expects one parameter, so it should be:
if test ! -f "$app"/logs
To get the log starting from the date given to the current end of the file, You should use sed (meaning "stream editor"), which is designed for noninteractive editing of files:
sed -n '/'$date'/,$ p' $app.log >$app.txt
The parameter -n is to only output, what matched and not the whole file.
Here I am telling sed to extract everything between two regular expressions, the first one, being the $date parameter given, the second one meaning "end of file". You could change this easily to extract all logs for one specific day or whatever.
As something seems to be unclear I am adding a complete example:
I have an apache log file, where the date logged looks like this [01/Apr/2015:22:31:21 +0200].
So I can do:
export date="01\/Apr\/2015"
It is important to note that I am escaping the slashes here, as they are going into a regex. This has to be done with everything that has some special meaning inside regexes too.
Then I can do:
sed -n '/'$date'/,$ p' myaccess.log
and I get everything logged for the 1st of April until the end of the file.
You could pass commands to vi from your script with a here document, if your task is as mechanical as you describe.
I'm talking about something like this
#!/bin/bash
read -p " enter the App name" : app
read =p " enter the Date in this Format 10 Jan 20:01:00" : date
FILE=/logs/$app.log
# Check if $app exists or not
if test ! -f "$app" /logs
then
echo "Error - $app not found."
else
vi $FILE <<-_EOF_
/date
.=
G
first_line,Last_line w filname.log
etc, etc
_EOF_
fi
I just wrote this as an idea that you mighr find fun to explore. I know the code will not work.
I'd use awk:
awk -v date="$date" '$0 ~ date {p=1} p' file.in > file.out
That will start printing lines when the date is first seen, until the end of the line.
I have a folder with lots of files having a pattern, which is some string followed by a date and time:
BOS_CRM_SUS_20130101_10-00-10.csv (3 strings before date)
SEL_DMD_20141224_10-00-11.csv (2 strings before date)
SEL_DMD_SOUS_20141224_10-00-10.csv (3 strings before date)
I want to loop through the folder and extract only the part before the date and output into a file.
Output
BOS_CRM_SUS_
SEL_DMD_
SEL_DMD_SOUS_
This is my script but it is not working
#!/bin/bash
# script variables
FOLDER=/app/list/l088app5304d1/socles/Data/LEMREC/infa_shared/Shell/Check_Header_T24/
LOG_FILE=/app/list/l088app5304d1/socles/Data/LEMREC/infa_shared/Shell/Check_Header_T24/log
echo "Starting the programme at: $(date)" >> $LOG_FILE
# Getting part of the file name from FOLDER
for file in `ls $FOLDER/*.csv`
do
mv "${file}" "${file/date +%Y%m%d HH:MM:SS}" 2>&1 | tee -a $LOG_FILE
done #> $LOG_FILE
Use sed with extended-regex and groups to achieve this.
cat filelist | sed -r 's/(.*)[0-9]{8}_[0-9][0-9]-[0-9][0-9].[0-9][0-9].csv/\1/'
where filelist is a file with all the names you care about. Of course, this is just a placeholder because I don't know how you are going to list all eligible files. If a glob will do, for example, you can do
ls mydir/*.csv | sed -r 's/(.*)[0-9]{8}_[0-9][0-9]-[0-9][0-9].[0-9][0-9].csv/\1/'
Assuming you wont have numbers in the first part, you could use:
$ for i in *csv;do str=$(echo $i|sed -r 's/[0-9]+.*//'); echo $str; done
BOS_CRM_SUS_
SEL_DMD_
SEL_DMD_SOUS_
Or with parameter substitution:
$ for i in *csv;do echo ${i%_*_*}_; done
BOS_CRM_SUS_
SEL_DMD_
SEL_DMD_SOUS_
When you use ${var/pattern/replace}, the pattern must be a filename glob, not command to execute.
Instead of using the substitution operator, use the pattern removal operator
mv "${file}" "${file%_*-*-*.csv}.csv"
% finds the shortest match of the pattern at the end of the variable, so this pattern will just match the date and time part of the filename.
The substitution:
"${file/date +%Y%m%d HH:MM:SS}"
is unlikely to do anything, because it doesn't execute date +%Y%m%d HH:MM:SS. It just treats it as a pattern to search for, and it's not going to be found.
If you did execute the command, though, you would get the current date and time, which is also (apparently) not what you find in the filename.
If that pattern is precise, then you can do the following:
echo "${file%????????_??-??-??.csv}" >> "$LOG_FILE"
using grep:
ls *.csv | grep -Po "\K^([A-Za-z]+_)+"
output:
BOS_CRM_SUS_
SEL_DMD_
SEL_DMD_SOUS_