So I have testfile which contains
Line one
Another line
and this is the third line
My script reads this file, does some stuff and I end up with a variable that should contain it. Kind of doing
filevar=$(cat testfile)
(the important thing here is that I cannot access the file directly).
I'll be using the contents of that variable to generate an HTML code and one things I have to do is to add <br> to the end of each line. The problem is, there doesnt seem to any EOLs in my var:
echo $filevar
Line one Another line and this is the third line
How do I read the file properly to keep the EOLs? Once I have that I can simply sed s/$/<br>/g, but till then...
thanks!
how about changing IFS?
#!/bin/bash
IFS=""
filevar=$(cat test)
echo $filevar
this will output:
Line one
Another line
and this is the third line
I can't understand why do you need to read the file into the variable. Why don't you simply do this:
sed 's|$|<br/>|' testfile
UPDATE:
If you really want to get the EOL back in your variable. Try this (notice the quotes):
echo "$filevar"
But I still can't understand, why you can cat the file but not access the file
As a solution, I would suggest the following script:
while read LINE
do
echo ${LINE} '<br />' # Implement your core logic here.
done < testfile
Instead of doing echo $filevar do echo "$filevar" (note the double quotes). This will send a single argument to echo and then you can pipe this to sed.
With sed, this will be treated as 3 lines, so you do not need the g option. This works with me (bash and cygwin):
echo "$filevar" | sed 's/$/<br>/'
You need to set the IFS variable to contain just a newline, and then reference the filevar variable without quotes.
$ filevar='Line one
Another line
and this is the third line'
$ for word in $filevar; do echo "$word<br>"; done
Line<br>
one<br>
Another<br>
line<br>
and<br>
this<br>
is<br>
the<br>
third<br>
line<br>
$ for word in "$filevar"; do echo "$word<br>"; done
Line one
Another line
and this is the third line<br>
$ (IFS=$'\n'; for word in $filevar; do echo "$word<br>"; done)
Line one<br>
Another line<br>
and this is the third line<br>
Related
I have a bash script which asks for two arguments with a space between them. Now I would like to automate filling out the prompt in the command line with reading from a text file. The text file contains a list with the argument combinations.
So something like this in the command line I think;
for line in 'cat text.file' ; do script.sh ; done
Can this be done? What am I missing/doing wrong?
Thanks for the help.
A while loop is probably what you need. Put the space separated strings in the file text.file :
cat text.file
bingo yankee
bravo delta
Then write the script in question like below.
#!/bin/bash
while read -r arg1 arg2
do
/path/to/your/script.sh "$arg1" "$arg2"
done<text.file
Don't use for to read files line by line
Try something like this:
#!/bin/bash
ARGS=
while IFS= read -r line; do
ARGS="${ARGS} ${line}"
done < ./text.file
script.sh "$ARGS"
This would add each line to a variable which then is used as the arguments of your script.
'cat text.file' is a string literal, $(cat text.file) would expand to output of command however cat is useless because bash can read file using redirection, also with quotes it will be treated as a single argument and without it will split at space tab and newlines.
Bash syntax to read a file line by line, but will be slow for big files
while IFS= read -r line; do ... "$line"; done < text.file
unsetting IFS for read command preserves leading spaces
-r option preserves \
another way, to read whole file is content=$(<file), note the < inside the command substitution. so a creative way to read a file to array, each element a non-empty line:
read_to_array () {
local oldsetf=${-//[^f]} oldifs=$IFS
set -f
IFS=$'\n' array_content=($(<"$1")) IFS=$oldifs
[[ $oldsetf ]]||set +f
}
read_to_array "file"
for element in "${array_content[#]}"; do ...; done
oldsetf used to store current set -f or set +f setting
oldifs used to store current IFS
IFS=$'\n' to split on newlines (multiple newlines will be treated as one)
set -f avoid glob expansion for example in case line contains single *
note () around $() to store the result of splitting to an array
If I were to create a solution determined by the literal of what you ask for (using a for loop and parsing lines from a file) I would use iterations determined by the number of lines in the file (if it isn't too large).
Assuming each line has two strings separated by a single space (to be used as positional parameters in your script:
file="$1"
f_count="$(wc -l < $file)"
for line in $(seq 1 $f_count)
do
script.sh $(head -n $line $file | tail -n1) && wait
done
You may have a much better time using sjsam's solution however.
i want to replace first line in another text file using sed.
In the replaced line must be $variable.
The only (reasonably simple) way to do this safely is to
Remove the first line.
Combine the modified file with the new first line in a temporary file.
Replace the original file with the temporary file.
One way to do that is
{ printf '%s\n' "$variable"; sed '1d' original.txt; } > tmp.txt && mv tmp.txt original.txt
var1=$(systemd-escape $var)
sed -ri '1s#^.*$#'"$var1"'#' filename
Where $var references your dynamic variable, we concentrate on the first line and then replace anything from the start to the end of the line with the var variable. Pay attention to the position of the single quotes which will encompass the expansion of the variable into the sed statement.
I am trying to replace a specific line in a txt file with my shell script, for example;
cat aa.txt:
auditd=0
bladeServerSlot=0
When I run my script I would like to change "bladeServerSlot" to 12 as following;
cat aa.txt:
auditd=0
bladeServerSlot=12
Could you please help me?
Using sed and backreferencing:
sed -r '/bladeServerSlot/ s/(^.*)(=.*)/\1=12/g' inputfile
Using awk , this will search for the line which contains bladeServerSlot and replace the second column of that line.
awk 'BEGIN{FS=OFS="="}/bladeServerSlot/{$2=12}1' inputfile
perl -pe 's/bladeServerSlot=\K\d+/12/' aa.txt > output.txt
The \K is a particular form of the positive lookbehind, which discards all previous matches. So we need to replace only what follows. The s/ is applied by default to $_, which contains the current line. The -p prints $_ for every line, so all other lines are copied. We redirect output to a file.
Is it really necessary to replace the line in your example? As bladeServerSlot is a variable you could reset the value.
bladeServerSlot=`any command`
Or you could just let this variable be filled by a Parameter provided to this script.
bladeServerSlot=$1
With $1being the first parameter of your script. I think this would be the cleaner way do solve your issue than to do fancy regex here. The sed/perl solutions will work, but they are not very clear to other people reading your script.
Say I have some arbitrary multi-line text file:
sometext
moretext
lastline
How can I remove only the last character (the e, not the newline or null) of the file without making the text file invalid?
A simpler approach (outputs to stdout, doesn't update the input file):
sed '$ s/.$//' somefile
$ is a Sed address that matches the last input line only, thus causing the following function call (s/.$//) to be executed on the last line only.
s/.$// replaces the last character on the (in this case last) line with an empty string; i.e., effectively removes the last char. (before the newline) on the line.
. matches any character on the line, and following it with $ anchors the match to the end of the line; note how the use of $ in this regular expression is conceptually related, but technically distinct from the previous use of $ as a Sed address.
Example with stdin input (assumes Bash, Ksh, or Zsh):
$ sed '$ s/.$//' <<< $'line one\nline two'
line one
line tw
To update the input file too (do not use if the input file is a symlink):
sed -i '$ s/.$//' somefile
Note:
On macOS, you'd have to use -i '' instead of just -i; for an overview of the pitfalls associated with -i, see the bottom half of this answer.
If you need to process very large input files and/or performance / disk usage are a concern and you're using GNU utilities (Linux), see ImHere's helpful answer.
truncate
truncate -s-1 file
Removes one (-1) character from the end of the same file. Exactly as a >> will append to the same file.
The problem with this approach is that it doesn't retain a trailing newline if it existed.
The solution is:
if [ -n "$(tail -c1 file)" ] # if the file has not a trailing new line.
then
truncate -s-1 file # remove one char as the question request.
else
truncate -s-2 file # remove the last two characters
echo "" >> file # add the trailing new line back
fi
This works because tail takes the last byte (not char).
It takes almost no time even with big files.
Why not sed
The problem with a sed solution like sed '$ s/.$//' file is that it reads the whole file first (taking a long time with large files), then you need a temporary file (of the same size as the original):
sed '$ s/.$//' file > tempfile
rm file; mv tempfile file
And then move the tempfile to replace the file.
Here's another using ex, which I find not as cryptic as the sed solution:
printf '%s\n' '$' 's/.$//' wq | ex somefile
The $ goes to the last line, the s deletes the last character, and wq is the well known (to vi users) write+quit.
After a whole bunch of playing around with different strategies (and avoiding sed -i or perl), the best way i found to do this was with:
sed '$! { P; D; }; s/.$//' somefile
If the goal is to remove the last character in the last line, this awk should do:
awk '{a[NR]=$0} END {for (i=1;i<NR;i++) print a[i];sub(/.$/,"",a[NR]);print a[NR]}' file
sometext
moretext
lastlin
It store all data into an array, then print it out and change last line.
Just a remark: sed will temporarily remove the file.
So if you are tailing the file, you'll get a "No such file or directory" warning until you reissue the tail command.
EDITED ANSWER
I created a script and put your text inside on my Desktop. this test file is saved as "old_file.txt"
sometext
moretext
lastline
Afterwards I wrote a small script to take the old file and eliminate the last character in the last line
#!/bin/bash
no_of_new_line_characters=`wc '/root/Desktop/old_file.txt'|cut -d ' ' -f2`
let "no_of_lines=no_of_new_line_characters+1"
sed -n 1,"$no_of_new_line_characters"p '/root/Desktop/old_file.txt' > '/root/Desktop/my_new_file'
sed -n "$no_of_lines","$no_of_lines"p '/root/Desktop/old_file.txt'|sed 's/.$//g' >> '/root/Desktop/my_new_file'
opening the new_file I created, showed the output as follows:
sometext
moretext
lastlin
I apologize for my previous answer (wasn't reading carefully)
sed 's/.$//' filename | tee newFilename
This should do your job.
A couple perl solutions, for comparison/reference:
(echo 1a; echo 2b) | perl -e '$_=join("",<>); s/.$//; print'
(echo 1a; echo 2b) | perl -e 'while(<>){ if(eof) {s/.$//}; print }'
I find the first read-whole-file-into-memory approach can be generally quite useful (less so for this particular problem). You can now do regex's which span multiple lines, for example to combine every 3 lines of a certain format into 1 summary line.
For this problem, truncate would be faster and the sed version is shorter to type. Note that truncate requires a file to operate on, not a stream. Normally I find sed to lack the power of perl and I much prefer the extended-regex / perl-regex syntax. But this problem has a nice sed solution.
I am trying to append a variable at the last character of a specific line of a file from a bash script.
The file is called myfile.txt and what I want to do is to append the contents of a variable named VERSION right after the last character of the line of the file that contains the unique string MYVERSION.
That is, if in this line there is the following:
MYVERSION=0.1
and VERSION="-custom_P1" then, I want to have the following:
MYVERSION=0.1-custom_P1
Thank you all for the help.
Try this:
sed -i "/^MYVERSION=/ s/\$/$VERSION/" myfile.txt
The idea is that it finds a line that starts with MYVERSION= and then replaces the end of that line with the contents of the $VERSION environment variable.
Edit: originally I wasn't sure if the first $ needed to be escaped, but #sehe's comment and its upvoters convinced me that it does.
Try
sed -e "s/^MYVERSION=/MYVERSION=.*$/&${VERSION}/g" < myfile.txt
The command appends the value of VERSION to the line with 'MYVERSION='