I am trying to append a variable at the last character of a specific line of a file from a bash script.
The file is called myfile.txt and what I want to do is to append the contents of a variable named VERSION right after the last character of the line of the file that contains the unique string MYVERSION.
That is, if in this line there is the following:
MYVERSION=0.1
and VERSION="-custom_P1" then, I want to have the following:
MYVERSION=0.1-custom_P1
Thank you all for the help.
Try this:
sed -i "/^MYVERSION=/ s/\$/$VERSION/" myfile.txt
The idea is that it finds a line that starts with MYVERSION= and then replaces the end of that line with the contents of the $VERSION environment variable.
Edit: originally I wasn't sure if the first $ needed to be escaped, but #sehe's comment and its upvoters convinced me that it does.
Try
sed -e "s/^MYVERSION=/MYVERSION=.*$/&${VERSION}/g" < myfile.txt
The command appends the value of VERSION to the line with 'MYVERSION='
Related
I need to read a file into an array and concatenate a string at the end of each line. Here is my bash script:
#!/bin/bash
IFS=$'\n' read -d '' -r -a lines < ./file.list
for i in "${lines[#]}"
do
tmp="$i"
tmp="${tmp}stuff"
echo "$tmp"
done
However, when I do this, an action of replace happens, instead of concatenation.
For example, in the file.list, we have:
http://www.example1.com
http://www.example2.com
What I need is:
http://www.example1.comstuff
http://www.example2.comstuff
But after executing the script above, I get things as below on the terminal:
stuff//www.example1.com
stuff//www.example2.com
Btw, my PC is Mac OS.
The problem also occurs while concatenating strings via awk, printf, and echo commands. For example echo $tmp"stuff" or echo "${tmp}""stuff"
The file ./file.lst is, most probably, generated on a Windows system or, at least, it was saved using the Windows convention for end of line.
Windows uses a sequence of two characters to mark the end of lines in a text file. These characters are CR (\r) followed by LF (\n). Unix-like systems (Linux and macOS starting with version 10) use LF as end of line character.
The assignment IFS=$'\n' in front of read in your code tells read to use LF as line separator. read doesn't store the LF characters in the array it produces (lines[]) but each entry from lines[] ends with a CR character.
The line tmp="${tmp}stuff" does what is it supposed to do, i.e. it appends the word stuff to the content of the variable tmp (a line read from the file).
The first line read from the input file contains the string http://www.example1.com followed by the CR character. After the string stuff is appended, the content of variable tmp is:
http://www.example1.com$'\r'stuff
The CR character is not printable. It has a special interpretation when it is printed on the terminal: it sends the cursor at the start of the line (column 1) without changing the line.
When echo prints the line above, it prints (starting on a new line) http://www.example1.com, then the CR character that sends the cursor back to the start of the line where is prints the string stuff. The stuff fragment overwrites the first 5 characters already printed on that line (http:) and the result, as it is visible on screen, is:
stuff//www.example1.com
The solution is to get rid of the CR characters from the input file. There are several ways to accomplish this goal.
A simple way to remove the CR characters from the input file is to use the command:
sed -i.bak s/$'\r'//g file.list
It removes all the CR characters from the content of file file.list, saves the updated string back into the file.list file and stores the original file.list file as file.list.bak (a backup copy in case it doesn't produce the output you expect).
Another way to get rid of the CR character is to ask the shell to remove it in the command where stuff is appended:
tmp="${tmp/$'\r'/}stuff"
When a variable is expanded in a construct like ${tmp/a/b}, all the appearances of a in $tmp are replaced with b. In this case we replace \r with nothing.
I'm guessing it's have something to do with the Carriage Return character.
Did your file.list created on windows? If so, try to use dos2unix before running the script.
Edit
You can check your files using the file command.
Example:
file file.list
If you saved the file in Windows Notepad like this:
Then it will probably come up like this:
file.list: ASCII text, with no line terminators
You can use built in tools like iconv to convert the encodings. However for a simple use like this, you can just use a command that works for multiple encodings without any conversion necessary.
You could simply buffer the file through cat, and use a regular expression that applies to either:
Carriage return followed by line terminator, or
Line terminator on it's own
Then append the string.
Example:
cat file.list | grep -E -v "^$" | sed -E -e "s/(\r?$)/stuff/g"
Will work with ASCII text, and ASCII text with no line terminators.
If you need to modify a stream to append a fixed string, you can use sed or awk, for instance:
sed 's/$/stuff/'
to append stuff to the end of each line.
using "dos2unix file.list" would also solve the problem
I am trying to replace a specific line in a txt file with my shell script, for example;
cat aa.txt:
auditd=0
bladeServerSlot=0
When I run my script I would like to change "bladeServerSlot" to 12 as following;
cat aa.txt:
auditd=0
bladeServerSlot=12
Could you please help me?
Using sed and backreferencing:
sed -r '/bladeServerSlot/ s/(^.*)(=.*)/\1=12/g' inputfile
Using awk , this will search for the line which contains bladeServerSlot and replace the second column of that line.
awk 'BEGIN{FS=OFS="="}/bladeServerSlot/{$2=12}1' inputfile
perl -pe 's/bladeServerSlot=\K\d+/12/' aa.txt > output.txt
The \K is a particular form of the positive lookbehind, which discards all previous matches. So we need to replace only what follows. The s/ is applied by default to $_, which contains the current line. The -p prints $_ for every line, so all other lines are copied. We redirect output to a file.
Is it really necessary to replace the line in your example? As bladeServerSlot is a variable you could reset the value.
bladeServerSlot=`any command`
Or you could just let this variable be filled by a Parameter provided to this script.
bladeServerSlot=$1
With $1being the first parameter of your script. I think this would be the cleaner way do solve your issue than to do fancy regex here. The sed/perl solutions will work, but they are not very clear to other people reading your script.
Lets say I have a file which is something like this:
"Testing is important"
Nothing is impossible
The output should be:
Nothing is impossible
This means the sed removed everything before new line. Also, I need to make sure it works on bash on windows.
Please help.
You can try this
sed '1,/^\s*$/d' file
\s is whitespace, it's same with
sed '1,/^[[:blank:]]*$/d' file
Sed supports addressing lines both as numbers and as matching regex. In your case, you can delete all lines starting from 1, and ending with an empty line:
sed -e '1,/^$/d'
On Windows your files may contain contain carriage returns, in which case you can use:
sed -e '1,/^\r*$/d'
(assuming GNU sed)
To allow for more than one blank line and multiple lines after the final blank line, you could use something like this:
awk 'BEGIN{RS=ORS=""}{a=$0}END{print a}' file
This unsets the Record Separator RS, so that each block/paragraph is treated as a separate record. It assigns each record to the variable a, then prints the last value of a once the file has been processed. The Output Record Separator ORS is also unset, so that no newline is appended to the final block.
I am trying to add a line to a bash script that does a bunch of other stuff and what I want it to do is write to the end of a line in another file. I have a file with a line of IP addresses which is all one line. This script that I have written asks for user input and one of those things it asks for is an IP address. That gets stored as a variable inside_ip I want to write that to the end of a line in another file. I found a similar question and the solution was
sed -i.bck '$s/$/yourText2/' list.txt
I tried to put that in a file with
sed -i.bck '$s/$/ $inside_ip/' list.txt
but it actually writes $inside_ip to the end of the file, so I just need it to print the variable.
Does the following work for you?
echo $inside_ip >> list.txt
use "'s instead of ''s as in sed -i.bck "s/$/ $inside_ip/" list.txt
Single quotes stop variables from being expanded. Double quotes allow them to be expanded. Hence:
sed -i.bck '$s/$/ '"$inside_ip/" list.txt
That protects the $s in single quotes; you want sed to see the $ and the s, not the value of your (probably unset) shell variable $s. Of course, if the file only contains one line, then the leading $ is not critical; you could leave it out, or replace it with 1. The /$/ would be left alone anyway, but the double quotes following expand the variable, preserving any spaces inside it (though IP addresses don't usually contain spaces).
So I have testfile which contains
Line one
Another line
and this is the third line
My script reads this file, does some stuff and I end up with a variable that should contain it. Kind of doing
filevar=$(cat testfile)
(the important thing here is that I cannot access the file directly).
I'll be using the contents of that variable to generate an HTML code and one things I have to do is to add <br> to the end of each line. The problem is, there doesnt seem to any EOLs in my var:
echo $filevar
Line one Another line and this is the third line
How do I read the file properly to keep the EOLs? Once I have that I can simply sed s/$/<br>/g, but till then...
thanks!
how about changing IFS?
#!/bin/bash
IFS=""
filevar=$(cat test)
echo $filevar
this will output:
Line one
Another line
and this is the third line
I can't understand why do you need to read the file into the variable. Why don't you simply do this:
sed 's|$|<br/>|' testfile
UPDATE:
If you really want to get the EOL back in your variable. Try this (notice the quotes):
echo "$filevar"
But I still can't understand, why you can cat the file but not access the file
As a solution, I would suggest the following script:
while read LINE
do
echo ${LINE} '<br />' # Implement your core logic here.
done < testfile
Instead of doing echo $filevar do echo "$filevar" (note the double quotes). This will send a single argument to echo and then you can pipe this to sed.
With sed, this will be treated as 3 lines, so you do not need the g option. This works with me (bash and cygwin):
echo "$filevar" | sed 's/$/<br>/'
You need to set the IFS variable to contain just a newline, and then reference the filevar variable without quotes.
$ filevar='Line one
Another line
and this is the third line'
$ for word in $filevar; do echo "$word<br>"; done
Line<br>
one<br>
Another<br>
line<br>
and<br>
this<br>
is<br>
the<br>
third<br>
line<br>
$ for word in "$filevar"; do echo "$word<br>"; done
Line one
Another line
and this is the third line<br>
$ (IFS=$'\n'; for word in $filevar; do echo "$word<br>"; done)
Line one<br>
Another line<br>
and this is the third line<br>