I'm quite new to bash scripting. I have a script where I want to extract part of the value of a particular line in a separate config file and then use that value as a variable in the script.
For example:
Line 75 in a file named config.cfg
"ssl_cert_location=/etc/ssl/certs/thecert.cer"
I want just the value at the end of "thecert.cer" to then use in the script. I've tried awk and various uses of grep but I can't quite get just the name of the certificate.
Any help would be appreciated. Thanks
These are some examples of the commands I ran:
awk -F "/" '{print $4}' config.cfg
grep -o *.cer config.cfg
Is this possible to extract the value on that line and then edit the output so it just contains the name of the certificate file?
This is a pure Bash version of the basic functionality of basename:
cert=${line##*/}
which removes everything up to and including the final slash. It presupposes that you've already read the line.
Or, using sed:
cert=$(sed -n '75s/^.*\///p' filename)
or
cert=$(sed -n '/^ssl_cert_location=/s/^.*\///p' filename)
This gets the specified line based on the line number or the setting name and replaces everything up to and including the final slash with nothing. It ignores all other lines in the file (unless the setting is repeated in the case of the text match version). The text match version is better because it works no matter what line number the setting is on.
grep uses regular expressions (as does sed). The grep command in your command appears to have a glob expression which won't work. One way to use grep (GNU grep) is to use the PCRE feature (Perl Compatible Regular Expressions):
cert=$(grep -Po '^ssl_cert_location=.*/\K.*' filename)
This works similarly to the sed command.
I have anchored the regular expressions to the beginning of the line. If there may be leading white spaces (the line may be indented), change the regex so it looks something like this:
^[[:space:]]*ssl_cert_location=
which works for both indented and unindented lines.
There are many variants, but a simple one that comes to mind with grep is first getting the line, then matching only non-slashes at the end of the line:
<config.cfg grep '^ssl_cert_location=' | grep -o '[^/]*$'
Why didn't your grep command (grep -o *.cer config.cfg) work? Becasue *.cer is a shell glob pattern and will be expanded by the shell to matching file names, even before the grep process is even started. If there are no matching files, it will be passed verbatim, but * in regular expressions is a quantifier which needs a preceeding expression. . in regex is "match any single character". So what you wanted is probably grep -o '.*\.cer', but .* matches anything, including slashes.
An awk solution would look like the following:
awk -F/ '/^ssl_cert_location=/{print $NF}' config.cfg
It uses "/" as separator, finds only lines starting with "ssl_cert_location" and then prints the last (NF) field in from this line.
Or an equivalent sed solution, which matches the same line and then deletes everything including the last slash:
sed -n '/^ssl_cert_location=/s#^.*/##p' config.cfg
To store the output of any command in a variable, use command substitution:
var="$(command with arguments)"
I have file which contains text at line 30 which is:
Icon="\<some path which we do not know\icon.png"
I want to replace above with:
Icon="\home\user\Img\Icons\icon.png"
What is the best way to do it.?
Thanks.
Best way:
perl -pi -e 's/\\<some path which we do not know/\\home\\user\\Img\\Icons/' text.txt
Perl approach is more preferred than sed, because of Unix compatibility.
You can either use an editor to do this manually or if you prefer to do it non interactively, you can use a small shell pipeline and sed.
sed `3 s/big path/custom path/` input_file.txt
where 3 is the line number, big path is what you want to replace and custom path is what you to want to replace it with. input_file.txt is your input file. This will print the replaced file onto the screen which you can redirect into another file using the > operator.
As a concrete example, suppose I have this file (input_file.txt)
Header
Random test
/bad/path/to/some/directory/icon.png
/bad/path/to/some/directory/icon.png
Footer
Now I'm going to run my command like so.
cat input_file.txt | sed '4 s/\/bad\/path\/to\/some\/directory\//\/home\/noufal\//'
and I get
Header
Random test
/bad/path/to/some/directory/icon.png
/home/noufal/icon.png
Footer
Notice that it has changed only the 4th line. The extra \ characters in the command are to escape the / character which has special meaning for sed.
you can use vim to find and replace your strings http://vim.wikia.com/wiki/Search_and_replace or use 'sed' command
For example I have:
Hello1 :
Hello2 :
Hello3 :
How Could I delete all of these with a shell script. The number reaches up all the way to 1000.
sed -i '/^Hello[[:digit:]]\+\>/d' file.txt
Or, if you want to output to a different file:
sed '/^Hello[[:digit:]]\+\>/d' file.txt > newfile.txt
If you wish to delete all the lines that contain only Hello(number) : use below :
Sample Input in file
Hell
Hello1 :
No hello stuff here
Unjulating stuff
Hello2 :
Some sentence
Hello99 :
Script
sed -Ei '/^Hello[[:digit:]]+ :$/d' file
Sample Output in the modified file
Hell
No hello stuff here
Unjulating stuff
Some sentence
What happens above
Using the ^ in the pattern we check for the beginning of the line.
We check the pattern Hello(number) : using Hello[[:digit:]]+ :$. Note that I used -E to enable sed extended regular expressions so I need not escape the + ie (\+). Here [[:digit:]] is a class which contains
all the decimal digits and + is used to check if the pattern before it matches at least one time.
Check the end of the line using $
For a matched pattern, delete it line using the d option
I have also used the sed inplace edit option -i so that the changes
are directly saved to the file.
If you wish to change the a line the begins with Hello(number) : then use the below script
sed -Ei '/^Hello[[:digit:]]+ :/d' file
You might have notices that I just removed the $, so our pattern matches any line that starts with Hello(number) :
Hope this helps.
I need to read a file line by line in Linux, find a substring in each line, remove it and place it at the end of that line.
Example:
Line in the original file:
a,b,c,substring,d,e,f
Line in the output file:
a,b,c,d,e,f,substring
How do I do it with the Linux command? Thanks!
sed '/substring/{ s///; s/$/substring/;} '
will handle a fixed substring. Note that if substring begins with a ,, this handles your example case well. If the substring is not fixed but may be a general regular expression:
sed 's/\(substring\)\(.*\)/\2\1'
If you are looking for general csv parsing, you should rephrase the question. (It will be difficult to apply this solution to find a fixed string at the start of a line if you are thinking of the input as comma separated fields.)
I always prefer to use perl's command line to do such regex tasks - perl is powerful enough to cover awk and sed in most of my usages, and both available in windows and linux, it is just easy and handy to me, so the solution in perl would be like:
perl -ne "s/^(.*?)(?:(?<comma>,)(?<substr>substring)|(?<substr>substring)(?<comma>,))(?<right>.*)$/$1$+{right}$+{comma}$+{substr}/; print" input.txt > output.txt
or a simpler one:
perl -lpe "if(s/(,substring|substring,)//){ s/$/,substring/ }" input.txt > output.txt
input.txt
substring,a,b,c,d,e,f
a,b,c,substring,d,e,f
a,b,c,d,e,f,substring
substring,a
a,substring
substring
a
output.txt
a,b,c,d,e,f,substring
a,b,c,d,e,f,substring
a,b,c,d,e,f,substring
a,substring
a,substring
substring
a
You can edit based on your actual input:
If there are any space between words and commas
If you are using tab as separator
Some explanation of the command line:
use perl's -n -e options: -n means process the input line by line in a loop; -e means one line program in the command line
use perl's -l -p options: -l means process multilines; -p means always print
The one line program is just a regex replacement and a print
(?:pattern) means group but don't capture the match
(?<comma>) is a named group, you then need to use $+{comma} hash to access it
I am trying to append a variable at the last character of a specific line of a file from a bash script.
The file is called myfile.txt and what I want to do is to append the contents of a variable named VERSION right after the last character of the line of the file that contains the unique string MYVERSION.
That is, if in this line there is the following:
MYVERSION=0.1
and VERSION="-custom_P1" then, I want to have the following:
MYVERSION=0.1-custom_P1
Thank you all for the help.
Try this:
sed -i "/^MYVERSION=/ s/\$/$VERSION/" myfile.txt
The idea is that it finds a line that starts with MYVERSION= and then replaces the end of that line with the contents of the $VERSION environment variable.
Edit: originally I wasn't sure if the first $ needed to be escaped, but #sehe's comment and its upvoters convinced me that it does.
Try
sed -e "s/^MYVERSION=/MYVERSION=.*$/&${VERSION}/g" < myfile.txt
The command appends the value of VERSION to the line with 'MYVERSION='