Bash script to copy from one line and replace another line with the copy - linux

I am looking to write a bash script for something slightly more complicated than the usual find/replace via sed. I have a book called bash Cookbook that I have been trying to glean some inspiration from but I am not getting very far.
Basically I am trying to write a script to update the version numbers in a bunch of maven pom.xml files automatically. Here is the general setup I am looking at:
<!-- TEMPLATE:BEGIN
<version>##VERSION##</version>
-->
<version>1.0.0</version>
<!-- TEMPLATE:END -->
After running the script (with the new version number 1.0.1) I'd like the file to read this instead:
<!-- TEMPLATE:BEGIN
<version>##VERSION##</version>
-->
<version>1.0.1</version>
<!-- TEMPLATE:END -->
So this would be in the actual release pom file, with 1.0.0 being the current version (and I am trying to replace it with 1.0.1 or something). Obviously the version number will be changing so there isn't a good way to do a find/replace (since the thing you want to find is variable). I am hoping to be able to write a bash script which can
replace ##VERSION## with the actual version number
delete the current version line
write the updated version line on the line before the TEMPLATE:END (while preserving the ##VERSION## in the file - possibly do this by writing template out to a temp file, doing replacement, then back in?)
I can sort of do some of this (writing out to a new file, doing replacement) using an ant script a la
<replace file="pom.xml">
<replacefilter
token="##VERSION##"
value="${version}"/>
</replace>
But I am not sure what the best ways to a.) delete the line with the old version or b.) tell it to copy the new line in the correct place are. Anyone know how to do this or have any advice?

Assuming the new version number is in a shell variable $VERSION, then you should be able to use:
sed -e '/<!-- TEMPLATE:BEGIN/,/<!-- TEMPLATE:END -->/{
s/<version>[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*<\/version>/<version>'$VERSION'<\/version>/
}'
Note that this ignores the template version line with ##VERSION##, but only matches a three-part version number that appears between the lines containing TEMPLATE:BEGIN and TEMPLATE:END, leaving everything else (including other lines containing a <version>...</version> element) alone.
You can decide how to do file overwriting (maybe your version of sed is from GNU and it does that automatically on request with the -i option), etc. You might also be able to use more powerful regular expression notations that lead to more compact matches. However, that should work on most versions of sed without change.

The steps you outlined (1-3) read as if you do not actually care to perform the replacement in accordance to the templated rules defined within the comments.
As such, here is some code that behaves verbosely as you outlined:
#!/bin/bash
file=$1
newversion=$2
sed -i $file -e "s|<version>\([^#]*\)</version>|<version>$newversion</version>|"
Run it:
chmod +x yourscript.sh
./yourscript.sh filetoupdate.xml 1.0.1

use 5.010;
use strictures;
use Perl::Version qw();
use XML::LibXML qw();
my $dom = XML::LibXML->load_xml(location => 'pox.xml');
for my $node ($dom->findnodes('//version')) {
my $version = Perl::Version->new($node->textContent);
$version->inc_subversion;
$version->stringify;
$node->removeChildNodes;
$node->appendText($version);
};
say $dom->toString;

Related

How to comment all the uncommented lines in a file using puppet module

I have a sshd_config configuration file which contains commented as well as uncommented lines. I want to comment all the uncommented lines in that file using puppet. Is there any optimal/simple way to do this? Or is there a way to run bash command (maybe sed to replace) via puppet? I am not sure that using bash command is a right approach.
It would be really helpful is someone guides me with this. Thanks in advance!
Is there any optimal/simple way to do this?
There is no built-in resource type or well-known module that specifically ensures that non-blank lines of a file start with a # character.
Or is there a way to run bash command (maybe sed to replace) via puppet?
Yes, the Exec resource type. That's your best bet short of writing a custom resource type.
I am not sure that using bash command is a right approach.
In a general sense, it's not. Appropriate, specific resource types are better than Exec. But when you don't have a suitable one and can't be bothered to make one, Exec is available.
It might look like this:
# The file to work with, so that we don't have to repeat ourselves
$target_file = '/etc/ssh/sshd_config'
exec { "Comment uncommented ${target_file} lines":
# Specifying the command in array form avoids complicated quoting or any
# risk of Puppet word-splitting the command incorrectly
command => ['sed', '-i', '-e', '/^[[:space:]]*[^#]/ s/^/# /', $target_file],
# If we didn't specify a search path then we would need to use fully-qualified
# command names in 'command' above and 'onlyif' below
path => ['/bin', '/usr/bin', '/sbin', '/usr/sbin'],
# The file needs to be modified only if it contains any non-blank, uncommented
# lines. Testing that via an 'onlyif' ensures that Puppet will not
# run 'sed' or (more importantly) report the file changed when it does
# not initially contain any lines that need to be commented
onlyif => [['grep', '-q', '^[[:space:]]*[^#]', $target_file]],
# This is the default provider for any target node where the rest of this
# resource would work anyway. Specifying it explicitly will lead to a more
# informative diagnostic if there is an attempt to apply this resource to
# a system to which it is unsuited.
provider => 'posix',
}
That does not rely on bash or any other shell to run the commands, but it does rely on sed and grep being available in one of the specified directories. In fact, it relies specifically on GNU sed or one that supports an -i option with the same semantics. Notably, that does not include BSD-style sed, such as you will find on macOS.

How to replace a matching pattern inside a file to another using bash

My script has many lines starting with slo. How can I replace all the strings that are starting with slo to fwd using bash commands? Any help would be appreciated.
Here is a snippet of my script
template_version: 2018-03-02
resources:
instance01:
type: ../../../templates/nf.yaml
properties:
vm_name: 'slol2lvdl1'
vm_flavour: 'dns_19te'
image_name: 'pdns_dnsd_slo_211214121207'
vm_az: 'az-1'
vm_disk_root_size: '50'
vm_disk_data_size: '50'
network_mgmt_refs: 'int:dns_ox_slo_507:c3dns_slo_live_nc_vnns_pcg'
My requirement is to replace all slo to fwd in the above code. I have 5 files like this in the same directory.
sed is the go-to for file content replacements with regular expressions. If every slo you want to replace is between _ characters it's fairly easy with a command like this (in GNU sed which ships with just about all linuxes):
sed -i -e 's/_slo_/_fwd_/g' files to replace
-i replaces the text inline, replacing existing file contents with updated contents.
If not all slo are within _ characters you need to worry about unintentional matches.
Be sure to make a backup of these files or if they're in a git repo work from a clean state in case you don't like the change. Using git to track the changes might make sense even if you don't currently have the files in a git repo as this will make it trivial to compare before and after.
sed -i 's/slo/fwd/' worked! Also found many alternatives but sed was straight forward!

Script shell for renaming and rearranging files

I would like to rearrange and rename files.
I have this tree structure of files :
ada/rda/0.05/alpha1_freeSurface.md
ada/rda/0.05/p_freeSurface.md
ada/rda/0.05/U_freeSurface.md
ada/rda/0.1/alpha1_freeSurface.md
ada/rda/0.1/p_freeSurface.md
ada/rda/0.1/U_freeSurface.md
I want that files will be renamed and rearranged like this structure below:
ada/rda/ada-0.05-alpha1.md
ada/rda/ada-0.05-p.md
ada/rda/ada-0.05-U.md
ada/rda/ada-0.1-alpha1.md
ada/rda/ada-0.1-p.md
ada/rda/ada-0.1-U.md
Using the perl rename (sometimes called prename) utility:
rename 's|ada/rda/([^/]*)/([^_]*).*|ada/rda/ada-$1-$2.md|' ada/rda/*/*
(Note: by default, some distributions install a rename command from the util-linux package. This command is incompatible. If you have such a distribution, see if the perl version is available under the name prename.)
How it works
rename takes a perl commands as an argument. Here the argument consists of a single substitute command. The new name for the file is found from applying the substitute command to the old name. This allows us not only to give the file a new name but also a new directory as above.
In more detail, the substitute command looks like s|old|new|. In our case, old is ada/rda/([^/]*)/([^_]*).*. This captures the number in group 1 and the beginning of the filename (the part before the first _) in group 2. The new part is ada/rda/ada-$1-$2.md. This creates the new file name using the two captured groups.
You can use basename and dirname functions to reconstruct the new filename:
get_new_name()
{
oldname=$1
prefix=$(basename $oldname _freeSurface.md)
dname=$(dirname $oldname)
basedir=$(dirname $dname)
dname=$(basename $dname)
echo "$basedir/ada-$dname-$prefix.md"
}
e.g. get_new_name("ada/rda/0.05/alpha1_freeSurface.md") will show ada/rda/ada-0.05-alpha1.md in console.
Then, you can loop through all your files and use mv command to rename the files.

How to add block of code within a file?

I have this setup in some setting file:
<datasource jndi-name="java:jboss/datasources/DS" pool-name="mysqlDS">
<connection-url>jdbc:mysql://localhost:3306/DSDB</connection-url>
<driver>mysqlDriver</driver>
<pool>
<min-pool-size>10</min-pool-size>
<max-pool-size>200</max-pool-size>
<prefill>true</prefill>
<use-strict-min>false</use-strict-min>
<flush-strategy>FailingConnectionOnly</flush-strategy>
</pool>
<security>
<security-domain>csa-encryption-sec</security-domain>
</security>
#block of code needs to be inserted HERE
</datasource>
And this is the block of code that needs to be inserted:
<validation>
<check-valid-connection-sql>SELECT 1</check-valid-connection-sql>
<background-validation>true</background-validation>
<valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"></valid-connection-checker>
<exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"></exception-sorter>
</validation>
I only had used sed with simples things, but I don't know how to use it properly, because I don't want to replace any text, just add the block of code inside of the other one. Any idea or suggestions ?
With sed, you can do it with the r command as follow:
If you have the block of code in a file (named validation):
sed "14r validation" standalone.xml
If you have it in a variable ($validation):
sed "14r"<(cat <<<"$validation") file
In the first method, sed just inserts the file validation.
In the second one, we have to make a file from the data in $validation.
A way to accomplish this is using bash's Here Strings (I noticed you are using bash in your self answer) and Process Substitution.
With cat <<<"$validation" we're supplying the expansion of $validation to cat's standard input.
This is what I have developed so far. I'm worried if it is an intelligent approach.
Because to me, it doesn't look elegant :(
#!/bin/bash
validation='\<validation\> \
\<check-valid-connection-sql\>SELECT 1\<\/check-valid-connection-sql\> \
\<background-validation\>true\<\/background-validation\> \
\<valid-connection-checker class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLValidConnectionChecker"\>\<\/valid-connection-checker\> \
\<exception-sorter class-name="org.jboss.jca.adapters.jdbc.extensions.mysql.MySQLExceptionSorter"\>\<\/exception-sorter\> \
\<\/validation\>'
sed -i "15i $validation" standalone.xml
how often do you need to do this? (i.e., is that validation something that is enabled/disabled constantly?).
How about git? Just use a source-control system, then you can have a version of the file with the extra block and another one without the extra block.
Just an idea.

Trying to 'grep' links from downloaded html pages in bash shell environment without cut, sed, tr commands (only e/grep)

In Linux shell, I am trying to return links to JPG files from the downloaded HTML script file. So far I only got to this point:
grep 'http://[:print:]*.jpg' 'www_page.html'
I don't want to use auxiliary commands like 'tr', 'cut', 'sed' etc...'lynx' is okay!
Using grep alone without massaging the file is doable but not recommended as many have pointed out in the comments.
If you can loosen up your requirements a bit then you can use html tidy to massage the downloaded HTML file so that each html entities are on a single line so that the regular expression can be simpler like you wanted, something like this:
$ tidy file.html|grep -o 'http://[[:print:]]*.jpg'
Note the use of "-o" option to grep to print only the matching part of the input

Resources