I have one file which contains key value pair
properties.env
development_port=8080
development_type=tcp
staging_port=8081
development_type=http
I need to read value from properties.env and replace it in another file based on some environment variable value lets say ENVIRONMENT=staging/development
config.yml
server:
adminConnectors:
-
port: 18001
type: http
The final output of config.yml should be if ENVIRONMENT=staging
server:
adminConnectors:
-
port: 8081
type: http
How can i achieve this using sed command?
Since the properties file looks like it has the right syntax you could just source it, and then replace the entire port line:
source properties.env
sed --in-place "s/port.*/port: $staging_port/" config.yml
This, of course, assumes the "port" line appears only once in your config.
Related
I'm new to shell script.
I have a YAML file that consists of placeholder . I need to replace all the occurrences of placeholder <test-name> with a value sweta
script.sh
Name="sweta"
Below is the example of YAML file
metadata:
name: <test-name>-svc
namespace: abc
spec:
selector:
app: <test-name>
ports:
- protocol: TCP
name: http
port: 80
targetPort: 8080
My expected output is:
metadata:
name: sweta-svc
namespace: abc
spec:
selector:
app: sweta
ports:
- protocol: TCP
name: http
port: 80
targetPort: 8080
Can someone help me?
Appreciate all your help. Thanks in advance!
There are two easy ways to accomplish your search and replace <test-name> string with sweta
Note: Assuming your yaml file name is config.yaml
Option-1: Using stream editor command 'sed' as below:
To overwrite the original file
sed -i 's/<test-name>/sweta/g' config.yaml
To write the output to a different file
sed 's/<test-name>/sweta/g' config.yaml > updated_config.yaml
The syntax of sed command is sed -i 's/original/new/g' file.txt
where
sed is the command name
-i stands for in-place edit. This option is used to overwrite the original file. If you don't use this, you have to redirect output to a different file as done in second command above.
s stands for substitute
original is the search string
new is the replace string
g is for global (replace all occurrences). Omitting this will just replace only the first occurence
file.txt is the text file (Linux doesn't use extensions like .txt but people often name files with such extensions as a convention to denote file type)
Option-2: Use 'awk' command:
awk is a very powerful text processing command with its own language syntax, but this search and replace is quite easy:
awk '{gsub("<test-name>", "sweta"); print $0}' config.yaml > updated_config.yaml
I am trying to comment a line within a .yml file through the command line, I am using this command sudo sed -i '/<pattern_to_find>/s/^#//g' /etc/metricbeat/metricbeat.yml an it works fine as long as the line is not idented, but I am loking to change an idented line such as:
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
# host: "localhost:5601"
# Kibana Space ID
any ideas how to achieve this? I cannot figure it out
Although I strongly encourage you not use the -i option to sed, I will not discuss that beyond mentioning it here. You just need to change your search pattern. Try:
sed -E '/^( *)#( host: "localhost:560!")/s//\1 \2/'
This will fail to match hard tabs. Depending on how much portability you want between different versions of said, that can be a pain to deal with. But the following should work:
sed -E '/^([[:space:]]*)#( host: "localhost:560!")/s//\1 \2/'
I want to uncomment a line in file sshd_config by using Ansible and I have the following working configuration:
- name: Uncomment line from /etc/ssh/sshd_config
lineinfile:
dest: /etc/ssh/sshd_config
regexp: '^#AuthorizedKeysFile'
line: 'AuthorizedKeysFile .ssh/authorized_keys'
However this config only works if the line starts by #AuthorizedKeysFile, but it won't work if the line starts by # AuthorizedKeysFile or # AuthorizedKeysFile (spaces between # and the words).
How can I configure the regexp so it won't take into account any number of spaces after '#'?
I've tried to add another lineinfile option with a space after '#', but this is not a good solution:
- name: Uncomment line from /etc/ssh/sshd_config
lineinfile:
dest: /etc/ssh/sshd_config
regexp: '# AuthorizedKeysFile'
line: 'AuthorizedKeysFile .ssh/authorized_keys'
If you need zero or more white spaces after the '#' character, the following should suffice:
- name: Uncomment line from /etc/ssh/sshd_config
lineinfile:
dest: /etc/ssh/sshd_config
regexp: '^#\s*AuthorizedKeysFile.*$'
line: 'AuthorizedKeysFile .ssh/authorized_keys'
The modification to your original code is the addition of the \s* and the .*$ in the regex.
Explanation:
\s - matches whitespace (spaces, tabs, line breaks and form feeds)
* - specifies that the expression to it's left (\s) can have zero or more instances in a match
.* - matches zero or more of any character
$ - matches the end of the line
Firstly, you are using the wrong language. With Ansible, you don't tell it what to do, but define the desired state. So it shouldn't be Uncomment line form /etc/ssh/sshd_config, but Ensure AuthorizedKeysFile is set to .ssh/authorized_keys.
Secondly, it doesn't matter what the initial state is (if the line is commented, or not). You must specify a single, unique string that identifies the line.
With sshd_config this is possible as the AuthorizedKeysFile directive occurs only once in the file. With other configuration files this might be more difficult.
- name: Ensure AuthorizedKeysFile is set to .ssh/authorized_keys
lineinfile:
dest: /etc/ssh/sshd_config
regexp: AuthorizedKeysFile
line: 'AuthorizedKeysFile .ssh/authorized_keys'
It will match any line containing AuthorizedKeysFile string (no matter if it's commented or not, or how many spaces are there) and ensure the full line is:
AuthorizedKeysFile .ssh/authorized_keys
If the line were different, Ansible will report "changed" state.
On the second run, Ansible will find the AuthorizedKeysFile again and discover the line is already in the desired state, so it will end the task with "ok" state.
One caveat with the above task is that if any of the lines contains a comment such as a real, intentional comment (for example an explanation in English containing the string AuthorizedKeysFile), Ansible will replace that line with the value specified in line.
I should caveat this with #techraf's point that 99% of the time a full template of a configuration file is almost always better.
Times I have done lineinfile include weird and wonderful configuration files that are managed by some other process, or laziness for config I don't fully understand yet and may vary by distro/version and I don't want to maintain all the variants... yet.
Go forth and learn more Ansible... it is great because you can keep iterating on it from raw bash shell commands right up to best practice.
lineinfile module
Still good to see how best to configuration manage one or two settings just a little better with this:
tasks:
- name: Apply sshd_config settings
lineinfile:
path: /etc/ssh/sshd_config
# might be commented out, whitespace between key and value
regexp: '^#?\s*{{ item.key }}\s'
line: "{{ item.key }} {{ item.value }}"
validate: '/usr/sbin/sshd -T -f %s'
with_items:
- key: MaxSessions
value: 30
- key: AuthorizedKeysFile
value: .ssh/authorized_keys
notify: restart sshd
handlers:
- name: restart sshd
service:
name: sshd
state: restarted
validate don't make the change if the change is invalid
notify/handlers the correct way to restart once only at the end
with_items (soon to become loop) if you have multiple settings
^#? the setting might be commented out - see the other answer
\s*{{ item.key }}\s will not match other settings (i.e. SettingA cannot match NotSettingA or SettingAThisIsNot)
Still might clobber a comment like # AuthorizedKeysFile - is a setting which we have to live with because there could be a setting like AuthorizedKeysFile /some/path # is a setting... re-read the caveat.
template module
- name: Configure sshd
template:
src: sshd_config.j2
dest: /etc/ssh/sshd_config
owner: root
group: root
mode: "0644"
validate: '/usr/sbin/sshd -T -f %s'
notify: restart sshd
handlers:
- name: restart sshd
service:
name: sshd
state: restarted
multiple distro support
And if you are not being lazy about supporting all your distros see this tip
- name: configure ssh
template: src={{ item }} dest={{ SSH_CONFIG }} backup=yes
with_first_found:
- "{{ ansible_distribution }}-{{ ansible_distribution_major_version }}.sshd_config.j2"
- "{{ ansible_distribution }}.sshd_config.j2"
https://ansible-tips-and-tricks.readthedocs.io/en/latest/modifying-files/modifying-files/
(needs to be updated to a loop using the first_found lookup)
Is it possible to achieve the same goal with replace module.
https://docs.ansible.com/ansible/latest/modules/replace_module.html
- name: Uncomment line from /etc/ssh/sshd_config
replace:
path: /etc/ssh/sshd_config
regexp: '^\s*#+AuthorizedKeysFile.*$'
replace: 'AuthorizedKeysFile .ssh/authorized_keys'
If you want to simply uncomment a line without setting the value, you can use replace with backreferences, eg (with a handy loop):
- name: Enable sshd AuthorizedKeysFile
replace:
path: /etc/ssh/sshd_config
# Remove comment and first space from matching lines
regexp: '^#\s?(\s*){{ item }}(.+)$'
replace: '\1{{ item }}\2'
loop:
- 'AuthorizedKeysFile'
This will only remove the first space after the #, and so retain any original indenting. It will also retain anything after the key (eg the default setting, and any following comments)
Thanks to the other helpful answers that provided a solid starting point.
I tried to append a value to the ssh config file but it seems that it doesn't work if specify two different values, it reads the first not the second.
It seems sensible, instead of trying to re-write the entire file, to just modify the value.
Currently the value is:
Port 22
I want to go in and change the value to:
Port 222
The best I can come up with is this:
sed -c -i "s/\(Port *= *\).*/\1$REPLACEMENT_VALUE/" /etc/ssh/sshd_config
However I know this isn't right because it's working in a way where it is expecting to have an equals sign between the value and the variable.
I need the script to do something like this:
Look for the variable (i.e Port)
Switch this line with my own line
I think you mean this,
sed 's/.*\bPort\b.*/REPLACEMENT_VALUE/' file
Example:
$ REPLACEMENT_VALUE="Port 222"
$ echo -e "foo\nPort 22" | sed "s/.*\bPort\b.*/$REPLACEMENT_VALUE/"
foo
Port 222
In the lineinfile module, it replaces the full line.
If the line is long I have to repeat the whole line again.
Let us suppose I want to replace the single word in the file:
#abc.conf
This is my horse
this is the playbook:
- lineinfile: dest=abc.conf
state=present
regexp='horse'
line='This is my dog'
backup=yes
is there any way to achieve someting like sed 's/horse/dog/g' ?
New module replace available since 1.6 version:
- replace:
dest=abc.conf
regexp='horse'
replace='dog'
backup=yes
You can use backreferences to retrieve other parts(that should not be changed) of the line:
- lineinfile: dest=abc.conf
state=present
regexp='^(.*)horse(.*)$'
line='\1dog\2'
backup=yes
backrefs=yes
If you need to do more replace operations in one block and you have the file locally, you might want to consider using template, which substitutes variables in the template file and copies the file to the remote:
- template: src=/mytemplates/foo.j2 dest=/etc/file.conf
In the local file you can write a variable with ansible sintax like
{{variable}}
and it will be substituted if it is in the scope of the script. Here the docs.