Dynamically assign master/slave variables in Ansible role - linux

I have a simple mariadb role which permits to setup master/slave replication on two servers. In order to do this, I have to define in my inventory my 2 nodes like this:
node1 master=true
node2 slave=true
This way, I can setup one role to setup master/slave replication using Ansible when statement playing with this vars.
- name: Setup master conf
template: >-
src="templates/master.conf.j2"
dest="{{ master_config_file }}"
when:
- master is defined
Now, I would like to get something more automatic that could dynamically and randomly assign a master variable to one node, and slave variable to all other nodes.
I have seen some Ansible doc about variables and filters, but none of them seems to be adapted to that. I guess that I have to develop my own Ansible variable plugin to do that.

You can utilise facts.d. Something like this:
- hosts: all
become: yes
tasks:
- file:
path: /etc/ansible/facts.d
state: directory
- shell: echo '{{ my_facts | to_json }}' > /etc/ansible/facts.d/role.fact
args:
creates: /etc/ansible/facts.d/role.fact
vars:
my_facts:
is_master: "{{ true if play_hosts.index(inventory_hostname) == 0 else false }}"
register: role_fact
# refresh facts if fact has been just added
- setup:
when: role_fact | changed
- set_fact:
is_master: "{{ ansible_local.role.is_master }}"
- debug:
var: is_master
This will create role.fact on remote nodes if it is not there and use is_master fact from it. During subsequent runs ansible_local.role.is_master is fetched automatically.

You can use a dynamic group to do that. Another use case : You don't know which node is the master because it is elected, and you need to performs actions only on master.
To use a dynamic group, you need to define two pays in your playbook :
First one in order to determine which node is the master and add it in a dynamic group, you need to use a command
Then execute tasks on master, slaves
Following playbook determine which nodes are masters and slaves and execute a play on each types :
- hosts: all
tasks:
- shell: <command on node to retrieve node type>
register: result__node_type
- name: If node is a master, add it in masters group
add_host:
name: "{{ inventory_hostname }}"
groups: temp_master
when: result__node_type.stdout == "MASTER"
- name: If node is a slave, add it in slaves group
add_host:
name: "{{ inventory_hostname }}"
groups: temp_slave
when: result__node_type.stdout == "SLAVE"
- name: No master found, then assign first one (or random if you want) to masters group
add_host:
name: "groups['all'][0]"
groups: temp_master
run_once: yes
when: groups['temp_master'] | length == 0
- name: No slave found, then assign others to slaves group
add_host:
name: "groups['all'][0]"
groups: temp_slave
run_once: yes
with_items: "{{ groups['all'][1:] }}"
when: groups['temp_slave'] | length == 0
- hosts: temp_master
gather_facts: false
tasks:
- debug:
msg: "Action on master {{ ansible_host }}"
- hosts: temp_slave
gather_facts: false
tasks:
- debug:
msg: "Action on slave {{ ansible_host }}"

Related

How to pass value to variable in list, to pick one by one using command "ansible-playbook test.yml --extra-var variable1=ntpd,crond,sysconf"

I tried to pass value manually to a specific variable by using command
ansible-playbook test.yml --extra-var variable1=ntpd
Can you please help me to know how to pass value to variable in list, to pick one by one using command?
I've tried as below did not work
ansible-playbook test.yml --extra-var "variable1=ntpd,crond,sysconf"
Tried as but no luck
ansible-playbook test.yml --extra-var "variable1=ntpd,crond,sysconf"
ansible-playbook test.yml -e '{"variable1":["ntpd", "crond"]}'
The playbook should pick 1st value as ntpd and then second value as crond and so on.
You may have a look into the following minimal example playbook
---
- hosts: localhost
become: false
gather_facts: false
tasks:
- name: Show provided content and type
debug:
msg:
- "var1 contains {{ var1 }}"
- "var1 is of tpye {{ var1 | type_debug }}"
which will called with
ansible-playbook extra.yml --extra-vars="var1=a,b,c"
resulting into an output of
TASK [Show provided content and type] ******
ok: [localhost] =>
msg:
- var1 contains a,b,c
- var1 is of tpye unicode
I understand that you like to get a list out of it for further processing. Since there is a string of comma separated values this is quite simple to achieve, just split the string on the separator.
- name: Create list and loop over elements
debug:
msg: "{{ item }}"
loop: "{{ var1 | split(',') }}"
will result into an output of
TASK [Create list and loop over elements] ******
ok: [localhost] => (item=a) =>
msg: a
ok: [localhost] => (item=b) =>
msg: b
ok: [localhost] => (item=c) =>
msg: c
As the example show packages names, the use cause might be to install or validate packages and which are provided in a list of comma separated values (pkgcsv).
In example for the yum module – Manages packages with the yum package manager and because of the Notes
When used with a loop: each package will be processed individually, it is much more efficient to pass the list directly to the name option.
one should better proceed further with
- name: Install a list of packages with a list variable
ansible.builtin.yum:
name: "{{ pkgcsv | split(',') }}"
or, as the module has the capabilities to do so, one could also leave the CSV just as it is
- name: Install a list of packages with a list variable
ansible.builtin.yum:
name: "{{ pkgcsv }}"
Further Reading
As this could be helpful for other or in future cases.
Is there any way to validate whether a file is passed as extra vars in ansible-playbook?

Ansible best practice for breaking a string into a list of max length substrings?

This shows what I am trying to do
---
- hosts: localhost
gather_facts: false
vars:
str: "abcdefg"
str_parts: []
tasks:
- name: Break string into list of max 3 character parts
set_fact:
str_parts: "{{ str_parts + [ str[:3] ] }}"
str: "{{ str[3:] }}"
until: str == ""
Running it with -vvv shows the loop code is executed only once. str_parts gains single member "abc", str is changed to "defg" and the log shows "FAILED - RETRYING: Break string into list of max 3 character parts" messages until it times out
Why doesn't the loop, er, loop?
I could solve it by using a command or shell module to insert commas at the break points and then using {{ str | split(",") }} but a pure Ansible solution would be nicer
Edit: the behavior the subject of bug set_fact won't update a fact while looping (breaking change)
For example
- set_fact:
str_parts: "{{ str|batch(3)|map('join') }}"
gives
str_parts:
- abc
- def
- g
It is possible to select matching items only, e.g.
- set_fact:
str_parts: "{{ str|batch(3)|map('join')|select('match', '^.{3}$') }}"
gives
str_parts:
- abc
- def

Ansible how to find out dictionary out of list of dictionaries

$ more defaults/mail.yaml
---
envs:
- dev:
acr-names:
- intake.azurecr.io
- dit.azurecr.io
- dev.azurecr.io
subscription-id: xxx
- uat:
acr-names:
- stagreg.azurecr.io
subscription-id: yyy
- prod:
acr-names:
- prodreg.azurecr.io
subscription-id: zzz
I want to write a ansible play to copy the image between registries in azure https://learn.microsoft.com/en-us/azure/container-registry/container-registry-import-images#import-from-a-registry-in-a-different-subscription
The play should accept 2 parameters. source_image and target_image, so the play will import the image from source to destination
For Ex:
ansible-playbook sync-docker-image.yml -e source_image=dit.azurecr.io/repo1:v1.0.0.0 -e target_image=stagreg.azurecr.io/stage-repo:latest
2 questions:
Here how can I find out the which env(dev,uat or prod) the source_image or target_image belongs to in ansible playbook, based on env, I want to choose the subscription-id. So from the above example, I want to create 2 variables called source_subscription and target_subscription and assign them to dev, uat subscriptions respectively.
In YAML, is it possible to access a variable in list of dictionaries based on key, for example something like envs[dev]?
Thanks
First - if possible - when you only have the three stages, don't use a list of dict items in envs. I asume they are already named, so use:
envs:
dev:
acr-names:
- ...
subscription-id: xxx
uat:
acr-names:
- ...
subscription-id: yyy
prod:
acr-names:
- ...
subscription-id: zzz
This would make it easier to access the stages via envs.dev or envs.uat etc. So you need to iterate only over envs.dev.acr-names (maybe use _ instead of -, otherwise you'll get in trouble later). Inside the iteration you can use the when condition to check the item against your source:
- name: "Facts"
set_fact:
envs:
dev:
acr_names:
- intake.azurecr.io
- dit.azurecr.io
- dev.azurecr.io
subscription_id: xxx
uat:
acr_names:
- stagreg.azurecr.io
subscription_id: yyy
prod:
acr_names:
- prodreg.azurecr.io
subscription_id: zzz
source_image: "dit.azurecr.io/repo1:v1.0.0.0"
target_image: "stagreg.azurecr.io/stage-repo:latest"
- name: "Identify source subscription"
set_fact:
source_subscription: "{{ envs.dev.subscription_id }}"
when:
- "item in source_image"
- "source_subscription is undefined"
loop: "{{ envs.dev.acr_names }}"
If it isn't possible to change the dict (because you have "many"), you need to iterate over the items in envs. If possible, do not create "random" keys but use "name"d item. So a structure like this would be better
envs:
- name: dev
acr_names:
- ...
subscription_id: xxx
- name: uat
acr_names:
- ...
subscription_id: yyy
...
So you iterate over the items in envs and then iterate over item.acr_names to find your system. This is more complicated, because you loop over a list and iterate then over items in that list. I think, this isn't possible with one single task. But with the given structure the problem is - the string in source_target is not exactly what is in acr_names. So remove anything after the slash and then you can use a different method to search for a string in a list.
- name: "Identify source subscription"
set_fact:
source_subscription: "{{ env.subscription_id }}"
when:
- "source_image.split('/')[0] in env.acr_names"
- "source_subscription is undefined"
loop: "{{ envs }}"
loop_control:
loop_var: env
You could also use the split filter in the first example without looping over envs.dev etc.
- name: "Show result"
set_fact:
source_subscription: "{{ envs.dev.subscription_id }}"
when:
- "source_image.split('/')[0] in envs.dev.acr_names"
If you really need to use your given structure, then you need to iterate over the envs. It countains a dictionary with a random key as root element. That makes it very complicated. In that case you need to loop over it, include a separate tasks file with include_tasks and inside that tasks list, you need the filter lookup('dict',env) to get a special dict and you can access item.keyanditem.value.acr_namesanditem.value.subscription_id` to access the values inside the dict. I wouldn't recommend that.
- name: "Identify source subscription"
include_tasks: find_env.yml
loop: "{{ envs }}"
loop_control:
loop_var: env
and find_env.yml contains:
- name: "Show result"
set_fact:
source_subscription: "{{ env[item.key].subscription_id }}"
when:
- "source_image.split('/')[0] in env[item.key].acr_names"
- "source_subscription is undefined"
loop: "{{ env | dict2items }}"
All of this must be done twice for source and target.

${{ if }} Syntax in Azure DevOps YAML

I have a pool of self hosted VMs (MyTestPool) half of which is dedicated to installing & testing a 'ON' build (that has few features turned on) and a 'OFF' build (that's a default installation). All my test agent VMs have 'ALL' as a user defined capability. Half of them are also tagged 'ON', and the other half 'OFF'.
Now I have 2 stages called DEPLOYOFF & DEPLOYON that can be skipped if a Variable Group variable 'skipDeployOffStage' or 'skipDeployOnStage' is set to true. What I would like to do is to use 'ALL' as an agent demand if only ON / OFF is to be tested. If both ON & OFF are to be tested, the appropriate stage would demand their respective 'ON' or 'OFF' VMs.
Question: The ${{ if }} DOES NOT WORK.
trigger: none
pr: none
pool: 'MyBuildPool'
variables:
- group: TEST_IF_VARIABLE_GROUP
- name: subPool
value: 'ON'
- name: useAllPool
value: $[ or(eq( variables.skipDeployOnStage, true), eq( variables.skipDeployOffStage, true)) ]
stages:
- stage: DEPLOYOFF
condition: ne(variables['skipDeployOffStage'], 'true')
variables:
# The test stage's subpool
${{ if variables.useAllPool }}:
subPool: 'ALL'
${{ if not(variables.useAllPool) }}:
subPool: 'OFF'
pool:
name: 'MyTestPool'
demands:
- ${{ variables.subPool }}
jobs:
- job:
steps:
- checkout: none
- pwsh: |
Write-Host ("Value of useAllPool is: {0}" -f '$(useAllPool)')
Write-Host ("Value of VG variable 'skipDeployOnStage' is {0} and 'skipDeployOffStage' is {1}" -f '$(skipDeployOnStage)', '$(skipDeployOffStage)')
Write-Host ("Subpool is {0}" -f '$(subPool)')
displayName: 'Determined SubPool'
The Output when one of the flags is false:
2020-08-02T18:39:05.5849160Z Value of useAllPool is: True
2020-08-02T18:39:05.5854283Z Value of VG variable 'skipDeployOnStage' is true and 'skipDeployOffStage' is false
2020-08-02T18:39:05.5868711Z Subpool is ALL
The Output when both are false:
2020-08-02T18:56:40.5371875Z Value of useAllPool is: False
2020-08-02T18:56:40.5383258Z Value of VG variable 'skipDeployOnStage' is false and 'skipDeployOffStage' is false
2020-08-02T18:56:40.5386626Z Subpool is ALL
What am I missing?
There are two issues that cause your code to run incorrectly:
1.The ${{if}}:
The way you write ${{if}} is incorrect, and the correct script is:
${{ if eq(variables['useAllPool'], true)}}:
subPool: 'ALL'
${{ if ne(variables['useAllPool'], true)}}:
subPool: 'OFF'
2.The definition of variables.useAllPool:
You use a runtime expression ($[ <expression> ]), so when the ${{if}} is running, the value of variables.useAllPool is '$[ or(eq( variables.skipDeployOnStage, true), eq( variables.skipDeployOffStage, true)) ]' instead of true or false.
To solve this issue, you need to use compile time expression ${{ <expression> }}.
However, when using compile time expression, it cannot contain variables from variable groups. So you need to move the variables skipDeployOnStage and skipDeployOffStage from variable group to YAML file.
So, you can solve the issue by the following steps:
1.Delete the variables skipDeployOnStage and skipDeployOffStage from the variable group TEST_IF_VARIABLE_GROUP.
2.Modify the YAML file:
trigger: none
pr: none
pool: 'MyBuildPool'
variables:
- group: TEST_IF_VARIABLE_GROUP
- name: subPool
value: 'ON'
- name: skipDeployOnStage
value: true
- name: skipDeployOffStage
value: false
- name: useAllPool
value: ${{ or(eq( variables.skipDeployOnStage, true), eq( variables.skipDeployOffStage, true)) }}
stages:
- stage: DEPLOYOFF
condition: ne(variables['skipDeployOffStage'], 'true')
variables:
# The test stage's subpool
${{ if eq(variables['useAllPool'], true)}}:
subPool: 'ALL'
${{ if ne(variables['useAllPool'], true)}}:
subPool: 'OFF'
pool:
name: 'MyTestPool'
demands:
- ${{ variables.subPool }}
jobs:
- job:
steps:
- checkout: none
- pwsh: |
Write-Host ("Value of useAllPool is: {0}" -f '$(useAllPool)')
Write-Host ("Value of VG variable 'skipDeployOnStage' is {0} and 'skipDeployOffStage' is {1}" -f '$(skipDeployOnStage)', '$(skipDeployOffStage)')
Write-Host ("Subpool is {0}" -f '$(subPool)')
displayName: 'Determined SubPool'
You can modify the value of skipDeployOnStage and skipDeployOffStage in YAML file to test whether this solution works.
My requirement:
Have a self hosted VM pool of say, 20 machines
10 of them have "OFF" as a User Capability (Name as OFF & value as blank) 10 of them have
"ON" as a User capability (Name as ON & value as blank)
All of them have a "ALL" as an additional user capability
The pipeline basically installs 2 variations of the products and runs tests on them on the designated machines in the pool (ON / OFF)
When the user wants to run both OFF & ON deployments, I have stages that demand OFF or ON run
What I want to do is when the user wants only ON or OFF deployment, to save time I want to use all 20 of the machines, deploy and test so I can reduce overall testing time.
I was trying vainly, to replace the pool demand at run time as I did not want to update user capabilities for 20-50 VMs just prior to the run every time. That's what I have to do if use the traditional demand syntax:
pool:
name: 'MyTestPool'
demands:
# There's no OR or other syntax supported here
# LVALUE must be a built-in variable such as Agent.Name OR a User capability
# I will have to manually change the DeploymentType before the pipeline is run
- DeploymentType -equals $(subPool)
So, I was trying to ascertain the value of the subPool at run time and use the below syntax so I don't have to manually configure the user capabilities before run.
pool:
name: ${{ parameters.pool }}
demands:
- ${{ parameters.subPool}}
Here's my research:
You can definitely mix compile & run time expressions. The compile time expression evaluates to whatever the value was at the compile time. If it's an expression or a variable (and not a constant), the entire expression or variable is substituted as Jane Ma-MSFT notes.
For example, I have been able to use queue time variables in compile time expressions without issues. For example, I've used which pool to use as a queue time variable and then pass on the same to a template which uses a compile time syntax for the value.
parameters:
- name: pool
type: string
jobs:
- job: ExecAllPrerequisitesJob
displayName: 'Run Stage prerequisites one time from a Single agent'
pool:
name: ${{ parameters.pool }}
However, the real issue is where you're using the compile time expression. Essentially, in the above, the entire ${{ parameters.pool }} gets replaced by $(buildPool), a queue time variable at compile time. But, pool supports using a variable for the pool name. This is so convoluted and undocumented where you can use expressions, variables (compile or run) and where you must use constants.
One such example:
jobs:
- job: SliceItFourWays
strategy:
parallel: 4 # $[ variables['noOfVMs'] ] is the ONLY syntax that works to replace 4
In some places, such as Pool demands, Microsoft's YAML parser dutifully replaces the variable. However, the run time doesn't support custom variables as LVALUE.
It only supports evaluation of custom run time variables in the RVALUE portion of the demand.
pool:
name: ${{ parameters.buildPool }} # Run time supports variables here
demands:
- ${{ parameters.subPool }} # LVALUE: Must be resolved to a CONSTANT at compile time
- poolToRunOn -equals '${{ parameters.subPool }}' # RVALUE... custom user capability. This evaluates and applies the demand correctly.
Conclusion:
MICROSOFT'S YAML IMPLEMENTATION SUCKS!

Building lists based on matched attributes (ansible)

Trying to build a list of servers that match an attribute (in this case and ec2_tag) to schedule specific servers for specific tasks.
I'm trying to match against selectattr with:
servers: "{{ hostvars[inventory_hostname]|selectattr('ec2_tag_Role', 'match', 'cassandra_db_seed_node') | map(attribute='inventory_hostname') |list}}"
Though I'm getting what looks like a type error from Ansible:
fatal: [X.X.X.X]: FAILED! => {"failed": true, "msg": "Unexpected templating type error occurred on ({{ hostvars[inventory_hostname]|selectattr('ec2_tag_Role', 'match', 'cassandra_db_seed_node') | map(attribute='inventory_hostname') |list}}): expected string or buffer"}
What am I missing here?
When you build a complex filter chain, use debug module to print intermediate results... and add filter one by one to achieve desired result.
In your example, you have mistake on the very first step: hostvars[inventory_hostname] is a dict of facts for your current host only, so there is nothing to select elements from.
You need a list of hostvars' values, because selectattr is applied to a list, not a dict.
But in Ansible hostvars is a special variable and is not actually a dict, so you can't just call .values() on it without jumping through some hoops.
Try the following code:
- hosts: all
tasks:
- name: a kind of typecast for hostvars
set_fact:
hostvars_dict: "{{ hostvars }}"
- debug:
msg: "{{ hostvars_dict.values() | selectattr('ec2_tag_Role','match','cassandra_db_seed_node') | map(attribute='inventory_hostname') | list }}"
You can use the group_by module to create ad-hoc groups depending on the hostvar:
- group_by:
key: 'ec2_tag_role_{{ ec2_tag_Role }}'
This will create groups called ec2_tag_role_* which means that later on you can create a play with any of these groups:
- hosts: ec2_tag_role_cassandra_db_seed_node
tasks:
- name: Your tasks...

Resources