Appending a key to the top of an array - puppet

I have some hiera not unlike the following (I know this is invalid hiera with two keys... bare with me):
an::example::rule_files:
my_rules:
groups:
- name: my_rules
rules:
- alert: highCPU
expr: CPU > 90
for: 5m
annotations:
summary: "CPU is too high"
description: "CPU should be less than 90"
someone_elses_rules:
groups:
- name: someone_elses_rules
rules:
- alert: highCPU
expr: CPU > 70
for: 5m
annotations:
summary: "CPU is too high"
description: "CPU should be less than 70 on someone else's system"
I'm trying to turn this into a yaml file (the key is the filename). Now I know this is invalid hiera and I can remove the groups key to get this working (exactly what I've done), however when I try to reinsert it into the array, I can't get the formatting right. Here's the puppet code I'm using:
$alert_files = hiera('an::example::rule_files'),
$alert_files.each | String $alerts_file_name, Array $alert_config_pre | {
$prefix = [ "groups:" ]
$alert_config = $prefix + $alert_config_pre
file { "/etc/prometheus/${alerts_file_name}.rules":
ensure => file,
content => $alert_config.to_yaml,
}
}
Here's what I want:
cat /etc/prometheus/my_rules.rules
---
groups:
- name: my_rules
rules:
- alert: highCPU
expr: CPU > 90
for: 5m
annotations:
summary: CPU is too high
description: CPU should be less than 90
and here's what I get:
---
- 'groups:'
- name: my_rules
rules:
- alert: highCPU
expr: CPU > 90
for: 5m
annotations:
summary: CPU is too high
description: CPU should be less than 90
Any help would be massively appreciated. I feel like this should be simple but I've not really made any progress (I can't even remove the quotes from the word groups). If this is possible in either hiera or puppet (perhaps I've defined the hiera wrong) then great; any progress I can make in any way will be really appreciated.

This ...
$alert_files = hiera('an::example::rule_files'),
$alert_files.each | String $alerts_file_name, Array $alert_config_pre | {
... depends on the data associated with key an::example::rule_files to be a Hash with String keys and Array values. In the YAML presented at the top of the question, that item is instead a hash with String keys and Hash values. Inasmuch as the data seem to match the wanted file content, the problem seems to be not with the YAML (except for the inconsistent indentation), but rather with the Puppet code.
To work as you appear to want with the data you want, the Puppet code might look more like so:
$alert_files = lookup('an::example::rule_files'),
$alert_files.each |String $alerts_file_name, Hash $alert_config| {
file { "/etc/prometheus/${alerts_file_name}.rules":
ensure => 'file',
content => $alert_config.to_yaml,
}
}
Note that I have switched from the deprecated hiera() function to its replacement, lookup().

Related

How to modify the alert manager's threshold value

I use a node exporter program to monitor the CPU's threshold and receive notifications.
The example below is a rule file that sets conditions when it is more than 80%.
alert: HostHighCpuLoad
expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[2m])) * 100) > 80
for: 0m
labels:
severity: warning
annotations:
summary: Host high CPU load (instance {{ $labels.instance }})
description: "CPU load is > 80%\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
I need to receive a threshold from the web UI and change the threshold set in the rule file to 70%.
Can I convert the threshold into a variable?
expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[2m])) * 100) > {THRESHOLD_VALUE}
Or I wonder if I should make a script that modifies the 80% value to 70%.
Please give me advice.

Is there any way to reduce this code of prometheus alert expressions? I have multiple similar expression only the source instance is different

Suppose I am getting the metrics from a service in event_processing_bucket tag
where instance are like source=ONE, source=TWO, source=THREE ...... TEN
Currently I am using the following way to get the alert, but here I have written a separate expression just because i have to get data for every single source.
Is there any way to reduce this duplicate code. so that i could write only one alert rule and it will alert for all separately based on its respective value
Here are the prometheus alert expressions,
- alert: ONE_SLA_GREATER_THAN_5DAYS
expr: sum(rate(event_processing_bucket{source="ONE"}[1m])) > 5
for: 1m
labels:
severity: warning
team: mySlackChannel
annotations:
description: ONE_SLA is GREATER_THAN_5DAYS
summary: ONE_SLA is GREATER_THAN_5DAYS
- alert: TWO_SLA_GREATER_THAN_5DAYS
expr: sum(rate(event_processing_bucket{source="TWO"}[1m])) > 5
for: 1m
labels:
severity: warning
team: mySlackChannel
annotations:
description: TWO_SLA is GREATER_THAN_5DAYS
summary: TWO_SLA is GREATER_THAN_5DAYS
.
.
.
- alert: TEN_SLA_GREATER_THAN_5DAYS
expr: sum(rate(event_processing_bucket{source="TEN"}[1m])) > 5
for: 1m
labels:
severity: warning
team: mySlackChannel
annotations:
description: TEN_SLA is GREATER_THAN_5DAYS
summary: TEN_SLA is GREATER_THAN_5DAYS
Please guide me to write single expression code if possible. if not please specify.
Thanks in advance!!
One way is to group by
histogram_quantile(0.95, sum(increase(event_bucket[5m])) by (le, source)) > 5
later result values can be used to trigger those many alerts

How do I declare and use a variable in the yaml file that is formatted for pyresttest?

So, a brief description of what I want, what my issue is, and what I have tried.
I want to declare and use a dictionary variable for my tests in pyrest, specifically for the [url, body] section so that I can conduct my POST tests targeting a specific endpoint and with a preformatted body.
Here is how mytest.yml file is structured:
- data:
- id: 63
- rate: 25
... a sizable set of field for reasons ...
- type: lab_test_authorization
- modified_at: ansible_date_time.datetime # Useful way to generate
- test:
- url: "some-valid-url/{the_url_question}" # data['special_key']
- method: 'POST'
- headers : {etc..etc}
- body: '{ "data": ${the_body_question} }' # data (the content)
Now the problem starts in my lack of understanding why (if true) does pyrest does not have support for dictionary mappings. I understand yaml supports these feature but am not sure if pyrest can parse through it. Knowing how to call and use dictionary variable in my url and body tags would be significantly helpful.
As of right now, if I try to convert my data Sequence into a data Dictionary, I will get an error stating:
yaml.parser.ParserError: while parsing a block mapping
in "<unicode string>", line 4, column 1:
data:
^
expected <block end>, but found '-'
in "<unicode string>", line 36, column 1:
- config:
I'm pretty sure there are gaps in my knowledge regarding how yaml and pyresttest interact with each other, so any insight would be greatly appreciated.

hiera() function and YAML hash lookups

How do I rewrite this YAML so it is more structured, then reference it in Puppet using hiera function?
Currently, I am working with a hieradata syntax that looks very flat and hard to read.
service::proxy::behind_reverse_proxy: true
service::proxy::proxy_timeout: 300
service::proxy::serverlist:
- host1.fqdn
- host2.fqdn
And grabbed these in a params.pp file, for example
$behind_reverse_proxy = hiera('service::proxy::behind_reverse_proxy', 'False')
$serverlist = hiera('service::proxy::serverlist')
I thought I could rewrite the YAML like so in an effort to make it more readable...
service::proxy:
behind_reverse_proxy: true
proxy_timeout: 300
serverlist:
- host1.fqdn
- host2.fqdn
And updated the params.pp file according to
Hiera Key.subkey syntax
interacting with structured data
$behind_reverse_proxy = hiera('service::proxy.behind_reverse_proxy', 'False')
$serverlist = hiera('service::proxy.serverlist')
However upon puppet agent -t that resulted in
Error 400 on SERVER: Could not find data item service::proxy.serverlist in any Hiera data file and no default supplied
I think these are relevant
[user#server ~]$ facter -y | grep 'version'
facterversion: 2.4.4
puppetversion: 3.8.2
Following up on my comment about how you can access your restructured data:
service::proxy:
behind_reverse_proxy: true
proxy_timeout: 300
serverlist:
- host1.fqdn
- host2.fqdn
In your manifest, instead of this ...
$behind_reverse_proxy = hiera('service::proxy.behind_reverse_proxy', 'False')
$serverlist = hiera('service::proxy.serverlist')
... you might do this:
$proxy_info = merge(
{ 'behind_reverse_proxy' => false, 'serverlist' => [] },
hiera('service::proxy', {})
)
$behind_reverse_proxy = $proxy_info{'behind_reverse_proxy'}
$serverlist = $proxy_info{'serverlist'}
The merge() function is not built-in, but rather comes from Puppet's (formerly PuppetLabs's) widely-used stdlib module. There's a good chance that you are already using that module elsewhere, but even if not, it may be well worth your while to introduce it to your stack.
I've never used Hiera, but I think the problem is that you have a sequence (array) when you wanted a mapping (hash).
In the below YAML, the value of the service::proxy key is a sequence with three elements, each of which is a mapping with one key:
service::proxy:
- behind_reverse_proxy: true
- proxy_timeout: 300
- serverlist:
- host1.fqdn
- host2.fqdn
What you probably wanted, though, was for service::proxy to be a mapping with three keys:
service::proxy:
behind_reverse_proxy: true
proxy_timeout: 300
serverlist:
- host1.fqdn
- host2.fqdn
The examples in the Hiera docs you linked to seem to support this.

Building lists based on matched attributes (ansible)

Trying to build a list of servers that match an attribute (in this case and ec2_tag) to schedule specific servers for specific tasks.
I'm trying to match against selectattr with:
servers: "{{ hostvars[inventory_hostname]|selectattr('ec2_tag_Role', 'match', 'cassandra_db_seed_node') | map(attribute='inventory_hostname') |list}}"
Though I'm getting what looks like a type error from Ansible:
fatal: [X.X.X.X]: FAILED! => {"failed": true, "msg": "Unexpected templating type error occurred on ({{ hostvars[inventory_hostname]|selectattr('ec2_tag_Role', 'match', 'cassandra_db_seed_node') | map(attribute='inventory_hostname') |list}}): expected string or buffer"}
What am I missing here?
When you build a complex filter chain, use debug module to print intermediate results... and add filter one by one to achieve desired result.
In your example, you have mistake on the very first step: hostvars[inventory_hostname] is a dict of facts for your current host only, so there is nothing to select elements from.
You need a list of hostvars' values, because selectattr is applied to a list, not a dict.
But in Ansible hostvars is a special variable and is not actually a dict, so you can't just call .values() on it without jumping through some hoops.
Try the following code:
- hosts: all
tasks:
- name: a kind of typecast for hostvars
set_fact:
hostvars_dict: "{{ hostvars }}"
- debug:
msg: "{{ hostvars_dict.values() | selectattr('ec2_tag_Role','match','cassandra_db_seed_node') | map(attribute='inventory_hostname') | list }}"
You can use the group_by module to create ad-hoc groups depending on the hostvar:
- group_by:
key: 'ec2_tag_role_{{ ec2_tag_Role }}'
This will create groups called ec2_tag_role_* which means that later on you can create a play with any of these groups:
- hosts: ec2_tag_role_cassandra_db_seed_node
tasks:
- name: Your tasks...

Resources