I have a use-case to setup monitoring system for a fleet of servers.
I was able to do it but, now i have to filter them and show these servers Names instead of server IP's.
I mean the data i get from node_exporter is like this:
ex: up{instance="10.21.88.12:9100",job="node_exporter"}
i want this like:
ex: up{instance="ServerA",job="node_exporter"} (replace 10.21.88.12:9100 with a name "ServerA")
can i do that.
or can i at-least add my own key value pair in the metric data like:
up{instance="10.21.88.12:9100",server_name="ServerA",job="node_exporter"}
Thank you in advance.
You can use the relabel_config option.
If you add a key that not exist in the label, it will be added at the label.
if you add a key that exist in the label, it will be updated at the label.
relabel_configs:
- source_labels: [ __address__ ]
target_label: server_name
replacement: 'ServerA'
If used as above, server_name will be added to the label.
relabel_configs:
- source_labels: [ __address__ ]
target_label: instance
replacement: 'ServerA'
If used as above, the instance key value in the label will be changed to ServerA.
Related
context: I'm adding a new resource to TF Provider (using SDKv2) with roughly the following schema:
resource "player" "football" {
type = "FOOTBALL"
...
config = {
"dribbling" = "50"
"speed" = "90"
"position" = "GOALKEEPER"
}
}
that I represent as:
"config": {
Type: schema.TypeMap,
Elem: &schema.Schema{
Type: schema.TypeString,
},
Required: true,
ForceNew: true,
},
The important detail here for different palyer instances' types there'll be a different set of required attributes (dribbling, speed, position for football and height, can_dunk, arm_span for basketball) -- all players share the same API endpoint so I introduced just one resource to cover them all.
I'd like to support the ability of importing players and apparently READ response includes a bunch of fields that are optional on create (and I suspect most of the users won't have them in Terraform configuration file) which results in the fact that I've got a state difference when saving the whole config like:
d.Set("config", player.GetConfig()) # GetConfig includes a bunch of new attributes (optional on a create or even computed)
So I've got a question: which of the following 2 options is preferable:
Implement DiffSuppressFunc for a config attribute where I'll be ignoring these optional fields (the downside is I'll have an implicit drift between main.tf and TF state file).
Be more restrictive when writing configs to TF state file:
instead of
d.Set("config", player.GetConfig())
# filtered config will match config in main.tf
filteredConfig = ...
d.Set("config", filteredConfig)
In some other Terraform providers that deal with similar situations (where a particular argument has a mixture of configuration-provided and remote-system-provided nested values), the resource type implementation takes a compromise position of effectively exposing the same data in two different attributes, where one of them represents what the user configured and the other represents the full data returned by the remote system. For example, you might have config to be set in the configuration, and expanded_config representing the full set of elements the server decided.
There is a challenge with that approach in that you'll probably need a special rule in your Read function to somehow decide if a change you detect in the remote system constitutes "drift" relative to the configuration or if it's just an additional element added by the server.
From what you described it seems like the rule could be that any key that's present in config in the prior state (that is, the values visible to d.Get inside Read before you call d.Set) would have its value overwritten by what the server returned, but any keys that were not present before are ignored entirely. This would create the effect then that any key the author specified in the configuration is considered "managed by Terraform" while any other key is only read by Terraform and not directly managed.
If you adopt that strategy then it's worth keeping in mind what will happen in a situation where the user has changed the configuration to include a new key or to remove a previously-present key. The Read operation is in terms of the previous state rather than the configuration, so that function will see the keys that were present at the end of the last apply, not the keys currently present in the configuration. In particular this means that if an author adds a new key that the server was already tracking then it will appear in the subsequent plan as being added, even though it might technically be more appropriate to show it as an in-place update ~ or a no-op. This is an example of the compromises we sometimes need to make in order to adapt remote APIs to fit within Terraform's model of resource instances.
I have a project on which I have to use terraform and in the end of the terraform, I need to append consul key's values places on a /path. I have the following:
resource "consul_keys" "write" {
datacenter = "dc1"
token = "xxxx-x-x---xxxxxx--xx-x-x-x"
key {
path = "path/to/name"
value = jsonencode([
{
cluster_name = "test", "region" : "us-east1"
},
{
cluster_name = "test2", "region" : "us-central1"
}
])
}
}
But if I run the terraform again with new values, it deletes all previous values and update new values.
Any way I can keep appending the values keeping previous values as it is?
The consul_keys resource type in the hashicorp/consul provider only supports situations where it is responsible for managing the entirety of the value of each of the given keys. This is because the underlying Consul API itself treats each key as a single atomic unit, and doesn't support partial updates of the sort you want to achieve here.
If you are able to change the design of the system that is consuming these values, a different way to get a comparable result would be to set aside a particular key prefix as a collection of values that the consumer will merge together after reading them. Consul's Read Key API includes a mode recurse=true which allows you to provide a prefix to read all of the entries with a given prefix in a single request.
By designing your key structure this way, you can use a separate keys for the data that Terraform will provide and the data provided by each other system that will generate data under this shared prefix. These different systems can therefore each maintain their own designated sub-key and not need to take any special extra steps to preserve existing data already stored at that location.
If you are using consul-template then consul-template's ls function wraps the multi-key lookup I described above.
If you are reading the data from Consul in some other Terraform configuration, the consul_key_prefix data source similarly implements the operation of fetching all key/value pairs under a given prefix.
I'm currently using hiera to set all my class parameters for the Puppet forge Gitlab module.
cat hieradata/nodes/example.yaml
---
gitlab::backup_cron_enable: true
gitlab::gitlab_rails:
backup_keep_time: 604800
backup_path: /opt/gitlab_backup
gitlab_default_can_create_group: false
initial_root_password: foobar
...
cat site/profiles/manifests/gitlab.rb
class profile::gitlab {
include gitlab
}
This code works as intended but I'd like to redact the password values in the log output and reports.
I tried to use hiera_options to convert the sensitive values but Puppet still displays the unredacted values.
cat hieradata/nodes/example.yaml
---
lookup_options:
gitlab::gitlab_rails::initial_root_password:
convert_to: "Sensitive"
gitlab::backup_cron_enable: true
gitlab::gitlab_rails:
backup_keep_time: 604800
backup_path: /opt/gitlab_backup
gitlab_default_can_create_group: false
initial_root_password: foobar
...
What is the best way to redact all sensitive values whilst using hiera to define the class parameters?
You need to have the password as a separate key in order for the auto conversion to take effect. The key that is looked up is bound to a hash, and it is not possible to address individual values in a hash with lookup_options (it is the entire hash that is looked up).
You can make an individual value Sensitive by using an alias and binding the password in clear text to a separate key - like this:
cat hieradata/nodes/example.yaml
---
lookup_options:
gitlab::gitlab_rails::initial_root_password:
convert_to: "Sensitive"
gitlab::backup_cron_enable: true
gitlab::gitlab_rails:
backup_keep_time: 604800
backup_path: /opt/gitlab_backup
gitlab_default_can_create_group: false
initial_root_password: '%{alias("gitlab::gitlab_rails::initial_root_password")}'
gitlab::gitlab_rails::initial_root_password: 'foobar'
...
With this approach you could also use EYAML or some other secure hiera backend to store the password in encrypted form. Such a backend may already return decrypted values wrapped in Sensitive - this is for example done by the Vault backend.
However, even if you get past the first hurdle, the result depends on what the gitlab module does with the hash now containing a Sensitive value. If it just passes the value for initial_root_password on it may work, but if it is doing any operation on this value (like checking if it is an empty string for example) it may fail. If you are unlucky it may seem to work but you may end up with the password "redacted" :-). Contact the maintainers of the module if it does not work and request that they support having the password as a Sensitive value instead of a String.
I want to create a DataSore through ssoadm.jsp because I use endpoint url in order to automatize process of configuration.
[localhost]/ssoadm.jsp?cmd=create-datastore
I put:
domain name (previously created with default coniguration): myDomain
data store name: myDataStore
type of DataStore: LDAPv3
Attribut values: LDAPv3=org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo
Then I got something like: Attribute name "LDAPv3" doesn't match with service schema. What am I supposed to put in those fields "Attribut values" pls? An example is given:
"sunIdRepoClass=com.sun.identity.idm.plugins.files.FilesRepo"
PS: I dont want to create datastore from [Localhost]/realm/IDRepoSelectType because there is jato.pageSession that i can't automaticly get.
PS2: it is my first time asking a question on Stackoverflow, sorry if my question didn't fit with the expectation. I tried my best.
ssoadm.jsp?cmd=list-datastore-types
shows the list of user data store types
Every user data store type has specific attributes to be set. Unfortunately those are not explicitly documented. The service attributes are defined in the related service definition XML template, which is loaded (after potential tag swapping) into the OpenAM configuration data store during initial configuration. For the user data stores you can find them in OPENAM_CONFIGURATION_DIRECTORY/template/xml/idRepoService.xml
E.g. for user data store type LDAPv3 the following service attributes are defined
sunIdRepoClass
sunIdRepoAttributeMapping
sunIdRepoSupportedOperations
sun-idrepo-ldapv3-ldapv3Generic
sun-idrepo-ldapv3-config-ldap-server
sun-idrepo-ldapv3-config-authid
sun-idrepo-ldapv3-config-authpw
openam-idrepo-ldapv3-heartbeat-interval
openam-idrepo-ldapv3-heartbeat-timeunit
sun-idrepo-ldapv3-config-organization_name
sun-idrepo-ldapv3-config-connection-mode
sun-idrepo-ldapv3-config-connection_pool_min_size
sun-idrepo-ldapv3-config-connection_pool_max_size
sun-idrepo-ldapv3-config-max-result
sun-idrepo-ldapv3-config-time-limit
sun-idrepo-ldapv3-config-search-scope
sun-idrepo-ldapv3-config-users-search-attribute
sun-idrepo-ldapv3-config-users-search-filter
sun-idrepo-ldapv3-config-user-objectclass
sun-idrepo-ldapv3-config-user-attributes
sun-idrepo-ldapv3-config-createuser-attr-mapping
sun-idrepo-ldapv3-config-isactive
sun-idrepo-ldapv3-config-active
sun-idrepo-ldapv3-config-inactive
sun-idrepo-ldapv3-config-groups-search-attribute
sun-idrepo-ldapv3-config-groups-search-filter
sun-idrepo-ldapv3-config-group-container-name
sun-idrepo-ldapv3-config-group-container-value
sun-idrepo-ldapv3-config-group-objectclass
sun-idrepo-ldapv3-config-group-attributes
sun-idrepo-ldapv3-config-memberof
sun-idrepo-ldapv3-config-uniquemember
sun-idrepo-ldapv3-config-memberurl
sun-idrepo-ldapv3-config-dftgroupmember
sun-idrepo-ldapv3-config-roles-search-attribute
sun-idrepo-ldapv3-config-roles-search-filter
sun-idrepo-ldapv3-config-role-search-scope
sun-idrepo-ldapv3-config-role-objectclass
sun-idrepo-ldapv3-config-filterrole-objectclass
sun-idrepo-ldapv3-config-filterrole-attributes
sun-idrepo-ldapv3-config-nsrole
sun-idrepo-ldapv3-config-nsroledn
sun-idrepo-ldapv3-config-nsrolefilter
sun-idrepo-ldapv3-config-people-container-name
sun-idrepo-ldapv3-config-people-container-value
sun-idrepo-ldapv3-config-auth-naming-attr
sun-idrepo-ldapv3-config-psearchbase
sun-idrepo-ldapv3-config-psearch-filter
sun-idrepo-ldapv3-config-psearch-scope
com.iplanet.am.ldap.connection.delay.between.retries
sun-idrepo-ldapv3-config-service-attributes
sun-idrepo-ldapv3-dncache-enabled
sun-idrepo-ldapv3-dncache-size
openam-idrepo-ldapv3-behera-support-enabled
It might be best that you create an user data store instance via console and then use ssoadm.jsp?cmd=show-datastore to list the properties. You would get a long list of attriutes ... to much to show here.
When you create the data store, make sure you specify the password for the bind DN using property
sun-idrepo-ldapv3-config-authpw=PASSWORD
Hallo I am building in Hiera / Puppet a data structure for creating mysql / config files. My goal ist to have some default values which can be overwritten with a merge. It works until this point.
Because we have different mysql instances on many hosts I want to automaticly configure some paths to be unique for every instance. I have the instance name as a hash (name) of hashes in the Namespace: our_mysql::configure_db::dbs:
In my case I want to lookup the instance names like "sales_db' or 'hr_db' in paths like datadir, but I can not find a way to lookup the superior keyname.
Hiera data from "our_mysql" module represents some default values:
our_mysql::configure_db::dbs:
'defaults':
datadir: /var/lib/mysql/"%{lookup('lookup to superior hash-key name')}"
log_error: /var/log/mysql/"%{lookup('lookup to superior hash-key name')}".log
logbindir: /var/lib/mysql/"%{lookup('lookup to superior hash-key name')}"
db_port: 3306
...: ...
KEY_N: VALUE_N
Hiera data from node definiton:
our_mysql::configure_db::dbs:
'sales_db':
db_port: "3317"
innodb_buffer_pool_size: "1"
innodb_log_file_size: 1GB
innodb_log_files_in_group: "2"
server_id: "1"
'hr_db':
db_port: "3307"
I now how to do simple lookups or to iterate by
.each | String $key, Hash $value | { ... }
but I have no clue how to reference a key from a certain hierarchy level. Searching all related topics to puppet and hiera didn't help.
Is it possible an any way and if yes how?
As I understand the question, I think what you hope to achieve is that, for example, when you look up our_mysql::configure_db::dbs.sales_db key, you get a merge of the data for that (sub)key and those for the our_mysql::configure_db::dbs.defaults subkey, AND that the various %{lookup ...} tokens in the latter somehow resolve to the string sales_db.
I'm afraid that's not going to happen. The interpolation tokens don't even factor in here -- Hiera simply won't perform such a merge at all. I guess you have a hash-merge lookup in mind, but that merges only identical keys and subkeys, so not our_mysql::configure_db::dbs.sales_db and our_mysql::configure_db::dbs.defaults. Hiera provides for defaults for particular keys in the form of data recorded for those specific keys at a low-priority level of the data hierarchy. The "defaults" subkey you present, on the other hand, has no special meaning to the standard Hiera data providers.
You can still address this problem, just not entirely within the data. For example, consider this:
$dbs = lookup('our_mysql::configure_db::dbs', Hash, 'deep')
$dbs.filter |$dbname, $dbparms| { $dbname != 'defaults' }.each |$dbname, $dbparms| {
# Declare a database using a suitable resource type. "my_mysql::database" is
# a dummy resource name for the purposes of this example only
my_mysql::database {
$dbname:
* => $dbparams;
default:
datadir => "/var/lib/mysql/${dbname}",
log_error => "/var/log/mysql/${dbname}.log",
logbindir => "/var/lib/mysql/${dbname}",
* => $dbs['defaults'];
}
}
That supposes data of the form presented in the question, and it uses the data from the defaults subkey where those do not require knowledge of the specific DB name, but it puts the patterns for various directory names into the resource declaration, instead of into the data. The most important things to recognize are the use of the splat * parameter wildcard for obtaining multiple parameters from a hash, and the use per-expression resource property defaults by use of the default keyword in a resource declaration.
If you wanted to do so, you could push more details of the directory names back into the data with a little more effort (and one or more new keys).