We are using the roles pattern in puppet with hiera, meaning we have these lines in hiera.yaml:
- name: "Roles data"
path: "roles/%{::server_role}.yaml"
We have a custom fact that produces the role name when facter runs, but we would like to move this into hiera. Instead of the server_role variable being produced by facter, we want to specify the server_role inside of hiera, and let that variable be referenced elsewhere in hiera. Something like this:
hiera.yaml:
- name: "Per-node data"
path: "nodes/%{trusted.certname}.yaml"
- name: "Roles data"
path: "roles/%{lookup(server_role)}.yaml"
nodes/hostname.yaml:
server_role: foo_bar
I have seen this question, which says to use hiera() or lookup() but when I try to use those, I get this error message:
Interpolation using method syntax is not allowed in this context
So how can I use a hiera variable that's defined elsewhere in hiera?
Edit:
The prototypical code examples for defining roles could use any fact that's known to facter, often giving examples that are based on hostname. When you can't embed server config into hostname, a common(ish) workaround is to write a file such as /etc/server_role, but it seems to defeat the purpose of config management, when you need to ssh into a machine and edit a file. As the other comments & answer here so far mentioned, you could use an ENC, but again, the goal here is not to have config stored outside of version control. In fact, we have foreman as an ENC and we make a practice to never use it that way because then upgrades and other maintenance become unsustainable.
We could write a class which will pick up data from hiera, write it to /etc/server_role, and on the next puppet run, facter will pick that up and send it back to hiera, so then we'll have the server_role fact available to use in hiera.yaml. As gross as this sounds, so far, it's the best known solution. Still looking for better answers to this question.
Thanks.
As #MattSchuchard explained in comments, you cannot interpolate Hiera data into your Hiera config, because the config has to be known before the data can be looked up.
If you need a per-role level in your data hierarchy then an alternative would be to assign roles to machines via an external node classifier. You don't need it to assign any classes, just the server_role top-scope variable and probably also environment.
On the other hand, maybe you don't need a per-role level of your general hierarchy in the first place. Lots of people do roles & profiles without per-role data, but even if you don't want to do altogether without then it may be that module-specific data inside the module providing your role classes could be made to suffice.
Related
The Infra team in my company has provided us with sample overthere.SshHost under 'Infrastructure' in XL-Deploy UI that has a predefined private key file and passphrase which is not shared with us.
We are asked to duplicate this file manually in the UI, rename it and create infra entries for our application.
How can I achieve this with puppet?
Lets say the sample file is placed under: Infrastructure/Project1/COMMONS/Template_SshHost
and I need to create an overthere.SshHost under Infrastructure/Project1/UAT/Uat_SshHost and Infrastructure/Project1/PREPROD/Preprod_SshHost by copying the sample file.
Thanks in advance!
You can sync a target file with another file accessible via the local file system by using a File resource whose source attribute specifies the path to the original. You can produce a modified copy in a variety of ways, such as by applying one or more File_line resources (from stdlib) or by applying an appropriate script via an Exec resource.
But if you go that route then you have to either
accept that the target file will be re-synced on every Puppet run, OR
set the File resource's replace attribute to false, in which case changes to the original file will not be propagated into the customized copy.
The latter is probably the more acceptable choice for most people. Its file-copying part might look something like this:
$project_dir = '/path/to/Infrastructure/Project1'
file { "${project_dir}/UAT/Uat_SshHost/overthere.SshHost":
ensure => 'file',
source => "${project_dir}/COMMONS/Template_SshHost/overthere.SshHost",
replace => false,
}
But you might want to consider instead writing a custom type and provider for the target file. That would allow you to incorporate changes from the original template without re-syncing the file on every run, and it would give you a lot more flexibility with respect to the customizations you need to apply. It would also present a simpler interface for you to use in your manifests, which could make managing these easier. But, of course, that's offset by the cost is that writing and maintaining a custom type and provider. Only you can determine whether that would be a worthwhile trade-off.
I am trying to set a module's source (this IS NOT a resource) based on a conditional trigger but it looks like the module is getting fired before the logic is applied:
module "my_module" {
source = "${var.my_field == "" ? var.standard_repo : var.custom_repo}"
stuff...
more stuff...
}
I have created the standard_repo and custom_repo vars as well and defined with URLs for respective repos (using git:: -- this all works w/o conditional)
All this being said, anyone know of a way to implement this conditional aspect? (again, this is a module and not a resource)
I tried using duplicate modules and calling based off the var value but this, too, does not work (condition is never met, even when it is):
repo = ["${var.my_field == "na" ? module.my_module_old : module.my_module_new}"]
One way to achieve this is described in this post
Basically, a common pattern is to have several folders for different environments such as dev/tst/prd. These environments often reuse large parts of the codebase. Some may be abstracted as modules, but there is still often a large common file which is either copy-pasted or symlinked.
The post offers a way that doesn't conditionally disable based on variables but it probably solves your issue of enabling a module based on different enviornments. It makes use of the override feature of terraform and adds a infra_override.tf file. Here, it defines a different source for the module which points to an empty directory. Voila, a disabled module.
Variables are not allowed to be used in the module source parameter. There also does not seem to be a plan for this to change. https://github.com/hashicorp/terraform/issues/1439 . Creating a wrapper script , or using something like mustache http://mustache.github.io/ seems to be the best way to solve the problem.
I'm new to configuration management, just FYI.
I'm trying to puppetize elasticsearch, and want to have a master list of elasticsearch nodes in a file (which can be used for multiple things, not just this purpose).
I would like to add elasticsearch.yml via an ERB template and expand the list of FDQN's into the discovery.zen.ping.unicast.hosts: [] param.
For example I have an external file called es_hosts in module/files that contains:
host1.domain.com
host2.domain.com
host3.domain.com
host4.domain.com
Then when puppet builds the ERB template have this in the param:
discovery.zen.ping.unicast.hosts: ["host1.domain.com", "host2.domain.com", "host3.domain.com", "host4.domain.com"]
I've tried a few things, but I can't get my head wrapped around it.
I would be using this list for other things like building firewall rules, etc, so I'd like to have one master list for reference that can be updated by my team.
Thanks for any help!
Rather than have a list in a file, it would be better to have it in Hiera, since defining lists and other external data is specifically what Hiera is for.
(If you have not used Hiera yet, you definitely should read up on it.)
So in Hiera you would have:
---
es_hosts:
- host1.domain.com
- host2.domain.com
- host3.domain.com
- host4.domain.com
In your manifest, you would read that in from Hiera using the hiera function:
$es_hosts = hiera('es_hosts')
(Note that instead of the hiera function, we often use Puppet's Automatic Parameter Lookup feature instead to read data into our manifests from Hiera, but your requirement here - a list of ES hosts to be used in multiple contexts - suggests you will want this list not to be bound to a specific class input. If this does not make sense to you right now, you will need to learn about Parameterised Classes and Automatic Parameter Lookup, but it's otherwise not relevant to this answer.)
Finally, in your ERB template you would have:
discovery.zen.ping.unicast.hosts: ["<%= #es_hosts.join('", "') %>"]
Pay attention to the fact that the $es_hosts variable from your manifest is accessed via a Ruby instance variable #es_hosts in your ERB template.
Finally, note that there is an Elasticsearch Puppet module available on the Puppet Forget here. You may find that using that module is better than writing your own.
Is there anyway to normalize the title value through a munge call or something similiar?
I have a custom type/provider that manages local group membership on Windows machines (we needed more granularity over group membership, than just inside the user/group types.
group_member{"Group => Member":
ensure=>present,
}
group_member{"Group => DOMAIN\Member":
ensure=>present,
}
resources{'group_member':
purge =>true,
}
In general, when everything is consistent everything works great. The issue comes in that windows/active directory are not case sensitive. Therefore, when self.instances gets called, It generates a resource Group_member["Users => DOMAIN\SomeGroup"], but in our puppet manifest, Users is hardcoded, DOMAIN comes from a fact, and SomeGroup comes from a hiera value.
We've been fighting with the case sensitivity of puppet for about 5 years on this, and most of our administrators have gotten used to ensuring case matches between domain and yaml configurations.
Now, we have a new problem... somehow DOMAIN is messing up. On a new testing domain, the NETBIOS name is lowercase. The fact we had returning DOMAIN still returns uppercase, but Windows ADSI is returning a lowercase form. This wouldn't be an issue if we wern't using resources{'group_member': purge=>true}, but now in this test environment the groups get added (through their respective .pp files) and removed (through resources{'group_member': purge=>true}}) every puppet run.
Ideally i'd like to just normalize everything to lowercase in self.instances and in all of our .pp files, but we have 400 puppet modules and we use group_member 120 times, managed by quite a few different teams. It would be significantly easier if I could just munge the title metaparameter.
Any suggestions?
So, the way i'm handling this is to rename the existing type/provider and wrapping it in a define type that downcase's the title.
I have a root level variable, whose value is set at run time via a tfvar file. The tfvar file used and the var value can vary. Now I want to use this variable inside a module, the terraform way to do this is to set it up as a module variable and pass the root var when creating the module.
Except, I have this module used in our infrastructure several hundred times. Is repeating this variable 100s of times the only way to do this? Can the module access root namespace to grab the variable value?
I'm half tempted to use an external data script in the module to fetch the value instead, except, I wont know which tfvars file will be in effect at runtime..
Unfortunately, the only Terraform supported way is to pass the value into the module as a variable. Terraform has made me a copy/paste expert.
The only other approach that comes to mind that Terraform supports is making creative use of the External Data Source.