I'd like to preface by saying I'm new to puppet. I have been working with it via vagrant and am starting to feel comfortable writing manifests, but I lack perhaps the experience, or intuition, that can answer my question.
I am trying to grasp how puppet is scoped, and where lines are drawn. I am specifically interested in how this applies to modules and their creation and use.
A more concrete example: the puppletlabs-nginx module. So hypothetically I'm going along my merry way, creating a manifest for a given server role; say it's a dead-simple static file webserver, and I'd like to use nginx. The module will clearly help me with that; there's try_files support and such. I can even ramp up to reverse-proxying via this module. But what if things get stickier? What if there's something I want to do programmatically that I cannot do with the module?
Well, perhaps the short answer is to fix it myself, do a pull request, and go along my merry way. But where does that stop? Is the goal of a community puppet module to support every facet of a given software package? That seems unmanageable. On the other hand, doesn't that create a bunch of mostly-baked modules, build solely from use cases?
Then, there's an analog to Android UI: there are setter methods for what I think is most XML UI definitions. In puppet if feels similar. You can build a config file programmatically, or create it by filling in an ERB template. In other words, I feel the line in puppet between programmatic creation of configuration files and the templated creation of configuration files is blurred; I found no best way with Android and so I don't know which is the way to go with puppet.
So, to the question: what constitutes the ideal puppet module? Should it rely more on templates? On the manifest? Should it account for all configuration scenarios?
From a further-withdrawn perspective it almost seems I want something more opinionated. Puppet's power seems to be flexibility and abstraction, but the modules that are out there feel inconsistent and not as fleshed out.
Thanks for reading...
Thanks to Mark. In just a small amount of time I've switched over to playing with Chef and the modules seem better in regards to many of the concerns I voiced.
In short i can explain you about puppet.
Puppet is nothing but it is an IT automation tool where we can install software's on other machines by creating manifests (recipes or scripts) on master for those softwares to be installed on target machine.
Here master indicates implementation of puppet manifests for softwares.
Target machine indicates agent where softwares to be installed.
Puppet module constitutes of following structure where we do this in master.
In master the path is /etc/puppet/modules to enter into modules directory,you have mentioned puppletlabs-nginx module.so now we can take this module as an example.
After modules directory we have to create files and manifests directories.furthermore,in manifests directory we will create .pp files.For instance, install.pp,uninstall.pp.this is how module structure will be.we usually run these scripts by using few resources like package,service,file,exec etc.
Templates play a minor role in puppet manifests just to hardcore the values.it is not a major part of puppet.Manifests has a great importance in puppet.
For automating any software by using puppet we can follow the above structure.
Thank you.
The PuppetLabs solution here is to use different types of modules for each function -- Components, Profiles, and Roles. See the presentation Designing Puppet: Roles/Profiles Pattern for more information.
The modules available from PuppetForge are of the "Component" type. Their purpose is to be as flexible as possible, while focusing on a single logical piece of software -- e.g., either the apache httpd server OR apache tomcat, but not both.
The kinds of modules that you might write to wrap around one of those Component modules would be a perfect example of a "Profile" module. This could tie apache httpd together along with tomcat and jboss and maybe also some other components like mysql and php. But it's one logical stack, with multiple components. Or maybe you have a LAMP Profile module, and a separate tomcat+jboss Profile module.
The next level up is "Role" modules. They shouldn't have anything in them other than "include" statements that point at the appropriate Profile modules.
See the PuppetLabs presentation for more details, but this logic is pretty similar to what is seen in the Chef world with "wrapper cookbooks".
Related
We have a multi repo structure for our terraform scripts and there are lot of shared resources among these repo's. We want to restructure these repo's using modularization like calling one resource from one repo to another repo using module source . What is the best approach for cleaning the scripts and modularize?
It's difficult to define a "best" approach as there will be tradeoffs that depend on your particular setup, particularly when it comes to cleaning and modularizing your terraform files.
You're best bet for getting a specific answer is to ask a more specific question. I'd suggest starting with the HashiCorp Learn tutorial on Reuse Configuration with Modules
which includes a section on securely sharing your modules in a private module registry.
If you work through those sections and hit a particular problem, you'll almost certainly get a response here if you ask for help on solving that specific problem.
Not sure there is going to be a right or wrong answer for this one, but I am just interested how people manage Terraform in the real world? In terms of do you use modules, different environments and collaborations.
At the moment we are planning on having a production, dev and test environments. All similar.
Now at the moment I have made my terraform files in a way that define individual components of AWS, so say one for, VPC, IAM, EC2, Monitoring (CloudWatch + CloudTrail + CloudConfig) etc. And there is one variable file and .tfvars for the above, so the files are portable (all environments will be the same). So if you need to change something its all in one place. Also means if we have a specific project running I can create a tf file defining all the resource for the project and drop it in, then once its completed remove it.
Each environment has its own folder structure on our Terraform server.
Is this too simplistic? I keep looking at module.
Also does anyone have experience of collaboration with Terraform, as in different teams? I have been looking at things like Atlantis to tie into GitHub, so any changes need to be approved. But also at the sametime with the correct IAM role I can limit what Terraform can change.
Like I said may not be a wrong of right answer just interested in how people are managing terraform and their experiences.
Thanks
My answer is just an use case...
We are using terraform for an application deployed for several customers each having small specific configuration features.
We have only one CVS repository. We don't use CVS branches mechanism.
For each folder, we have remote states at least to share states between developers.
We are using one global folder having remote states also to share states between customers configurations
We are using one folder per customer and using workspaces (former environment) for each context for each customer (prod:blue/green, stage)
For common infrastructure chunks shared by all customers, we use module
We mainly use variables to reduce the number of specific files in each customer folders.
Hope this will help you...
I would like to use Terraform programmatically like an API/function calls to create and teardown infrastructure in multiple specific steps. e.g reserve a couple of eips, add an instance to a region and assign one of the IPs all in separate steps. Terraform will currently run locally and not on a server.
I would like to know if there is a recommended way/best practices for creating the configuration to support this? So far it seems that my options are:
Properly define input/output, heavily rely on resource separation, modules, the count parameter and interpolation.
Generate the configuration files as JSON which appears to be less common
Thanks!
Instead of using Terraform directly, I would recommend a 3rd party build/deploy tool such as Jenkins, Bamboo, Travis CI, etc. to manage the release of your infrastructure managed by Terraform. Reason being is that you should treat your Terraform code in the exact same manner as you would application code (i.e. have a proper build/release pipeline). As an added bonus, these tools come integrated with a standard api that can be used to execute your build and deploy processes.
If you choose not to create a build/deploy pipeline, your other options are to use a tool such as RunDeck which allows you to execute arbitrary commands on a server. It also has the added bonus of having a excellent privilege control system to only allow specified users to execute commands. Your other option could be to upgrade from the Open Source version of Terraform to the Pro/Premium version. This version includes an integrated GUI and extensive API.
As for best practices for using an API to automate creation/teardown of your infrastructure with Terraform, the best practices are the same regardless of what tools you are using. You mentioned some good practices such as clearly defining input/output and creating a separation of concerns which are excellent practices! Some others I can recommend are:
Create all of your infrastructure code with idempotency in mind.
Use modules to separate the common shared portions of your code. This reduces the number of places that you will have to update code and therefore the number of points of error when pushing an update.
Write your code with scalability in mind from the beginning. It is much simpler to start with this than to adjust later on when it is too late.
From Puppet Best Practices:
The Puppet Labs documentation describes modules as self-contained bundles of code and data.
Ok it's clear.
A single module can easily manage a single application.
So, puppetlabs-apache manages Apache only, puppetlabs-mysql manages MySQL only.
.... So, my module my_company-mediawiki manages Mediawiki only (i suppose... with database and virtual host... because a module is self-contained bundles of code and data).
Modules are most effective when the serve a single purpose, limit dependencies, and concern themselves only with managing system state relating to their named purpose.
But my_company-mediawiki needs to depend by:
puppetlabs-mysql: to create database;
puppetlabs-apache: to manage a virtual host.
And... from a rapid search I understand that many modules refer to other modules.
But...
They provide complete functionality without creating dependencies on any other modules, and can be combined as needed to build different application stacks.
Ok, a good module is self-contained and has no dependencies.
So I have to necessarily use the pattern roles and profiles to accomplish these best practices? Or I'm confused...
The Puppet documentation's description of modules as self-contained is more aspirational than definitive. Don't read too much into it, or into others' echoes of it. Modules are quite simply Puppet's next level of code organization above classes and defined types, incorporating also plug-ins and owned data.
Plenty of low-level modules indeed have no cross-module dependencies, but such dependencies inescapably arise when you start forming aggregations at a level between that and whole node configurations. There is nothing inherently wrong with that. The Roles & Profiles pattern is a good way to structure such aggregations, but it is not the only way, and in any case it does not avoid cross-module dependencies because role and profile classes, like any other, should themselves belong to modules.
We are a mixed linux/windows shop that successfully adopted Puppet for Config Mgmt a while ago. We'd like to drop ansible in as our deployment orchestration tool (research suggests that puppet doesn't do this very well) but have questions about how to integrate the two products.
Today, puppet is the source of truth with respect to environment info (which nodes belong to which groups etc). I want to avoid duplicating this information in ansible. Are there any best practices with regards to sharing environment information between the two products?
One way to reduce the amount of duplicated state between the systems is to use Ansible's "Dynamic Inventory" support. Instead of defining your hosts/groups in a text file, you use a script that pulls the same data from somewhere else. This could be PuppetDB, Foreman, etc and is going to depend on your environment.
Writing a new script is also pretty simple, it just needs to be any executable (bash/python/ruby/etc) that returns json in a specific format.
Lastly, it is possible to roll out new releases with puppet, but it is easier with a "microservice" like release process. Ensuring apps/services/databases remain backwards compatible across versions can make pushing out releases trivial with puppet and your favorite package manager.
Using Puppet and Mcollective should be the way to go if you are looking for a solution from puppetlabs
https://puppetlabs.com/mcollective