I have a very simple use for puppet; I create and delete students from a test server. I would like to store some configuration variables in another file and then just do a simple: include 'variables.pp' at the top of my manifest file. Is there a simple way to do this? I have consulted:
https://docs.puppet.com/puppet/latest/reference/lang_classes.html#using-include
https://puppet.com/blog/problem-separating-data-from-puppet-code
None of which give a simple solution. If I can only use the solutions above, I'll just hardcode it in my manifest.
Puppet 3 has an import statement that should serve your purpose. It is deprecated (and removed from Puppet 4), but it will do the job for you in Puppet 3:
variables.pp
# top-scope variables:
$var1 = 'foo'
$var2 = 'bar'
main.pp
import 'variables.pp'
# demo
notify { "var1 = ${var1}; var2 = ${var2}": }
Note that Puppet's import does not perform text interpolation; it is more like Python's import, making complete declarations from another manifest visible in the importing manifest.
Related
I got the idea that node js its not just for web application for example I can create a console application with node (cli) .
and already I have an interest in how I can make a cli app that create files and modify existing files for example something like angular cli with one command "ng generate component" its :-
1- create a set of files
2- modify app.module file
a. add import statement for generated component
b. add generated component in declarations array
and after a lot of search I got that first step can be handled in some way with node file system module.
but i don't know how they modify "app.module" file by just adding some syntax in its right place for instance adding new import statement after all exists import statements also adding the component name in declarations array as a last item
I'm really appreciate any help maybe with some code example if possible and thanks in advance
After some searches i found this answer:
There a couple of ways of editing a file, the most reliable is perhaps
the most complex one which can be done by parsing the file (Generating
an abstract syntax tree) update the new ast and pass it to a code
generator which will output the new string (code) of the modified ast.
Another option is to use regular expressions to know where add to
certain statements. For example there would be a regex to match import
statements to lines, you will map the lines of the file to this regex
where you'll get an array of booleans denoting whether the line is an
import stmt. Or not, once the import stmts are finished you can insert
a new line with the new import statement in the original lines array.
A third option is to regenerate the file all at once everytime, but
this means that you'll have to a ctx of the project (ctx = object
contains some details.
and as reference i found that AST (Abstract Syntax Tree) is more reliable way also it's not that hard this is some links that helped me a lot to know what is AST in simple way and how to deal with it
1- What is AST and how to understand it and how to use it https://www.youtube.com/watch?v=tM_S-pa4xDk
2- and this is an amazing article about "Write Code to Rewrite Your Code with jscodeshift" https://www.toptal.com/javascript/write-code-to-rewrite-your-code
https://www.youtube.com/watch?v=tM_S-pa4xDk
I am trying to force a Puppet class that creates a file to be processed before another class that needs this file to exist to run properly. Following the Puppet article Language: Containment of resources I am using contain.
My code does not work and I do not understand why. It gives this error:
Error: Evaluation Error: Error while evaluating a Function Call, Failed to parse template testing/def.erb:
Filepath: /root/local/testing/templates/def.erb
Line: 1
Detail: No such file or directory # rb_sysopen - /tmp/abc
at /root/local/test2.pp:16:16 on node example.com
Here is the code (stripped down):
### test2.pp
class klass1 {
file { '/tmp/abc':
content => 'xxx',
}
}
# Stage 0 creates the file /tmp/abc.
class stage0 {
contain klass1
}
# Stage 1 uses the contents of /tmp/abc to create the
# file /tmp/def.
class stage1 {
file { '/tmp/def':
content => template('testing/def.erb'),
}
}
# Try to force stage0 to be loaded before stage1.
include stage0
class { 'stage1':
require => Class['stage0']
}
### testing/templates/def.erb
Contents: <%= File.read("/tmp/abc") %>
I am using Puppet 5.3.3.
The issue here does not relate to containment, but to the dependency in your template at compile time on the call to File.read("/tmp/abc").
Ordinarily, compilation occurs on the Puppet Master a.k.a. Puppet Server, and the template function also runs at this time. Thus, your template def.erb attempts to read from a nonexistent file at compile time on the Puppet Master.
A better solution is likely to be define the content of file /tmp/abc in Puppet itself as data or a variable and then pass that variable to the template function, and so remove the dependency on reading from the file on disk altogether.
Without fully understanding why you were trying to separate this file content into multiple classes in the first place, I can't really comment any further.
Puppet is a declarative language that is typically used to ensure certain state of resources. But if you really need make a decision based on local code evaluation, you're basically left with 2 options:
Firstly, use facter, in some module create lib/facter/my_fact.rb:
#!/usr/bin/env ruby
require 'facter'
Facter.add(:load1) do
confine kernel: 'Linux'
setcode do
Facter::Util::Resolution.exec("cat /proc/loadavg | awk '{print $1}'")
end
end
Then your "function" result will be available via facter load1 (from shell) or $facts['load1'] from Puppet code. Such code is always evaluated before applying Puppet's catalog. If your function doesn't take arguments, this might be a good option. Note this is a silly example, the load is already available via facter load_averages.1m (though the usefulness of such fact is questionable). Using too many facts isn't good idea, it would prolong time required for applying a Puppet catalog. There's a soft limit for number of facts on puppetserver.
Second option would be using Deferred function. Evaluation of such function is delayed into later phase of catalog application (won't be evaluated on compile server).
Puppet code looks like this:
$value = Deferred("mymodule::load", ["1m"])
and the actual implementation should be in a Puppet module in a Ruby function, e.g. lib/puppet/functions/load.rb:
Puppet::Functions.create_function(:'mymodule::load') do
dispatch :load do
param 'String', :load_type
return_type 'String'
end
def load(load_type)
case load_type
when '1m'
avgs[0]
when '5m'
avgs[1]
else
raise "#{load_type} not supported"
end
end
end
The advantage is the latter approach is that you can pass multiple arguments. Although returning more complex types than String or Numeric doesn't seem to currently supported.
I am trying to set a module's source (this IS NOT a resource) based on a conditional trigger but it looks like the module is getting fired before the logic is applied:
module "my_module" {
source = "${var.my_field == "" ? var.standard_repo : var.custom_repo}"
stuff...
more stuff...
}
I have created the standard_repo and custom_repo vars as well and defined with URLs for respective repos (using git:: -- this all works w/o conditional)
All this being said, anyone know of a way to implement this conditional aspect? (again, this is a module and not a resource)
I tried using duplicate modules and calling based off the var value but this, too, does not work (condition is never met, even when it is):
repo = ["${var.my_field == "na" ? module.my_module_old : module.my_module_new}"]
One way to achieve this is described in this post
Basically, a common pattern is to have several folders for different environments such as dev/tst/prd. These environments often reuse large parts of the codebase. Some may be abstracted as modules, but there is still often a large common file which is either copy-pasted or symlinked.
The post offers a way that doesn't conditionally disable based on variables but it probably solves your issue of enabling a module based on different enviornments. It makes use of the override feature of terraform and adds a infra_override.tf file. Here, it defines a different source for the module which points to an empty directory. Voila, a disabled module.
Variables are not allowed to be used in the module source parameter. There also does not seem to be a plan for this to change. https://github.com/hashicorp/terraform/issues/1439 . Creating a wrapper script , or using something like mustache http://mustache.github.io/ seems to be the best way to solve the problem.
Is it possible to reference environment variables in logstash configuration?
In my case, i want to make my elasticsearch address configurable that i have set in the environment.
With logstash 2.3, you can set environment variable references into Logstash plugins configuration using ${var} or $var.
https://www.elastic.co/guide/en/logstash/current/environment-variables.html
Before logstash 2.3, you can use the "environment" filter plugin which is community maintained.
Documentation at : https://www.elastic.co/guide/en/logstash/current/plugins-filters-environment.html#plugins-filters-environment-add_field_from_env
How to install this plugin:
$LOGSTASH_HOME/bin/plugin install logstash-filter-environment
Source code at : https://github.com/logstash-plugins/logstash-filter-environment
The main part is:
# encoding: utf-8
require "logstash/filters/base"
require "logstash/namespace"
# Set fields from environment variables
class LogStash::Filters::Environment < LogStash::Filters::Base
config_name "environment"
# Specify a hash of fields to the environment variable
# A hash of matches of `field => environment` variable
config :add_field_from_env, :validate => :hash, :default => {}
public
def register
# Nothing
end # def register
public
def filter(event)
return unless filter?(event)
#add_field_from_env.each do |field, env|
event[field] = ENV[env]
end
filter_matched(event)
end # def filter
end # class LogStash::Filters::Environment
I can hardly believe, that these are the only solutions left: Hacking logstash or using some kind of templating system to re-write the config.
Actually, I do not want to touch or tweak the config for different deployment-scenarios: All I want is to pass in some parameters to connect logstash to the outside world (e.g. where elasticsearch is located, usernames/credentials to connect to other systems). I googled for an hour now an all I could find were these awkwardly complicated solutions for this very simple and common problem.
I sincerely hope, that someone comes up with a better idea like
%{ENV[ELASTICSEARCH_HOST]}}
That's not directly supported, no.
However, if you're running a version later than 1.4.0, it would be pretty trivial to edit elasticsearch.rb to add this feature. Around line 183:
client_settings["network.host"] = #bind_host if #bind_host
You could tweak it to read an environment variable:
if ENV["ESHOST"].nil? then
client_settings["network.host"] = ENV["ESHOST"]
else
client_settings["network.host"] = #bind_host if #bind_host
end
If you prefer, you can run Logstash with the -e command-line option to pass config via STDIN. You could cat in some file with special tokens that you've replaced with your environment variable(s).
The logstash configuration as of this writing is just a configuration file, but it's not a programing language. Thus, it has a few reasonable "limitations", for example, it cannot reference environment variables, cannot pass parameters it, hard to reuse other configuration file. Those limitations would make the logstash configuration file hard to maintain when the configuration file grows or you want to adjust its behavior on the fly.
My approach is to use template engine to generate the logstash configuration file. I used Jinja2 in Python.
For example, the elastic search output could be templated as
output {
elasticsearch {
index => {{ es_index_name }}
host => {{ es_hostname }}
}
}
Then I wrote a simple python code using Jinja2 to generate the logstash configuration file, and the value of es_index_name and es_hostname could be passed via the Python script argument. See here for Jiaja2 tutorial: http://kagerato.net/articles/software/libraries/jinja-quickstart.html
In this way, a big logstash configuration could be splitted into reusable pieces and its behavior can be adjusted on the fly.
As explained in logstash-issues
Connections are established at plugin registration time (during initialization, as they almost certainly should be), but field interpolation (like %{escluster}) is an event-processing-time operation. As such, host isn't really eligible for this behavior.
So unless input or output plugin will natively supports %{foo} syntax, doing any environment variable evaluation at the stage of event filtering is too late for the input and output plugin to take advantage of it.
.conf file support environment variables.
you just have to export the environment variable:
export EXAMPLE_VAR=123
and use it in the configuration file this way:
${EXAMPLE_VAR}
The cucumber-jvm javadocs states that purpose of the glue element is to specify the location of the stepdefinitions and hooks. However, this doesn't seem to work for me. Lets say I have my features in directory a, and my step definitions in directory b. Then,
#Cucumber.Options(
features= "directory_a",
glue="directory_b"
)
will load my feature files from directory_a, but, it doesn't load my step definitions from directly_b. However, if I use
#Cucumber.Options(
features= {"directory_a", "directory_b"}
)
then my features from directory_a is loaded, and my step definitions from directory_b are also picked up. Which is exactly what I want, however, I don't understand why the former isn't working? I'm guessing it has something to do with it expecting the URI to be formatted differently (maybe i need to prepend a classpath:// or something like that), but I can't find any information on this in the documentation.
I have successfully used something like:
#RunWith(Cucumber.class)
#Cucumber.Options(
//this code will only look into "features/" folder for features
features={"classpath:features/"},
glue = { "com.mycompany.cucumber.stepdefinitions", "com.mycompany.cucumber.hooks" },
format = { "com.mycompany.cucumber.formatter.RuntimeInfoCatcher", "json:target/cucumber.json" },
tags = { "#working" }
)
public class CucumberStarterIT {
}
Looking at the doc at http://cukes.info/api/cucumber/jvm/javadoc/cucumber/api/junit/Cucumber.Options.html it specifies the options to be of type String[] so perhaps it's not expected to work "well" if you don't give it a single-value list. Try glue={"directory_b"} and see what happens for you.
I had this problem too... and so far it seems to be that:
"features" is looking for a filesystem path:
features = "src/foo/bar"
whereas "glue" is looking for a package name:
glue = "foo.bar"
Not sure why they are different, but this seems to be working for me.
Hi as per my knowledge it all depends on the structure of you project. For example if you add the "Directory_a" ( directory which contains feature files) in the root level and StepDefinition, Hooks at src > test > java "Directory_b" And the TestRunner class at the same level ( src > test > java ) in "Directory_c"
Dir_a
|
src
|---main
|---test
|------java
|------Dir_b
|------Dir_c
You saying "Dir_b" while you are in the "Dir_c" It will identify "Dir_b" or any directory in same level with out any additional paths so,
It will be
glue = {"Dir_b"},
But when you look at the directory that includes feature file you have to give the path from the root level
In this case it's
features = {"Dir_a"}
or Giving the actual path eg :- "E://Project_Name//Dir_a" should work too
If your feature directory is NOT in root level make sure you give the path like "src/path to feature directory"
It will work fine :)