I am trying to write duplicate node blocks in site.pp file. This site.pp file I am generating from Java code. When I do test 'puppetd --test' I am not getting other node blocks changes on client.
site.pp
node "puppetclient1.domain.com" {
file { "twc-bind-9.9.4-0.noarch.rpm" :
source => "puppet:///files/modules/BIND/twc-bind-9.9.4-0.noarch.rpm",
}
}
node "puppetclient1.domain.com" {
package { "twc-bind" :
source => "/opt/test/files/twc-bind-9.9.4-0.noarch.rpm",
provider => "rpm",
ensure => "latest",
}
}
node "puppetclient1.domain.com" {
service { "named" :
subscribe => File["/opt/test/files/twc-bind-9.9.4-0.noarch.rpm"],
ensure => "running",
}
}
I'm pretty sure that puppet will match against the first node it finds.
You need to make your Java code a little bit smarter and add all of the resources into a single node, i.e.
node "puppetclient1.domain.com" {
file { "twc-bind-9.9.4-0.noarch.rpm" :
source => "puppet:///files/modules/BIND/twc-bind-9.9.4-0.noarch.rpm",
}
package { "twc-bind" :
source => "/opt/test/files/twc-bind-9.9.4-0.noarch.rpm",
provider => "rpm",
ensure => "latest",
}
service { "named" :
subscribe => File["/opt/test/files/twc-bind-9.9.4-0.noarch.rpm"],
ensure => "running",
}
}
Or another option would be to use node inheritance.
If you'll have to deal with hundred of resources and thousands of boxes, you should care about make a good design and modeling. Put your resources into classes, and then classes into more general classes and then, put classes into boxes. And use hiera or parameterized classes or both to change resources
class twc-bind {
file { "/opt/test/files/twc-bind-9.9.4-0.noarch.rpm" :
source => "puppet:///files/modules/BIND/twc-bind-9.9.4-0.noarch.rpm",
}
package { "twc-bind" :
source => "/opt/test/files/twc-bind-9.9.4-0.noarch.rpm",
provider => "rpm",
ensure => "latest",
}
service { "named" :
ensure => "running",
}
File["twc-bind-9.9.4-0.noarch.rpm"]->Package["twc-bind"]->Service["named"]
}
node "puppetclient1.domain.com" {
class { "twc-bind" :
}
}
If you're using Java to generate manifests, you shuold model your Java classes too.
Related
I have multiple logstash pipelines set up on server that feeds data in Index. Every pipelines adds bunch of fields at the first level of Index along with their nested level.
I already have kpi1 and kpi2 values inside Metrics => data with Metrics being nested array. And I have a requirement to add a new pipeline that will feed the value of kpi3. Here is my filter section in the new pipeline that I created:
filter {
ruby {
code => "
event.set('kpi3', event.get('scoreinvitation'))
"
}
mutate {
# Rename the properties according to the document schema.
rename => {"kpi3" => "[metrics][data][kpi3]"}
}
}
It overwrites the Metrics section ( may be because it is an array??). Here is my mapping :
"metrics" : {
"type" : "nested",
"properties" : {
"data" : {
"properties" : {
"kpi1" : {
....
}
}
}
"name" : {
"type" : "text",
....
}
}
}
How can I keep the existing fields (and values) and still add the new fields inside Metrics => Data ? Any help is appeciated.
The Logstash pipeline looks good, however your mapping doesn't make much sense to me if I'm understanding your requirement correctly.
The metrics property doesn't have to be of type nested. In fact, the metrics property is just a json namespace that contains sub-fields / -objects.
Try the following mapping instead
"metrics": {
"properties": {
"data": {
"properties": {
"kpi1": {
# if you want to assign a value to the kpi1 field, it must have a type
}
}
},
"name": {
"type": "text"
}
}
}
I have a strange problem with a logstash filter, that was working up until yesterday.
This is my .conf file:
input {
beats {
port => 5044
}
}
filter {
if "access.log" in [source] {
grok {
match => { "message" => "%{GREEDYDATA:messagebefore}\[%{HTTPDATE:real_date}\]\ %{GREEDYDATA:messageafter}" }
}
mutate {
replace => { "[message]" => "%{messagebefore} %{messageafter}" }
remove_field => [ "messagebefore" ]
remove_field => [ "messageafter" ]
}
date {
match => [ "real_date", "dd/MMM/YYYY:HH:mm:ss Z" ]
}
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
}
}
The issue is that in the output, the derived variables %messagebefore and %message after are coming through as literal text, rather than the content.
Example:
source:/var/log/nginx/access.log message:%{messagebefore} %{messageafter}...
The strange thing is that this was working fine before yesterday afternoon. I also appreciate that this is probably not the best way to process nginx logs, but I'm using this one as an example only as it's affecting all of my other configuration files as well.
My environment:
ELK stack running as a docker container on Centos 7 derived from docker.io/sebp/elk.
Filebeat running on Centos 7 client.
Any ideas?
Thanks.
Solved this myself, and posting here in case anyone gets the same issue.
When building the docker container, I inadvertently left behind another .conf file that also contained reference to access.log. The two .conf files were clashing as logstash was processing both. I deleted the erroneous file and it has all started working.
I'm trying to use class inheritance for puppet. There is a base class named foo and an inherited class named bar. Overriding file or package resources are quite fine and works properly. But at the same time i'm using a custom module for configuring sudo.
The problem arises when i try to override sudo class from inside class bar. Puppet classes are as follows:
class foo {
$nrpe_plugin_pkgs = [ .... ]
service { 'nrpe service':
..
}
package { $nrpe_plugin_pkgs:
..
}
file { '/etc/nagios/nrpe.cfg':
..
}
file { '/etc/resolv.conf':
..
}
class { 'sudo':
purge => true,
config_file_replace => true,
}
sudo::conf { 'sudo_conf':
priority => 10,
content => [
'%gr1 ALL=(userfoo) NOPASSWD: ALL',
'%gr1 ALL=(root) NOPASSWD: /usr/bin/wbinfo *',
]
}
}
class bar inherits foo {
File['/etc/resolv.conf'] {
..
}
sudo::conf { 'sudo_conf':
priority => 10,
content => [
'%gr2 ALL=NOPASSWD:/bin/chown userbar\:gr2 /dirbar/*',
'%gr2 ALL=NOPASSWD:/bin/chown -R userbar\:gr2 /dirbar/*',
'%gr2 ALL=NOPASSWD:/bin/chmod * /dirbar/*',
]
}
}
I just want to customize only resolv.conf and sudo config, but i'm getting error as follows:
Error while evaluating a Resource Statement, Evaluation Error: Error
while evaluating a Resource Statement, Duplicate declaration:
Sudo::Conf[sudo_conf] is already declared in file
/etc/puppetlabs/code/environments/foobar_servers/manifests/foobar.pp:80;
cannot redeclare at
/etc/puppetlabs/code/environments/foobar_servers/manifests/foobar.pp:335
at
/etc/puppetlabs/code/environments/foobar_servers/manifests/foobar.pp:335:3
on node foobartest01
/etc/sudoers.d/10_sudo_conf file expected to be created. How can i achieve that?
Using: Puppet 4.9 Community version.
Any help appreciated.
The resolv.conf override works here, because you're using the correct syntax:
class bar inherits foo {
File['/etc/resolv.conf'] {
..
}
But the syntax for the next line where you try to override is different, so it fails:
sudo::conf { 'sudo_conf':
This is the syntax to declare a new resource (hence why you get a duplicate error) rather than to override an existing resource. It should be:
Sudo::Conf['sudo_conf'] {
priority => 10,
content => [
'%gr2 ALL=NOPASSWD:/bin/chown userbar\:gr2 /dirbar/*',
'%gr2 ALL=NOPASSWD:/bin/chown -R userbar\:gr2 /dirbar/*',
'%gr2 ALL=NOPASSWD:/bin/chmod * /dirbar/*',
]
}
I am new to the whole ElasticSearch framework and have downloaded an installed the logstash-input-jmx plugin and now I need to test my configuration but I can't find in any of the LogStash documentation exactly how to test the plugin. All they have in the plugin documentation in GitHub is a sentence down at the bottom that says to start LogStash and test your plugin, they don't tell you exactly how to accomplish that. As a matter of fact that seems to be the standard blurb for all of the plugins which isn't very helpful if you're coming in without any knowledge of the framework.
Here are some details for my configuration if that helps:
logstash.conf
input {
jmx
{
path => "file://*machinename*/D$/LS/logstash-5.1.1/config/jmx"
polling_frequency => 15
type => "jmx"
}
}
filter {
it [type] == "jmx" {
if ("Memory.HeapMemoryUsage" in [metric_path] or "Memory.NonHeapMemoryUsage" in [metric_path]) {
ruby {
code => "event['memoryUsage'] = event['metric_value_number'] * 100"
add_tag => [ "memoryUsage" ]
}
}
}
}
jmx.conf:
{
"host" : *ip address of machine*,
"port" : *jmx listener port*,
"queries" : [
"object_name" : "java.lang:type=Memory",
"object_alias" : "Memory"
]
}
TIA,
Bill
Figured it out by doing a complete uninstall/reinstall of the framework and found a very good tutorial on Ivan Krizsan's blog (https://www.ivankrizsan.se/2015/09/27/jmx-monitoring-with-the-elk-stack/) that was instrumental in helping me get the plug-in up and running.
I want to provision multiple sets of things on a server using existing puppet modules the simplest example would be:
file { "/var/www/MYVARIABLEHERE":
ensure => "directory",
}
mysql::db { MYVARIABLEHERE:
user => MYVARIABLEHERE,
password => MYVARIABLEHERE,
host => 'localhost',
grant => ['all'],
}
Is there a way to abstract this out so that I can have say an array of pre defined options and then pass them into existing puppet modules so I don't end up with a manifest file that's thousands of lines long?
As per the answer below I have setup:
define mySites {
mysql::db { $name:
user => $name,
password => $name,
host => 'localhost',
grant => ['all'],
}
file { "/var/www/${name}.drupal.dev":
ensure => "directory",
}
}
I then call:
mySites {"site": $name => "test", }
and get the following error:
Could not parse for environment production: Syntax error at 'name'; expected '}'
You could use a define type to simplify as much :
define mydef( $usern, $passn) {
file { "/var/www/$usern":
ensure => "directory",
}
mysql::db { $usern :
user => $usern,
password => $passn,
host => "localhost",
grant => ['all'],
}
}
# You have to call the define type for each cases.
mydef{"u1": usern => "john", password => "pass", }
# It might be possible to provide multiple arrays to a define
# type if you use puppet's future parser