I have a module with multiple manifests. I have two types of VMs (for different functionality with "APP" and "PRT" as substring in their names). I have few manifests common for both the VMs but some manifests are specific to the VMs.
$hostname = generate('hostname').
I have the hostname.
All I need to implement is:
if(hostname matches regex(APP)){
execute manifest a.pp & b.pp
}
else if (hostname matches regex(PRT)){
execute manifest a.pp & bc.pp
}
How can I do this with puppet?
In your site.pp (main manifest):
node /.*APP.*/ {
class { 'a': }
class { 'b': }
}
node /.*PRT.*/ {
class { 'a': }
class { 'bc': }
}
If your puppet codebase becomes more sophisticated then you can do more sophisticated class inclusion here, but this gets the job done for the case you are working with.
Note relevant documentation here: https://docs.puppet.com/puppet/latest/reference/lang_node_definitions.html
Related
I'm learning puppet (v6), and trying to understand how to set class parameters when a specific node needs an additional parameter, but uses the same class. Maybe a little fuzzy on the terminology, but here's what I'm working on:
MyNode1 needs sshd configured to use a banner and timeout, so using ghoneycutt-ssh, I include the ssh class with parameters:
/modules/MyModule/manifests/MySSH.pp
# Configures SSH
class MyModule::MySSH {
# Using ssh module
class { '::ssh':
sshd_client_alive_count_max => 0,
sshd_client_alive_interval => 900,
sshd_config_banner => '/etc/MyBanner.txt',
}
}
Now I have a second node MyNode2, which requires MySSH above, and also needs to disable forwarding. I started with something like this, where I define only the additional parameter in its own class:
/modules/MyModule/manifests/MySSH_Node2.pp
class MyModule::MySSH_Node2 {
class { '::ssh':
sshd_allow_tcp_forwarding => 'no',
}
}
Then define MyNode2 to include both in my site definition, hoping that puppet merges my ::ssh definitions:
/manifests/site.pp
node MyNode1 {
include MyModule::MySSH
}
node MyNode2 {
include MyModule::MySSH
include MyModule::MySSH_Node2
}
I understand that the above example doesn't work due to error Duplicate declaration: Class[Ssh]. I also tried overriding the class with a new parameter:
class MyModule::MySSH_Node2 {
Class[ssh] {
sshd_allow_tcp_forwarding => 'no',
}
}
But it seems this is not allowed either: Error: Resource Override can only operate on resources, got: Class[ssh]-Type
I'm not sure what the best way to add parameters is. I know I can create a manifest that includes all the parameters needed for this node and apply that instead, but then I end up with duplicate code everywhere.
Is there a reasonable way in modern puppet to assign and merge class parameters like this in puppet?
I understand at a basic level the various moving parts of build.gradle build scripts but am having trouble tying it all together.
In Apache Spark standalone mode, just trying to start a master and worker on the same box from build.gradle. (Later will extend with call with $SPARK_HOME/sbin/start-slaves with the proper argument for masterIP.)
Question: How can I assign my IP address to a variable in Groovy/build.gradle so I can pass it to a command in an Exec task? We want this to run on a couple different development machines.
We have a (I think fairly standard) /etc/hosts config with the FQDN and hostname assigned to 127.0.1.1. The driver gets around this OK but starting master and slaves with hostnames is not an option, I need the ip address.
I am trying:
task getMasterIP (type: Exec){
// declare script scope variable using no def or
executable "hostname"
args += "-I"
// need results of hostname call assigned to script scope variable
sparkMasterIP = <resultsOfHostnameCall>
}
// added this because startSlave stops if Master is already running
task startSlaveOnly(dependsOn:'getMasterIP', type: Exec){
executable "/usr/local/spark/sbin/start-slave.sh"
args += "spark://$sparkMasterIP:7077"
doLast {
println "enslaved"
}
}
// now make startSlave call startSlaveOnly after the initial startMaster
task startSlave(dependsOn:'startMaster', type: Exec) {
finalizedBy 'startSlaveOnly'
}
When I try something like suggested in the docs for Exec for Groovy calls:
task getMasterIP (type: Exec){
// declare script scope variable using no def or
sparkMasterIP = executable "hostname"
args += "-I"
}
I get a warning that executable is not recognized.
The " for a little more background on what I am thinking" section, not the main question.
Googling "build.gradle script scope variables" and looking at the first two results, in the basic docs I only see one type of variable and ext properties to be used.
16.4. Declaring variables -- There are two kinds of variables that can be declared in a build script: local variables and extra properties.
But in this other Gradle doc Appendix B. Potential Traps I am seeing two kinds of variables scopes aside from the ext properties:
For Gradle users it is important to understand how Groovy deals with
script variables. Groovy has two types of script variables. One with a
local scope and one with a script-wide scope.
With this example usage:
String localScope1 = 'localScope1'
def localScope2 = 'localScope2'
scriptScope = 'scriptScope'
I am assuming I should be using script-scope variables with no "def" or type declaration.
To fetch local IPs:
// Return all IPv4 addresses
def getLocalIPv4() {
def ip4s = []
NetworkInterface.getNetworkInterfaces()
.findAll { it.isUp() && !it.isLoopback() && !it.isVirtual() }
.each {
it.getInetAddresses()
.findAll { !it.isLoopbackAddress() && it instanceof Inet4Address }
.each { ip4s << it.getHostAddress() }
}
return ip4s
}
// Optionally, return all IPv6 addresses
def getLocalIPv6() {
def ip6s = []
NetworkInterface.getNetworkInterfaces()
.findAll { it.isUp() && !it.isLoopback() && !it.isVirtual() }
.each {
it.getInetAddresses()
.findAll { !it.isLoopbackAddress() && it instanceof Inet6Address }
.each { ip6s << it.getHostAddress() }
}
return ip6s
}
task printIP() doLast {
println getLocalIPv4()
println getLocalIPv6()
}
The two functions above return a list of IPv4 or IPv6 addresses respectively. You might notice that I'm skipping all localhosts, interfaces that are not up, all loopbacks and virtual interfaces. If you want to use the first IPv4 address, you can use it elsewhere as:
getLocalIPv4()[0]
or in your case:
args += "spark://"+ getLocalIPv4()[0] + ":7077"
I found this post that appears to be a more straightforward way of doing this but it limited to Linux platforms, hostname -I doesn't work in Windows and maybe not all Linux distros?
getting hostname
assigning it to variable
using in a build.gradle
task
Here's the task I built as a result, the accepted answer is much better and more universal, this is just for another way of looking at it
task getMasterIP{
doLast {
new ByteArrayOutputStream().withStream { os ->
def result = exec {
executable = 'hostname'
args += '-I'
}
ext.ipAddress = os.toString()
}
}
}
RaGe's answer does a better job of looking at all interfaces on all platforms
I am trying to create Puppet module to setup my web server.
What i want - is to split configuration to logical modules (manifests by services: webserver, database, ftp etc.) But I can not figure out how to use additional manifests in init.pp
I am going to use it only with puppet apply not server-client configuration.
My text module manifest (kp/manifests/init.pp):
class kp {
include kp::testfile
}
include kp
And additional manifest (kp/manifests/testfile.pp)
define kp::testfile {
$value = template("kp/some.erb")
file { 'testfile':
path => '/tmp/my.txt',
ensure => file,
content => $value
}
}
Documentation says:
If a class is defined in a module, you can declare that class by name in any manifest. Puppet will automatically find and load the manifest that contains the class definition.
But when I run puppet apply init.pp I am getting error message
Could not find class kp::testfile for myhost.com at /myDir/puppetModules/kp/manifests/init.pp:2 on node vagrant.example.com
Facts
/myDir/puppetModules/ is in modulepath so no problems here
Puppet version v2.7.11
Ubuntu 12.04 LTS
What I am doing wrong?
Thanks in advance!
Your kp::testfile is a defined type, not a class. To use a defined type you need to declare it like:
kp::testfile { 'name': }
Try redefining kp::testfile like
class kp::testfile {
$value = template("kp/some.erb")
file { 'testfile':
path => '/tmp/my.txt',
ensure => file,
content => $value
}
}
and you may have better luck.
I want to override parameters of base nodes. What I want to get is a pattern like this:
# File manifests/nodes.pp
node myDefault {
class { 'my::common::puppet_setup':
service => 'enable',
pushable => 'disable',
}
# Do lots of default things ...
}
node 'myFirstNode' inherits myDefault {
# Do something ...
}
node 'mySecondNode' inherits myDefault {
class { 'my::common::puppet_setup::params':
service => 'disable',
pushable => 'enable',
}
}
I understood the the puppet documentation, i could do this by writing my module like this:
# File modules/my/manifests/common/puppet_setup.pp
class my::common::puppet_setup (
$pushable = $my::common::puppet_setup::params::pushable,
$service = $my::common::puppet_setup::params::service,
) inherits my::common::puppet_setup::params {
# package that configures puppet node
# input value validation
validate_re($pushable, ['^enable$', '^disable$', '^ignore$', ])
validate_re($service, ['^enable$', '^disable$', '^ignore$', '^cron$', ])
# setup puppet, start or disable agent, put ssh keys for push ...
}
class my::common::puppet_setup::params {
$pushable = 'enable'
$service = 'enable'
$puppetserver = 'puppet.my.site.de'
case $::osfamily {
'Debian': {
}
default: {
fail("not implemented yet for {::operatingsystem}")
}
}
}
The Documentations on puppet website says:
When a derived class is declared, its base class is automatically declared first (if it wasn’t already declared elsewhere).
But i get this error (some indentation added):
mySecondNode# puppet agent --test --environment dev_my
Error: Could not retrieve catalog from remote server:
Error 400 on SERVER: Duplicate declaration:
Class[My::Common::Puppet_setup::Params] is already declared;
cannot redeclare at /.../puppet/manifests/nodes.pp:16 on node mySecondNode
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run
I'm reading on this for a week and i guess my understanding ist totally wrong somewhere, although i used the puppetlabs ntp modules as an example.
what am i missing?
You should check Inheritance section from http://docs.puppetlabs.com/puppet/latest/reference/lang_node_definitions.html
Puppet treats node definitions like classes. It does not mash the two together and then compile the mix; instead, it compiles the base class, then compiles the derived class, which gets a parent scope and special permission to modify resource attributes from the base class.
One of the good solutions is to use roles and profiles, there's a great blog post about it:
http://garylarizza.com/blog/2014/02/17/puppet-workflow-part-2/
You can use virtual resources :
http://docs.puppetlabs.com/guides/virtual_resources.html
I am experimenting with Gradle to build a few jars, rather than maintain a list of classes that hold EJBs so that I can deploy them separately I thought it might be neat to scan the classes when making the jar.
Rather than load the classes and use reflection to get the annotations I thought it may be simpler to scan the classes with asm, hence the chuncky ClassReader in one of the tasks.
I don't think this is the issue so can be ignored, basically I have 2 tasks that I use to define the contents of the jars, both report that different content is going into them via the eachFile print out, however when I look in the publish repository location both files and associated sha1 are identical.
Either Gradle is broken or, more likely, I've done something crazy but can't see what it is, can anyone help?
By the way if I disable the publish of either of the jar files the one that does get created is correct so I think it's something wrong with the publish rather than the jarring up, but could be wrong.
// ASM is used to interpret the class files, this avoids having to load all classes in the vm and use reflection
import org.objectweb.asm.*
task ejbJar(type: Jar) {
//outputs.upToDateWhen { false }
from "${project.buildDir}/classes/main"
eachFile { println "EJB server: ${name}" }
include getEjbClassFiles(project.buildDir)
}
task clientEjbJar(type: Jar) {
//outputs.upToDateWhen { false }
from "${project.buildDir}/classes/main/com/company/core/versioner"
eachFile { println "Client EJB ${name}" }
include '**/*'
}
artifacts {
archives clientEjbJar
archives ejbJar
}
String[] getEjbClassFiles(base) {
def includedFiles = []
def baseDir = project.file("${base}/classes/main")
def parentPath = baseDir.toPath()
if (baseDir.isDirectory()) {
baseDir.eachFileRecurse(groovy.io.FileType.FILES) { file ->
if(file.name.endsWith('.class')) {
//get hold of annotations in there --- org.objectweb.asm.Opcodes.ASM4
def reader = new ClassReader(file.bytes).accept(
new ClassVisitor(Opcodes.ASM4) {
public AnnotationVisitor visitAnnotation(String desc, boolean visible) {
if(desc.equals("Ljavax/ejb/Stateless;") ||
desc.equals("Ljavax/ejb/Stateful;")) {
includedFiles += parentPath.relativize(file.toPath())
}
return null //no interest in actually visiting the annotation values
}
},
ClassReader.SKIP_DEBUG | ClassReader.EXPAND_FRAMES | ClassReader.SKIP_FRAMES | ClassReader.SKIP_CODE
)
}
}
}
return includedFiles
}
publishing {
publications {
mypub(IvyPublication) {
artifact(ejbJar) {
name 'ejb'
}
artifact(clientEjbJar) {
name 'client-ejb'
}
}
}
repositories {
ivy {
name 'personal'
url "${ant['developer.repository']}/"
layout 'pattern', {
artifact "[organisation]/[module]/[artifact]/[revision]/[type]/[artifact]-[revision].[ext]"
ivy "[organisation]/[module]/[type]/[revision]/[type]/[type]-[revision].[ext]"
}
}
}
}
I did break the thing down into a simpler form as I thought it may be a Gradle bug.
The simplified form was:
apply plugin: 'java'
apply plugin: 'ivy-publish'
task bigJar(type: Jar) {
from "${rootDir}/src/main/resources"
include '**/*'
}
task smallJar(type: Jar) {
from "${rootDir}/src/main/resources/A/B"
include '**/*'
}
group 'ICantBeEmpty'
artifacts {
archives bigJar
archives smallJar
}
publishing {
publications {
mypub(IvyPublication) {
artifact(bigJar) { name 'biggie' }
artifact(smallJar) { name 'smallie' }
}
repositories {
ivy {
name 'personal'
url "c:/temp/gradletest"
layout 'pattern', {
artifact "[organisation]/[module]/[artifact]/[revision]/[type]/[artifact]-[revision].[ext]"
ivy "[organisation]/[module]/[type]/[revision]/[type]/[type]-[revision].[ext]"
}
}
}
}
}
This results in 2 files in c:/temp/gradletest/ICantBeEmpty/report-bug/biggie/unspecified/biggie-unspecified.jar and c:/temp/gradletest/ICantBeEmpty/report-bug/smallie/unspecified/smallie-unspecified.jar
Both of these files are identical, however I think I know why see my later answer.
Whilst looking at some configurations I noticed some odd behaviour that led me to a resolution of this issue, and it is a Gradle bug.
In my build I had a scratch task doing
configurations.archives.artifacts.each { println it }
This gave me 5 different lines output, however changing it to this
configurations.archives.artifacts.each { println it.file }
produced the same filename 5 times.
It turns out this is related to my issue, although the artifacts are there as separate entities the name used to uniquely identify them was the same so the same file was always chosen during a publish. The name of the artifacts is given by ${baseName}-${appendix}-${version}-${classifier}.${extension} by default in the java plugin. This means that if neither appendix or classifier is specified then the artifact will have the same name.
I tested this using the above sample code by adding an appendix name
task bigJar(type: Jar) {
appendix = 'big'
from "${rootDir}/src/main/resources"
include '**/*'
}
task smallJar(type: Jar) {
appendix = 'small'
from "${rootDir}/src/main/resources/A/B"
include '**/*'
}
Using this rather than the code from the question produces 2 different jars.
It's not a complete answer but is a good enough work around, if I add a new publication definition I can publish the artifacts that I want to to the location that I want, the only downside is that it will create another gradle task which isn't ideal.
publications {
mypub(IvyPublication) {
artifact(ejbJar) {
name 'ejb'
}
}
newpub(IvyPublication) {
artifact(clientEjbJar) {
name 'client-ejb'
}
}
}
The above answer works in the short term, however does reveal yet another short coming in the Gradle world enter link description here
Not sure Gradle is all it could be at the moment, and so far no one has answered my questions so maybe it's not that actively developed!!
I'm no expert in this part of Gradle, but the functionality you are using is marked as "incubating"; you are using the new publishing feature which might or might not be complete. Perhaps you should use the old way of doing things. You also seem to be mixing both ways by using the artifacts closure.