Override default cookbook chef variables - apache-spark

I am looking for specific instructions on how to override default values in a third party cookbook. For example, i am using apache_spark cookbook (https://github.com/clearstorydata-cookbooks/apache_spark)
And i want to override the attribute
default['apache_spark']['standalone']['master_host']
I tried it my making a main recipe, in which i add
node.default['apache_spark']['standalone']['master_host'] = 'foo.com'
And execute it using chef solo like:
run_list(
'recipe[main]',
'recipe[apache_spark::spark-standalone-worker]'
)
But that does not seem to work. Any suggestions on how it needs to be done?
My main recipe is here https://github.com/Vibhuti/chef-main

The correct fix would be to make a wrapper cookbook and set your values in the cookbook's attributes file (main/attributes/default.rb):
override['apache_spark']['standalone']['master_host'] = 'foo.com'
Also make sure you add a dependency in main's metadata.rb to force the load ordering to be correct:
depends 'apache_spark'

Related

Terraform conditional source in MODULE

I am trying to set a module's source (this IS NOT a resource) based on a conditional trigger but it looks like the module is getting fired before the logic is applied:
module "my_module" {
source = "${var.my_field == "" ? var.standard_repo : var.custom_repo}"
stuff...
more stuff...
}
I have created the standard_repo and custom_repo vars as well and defined with URLs for respective repos (using git:: -- this all works w/o conditional)
All this being said, anyone know of a way to implement this conditional aspect? (again, this is a module and not a resource)
I tried using duplicate modules and calling based off the var value but this, too, does not work (condition is never met, even when it is):
repo = ["${var.my_field == "na" ? module.my_module_old : module.my_module_new}"]
One way to achieve this is described in this post
Basically, a common pattern is to have several folders for different environments such as dev/tst/prd. These environments often reuse large parts of the codebase. Some may be abstracted as modules, but there is still often a large common file which is either copy-pasted or symlinked.
The post offers a way that doesn't conditionally disable based on variables but it probably solves your issue of enabling a module based on different enviornments. It makes use of the override feature of terraform and adds a infra_override.tf file. Here, it defines a different source for the module which points to an empty directory. Voila, a disabled module.
Variables are not allowed to be used in the module source parameter. There also does not seem to be a plan for this to change. https://github.com/hashicorp/terraform/issues/1439 . Creating a wrapper script , or using something like mustache http://mustache.github.io/ seems to be the best way to solve the problem.

Jenkins Extended Choice Parameter Plugin Groovy Script

So in the website for the plugin, the change-log for Version 0.61 (Mar 30, 2016) mentions that they have exposed Jenkins and project bindings for use in a groovy script. I'm currently trying to set a parameter (multi select) and i want it to have different options based on the value of another parameter which is set before this one in the Jenkins job. However i'm not able to get the value of that other parameter in the script.
My problem is very similar to the one explained here
I've tried using ${param}, $(param), $param, bindings.variables.get('param') in the script, and none of it works
I've also tried to set that parameter in the Variable bindings section of the script but still no success.
Any ideas about the correct way to obtain the value?
I set the variable binding as param on the other field that you want to dynamically populate and used param (please note that i didnt use $ ${} or binding.get etc). It worked for me.
As a side note, Will active choice reactive parameter work in your case?

Can we merge JAVA_OPTS variables through recipe in chef?

I have installed Java through some cookbook and have set some default variables, now I want to add some more variables (Application specific) through my cookbook. How can I do that through Recipes in Chef. I tried to pass some variables in setenv.sh but it is overriding the default values instead I want to merge the variables and override existing variable values.
My code in setenv.sh:
export JAVA_OPTS="$JAVA_OPTS -Xmx2048m"
where $JAVA_OPTS - default variables
First way to do it would be to update the attribute defining the bases values in your application cookbook, as the attributes are read before the recipes are evaluated your file would end up with the correct values.
You're not saying with which cookbook you're using so I'll base the example on the tomcat cookbook
This cookbook define an attribute default['tomcat']['java_options'] = '-Xmx128M -Djava.awt.headless=true'
The easiest way it to complement this by using something like
default['tomcat']['java_options'] = "#{default['tomcat']['java_options']} -Xmx2048m"
The obvious problem is that you end up with 2 -Xmx values, usually the JVM will take the latest but it become hard to find what options is at which value when there's a lot of overwriting.
Second option is to take advantage of jvmargs cookbook wich gives helpers functions to deine the java options and use in your setenv.sh template at end.

Chef wrapper cookbooks only apply internal cookbook once

I have a cookbook "blah-deploy-nodejs-from-git" cookbook that installs a nodejs codebase from GIT and calls NPM install on the directory. It has the following attributes
git_repo
branch
destination
I have then written cookbooks that wrap that cookook for inidividual sites, that need to get installed. In this particar case "blah-pricing" and "blah-notifications" which have different overriding attributes:
me#me cat cookbooks/blah-svc-pricing/attributes/default.rb
node.override[:blah_deploy_nodejs_from_git][:destination] = "/var/blah/pricing"
node.override[:blah_deploy_nodejs_from_git][:branch] = "master"
node.override[:blah_deploy_nodejs_from_git][:git_repo] = "https://hqdevgit01.blah.lan/micro-services/blah-pricing.git"
me#me:~/chef-repo$ cat cookbooks/blah-svc-notifications/attributes/default.rb
node.override[:blah_deploy_nodejs_from_git][:destination] = "/var/blah/notifications"
node.override[:blah_deploy_nodejs_from_git][:branch] = "master"
node.override[:blah_deploy_nodejs_from_git][:git_repo] = "https://hqdevgit01.blah.lan/micro-services/blah-notifications.git"
And then the recipe is the same in both cases:
include_recipe 'blah-deploy-nodejs-from-git'
Unfortunately it is applying the inner recipe only once even though my node has both cookbooks applied to it. My understanding was that wrapper cookbooks are used to customize a cookbook and make it unique.
Can encapsulate the inner cookbook to two different cookbooks, with different attributes, and have the wrapper cookbooks both apply that inner recipe? OR Am I going to have to completely replicate the code that is in the inner cookbook?
This is due to a basic misunderstanding of how chef works. Recipes are not meant to be a procedure for how to do something, they are meant to be a declaration of what that something should look like. As such, you need to think of them as describing the end state, not the process for getting there.
Thus, chef will never run a recipe twice. And attributes really should not be changed mid run (unless they are updated to indicate something that happens mid run. Luckily, there are other chef capabilities that can solve your problem. You need either a definition or an LWRP (light weight resource provider)
Definitions are just groups of resources that are often repeated. So you can create a definition and then later call it multiple times in the same recipe with different attributes. Much like what you currently are doing with your recipe.
While definitions are sometimes appropriate, LWRPs are generally more powerful, and have become the prefered approach for most repetitive (library like) task in Chef. With LWRPs you will define a new chef primitive (much like file, service, etc), and then write the code for accomplishing the goal of that primitive. You can then use these resources anywhere in your recipes. In your case, you'd have an npm_deployer resource that took attributes for repo, branch, and destination. It would then do the work that is currently in your deployer recipe. Your "wrapper" recipes would then stop calling include_recipe and instead just declare a npm_deploy resource and pass in the attributes needed.

Is GroovyResourceLoader ever called outside the main ClassLoader?

Ive setup a GroovyResourceLoader and it seems to get requests for groovy scripts as necessary. I was just wondering is it specially used anywhere besides class loading ? Is there any benefit in simply wrapping a ClassLoader and loading *.groovy files there as opposed to using a GRL ? Are they just different ways to the same end ?
GroovyResourceLoader (GRL) is used by GroovyClassLoader (GCL) and at least since Groovy 1.8 indirectly through GroovyScriptEngine (GSE). But GSE loads it through a GCL as well.
But what GRL does is to "locate" the scripts and return an URL to the location. What GCL does is use the URL returned by a GRL to get the source and compile it to create the class, which is then available for loading.
GRL is a backend for GCL. So they are not different ways to the same end. True, you still have to do more things to actually execute the script code (unless it is precompiled), but "get the script source, compile it, make a Class out of it and finally execute it" are the steps you always have to do. In our GRL/GCL discussion, GRL does part of the first step, GCL itself does the third step. Step 2 is done by CompilationUnit inside GCL and the last step is yours to be done. There are other ways to complete these steps of course, but that's out of the scope for this discussion.

Resources