I try to manage a user password files with bcrypt hashed passwords. The password hashes will be created via puppet on each run.
So now i have the problem that the file content of the user file is changed on each run.
My first idea was to manage a check file which contains a md5sum of the complete user hash to monitor any change on it. And i try to change the user file only if the check file was changed.
But this does not work. This is because puppet file resources doesn't react on refresh events i guess?
This was my last try with an exec as extra instance.
$md5sum = md5(String($input_users_hash))
file { "${filename}.serial" :
ensure => present,
content => $md5sum,
notify => Exec['md5sum_check']
}
exec { 'md5sum_check':
command => '/bin/true',
unless => "test `cat /opt/tomcat/shared/classes/users.properties.serial` == ${md5sum}",
notify => File[$filename],
}
file { $filename :
content => template($users_properties_template),
require => Exec['md5sum_check']
}
Any useful hints to solve this and to prevent a flapping resource?
In this case, my recommendation is that you store the bcrypt hashes in a hiera yaml on puppet server, and they are generated by something other than puppet (i.e. during the point where you create the new users).
If that isn't possible, and your avoiding directory service account management, then you need to find a bcrypt implementation that lets you choose set the salt. Then for each username, you can hash it, and use it to stablize the salt for a given user, which should stabilize the passwourd output.
Related
I have following problem with my Puppet installation:
I would like to copy (overwrite) a file only, if a new version of RPM package was installed.
I have something like this:
package { 'my-rpm':
ensure => $version,
source => $rpm_file,
notify => File[$my_file],
}
file { $my_file:
ensure => present,
source => $my_file_template,
replace => true, # Overwrite (default behaviour). Needed in case new RPM was installed.
}
The problem is, that the "file" get executed also, if no new version of RPM was installed. This happens, since I change the $my_file file afterwards using "file_line"
file_line { 'disable_my_service':
ensure => present,
path => $my_file,
line => ' <deployment name="My.jar" runtime-name="My.jar" enabled="false">',
match => ' <deployment name="My.jar" runtime-name="My.jar">',
}
This change of the content of the $my_file triggers copying fresh version from the template on each and every Puppet run.
I could add "repace => false" to my file copy define, but this would break any further updates...
The long story short: I have the following loop
Copy file -> change file -> copy file -> ...
How can I break this loop?
UPDATE:
Clarification:
The "file_line" define is executed optionally, controlled by a Puppet hiera-property and so the "enabled" part can't be included in the RPM.
The entire file can't be turned into a template (IMHO). The problem: Puppet module must be able to install different (future) versions of the file.
The problem remains unsolved for the time being.
I think the problem you're getting here is that you're trying to manage $my_file using both the file and file_line resource types and this is going to cause the file to change during the catalog application.
Pick one or the other, manage it as a template or by file line.
I suspect what's happening here during the Puppet run is the file resource changes $my_file to look like this;
<deployment name="My.jar" runtime-name="My.jar">
Because that's what is in the template then, the file_line resource changes it to;
<deployment name="My.jar" runtime-name="My.jar" enabled="false">
Then on the next run the exact same thing happens, file changes $my_file to match the template and then file_line changes it to modify that line.
I would remove the notify => File[$my_file], it's not actually doing anything, you're defining the desired state in code so if that file changes for any reason, manual change or RPM update, Puppet is going to bring that file back into the desired state during the run. You may want to consider;
file { $my_file:
ensure => present,
source => $my_file_template,
require => Package['my-rpm'],
}
This ensures the file desired state is enforced after the package resource so if the package changes the file the file will be corrected in the same run.
https://puppet.com/docs/puppet/7.4/lang_relationships.html
You may also want to consider;
file { $my_file:
ensure => present,
source => $my_file_template,
require => Package['my-rpm'],
notify => Service['my-service'],
}
So the service provided by the rpm restarts when the config file is changed.
Copy overwriting a file only on RPM update
The problem is, that the "file" get executed also, if no new version of RPM was installed. This happens, since I change the $my_file file afterwards using "file_line"
Yes, File resources in a node's catalog are applied on every run. In fact, it's best to take the view that every resource that makes it into in a node's catalog is applied on every run. Resources' attributes affect what applying them means and / or what it means for them to be in sync, not whether they are applied at all. In the case of File, for example, setting replace => false says that as long as the file initially exists, its content is in sync (and therefore should not be modified), whereas replace => true says that the file's content is in sync only if it is an exact match to the specified source or content.
Generally speaking, it does not work well to manage the same or overlapping physical resources via multiple Puppet resources, and that's what you're running into here. The most idiomatic approach when you run into a problem with that is often to write a custom resource type with which to manage the target object in detail. But in this case, it looks like you could work around the issue by using an Exec to perform the one-time post-update copy:
package { 'my-rpm':
ensure => $version,
source => $rpm_file,
}
~> exec { "Start with default ${my_file}":
command => "cp '${my_file_template}' '${my_file}'",
# this is important:
refreshonly => true,
}
-> file { $my_file:
ensure => 'file',
replace => false,
# no source or content
owner => 'root', # or whatever
group => 'root', # or whatever
mode => '0644',
# ...
}
-> file_line { 'disable_my_service':
ensure => present,
path => $my_file,
# ...
}
You can, of course, use relationship metaparameters instead of the chaining arrows if you prefer or have need.
That approach gives you:
management of the package via the package manager;
copying the packaged default file to the target file only when triggered by the package being updated (by Puppet -- you won't get this if the package is updated manually);
managing properties of the file other than its contents via the File resource; and
managing a specific line of the file's contents via the File_line resource.
How can I make the below logic work? My aim is to compare the value of custom fact $environment and the content of the file /etc/facter/facts.d/oldvalue.
If the custom fact $environment is not equal to the content of file /etc/facter/facts.d/oldvalue, then execute the following code.
exec {'catenvchange' :
command => "/bin/cat /root/oldvalue"}
if $environment != exec['catenvchange'] {#code#}
Exec resources do not work that way. In fact, no resource works that way, or any way remotely like that. Moreover, the directory /etc/facter/facts.d/ serves a special purpose, and your expectation for how it might be appropriate to use a file within is not consistent with that purpose.
What you describe wanting to do looks vaguely like setting up an external fact and testing its value. If you drop an executable script named /etc/facter/facts.d/anything by some means (manually, plugin sync, File resource, ...) then that script will be executed before each Puppet run as part of the process of gathering node facts. The standard output generated by the script would be parsed for key=value pairs, each defining a fact name and its value. The facts so designated, such as one named "last_environment" will be available during catalog building. You could then use it like so:
if $::environment != $::last_environment {
# ...
}
Update:
One way to use this mechanism to memorialize the value that a given fact, say $::environment, has on one run so that it can be read back on the next run would be to declare a File resource managing an external fact script. For example,
file { '/etc/facter/facts.d/oldvalues':
ensure => 'file',
owner => 'root',
group => 'root',
mode => '0755',
content => "#!/bin/bash\necho 'last_environment=${::environment}'\n"
}
I'm concerned about security of database passwords in sqoop batches (no interactive input).
In the old days, for a sqoop batch, the only thing you could do was to pass it on the command line using --password, but then the password was easy to read with a simple ps command.
Now we have that --password-file option, but it requires to store the password unencrypted on the disk and that's not really a "secure" practice, nor is it very convenient to have individual files for individual parameters.
I was thinking of storing the encrypted password in a configuration file, and dynamically decrypt it, store it in a temporary file, setting the rights (using a chmod command), calling sqoop, and then deleting the file... But I may miss a less cumbersome way ? How do you deal with it ?
#abeaamase has the best answer at this time in his stackoverflow 23916985 response from 3/2015 found here. Essentially, we should upgrade to sqoop > 1.4.5 and use a java keystore (JKES), org.apache.sqoop.util.password.CryptoFileLoader, or a loader defined in our own class.
The provided CryptoFileLoader has the disadvantage of presuming the crypto passphrase and salt will be provided as -D parameters to drive system properties (which are open to snooping via ps) or in plain text in the configuration XML.
I initially discovered these parameters in this blog from 3/2015 having missed it earlier (it lacks a heading but you'll find it if you look at step 3 part 2).
Surprisingly, it is not the recommended practice, and does not appear in the sqoop 1.4.5 docs.
Before the availibilty of the --password-file option, I made a patch for sqoop to read the password from in input stream in a non interactive way when using the -P command.
That way, I could unencrypt the password from a configuration file, and call sqoop with that password using a stdin pipe, without using a file or a command line where the plain password could be seen.
Edit file src/java/org/apache/sqoop/SqoopOptions.java
Replace the securePasswordEntry function code by
private String securePasswordEntry() {
try {
return new String(System.console().readPassword("Enter password: "));
}
// PATCH Bouygues Telecom - read password from pipe if launched in non-interactive mode
catch (NullPointerException e) {
try {
final java.io.BufferedReader reader = new java.io.BufferedReader(
new java.io.InputStreamReader(System.in));
return reader.readLine();
}
catch (java.io.IOException excep) {
LOG.error("It seems that you have launched a Sqoop metastore job via");
LOG.error("Oozie with sqoop.metastore.client.record.password disabled.");
LOG.error("But this configuration is not supported because Sqoop can't");
LOG.error("prompt the user to enter the password while being executed");
LOG.error("as Oozie tasks. Please enable sqoop.metastore.client.record");
LOG.error(".password in sqoop-site.xml, or provide the password");
LOG.error("explicitly using --password in the command tag of the Oozie");
LOG.error("workflow file.");
}
return null;
}
}
What is cumbersome is to have to re-patch every new release of Sqoop... I should maybe submit a jira (with a low confidence that my patch will be taken into account), or move to the --password-file option the way you wanted to.
I want to create a user for a service (postgres, rabbitmq...) using a random generated password. This password should then be written to a file on the host. This file, containing env vars is then used by an application to connect to those services.
I don't want to store these passwords elsewhere.
postgresql::server::db { $name:
user => $name,
password => postgresql_password($name, random_password(10)),
}
Then i want to insert this password in the form PG_PASS='the same password' into a config file but the whole thing should happen only if the user is not already present.
In pure Puppet
A trick is to define a custom type somehow like :
define authfile($length=24,$template,$path) {
$passwordfile = "/etc/puppet/private/${::hostname}/${::title}"
$password = file($passwordfile,'/dev/null')
##exec { "generate-${title}":
command => "openssl rand -hex -out '$passwordfile' 24",
creates => $passwordfile,
tag => 'generated_password'
}
file { $path:
content => template($template);
}
}
And on your puppetmaster, have something like :
Exec<|| tag = 'generated_password' ||>
You can then pass in the $template variable the name of a template that will have the variable $password available. You will need to be careful about how these authfile types are defined (as it creates files on the puppetmaster, you will want to take care of malicious facts), and you will need to run puppet once on the host (so that the exported resources is created), once on the puppetmaster (so that the secret file is generated), then once again on the host (so that the secret file is read) for it to work.
With a custom function
Another solution is to write a custom function, random_password, that will use the fqdn and sign it with a secret that is stored on the puppetmaster (using a HMAC) to seed the password. That way you will have a password that can't be guessed without getting the secret, and no extra puppet roundtrips.
I haven't tried it myself yet. But trocla looks exactly like what you're looking for. Here's a little intro.
EDIT: After having now tried out trocla I can tell you that it works like a charm :-)
You need to have the trocla module installed to use it from puppet.
I know I can sync directory in Puppet:
file { 'sqls-store':
path => '/some/dir/',
ensure => directory,
source => "puppet:///modules/m1/db-updates",
recurse => true,
purge => true
}
So when the new files are added they are copied to '/some/dir/'. However what I need is to perform some action for every new file. If I "Subscribe" to such resource, I don't get an array of new files.
Currently I created external shell script which finds new files in that dir and executes action for each of them.
Naturally, I would prefer not to depend on external script. Is there a way to do that with Puppet?
Thanks!
The use case for that is applying changes to DB schema that are being made from time to time and should be applied to all clients managed by puppet. In the end it's mysql [args] < update.sql for every such file.
Not sure I would recommend to have puppet applying the db changes for me.
For small db, it may work but for real world db... you want to be aware of when and how these kind of changes got applied (ordering of the changes, sometime require temp disk space adjustement, db downtime, taking backup before/after, reorg,...), most of the times your app should be adapted at the same time. You want more orchestration (and puppet isn't good at orchestration)
Why not using a tool dedicated to this task like
liquid-base
rails db migrations and capistrano
...
A poor men solution would be to use vcs-repo module and an exec to list modified files since last "apply".
I agree with mestachs, puppet dealing with db updates it's not a great idea
You can try some kind of define:
define mydangerousdbupdate($name, $filename){
file { "/some/dir/$filename":
ensure => present,
source => "puppet:///modules/m1/db-updates/$filename",
}
exec{"apply $name":
command => "/usr/bin/mysql [args] < /some/dir/$filename > /some/dir/$filename.log",
creates => "/some/dir/$filename.log"
}
}
And then, you can instantiate with the different patches, in the preferred order
mydangerousdbupdate{"first_change":
name => "first",
filename => "first.sql",
}->mydangerousdbupdate{"second_change":
name => "second",
filename => "second.sql",
}