I am modifying a file utilizing puppet file_line resource but each time puppet runs, its triggering a refresh even though no other change has been made after the first puppet run.
file_line { 'log_format_combined':
ensure => present,
path => '/etc/apache2/apache2.conf',
line => 'LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined',
match => '^LogFormat "%h %l %u %t ."%r." %>s %b ."%{Referer}i." ."%{User-Agent}i."" combined',
}
What can I do to only trigger a refresh if a change is made to the file?
I tested your code and it works perfectly, and the file_line makes sure the code is executed only once if no change has made.
So when I run puppet for the first time I get this
Notice: /Stage[main]/Main/Node[default]/File_line[log_format_combined]/ensure: created
and then nothing when I run the puppet agent again.
Are you sure your file hasn't changed after the puppet agent run ? don't you have another puppet resource that change that file ?
I used the same code to test in my setup ,there is no replacement happening after the first run, same with the refresh too.I suspect the refresh would be the outcome of some other code snippet.
Related
I have a sshd_config configuration file which contains commented as well as uncommented lines. I want to comment all the uncommented lines in that file using puppet. Is there any optimal/simple way to do this? Or is there a way to run bash command (maybe sed to replace) via puppet? I am not sure that using bash command is a right approach.
It would be really helpful is someone guides me with this. Thanks in advance!
Is there any optimal/simple way to do this?
There is no built-in resource type or well-known module that specifically ensures that non-blank lines of a file start with a # character.
Or is there a way to run bash command (maybe sed to replace) via puppet?
Yes, the Exec resource type. That's your best bet short of writing a custom resource type.
I am not sure that using bash command is a right approach.
In a general sense, it's not. Appropriate, specific resource types are better than Exec. But when you don't have a suitable one and can't be bothered to make one, Exec is available.
It might look like this:
# The file to work with, so that we don't have to repeat ourselves
$target_file = '/etc/ssh/sshd_config'
exec { "Comment uncommented ${target_file} lines":
# Specifying the command in array form avoids complicated quoting or any
# risk of Puppet word-splitting the command incorrectly
command => ['sed', '-i', '-e', '/^[[:space:]]*[^#]/ s/^/# /', $target_file],
# If we didn't specify a search path then we would need to use fully-qualified
# command names in 'command' above and 'onlyif' below
path => ['/bin', '/usr/bin', '/sbin', '/usr/sbin'],
# The file needs to be modified only if it contains any non-blank, uncommented
# lines. Testing that via an 'onlyif' ensures that Puppet will not
# run 'sed' or (more importantly) report the file changed when it does
# not initially contain any lines that need to be commented
onlyif => [['grep', '-q', '^[[:space:]]*[^#]', $target_file]],
# This is the default provider for any target node where the rest of this
# resource would work anyway. Specifying it explicitly will lead to a more
# informative diagnostic if there is an attempt to apply this resource to
# a system to which it is unsuited.
provider => 'posix',
}
That does not rely on bash or any other shell to run the commands, but it does rely on sed and grep being available in one of the specified directories. In fact, it relies specifically on GNU sed or one that supports an -i option with the same semantics. Notably, that does not include BSD-style sed, such as you will find on macOS.
How do I set file permission from list of file names from cat command?
For example, below command returns 3 file names:
$ cat /tmp/test | grep file
/etc/systemd/file_1.log
/etc/systemd/file_2.log
/etc/systemd/file_3.log
How do I use puppet to run the command, get the file names and then loop the 3 file names and set permission accordingly?
The files are resources and if you want to manage a resource you have to know it's there so dynamically created log files are not easy. If you know the file names already then you can use something like this and pass an array into the file resource.
file { ['/etc/systemd/file_1.log',
'/etc/systemd/file_1.log',
'/etc/systemd/file_1.log'] :
ensure => 'file',
mode => '0644',
owner => 'root',
group => 'root',
}
An other method might be to use an exec
exec { 'chmod 644 /etc/systemd/file_*.log':
path => ['/usr/bin', '/usr/sbin',],
}
But you really need something like an onlyif or unless or this is going to execute every 30 minutes and that breaks the idempotent rule we try and apply with Puppet code where things only change if they need correcting. So you're going to need a command line that'll test the permissions and return a boolean to the onlyif.
There are more details here https://puppet.com/docs/puppet/5.5/types/exec.html
A alternative (and the way I'd do it) would be to expose the contents of that file via an external fact which passes the list of files to Puppet to use in the catalog compilation. An external fact can be a bash script so I'd create a file called /etc/facter/facts.d/logfiles.sh, obviously I'd deploy this using Puppet.
#!/usr/bin/env bash
logfiles=($(grep file /tmp/test))
echo "logfiles=${logfiles[*]}"
Then in my Puppet code I'd have something like this;
$logfiles.each |String $logfile| {
file { $logfile :
ensure => 'file',
mode => '0644',
owner => 'root',
group => 'root',
}
}
So when the Puppet run happens the list of log files will be returned to Puppet via the facts and then each file listed is defined as a resource with the correct permissions.
How do I set file permission from list of file names from cat command?
There are two main alternatives, but I observe first that your example is of the output from grep, not cat, and that the cat in that example is superfluous. Nevertheless, those details don't change the big picture -- substantially the same approaches are applicable for data output by any command.
It would be more idiomatic to write a custom fact that captures the filenames (as of the time of each catalog request), and use that information to create the appropriate File resources.
Custom facts are not that hard, but the full details are more than would be appropriate for an SO answer. Supposing that you have a fact $facts['systemd_logs'] whose value is an array of the absolute filenames, you can compactly express the whole group of wanted File resources like so:
file { $facts['systemd_logs']:
mode => '0644',
}
(or whatever mode it is that you want).
It would be quicker (and dirtier) to use an Exec resource to run an appropriate command:
exec { 'ensure correct file permissions':
command => 'chmod 0644 $(/bin/grep file /tmp/test)',
onlyif => '/bin/grep -q file /tmp/test',
provider => 'shell',
}
My application requires the core file to be generated in a specific pattern.
How do I do this without affecting other processes?
And how do I do this when /proc is read-only?
man core tells us:
Piping core dumps to a program
Since kernel 2.6.19, Linux supports an alternate syntax for the
/proc/sys/kernel/core_pattern file. If the first character of this
file is a pipe symbol (|), then the remainder of the line is
interpreted as a program to be executed. Instead of being written to
a disk file, the core dump is given as standard input to the program.
Note the following points:
The program must be specified using an absolute pathname (or a
pathname relative to the root directory, /), and must immediately
follow the '|' character.
The process created to run the program runs as user and group
root.
Command-line arguments can be supplied to the program (since Linux
2.6.24), delimited by white space (up to a total line length of
128 bytes).
The command-line arguments can include any of the % specifiers
listed above. For example, to pass the PID of the process that is
being dumped, specify %p in an argument.
You can put a script there, like e.g.
| /path/to/myscript %p %s %c
You can detect which process is triggering the coredump: (man core):
%% a single % character
%p PID of dumped process
%u (numeric) real UID of dumped process
%g (numeric) real GID of dumped process
%s number of signal causing dump
%t time of dump, expressed as seconds since the Epoch, 1970-01-01
00:00:00 +0000 (UTC)
%h hostname (same as nodename returned by uname(2))
%e executable filename (without path prefix)
%E pathname of executable, with slashes ('/') replaced by exclama‐
tion marks ('!').
%c core file size soft resource limit of crashing process (since
Linux 2.6.24)
Now all you have to do is "do the default thing" for other processes than your own
I'm newbie on linux. I'm so sorry for asking this question again. But I am really appreciate if someone could help me on this. I have trouble on how to convert my linux log to csv file for more readable.
I have apache log as bellow:
[Sun Mar 01 06:01:30 2015] [error] [client 123.456.789.012] File does not exist: /var/www/html/
How can I separate them by column, using: Date (Sun Mar 01 06:01:30 2015), IP (123.456.789.012) only IP, Error Message (File does not exist) and Target (/var/www/html/)?
Thank you
There are many ways t achieve it in shell script. Will describe the method in detail and will give a sample example.
You have to identify the delimiter to partition your string and either you can use awk or sed command to partition the fields according to the delimiter
For example in you case you can consider ']' as delimiter s to break the line using the delimiter command will be as follows:
cat logfile | awk -F']' '{print "$1, $2, $3"}' > new_log_file.csv
The easiest way would be to use your own Logformat string. You can modify the standard LogFormat to use TAB instead of space as separator. The standard or Common Log Format usually named combined LogFormat looks like this:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
If you want a TAB separated file simply use this CustomLog statement in your server configuration file:
Customlog logs/tabbed-logfile "%h\t%l\t%u\t%t\t\"%r\"\t%>s\t%b\t\"%{Referer}i\"\t\"%{User-Agent}i\""
I'm frequently eye-balling apache and jboss logs on Linux with "less" and "tail -f" and would like to have particular lines that match a string to be highlighted in a color of my choice. Is there a way to do this?
I am typically connected via ssh from an MS-DOS command window.
Edit: Preferably, the solution would not modify the log file itself.
Don't have access to a terminal right now, but you can try this,
In Apache config file when you define the LogFormat, try using shell color codes. E.g.,
LogFormat "${START}%h %l %u %t \"%r\" %>s %b ${END}" common_color
where
START="\e[1;34m" # for a blue text, you can use other colors as well
END="\e[0m"
This should work in shell terminals