We have FOSS Puppet 3.8.1 running a master and at some point it started responding 500 to our clients at their request for catalog.
Having absolutely no idea what's wrong, I started searching for some logs and found nothing.
/etc/puppet/puppet.conf on master looks like
[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=/lib/facter
pluginsync=true
[agent]
server=our-app.com
certname=cert-name
There is a /var/log/puppet folder, but it is empty.
Can someone help with finding where the errors are logged?
On the puppetmaster, run this command
puppet config print |grep log
On my system this comes up with various useful looking paths like
masterhttplog = /var/log/puppet/masterhttp.log
httplog = /var/log/puppet/http.log
railslog = /var/log/puppet/rails.log
masterlog = /var/log/puppet/puppetmaster.log
puppetdlog = /var/log/puppet/puppetd.log
also for debugging the relationship of the puppetmaster and agents and for errors in manifest files the reports are useful
puppet config print |grep report
Note that the puppetmaster itself can also be a agent so there could well be stuff like a last_run_report.yaml which is concerning the puppetmaster configuring itself, but you maybe want to skip these
On redhat clones at least, the logging is configured in /etc/sysconfig/puppet:
PUPPET_LOG=/var/log/puppet/puppet.log
or
PUPPET_LOG=syslog
If you select syslog it will send the messages to /dev/log, from where they are picked up by your logging daemon (rsyslog), which will write the lines most probably in /var/log/messages.
The simplest way is to use a file instead of syslog.
Related
I have installed Snort and Barnyard2 following this guide.
I was about to install BASE but it requires PHP5 and it's no longer supported. I have PHP7 installed and cannot downgrade it.
After a bit of lurking I decided to use Graylog2 to view the logs.
Snort is configured to log in unified2 format then barnyard2 reads that and saves it to MySQL database.
As far as I understood (not much) logging to MySQL is pointless without BASE and I need to forward the logs to Graylog.
Now, should I
1) Remove Barnyard and tell Snort to log in human-readable format (not very efficient?) then send the logs to Graylog
2a) Tell Barnyard to stop sending the logs to MySQL but forward them directly to Greylog
2b) Tell Barnyard to just translate unified2 and write the human-readable log to another file then send it to Graylog.
In any case, How?
Snort, Barnyard and Graylog are on the same machine.
So, I've come up with this but I would like an opinion from someone who knows more than me.
Remove this from barnyard2 config file (example - Line 258 to 296)
output database: log, mysql, user=snort password=************** dbname=snort host=localhost sensor_name=sensor01
and add the proper output for syslog
output alert_syslog
(OR output log_syslog_full: sensor_name whatever, server 127.0.0.1, protocol udp, port 514 ?)
Then in syslog config (graylog documentation, here)
*.* #graylog.example.org:514;RSYSLOG_SyslogProtocol23Format
You can also configure barnyard to log directly to your Graylog server. Instead of logging via syslog to localhost, you just create a new syslog input on your Graylog server and point the barnyard syslog output to that ip/port.
We have several servers working with puppet as agents today, but I'm having a problem with a new server running CentOS 7. Normally I would update the /etc/sysconfig/puppet file with the puppet master name and then start the daemon and move to signing the certificate on the master. However, puppet agent doesn't appear to be reading the server = myhost.domain in my config file.
I get the following error in /var/log/messages:
puppet-agent[11133]: Could not request certificate: getaddrinfo: Name or service not known
I tried:
myserver:root$ puppet agent --configprint server
puppet
myserver:root$
but the /etc/sysconfig/puppet file has:
PUPPET_SERVER=myserver.domain.com
Can you please help me understand why puppet agent doesn't get the server from the config file?
The /etc/sysconfig/puppet file is not typically read by the Puppet agent. (I'm not very familiar with CentOS operations, but I suppose that this location might hold some settings that are external to the process, such as environment, command line switches etc.)
You will want to use the proper puppet configuration file:
/etc/puppet/puppet.conf for Puppet 3.x and earlier
/etc/puppetlabs/puppet.conf for Puppet 4.x
so ran the following:
"puppet agent --no-daemonize --verbose --onetime --server puppetmaster.xxx.com"
this started puppet properly, requested certificate and I was able to sign on master. Then added:
server = puppetmaster.xxx.com
to /etc/puppet/puppet.conf and "systemctl restart puppet"
and it worked. Thanks for posts here and other places.
I have an Rsyslog central server where multiple machine send log files and these log files are stored by machine IP.
$template DailyPerHostLogs,"/var/log/remote/%fromhost-ip%_%$YEAR%-%$MONTH%-%$DAY%.log"
*.* -?DailyPerHostLogs
This works fine however all the logs gets into the logmachines central logs as well like:
/var/log/messages
/var/log/auth
/var/log/cron
What is the best way to stop this from happening?
Thanks
Solved it, have to put the remote log acception rule first then this:
## before going to local log rules, drop remote logging, it's been
## processed in the "central logging" section
#
:hostname, !isequal, "biglogserver" ~
Then rest of the rsyslog.conf
If I understood your question right, you won't save the logs locally on the machines.
To do that you must remove from /etc/syslog.conf all the lines that describe local log files, and leave there only the line that send the logs remotely.
I'm new to puppet, but picking it up quickly. Today, I'm running into an issue when trying to run the following:
$ puppet agent --no-daemonize --verbose --onetime
**err: Could not request certificate: getaddrinfo: Name or service not known
Exiting; failed to retrieve certificate and waitforcert is disabled**
It would appear the agent doesn't know what server to connect to. I could just specify --server on the command line, but that will be of no use to me when this runs as a daemon in production, so instead, I specify the server name in /etc/puppet/puppet.conf like so:
[main]
server = puppet.<my domain>
I do have a DNS entry for puppet.<my domain> and if I dig puppet.<my domain>, I see that the name resolves correctly.
All puppet documentation I have read states that the agent tries to connect to a puppet master at puppet by default and your options are host file trickery or do the right thing, create a CNAME in DNS, and edit the puppet.conf accordingly, which I have done.
So what am I missing? Any help is greatly appreciated!
D'oh! Need to sudo to do this! Then everything works.
I had to use the --server flag:
sudo puppet agent --server=puppet.example.org
I actually had the same error but I was using the two learning puppet vm and trying run the 'puppet agent --test' command.
I solved the problem by opening the file /etc/hosts on both the master and the agent vm and the line
***.***.***.*** learn.localdomain learn puppet.localdomain puppet
The ip address (the asterisks) was originally some random number. I had to change this number on both vm so that it was the ip address of the master node.
So I guess for experienced users my advice is to check the /etc/hosts file to make sure that the ip addresses in here for the master and agent not only match but are the same as the ip address of the master.
for other noobs like me my advice is to read the documentation more clearly. This was a step in the 'setting up an agent vm' process the I totally missed xD
In my case I was getting same error but it was due to the cert which should been signed to node on puppetmaster server.
to check pending certs run following:
puppet cert list
"node.domain.com" (SHA256) 8D:E5:8A:2*******"
sign the cert to node:
puppet cert sign node.domain.com
Had the same issue today on puppet 2.6 on CentOS 6.4
All I did to resolve the issue was to check the usual stuff such as hosts and resolv.conf to ensure they were as expected (compared with a working server) and then;
Removed /var/lib/puppet directory rm -rf /var/lib/puppet
Cleared the certificate on the puppet master puppetca --clean
servername
Restarted the network service network restart
Re-ran puppet
Even though the resolv.conf was identical to the working server, puppet updated resolv.conf and immediately re-signed the certificate and replaced all the puppet lib files.
Everything was fine after that.
Adding log4j [1] in tomcat 6.0.x forces tomcat to produce logs in "catalina" file. However, the default catalina.out is still produced and populated with logs. So, questions:
Is it safe to delete catalina.out file (while server running)?
If yes, could this deletion be added to tomcat startup script? If yes, could anyone please point out the file and the required script?
Is it possible for tomcat to stop createing the catalina.out, since it is not nessesary anymore?
http://tomcat.apache.org/tomcat-6.0-doc/logging.html
Thanks in advance people!
Tomcat redirects its stdout and stderr to catalina.out. So direct out/err writes and log4j ConsoleAppender messages will go to catalina.out. See catalina.sh file for details. To disable it completely you can redirect stdout and stderr to /dev/null setting CATALINA_OUT environment variable:
export CATALINA_OUT=/dev/null
But I recommend to disable ConsoleAppender instead to reduce catalina.out size and monitor it periodically looking for error messages, that may go to stdout bypassing log4j.
If the catalina.out is deleted after tomcat is stopped, it will create a new catalina.out once tomcat starts again and it is totally safe.
But if you remove the catalina.out while tomcat is running, it will keep on logging to catalina.out which is removed already (reference of the file is hold by the tomcat) hence the space will not be released. So you will need to restart the tomcat sever to release the space. It is not recommended.
You can try following to disable writing to catalina.out :
Locate and Edit File: {CATALINA_BASE}/bin/catalina.sh
Locate "CATALINA_OUT" and replace the path with "/dev/null":
CATALINA_OUT="$CATALINA_BASE"/logs/catalina.out // Original Location
CATALINA_OUT=/dev/null // replace path to /dev/null
Is there a reason you'd want to delete the catalina.out file? It seems like it might lead to potentially missing important event messages. Perhaps consider just setting
org.apache.catalina.level=INFO
Otherwise I can't think of a reason that it would negatively impact the functionality of the container if you deleted it. On a *nix install its still writing to the file descriptor of a file that is diconnected from an inode(so otherwise unreachable) and on windows it won't let you delete it because the container will have a file lock.
logging docs - a quick rtfm and it looks like you should just be able to remove the handlers from logging.properties to discontinue producing this file