Puppet clients don't update - puppet

We are running puppet 2.7.11-1ubuntu2.4 (Ubuntu 12.04) on our clients and master. The clients don't seem to update automatically, but when I run:
sudo puppet agent --test
Everything works fine.
Current running processes on the client:
root 1764 1 0 Sep10 ? 00:00:05 /usr/bin/ruby1.8 /usr/bin/puppet agent
/etc/puppet/puppet.conf
[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
templatedir=$confdir/templates
prerun_command=/etc/puppet/etckeeper-commit-pre
postrun_command=/etc/puppet/etckeeper-commit-post
pluginsync=true
[master]
# These are needed when the puppetmaster is run by passenger
# and can safely be removed if webrick is used.
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY
[agent]
server=<URL_REMOVED>
configtimeout=300
/var/log/syslog.log
Sep 11 16:12:48 <HOSTNAME_REMOVED> puppet-agent[1764]: Did not receive certificate
Sep 11 16:14:48 <HOSTNAME_REMOVED> puppet-agent[1764]: Did not receive certificate
Sep 11 16:16:49 <HOSTNAME_REMOVED> puppet-agent[1764]: Did not receive certificate
Sep 11 16:18:49 <HOSTNAME_REMOVED> puppet-agent[1764]: Did not receive certificate
Sep 11 16:20:49 <HOSTNAME_REMOVED> puppet-agent[1764]: Did not receive certificate
/etc/default/puppet
# Defaults for puppet - sourced by /etc/init.d/puppet
# Start puppet on boot?
START=yes
# Startup options
DAEMON_OPTS=""
Does someone have an idea what could be wrong?

We actually recently found the cause of this problem.
Some nodes had a hostname in their puppet.conf that didn't match the hostname in the certificate of the server.
Also some nodes didn't use their FQDN when they contacted the server, which caused mismatches with the client certificates. We fixed that by adding the FQDN to /etc/hosts:
127.0.1.1 hostename.domain.edu hostename

Take a look at this Troubleshooting page. Not sure about your problem exactly, but I saw similar errors in my log: "Did not receive certificate". In my case these steps have helped me:
on master run
puppet cert clean <NODE NAME>
on agent:
rm -rf $(puppet agent --configprint ssldir)
puppet agent --test

Related

openLDAP Proxy fails with ldaps

This issue is driving me a bit insane.
I am trying to configure an openLDAP Proxy to an Active Directory, which works fine as long as I use unencrypted ldap to the AD. But I would like to secure the connection between the proxy and the AD via LDAPs.
What drives me crazy is that this worked before until I enabled LDAPs for the proxy itself. Now I can't get it running, even if I undo my changes.
This is how the slapd.conf looks like:
# Include schemas
include /etc/openldap/schema/corba.schema
include /etc/openldap/schema/core.schema
include /etc/openldap/schema/cosine.schema
include /etc/openldap/schema/duaconf.schema
include /etc/openldap/schema/dyngroup.schema
include /etc/openldap/schema/inetorgperson.schema
include /etc/openldap/schema/java.schema
include /etc/openldap/schema/misc.schema
include /etc/openldap/schema/nis.schema
include /etc/openldap/schema/openldap.schema
include /etc/openldap/schema/ppolicy.schema
include /etc/openldap/schema/collective.schema
include /etc/openldap/schema/local.schema
allow bind_v2
pidfile /var/run/openldap/slapd.pid
argsfile /var/run/openldap/slapd.args
modulepath /usr/lib/openldap
modulepath /usr/lib64/openldap
moduleload back_ldap
moduleload rwm
loglevel 256
#LDAPS Settings
TLSCipherSuite HIGH:MEDIUM:-SSLv3
TLSCertificateFile /etc/openldap/cacerts/server.crt
TLSCertificateKeyFile /etc/openldap/cacerts/server.key
TLSCACertificateFile /etc/openldap/cacerts/CAs.pem
### Database definition (Proxy to AD) #########################################
database ldap
readonly no
protocol-version 3
rebind-as-user
uri "ldaps://ad-server.internal.de:636"
suffix "dc=ad,dc=internal,dc=de"
Now when I search I get the below error (ldaps or ldap).
Both queries work just fine when I change the uri to "ldap://ad-server.internal.de:389"
ldapsearch -h localhost:389 -D "CN=admin,OU=User,DC=ad,DC=internal,DC=de" -W -b DC=ad,DC=internal,DC=de '(objectclass=person)'
ldapsearch -H ldaps://localhost:636 -D "CN=admin,OU=User,DC=ad,DC=internal,DC=de" -W -b DC=ad,DC=internal,DC=de '(objectclass=person)'
=>
ldap_bind: Server is unavailable (52)
additional info: Proxy operation retry failed
This is not a certificate error, as the direct ldapsearch from the same host works just fine:
ldapsearch -H ldaps://ad-server.internal.de:636 -D "CN=admin,OU=User,DC=ad,DC=internal,DC=de" -W -b DC=ad,DC=internal,DC=de '(objectclass=person)'
This is what the log looks like:
Jun 09 12:06:06 adproxy.internal.de slapd[94952]: conn=1000 fd=12 ACCEPT from IP=127.0.0.1:47458 (IP=0.0.0.0:389)
Jun 09 12:06:06 adproxy.internal.de slapd[94952]: conn=1000 op=0 BIND dn="cn=admin,ou=User,dc=ad,dc=internal,dc=de" method=128
Jun 09 12:06:06 adproxy.internal.de slapd[94952]: conn=1000 op=0 ldap_back_retry: retrying URI="ldaps://ad-server.internal.de:636" DN=""
Jun 09 12:06:06 adproxy.internal.de slapd[94952]: conn=1000 op=0 RESULT tag=97 err=52 text=Proxy operation retry failed
Jun 09 12:06:06 adproxy.internal.de slapd[94952]: conn=1000 op=1 UNBIND
Jun 09 12:06:06 adproxy.internal.de slapd[94952]: conn=1000 fd=12 closed
Any idea why this happens would be greatly appreciated.
Edit:
I did some more digging and it actually seems to be a certificate problem. The tcpdump for the connection returns this error:
TLSv1.2 Record Layer: Alert (Level: Fatal, Description: Unknown CA)
The AD certificate is self signed and unfortunately I have no saying in that, so I cannot change that.
In the standard config I get the same error when running ldapsearch. I can get ldapsearch running with two methods:
Set TLS_REQCERT never in ldap.conf
Set environment variable LDAPTLS_REQCERT=never
I know this is not ideal but this is not supposed to be the final configuration, I just want to get it running first.
However slapd seems to ignore both, because I still get the same error in the tcpdump and slapd debugging still gives me the following:
TLS trace: SSL_connect:before/connect initialization
TLS trace: SSL_connect:SSLv2/v3 write client hello A
TLS trace: SSL_connect:SSLv3 read server hello A
TLS certificate verification: depth: 0, err: 20, subject: /CN=ad-server.internal.de, issuer: /CN=ad-server.internal.de
TLS certificate verification: Error, unable to get local issuer certificate
TLS trace: SSL3 alert write:fatal:unknown CA
TLS trace: SSL_connect:error in error
TLS trace: SSL_connect:error in error
TLS: can't connect: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed (unable to get local issuer certificate).
I tried to add the self-signed certificate to the ldap.conf and to the trusted anchors for openssl, but both didn't work. I also noticed that I cannot get openssl to verify the self-signed certificate, while I can do the same on my PC...
My PC:
$ openssl version
OpenSSL 1.1.1h 22 Sep 2020
$ openssl verify server.crt
error server.crt: verification failed
CN = ad-server.internal.de
error 18 at 0 depth lookup: self signed certificate
$ openssl verify -CAfile server.crt server.crt
server.crt: OK
While on the server (REHL 7.9):
$ openssl version
OpenSSL 1.0.2k-fips 26 Jan 2017
$ openssl verify server.crt
server.crt: CN = ad-server.internal.de
error 20 at 0 depth lookup: unable to get local issuer certificate
$ openssl verify -CAfile server.crt server.crt
server.crt: CN = ad-server.internal.de
error 20 at 0 depth lookup: unable to get local issuer certificate
The result on my PC is the expected behaviour for self-signed certificates, as far as I know. I already checked, but it is the latest version available on RHEL 7.9
So does anyone have any idea how I can
get SLAPD to honor the TLS_REQCERT or LDAPTLS_REQCERT settings OR
openssl to verify the self-signed certificate
Thank you!
I finally found a possibility to ignore the certificate validation error. Which is not the optimal solution, but due to the self-signed certificate and the openssl issue in the RHEL7.9 version, I am for now content to have it running at all. The final solution should then be to make the AD department get a certificate from our CA, though I don't know if I can convince them to do so.
So while the LDAP proxy seems to ignore the TLS_REQCERT none setting in the ldap.conf, there is a possibility to do this directly in slapd.conf:
### Database definition (Proxy to AD) #########################################
database ldap
readonly no
protocol-version 3
rebind-as-user
uri "ldaps://ad-server.internal.de:636"
suffix "dc=ad,dc=internal,dc=de"
tls start tls_reqcert=never
The last line eliminates the verification error.

OpenVPN client is not working on Windows computer?

I try to run my OpenVPN client on my windows 10 machine in order to connect to a remote OpenVPN CentOS 7 server but it does not work. I get the error below:
Options error: --capath fails with 'C:\Users\Desktop\OpenVPN\ca.crt': No such process (errno=3)
Options error: --cert fails with 'C:\Users\Desktop\OpenVPN\Win10client.crt': No such process (errno=3)
Fri Mar 22 22:56:20 2019 WARNING: cannot stat file 'C:\Users\Desktop\OpenVPN\Win10client.key': No such process (errno=3)
Options error: --key fails with 'C:\Users\Desktop\OpenVPN\Win10client.key'
Fri Mar 22 22:56:20 2019 WARNING: cannot stat file 'C:\Users\Desktop\OpenVPN\myvpn.tlsauth': No such process (errno=3)
Options error: --tls-crypt fails with 'C:\Users\Desktop\OpenVPN\myvpn.tlsauth': No such process (errno=3)
This is the config that I have on my ovpn file:
client
tls-client
--capath C:\\Users\\Desktop\\OpenVPN\\ca.crt
--cert C:\\Users\\Desktop\\OpenVPN\\Win10client.crt
--key C:\\Users\\Desktop\\OpenVPN\\Win10client.key
--tls-crypt C:\\Users\\Desktop\\OpenVPN\\myvpn.tlsauth
remote-cert-eku "TLS Web Client Authentication"
proto udp
remote serveraddress 1194 udp
dev tun
topology subnet
pull
Assuming your config file is well done. Try to reinstall openvpn, and put your config file to the c:/program files/openvpn/config folder. Then you can start the openvpn Service. Therefore you dont need to use the Openvpn gui.

Jboss 7.0 Fails to start in Red Hat

Hi, i'm trying to run Jboss EAP 7.0.0 in Red Hat Enterprise Linux 7, the installation goes well until i need to start the service.
sudo service jboss-eap-rhel start
Redirecting to /bin/systemctl start jboss-eap-rhel.service
Job for jboss-eap-rhel.service failed. See 'systemctl status jboss-eap-rhel.service' and 'journalctl -xn' for details.
After reach for the service log, it shows that the JBoss EAP startup script has failed to start.
localhost.localdomain systemd1: Failed to start SYSV: JBoss EAP startup script.
systemctl status jboss-eap-rhel.service
jboss-eap-rhel.service - SYSV: JBoss EAP startup script
Loaded: loaded (/etc/rc.d/init.d/jboss-eap-rhel.sh)
Active: failed (Result: resources) since Wed 2017-05-17 05:35:37 EDT; 6min ago
Process: 16673 ExecStart=/etc/rc.d/init.d/jboss-eap-rhel.sh start (code=exited, status=0/SUCCESS)
Main PID: 6979
May 17 05:35:06 localhost.localdomain systemd[1]: Starting SYSV: JBoss EAP startup script...
May 17 05:35:06 localhost.localdomain jboss-eap-rhel.sh[16673]: Starting jboss-eap: chown: missing operand after ‘/var/run/jboss-eap’
May 17 05:35:06 localhost.localdomain jboss-eap-rhel.sh[16673]: Try 'chown --help' for more information.
May 17 05:35:37 localhost.localdomain jboss-eap-rhel.sh[16673]: jboss-eap started with errors, please see server log for details
May 17 05:35:37 localhost.localdomain jboss-eap-rhel.sh[16673]: [ OK ]
May 17 05:35:37 localhost.localdomain systemd[1]: PID file /var/run/jboss-eap/jboss-eap.pid not readable (yet?) after start.
May 17 05:35:37 localhost.localdomain systemd[1]: Failed to start SYSV: JBoss EAP startup script.
May 17 05:35:37 localhost.localdomain systemd[1]: Unit jboss-eap-rhel.service entered failed state.
i checked the jboss conf and the eap-rhel.sh looking for something wrong, including the standalone.xml and the standalone-full.xml, but everything looks to be ok.
the files of the jboss are in /usr/share right now (i have installed and unstalled several times in different folders trying to solve it, yes i have deleted remaining files before each installation).
just to be sure, i mention the steps i done after every installation:
the jboss-eap.conf was succefully edited. the user and the path of the jboss were changed to the right ones.
jboss-eap.conf copied to /etc/default
jboss-eap-rhel copied to /etc/init.d
I also opened it using
./standalone.sh -c standalone-full.xml
it throws this warning:
03:56:23,735 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 60) WFLYTX00 13: Node identifier property is set to the default value. Please make sure it is unique.
and doesn't work (because the service is still not active).
¿how can I start the service?
03:56:23,735 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 60) WFLYTX0013: Node identifier property is set to the default value. Please make sure it unique.
You dont have to worry about it unless you have enabled JTA. You can set unique value of node identifier in standalone-full.xml file like :
<subsystem xmlns="urn:jboss:domain:transactions:1.4">
<core-environment node-identifier="${jboss.tx.node.id}">
...
Regarding service, please verify steps you have followed http://www.dmartin.es/2014/07/jboss-eap-6-as-rhel-7-service/
If you're using JBoss 7.x, you can use the following CLI commands:
/host=master/server-config=server-one/system-property=jboss.tx.node.id:add(boot-time=true,value=master)
/host={slave-host}/server-config=server-one/system-property=jboss.tx.node.id:add(boot-time=true,value=slave2)
/profile={some-profile}/subsystem=transactions:write-attribute(name=node-identifier,value="${jboss.tx.node.id}")
:reload-servers(blocking=true)
This will add the following lines:
<subsystem xmlns="urn:jboss:domain:transactions:4.0">
<core-environment node-identifier="${jboss.tx.node.id}">
<process-id>
<uuid/>
</process-id>
</core-environment>
<recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager"/>
<object-store path="tx-object-store" relative-to="jboss.server.data.dir"/>
</subsystem>
In each profile section of the domain.xml configuration file (in domain controller), and:
<servers>
<server name="server-one" group="x-server-group" auto-start="true">
<system-properties>
<property name="jboss.tx.node.id" value="slave1" boot-time="true"/>
</system-properties>
</server>
</servers>
under each server definition in the host-slave.xml configuration file (in host controller).
External references:
https://access.redhat.com/solutions/748323
https://access.redhat.com/solutions/260023
https://issues.jboss.org/browse/JBEAP-11208

Apache: Failed to configure CA certificate chain

Pre-note: The certificates was purchased from a vendor and are valid till 2018
Our Apache for one of our servers (Ubuntu 12.04) crashed this morning. Trying to restart Apache kept giving us the following error message
[Wed Jun 03 12:21:51.875811 2015] [ssl:emerg] [pid 30534] AH01903: Failed to configure CA certificate chain!
[Wed Jun 03 12:21:51.875846 2015] [ssl:emerg] [pid 30534] AH02311: Fatal error initialising mod_ssl, exiting. See /var/log/apache2/error.log for more information
After removing the following line from apache config
SSLCertificateChainFile /etc/apache2/ssl/wck.bundle
Apache reloaded.
The server did not restart so I am sure no updates where done by accident.
I then proceeded to try and get it up and running on one of the 14.04 Ubuntu servers we own. The same problem occurred with the same certificates. I asked the guy who setup the 14.04 apache and he claims the problem we suddenly experienced today with the 12.04 server has always happened on the 14.04 server.
I tried reproducing the error on my local 14.04 by installing a new Apache and copying the certificates and one of the config files for one of the sites to my local machine. On my local machine after the setup everything worked perfectly.
I have tried comparing openssl versions, lib version between the two 14.04, but everything looks the same. I even upgraded both my local machine and the 14.04 server to ensure the libs and Apache version are identical, but the one works and the other one doesn't and I recon If I can solve this problem for the 14.04 Ubuntu server it will provide me with the information to get the ssl certificate chain up and running on the 12.04 Machine.
Does anyone have an idea why suddenly the 12.04 Ubuntu's Apache would stop working with the ssl certaficate chain and the 14.04 server also produces the same error, but my local 14.04 does not?
Any help would be appreciated.
Thanks in advance.

Puppet showing cert in list, but cannot 'find' it

I am setting up Puppet on a few test servers: bruno is the puppet master and oppenheimer is the agent. When I start the server on bruno I get this output:
bruno$ sudo puppet cert list
"oppenheimer.home" (SHA256) D4:**:**:**:0B:2A
bruno$ sudo puppet master --verbose --no-daemonize
Notice: Starting Puppet master version 3.4.3
I then go to start the agent on oppenheimer:
oppenheimer$ sudo puppet agent --test --server=bruno
Exiting; no certificate found and waitforcert is disabled
And when I look over at bruno again:
Info: access[^/catalog/([^/]+)$]: allowing 'method' find
Info: access[^/catalog/([^/]+)$]: allowing $1 access
Info: access[^/node/([^/]+)$]: allowing 'method' find
Info: access[^/node/([^/]+)$]: allowing $1 access
Info: access[/certificate_revocation_list/ca]: allowing 'method' find
Info: access[/certificate_revocation_list/ca]: allowing * access
Info: access[^/report/([^/]+)$]: allowing 'method' save
Info: access[^/report/([^/]+)$]: allowing $1 access
Info: access[/file]: allowing * access
Info: access[/certificate/ca]: adding authentication any
Info: access[/certificate/ca]: allowing 'method' find
Info: access[/certificate/ca]: allowing * access
Info: access[/certificate/]: adding authentication any
Info: access[/certificate/]: allowing 'method' find
Info: access[/certificate/]: allowing * access
Info: access[/certificate_request]: adding authentication any
Info: access[/certificate_request]: allowing 'method' find
Info: access[/certificate_request]: allowing 'method' save
Info: access[/certificate_request]: allowing * access
Info: access[/]: adding authentication any
Info: Inserting default '/status' (auth true) ACL
Info: Not Found: Could not find certificate oppenheimer.home
Info: Not Found: Could not find certificate oppenheimer.home
Info: Not Found: Could not find certificate oppenheimer.home
Info: Not Found: Could not find certificate oppenheimer.home
Info: Not Found: Could not find certificate oppenheimer.home
Notice that the server bruno does show the agent oppenheimer's cert before I start the server. So why can it not find the cert?
This is my config on the server:
bruno$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 bruno
10.0.0.7 bruno
10.0.0.10 oppenheimer
bruno$ cat /etc/puppet/puppet.conf
[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
templatedir=$confdir/templates
prerun_command=/etc/puppet/etckeeper-commit-pre
postrun_command=/etc/puppet/etckeeper-commit-post
certificate_revocation=false
server=bruno
[master]
# These are needed when the puppetmaster is run by passenger
# and can safely be removed if webrick is used.
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY
And here is the config on the agent:
oppenheimer$ cat /etc/hosts
127.0.0.1 localhost
127.0.1.1 oppenheimer
10.0.0.7 bruno
10.0.0.10 oppenheimer
oppenheimer$ cat /etc/puppet/puppet.conf
[main]
logdir=/var/log/puppet
vardir=/var/lib/puppet
ssldir=/var/lib/puppet/ssl
rundir=/var/run/puppet
factpath=$vardir/lib/facter
templatedir=$confdir/templates
prerun_command=/etc/puppet/etckeeper-commit-pre
postrun_command=/etc/puppet/etckeeper-commit-post
certificate_revocation=false
server=bruno
[master]
# These are needed when the puppetmaster is run by passenger
# and can safely be removed if webrick is used.
ssl_client_header = SSL_CLIENT_S_DN
ssl_client_verify_header = SSL_CLIENT_VERIFY
[agent]
server=bruno
Both the machines are running Ubuntu Linux 14.04 with the latest updates.
You have to sign the certificate. If the certificate was signed already then it would not show up in the output of puppet cert list.
# puppet cert sign oppenheimer.home
Then puppet agent should run successfully.
Hope this helps.

Resources