I have installed foreman (v1.9.0) on a RHEL 7.1 VM as per the official documentation.
My current environment consists of:
1 x Puppet master/foreman
2 x agents (RHEL 6.5 & 7)
Foreman is configured as the ENC which will service 9 environments (inc production).
A smart proxy has been configured in foreman for the foreman/puppetmaster.
There were no issues on the agents in generating the CSR and getting it signed by the puppet master.
When I run the puppet agent on a remote machine with the command
puppet agent --no-daemonize --server <FQDN> --trace
I get the following errors
Warning: Unable to fetch my node definition, but the agent will continue:
Warning: Error 400 on SERVER: Failed to find < agent FQDN> via exec: Execution of '/etc/puppet/node.rb < agent FQDN>' returned 1:
Running the referenced command on the puppet master
sudo -u puppet /etc/puppet/node.rb <agent FQDN>
Returns the error:
Error retrieving node < agent FQDN> Net::HTTPNotFound
Check Foreman's /var/log/foreman/production.log for more information.
Researching this error on the web
"Foreman with puppet node.rb error 404 Not Found"
provides the standard response of the puppet master is not aware (DNS, /etc/hosts) of the agent.
In my case this is not the issue as running the same node.rb command against the puppetmaster returns the same error. A simple ping test confirms that DNS is correctly setup.
There are no logs generated in /var/log/foreman/production.log (inc debug) when this fails, although there is a http 404 error generated in /var/log/httpd/foreman-ssl_access_ssl.log
< IP> - - [30/Sep/2015:15:13:29 +1000] GET /node/< agent FQDN>?format=yml HTTP/1.1" 404 48 "-" "Ruby"
node.rb references foreman.yaml which is correct in relation to the listed entries I can confirm like:
:url:
:ssl_ca:
:ssl_cert:
:ssl_key:
etc
It is also the default file generated by the install process with no changes.
The agents are in foreman, but to achieve that once I configured puppet.conf and generated/signed the agent certs I ran the command
puppet agent -t <puppet master FQDN>
the agents were not imported into foreman until I manually ran the command
foreman-rake puppet:import:hosts_and_facts
calling one of the hosts in foreman and then clicking on 'YAML' shows correct information for each agent.
I have even pulled the information out of the certificates using openssl to confirm that they are correct.
UPDATE:
I have run the command
curl -k https://< Puppet Master FQDN>/foreman
This returns the output
<html><body>You are being redirected.</body></html>
This generates the log entry in
~/foreman/production.log
2015-10-01 10:04:57 [app] [I]
|
| Starged GET "/foreman/" for < PUPPET MASTER IP> at 2015-10-01 10:04:57 +1000
2015-10-01 10:04:57 [app] [I] Processing by DashboardController#index as /
2015-10-01 10:04:57 [app] [I]Redirected to https://< PUPPET MASTER>/foreman/users/login
2015-10-01 10:04:57 [app] [I] Filter chain halted as :require_login
rendered or redirected
2015-10-01 10:04:57 [app] [I]Completed 302 Found in 7ms (Active Record: 0.6ms)
While in ~/httpd/foreman-ssl_access_ssl.log I get this
< PUPPET MASTER IP> - - [01/Oct/2015:10:05:26 +1000] "GET /foreman/ HTTP/1.1" 302 129 "-" "curl/7.29.0"
Other relevant log entries in the same log would be
< PUPPET MASTER IP> - - [01/Oct/2015:09:53:28 +1000] "POST /api/hosts/facts /HTTP/1.1" 404 27 "-" "Ruby"
< PUPPET MASTER IP> - - [01/Oct/2015:09:53:28 +1000] "GET /node/< PUPPET MASTER>?format=yml HTTP/1.1" 404 52 "-" "Ruby"
So the question to ask is what can I do to the Puppet/Foreman HTTP config to resolve the 404 errors
It looks like the ENC script is requesting paths at the root of the web server (/api) while Foreman itself is hosted at a sub-URI (/foreman/). It should be hitting /foreman/api/hosts/facts, /foreman/node/ etc, which is why Apache's returning a 404, but Foreman isn't seeing and logging the request.
You can change the URL that node.rb uses in /etc/puppet/foreman.yaml, via the :url: setting. Add the /foreman suffix here.
Alternatively if you used the Foreman installer then you could re-run it with --puppet-server-foreman-url=https://example.com/foreman which should do the same thing.
Related
I have a gitlab instance with a public IP set up and running. However, trying to add a gitlab-runner fails, because the reverse lookup for the runner does not seem to work as intended. When registering the runner, I can see an incoming POST request with HTTP status 200 on nginx on the gitlab instance
1.2.3.4 - - [09/Oct/2021:21:41:51 +0200] "POST /api/v4/runners HTTP/1.1" 200 179 "-" "gitlab-runner 14.3.2 (14-3-stable; go1.13.8; linux/amd64)"
but the runner registration fails.
sudo gitlab-runner --debug register --non-interactive --url https://gitlab.example.org/ --registration-token asdasdasdasd --executor shell --tag-list shared --run-untagged=true --locked=false --access-level=not_protected
Runtime platform arch=amd64 os=linux pid=2488 revision=e0218c92 version=14.3.2
Checking runtime mode GOOS=linux uid=0
Running in system-mode.
Dialing: tcp gitlab.example.org:443 ...
ERROR: Registering runner... failed runner=LQ_-Gq-k status=200 OK
PANIC: Failed to register the runner. You may be having network problems.
The message is a bit misleading...
How to show what happens in gitlab after the connection was made? I mean the HTTP response is officially a 200.
The runner runs behind a NAT and has no public IP, but the gitlab instance has. The route and DNS settings are working to be able to resolve the runner.
Gitlab runs behind an Nginx reverse proxy, but the proxy is not controlled by gitlab-ctl. Since this was a fresh install, I checked my vhost definition and figured that it may be the case, because the incoming requests to register seemed to be eaten by nginx somehow. gitlab-ctl tail did not even log the incoming request to register the runner.
With that assumption, I enabled the nginx shipped with gitlab in gitlab.rb
nginx['enable'] = true
and ran gitlab-ctl reconfigure just to disable nginx in gitlab.rb afterwards
nginx['enable'] = false
and run gitlab-ctl reconfigure again.
After that procedure, nginx configs are in /var/opt/gitlab/nginx/conf/. I replaced my vhost definition with gitlab-http.conf and adapted it further.
Now it all works as intended. My vhost template was completely outdated.
I use the ddns, so that you need to resolve the domain name by editing the hosts file, in my case is ubuntu:
/etc/hosts
add the ip address of your gitlab server ip, like this:
192.168.2.180 yourgitlab.yourdomain.com
then, execute the gitlab-runner register again, it should be successful.
sudo gitlab-runner register --url https://<yourgitlab.yourdomain.com> --registration-token <your token>
While migrating from puppet 5 to 6 (6.20), it was noticed that puppet cert ca list --all was deprecated with puppetserver ca list --all. When we execute this command we get an error:
Fatal error when running action 'list' Error: Failed connecting to
https://puppet:8140/puppet-ca/v1/certificate_statuses/any_key Root
cause: Failed to open TCP connection to puppet:8140 (getaddrinfo: Name
or service not known)
This error indicates that puppet is looking for a host names puppet whereas the puppetserver host alias in /etc/hosts is puppetserver.
When we add another alias as puppet to /etc/hosts then this command gets executed successfully. I have 2 queries on this
Is there a mechanism to execute this command without modifying /etc/hosts
What is the output structure of this command. I need to parse this using a script. In version 5 the format was having a prefix of +, - etc, which does not exists now.
Concerning 1-st question.
Now days it's due to default hard-coded or configured hostname under the hood.
Name may be defined in puppet.conf and no need to support /etc/hosts
[server]
server = fqdn.fqdn
or
[server]
ca_server = fqdn.fqdn
As described in configuration manual at https://puppet.com/docs/puppet/7/configuration.html#ca-server
ca_server
The server to use for certificate authority requests. It's a separate server because it cannot and does not need to horizontally scale.
Default: $server
However there is life hack:
$ cat /etc/hosts | grep puppet
127.0.0.1 puppetmaster puppet
And puppetserver ca will succeed too in certain cases.
I am trying to set up Puppet for DevOps. I have puppet server in Ubuntu 14.04 and puppet-agent in Windows 10. When I am generating certificate for the first time from puppet-agent (Windows 10) the SSL certficate is generating without any issues and even I can sign the same certificate from puppetserver (Ubuntu 14.04), however after signing when I am trying to update the status in puppet-agent (Windows 10) by "puppet agent -t" getting error as,
Error: Could not request certificate: SSL_connect returned=1 errno=0 state=error: certificate verify failed: [unable to get local issuer certificate for /CN=username-virtualbox.domain.com]
The puppet configuration file details:
puppet.config (puppetserver [Ubuntu]):
Troubleshooting steps already tried,
Time zone in both the environments are in sync
Deleted "ssl" folders containing the details of the ssl certificates from both the environments several times and re-tried
8140 ports are enabled on both Windows and Ubuntu
I faced the same problem,
Try to run cmd as admin
I am having the same issue -- been working it for a few weeks now. I cannot guarantee yet that mine is working correctly all the time. Here are some steps I have taken. I hope they are helpful to others.
I am running Puppet Enterprise 2018.1.4. Puppet Agent 5.5.6 on RHEL 7.4.
1) The SSL routine uses a time stamp. Ensure the time is the same between Master & Client.
2) clean/remove the agent cert from the Master AND the Client. On my RHEL, the Client cert is is in /etc/puppetlabs/puppet/ssl/* -- remove any files with the agent name in here.
3) Make sure your puppet enabled on your agent: puppet agent --enable
4) If a client does not contact the puppet master "for a while" the master will drop the client from it's node list, but NOT remove the cert. In theory, the master SHOULD return the node to an active status.
5) Can you run the puppet agent on the master & get the expected results? If not -> problem with puppet code, otherwise, problem with agent.
6) Is puppet.conf configured correctly? Under the [main] section, do you have the server entry correct? Under [agent] are you set to the correct environment? Is noop set to true?
7) It is possible that you have an error in a puppet module that is causeing the agent to exit quietly. Run puppet parser validate on all of your .pp files
8) Can the master resolve the IP address of the master and the client? Can the client resolve the the IP address of the master and the client? Is resolv.conf set correctly on both hosts?
9) hostnames of the client & master should be correct. Each server should know it's shortname, FQDN and IP. On RHEL, I run: hostname; hostname -f; and hostname -i, respectively.
10) File permissions on all the directories & modules should be correct. Check out a working module, see it's owner, group & permissions. Ensure your module is the same.
11) Only root/admin can correctly run puppet agent.
12) On RHEL, the logs are under /var/log/puppet. Do you see any errors there?
13) run puppet agent with the --debug or the --trace option in addition to -t. Pipe this output to a file and see if you can spot any errors.
14) Can you force the master to run the puppet agent on the client successfully?
Many of these things have been narrowing down my issue. I don't know yet if it is fixed, as It takes a while for a node to drop out. Hopefully these will fix your issue.
Hope it helps. There are LOTS of things that could be going wrong.
I am trying to setup puppet master and puppet agent in aws ec2 instances-linux ami. when i run my puppet agent to generate the certificate for the master to sign i encounter the below error.
Puppet Master:
[root#ip-10-**-*-*** /]# sudo yum install puppet-server
[root#ip-10-**-*-*** /] sudo service puppetmaster start
Starting puppetmaster: [ OK ]
Puppet Agent:
[root#ip-10-**-*-*** /]# sudo yum install puppet
[root#ip-10-**-*-*** /]
File excerpt /etc/puppet/puppet.conf
[main]
server = hostname
[root#ip-10-**-*-*** /] sudo service puppet start
Starting puppet: [ OK ]
[root#ip-10-**-*-*** /]# puppet agent -t
info: Creating a new SSL key for ip-10-**-*-***.dev.abc.net
info: Caching certificate for ca
info: Creating a new SSL certificate request for ip-10-**-*-***.dev.abc.net
info: Certificate Request fingerprint (md5): C2:F0:B1:2C:19:39:9E:D6:39:24:18:28
Exiting; no certificate found and waitforcert is disabled
Puppet Master:
[root#ip-10-**-*-*** /]# puppet cert list
"ip-10-**-*-***.dev.abc.net" (C2:F0:B1:2C:19:39:9E:D6:39:24:18:28:F6:DA:5D:FE)
[root#ip-10-**-*-*** /]# puppet cert sign ip-10-**-*-***.dev.abc.net
notice: Signed certificate request for ip-10-**-*-***.dev.abc.net
notice: Removing file Puppet::SSL::CertificateRequest ip-10-**-*-***.dev.abc.net at '/var/lib/puppet/ssl/ca/requests/ip-10-**-*-***.dev.abc.net.pem'
Puppet Agent:
[root#ip-10-**-*-*** /]# puppet agent -t
info: Caching certificate for ip-10-**-*-***.dev.abc.net
err: Could not retrieve catalog from remote server: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [certificate signature failure for /CN=ip-10-**-*-***.dev.abc.net]
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run
err: Could not send report: SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed: [certificate signature failure for /CN=ip-10-**-*-***.dev.abc.net]
Can anyone please help me resolve this issue.
Yes, I know this is an old post. It still needs an answer, as I am having the same issue -- been working it for a few weeks now. I cannot guarantee yet that mine is working correctly all the time. Here are some steps I have taken. I hope they are helpful to others.
I am running Puppet Enterprise 2018.1.4. Puppet Agent 5.5.6 on RHEL 7.4.
1) The SSL routine uses a time stamp. Ensure the time is the same between Master & Client.
2) clean/remove the agent cert from the Master AND the Client. On my RHEL, the Client cert is is in /etc/puppetlabs/puppet/ssl/* -- remove any files with the agent name in here.
3) Make sure your puppet enabled on your agent: puppet agent --enable
4) If a client does not contact the puppet master "for a while" the master will drop the client from it's node list, but NOT remove the cert. In theory, the master SHOULD return the node to an active status.
5) Can you run the puppet agent on the master & get the expected results? If not -> problem with puppet code, otherwise, problem with agent.
6) Is puppet.conf configured correctly? Under the [main] section, do you have the server entry correct? Under [agent] are you set to the correct environment? Is noop set to true?
7) It is possible that you have an error in a puppet module that is causeing the agent to exit quietly. Run puppet parser validate on all of your .pp files
8) Can the master resolve the IP address of the master and the client? Can the client resolve the the IP address of the master and the client? Is resolv.conf set correctly on both hosts?
9) hostnames of the client & master should be correct. Each server should know it's shortname, FQDN and IP. On RHEL, I run: hostname; hostname -f; and hostname -i, respectively.
10) File permissions on all the directories & modules should be correct. Check out a working module, see it's owner, group & permissions. Ensure your module is the same.
11) Only root/admin can correctly run puppet agent.
12) On RHEL, the logs are under /var/log/puppet. Do you see any errors there?
13) run puppet agent with the --debug or the --trace option in addition to -t. Pipe this output to a file and see if you can spot any errors.
14) Can you force the master to run the puppet agent on the client successfully?
Many of these things have been narrowing down my issue. I don't know yet if it is fixed, as It takes a while for a node to drop out. Hopefully these will fix your issue.
I have installed puppet agent (windows) and master successfully. When i download the node from agent it works fine (puppet agent -t ) and the nodes are pulled successfully.
But when i trigger the same from the puppet master, i get connection error as below.
Error: Host agent01 failed: Connection refused - connect(2)
I did a wireshark in agent machine and i could see the connection made to agent at port 8139 and is responded with a red ACK message.
Since the connection has reached the agent, am assuming agent rejected the connection. Could some one please help me fix this problem.
Puppet.conf at agent
[main]
server=puppetmaster.localsky.biz
autoflush=true
environment=production
pluginsync=true
listen = true
[agent]
auth.conf at agent:
path /
method find, search, save
auth yes
allow puppetmaster.localsky.biz
However, if i run puppet agent --no-daemonize --debug
from the Powershell console, the build gets pushed successsfully.
So the failure happens only if run the agent from Puppet Agent as windows service.
I landed up on the link below which says Kick is not possible for windows agents. Not sure how true this is...
https://ask.puppet.com/question/14234/port-8139-not-opened-when-running-as-a-windows-service/