I've got GitLab 10.5.6. I'd like to use Health Check information in my monitoring system. I can configure it by using Health Check endpoints with health check access token, but as this solution is depracated, I want to use IP whitelist. And I have some problems with it.
According to this article https://docs.gitlab.com/ee/administration/monitoring/ip_whitelist.html I edited /etc/gitlab/gitlab.rb and added this line (as this GitLab was installed around version 7 or even older I think):
gitlab_rails['monitoring_whitelist'] = ['127.0.0.0/8', '192.168.0.1', 'X.X.X.X', 'Y.Y.Y.Y']
where X.X.X.X is IP of my computer and Y.Y.Y.Y is IP of server with GitLab. After it I executed reconfiguration (gitlab-ctl reconfigure). And started tests... Below logs are from production.log file.
Execution of curl http://127.0.0.1:8888/-/readiness on server Y.Y.Y.Y returns proper JSON with expected data:
Started GET "/-/readiness" for 127.0.0.1 at 2018-03-24 20:01:31 +0100
Processing by HealthController#readiness as /
Completed 200 OK in 27ms (Views: 0.6ms | ActiveRecord: 0.5ms)
Execution of curl http://Y.Y.Y.Y:8888/-/readiness on server Y.Y.Y.Y returns error:
Started GET "/-/readiness" for Y.Y.Y.Y at 2018-03-24 21:20:04 +0100
Processing by HealthController#readiness as /
Filter chain halted as :validate_ip_whitelisted_or_valid_token! rendered or redirected
Completed 404 Not Found in 2ms (Views: 1.0ms | ActiveRecord: 0.0ms)
Accessing address http://Y.Y.Y.Y:8888/-/readiness through Firefox browser on computer X.X.X.X returns error:
Started GET "/-/readiness" for X.X.X.X at 2018-03-24 20:03:04 +0100
Processing by HealthController#readiness as HTML
Filter chain halted as :validate_ip_whitelisted_or_valid_token! rendered or redirected
Completed 404 Not Found in 2ms (Views: 0.8ms | ActiveRecord: 0.0ms)
Accessing address http://Y.Y.Y.Y:8888/-/readiness?token=ZZZZZZZZZZZZZ through Firefox browser on computer X.X.X.X returns proper JSON with expected data.
I don't have any idea what I can check more. Maybe there's lack of any more configuration in /etc/gitlab/gitlab.rb as it's quite old GitLab instance.
Related
I need to regularly and randomly test with Linux/C++/libcurl the responses of several servers that are available through a single DNS name, such as
$ host example.com
n1.example.com 1.2.3.4
n2.example.com 1.2.3.5
n3.example.com 1.2.3.6
The list changes. When I try https://example.com libcurl always uses the same IP for the span of the TTL, and I cannot switch to the next host. There is CURLOPT_DNS_CACHE_TIMEOUT setopt, but setting it to zero does not help - even if I fully recreate easycurl object I still get the same IP. Therefore, this does not help: curl - How to set up TTL for dns cache & How to clear the curl cache
I can of course manually resolve DNS names and iterate, but are there any options? Polling randomly is okay. I see curl uses c-ares. Is there a way to clean up the cache there and will it help?
I cannot do exactly what I need with curl without doing a resolve by myself, but there are findings for the others to share with:
First of all, as a well-written TCP client, curl will try the hosts from the DNS list from top to bottom until a successful connection is made. Since then it will use that host even if it returns some higher level error (such as SSL error or HTTP 500). This is good for all major cases.
Curl command line of newer curl versions has --retry and --retry-all-errors - but there are no such things in libcurl, unfortunately. The feature is being enhanced right now, and there is no release yet as of 2021-07-14 that will enumerate all DNS hosts until there is one that returns HTTP 200. Instead, the released curl versions (I tried 7.76 and 7.77) will always do retries with the same host. But the nightly build (2021-07-14) does enumerate all DNS hosts. Here is how it behaves for two retries and three inexisting hosts (note, the retries will happen if any host returns HTTP 5xx):
$ ./src/curl http://nohost.sureno --trace - --retry 2 --retry-all-errors
== Info: Trying 192.168.1.112:80...
== Info: connect to 192.168.1.112 port 80 failed: No route to host
== Info: Trying 192.168.1.113:80...
== Info: connect to 192.168.1.113 port 80 failed: No route to host
== Info: Trying 192.168.1.114:80...
== Info: connect to 192.168.1.114 port 80 failed: No route to host
== Info: Failed to connect to nohost.sureno port 80 after 9210 ms: No route to host
== Info: Closing connection 0
curl: (7) Failed to connect to nohost.sureno port 80 after 9210 ms: No route to host
Warning: Problem (retrying all errors). Will retry in 1 seconds. 2 retries
Warning: left.
== Info: Hostname nohost.sureno was found in DNS cache
== Info: Trying 192.168.1.112:80...
== Info: connect to 192.168.1.112 port 80 failed: No route to host
== Info: Trying 192.168.1.113:80...
== Info: connect to 192.168.1.113 port 80 failed: No route to host
== Info: Trying 192.168.1.114:80...
== Info: connect to 192.168.1.114 port 80 failed: No route to host
== Info: Failed to connect to nohost.sureno port 80 after 9206 ms: No route to host
== Info: Closing connection 1
curl: (7) Failed to connect to nohost.sureno port 80 after 9206 ms: No route to host
Warning: Problem (retrying all errors). Will retry in 2 seconds. 1 retries
This behavior can be very helpful for the users of libcurl, but unfortunately, these retry flags presently have no mapping to curl_easy_setopt. And as a result, if you give --libcurl to the command line you will not see any retry-related code
I tried to set up API Platform on my local machine to explore it.
I tried to performed all the operations according to API Platform's "Getting Started" page. So I downloaded the latest offical distribution which happens to be v2.4.2 (https://github.com/api-platform/api-platform/releases/tag/v2.4.2) and I started it using Docker.
I cannot however access the administration backend at http://localhost:81 receiving "Unable to retrieve API documentation."
I searched for help at https://api-platform.com/docs/admin/getting-started/, but it describes steps that seems to be already done in the distribution
How can I enable the admin component or debug what went wrong?
Edit (2019-04-14)
$ docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
40a5d5213cfe quay.io/api-platform/nginx "nginx -g 'daemon of…" 45 hours ago Up 6 minutes 0.0.0.0:8080->80/tcp apiplatformdemo_api_1
d53711c0ba0c quay.io/api-platform/php "docker-entrypoint p…" 45 hours ago Up 6 minutes 9000/tcp apiplatformdemo_php_1
2d4eb8d09e3e quay.io/api-platform/client "/bin/sh -c 'yarn st…" 45 hours ago Up 6 minutes 0.0.0.0:80->3000/tcp apiplatformdemo_client_1
abe3e3b41810 quay.io/api-platform/admin "/bin/sh -c 'yarn st…" 45 hours ago Up 6 minutes 0.0.0.0:81->3000/tcp apiplatformdemo_admin_1
4596a7f81cd8 postgres:10-alpine "docker-entrypoint.s…" 45 hours ago Up 6 minutes 0.0.0.0:5432->5432/tcp apiplatformdemo_db_1
c805fc2f11c9 dunglas/mercure "./mercure" 45 hours ago Up 6 minutes 443/tcp, 0.0.0.0:1337->80/tcp apiplatformdemo_mercure_1
Edit 2 (2019-04-14)
It is worth mentioning that although the API component at http://localhost:8080 works, the HTTPS variant at https://localhost:8443 does not. (Connection refused if I try to telnet it.)
Now it turned out it escaped my notice earlier that there is a message in the JS console saying there was a failed connection to https://localhost:8443. (It says about CORS, but I think the real reason is 8443 simply refuses connection). So although I entered the HTTP variant of Admin at http://localhost:81 it tried to access the API via HTTPS. What could be the reason HTTPS doesn't work?
Edit 3 (2019-04-15)
After looking into the logs of docker compose, I see it is relevant the Varnish container failed. h2-proxy depends on it and it is h2-proxy that governs the 8443 port.
cache-proxy_1 | Error:
cache-proxy_1 | Message from VCC-compiler:
cache-proxy_1 | Expected return action name.
cache-proxy_1 | ('/usr/local/etc/varnish/default.vcl' Line 67 Pos 13)
cache-proxy_1 | return (miss);
cache-proxy_1 | ------------####--
cache-proxy_1 |
cache-proxy_1 | Running VCC-compiler failed, exited with 2
cache-proxy_1 | VCL compilation failed
apiplatform242_cache-proxy_1 exited with code 2
h2-proxy_1 | 2019/04/15 08:09:17 [emerg] 1#1: host not found in upstream "cache-proxy" in /etc/nginx/conf.d/default.conf:58
h2-proxy_1 | nginx: [emerg] host not found in upstream "cache-proxy" in /etc/nginx/conf.d/default.conf:58
apiplatform242_h2-proxy_1 exited with code 1
I have solved this error by getting API Platform by cloning the current master and not download the tar.tgz release version (2.4.2)
git clone https://github.com/api-platform/api-platform.git
docker-compose build
docker-compose up -d
Works like a charm !
When we push to our repository, we expect a pipeline to run. However, the pipelines have stopped starting automatically when we push.
In addition, when we try to start the pipeline manually, not all the tags and branches are showing in the dropdown list of tags and branches to choose from. When we browse the repository in Gitlab, we can see the branches and the pushed commits.
Finally, in the /dashboard/projects page, the 'last updated' date of the project is out of date, saying "yesterday" rather than "10 mins ago" (which is what shows when viewing the repository within the project.
We recently migrated server and so would expect that there is some migration issue going on here. Does anyone have any ideas where to look to solve this problem (i.e. what sub-systems could be not working/configured correctly to produce this behaviour)?
Gitlab version: 9.4.2
Run with Docker using: https://hub.docker.com/r/gitlab/gitlab-ce/
Update
I tailed the logs while pushing to the repository, what follows is a chunk of logs around that time (starting with the SSH connection for the push). Potentially the 404 around prometheus is interesting, but I'm not sure that's unexpected (we're not using it):
==> /var/log/gitlab/sshd/current <==
2017-08-01_17:05:16.86559 Accepted publickey for git from (removed) port 57680 ssh2: RSA SHA256:(removed)
==> /var/log/gitlab/gitlab-rails/production.log <==
Started POST "/api/v4/internal/allowed" for 127.0.0.1 at 2017-08-01 17:05:17 +0000
==> /var/log/gitlab/gitlab-shell/gitlab-shell.log <==
I, [2017-08-01T17:05:17.088866 #2286] INFO -- : POST http://127.0.0.1:8080/api/v4/internal/allowed 0.01170
I, [2017-08-01T17:05:17.089030 #2286] INFO -- : gitlab-shell: executing git command <git-receive-pack /var/opt/gitlab/git-data/repositories/products/preside-ext-ems.git> for user with key key-2.
==> /var/log/gitlab/sshd/current <==
2017-08-01_17:05:17.20480 Received disconnect from x.x.x.x port 57680:11: disconnected by user
2017-08-01_17:05:17.20483 Disconnected from x.x.x.x port 57680
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/-/metrics" for 127.0.0.1 at 2017-08-01 17:05:18 +0000
Processing by MetricsController#index as HTML
Filter chain halted as :validate_prometheus_metrics rendered or redirected
Completed 404 Not Found in 1ms (Views: 0.4ms | ActiveRecord: 0.0ms)
Started POST "/api/v4/jobs/request" for 172.17.0.1 at 2017-08-01 17:05:18 +0000
==> /var/log/gitlab/gitlab-workhorse/current <==
2017-08-01_17:05:18.16504 gitlab.mycompany.com # - - [2017-08-01 17:05:18.158505651 +0000 UTC] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-ci-multi-runner 9.4.1 (9-4-stable; go1.8.3; linux/amd64)" 0.006484
==> /var/log/gitlab/nginx/gitlab_access.log <==
172.17.0.1 - - [01/Aug/2017:17:05:18 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "-" "gitlab-ci-multi-runner 9.4.1 (9-4-stable; go1.8.3; linux/amd64)"
==> /var/log/gitlab/gitlab-rails/production.log <==
Started POST "/api/v4/jobs/request" for 172.17.0.1 at 2017-08-01 17:05:23 +0000
==> /var/log/gitlab/gitlab-workhorse/current <==
2017-08-01_17:05:23.16534 gitlab.mycompany.com # - - [2017-08-01 17:05:23.159064793 +0000 UTC] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "" "gitlab-ci-multi-runner 9.4.1 (9-4-stable; go1.8.3; linux/amd64)" 0.006235
==> /var/log/gitlab/nginx/gitlab_access.log <==
172.17.0.1 - - [01/Aug/2017:17:05:23 +0000] "POST /api/v4/jobs/request HTTP/1.1" 204 0 "-" "gitlab-ci-multi-runner 9.4.1 (9-4-stable; go1.8.3; linux/amd64)"
Not exactly an answer - but I have wiped out the server and rebuilt from scratch. Manually recreating each project and importing the repositories for each project.
A royal PITA, but everything is working as expected.
I can only guess that either something was setup on the host server that was causing issues (I did a clean install on the host to start again), or that there was something about simply copying over all our configuration and data dirs from the old server to the new server that caused issues (seems unlikely).
Not much help I'm afraid :(
I want to create a zabbix proxy and a zabbix agent and setup the agent to speak through the proxy.I have created docker containers for this (zabbix-proxy and zabbix-agent).
proxy.conf:
Server=192.10.30.58 # address of server
ServerPort=10051
Hostname=DFS
agent.conf:
Server=ZabbixProxy # the zabbix-proxy container name
ListenPort=10050
Hostname=Agent
I have created also in zabbix :
A proxy named DFS.
A host named DFS and 192.10.30.3:10051
A host named Agent and 192.18.0.4:10050 (an internal IP where the agent is running)
I can see data from Monitoring-> Latest Data for both the proxy and the agent.
So, it work.
But, in my log I can see that for the agent it gives me:
INFO success: zabbix-agentd entered RUNNIG state, process has stayed up for > than 1 seconds (startsecs)
failed to accept an incoming connection: connection from "192.10.30.58" rejected, allowed hosts: "ZabbixProxy"
(The 192.10.30.3:10051 is the external ip of proxy)
It seems that the agent tries to speak with the server also but I don't know why.
If in agent.conf instead of ZabbixProxy (the name of the zabbix proxy container) I put the address of proxy 192.10.30.3 , then I still have the same errors and also I can't get Latest data for the agent.
I I use ServerActive=ZabbixProxy or ServerActive=192.10.30.3:10051, I am receiving:
...
INFO spawned: 'zabbix-agentd' with pid 51
2017-04-12 16:37:55,916 INFO exited: zabbix-agentd (exit status 1; not expected)
2017-04-12 16:37:57,928 INFO spawned: 'zabbix-agentd' with pid 52
2017-04-12 16:37:57,988 INFO exited: zabbix-agentd (exit status 1; not expected)
2017-04-12 16:38:01,001 INFO spawned: 'zabbix-agentd' with pid 53
2017-04-12 16:38:01,061 INFO exited: zabbix-agentd (exit status 1; not expected)
2017-04-12 16:38:02,063 INFO gave up: zabbix-agentd entered FATAL state, too many start retries too quickly
and of course now the agent doesn't work at all.
Parameter Server is for passive items - incoming connections to the agent. Agent connects to the server (or proxy) based on the parameter ServerActive, which seems to be misconfigured in your case.
Ubuntu 10.04 system. new Plone install, went fine and created some content, everything seemed fine. New kernel update and a reboot later, Plone is running but will not present any pages to a browser. In fact, a browser attempt just times out. I can telnet to the port 8080 on the system and send an HTTP get by hand and nothing comes back. The log file for client1 in a zeo install keeps repeating:
2011-08-10T16:59:57 INFO ZServer HTTP server started at Wed Aug 10 16:59:57 2011
Hostname: 0.0.0.0
Port: 8080
------
2011-08-10T16:59:57 INFO Zope Set effective user to "plone"
------
2011-08-10T17:00:02 INFO ZEO.ClientStorage zeostorage ClientStorage (pid=24596) created RW/normal for storage: '1'
------
2011-08-10T17:00:02 INFO ZEO.cache created temporary cache file '<fdopen>'
------
2011-08-10T17:00:02 INFO ZEO.ClientStorage zeostorage Testing connection <ManagedClientConnection ('127.0.0.1', 8100)>
------
2011-08-10T17:00:02 INFO ZEO.zrpc.Connection(C) (127.0.0.1:8100) received handshake 'Z3101'
------
2011-08-10T17:00:02 INFO ZEO.ClientStorage zeostorage Server authentication protocol None
------
2011-08-10T17:00:02 INFO ZEO.ClientStorage zeostorage Connected to storage: ('dns', 8100)
------
2011-08-10T17:00:02 INFO ZEO.ClientStorage zeostorage No verification necessary -- empty cache
------
2011-08-10T17:00:22 INFO ZServer HTTP server started at Wed Aug 10 17:00:22 2011
Hostname: 0.0.0.0
Port: 8080
I haven't been able to find any other info on what is causing this, nor can I find any documentation on debugging a Plone install.
Thanks for any help you can provide.
Forgive the aborted answer, misread the log snippet. The repeated log entries you're seeing are what you'd expect to see from repeated restarts. Are you repeatedly restarting the instance? If not, then in it seems your instance is restarting on it's own. Shut down the instance and start it using "bin/instance fg" and see if that gives you more information.