Check GitLab API access: FAILED. code: 500 and all pages giving 500 error - gitlab

I am getting 500 error code when viewing pages and doing:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production
Outputs
Check GitLab API access: FAILED. code: 500
gitlab-shell self-check failed
Try fixing it:
Make sure GitLab is running;
Check the gitlab-shell configuration file:
sudo -u git -H editor /home/git/gitlab-shell/config.yml
Please fix the error above and rerun the checks.

Checking the logs for production located at /home/git/gitlab/log/production.log
tail -n 100 /home/git/gitlab/log/production.log
I see the last few errors are related to:
Errno::EACCES (Permission denied - connect(2) for /var/run/redis/redis.sock):
config/initializers/rack_attack_git_basic_auth.rb:5:inblock in `'
So from the command line:
cd /var/run/redis/
ls -l
Output
-rw-r--r-- 1 redis redis 5 Mar 29 16:39 redis-server.pid
srwxrwxr-x 1 root root 0 Mar 29 16:39 redis.sock
Here I think the file itself redis.sock has the wrong permissions or owned by the wrong user (root), lets changed that to redis:
chown redis:redis redis.sock
ls -l
Output
-rw-r--r-- 1 redis redis 5 Mar 29 16:39 redis-server.pid
srwxrwxr-x 1 redis redis 0 Mar 29 16:39 redis.sock
Doing this now everything works, I can now push/pull to gitlab from terminal, and also view pages without getting a 500 error.
There maybe different causes for a 500 error (though most times its permission errors like these) so its important to check the logs.

For me a port change in /etc/redis.conf from 0 to the standard port 6379 in combination with renaming the rescue.yml ind rescue.yml_old to disable it and setting the servername in apache2 hardcoded sovled the issue.
I've to look more inside tomorrow, but I'm quite sure the main problem was the FQDM of the apache server.
If you get a info message like this, when you restart apache2 :
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using xxxxxxx. Set the 'ServerName' directive globally to suppress this message
You might put ServerName blablabla.com e.g. in /etc/apache2/ports.conf . Restart Apache2 or restart the whole server ( redis etc. ) and try again.

Related

How to restart apache2 when I get a pid conflict and php not processed by the server

There are processes named 'apache2' running which do not match y our pid file which are left untouched in the name of safety, Please review the s ituation by hand.
Following Test Were Done
no of pids -> sudo pidof apache2
3501 3500 3498 3497
Remove 3497 id and sudo service apache2 start - >
Error:
* Starting web server apache2 *
* The apache2 configtest failed.
Output of config test was:
apache2: Syntax error on line 140 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/authz_default.load: Cannot load /usr/lib/apache2/modules/mod_authz_default.so into server: /usr/lib/apache2/modules/mod_authz_default.so: cannot open shared object file: No such file or directory
Action 'configtest' failed.
The Apache error log may have more information.
Need your assist ....
even php not processed in the server
php 5.5 not compatible with precise64 version..so I tried with trusty64 all work fine..

Permission denied to read file owned by user

I'm trying to restart (it is running) MongoDB on Ubuntu 14.04 but keep getting failures with this line in the log error getting file /srv/mongodb/keyfile: Permission denied
Here's directory structure
drwxr-xr-x 3 root root 4096 Jan 31 05:54 srv/
drw------- 2 mongodb mongodb 4096 Jan 31 07:07 mongodb/
-rw------- 1 mongodb mongodb 876 Jan 31 07:07 keyfile
The user running mongod is "mongodb", group "mongodb" so it should be available for reading.
If I change permissions on mongodb/ and keyfile to -rw-r--r-- it becomes readable, but mongo claims it's too permissive of course
Going cookoo with this thing already. Any suggestion what's wrong here?
I've had the same problem before, and the solution I found and was not documented had to do with the context. If you are in Linux try:
ls -lahZ
That will display in list, all files, sizes in human readable and Z for contexts, I noticed that the keyFile must have "system_u:object_r:mongod_var_lib_t:s0" context, so it gets fixed with:
chcon system_u:object_r:mongod_var_lib_t:s0 mongodb-keyfile
Hope that helps!!
I got the mongod service working by changing the owner of the key file to mongod
sudo chown mongod:mongod mongodb.key
the mongod process owner is mongod which tries to access the file
In addition to read and write permission for a directory, usually you must have an execute permission as well (more info at http://en.wikipedia.org/wiki/File_system_permissions#Permissions). So, you have to set the following permissions:
chmod 700 /srv/mongodb/
chmod 600 /srv/mongodb/keyfile
Since none of the answers here solved my problem, and I had the same problem exactly, I'll try to post what worked for me.
My solution:
Stop Mongo
Edit mongod.conf and temproraily comment out the attribute keyFile: /path/to/key
Start Mongo
sudo chown mongod </path/to/key>
sudo chmod 400 </path/to/key>
Remove comment from KeyFile attribute in mongod.conf
Restart Mongo
Hope this helps

Nagios - Error: Could not read object configuration data

We have a working version of Nagios 3.2.0 running on two other servers but on these new server we installed it using apt-get install but keep getting the following problem.
The permissions and config file is all default from install and everything seems correct when comparing it to the working versions we have of Nagios...
Note: The Nagios website is running but when i click on any option on the left i get this...
Error: Could not read object configuration data!
Verify configuration options using the -v command-line option to check for errors.
Check the Nagios log file for messages relating to startup or status data errors.
Running
sudo /usr/sbin/nagios3 -v /etc/nagios3/nagios.cfg
Gives:
...
Total Warnings: 0
Total Errors: 0
Things look okay - No serious problems were detected during the pre-flight check
/var/log/nagios3/nagios.log gives no errors, just a bunch of:
...
[1389054450] Auto-save of retention data completed successfully.
[1389058050] Auto-save of retention data completed successfully.
Not a permission issue either tried changing everything to nagios user and group but no luck:
drwxr-xr-x 4 root root 4096 2013-12-10 16:35 .
drwxr-xr-x 141 root root 12288 2014-01-06 16:27 ..
-rw-r--r-- 1 root root 1905 2011-06-08 01:07 apache2.conf
-rw-r--r-- 1 root root 12218 2011-06-08 01:08 cgi.cfg
-rw-r--r-- 1 root root 2441 2011-06-08 01:08 commands.cfg
drwxr-xr-x 2 root root 4096 2013-12-10 16:35 conf.d
-rw-r--r-- 1 root root 26 2013-12-10 16:35 htpasswd.users
-rw-r--r-- 1 root root 43769 2011-06-08 01:08 nagios.cfg
-rw-r----- 1 root nagios 1293 2011-06-08 01:07 resource.cfg
drwxr-xr-x 2 root root 4096 2013-12-10 16:35 stylesheets
Changed it back to root.
Stopped Nagios and restarted but still nothing. Please help!
This is a permission issue, webserver user should be able to accees nagios directory structure.
try the following . assuming your web server is not running as the root user.
chgrp -R webserver_user /var/cache/nagiosx/
and also check other nagios directories for correct permissions.
run nagios as daemon
/usr/local/nagios/bin/nagios -d /usr/local/nagios/etc/nagios.cfg
Solved it a few days ago. Apache on these servers had been configured long ago to run as a different user, not www-data. This did not solve the problem we had back then so we left it as is and forgot about apache. Now that we have Nagios running on apache this caused the problem.
So it was a permission issue in the end.
Note: When checking if it is a permission issue, don't just check the file and directory permissions, check that what needs access to them is indeed running as that user in its configuration. In this case it was changed in /etc/apache2/envvars
Lesson learnt.
In my case (Nagios 4, Centos 7), I got this error because of selinux permissions (although no traces of any denies were logged to syslog). Simple way to test if the issue is selinux:
Load page, confirm failue.
Run command getenforce, should return Enforcing.
Run command setenforce 0.
Reload page. If it now works, the error is related to selinux permissions.
Optional:
Run command setenforce 1.
Reload page, it should fail again.
There was a SELinux issue, and if you don't like to turn off the security,
you can create a rule just for this situation.
to see the error in human readable format
audit2allow -a -w
have audit2allow create a te allow rule
audit2allow -a -M nagios-www
semange nagios-www.pp
refresh and see it works
The nagios configuration file is referenced in the cgi config file, /etc/nagios3/cgi.cfg. Check if that file contains the line
main_config_file=/etc/nagios3/nagios.cfg
In my case it was permissions on /var/cache/nagios3. You can confirm it by straceing the CGI, i.e. sudo su -s /bin/sh -c 'REQUEST_METHOD=GET strace -fo /tmp/strace.log /usr/lib/cgi-bin/nagios3/status.cgi' www-data.
In my case the web server didn't have permission to access /var/log/nagios/nagios.log.
Had the same problem after an update from Nagios 3.5 to Nagios 4.3 using the RPM packages from EPEL. Solved installing the appropriate package containing the SELinux profile (yum install nagios-selinux). For more details: https://serverfault.com/q/894349/217522
In my case I had to bring broader permissions to objects.cache file. In my case my file is at:
/usr/local/nagios/var/objects.cache
hope this helps

Cache/Log permissions in Symfony2 on Amazon Linux AMI

I've set up Symfony2 on the AWS Linux AMI (a free tier micro instance) and set up PHP-FPM and nginx successfully.
My Symfony app works.. until I use the command line to clear the cache. When I do this the /app/logs and app/cache/ folder becomes owned by root and the app can no longer write to it.
I can manually chmod 777 / chown nginx:nginx these files after each cache clear, but there must be a better way?
I've tried using a bash function to su nginx; before it clears the cache, but that doesn't work.
Before cache folder before cache clear:
drwxrwxrwx 11 nginx nginx 4096 Nov 4 13:23 dev
drwxrwxrwx 10 nginx nginx 4096 Nov 4 20:39 prod
And after:
drwxrwxrwx 11 nginx nginx 4096 Nov 4 13:23 dev
drwxr-xr-x 10 root root 4096 Nov 4 20:39 prod
Is this something I can fix in the app, or is there a way of setting up the permissions so they don't become rooted?
Edit: I should also mention I tried:
sudo chmod +a "nginx allow delete,write,append,file_inherit,directory_inherit" app/cache app/logs
But the server doesn't recognise +a.
Also tried using:
setfacl -R -m u:nginx:rwx app/cache app/logs
Which doesn't error, but doesn't fix the issue.
sudo -u nginx php app/console cache:clear
Edit:
Can't accept this answer yet but this worked for me:
So I successfully enabled ACL on my mount following the guide here: http://blog.dsyph3r.com/2011/09/symfony2-using-setfacl-for-cache-and.html
For newbies like, me note the label of the drive is (somewhat strangely) "Label=/" in fstab.
Once that was done I used
setfacl -R -m u:nginx:rwx app/cache app/logs
setfacl -dR -m u:nginx:rwx app/cache app/logs
Now I can clear cache at will it seems.
Original (only use if you cannot or will not enabled ACL):
I followed a help page which got me to set umask(0000); in numerous places, namely the app/app*.phps and the app/console script. This seems to have worked, but I'm not sure of the implications.
Leaving question open for non-umask solutions. Looking into activating ACL on my mount, but new to that and it all seems.. strangely named.

Varnish Cache log not working

varnishlog is returning:
_.vsm: No such file or directory
Has anyone else seen this before?
It looks like varnishlog is not pointing to the correct directory, or has not access to it.
Please check the command line options of varnishd. If the deamon run with -n <instancename> argument, you have to add it to varnishlog as well.
The second thing, is to see the permissions of varnish directory.
In order to see the current directory used, you must log into root and run the command below :
$ lsof -p <PID of varnishd> | grep vsm
Once revealed, you just had to be sure the full path has read permission for your user.
In Varnish 4.1 the root cause can be due to incorrect rights for reading _.vsm file. For example:
# service varnishncsa start
* Starting HTTP accelerator log deamon [fail]
Can't open log - retrying for 5 seconds
Can't open VSM file (Cannot open /var/lib/varnish/dev-me/_.vsm: Permission denied
Varnishncsa works from varnishlog user. But /var/lib/varnish/dev-me/_.vsm can be readable from varnish group or root user only:
# ls -l /var/lib/varnish/dev-me/_.vsm
-rw-r----- 1 root varnish 84934656 Apr 15 05:58 /var/lib/varnish/dev-me/_.vsm
So you can fix this problem in the following way:
# usermod -a -G varnish varnishlog
# id varnishlog
uid=110(varnishlog) gid=116(varnishlog) groups=116(varnishlog),115(varnish)
And now you can start varnishncsa.
In our case the hostname of the server was changed.
If you do not specify an instance name, varnish uses the hostname. It was looking for a directory holding the shared memory logging configuration with the new hostname, but the instance was still running from the directory with the old hostname.
Restarting varnish solved the problem.
I just had the same error message while trying to issue varnishadm commands. Turned out that I renamed my machine without stopping varnish. There was some directory in /var/varnish/ corresponding to the machine name that varnish needed access to. "sudo service varnish restart" fixed this for me.

Resources