GitLab projects failed their last repository check - gitlab

I have received a gitlab alert mail stating that "One project failed its last repository check".
I did check the error in Gitlab admin panel as "Last repository check (just now) failed. See the 'repocheck.log' file for error messages." As suggested in Admin Panel in Gitlab, i have checked the repocheck.log file and the error is as below.
"Could not fsck repository: error: Could not read 0f188244898707e6090498bc03aafd8ac25e776e
failed to parse commit 0f188244898707e6090498bc03aafd8ac25e776e from object database for commit-graph
error: Could not read 4ab7111f3f8f1083cee8e33ec033c18edfefb0e9"
This happened the same with another repo last week. Even that had similar error message and it is not resolved yet. Tried to clone the same repo in another gitlab instance to recreate the issue but the repo check there seems to be fine.
Unable to find proper solution for that. Could any one please help on this.

It seems to be an active issue: https://gitlab.com/gitlab-org/gitaly/-/issues/2359#note_966195929 The issue is a couple of years old, but the note I link to, has updates from today, including info on fixes and workarounds.
In any case: The commit-graph is a relatively new feature and since it is a form of index into packed data, it can be recreated. So the corrupted repo can be easily fixed without loss of data.

It can be reproduced like this:
Visit a commit graph: Project sidebar / Repository / Graph
Delete one of the visible commits by rewriting history:
git checkout <branch>
git reset head~1
git commit -am "Replace last commit with a new one"
git push --force
Refresh the page of step 1
Now GitLab will send you "A commit graph at GitLab projects failed their last repository check" when it runs the repository check on a scheduled interval.
I'd consider it a bug that admins and maintainers get a warning about this, since nothing is wrong really. See the GitLab issues linked by Anders Bandholm for more details.

I've now come across and fixed this twice, I found the fix on the forum:
Find all your failed repos at your instance's admin panel: https://your-gitlab-instance/admin/projects?last_repository_check_failed=1
Open the repo from that page and copy down its Gitaly relative path, it should look like #hashed/d4/73/c530f048efdf2711df6fa15198ff48003583303624f8b97c174fadc2cab5e582.git
NOTE! The following commands should be run with the user that gitlab runs as. You can do this with sudo su [username], in my case the username was git. They will work with sudo, but it may create files or folders that the gitlab user won't be able to access.
Run the fsck command using the repo's relative path (This should output the same text as your /var/log/gitlab/gitlab-rails/repocheck.log file):
/opt/gitlab/embedded/bin/git -C /var/opt/gitlab/git-data/repositories/[Gitlab relative path] fsck
# example
/opt/gitlab/embedded/bin/git -C /var/opt/gitlab/git-data/repositories/\#hashed/d4/73/c530f048efdf2711df6fa15198ff48003583303624f8b97c174fadc2cab5e582.git fsck
Run the gc command using the repo's relative path. This may take a minute to complete.
/opt/gitlab/embedded/bin/git -C /var/opt/gitlab/git-data/repositories/[Gitlab relative path] gc
# example
/opt/gitlab/embedded/bin/git -C /var/opt/gitlab/git-data/repositories/\#hashed/d4/73/c530f048efdf2711df6fa15198ff48003583303624f8b97c174fadc2cab5e582.git gc
Run the fsck command again from step 3 to check that the command was successful
On the repo's page from step 2, click the blue Trigger repository check button

Related

Git checkout untracked issue

I'm collaborating with a few other people on a Drupal website which we are version controlling Git. We setup a local Git repository containing our commits.
After a colleague pushed some updates and I fetched and merged into my local dev branch, I began experiencing the following problems:
user#server:/var/www/Intranet/sites/intranet/modules/custom$ git checkout dev
error: The following untracked working tree files would be overwritten by checkout:
themes/bigcompany/panels/layouts/radix_bryant_flipped/radix-bryant-flipped.png
themes/bigcompany/panels/layouts/radix_bryant_flipped/radix-bryant-flipped.tpl.php
themes/bigcompany/panels/layouts/radix_bryant_flipped/radix_bryant_flipped.inc
Please move or remove them before you can switch branches.
Aborting
The issue above typically shows up when I try to checkout into other branches which fails and I am effectively trapped in my current branch.
Referring to this question, there is a suggestion my issue is related to the gitignore file. However, my .gitignore file has nothing indicating any part of my themes directory should be ignored as the following shows:
# .gitignore for a standard Drupal 7 build based in the sites subdirectory.
# Drupal
files
settings.php
settings.*.php
# Sass.
.sass-cache
# Composer
vendor/
# Migrate sourec files
modules/custom/haringeygovuk_migrate/source_data
As mentioned above, my attempts to execute git checkout into any branch fails with the message above. I decided to force it with the -f switch and successfully switched into my target branch but I lost a couple of hundred lines of code - which I'd love to avoid going forward.
I work on a Linux-Ubuntu VirtualBox which my colleagues prefer working in a WAMP setup and use the Git Bash terminal emulator for executing the Git commands. Could the difference in environments be causing these serious issues?
How can I resolve this issue?
Well, the situation is rather simple. You, in your current branch, don't have certain files under the control of Git, but at the same time, you have those files in your working tree. The branch you're trying to switch to, has those files, so git would need to override files in the working tree to perform checkout.
To prevent possible data loss, Git stops the process of switching the branches and notifies you that you should either add those files under the control of Git in a separate commit in your current branch, and only then perform the switch, or simply remove those files from the git way.
Likely you have chosen the second way. Generally you should "force" any operation only if you really understand what you're doing.

Bare Git repo cannot add files or commit files

So I have been plagued with this weird git problem that myself and a few other developers have not been able to solve. Here it is:
I created a bare repo for managing website changes using git on test server.
For this example the repo is here: /home/website/website.git
The website public root would be here: /home/website
I created the repo by doing this command: git init --bare
inside the git repo directory "website.git"
Next I have my local repo on a machine elsewhere. This is a standard git repo. I build the site get it ready to deploy. When its ready I push it to the bare repo. From my local repo.
There is a post-receive hook that checks the latest file tree out into the public root of the website. So when I change things on the local repo and test them in the localhost environment, once satisfied I can push them to the live server.
Here is the problem I face:
I can push fine. No issues. All works as expected. Code gets checked out to public root. Everybody is happy and goes on with their life.
BUT!!!:
The site is a CMS site. Users log in to it and upload things. Files get created on the public root of the website which is the GIT_WORK_TREE.
So NBD right?! I can just commit the files every now and then from the live bare repo and pull them back to my local environment like I have before. So I log into SSH on the server. Navigate to /home/website/website.git
Then run this command:
GIT_WORK_TREE=/home/website/ git add ../
I get this mess:
error: unable to create temporary sha1 filename ./objects/cb: No such file or directory
error: error_log: failed to insert into database
error: unable to index file error_log
fatal: adding files failed
I have done this before on other servers and it worked fine from what I remember. So I was like WTF, must be something strange on this server. I went to another server I have and replicated the EXACT same steps. Got the EXACT same problem. So now I fear I am loosing my sanity and maybe these previous git experiences are all made up in my head.... *Well, lets not go that far yet.... :)
Maybe somebody can help me out here. I have used git plenty and can't seem to crack this one.
Oh, some other maybe useful specs:
running CENTOS 6.2
I double checked all permissions. I even tried changing everything to 777 recursively just to make sure Im not loosing it somehow. Made sure all the files are owned by the correct user. chowned recursively. I also tried the standard solution to this problem which is described here: https://answers.atlassian.com/questions/132671/git-commit-fails-with-sourcetree-error-unable-to-create-temporary-sha1-filename-git-objects-d8-file-exists
That didnt work either. Not sure where to go from here.
PLEASE HELP ME!!!!
I feel like Linus is playing cruel tricks on me right now.
You have a bare git repository in /home/website/website.git which is inside of a git repository in /home/website? And then you try to trick the bare git repository into having a working tree using GIT_WORK_TREE?
Maybe it is worth understanding this setup but only if you intend to be a git developer. As you are a user focused on delivering website functionality, I suggest using a standard git setup.
Move the bare git repository elsewhere:
$ mkdir /home/repo
$ mv /home/website/website.git /home/repo/website.git
$ cd /home/website
$ git remote set-url origin /home/repo/website.git
I got the solution. This is it. I was running the above command from the git repo.
Turns out the command should be run from the work tree and altered to look like this:
GIT_WORK_TREE=/home/website/ git --git-dir="./website.git/" add ./

Pull not working - TortoiseGIT / Windows 7 / GIT on Debian + gitolite

i have a weird issue. Im using TortoiseGIT (Win7) and my repositories are placed on a vritual server (Debian), where im using gitolite and SSH keys.
I can clone the repository to my PC, i can run Fetch, Push, Commit, Sync .. everything, but when trying to Pull the changes from server Pushed by other contributor, the following error appears:
git.exe pull -v --progress "origin"
fatal: 'pull' appears to be a git command, but we were not
able to execute it. Maybe git-pull is broken?
git did not exit cleanly (exit code 128)
I don't understand, why just the pull command is not working .. thanks for any help.
I can make a clone of the repository, with the contributed changes .. but can not Pull the changes to created repository on my PC.
I encountered this same issue after changing the git Bash executable sh.exe to be always run as administrator (to get round another problem). It then left git unable to access it under certain scenarios and caused various "Maybe git-* is broken?" errors. Perhaps this might help someone...
Uninstalling old Git and reinstalling the latest build fixed this issue for me.
Here's a link to the installers
Link to get installers
My exact error message was
C:\Program Files (x86)\Git/libexec/git-core\git-pull: line 304: exec: git-merge: not found
fatal: 'pull' appears to be a git command, but we were not
able to execute it. Maybe git-pull is broken?
The error message is very much linked to Git, and comes from help.c:
static const char bad_interpreter_advice[] =
N_("'%s' appears to be a git command, but we were not\n"
"able to execute it. Maybe git-%s is broken?");
That is similar to issue 40 (of another GUI, here terminal-ide).
In that case, it was due to the remote Git installation, which was incomplete
(Comments 3 of issue 19)
git-merge was also missing from install, can be fixed with
$> ln -s git git-merge
in system/bin/
The resolution might not be exactly the same in your case, but it could be related to a faulty Git installation.
I see that you're able to run "git fetch". If you can also run "git merge", running the sequence "git fetch" followed by "git merge" will accomplish the same thing as "git pull".
Source:
http://git-scm.com/docs/git-pull

Creating a Subversion branch

I have a Subversion repository (running SVN 1.6.6) on an Ubuntu Linux server, and I'm trying to create a branch using the TortoiseSVN GUI; However, I get an error back:
COPY c:\work\repositoryWorkingcopy\ to http://svnserver/svn/repository/Oct13, Revision HEAD
Error
'/svn/repository/!svn/bc/234/branches'
path not found
I logged into my Linux box to see if the "branches" folder was NOT available. However, I could see that /var/svn/repository/branches existed.
So, what's the problem and how do I fix it?
I tried the following command (as per http://svnbook.red-bean.com/en/1.1/ch04s02.html#svn-ch-4-sect-2.1) as well as from the command line and got a similar error:
c:\workingcopy> svn copy http://reposerver/svn/repository/ http://reposerver/svn/repository/branches/BFI_Oct13/ -m "Creating a branch"
svn: '/svn/repository/!svn/bc/235/branches' path not found
What should I do?
Also, svn info on the main directory of the working copy gives the following information.
Path: .
URL: http://reposerver/svn/myrepository
Repository Root: http://reposerver/svn/myrepository
Repository UUID: 7a31d3c0-b288-4695-aecd-3f9dda2861ab
Revision: 235
Node Kind: directory
Schedule: normal
Last Changed Author: anjan
Last Changed Rev: 235
Last Changed Date: 2010-10-13 20:39:31 +0530 (Wed, 13 Oct 2010)
When you say:
I logged into my Linux box to see if the "branches" folder was NOT available. However, I could see that /var/svn/repository/branches existed.
Do you mean that the actual directory exists in the filesystem (which will do you absolutely no good), or that it exists in the repository?
Possible fix:
$ svn mkdir http://reposerver/svn/repository/branches
$ svn copy http://reposerver/svn/repository/ http://reposerver/svn/repository/branches/BFI_`date +%b%d`/ -m "Creating a branch"
Generally speaking, you'll want to branch on the server, make your changes in the branch, and check those changes in.
So, the first step is to create the branch, which is just a copy command. In TortoiseSVN, you need the URL to (presumably) the trunk, and you'll need to define the URL of the branch. In a more standard repository setup, the trunk might be http://svnserver/svn/repository/trunk and your branch might be http://svnserver/svn/repository/branches/Oct13.
Then, you'll want to switch your working copy to the branch. You should have the option to do so without losing your local changes.
Finally, you would check your changes in, and they should go to the branch from there.
I'm more accustomed to the command-line interface for SVN, but I have used TortoiseSVN. It's been a while, though, so leave a comment if you still have trouble or this doesn't make sense.

svn Merge Problem (3 levels of svn)

I am new to use svn and the company in which I work uses three levels (I don't know whether this is a correct word to use here) of svn. I mean the developers are provided a working directory on a testing server. When we commit, it goes to the dev server. When a manager commits it from there it goes to production server. I am a developer here and one of my files is giving error (conflict) when I commit from directory. Not only, but also it gives conflict when manager tries to commit. I am now given access as manager too but I am still unable to resolve it.
What I've tried till now:
svn update
svn delete
svn commit
It gives conflict on all of these operations.
Earlier on a simple error happened and the manager preferred to just delete file on dev, copy it manually and then commit from there. I don't know this may be a reason of this problem or not.
Please help me resolve this issue. I've read some things in read-bean book too but to no avail yet.
Thanks
Ok, here's the update. The actual problem is that a file (ex lib/a.php) used to be in my working directory as well as in dev and production servers. Now it was deleted by someone (using del command, not svn delete) from dev server. Now question here is how I add it again so that it becomes part of svn again. The simple svn add doesn't work.
Update 2
From one of the answers below I understood that its a tree conflict. Some searching brought me to http://svnbook.red-bean.com/nightly/en/svn.tour.treeconflicts.html . Following the instructions, I took the backup of the file and then svn delete it from everywhere. Then I svn add it to my directory, commit it and tried to update dev and production. End result is that it doesn't go there. No error is shown either.
svn info in my directory shows complete info of the file but on dev and production it shows
file_name: (Not a versioned resource)
:S
Any more ideas please?
Alternatively you can take backup of the file ,then say svn revert filename insert you new code.Do a svn up just to make sure you do not have any conflicts,and then commit
Or
fix the conflicts in the file and then you can say svn resolved filename and then you can continue operations on the file
Update:If your file is deleted using rm or del command use svn revert filename to get it back and you do not have to add it again.Just put in your new changes and say svn ci -m"your comments" filename
svn revert will fetch back the last checked in copy into SVN and it wouldnt have your any changes made before the user had used del command
Update 2:After u say svn delete ,u need to commit it until u get the message Deleting filename with a new revision number.Then add the file using svn add command,then commit again.Once this is done you can check the svn info, let me know..
Use svn status command to know the status of the file
The only problem apart from this i can think of is this the directory may not have been added.Is this a new directory?
ah, the old tree conflict problem.
The issue is that SVN is letting you know that you're adding a file that used to be there but it cannot tell whether you're trying to delete it, add it or just update it! So it does the only thing it can - flags a conflict so you can sort it out and fix it. Its basically a conflict on the directory level (rather than a conflict of a file's contents).
What you do is resolve the error (as others have pointed out), then update the directory to get the original file back, then commit your changes. Note that the file was never deleted from SVN - its still in the repo, and if you checkout out a new WC, you'd get the file.
Try to resolve the conflicts then commit again:
svn resolve --accept working

Resources