I pushed .png files, each of which is 2+MB file size and tracked by git-lfs, to my gitlab.com repository, say repo_a. In CI job on another repo repo_b where git-lfs is installed, repo_a is cloned. Now I see the size of all .png files are 132, which seems to be the same volume as sha output (as the following. Note: some values are populated for privacy):
$ git show HEAD:file-a.png | tee sha_temp
version https://git-lfs.github.com/spec/v1
oid sha256:shashashaaaashashashaaaashashashaaaashashashaaaashashashaaaa
size 2430019
$ ls -l sha_temp
-rw-rw-r-- 1 crookednoodle crookednoodle 132 Nov 7 05:35 sha_temp
However, On my computer instead on Gitlab CI, I can see the original files when I git clone the repo_a.
This makes me feel that the content of these files are still pointers, not the original files. I also noticed that on my computer, I see in the output the original files are downloaded like this:
Downloading file-a.png (2.5 MB)
But I don't see this in the output on CI job.
Obviously related, subsequent process that opens the images by OpenCV fails.
What is wrong?
Managed to solve (get around) the issue by myself. In the targeted repo, I modified the CI script to run git lfs pull.
Related
I am trying to write a git pre-receive hook that rejects LFS files larger than a certain size (among other things).
I am trying to execute git lfs ls-files -l -s <new-ref-value> in my script, but it returns
2ec20be70bb1be824e124a61eabac40405d60de62c76d263eff9923f18c098ed - binary.dll (63 B)
Could not scan for Git LFS tree: missing object: a405ce05ac78ea1b820d036676831a474ddf8f90
I cannot even ignore the error message because it stops after the first file.
I guess that the problem has to do with the fact that the commits have not been "validated" on the remote yet. The frustrating thing is that the information that I need (new file paths + sizes) is accessible since it's printed for the first file.
Is there a way to run the git lfs ls-files command with the new ref value successfully at this stage?
Can I obtain the list of the added file paths and sizes in any other way?
EDIT: If that's relevant in any way, the Git server is a GitLab instance in its default configuration.
I uploaded some files to git LFS and I went over my storage limit. Now, the files I uploaded don't show up in the repo, but I'm still over my data limit and I can't upload anything.
Deleting local Git LFS files
You can delete files from your local Git LFS cache with the git lfs prune command:
$ git lfs prune
✔ 4 local objects, 33 retained
Pruning 4 files, (2.1 MB)
✔ Deleted 4 files
This will delete any local Git LFS files that are considered old. An old file is any file not referenced by:
the currently checked out commit
a commit that has not yet been pushed (to origin, or whatever
lfs.pruneremotetocheck is set to)
a recent commit
for details please go through this link https://www.atlassian.com/git/tutorials/git-lfs
Currently that is not possible via lfs command line. From the Atlassian Git LFS tutorial:
The Git LFS command-line client doesn't support pruning files from the server, so how you delete them depends on your hosting provider. In Bitbucket Cloud, you can view and delete Git LFS files via Repository Settings > Git LFS
GitHub even suggest recreating the repo:
To remove Git LFS objects from a repository, delete and recreate the repository. When you delete a repository, any associated issues, stars, and forks are also deleted.
But it is still good idea to use tools like BFG to clear out large files in history before moving around.
This question already has answers here:
How do I move my local Git repository to a remote Git repository
(9 answers)
Closed 6 years ago.
I apologize if this has been asked somewhere else and I missed it. I've found many questions and answers that are close to what I'm trying to do, but none that fit exactly.
I have local files and projects in a local git repository. I created the repository a few months back as a way to safeguard and track my own work. The rest of our company uses SVN so I am the first to use git here. I set it up in my home directory on a Linux server. This repo contains many commits, branches, and tags that span my work over the past few months.
I want to move that repo to a more accessible location within our company's Linux server (not in my home directory) so others can clone and work with the entire repo (including all the tags, branches, history, etc.) as they see fit.
I've thought about copying or moving the entire file structure, but I want it to be a single file - i.e. myRepo.git.
How do I do this so that myRepo.git contains all the information I've committed such that when someone clones this repo and runs 'gitk --all', they will see all the branches, tags, etc.?
Thanks for your help.
You can git clone --bare your repo from somewhere other people can access it.
The most basic remote Git repo is nothing but a bare clone. A bare clone means it doesn't have a checkout, it's just the contents of the .git directory. Having a checkout would mean people might try to use that checkout as their own git working directory and make changes and that would get confusing with other people pushing changes to the repository.
For example, I'll make a new repository in foo/ and commit a file.
$ git init foo
Reinitialized existing Git repository in /Users/schwern/tmp/foo/.git/
$ rm -rf foo
$ git init foo
Initialized empty Git repository in /Users/schwern/tmp/foo/.git/
$ cd foo
$ touch this
$ git add this
$ git ci -m 'First commit'
[master (root-commit) 1a0bddf] First commit
1 file changed, 0 insertions(+), 0 deletions(-)
create mode 100644 this
Then I'll make a bare clone of it into bar/ (cloning doesn't have to happen over a network).
$ git clone --bare foo bar
$ ls
HEAD config description hooks info objects packed-refs refs
Instead of the checkout, it has the contents of the .git directory.
Now I can clone from that.
$ git clone bar baz
Cloning into 'baz'...
done.
$ cd baz/
$ git log
commit 1a0bddf415f48d71fa3aacc79c07c233d91ed4a9 (HEAD -> master, origin/master, origin/HEAD)
Author: Michael G. Schwern <schwern#pobox.com>
Date: Tue Feb 14 12:01:13 2017 -0800
First commit
That's cloning on a local filesystem. Git will, of course, clone over a network. Typically it uses ssh or https. git clone user#host/path is really sshing into host as user and copying what's in path.
So you can put your repo anywhere the other folks have ssh access, like a dev server, or the same host that has the SVN server.
That's the most bare bones way to share your Git repository. There are plenty of more sophisticated Git servers out there that provide web access and more protocols. Everything from the very sparse GitWeb to Gitlab, a Github clone you can host locally for free.
You can read more about this in Pro Git's chapter on Git Servers.
A fully functional git repo is not a single file, and there is no way to make it so. That said, copying your working repo isn't the best thing to do either.
If having a single file is more important that having the repo be fully functional - i.e. you don't care if your coworkers can push to it - then you can create a bundle file.
git bundle create /path/to/my/repo.bundle --all
Your coworkers can clone from this, add it as a remote, pull or fetch from it... but they can't push to it (and neither can you; to add to it you'd have to recreate it).
A shared repo to be cloned from should normally be a bare repo. This is roughly equivalent to a copy of the .git directory from your working repo. If your existing repo is at /path/to/my/repo, you can create a suitable bare repo at /path/to/shared/repo by saying
cd /path/to/shared
git clone --mirror /path/to/my/repo
Note that I used --mirror instead of --bare; this has to do with how refs are copied and will work better if you intend this to be treated as the origin henceforth.
And yet at this point /path/to/my/repo is still considered the upstream. To fix that
cd /path/to/shared/repo
git remote remove origin
cd /path/to/my/repo
git remote add origin /path/to/shared/repo
I am new to git and I keep all my git repos in one folder inside an NTFS disk so that both Windows and Linux partitions can see it. The problem is that executable files in that folder can't be given execution permissions in Linux.
For the repos that contain executable files, I've been working in a directory of the Linux partition, so that I can execute them, and then once I'm done editing them, I copy them back to the relevant repo folder to keep version-controlling them.
Side note (for GolezTrol):
When I say executables, I mean, for example, *.cpp or *.java files. If I want to compile those on an NTFS partition, I will get an error if I try to run the executable. So I can't work in the NTFS folder, I have to work in the Linux partition. Think for example, the workspace folder in Eclipse. I can't make any app work if I place it in the NTFS folder. So I have to move it to the Linux partition, edit my files, and then copy the relevant files back to the repo folder in the NTFS partition. I'm not tracking the executables, I'm tracking the files that produce those executables. I hope this clears it up.
I was wondering if there is a way to keep the version-control of those files in their repo folder, while the files live in the Linux partition, so that I don't have to copy them back when I'm finished editing, and I can still see them from Windows. Something like a link to the executables folder that lives in the repos folder, maybe (it would probably be nice to be able to edit them from Windows too, as if they really lived in the repo folder, but I guess that won't be possible).
I searched SO and found a similar question. I tried the first answer, although it's not exactly what I'm aiming for, but it didn't work for me. I placed the .git file in my executables folder, containing:
gitdir: path/to/repo/in/NTFS/disk/.git
and then initialized the repo, but I got an error:
$ git init
fatal: Not a git repository: path/to/repo/in/NTFS/disk/.git
So I went back to the repo folder (in the NTFS disk), did git init and came back to the executables folder. Now I get a different error:
$ git init
fatal: Invalid gitfile format: path/to/executables/folder
I think I didn't understand the answer.
Regarding the second answer, I couldn't understand how to implement it either.
So, wrapping up, my question is... how can I have the executable files in a folder in the Linux partition but version control them from the repo folder in the NTFS disk? Would it be possible to make Windows see those files?
P.S. I would like to avoid (if possible) getting into mounting or editing fstab files.
P.S2. This question is different to the one I linked. I'm thinking more of a sort of link to the executables folder from the repo folder. It would be ideal to see the files from Windows too. If that is not possible, I'm open to other answers, since the answers in the linked question didn't work for me. Had those answers solved my problem, I wouldn't be asking this question.
Update:
My executables directory:
$ tree -L 1 -aF --dirsfirst
.
├── file.txt
└── .git
0 directories, 2 files
Contents of file.txt: file
Contents of .git: gitdir: /media/admin/DATA/github/af-62/.git
My NTFS directory (that is, /media/admin/DATA/github/af-62):
$ tree -L 1 -aF --dirsfirst
.
├── .git/
├── .gitignore
└── LICENSE.md
1 directory, 2 files
(Please don't pay too much attention to the files, they're placeholder files for now.)
The -a option shows hidden files, as those whose name starts with a dot, and the --dirsfirst shows directories first, so in the first case, .git is at the end while in the second .git is at the beginning.
In the NTFS directory:
$ git log
commit f8f7e3bac01f51e5d819a31e28c1a42c181b0407
Author: private <user#example.com>
Date: Wed Jul 15 11:50:44 2015 +0200
First commit.
$ git checkout /path/to/executables/folder/file.txt
fatal: /path/to/executables/folder/file.txt: '/path/to/executables/folder/file.txt' is outside repository
In the executables directory:
$ git init
fatal: Invalid gitfile format: /path/to/executables/folder/.git
$ git checkout file.txt
fatal: Invalid gitfile format: .git
$ git checkout /media/admin/DATA/github/af-62/LICENSE.md
fatal: Invalid gitfile format: .git
According to jvdm's linked resource, "[...the .git file...] It must point to a valid Git bare repository", but /media/admin/DATA/github/af-62/ is a valid repository, isn't it? What am I doing wrong?
You need to tell git that /path/to/executables/folder is the working
dir of the git repository /path/to/repo/in/NTFS/disk/.git, one way
to do it:
cd /path/to/executables/folder
echo gitdir: /path/to/repo/in/NTFS/disk/.git > .git
Now notice that a git status (if /path/to/executables/folder was
empty) will show all your files as deleted. Then you can:
git checkout .
To checkout them into your working dir.
I have an old Synology DS-106j server where I tried to install git using ipkg command. The installation went smoothly, but git failed to work correctly. I am currently learning how to use git, so I don't know if it is a bug from git with the version I am using or something else is wrong.
What I did was create a new local repository with a specified name, add a new file, commit it, and got an error:
NAS_SERVER> git init Test
Initialized empty Git repository in /root/Test/.git/
NAS_SERVER> ls
Packages.gz git_1.8.4.2-1_powerpc.ipk
Test
NAS_SERVER> cd Test
NAS_SERVER> git status
# On branch master
#
# Initial commit
#
nothing to commit (create/copy files and use "git add" to track)
NAS_SERVER> touch Test.cs
NAS_SERVER> ls
Test.cs
NAS_SERVER> git add *
NAS_SERVER> git status
# On branch master
#
# Initial commit
#
# Changes to be committed:
# (use "git rm --cached <file>..." to unstage)
#
# new file: Test.cs
#
NAS_SERVER> git commit -m "Test"
fatal: 57e2b8c52efba71d84c56bf6f37581686b9061a3 is not a valid object
I thought...maybe I did something wrong, so I used git on Windows OS and try a push. Still an error. Transfer the whole repository to the server and check the status. It seems fine. Try a commit. Still the same result. What worse is that I can't update git version without having to compile it, which I don't even know how to do so. Any suggestion to what might be wrong?
If your goal is to push into a git repo located on the synology disk(s) for backup purposes I'd recommend a different approach which would avoid having to install a rather old git version on the synology box itself (which could lead to problems if/when using a newer git version on the windows machine).
Export a samba share from synology, mount it on windows and use the windows git to create the backup repo (maybe even a bare repo, eventually group shared if you plan to share work with other people). Then push from your working repo into this backup repo - all on the windows box. In this scenario the synology box doesn't need git installed, it just serves files (i.e. its original job).
I'm using such setup but with a linux machine instead of a windows one and with the bare repo on the synology disks exported via NFS instead of Samba.