Here is a question keeps me puzzled for a long time as I cannot find nice and more-or-less universal solution to it.
Lets say there are two git branches available, production and dev; each uses it's own configurable parameters for some tasks (i.e. credentials, build path, test/deployment scripts switches et cetera). Same time implementation scripts & code is common for both branches.
Now, the problem arises is - how to maintain branch-specific configuration within git repository the way required. One of common solutions out there is to use 'template' configuration file within git, symlinking and ignoring the concrete for specific branch, like
$ cat .gitignore | grep conf
/concrete.conf
$ ls -l *.conf
lrwxrwxrwx 1 1000 100 12 Oct 29 10:23 concrete.conf -> generic.conf
-rw-r--r-- 1 1000 100 0 Oct 29 10:16 generic.conf
breaking and adjusting concrete.conf on development box next. Yet not a solution I'm in pursuit for.
My requirements (ok, wishes) are:
support separate configuration file per git branch, and
keep branch-specific configuration file managed by git same time
Is it even possible? Could be (actually, preferred to be file sourced by some other one, but it's none related...)
The best that has came to my mind so far is to use post-merge hooks to adjust per-branch configuration per se from some other source ignored by git, but it stinks from the very beginning.
Any suggestions on solution of problem described ?
PS: *nix-specific suggestions (i.e. using symlinks/hardlinks) suggestions is absolutely fine, I do do have any interest in M$ targets
It is possible, and does not involve symlinking.
You do not version my.config, but a template file my.config.tpl, and a value file (with values for each branches)
prod.conf
dev.conf
Then, you can use a content filter driver, using .gitattributes declaration.
(image from "Customizing Git - Git Attributes", from "Pro Git book")
The script generate my.config file by replacing placeholder values with the values of the <branch.conf> file, each time you checkout a branch.
You can know about the current branch name with:
branch=$(git rev-parse --symbolic --abbrev-ref HEAD)
The generated actual my.config remains ignored (by the .gitignore).
That means your actual working tree does not get "dirty".
The smudge script selects the correct value file and generates the correct web.config based on the template the smudge script is applied on during a git checkout.
See a complete example at "git smudge/clean filter between branches".
Related
Our Laravel project is using symlinks. Recently when I tried to pull from my colleague's work, I get this message:
CONFLICT (modify/delete): resources/lang/en/validation.php deleted in HEAD and modified in a262067feb430a072c1d3abf2ec500150212ff0f. Version a262067feb430a072c1d3abf2ec500150212ff0f of resources/lang/en/validation.php left in tree.
error: failed to symlink 'resources/lang/en/validation.php': File name too long
Upon trying to git rm the file, I am told it doesn't exist and is deleted in HEAD. Then when I pull I get the same message as above. Upon trying to touch the file and git add the file, and commit and then pull (in order to push my changes to the same branch), I get a similar error message:
CONFLICT (content): Merge conflict in resources/lang/en/validation.php
CONFLICT (modify/delete): resources/lang/en/auth.php deleted in HEAD and modified in a262067feb430a072c1d3abf2ec500150212ff0f. Version a262067feb430a072c1d3abf2ec500150212ff0f of resources/lang/en/auth.php left in tree.
error: failed to symlink 'resources/lang/en/auth.php': File name too long
I have tried to skip-worktree the file, assume-unchanged the file and to change the git config setting via git config --local core.longpaths true to allow long-paths. None have worked. I think it has to do with the symlink, but I haven't run the script yet and so I don't know how this is a barrier to pulling for git.
When I do try to run the symlink, I get this error message:
error: unable to create symlink resources/lang/en/auth.php: File name too long
error: unable to create symlink resources/lang/en/validation.php: File name too long
Long story short, I cannot git pull, and therefore cannot git push. What's the solution? I don't want to git push force it.
Running git pull is just running two Git commands:
First, git pull runs git fetch. This obtains any new commits needed for the second command.
Second, git pull runs ... well, this can be complicated. You are having it run the default, though: git merge.
Usually when git pull fails, one of these two commands that it runs is the one that actually failed. The second command fails more often unless you have a particularly flaky Internet connection. In your case, it's the git merge that failed.
The word failed is usually too strong, really. Most merges do not actually fail. They just stop in the middle of the operation due to a conflict (or two conflicts, in your particular case). But your merge is a little special. It really does have an internal failure, which repeats several times:
error: unable to create symlink resources/lang/en/auth.php: File name too long
error: unable to create symlink resources/lang/en/validation.php: File name too long
This is happening because your OS is placing a hard limit on the length of the target of a symbolic link. As you found:
It seems it was trying to make a symlink out of the content inside the file instead of the file name ...
Git's internal limits are much bigger than your OS's.
A symbolic link is just data, at one level, and that's how Git tends to store it (as a blob object, but one with mode 120000 rather than the normal file mode of 100644 or 100755). At another level, the data will be interpreted as a file name, and that file name tends to have a length limit, such as 1024 or 4096 bytes.
What would git show do?
git show will spill out the contents of the symlink, when pointed to a symbolic-link object.
$ git hash-object -w -t blob /usr/share/misc/termcap
d305cd8e161ecc8a78b0485d1926b9600efc6cb7
$ git update-index --add --cacheinfo 120000,d305cd8e161ecc8a78b0485d1926b9600efc6cb7,crazy
$ git commit -m "add crazy-long symlink"
[master dbb6e35] add crazy-long symlink
1 file changed, 4725 insertions(+)
create mode 120000 crazy
The normal tools will no longer work with this repository (which I made just to hold this crazy-long symlink):
$ git log | sed 's/#/ /'
commit dbb6e35967041fa4b03812866999ea0acd640dce
Author: Chris Torek <chris.torek gmail.com>
Date: Sun Nov 15 19:52:05 2020 -0800
add crazy-long symlink
commit c6e238c122dcd41410e7fdcfaa47ac112e935a35
Author: Chris Torek <chris.torek gmail.com>
Date: Sun Nov 15 19:51:58 2020 -0800
initial commit
$ git checkout HEAD^
This works fine, but trying to check out the second commit fails:
$ git checkout master
error: unable to create symlink crazy: File name too long
D crazy
Previous HEAD position was c6e238c initial commit
Switched to branch 'master'
What happens at this point is that Git simply leaves the symbolic link out of the working tree entirely. That's why it is in state D. You can still do work with the repository, but you cannot use the regular tools in the regular way.
With your merge, what you can do is delete the bad symbolic links entirely (safely), create correct (good) ones, and add them.
I am building a web application for an online "build your own" card game. In the application, I have a cards.json file that holds custom card data. This file is changed with fs whenever a user creates a card. Whenever I push local changes, the cards.json file gets overwritten on deploy. That means all the remote data gets lost on every deploy. How can I include a cards.json file remotely but not change the file whenever I push changes using git push heroku master?EDIT: I guess for clarification reasons, I have tried using a .gitignore as well as removing the file from the staging area. I'm not entirely sure, but I think the issue is that when the application is deployed the file is overwritten there.
So I just found out that the data created during runtime will always be deleted/reset.
https://devcenter.heroku.com/articles/dynos#ephemeral-filesystem
I guess the best fixes for anyone else who has this same issue are:
a) Look into Databases and Heroku Add-ons, or
b) This is very workaround, and there might be better ways to do it, but:
// Go into a new directory, and use
$ heroku ps:copy <FILENAME> --app <APPNAME>
// Then, copy+paste the data from this file into your main repo.
/* Now, each time you do this, you need to make sure you delete that file from the
* extra directory you created as ps:copy only works when the file doesnt exist locally.
*/
I think git fetch doesn't work in this instance, as it only pulls that unchanged file, rather than the changed one from the dyno.
Look up the .gitignore file in git, seems to me that's exactly what you're looking for.
If it doesn't recognize .gitignore properly at first:
git add [uncommitted changes you want to keep] && git commit
git rm -r --cached .
git add .
git commit -m "fixed untracked files"
In .gitignore add the cards.json along with the path .
eg. src/test/resources/testdata/cards.json
I have Serverless application that has two stages: develop (on branch develop) and master (on branch master).
It has a file called config.js like this:
API_URL: api.prod.some-server.com
But I want that "prod" changed to "dev" when I'm on develop branch / on develop stage environment.
It's kinda exhausting if I should change the API_URL back-and-forth (prod -> dev) when I changed the current branch/dev_stage.
Is there any possible to automate this?
Thank you
Is there any possible to automate this?
Yes: You would need to not version config.js (keep it private), but to generate it (with the right value in it).
The generation script will determine the name of the checked out branch with:
branch=$(git rev-parse --symbolic --abbrev-ref HEAD)
Or (since Git 2.22, Q2 2019)
branch=$(git branch --show-current)
That means you could:
version only a template file config.js.tpl
version value files named after the branches: config.js.dev, config.js.master: since they are different, there is no merge issue when merging or switching branches.
Finally, you would register (in a .gitattributes declaration) a content filter driver.
(image from "Customizing Git - Git Attributes", from "Pro Git book")
The smudge script, associated to the template file (config.js.tpl), would generate (automatically, on git checkout) the actual config.js file by looking values in the right config.js.<branch> value file.
The generated actual config.js file remains ignored (by the .gitignore).
See a complete example at "git smudge/clean filter between branches".
Symptoms
I have a strange issue where I want to checkout a different branch <branch2>, but when I try I get the message that
$ git checkout <branch2>
error: The following untracked working tree files would be overwritten by checkout:
<filename.xlsx>
Please move or remove them before you switch branches.
Aborting
However git ls-files lists <filename.xlsx> as a tracked file.
Workaround
If I do a git checkout -f <branch2> it works, and the <branch2> version of the excel file is present and correct (similarly if I rm <filename.xlsx> and then checkout I get the correct file.)
To move back to <branch1> I get the exact same issue (<filename.xlsx> is tracked, but for checkout appears to be untracked).
Testing/Troubleshooting
Investigating further this seems to be related to a Windows/Linux issue:
I work predominantly in a VirtualBox VM of Debian, so I use my git tooling and terminal there, and access via a VirtualBox shared folder.
Some of my code (this repository included) lives on the host system (Windows 10) as it is platform specific.
If I use Git for Windows this issue does not occur.
If I clone to a linux filesystem this issue does not occur.
If I clone from upstream to a different Windows folder and access from linux I get the same issue.
Other branches in the same repository that have the same file with different contents again do not suffer from the same issue (ie checkouts work as expected)
Other commits with the same version of the file have the same issue.
A small repo with just these two file versions does not exhibit the same behaviour, so, unfortunately, no easy way to create MCVE.
Final Thoughts
Honestly, it is something that is relatively easy to work around, and seems limited in scope so far to only these two copies of the , but it has me intrigued. If anyone with greater knowledge of git has any ideas on how to diagnose/fix the root cause of this issue I would be very grateful.
Extra debugging
Here is the results of git ls-files --debug from <branch1>
CodeSheet.xlsx
ctime: 1489465633:751038200
mtime: 1489465633:751038200
dev: 38 ino: 5432
uid: 0 gid: 999
size: 14752 flags: 0
Here is the results of git ls-files --debug from <branch2>
Codesheet.xlsx
ctime: 1489467487:720851100
mtime: 1489467487:720851100
dev: 38 ino: 5502
uid: 0 gid: 999
size: 12546 flags: 0
Just a guess: this could be a case of a file with a different case but the same name.
From the case-insensitive Windows OS perspective, it is tracked, but from a case-sensitive Debian OS perspective, it is not tracked.
Try:
git config core.ignorecase true
The OP Gavin mentions in the comments using "Changing capitalization of filenames in Git", with the git mv command.
We are working on integrating GitLab (enterprise edition) in our tooling, but one thing that is still on our wishlist is to create a merge request in GitLab via a command line (or batchfile or similar, for that matter). We would like to integrate this in our tooling. Searching here and on the web lead me to believe that this is not possible with native GitLab, but that we need additional tooling for that.
Am I correct? And what kind of tooling would I want to use for this?
As of GitLab 11.10, if you're using git 2.10 or newer, you can automatically create a merge request from the command line like this:
git push -o merge_request.create
More information can be found in the docs.
It's not natively supported, but it's not hard to throw together. The gitlab API has support for opening MR: https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/merge_requests.md#create-mr
You can use following utility.
Disclosure : I developed it.
https://github.com/vishwanatharondekar/gitlab-cli
You can create merge request using this.
Some of the features it has are.
Base branch is optional. If base branch is not provided. Current branch is used as base branch.
target branch is optional. If target branch is not provided, default branch of the repo in gitlab will be used.
Created pull request page will be opened automatically after successful creation.
If title is not supported with -m option value. It will be taken from in place editor opened. First line is taken as title.
In the editor opened third line onwards takes as description.
Comma separated list of labels can be provided with its option.
Supports CI.
Repository specific configs can be given.
squash option is available.
remove source branch option is available.
If you push your branch before this command (git push -o merge_request.create) it will not work. Git will response with Everything up-to-date and merge request will not be created (gitlab 12.3).
When I tried to remove my branch from a server (do not remove your local branch!!!) then it worked for me in this form.
git push --set-upstream origin your-branch-name -o merge_request.create
In addition to answering of #AhmadSherif, You can use merge_request.target=<branch_name> for declaring the target branch.
sample usage:
git push -o merge_request.create -o merge_request.target=develop origin feature
Simple This:
According to the Gitlab documents, you can define an alias for this command to simpler usage.
git config --global alias.mwps "push -o merge_request.create -o
merge_request.target=master -o merge_request.merge_when_pipeline_succeeds"
I made a shell function which opens up the GitLab MR web page with desired parameters.
Based on the directory with the git repo you are currently in, it:
Finds the correct URL to your repo.
Sets the source branch to the branch you're currently on.
As a optional first argument you can provide the target branch. Otherwise, GitLab defaults to your default branch, which is typically master.
gmr() {
# A quick way to open a GitLab merge request URL for the current git branch
# you're on. The optional first argument is the target branch.
repo_path=$(git remote get-url origin --push | sed 's/^.*://g' | sed 's/.git$//g')
current_branch=$(git rev-parse --abbrev-ref HEAD)
if [[ -n $1 ]]; then
target_branch="&merge_request[target_branch]=$1"
else
target_branch=""
fi
xdg-open "https://gitlab.com/$repo_path/merge_requests/new?merge_request[source_branch]=$current_branch$target_branch"
}
You can set more default values in the URL, like removing the source branch after merge:
&merge_request[force_remove_source_branch]=true
Or assignee to someone:
&merge_request[assignee_ids][]=12345
Or add a reviewer:
&merge_request[reviewer_ids][]=54321
You can easily find the possible query string parameters by searching the source of the GitLab MR webpage for merge_request[.
As of now, GitLab sadly does not support this, however I recently saw it on their issue tracker. It appears one can expect a 'native tool' in the upcoming months.
GitLab tweeted out about numa08/git-gitlab some time ago, so I guess this would be worth a try.
In our build script we just pop up the browser with the correct URL and let the developer write his comments in the form hit save to create the merge request. You get this url with the correct parameters by creating a merge request manually and copying the url of the form.
#!/bin/bash
set -e
set -o pipefail
BRANCH=${2}
....
git push -f origin-gitlab $BRANCH
open "https://gitlab.com/**username**/**project-name**/merge_requests/new?merge_request%5Bsource_branch%5D=$BRANCH&merge_request%5Bsource_project_id%5D=99999&merge_request%5Btarget_branch%5D=master&merge_request%5Btarget_project_id%5D=99999"
You can write a local git alias to open a Gitlab Merge Request creation page in the default browser for the currently checked-out branch.
[alias]
lab = "!start https://gitlab.com/path/to/repo/-/merge_requests/new?merge_request%5Bsource_branch%5D=\"$(git rev-parse --abbrev-ref HEAD)\""
(this is a very simple alias for windows; I guess there are equivalent replacements for "start" on linux and fancier aliases that work with github and bitbucket too)
As well as being able to immediately see&modify the details of the MR, the advantage of this over using the merge_request.create push option is that you don't need your local branch to be behind the remote for it to work.
You might additionally want to store the alias in the repo itself.
I use https://github.com/mdsb100/cli-gitlab
I am creating the MR from inside of a gitlab CI docker container based on alpine linux, so I include the install command in before-script (that could also be included in your image). All commands in the following .gitlab-ci.yml file, are also relevant for normal command line usage (as long as you have the cli-gitlab npm installed).
variables:
TARGET_BRANCH: 'live'
GITLAB_URL: 'https://your.gitlab.net'
GITLAB_TOKEN: $PRIVATE_TOKEN #created in user profile & added in project settings
before-script:
-apk update && apk add nodejs && npm install cli-gitlab -g
script:
- gitlab url $GITLAB_URL && gitlab token $GITLAB_TOKEN
- 'echo "gitlab addMergeRequest $CI_PROJECT_ID $CI_COMMIT_REF_NAME \"$TARGET_BRANCH\" 13 `date +%Y%m%d%H%M%S`"'
- 'gitlab addMergeRequest $CI_PROJECT_ID $CI_COMMIT_REF_NAME "$TARGET_BRANCH" 13 `date +%Y%m%d%H%M%S` 2> ./mr.json'
- cat ./mr.json
This will echo true if the merge request already exists, and echo the json result of the new MR if it succeeds to create one (also saving to a mr.json file).
Since GitLab 15.7 (Dec. 2022), the GitLab CLI glab is officially integrated to GitLab.
Introducing the GitLab CLI
The command line is one of the most important tools in a software engineer’s toolkit and the majority of their process and work revolve around tools available there. They customize their CLI with styles and extend it through applications to ensure maximum efficiency while performing tasks. The CLI is the backbone of scripts and workflows developers depend on to complete their work.
To support more developers where they’re already working, we’ve adopted the open source project glab, which will form the foundation of GitLab’s native CLI experience.
The GitLab CLI brings GitLab together with Git and your code, with no application or tab switching required.
You can read about our adoption of glab, our partnership with 1Password, and how to contribute to the project in our blog post.
A special thank you to Clement Sam for creating glab and trusting us with its future.
That means you can create a MR with glab mr create:
glab mr create -a username -t "fix annoying bug"