How to prevent git from committing two files with names differing only in case? - linux

We develop in a mixed environment - some people work on Macs and some work on Linux. This has proven to be a bit of a challenge at times, as those people who work on Linux are used to having their filesystems be case sensitive, so there's no issue committing (accidentally or otherwise) multiple files differing just by case. (e.g. FileName.ext versus filename.ext)
However, when the people on Macs go to check out the repository, having a case-insensitive filesystem means that the two files - differing only in case - overwrite each other and cause general havoc.
I know that there are various git settings to help people on case-insensitive filesystems work better with case changes (e.g. core.ignorecase), but these don't solve the issue where there's two different files in the repository, only differing by case.
I realize that the only way to fix it is to make sure the Linux people don't commit the two files differing only in case in the first place. -- Is there some setting in git which will pop up a warning or error if a user on a case-sensitive filesystem attempts to commit file(s) which would be confused with each other on a case-insensitive filesystem?

There's nothing built in (although there should be, no doubt). What you can do is provide a pre-commit hook that verifies that all names are OK and prevents the commit if not.
This hook only needs to be run on the Linux box (although making it work on Linux and Mac is easy, it's just Windows with its default impoverished toolbox that is problematic). You might want to add it to a side branch and give the Linux folks instructions on setting it up.
You may want to check branch names as well, as in git pre-commit or update hook for stopping commit with branch names having Case Insensitive match. (Interesting: the answer on this question is my own; I had forgotten it.)
First, let's write a "check for case conflict" function. This is just a matter of sorting with case-folding (so that "helloworld" and "helloWorld" are placed adjacent to each other), then using uniq -di to print any duplicate (after case-folding) strings, but no non-duplicates:
sort -f | uniq -di
If this produces any output, these are the "bad names". Let's capture the output in a temporary file and check its size, so we can print them to standard output as well:
#! /bin/sh
TF=$(mktemp)
trap "rm -f $TF" 0 1 2 3 15
checkstdin() {
sort -f | uniq -di > $TF
test -s $TF || return 0 # if $TF is empty, we are good
echo "non-unique (after case folding) names found!" 1>&2
cat $TF 1>&2
return 1
}
Now we just need to use it on files that will be committed, and perhaps on branch names as well. The former are listed with git ls-files, so:
git ls-files | checkstdin || {
echo "ERROR - file name collision, stopping commit" 1>&2
exit 1
}
You can fancy this up to use git diff-index --cached -r --name-only --diff-filter=A HEAD to check only added files, allowing existing case collisions to continue, and/or try to check things across many branches and/or commits, but that gets difficult.
Combine the above two fragments into one script (and test) and then simply copy it to an executable file named .git/hooks/pre-commit.
Checking branch names is a bit trickier. This really should happen when you create the branch name, rather than when you commit to it, and it's impossible to do a really good job on the client—it has to be done on a centralized server that has a proper global view.
Here is a way to do it on the server in a pre-receive script, in shell script rather than in Python (as in the linked answer). We still need the checkstdin function though, and you might want to do it in an update hook rather than a pre-receive hook, since you don't need to reject the entire push, just the one branch name.
NULLSHA=0000000000000000000000000000000000000000 # 40 0s
# Verify that the given branch name $1 is unique,
# even IF we fold all existing branch names' cases.
# To be used on any proposed branch creation (we won't
# look at existing branches).
check_new_branch_name() {
(echo "$1"; git for-each-ref --format='%(refname:short)' refs/heads) |
checkstdin || {
echo "ERROR: new branch name $1 is not unique after case-folding" 1>&2
exit 1 # or set overall failure status
}
}
while read oldsha newsha refname; do
... any other checks ...
case $oldsha,$refname in
$NULLSHA,refs/heads/*) check_new_branch_name ${refname#refs/heads/};;
esac
... continue with any other checks ...
done

Related

How to visually compare two revisions of a folder versioned by SVN on Linux?

I can compare a current folder state to its latest revision using the following command:
meld .
meld shows a list of changed files. One can click on each file and see a diff.
Is it possible to compare a current folder state to its specific revision (not the latest one)?
TortoiseSVN allows to compare arbitrary revisions, however it works on Windows only. SmartSVN is proprietary. SVN CLI is unusable for big changesets (CLI diff is fine for small commits, but it's very hard to use it for branch comparision with a lot of changes).
Maybe I could checkout the revision to compare into a separate folder and compare two folders using meld. But I guess there should be a simpler approach.
Preface
Question is offtopic here: as already mentioned in comment, it's question for Software Recommendations site
Face
Every versioned object with history in SVN can be referenced using PEG-revision for its history state
Folder in SVN-repo is object of versioning
In order to compare two folders, you have to have folder-diff tool (for your OS) and know (command-line) options for calling it
According to Slant's comparison:
Meld can be used on Linux for folder-diffing
Best folder-diff tool is Beyond Compare
From points 1-3 above it follows that Meld can be used for your task in form
meld folder#REV1 folder#REV2
Folder comparision is usually required for code review or branch merging. It seems that the simplest approach is as follows:
Find last trunk revision merged to a current brach
If the current branch was not merged from trunk, then find the branch creation revision
Checkout trunk with a specified revision to some folder
Compare the trunk folder and the brach folder
I didn't found any existing tools supporting it. Here is a bash script I use, maybe it will be useful for someone:
TRUNK_BASE_PATH=~/tmp
BRANCH_RELATIVE_URL=$(svn info --no-newline --show-item relative-url)
if [[ "$BRANCH_RELATIVE_URL" != *"/branches/"* ]]; then
echo "Run it in a branch. Relative URL should contain /branches/. Given: $BRANCH_RELATIVE_URL"
exit 1
fi
TRUNK_RELATIVE_URL=${BRANCH_RELATIVE_URL%%/branches/*}/trunk
echo "Trunk relative URL: $TRUNK_RELATIVE_URL"
ROOT_URL=$(svn info --no-newline --show-item repos-root-url)
TRUNK_URL=${ROOT_URL}${TRUNK_RELATIVE_URL:1}
echo "Trunk URL: $TRUNK_URL"
TRUNK_PATH=${TRUNK_BASE_PATH}/${TRUNK_URL#*://}
echo "Trunk local copy path: $TRUNK_PATH"
BRANCH_PATH=$(svn info --no-newline --show-item wc-root)
echo "Branch local copy path: $BRANCH_PATH"
SUBFOLDER=$(realpath --relative-to="$BRANCH_PATH" .)
echo "Comparing subfolders: $SUBFOLDER"
TRUNK_REVISION=$(svn mergeinfo --show-revs merged -R "$TRUNK_RELATIVE_URL" "$BRANCH_PATH" | tail -n 1)
if [[ -z "$TRUNK_REVISION" ]]; then
TRUNK_REVISION=$(svn log -r 1:HEAD --limit 1 --stop-on-copy -q | grep -oP "^r\K[0-9]+")
echo "Comparison with trunk#$TRUNK_REVISION from which the current branch was copied"
else
echo "Comparison with trunk#$TRUNK_REVISION with which the current branch was last merged"
fi
if [[ -d "$TRUNK_PATH/.svn" ]]; then
echo "Found .svn subfolder in the local trunk copy, updating it"
svn update -r $TRUNK_REVISION "$TRUNK_PATH"
else
echo "Not found .svn subfolder in the local trunk copy, checking out it"
svn checkout "$TRUNK_URL" -r $TRUNK_REVISION "$TRUNK_PATH"
fi
meld "$TRUNK_PATH/$SUBFOLDER" . &!

Execute p4 aliased command result in messed wrong order of lines in output

I have the following content inside my p4aliases.txt.
diff-cl $(target-cl) = diff -dl //...#$(EQ)$(target-cl)
Basically it diffs against your files in current workspace toward the target shelved files of changelist.
It is fine. I can execute it. But when I compare the result coming from above aliased command against the direct raw (non-aliased) command as follows
p4 diff -dl //...#=<target-cl>
the output lines of text from aliased command is in wrong order e.g. changes according to a certain file shows up first before a line of file shown, line orders are messed up. This is not the case if you execute with a non-aliased command.
Example
Expected result
==== //depot/common.h#none - x:\mydir\project\src\common.h ====
==== //depot/file.cpp#none - x:\mydir\project\src\file.cpp ====
3a4
> added line 1
==== //depot/file.h#none - x:\mydir\project\src\file.h ====
Actual result
3a4
> added line 1
==== //depot/common.h#none - x:\mydir\project\src\common.h ====
==== //depot/file.cpp#none - x:\mydir\project\src\file.cpp ====
==== //depot/file.h#none - x:\mydir\project\src\file.h ====
I have p4 version as of Rev. P4/NTX64/2021.1/2126753 (2021/05/12).
Perforce server version (got from p4 info) is Server version: P4D/LINUX26X86_64/2017.1/1574018 (2017/10/02).
How can I solve this issue?
Could this be a version too far away between client and server
Update
I have tested p4 client all the way down from 2016-2020 version by downloading old binaries from ftp.perforce.com (in directory perforce). No luck. Output still messed the same. So it's not the problem about version mismatch.
This looks like a bug in the p4 client. When the client does a diff, it's written by the ClientUser::Diff() method, which defaults to writing to stdout (i.e. it does not route the output through ClientUser::OutputText()):
https://workshop.perforce.com/projects/perforce_software-p4/files/2018-2/client/clientuser.cc#436
https://workshop.perforce.com/projects/perforce_software-p4/files/2018-2/client/clientuser.cc#573
Output from commands run as part of an alias go through the ClientUserStrBuf subclass, which buffers all of its output. The file headers, for example, are buffered by ClientUserStrBuf::OutputInfo():
https://workshop.perforce.com/projects/perforce_software-p4/files/2018-2/client/clientaliases.cc#1647
There isn't a ClientUserStrBuf::Diff() implementation, though, so that diff output goes straight to stdout while the headers are buffered and printed at the end (presumably after some post-processing) -- hence the diff output showing up first in the console.
The fix I'd make would be to have the base ClientUser::Diff() implementation route the output through OutputText() when no output file is provided, which seems like the least-surprise behavior; that'd fix the aliases behavior and might even make life a little easier for other client developers who would otherwise hit the same issue. If you have a support contract with Perforce you can file this as a bug report, or since the client is open source you can take a crack at fixing and building it yourself. I don't think there's a workaround that doesn't involve modifying the client source code.
Samwise has the correct approach to truly fix the problem at hands although it might take some effort to understand the code, and conduct the fix itself.
At any rate, if we took such approach we won't be able to take benefits of bug fixes and future updates as we will be stuck with 2018-2 version of p4 as it's the latest as it can be in which we can grab the source.
I would recommend to use WSL then interact with p4.exe (yes, a Windows-based binary) for Windows-based project, and p4 for Linux-based binary. If you didn't use WSL, please find the .bash_aliases-like solution as I have below to seamlessly solve aliases diff operation.
Put the following code into your ~/.bash_aliases
# p4 - fix of aliases diff operation
# platform independent, it will choose a correct binary path to execute properly
p4() { cmd="p4.exe" # default is Windows-based
# get the last argument value, if "-lx" passed in then we know it's linux
if [[ "${#: -1}" == "-lx" ]]; then
cmd="/usr/local/bin/p4"
fi
if [[ $1 == "diff-cl" ]]; then
if [ -z "$2" ]; then
echo "usage: p4 diff-cl <CL>"
return 1
fi
$cmd diff -dl //...#=$2 | diffp4 | less -r
elif [[ $1 == "diff-cl-fonly" ]]; then
if [ -z "$2" ]; then
echo "usage: p4 diff-cl-fonly <CL>"
return 1
fi
$cmd diff -Od -dl -ds //...#=$2 | diffp4 | grep ==== | less -r
else
$cmd "$#"
fi
}
then source ~/.bash_aliases.
What it does is to allow you to still use p4 with all of its original commands & arguments normally with exceptions of diff-cl (which is the same name of alias I've put into p4aliases.txt for Windows or ~/.p4aliases for Linux). You can safely remove diff-cl entry from p4's alias file, or just leave it there. What we have in ~/.bash_aliases file will intercept whenever such argument matches then execute the raw command, just that we don't have to type long command ourselves.
We can later remove such section in our ~/.bash_aliases file when upstream p4 has been fixed.
In else section, we just relay the the whole arguments, and it will be performed just as normally done.
Extra: diff-cl-fonly to list out only files (its depot path, and local workspace path) which have changes.

How can I get the last merged branch name in git

How can i get the last merged branch name in git from the remote
Tried
git log --first-parent --merges -1 --oneline
But not getting the branch name
Please help
In general, you cannot.
A merge commit may have, as its commit message, text of the form merge branch foo or merge branch foo of someurl, but to read that message, you must obtain the commit from the remote. Even so, there's no guarantee that branch foo exists any more, or that the name means anything if it does exist. Merging really works by commit hash IDs, not by branch names; branch names may evanesce.
If you think you need a branch name here, you are probably doing something wrong. Branch names do make sense here in other version control systems, but in Git, depending on them is unwise.
Here is the command you need to give (change the branch name from origin/master to whichever branch you're checking merges for):
git log --merges origin/master --oneline --grep='^Merge' -1 | grep -oe "[^']*[^']" | sed -n 2p
I had quite a bit of a hard time trying to solve this issue.
I had to get this information for my CircleCi pipeline, and to solve my problem I used the GitHub cli.
Specifically the pr list command: https://cli.github.com/manual/gh_pr_list
The following command gives you all the information of the last PR you merged into main
gh pr list -s merged -B main -L 1
Then you need to manipulate the string and get only the branch name. Which is the text before "MERGED"
I took it splitting the string and taking the penultimate element.

Redirect to file mysteriously does nothing in Bash

Background
I've a script. It's purpose is to generate config files for various system services from templates whenever my gateway acquires a new IP from my ISP. This process includes making successive edits with sed to replace $[template] strings in my custom templates with the correct information.
And to do that I've created a small function designed to take input from stdin, redirect it to a temporary file passed as an argument, and then move that file to replace the destination (and also, often, source) config file. The "edit-in-place dance", if you will.
I created a simple test script with the problematic function:
#!/bin/bash
inplace_dance() {
read -r -d '' data
printf '%s' "${data}" > "${1}~"
mv "${1}~" "${1}"
}
# ATTN: ls is only being used to generate input for testing. It is not being parsed.
ls -l ~/ | inplace_dance ~/test.out
Unfortunately, this works. So it's not the function itself. I also tried it with my custom logging utility (see "complications" below):
#!/bin/bash
. /usr/local/lib/logging.bash
log_identifier='test'
log_console='on'
inplace_dance() {
read -r -d '' data
printf '%s' "${data}" > "${1}~"
mv "${1}~" "${1}"
}
# ATTN: ls is only being used to generate input for testing. It is not being parsed.
bashlog 'notice' $(ls -l ~/ | inplace_dance '/home/wolferz/test.out')
This also works.
The Problem
In its original context, however, it does not work. I've confirmed that ${data} gets set just fine. And that ${1} contains the correct filename. What fails is the second line of the function. I've confirmed printf is being run (see, "Additional Info - Without The Redirect" below)... but the file its output is being redirected to is never created.
And I've been over the code a dozen-dozen times (not an exaggeration) and have yet to identify the cause. So, in desperation, I'm going to post it here and hope some kind soul will wade through my code and maybe spot the problem that I'm missing. I'd also happily take advice on improvements/replacements to my logging utility (in the hopes of tracking down the problem) or further troubleshooting steps.
Here is the original context. The important lines are 106-110, 136-140, 144-147, and 151-155
Additional Info
☛ PATH/Environment
The PATH is PATH=/usr/local/sbin:/usr/local/bin:/usr/bin. I believe this is being inherited from systemd (systemd=>dhcpcd.service=>dhcpcd=>dhcpcd-run-hooks=>dhcpcd.exit-hook).
dhcpcd-run-hooks (see "Complications" below) does clear the environment (keeping the above PATH) when it runs. Thus, I've added an example of the environment the script runs in to the "original context" gist. In this case, the environment when $reason == 'BOUND'. This is output by printenv | sort at the end of execution (and thus should show the final state of the environment).
NOTE: Be aware this is Arch Linux and the absence of /bin, /sbin, and /usr/sbin in the PATH is normal (they are just symlinks to /usr/bin anyway).
☛ Return Code
Inserting echo $? after the second line of the function gives me a return code of "0". This is true both with the redirect in line 2 and without (just the printf).
☛ Without The Redirect
Without the redirect, in the original context, the second line of the function prints the contents of ${data} to stdout (which is then captured by bashlog()) exactly as expected.
⚠️ Execute Instead of Source.
Turns out that $0 was /usr/lib/dhcpcd/dhcpcd-run-hooks rather than my script. Apparently dhcpcd-run-hooks doesn't execute the script... it sources it. I made some changes to line 196 to fix this.
♔ Aaaaaand that seems to have fixed all problems. ♔
I'm trying to confirm that was the silver bullet now... I didn't notice it was working till I had made several other changes as well. If I can confirm it I'll submit an answer.
Complications
What complicates matters quite a bit is that it's original context is a /etc/dhcpcd.exit-hook script. dhcpcd-run-hooks appears to eat all stderr and stdout which makes troubleshooting... unpleasant. I've implemented my own logging utility to capture the output of commands in the script and pass it to journald but it's not helping in this case. Either no error is being generated or, somehow, the error is not getting captured by my logging utility. The script is running as root and there is no mandatory access control installed so it shouldn't be a permissions issue.

Don't add "+" to linux kernel version

I am building linux kernel, if my kernel under git, then kernel version every time is:
Image Name: Linux-2.6.39+
If I am not using git, then everything is OK without any plus at the end.
I know that this done by scripts/setlocalversion script:
if test "$CONFIG_LOCALVERSION_AUTO" = "y"; then
# full scm version string
res="$res$(scm_version)"
else
# append a plus sign if the repository is not in a clean
# annotated or signed tagged state (as git describe only
# looks at signed or annotated tags - git tag -a/-s) and
# LOCALVERSION= is not specified
if test "${LOCALVERSION+set}" != "set"; then
scm=$(scm_version --short)
res="$res${scm:++}"
fi
fi
So, it is possible without code changes say to build system that no need to add "+" at the end of version line?
The plus sign at the end of your version string is there as an indicator that the kernel was built from modified sources (that is, there were non-committed changes). This is also indicated by the comments in scripts/setlocalversion.
To avoid the '+' being appended despite having a dirty working directory, simply set LOCALVERSION explicityly when running make:
make LOCALVERSION=
You may also have to change the configuration option CONFIG_LOCALVERSION_AUTO to n in your kernel config (.config) before building:
sed -i "s|CONFIG_LOCALVERSION_AUTO=.*|CONFIG_LOCALVERSION_AUTO=n|" .config
To prevent the script scripts/setlocalversion to append the + to the end of the kernel local version, create an empty .scmversion file in the root of the kernel sources.
touch .scmversion
this way, you'll be able to leave LOCALVERSION as is in the kernel configuration file, in case you want to append a local signature to the kernel name.
Manipulating scripts/setlocalversion seems to be the only way for me.
Force return in scm_version():
scm_version()
{
local short
short=false
**return**
Add this line to your local.conf if you're using yocto and imx soc
SCMVERSION_pn-linux-imx = ""
Tested on imx-4.9.88-2.0.0_ga release
Just comment the line as a workaround/quickfix in scripts/setlocalversion. Then the kernel version should be same as "make kernelversion".
# Check for git and a git repo.
if test -z "$(git rev-parse --show-cdup 2>/dev/null)" &&
head=$(git rev-parse --verify HEAD 2>/dev/null); then
# If we are at a tagged commit (like "v2.6.30-rc6"), we ignore
# it, because this version is defined in the top level Makefile.
if [ -z "$(git describe --exact-match 2>/dev/null)" ]; then
# If only the short version is requested, don't bother
# running further git commands
if $short; then
#echo "+" #comment this line
return
fi

Resources