Can I customize the svnadmin create process? - linux

I have many different repositories setup on my server. I need to have an identical post-commit hook file in every one of those repos. Simple enough for existing, but is there a way to have calls to svnadmin create automatically copy a post-commit stub file to the new hooks directory? Essentially I'm looking for a post-svnadmin-create hook. Thanks!

I think your best bet would be to wrap the call to svnadmin create in a script that creates the hooks after the repo.

Agreed as long as there is not some built-in way, which there seems not to be. I would have expected subversion to sport something like the customizable skeleton directory for new Linux users. Too bad.
Here is my wrapper with comments if anyone can find it useful - should be fairly extendable. If anyone notices any glaring gotchas in it, don't hesitate - I'm neither a bash nor Linux expert but I think I got most of it covered, and it works :)
# -----------------------------------------------------------------------
# A wrapper for svnadmin to allow post operations following repo creation - copying custom
# hook files into repo in this case. This should be run as root.
# capture input args; note that args[0] == $#[1] (this script name is not captured here)
args=("$#");
# redirect args to svnadmin in all cases - this script should not modify the behavior of svnadmin.
# note: the original binary "/binary_path/svnadmin" has been renamed "/binary_path/svnadmin-wrapped" and
# this script was then named "/binary_path/svnadmin" and given identical user:group & permissions as
# the original.
sudo -u svnuser svnadmin-wrapped ${args[#]};
# capture return code so we can return on exit; svnadmin returns 0 for success
eCode=$?;
# find out if sub-command to svnadmin was "create" and, if so, note the index of the directory arg,
# which is not necessarily going to be in the same position each time (options may be specified
# before the sub-command).
path_idx=0;
found=0;
for i in ${args[#]}
do
# track index; pre-incerement
((path_idx++));
if [ $i == "create" ]
then
# found repo path
((found++));
break;
fi
done
# we now know if the subcommand was create and where the repo path is - finish up as needed.
# note that this block assumes that our hook file stubs are /stub_path/ (owned by root)
# and that there exists a custom log file at /stub_path/cust-log (also owned by root).
d=`date`;
if [ $found != 0 ]
then
# check that the command succeeded
if [ $eCode == 0 ]
then
# check that the directory exists
if [ -d "${args[$path_idx]}/hooks" ]
then
# copy our custom hooks into place
sudo -u svnuser cp "/stub_path/post-commit" "${args[$path_idx]}/hooks/post-commit";
sudo -u svnuser cp "/stub_path/post-revprop-change" "${args[$path_idx]}/hooks/post-revprop-change";
else
# unlikey failure; set custom error code here; log issue
echo "$d svnadmin wrapper error: svnadmin 'create' succeeded but the 'hooks' directory was not found! Params: ${args[#]}" >> "/stub_path/cust-log";
let "eCode=1325";
fi
else
# tried to create but svnadmin failed; log issue
echo "$d svnadmin wrapper error: svnadmin 'create' was called but failed! Params: ${args[#]}" >> "/stub_path/cust-log";
fi
fi
exit $eCode;
-Thanks to all who host and post!

Related

Execute p4 aliased command result in messed wrong order of lines in output

I have the following content inside my p4aliases.txt.
diff-cl $(target-cl) = diff -dl //...#$(EQ)$(target-cl)
Basically it diffs against your files in current workspace toward the target shelved files of changelist.
It is fine. I can execute it. But when I compare the result coming from above aliased command against the direct raw (non-aliased) command as follows
p4 diff -dl //...#=<target-cl>
the output lines of text from aliased command is in wrong order e.g. changes according to a certain file shows up first before a line of file shown, line orders are messed up. This is not the case if you execute with a non-aliased command.
Example
Expected result
==== //depot/common.h#none - x:\mydir\project\src\common.h ====
==== //depot/file.cpp#none - x:\mydir\project\src\file.cpp ====
3a4
> added line 1
==== //depot/file.h#none - x:\mydir\project\src\file.h ====
Actual result
3a4
> added line 1
==== //depot/common.h#none - x:\mydir\project\src\common.h ====
==== //depot/file.cpp#none - x:\mydir\project\src\file.cpp ====
==== //depot/file.h#none - x:\mydir\project\src\file.h ====
I have p4 version as of Rev. P4/NTX64/2021.1/2126753 (2021/05/12).
Perforce server version (got from p4 info) is Server version: P4D/LINUX26X86_64/2017.1/1574018 (2017/10/02).
How can I solve this issue?
Could this be a version too far away between client and server
Update
I have tested p4 client all the way down from 2016-2020 version by downloading old binaries from ftp.perforce.com (in directory perforce). No luck. Output still messed the same. So it's not the problem about version mismatch.
This looks like a bug in the p4 client. When the client does a diff, it's written by the ClientUser::Diff() method, which defaults to writing to stdout (i.e. it does not route the output through ClientUser::OutputText()):
https://workshop.perforce.com/projects/perforce_software-p4/files/2018-2/client/clientuser.cc#436
https://workshop.perforce.com/projects/perforce_software-p4/files/2018-2/client/clientuser.cc#573
Output from commands run as part of an alias go through the ClientUserStrBuf subclass, which buffers all of its output. The file headers, for example, are buffered by ClientUserStrBuf::OutputInfo():
https://workshop.perforce.com/projects/perforce_software-p4/files/2018-2/client/clientaliases.cc#1647
There isn't a ClientUserStrBuf::Diff() implementation, though, so that diff output goes straight to stdout while the headers are buffered and printed at the end (presumably after some post-processing) -- hence the diff output showing up first in the console.
The fix I'd make would be to have the base ClientUser::Diff() implementation route the output through OutputText() when no output file is provided, which seems like the least-surprise behavior; that'd fix the aliases behavior and might even make life a little easier for other client developers who would otherwise hit the same issue. If you have a support contract with Perforce you can file this as a bug report, or since the client is open source you can take a crack at fixing and building it yourself. I don't think there's a workaround that doesn't involve modifying the client source code.
Samwise has the correct approach to truly fix the problem at hands although it might take some effort to understand the code, and conduct the fix itself.
At any rate, if we took such approach we won't be able to take benefits of bug fixes and future updates as we will be stuck with 2018-2 version of p4 as it's the latest as it can be in which we can grab the source.
I would recommend to use WSL then interact with p4.exe (yes, a Windows-based binary) for Windows-based project, and p4 for Linux-based binary. If you didn't use WSL, please find the .bash_aliases-like solution as I have below to seamlessly solve aliases diff operation.
Put the following code into your ~/.bash_aliases
# p4 - fix of aliases diff operation
# platform independent, it will choose a correct binary path to execute properly
p4() { cmd="p4.exe" # default is Windows-based
# get the last argument value, if "-lx" passed in then we know it's linux
if [[ "${#: -1}" == "-lx" ]]; then
cmd="/usr/local/bin/p4"
fi
if [[ $1 == "diff-cl" ]]; then
if [ -z "$2" ]; then
echo "usage: p4 diff-cl <CL>"
return 1
fi
$cmd diff -dl //...#=$2 | diffp4 | less -r
elif [[ $1 == "diff-cl-fonly" ]]; then
if [ -z "$2" ]; then
echo "usage: p4 diff-cl-fonly <CL>"
return 1
fi
$cmd diff -Od -dl -ds //...#=$2 | diffp4 | grep ==== | less -r
else
$cmd "$#"
fi
}
then source ~/.bash_aliases.
What it does is to allow you to still use p4 with all of its original commands & arguments normally with exceptions of diff-cl (which is the same name of alias I've put into p4aliases.txt for Windows or ~/.p4aliases for Linux). You can safely remove diff-cl entry from p4's alias file, or just leave it there. What we have in ~/.bash_aliases file will intercept whenever such argument matches then execute the raw command, just that we don't have to type long command ourselves.
We can later remove such section in our ~/.bash_aliases file when upstream p4 has been fixed.
In else section, we just relay the the whole arguments, and it will be performed just as normally done.
Extra: diff-cl-fonly to list out only files (its depot path, and local workspace path) which have changes.

How does one create a wrapper around a program?

I want to learn to create a wrapper around a program in linux. How does one do this? A tutorial reference web-page/link or example will do. To clarify what I want to learn, I will explain with an example.
I use vim for editing text files. And use rcs as my simple revision control system. rcs allows you to check-in and checkout-files. I would like to create a warpper program named vir which when I type in the shell as:
$ vir temp.txt
will load the file temp.txt into rcs with ci -u temp.txt and then allows me to edit the file using vim.
When I get out and go back in, It will need to check out the file first, using ci -u temp.txt and allow me to edit the file as one normally does with vim, and then when I save and exit, it should check-in the file using co -u temp.txt and as part of that I should be able to add a version control comment.
Basically, all I want to be doing on the command line is:
$ vir temp.txt
as one would with vim. And the wrapper should take care of the version control for me.
Take a look at rcsvers.vim, a vim plugin for automatically saving versions in RCS; you could modify that. There are also other RCS plugins for vim at vim.org
I have a wrapper to enhance the ping command (using zsh) it could, maybe help you:
# ping command wrapper - Last Change: out 27 2019 18:47
# source: https://www.cyberciti.biz/tips/unix-linux-bash-shell-script-wrapper-examples.html
ping(){
# Name: ping() wrapper
# Arg: (url|domain|ip)
# Purpose: Send ping request to domain by removing urls, protocol, username:pass using system /usr/bin/ping
local array=( $# ) # get all args in an array
local host=${array[-1]} # get the last arg
local args=${array[1,-2]} # get all args before last arg in $#
#local _ping="/usr/bin/ping"
local _ping="/bin/ping"
local c=$(_getdomainnameonly "$host")
[ "$host" != "$c" ] && echo "Sending ICMP ECHO_REQUEST to \"$c\"..."
# pass args and host
# $_ping $args $c
# default args for ping
$_ping -n -c 2 -i 1 -W1 $c
}
_getdomainnameonly(){
# Name: _getdomainnameonly
# Arg: Url/domain/ip
# Returns: Only domain name
# Purpose: Get domain name and remove protocol part, username:password and other parts from url
# get url
local h="$1"
# upper to lowercase
local f="${h:l}"
# remove protocol part of hostname
f="${f#http://}"
f="${f#https://}"
f="${f#ftp://}"
f="${f#scp://}"
f="${f#scp://}"
f="${f#sftp://}"
# Remove username and/or username:password part of hostname
f="${f#*:*#}"
f="${f#*#}"
# remove all /foo/xyz.html*
f=${f%%/*}
# show domain name only
echo "$f"
}
What it hides the local ping using a function called "ping", so if your script has precedence on your path it will find at first the function ping. Then inside the script I define an internal variable called ping that points out to the real ping command:
local _ping="/bin/ping"
You can also notice that the args are stored in one array.

How do I rerun a bash script skipping over lines which have previously run sucesfully?

I have a bash script which acts as a wrapper for an analysis pipeline. If the script errors out I want to be able to run the script from the point at which the errors occurred by simply re-running the original command. I have set two different traps; one which will remove the last file being generated on a non-zero exit from my script, the other will remove all the temporary files on exit signal = 0 and essentially cleans up the file system at the end of the run. I turned on noclobber in the bash environment which allows my script to skip over lines of the script where files have already been written but this will only do this if I do not set the non-zero exit trap. As soon as I set this trap then it will exit at the first line where noclobber IDs a file it will not overwrite. Is there a way for me to skip over lines of code that have successfully run previously rather than having to re-run my code from the start? I know I could use conditional statements for each line but I thought there might be a neater way of doing this.
set -o noclobber
# Function to clean up temporary folders when script exits at the end
rmfile() { rm -r $1 }
# Function to remove the file being currently generated
# Function executed if script errors out
rmlast() {
if [ ! -z "$CURRENTFILE" ]
then
rm -r $1
exit 1
fi }
# Trap to remove the currently generated file
trap 'rmlast "$CURRENTFILE"' ERR SIGINT
#Make temporary directory if it has not been created in a previous run
TEMPDIR=$(find . -name "tmp*")
if [ -z "$TEMPDIR" ]
then
TEMPDIR=$(mktemp -d /test/tmpXXX)
fi
# Set CURRENTFILE variable
CURRENTFILE="${TEMPDIR}/Variants.vcf"
# Set CURRENTFILE variable
complexanalysis_tool input_file > $CURRENTFILE
# Set CURRENTFILE variable
CURRENTFILE="${TEMPDIR}/Filtered.vcf"
complexanalysis_tool2 input_file2 > $CURRENTFILE
CURRENTFILE="${TEMPDIR}/Filtered_2.vcf"
complexanalysis_tool3 input_file3 > $CURRENTFILE
# Move files to final destination folder
mv -nv $TEMPDIR/*.vcf /test/newdest/
# Trap to remove temporary folders when script finishes running
trap 'rmfile "$TEMPDIR"' 0
Update:
I have been offered answers suggesting the use of the make utility. I want to make use of its inbuilt utility to check if a dependency has been fulfilled.
In my hands the makefile suggested by VK Kashyap does not seem to skip execution for previously accomplished tasks. So for example I ran the script above and interrupted the script when it was running filtered.vcf with ctrl c. When I rerun the script again it runs from the beginning again i.e. starts again at varaints.vcf. Am I missing something in order to get the makefile to show sources as being fullfilled?
Answer to update:
OK this is a rookie mistake but since I am not familiar with generating makefiles I will post this explanation of my error. The reason my makefile was not rerunning from the exit point was that I had named the targets a different name to the output files being generated. So as VK Kashyap quite correctly answered if you name the targets eg.
variants.vcf
filtered.vcf
filtered2.vcf
the same as the output files being generated then the script will skip previously accomplished tasks.
make utility might be an answer for the thing you want to achive.
it has inbuilt dependecy checking (the stuff which you are trying to achive with tmp files)
#run all target when all of the files are available
all: variants.vcf filtered.vcf filtered2.vcf
mv -nv $(TEMPDIR)/*.vcf /test/newdest/
variants.vcf:
complexanalysis_tool input_file > variants.vcf
filtered.vcf:
complexanalysis_tool2 input_file2 > filtered.vcf
filtered2.vcf:
complexanalysis_tool3 input_file3 > filtered2.vcf
you may use bash script to invoke this make file as:
#/bin/bash
export TEMPDIR=xyz
make -C $TEMPDIR all
make utility will check itself for already accomplished task and skip execution for done stuffs. it will continue where you had the error finishing the task.
you can find more details on internet about exact syntax for makefile.
there is no built-in way to do that.
however, you could brew something like that by keeping track of the last successful line and building your own goto statement, as described here and in Is there a "goto" statement in bash? (just replace the 'labels' with actual line-numbers).
however, the question is whether this is really a smart idea.
a better way is to only run the commands needed, not the commands not-yet-executed.
this could be done either by explicit conditionals in your bash-script:
produce_if_missing() {
# check if first argument is existing
# if not run the rest of the arguments and pipe it into the first one
local curfile=$1
shift
if [ ! -e "${curfile}" ]; then
$# > "${curfile}"
fi
}
produce_if_missing Variants.vcf complexanalysis_tool input_file
produce_if_missing Filtered.vcf complexanalysis_tool2 input_file2
or using tools that are made for such things (see VK Kahyap's answer using make, though i prefer using variables in the make-rules to minimize typos):
Variants.vcf: input_file
complexanalysis_tool $^ > $#
Filtered.vcf: input_file
complexanalysis_tool2 $^ > $#

Integrate a built-in Update function in shell in order to receive OTA updates when there are available

i'm struggled here with this thing that it would be awesome if it's gonna be integrated.
Well my idea is to, create a function which it will be runned at a certain time which is gonna to check if there is a new version of the script . But i don't know how to put the commands together.
I already have a sort of sketch Here:
SCRIPT_NAME="$0"
ARGS="$#"
NEW_FILE="/tmp/blog.sh"
VERSION="1.0"
check_upgrade () {
# check if there is a new version of this file
# here, hypothetically we check if a file exists in the disk.
# it could be an apt/yum check or whatever...
[ -f "$NEW_FILE" ] && {
# install a new version of this file or package
# again, in this example, this is done by just copying the new file
echo "Found a new version of me, updating myself..."
cp "$NEW_FILE" "$SCRIPT_NAME"
rm -f "$NEW_FILE"
# note that at this point this file was overwritten in the disk
# now run this very own file, in its new version!
echo "Running the new version..."
$SCRIPT_NAME $ARGS
# now exit this old instance
exit 0
}
I know it's possible to do this, but i didn't found anything useful on internet.
Every advice will be much appreciated.
Assuming the script is always running, make another script that curls the file and checks it against the original. Something like:
if [ version newer ]; then
kill old verseion
mv "new version" 'old version"
./new version
else:
delete tmp file
fi
Run it with cron at intervals you see fit

Don't add "+" to linux kernel version

I am building linux kernel, if my kernel under git, then kernel version every time is:
Image Name: Linux-2.6.39+
If I am not using git, then everything is OK without any plus at the end.
I know that this done by scripts/setlocalversion script:
if test "$CONFIG_LOCALVERSION_AUTO" = "y"; then
# full scm version string
res="$res$(scm_version)"
else
# append a plus sign if the repository is not in a clean
# annotated or signed tagged state (as git describe only
# looks at signed or annotated tags - git tag -a/-s) and
# LOCALVERSION= is not specified
if test "${LOCALVERSION+set}" != "set"; then
scm=$(scm_version --short)
res="$res${scm:++}"
fi
fi
So, it is possible without code changes say to build system that no need to add "+" at the end of version line?
The plus sign at the end of your version string is there as an indicator that the kernel was built from modified sources (that is, there were non-committed changes). This is also indicated by the comments in scripts/setlocalversion.
To avoid the '+' being appended despite having a dirty working directory, simply set LOCALVERSION explicityly when running make:
make LOCALVERSION=
You may also have to change the configuration option CONFIG_LOCALVERSION_AUTO to n in your kernel config (.config) before building:
sed -i "s|CONFIG_LOCALVERSION_AUTO=.*|CONFIG_LOCALVERSION_AUTO=n|" .config
To prevent the script scripts/setlocalversion to append the + to the end of the kernel local version, create an empty .scmversion file in the root of the kernel sources.
touch .scmversion
this way, you'll be able to leave LOCALVERSION as is in the kernel configuration file, in case you want to append a local signature to the kernel name.
Manipulating scripts/setlocalversion seems to be the only way for me.
Force return in scm_version():
scm_version()
{
local short
short=false
**return**
Add this line to your local.conf if you're using yocto and imx soc
SCMVERSION_pn-linux-imx = ""
Tested on imx-4.9.88-2.0.0_ga release
Just comment the line as a workaround/quickfix in scripts/setlocalversion. Then the kernel version should be same as "make kernelversion".
# Check for git and a git repo.
if test -z "$(git rev-parse --show-cdup 2>/dev/null)" &&
head=$(git rev-parse --verify HEAD 2>/dev/null); then
# If we are at a tagged commit (like "v2.6.30-rc6"), we ignore
# it, because this version is defined in the top level Makefile.
if [ -z "$(git describe --exact-match 2>/dev/null)" ]; then
# If only the short version is requested, don't bother
# running further git commands
if $short; then
#echo "+" #comment this line
return
fi

Resources