I have a bunch of directories to process, so I start a for loop like this:
foreach n (1 2 3 4 5 6 7 8)
Then I have a bunch of commands where I am copying over a few files from different places
cp file1 dir$n
cp file2 dir$n
but I have a couple commands where the $n is in the middle of the command like this:
cp -r dir$nstep1 dir$n
When I run this command, the shell complains that it cannot find the variable $nstep1. What i want to do is evaluate the $n first and then concatenate the text around it. I tried using `` and (), but neither of those work. How to do this in csh?
In this respect behavior is similar to POSIX shells:
cp -r "dir${n}step1" "dir${n}"
The quotes prevent string-splitting and glob expansion. To observe what this means, compare the following:
# prints "hello * cruel * world" on one line
set n=" * cruel * "
printf '%s\n' "hello${n}world"
...to this:
# prints "hello" on one line
# ...then a list of files in the current directory each on their own lines
# ...then "cruel" on another line
# ...then a list of files again
# ... and then "world"
set n=" * cruel * "
printf '%s\n' hello${n}world
In real-world cases, correct quoting can thus be the difference between deleting the oddly-named file you're trying to operate on, and deleting everything else in the directory as well.
Related
NOTE BEFORE READING: The following question is described very precisely and that is the reason for the length of a question. If you want to understand the problem, it's better to read the entire thing. Many thanks for all the answers!
I am working on a bash script (.sh file) which will check certain values in every file of a directory. Bash script will be executed in a pre-commit (pre-commit is not a part of the question).
There is a directory that contains multiple .c files in multiple subdirectories. I want to check a part of two lines which are NOT in every .c file but only in some of them. The structure of a file that contains the useful information is as following:
/*
## SYMBOL = some_symbol1
## A2L_TYPE = PARAMETER
.
.
.
#! DEFAULT = some_value1
## END
*/
some_symbol1 = some_value1
/*
## SYMBOL = some_symbol2
## A2L_TYPE = PARAMETER
.
.
.
#! DEFAULT = some_value2
## END
*/
some_symbol2 = some_value2
This kind of structure is automatically generated by another script.
I want to check if some_value1 (in comment) is equal to some_value1 (in variable).
There are hundreds of these variable in each .c file (not necessarily in each .c file).
The main functionality of a script should be:
Check some_value1 in comment and variable and throw an error if they are not the same. Script has to go through EVERY .c file in a directory (bash is in root) and ALL subdirectories to find previously mentioned structure.
Value of variable can be something as 0.06F, where in comment, there is 0.06 (compare only the numbers)
Value of variable can also be an array: { 0.0F, 0.45F, 0.3F } where in the comment, there is [ 0.0, 0.45, 0.3 ] (without F and difference in braces)
To summarize:
I want to build a check script that compares some_value1 (in comment) and some_value1 (in variable) and throw an error if they don't match
Useful information is not in EVERY .c file but only in some of them (don't know which)
Values after #! DEFAULT is a comment where the value of variable is a number (maybe this is not that important?)
between A2L_TYPE and DEFAULT, there can be desired number of unimportant stuff. (still a comment)
What I tried so far is for loop through every .c file and a nested for loop to read every line in each .c file. What I wanted to implement was a grep command inside for loop to check each line if there is a #! DEFAULT pattern and save it to the variable.
Latest code that I tried:
!/bin/bash
shopt -s globstar
for d in */**/*.c
do
while IFS="" read -r p || [ -n "$p" ]
do
grep -P "#! DEFAULT" $d
done < $d
done
This is currently not working because it gives an error that certain grep targets are directories
If any has any questions, I will try to explain it better.
# search for files with extension ".c"
# execute awk on any matches, using '= ' as field separator
find . -type f -name '*.c' -exec awk -F'=[[:space:]]*' '
# check if first three lines match template
( NR==1 && /^\/\*/ ) ||
( NR==2 && /^## SYMBOL = / ) ||
( NR==3 && /^## A2L_TYPE = PARAMETER/ ) { ok++ }
# template mismatch - skip this file
( NR==4 && ok!=3 ) {
printf "%s : ignored\n", FILENAME
nextfile
}
# store first occurrence of some_value1
# note line number where second occurrence expected
/^#! DEFAULT =/ { v[1]=v1=$2; n=NR+3 }
# test second occurrence
NR==n {
v[2]=v2=$2;
# prune everything except numbers and array delimiters
for (s in v) gsub(/[^0-9.,]/,"",v[s]);
# output result
# match exactly or only number list
printf "%s #(%d,%d) : ", FILENAME,n-3,n
if (v1==v2 || v[1]==v[2])
printf "match (%s)==(%s)\n", v1,v2
else
printf "mismatch (%s)!=(%s)\n", v1,v2
# no need to check rest of this file
# elide to check multiple values per file
nextfile
}
' {} +
I am wanting to pass a string variable in a ssh command. You can see in the code below I ssh to a server then cd to a directory that I pass a variable to. (cd $orig)
The variable is pulled from a file that I read in and put into an array.
I think that is where my error is because there might be unwanted hidden characters after I used the split command to read in from the file.
Here is the error I get:
ksh: /OnSight/jetplan/scripts/release/jscripts^M: not found
Can't open perl script "AddAlias.pl": No such file or directory
/OnSight/users/onsadm
SSHing to densbp53
/OnSight//scripts/release/jscripts
It can't find my script because the CD to the folder fails.
Sometimes the error says that 'end of file' can't be found. Like I'm doing a CD command with a EOF hidden symbol.
And here is the code:
for(my $j=0; $j < $#servName+1; $j++)
{
print "\nSSHing to $servName[$j]\n\n";
my $orig = $scriptfileLoc[$j];
#my $chopped = chop($orig);
chop($orig);
chomp($orig);
print ("\n$orig\n");
$sshstart = `ssh $servName[$j] "cd $orig; pwd; perl AddAlias.pl $aliasName $aliasCommand $addperl $servProfileLoc[$j]"`;
print $sshstart;
}
It outputs the $orig variable and it looks fine after the chop and chomp. (Which I've done both by themselves and still got the same error) So I pass it in my SSH command and it doesnt work.
I have a server file that holds all the server information, and yes it looks repetative I know.
densbp40:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp41:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp42:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp43:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp50:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp51:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp52:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp53:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp60:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp61:/export/home/.profile:/OnSight/scripts/release/jscripts
densbp62:/export/home/.profile:/OnSight/scripts/release/jscripts
tulsbp40:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
tulsbp41:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
tulsbp42:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
tulsbp43:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
tulsbp50:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
tulsbp51:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
tulsbp52:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
tulsbp53:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
densbcp1:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
densbcp2:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
densmsv1:/OnSight/.profile:/OnSight/scripts/jscripts
denamdp1:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
denamap1:/OnSight/users/profile:/OnSight/scripts/release/jscripts
denamap2:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
denfpev1:/OnSight/users/.profile:/OnSight/scripts/release/jscripts
This script asks the user to choose to send a file to ALL servers or just one.
To remove CR (^M) from the end of lines, use the following regex:
$orig =~ s/\r$//gm;
Anchoring at the line end guarantees that any other carriage return characters are not removed from your input. (You probably don't them there either, but to normalize line endings, it's better to not touch other characters).
g enables global matches (not only the first) and m enables multiline mode, so that $ matches the end of each line in a multiline string, not only the end of the string.
"^M" is carriage return a.k.a "\r". Use regex to remove it:
$orig =~ s/\r//g;
I know I can do this to reflect just last 2 directories in the PS1 value.
PS1=${PWD#"${PWD%/*/*}/"}#
but lets say we have a directory name that's really messy and will reduce my working space , like
T-Mob/2021-07-23--07-48-49_xperia-build-20191119010027#
OR
2021-07-23--07-48-49_nokia-build-20191119010027/T-Mob#
those are the last 2 directories before the prompt
I want to set a condition if directory length of either of the last 2 directories is more than a threshold e.g. 10 chars , shorten the name with 1st 3 and last 3 chars of the directory (s) whose length exceeds 10
e.g.
2021-07-23--07-48-49_xperia-build-20191119010027 &
2021-07-23--07-48-49_nokia-build-20191119010027
both gt 10 will be shortened to 202*027 & PS1 will be respectively
T-Mob/202*027/# for T-Mob/2021-07-23--07-48-49_xperia-build-20191119010027# and
202*027/T-Mob# for 2021-07-23--07-48-49_nokia-build-20191119010027/T-Mob#
A quick 1 Liner to get this done ?
I cant post this in comments so Updating here. Ref to Joaquins Answer ( thx J)
PS1=''`echo ${PWD#"${PWD%/*/*}/"} | awk -v RS='/' 'length() <=10{printf $0"/"}; length()>10{printf "%s*%s/", substr($0,1,3), substr($0,length()-2,3)};'| tr -d "\n"; echo "#"`''
see below o/p's
/root/my-applications/bin # it shortened as expected
my-*ons/bin/#cd - # going back to prev.
/root
my-*ons/bin/# #value of prompt is the same but I am in /root
A one-liner is basically always the wrong choice. Write code to be robust, readable and maintainable (and, for something that's called frequently or in a tight loop, to be efficient) -- not to be terse.
Assuming availability of bash 4.3 or newer:
# Given a string, a separator, and a max length, shorten any segments that are
# longer than the max length.
shortenLongSegments() {
local -n destVar=$1; shift # arg1: where do we write our result?
local maxLength=$1; shift # arg2: what's the maximum length?
local IFS=$1; shift # arg3: what character do we split into segments on?
read -r -a allSegments <<<"$1"; shift # arg4: break into an array
for segmentIdx in "${!allSegments[#]}"; do # iterate over array indices
segment=${allSegments[$segmentIdx]} # look up value for index
if (( ${#segment} > maxLength )); then # value over maxLength chars?
segment="${segment:0:3}*${segment:${#segment}-3:3}" # build a short version
allSegments[$segmentIdx]=$segment # store shortened version in array
fi
done
printf -v destVar '%s\n' "${allSegments[*]}" # build result string from array
}
# function to call from PROMPT_COMMAND to actually build a new PS1
buildNewPs1() {
# declare our locals to avoid polluting global namespace
local shorterPath
# but to cache where we last ran, we need a global; be explicit.
declare -g buildNewPs1_lastDir
# do nothing if the directory hasn't changed
[[ $PWD = "$buildNewPs1_lastDir" ]] && return 0
shortenLongSegments shorterPath 10 / "$PWD"
PS1="${shorterPath}\$"
# update the global tracking where we last ran this code
buildNewPs1_lastDir=$PWD
}
PROMPT_COMMAND=buildNewPs1 # call buildNewPs1 before rendering the prompt
Note that printf -v destVar %s "valueToStore" is used to write to variables in-place, to avoid the performance overhead of var=$(someFunction). Similarly, we're using the bash 4.3 feature namevars -- accessible with local -n or declare -n -- to allow destination variable names to be parameterized without the security risk of eval.
If you really want to make this logic only apply to the last two directory names (though I don't see why that would be better than applying it to all of them), you can do that easily enough:
buildNewPs1() {
local pathPrefix pathFinalSegments
pathPrefix=${PWD%/*/*} # everything but the last 2 segments
pathSuffix=${PWD#"$pathPrefix"} # only the last 2 segments
# shorten the last 2 segments, store in a separate variable
shortenLongSegments pathSuffixShortened 10 / "$pathSuffix"
# combine the unshortened prefix with the shortened suffix
PS1="${pathPrefix}${pathSuffixShortened}\$"
}
...adding the performance optimization that only rebuilds PS1 when the directory changed to this version is left as an exercise to the reader.
Probably not the best solution, but a quick solution using awk:
PS1=`echo ${PWD#"${PWD%/*/*}/"} | awk -v RS='/' 'length()<=10{printf $0"/"}; length()>10{printf "%s*%s/", substr($0,1,3), substr($0,length()-2,3)};'| tr -d "\n"; echo "#"`
I got this results with your examples:
T-Mob/202*027/#
202*027/T-Mob/#
I'm having a weird problem with the sed command.
I have a script that take a c file, copy it X times and then replace the name of the functions inside it by adding number to the name.
For example:
originalFile.c contains these functions check0, check1 check2
The script will generate those file:
originalFile1.c: check0 check1 check2
originalFile2.c: check3 check4 check5
originalFile3.c: check6 check7 check8
... and so on.
Now the problem... If I generate enough files so the number goes up to 10,20 or more I noticed something in the name of the function. The first function of the file is renamed incorrectly but the other are corrects. For example:
originalFileX.c: __check165__ check16 check17
...
originalFileZ.c: __check297__ __check298__ check29 -> in this file 2 names are incorrects.
Also, If I print the name with echo everything is correct. Do you have any idea what could be wrong?
Here is my script (I run it under OSX):
#!/bin/bash
NUMCHECK=3
# $1: filename
# $2: number of function in the file
# $3: number of function I want to generate
# $4: function basename
function replace_name() {
FILE_NUM=$((($3+($2-1))/$2))
TMP=0
for (( i=1; i<$FILE_NUM+1; i++ ))
do
cp $1.mm test/$1$i.mm
for (( j=0; j<$2; j++ ))
do
OLDNAME="$4$j"
NEWNAME="$4$TMP"
echo $OLDNAME:$NEWNAME
sed -i "" "s/$OLDNAME/$NEWNAME/g" test/$1$i.mm
TMP=$(($TMP+1))
done
done
}
replace_name check $NUMCHECK 60 check
Youre doing 3 runs of the sed in each file. Just imagine the following
sed -i s/check0/check150/g test/check51.mm
sed -i s/check1/check151/g test/check51.mm
sed -i s/check2/check152/g test/check51.mm
The
s/check0/check150/g changes the check0 to check150 - ok
s/check1/check151/g will change the previous check150 to check15150 (because it finds the check1 string in the check150 too, from the previous step).
etc...
You need more precisely define your regex. because here isn't any example input, can't help more.
I'm looking for a standard tool capable of taking all of its arguments and turning it into a single string suitable for use as multiple arguments in an automatically generated bash/sh/zsh script. Such a command is extremely useful in various disciplines of script-fu. An example of its usage:
% shsafe 'A big \nasty string '\'' $HOME $PATH' 'another string \\'
'A big \nasty string '\'' $HOME $PATH' 'another string \\'
Using it in another script:
% sshc host rm 'file/with spaces and $special chars'
where sshc contains
#!/bin/bash
# usage: sshc host command [arg ...]
# Escapes its arguments so that the command may contain special
# characters. Assumes the remote shell is sh-like.
host=$1
shift
exec ssh "$host" "$(shsafe "$#")"
Another example:
#!/bin/bash
# Run multiple commands in a single sudo session. The arguments of
# this script are passed as arguments to the first command. Useful if
# you don't want to have to type the password for both commands and
# the first one takes a while to run.
sudo bash -c "pacman -Syu $(shsafe "$#") && find /etc -name '*.pacnew'"
I couldn't find a suitable solution to this problem in the pre-existing commands, so I made up my own, called shsafe. It uses the fact that single quotes, '', turn off absolutely all shell expansion, except for ' itself.
shsafe:
#!/usr/bin/env python
from sys import *
n = len(argv)
if n == 1:
exit(0)
i = 1
while True:
stdout.write("'" + argv[i].replace("'", "'\\''") + "'")
i += 1
if i == n:
break
stdout.write(' ')
stdout.write('\n')
Is there any standard tool capable of doing this to its arguments?
Note that the printf command with a format string consisting of just the %q formatter is not good enough for this, because it won't keep multiple arguments separated:
% printf %q arg1 arg2
arg1arg2
I did eventually figure out a decent way of doing this:
% printf "$'%q' " 'crazy string \ $HOME' 'another\ string'
$'crazy\ string\ \\\ \$HOME' $'another\\\ string'
It's a little error prone what with the quotes everywhere, so it's not ideal, IMO, but it's a solid solution that should work anywhere. If it's being used a lot, you could always turn it into a shell function:
shsafe () {
printf "$'%q' " "$#"
}