I am trying to write a bash script. In this script I want user to enter a path of a directory. Then I want to append some strings at the end of this string and build a path to some subdirectories.
For example assume user enters an string like this:
/home/user1/MyFolder
Now I want to create 2 subdirectories in this directory and copy some files there.
/home/user1/MyFolder/subFold1
/home/user1/MyFolder/subFold2
How can I do this?
The POSIX standard mandates that multiple / are treated as a single / in a file name. Thus
//dir///subdir////file is the same as /dir/subdir/file.
As such concatenating a two strings to build a complete path is a simple as:
full_path="$part1/$part2"
#!/bin/bash
read -p "Enter a directory: " BASEPATH
SUBFOLD1=${BASEPATH%%/}/subFold1
SUBFOLD2=${BASEPATH%%/}/subFold2
echo "I will create $SUBFOLD1 and $SUBFOLD2"
# mkdir -p $SUBFOLD1
# mkdir -p $SUBFOLD2
And if you want to use readline so you get completion and all that, add a -e to the call to read:
read -e -p "Enter a directory: " BASEPATH
Won't simply concatenating the part of your path accomplish what you want?
$ base="/home/user1/MyFolder/"
$ subdir="subFold1"
$ new_path=$base$subdir
$ echo $new_path
/home/user1/MyFolder/subFold1
You can then create the folders/directories as needed.
One convention is to end directory paths with / (e.g. /home/) because paths starting with a / could be confused with the root directory. If a double slash (//) is used in a path, it is also still correct. But, if no slash is used on either variable, it would be incorrect (e.g. /home/user1/MyFoldersubFold1).
The following script catenates several (relative/absolute) paths (BASEPATH) with a relative path (SUBDIR):
shopt -s extglob
SUBDIR="subdir"
for BASEPATH in '' / base base/ base// /base /base/ /base//; do
echo "BASEPATH = \"$BASEPATH\" --> ${BASEPATH%%+(/)}${BASEPATH:+/}$SUBDIR"
done
The output of which is:
BASEPATH = "" --> subdir
BASEPATH = "/" --> /subdir
BASEPATH = "base" --> base/subdir
BASEPATH = "base/" --> base/subdir
BASEPATH = "base//" --> base/subdir
BASEPATH = "/base" --> /base/subdir
BASEPATH = "/base/" --> /base/subdir
BASEPATH = "/base//" --> /base/subdir
The shopt -s extglob is only necessary to allow BASEPATH to end on multiple slashes (which is probably nonsense). Without extended globing you can just use:
echo ${BASEPATH%%/}${BASEPATH:+/}$SUBDIR
which would result in the less neat but still working:
BASEPATH = "" --> subdir
BASEPATH = "/" --> /subdir
BASEPATH = "base" --> base/subdir
BASEPATH = "base/" --> base/subdir
BASEPATH = "base//" --> base//subdir
BASEPATH = "/base" --> /base/subdir
BASEPATH = "/base/" --> /base/subdir
BASEPATH = "/base//" --> /base//subdir
I was working around with my shell script which need to do some path joining stuff like you do.
The thing is, both path like
/data/foo/bar
/data/foo/bar/
are valid.
If I want to append a file to this path like
/data/foo/bar/myfile
there was no native method (like os.path.join() in python) in shell to handle such situation.
But I did found a trick
For example , the base path was store in a shell variable
BASE=~/mydir
and the last file name you wanna join was
FILE=myfile
Then you can assign your new path like this
NEW_PATH=$(realpath ${BASE})/FILE
and then you`ll get
$ echo $NEW_PATH
/path/to/your/home/mydir/myfile
the reason is quiet simple, the "realpath" command would always trim the terminating slash for you if necessary
This should works for empty dir (You may need to check if the second string starts with / which should be treat as an absolute path?):
#!/bin/bash
join_path() {
echo "${1:+$1/}$2" | sed 's#//#/#g'
}
join_path "" a.bin
join_path "/data" a.bin
join_path "/data/" a.bin
Output:
a.bin
/data/a.bin
/data/a.bin
Reference: Shell Parameter Expansion
I ended up concating all arguments ($*) with '/' (IFS=/ - Internal Field Separator) as separator and then removing all repeating '/' (tr -s /).
My function looks like this:
join_paths() {
(IFS=/; echo "'$*'" | tr -s /)
}
#Dunes' basic solution, but accounting for empty strings:
[ -z "$rootpath" ] && rootpath="."
fullpath="${rootpath}/${subdir}"
Related
NOTE BEFORE READING: The following question is described very precisely and that is the reason for the length of a question. If you want to understand the problem, it's better to read the entire thing. Many thanks for all the answers!
I am working on a bash script (.sh file) which will check certain values in every file of a directory. Bash script will be executed in a pre-commit (pre-commit is not a part of the question).
There is a directory that contains multiple .c files in multiple subdirectories. I want to check a part of two lines which are NOT in every .c file but only in some of them. The structure of a file that contains the useful information is as following:
/*
## SYMBOL = some_symbol1
## A2L_TYPE = PARAMETER
.
.
.
#! DEFAULT = some_value1
## END
*/
some_symbol1 = some_value1
/*
## SYMBOL = some_symbol2
## A2L_TYPE = PARAMETER
.
.
.
#! DEFAULT = some_value2
## END
*/
some_symbol2 = some_value2
This kind of structure is automatically generated by another script.
I want to check if some_value1 (in comment) is equal to some_value1 (in variable).
There are hundreds of these variable in each .c file (not necessarily in each .c file).
The main functionality of a script should be:
Check some_value1 in comment and variable and throw an error if they are not the same. Script has to go through EVERY .c file in a directory (bash is in root) and ALL subdirectories to find previously mentioned structure.
Value of variable can be something as 0.06F, where in comment, there is 0.06 (compare only the numbers)
Value of variable can also be an array: { 0.0F, 0.45F, 0.3F } where in the comment, there is [ 0.0, 0.45, 0.3 ] (without F and difference in braces)
To summarize:
I want to build a check script that compares some_value1 (in comment) and some_value1 (in variable) and throw an error if they don't match
Useful information is not in EVERY .c file but only in some of them (don't know which)
Values after #! DEFAULT is a comment where the value of variable is a number (maybe this is not that important?)
between A2L_TYPE and DEFAULT, there can be desired number of unimportant stuff. (still a comment)
What I tried so far is for loop through every .c file and a nested for loop to read every line in each .c file. What I wanted to implement was a grep command inside for loop to check each line if there is a #! DEFAULT pattern and save it to the variable.
Latest code that I tried:
!/bin/bash
shopt -s globstar
for d in */**/*.c
do
while IFS="" read -r p || [ -n "$p" ]
do
grep -P "#! DEFAULT" $d
done < $d
done
This is currently not working because it gives an error that certain grep targets are directories
If any has any questions, I will try to explain it better.
# search for files with extension ".c"
# execute awk on any matches, using '= ' as field separator
find . -type f -name '*.c' -exec awk -F'=[[:space:]]*' '
# check if first three lines match template
( NR==1 && /^\/\*/ ) ||
( NR==2 && /^## SYMBOL = / ) ||
( NR==3 && /^## A2L_TYPE = PARAMETER/ ) { ok++ }
# template mismatch - skip this file
( NR==4 && ok!=3 ) {
printf "%s : ignored\n", FILENAME
nextfile
}
# store first occurrence of some_value1
# note line number where second occurrence expected
/^#! DEFAULT =/ { v[1]=v1=$2; n=NR+3 }
# test second occurrence
NR==n {
v[2]=v2=$2;
# prune everything except numbers and array delimiters
for (s in v) gsub(/[^0-9.,]/,"",v[s]);
# output result
# match exactly or only number list
printf "%s #(%d,%d) : ", FILENAME,n-3,n
if (v1==v2 || v[1]==v[2])
printf "match (%s)==(%s)\n", v1,v2
else
printf "mismatch (%s)!=(%s)\n", v1,v2
# no need to check rest of this file
# elide to check multiple values per file
nextfile
}
' {} +
I are creating a .sh using bash for validate the api sub folders versions
The objective is validate this strings into APIS_BUILD var and find all .proto files into ./proto folder to compile into protobuffer Go file
# define subfolder apis to build
APIS_BUILD=(
prototests/v1/
prototests2/v2
testfolder
)
# the "testfolder" are a invalid folder
Test cases:
prototestes/v1 # valid
prototestes/v1/cobranca # valid
prototestes/v1/cobrnaca/faturamento # valid
outrapastacomarquivosproto/v1 # valid
prototests # invalid
/prototests # invalid
Then, I created this script for validate the APIS_BUILD string array
#!/usr/bin/env bash
# text color
RED='\033[0;31m' # RED
BLUE='\033[0;34m' # Blue
NC='\033[0m' # No Color
# Underline color
UCyan='\033[4;36m' # Cyan
# define subfolder apis to build
APIS_BUILD=(
prototests/v1
cobrancas/v1
)
DST_DIR="." # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder
# Compile proto file
# $1 = Filename to compile
function compile() {
protoc --go_out=$DST_DIR --proto_path=proto --go_opt=M$1=services \
--go_opt=paths=import --go-grpc_out=. \
$1
}
# Validate api_build's
function validateApiBuilds() {
for t in ${APIS_BUILD[#]}; do
IFS="/"
read -a SUBSTR <<<"$t"
if [ ${#SUBSTR[#]} -lt 2 ]; then
printf "${RED}The API_BUILD value ${UCyan}\"${t}\"${RED} are declare wrong, please declare [api_folder]/[version_folder] (example: prototest/v1)${NC}\n" 1>&2
exit 1
fi
done
}
validateApiBuilds
for filename in $(find $SRC_DIR -name '*.proto'); do
[ -e "$filename" ] || continue
echo $filename
done
The subfolder:
But I getting a strange behavior:
If run the .sh file with the validateApiBuilds function the return for $filename is always .
If run the .sh file without the validateApiBuilds function the return for $filename are getting the testservice.proto file
Pictures:
With validateApiBuilds function:
Without validateApiBuilds function:
All the variables:
# define subfolder apis to build
APIS_BUILD=(
prototests/v1
cobrancas/v1
)
DST_DIR="." # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder
Bash version:
$ bash --version
$ GNU bash, versão 4.4.19(1)-release (x86_64-pc-linux-gnu)
Obs.: I changed the validateApiBuilds function to use a regex validation for strings into API_BUILDS variable. But I really wanted to know the reason for this behavior.
edit 2: The make-proto.config file
# define subfolder apis to build
APIS_BUILD=(
prototests/v1
cobrancas/v1
)
DST_DIR="." # define the directory to store the build-in protofiles
SRC_DIR="./proto" # define the proto files folder
Use find better
for filename in $(anything) is always an antipattern -- it splits values on characters in IFS, and then expands each result as a glob. To make find emit completely unambiguous strings, use -print0:
while IFS= read -r -d '' filename; do
[ -e "$filename" ] || continue
echo "$filename"
done < <(find "$SRC_DIR" -name '*.proto' -print0)
Don't change IFS unnecessarily
Change your code to make the assignment to IFS be on the same line as the read, which will make IFS only be modified for that one command.
That is to say, instead of:
IFS=/
read -a SUBSTR <<<"$t"
...you should write:
IFS=/ read -a SUBSTR <<<"$t"
Given a string, I'd like to know if the string represents a valid file path. I've looked at [file isfile <string>] and [file isdirectory <string>] but it seems those two return false if the given string isn't pointing to an existing file/directory. I'm merely interested in the validity of the string.
Pretty much every string is a valid file path. Can you give some examples of what you consider invalid file paths?
You can use file pathtype to check if the path is absolute. Also file split and file normalize may be useful, depending on what you really need.
I don't know that there's a particular way to do that.
I suppose you could try:
set fn /home/bll/mytest.txt
set fail true
if { [file exists $fn] } {
set fail false
} else {
try {
set fh [open $fn w]
set fail false
file delete $fn
} on error {err res} {
}
}
Even if the above works, certain characters may cause issues if you use
the filename in an external command line.
I strip the following characters simply to avoid possible command line problems:
* : " ? < > | [:cntrl:]
I have this list as possibly problematical for unix:
* & [] ? ' " < > |
And this list for windows:
* : () & ^ | < > ' "
Also, a trailing dot is illegal for windows directory names.
And slashes or backslashes can cause issues as those are directory separators.
I want to make a alias for example if I type fullList it will print out a custom text with specific extension in full path listed the last modified the last something like
>fullList
file = /home/user/something/fileA.txt &
file = /home/user/something/fileB.txt &
file = /home/user/something/fileC.txt & <- the last modified.
If you want your exact example output, you're going to want to do something like.
#!/bin/bash
echo
for i in $(ls -trF *.txt); do
full_path="$(pwd $i)/$i"
echo "file = $full_path &"
done
And if you want to do a simple one line alias, do something like below.
> alias fullList="echo; for i in \$(ls -trF *.txt); do full_path=\"\$(pwd \$i)/\$i\"; echo \"file = \$full_path &\"; done"
> fullList
file = /some/path/oldest.txt &
file = /some/path/newer.txt &
...
file = /some/path/newest.txt &
Note, this is assuming you only want to find files, since the F flag for ls appends "/" to directories.
Function
Use a function instead of an alias. Henceforth, you will be able to pass argument
fullList() {
customText="$1"
for f in "$PWD"/* # list current dir files
do
printf "%s: %s\n" "$customText" "$PWD/$f"
done
}
Usage
Then run
$ fullList 'blabla'
blabla: /path/to/file1
blabla: /path/to/file2
blabla: /path/to/file2
Do you know about ?
The tree command ? Could be helpful to list directories content:
tree -f -L 1 $(pwd)/
/home
|-- /home/user1
`-- /home/user2
I'm making this tiny program in Shell:
#***************************************************************
# Function.
# NAME: chk_df
# Synopsis:
# Check if a local directory (dirName) exist and has a file (fileName).
#
#
# The return codes are the following:
# 99 : dirName does not exists
# 0 : dirName exists and has fileName
# 1 : dirName exists and has not fileName
#
# Parameters:
# In values: dirName <string> fileName <string>
# Out values: returnCode <int>
#
# How to use:
# chk_df dirName fileName
#***************************************************************
chk_df(){
# Check the number of arguments that could be passed.
# In this case, two, dirName, fileNAme.
if [[ ${##} != 2 ]]; then
echo "Error ...Use [Function]: chk_df <dirName> <fileName>"
echo "Ex: chk_df /foo lola.txt"
exit
fi
DIR=$1
FILE=$2
[[ ! -d $DIR ]] && return 99
[[ -d $DIR && ! -e $DIR/$FILE ]] && return 1
[[ -d $DIR && -e $DIR/$FILE ]] && return 0
}
Because I need to check if a file is in a directory, I did this (horrible?) patch $DIR/$FILE , but things like this could happen:
I) If we do: chk_df /foo lola.txt
We get: /foo/lola.txt
II) If we do: chk_df /foo/ lola.txt
We get: /foo//lola.txt [Notice the //]
In both cases the code seems to work. Why? I read that backslash acts like a space. So, could I put n backslash without unknown problems?
Could I leave it like that or it will bring problems? Is there a difference? UNIX assume it to the right way?
EXTRA QUESTION: why I can not do the returns with negative numbers? This is: return -1
/ , //, or any string of consecutive slashes have the same meaning according to the POSIX standard, with the exception that they may have a different meaning at the beginning of a path (so /foo and //foo may denote different objects). Linux does not use this exception, so any number of consecutive slashes always means the same thing as a single /.
(The exception is there to cater to the needs of other Unix-like systems that use leading // to denote a network path.)
There are no difference.
// = /
You can, in principle, use as many / separators as you want (until you start hitting PATH_MAX or some other hard limitation):
$ ls /usr/bin///////////////less
/usr/bin///////////////less
One problem you'll run into is if you ever want to test that two paths are the same[*], because /usr/bin/less and /usr/bin//less are the same path but are different strings. It can be useful to canonicalise paths before comparison.
[*] Ignoring the fact that different paths can refer to the same object.