I have a text file named creating.txt. In this file, each line has a path.
If line starts with D it means that I have to create directory.
If line starts with F, I have to create a file.
For example:
D:/root/F1/Cat1
F:/root/F1/file1.txt
D:/root/F1/Cat1/Cat2
I need to go line by line, first check if first letter is D or F, then ignore the : and the first letter and accordingly create a file or directory based on given path. If first letter in line is not D or F, just ignore the line.
Also if the file already exists I'm not allowed to create another one.
I tried using awk:
path=`awk { print } creating.txt`
touch $path
But the problem is this reads the whole file (not line by line) so there have to be only one line. Also I don't know how to ignore D: and F:.
It can be done with two 2 one-liners
I reproduced your input :
$ cat creating.txt
D:/root/F1/Cat1
F:/root/F1/file1.txt
D:/root/F1/Cat1/Cat2
You can use this to create the Directories
$ for path in `grep ^D creating.txt | cut -d: -f2`; do mkdir -p "$path";done
Then, for files
$ for file in `grep ^F creating.txt | cut -d: -f2`; do touch "$file";done
Explanation:
The grep, will select the lines beginning [as I used ^] by D (or F)
The cut, will catch the 2nd field using ':' as delimiter
Here is a bash script that will do as you asked:
#!/bin/bash
while IFS= read -r line; do
[[ $line =~ (.):(.*) ]]
[[ ${BASH_REMATCH[1]} == D ]] && mkdir -p "${BASH_REMATCH[2]}"
[[ ${BASH_REMATCH[1]} == F ]] && [[ ! -d ${BASH_REMATCH[2]%/*} ]] && mkdir -p "${BASH_REMATCH[2]%/*}"
[[ ${BASH_REMATCH[1]} == F ]] && touch "${BASH_REMATCH[2]}"
done < creating.txt
Related
The task is to read a file containing letters and numbers. Then put the letters in a text file inside a Letters directory and the numbers in a different text file in a numbers directory. The problem I'm having is I don't know how I would identify strings and integers within a file.
The file I'm using contains:
723
Xavier
Cat
323
Zebra
This is the code I've come up with.
#!/bin/bash
file = $1
mkdir Numbers
mkdir Letters
touch Numbers/Numbers.txt
touch Letters/Letters.txt
for x in $file; do
if [x == string];
then
echo x >> Letters/Letters.txt;
fi
if [x == integer];
then
echo x >> Numbers/Numbers.txt;
fi
Your original code has a lot of syntactical errors. I suggest going forward please submit a code which is at least syntactically correct.
Anyways, assuming you are new to unix shell scripting, here a quick solution to your problem
#!/bin/bash
file=$1
mkdir Numbers
mkdir Letters
touch Numbers/Numbers.txt
touch Letters/Letters.txt
while read line
do
if [[ $line =~ [^0-9] ]]
then
echo $line >> Letters/Letters.txt
else
echo $line >> Numbers/Numbers.txt
fi
done < $file
The question is vague.
This solves the question as posted, and handles lines with mixed
numbers and strings, (e.g.: "a1b2c3".), using tee and bash
process substitution, with tr to delete all undesired chars:
mkdir Numbers Letters
tee >(tr -d '[0-9]' > Letters/Letters.txt) \
>(tr -d '[a-zA-Z]' > Numbers/Numbers.txt) \
> /dev/null < inputfile
If the goal is to remove whole lines that are either all numbers
or all letters, try grep:
mkdir Numbers Letters
tee >(grep -v '^[0-9]*$' > Letters/Letters.txt) \
>(grep '^[0-9]*$' > Numbers/Numbers.txt) \
> /dev/null < inputfile
I am searching for one file, say "file1.txt", and output of find command is like below.
/home/nicool/Desktop/file1.txt
/home/nicool/Desktop/dir1/file1.txt
/home/nicool/Desktop/dir1/dir2/file1.txt
In above cases I want only common parent directory, which is "/home/nicool/Desktop" in above case. How it can be achieved using bash? Please help to find general solution for such problem.
This script reads lines and stores the common prefix in each iteration:
# read a line into the variable "prefix", split at slashes
IFS=/ read -a prefix
# while there are more lines, one after another read them into "next",
# also split at slashes
while IFS=/ read -a next; do
new_prefix=()
# for all indexes in prefix
for ((i=0; i < "${#prefix[#]}"; ++i)); do
# if the word in the new line matches the old one
if [[ "${prefix[i]}" == "${next[i]}" ]]; then
# then append to the new prefix
new_prefix+=("${prefix[i]}")
else
# otherwise break out of the loop
break
fi
done
prefix=("${new_prefix[#]}")
done
# join an array
function join {
# copied from: http://stackoverflow.com/a/17841619/416224
local IFS="$1"
shift
echo "$*"
}
# join the common prefix array using slashes
join / "${prefix[#]}"
Example:
$ ./x.sh <<eof
/home/nicool/Desktop1/file1.txt
/home/nicool/Desktop2/dir1/file1.txt
/home/nicool/Desktop3/dir1/dir2/file1.txt
eof
/home/nicool
I don't think there's a bash builtin for this, but you can use this script, and pipe your find into it.
read -r FIRSTLINE
DIR=$(dirname "$FIRSTLINE")
while read -r NEXTLINE; do
until [[ "${NEXTLINE:0:${#DIR}}" = "$DIR" || "$DIR" = "/" ]]; do
DIR=$(dirname "$DIR")
done
done
echo $DIR
For added safety, use -print0 on your find, and adjust your read statements to have -d '\0'. This will work with filenames that have newlines.
lcp() {
local prefix path
read prefix
while read path; do
while ! [[ $path =~ ^"$prefix" ]]; do
[[ $prefix == $(dirname "$prefix") ]] && return 1
prefix=$(dirname "$prefix")
done
done
printf '%s\n' "$prefix"
return 0
}
This finds the longest common prefix of all of the lines of standard input.
$ find / -name file1.txt | lcp
/home/nicool/Desktop
I am pretty new to Unix and have little exposure to shell script. I need to come up with a script that converts the file names from certain string values to special characters. This needs to be run in such a way all files under sub-directories also gets renamed.
For Example:
From: abc(GE)xyz(PR).txt changes
To: abc>xyz%.txt
I m ok to set if condition for all required special characters, but im not sure what options to pass and how to do it for all sub-directories.
Thanks,
Jeel
Here's one approach:
# given a filename, execute any desired replacements.
update_name() {
local orig_name_var=$1
local dest_name_var=$2
local orig_name=${!orig_name_var}
local new_name="$orig_name"
new_name=${new_name//(GE)/">"}
new_name=${new_name//(PR)/"%"} # repeat for additional substitutions
printf -v "$dest_name_var" "$new_name"
}
while IFS= read -r -d '' orig_name; do
update_name orig_name new_name
[[ $orig_name = $new_name ]] && continue
if ! [[ -e $orig_name ]]; then
orig_dirname=${orig_name%/*}
orig_basename=${orig_name##*/}
update_name orig_dirname new_dirname
if [[ -e $new_dirname/$orig_basename ]]; then
# we already renamed the directory this file is in
orig_name=$new_dirname/$orig_basename
fi
fi
mv -- "$orig_name" "$new_name"
done < <(find . '(' -name '*(GE)*' -o -name '*(PR)*' ')' -print0)
I have a script running that is checking multiples directories and comparing them to expanded tarballs of the same directories elsewhere.
I am using diff -r -q and what I would like is that when diff finds any difference in the recursive run it will stop running instead of going through more directories in the same run.
All help appreciated!
Thank you
#bazzargh I did try it like you suggested or like this.
for file in $(find $dir1 -type f);
do if [[ $(diff -q $file ${file/#$dir1/$dir2}) ]];
then echo differs: $file > /tmp/$runid.tmp 2>&1; break;
else echo same: $file > /dev/null; fi; done
But this only works with files that exist in both directories. If one file is missing I won't get information about that. Also the directories I am working with have over 300.000 files so it seems to be a bit of overhead to do a find for each file and then diff.
I would like something like this to work, with and elif statement that checks if $runid.tmp contains data and breaks if it does. I added 2> after the first if statement so stderr is sent to the $runid.tmp file.
for file in $(find $dir1 -type f);
do if [[ $(diff -q $file ${file/#$dir1/$dir2}) ]] 2> /tmp/$runid.tmp;
then echo differs: $file > /tmp/$runid.tmp 2>&1; break;
elif [[ -s /tmp/$runid.tmp ]];
then echo differs: $file >> /tmp/$runid.tmp 2>&1; break;
else echo same: $file > /dev/null; fi; done
Would this work?
You can do the loop over files with 'find' and break when they differ. eg for dirs foo, bar:
for file in $(find foo -type f); do if [[ $(diff -q $file ${file/#foo/bar}) ]]; then echo differs: $file; break; else echo same: $file; fi; done
NB this will not detect if 'bar' has directories that do not exist in 'foo'.
Edited to add: I just realised I overlooked the really obvious solution:
diff -rq foo bar | head -n1
It's not 'diff', but with 'awk' you can compare two files (or more) and then exit when they have a different line.
Try something like this (sorry, it's a little rough)
awk '{ h[$0] = ! h[$0] } END { for (k in h) if (h[k]) exit }' file1 file2
Sources are here and here.
edit: to break out of the loop when two files have the same line, you may have to do the loop in awk. See here.
You can try the following:
#!/usr/bin/env bash
# Determine directories to compare
d1='./someDir1'
d2='./someDir2'
# Loop over the file lists and diff corresponding files
while IFS= read -r line; do
# Split the 3-column `comm` output into indiv. variables.
lineNoTabs=${line//$'\t'}
numTabs=$(( ${#line} - ${#lineNoTabs} ))
d1Only='' d2Only='' common=''
case $numTabs in
0)
d1Only=$lineNoTabs
;;
1)
d2Only=$lineNoTabs
;;
*)
common=$lineNoTabs
;;
esac
# If a file exists in both directories, compare them,
# and exit if they differ, continue otherwise
if [[ -n $common ]]; then
diff -q "$d1/$common" "$d2/$common" || {
echo "EXITING: Diff found: '$common'" 1>&2;
exit 1; }
# Deal with files unique to either directory.
elif [[ -n $d1Only ]]; then # fie
echo "File '$d1Only' only in '$d1'."
else # implies: if [[ -n $d2Only ]]; then
echo "File '$d2Only' only in '$d2."
fi
# Note: The `comm` command below is CASE-SENSITIVE, which means:
# - The input directories must be specified case-exact.
# To change that, add `I` after the last `|` in _both_ `sed commands`.
# - The paths and names of the files diffed must match in case too.
# To change that, insert `| tr '[:upper:]' '[:lower:]' before _both_
# `sort commands.
done < <(comm \
<(find "$d1" -type f | sed 's|'"$d1/"'||' | sort) \
<(find "$d2" -type f | sed 's|'"$d2/"'||' | sort))
The approach is based on building a list of files (using find) containing relative paths (using sed to remove the root path) for each input directory, sorting the lists, and comparing them with comm, which produces 3-column, tab-separated output to indicated which lines (and therefore files) are unique to the first list, which are unique to the second list, and which lines they have in common.
Thus, the values in the 3rd column can be diffed and action taken if they're not identical.
Also, the 1st and 2nd-column values can be used to take action based on unique files.
The somewhat complicated splitting of the 3 column values output by comm into individual variables is necessary, because:
read will treat multiple tabs in sequence as a single separator
comm outputs a variable number of tabs; e.g., if there's only a 1st-column value, no tab is output at all.
I got a solution to this thanks to #bazzargh.
I use this code in my script and now it works perfectly.
for file in $(find ${intfolder} -type f);
do if [[ $(diff -q $file ${file/#${intfolder}/${EXPANDEDROOT}/${runid}/$(basename ${intfolder})}) ]] 2> ${resultfile}.tmp;
then echo differs: $file > ${resultfile}.tmp 2>&1; break;
elif [[ -s ${resultfile}.tmp ]];
then echo differs: $file >> ${resultfile}.tmp 2>&1; break;
else echo same: $file > /dev/null;
fi; done
thanks!
I need to find strings matching some regexp pattern and represent the search result as array for iterating through it with loop ), do I need to use sed ? In general I want to replace some strings but analyse them before replacing.
Using sed and diff:
sed -i.bak 's/this/that/' input
diff input input.bak
GNU sed will create a backup file before substitutions, and diff will show you those changes. However, if you are not using GNU sed:
mv input input.bak
sed 's/this/that/' input.bak > input
diff input input.bak
Another method using grep:
pattern="/X"
subst=that
while IFS='' read -r line; do
if [[ $line = *"$pattern"* ]]; then
echo "changing line: $line" 1>&2
echo "${line//$pattern/$subst}"
else
echo "$line"
fi
done < input > output
The best way to do this would be to use grep to get the lines, and populate an array with the result using newline as the internal field separator:
#!/bin/bash
# get just the desired lines
results=$(grep "mypattern" mysourcefile.txt)
# change the internal field separator to be a newline
IFS=$'/n'
# populate an array from the result lines
lines=($results)
# return the third result
echo "${lines[2]}"
You could build a loop to iterate through the results of the array, but a more traditional and simple solution would just be to use bash's iteration:
for line in $lines; do
echo "$line"
done
FYI: Here is a similar concept I created for fun. I thought it would be good to show how to loop a file and such with this. This is a script where I look at a Linux sudoers file check that it contains one of the valid words in my valid_words array list. Of course it ignores the comment "#" and blank "" lines with sed. In this example, we would probably want to just print the Invalid lines only but this script prints both.
#!/bin/bash
# -- Inspect a sudoer file, look for valid and invalid lines.
file="${1}"
declare -a valid_words=( _Alias = Defaults includedir )
actual_lines=$(cat "${file}" | wc -l)
functional_lines=$(cat "${file}" | sed '/^\s*#/d;/^\s*$/d' | wc -l)
while read line ;do
# -- set the line to nothing "" if it has a comment or is empty line.
line="$(echo "${line}" | sed '/^\s*#/d;/^\s*$/d')"
# -- if not set to nothing "", check if the line is valid from our list of valid words.
if ! [[ -z "$line" ]] ;then
unset found
for each in "${valid_words[#]}" ;do
found="$(echo "$line" | egrep -i "$each")"
[[ -z "$found" ]] || break;
done
[[ -z "$found" ]] && { echo "Invalid=$line"; sleep 3; } || echo "Valid=$found"
fi
done < "${file}"
echo "actual lines: $actual_lines funtional lines: $functional_lines"