erreur commande cut script shell - linux

while i want to execute this script, the execution was blocked at the cut command and the man cut was displayed
the script code
#!/bin/bash
for i in `cat newcontext` ;do
var1=`cut –f1 –d" " $i`
var2=`cut –f2 –d" " $i`
if [ $var2 = false ];then
for j in `cat adaptrules`;do
c=`cut -f1 -d" " $j`
cc=`cut -f2 -d" " $j`
if [ $c = $var1 ];then
r=$cc
fi
done
sed /$var1/d currentconfig>>newconfig
else
for k in `cat adaptrules`;do
var3=`cut –f1 –d" " $k`
var4=`cut –f2 –d" " $k`
if [ $var3 = $var1 ];then
action=$var4
fi
done
cat $action >> newconfig
fi
done

It's difficult to know if you are trying to read from a file named in the variables i, j, and k, or if you are trying to just parse the lines of newcontext and adaptrules. In either case you should simply not use cut at all. If you are attempting the latter, you can instead do something like:
#!/bin/bash
while read var1 var2 ; do
if test "$var2" = false; then
while read c cc ; do
if test "$c" = "$var1"; then
r=$cc
fi
done < adaptrules
<&- sed /$var1/d currentconfig>>newconfig #WTF: multiple iterations with only one line omitted?
else
while read var3 var4 ; do
if test "$var3" = "$var1"; then
action=$var4
fi
done < adaptrules
<&- cat $action >> newconfig # Does $action really name a file?
# I'm guessing this should be 'echo'
fi
done < newcontext
I find the formatting of the code in the question makes it difficult to read, so I will not spend a lot of time verifying that this logic matches yours. It appears that the variable r is totally unused, the sed and the cat seem incorrect...but this gives you the idea.
Also, it might be stylistically cleaner to write the inner portion as:
if test "$var2" = false; then
while read c cc; do
...
done
else
while read var3 var4; do
...
done
...
fi < adaptrules
Note that you need to be careful that none of the commands inside the outer while loop consume anything from stdin. (hence the <&- which closes that file descriptor for sed and cat, which is not strictly necessary in these cases but demonstrates how to ensure that you do not inadvertently consume from the input to the loop.) It's probably cleaner to do things like:
while read -u 5 var1 var2; do
...
done 5< newcontext

the content of text files
my script generates a new configuration (newconfig) using the context data (newcontext), the current configuration and adaptations rules.

You would need to execute the cut command with example:
var1=$(cut -f1 -d " " <<< $i)
You are trying to execute the cut command as if $i contains a filename when it actually contains text.

my script is intended to generate a new configuration (newconfig). this configuration is generated using the context data (new context), adaptations rules (adaptrules) and the current configuration (currentconfig). the script works as follow: for each contextdata (the field var1 in newcontext), we look for its action (the field cc in adaptrules). then we verify if this action is present in the currentconfiguration. then, if the action of the selected contextdata is present in the currentconfig and the field var2 of contextdata is equal to false, then the action is deleted from the currentconfig else if the action is absent in the currentconfig and the var2 of contextdata is equal to true, so the action is added to the currentconfig. I had modified my script but it generated an error "sed, command c expects followed by text".
the new code is as follows
#!/bin/bash
while read var1 var2 ;do
while read c cc;do
if test "$c" = "$var1" ;then
r=$cc
fi
done <adaptrules
exist=false
while read var; do
if test "$var" = "$r";then
exist=true
fi
done < currentconfig
if test "$var2" = false && test "$exist" = true; then
sed -i "/$r/d" currentconfig
fi
if test "$var2" = true && test "$exist" = false; then
echo "$r">> currentconfig
fi
done < newcontext
thank you

Related

Why variable changed after pipe?

The related stub is like:
tag=('*' '#')
i=0
function output()
{
ifs="$IFS"
IFS=$'\n'
for line in $#
do
echo $'\t' "${tag[$i]}" $line
done
IFS="$ifs"
echo $i
i=$((i+1))
echo $i
i=$((i%2))
echo $i
}
output a|tee README
output b
What I want to do is:
Every time execute output to output a message block, different prefix(${tag[$ind]}) can be used for distinguishing itself from context. Besides, part-message can be redirect to file.
Result of it is:
* a
0
1
1
* b
0
1
1
With the pipe |tee README, variable $i had been reset to 0.
Why it happened and can I implement the function by this train of thought?
Thanks.
It happens becase, as stated at Bash manual, each command in a pipeline is executed as a separate process (i.e., in a subshell).
In order to preserve i variable value I suggest you to enclose the two output calls into a single shell process as follow:
#!/bin/bash
tag=('*' '#')
i=0
function output()
{
ifs="$IFS"
IFS=$'\n'
for line in $#
do
echo $'\t' "${tag[$i]}" $line
done
IFS="$ifs"
echo $i
i=$((i+1))
echo $i
i=$((i%2))
echo $i
}
(
output a
output b
) | tee README

How to make "dictionary" with shell functions?

This is my code:
#!/bin/sh
echo "ARGUMENTS COUNT : " $#
echo "ARGUMENTS LIST : " $*
dictionary=`awk '{ print $1 }'`
function()
{
for i in dictionary
do
for j in $*
do
if [ $j = $i ]
then
;
else
append
fi
done
done
}
append()
{
ls $j > dictionary1.txt
}
function
I need using unix shell functions make "dictionary". For example: I write in arguments default word, example hello. Then my function checks the file dictionary1 if that word is existing in the file. If not - append that word in file, if it's already exist - do nothing.
For some reason, my script does not work. When I start my script, it waits for something and that's it.
What I am doing wrong? How can I fix it?
An implementation that tries to care about both performance and correctness might look like:
#!/usr/bin/env bash
# ^^^^- NOT sh; sh does not support [[ ]] or <(...)
addWords() {
local tempFile dictFile
tempFile=$(mktemp dictFile.XXXXXX) || return
dictFile=$1; shift
[[ -e "$dictFile" ]] || touch "$dictFile" || return
sort -um "$dictFile" <(printf '%s\n' "$#" | sort -u) >"$tempFile"
mv -- "$tempFile" "$dictFile"
}
addWords myDict beta charlie delta alpha
addWords myDict charlie zulu
cat myDict
...has a final dictionary state of:
alpha
beta
charlie
delta
zulu
...and it rereads the input file only once for each addWords call (no matter how many words are being added!), not once per word to add.
Don't name a function "function".
Don't read in and walk through the whole file - all you need is to know it the word is there or not. grep does that.
ls lists files. You want to send a word to the file, not a filename. use echo or printf.
sh isn't bash. Use bash unless there's a clear reason not to, and the only reason is because it isn't available.
Try this:
#! /bin/env bash
checkWord() {
grep -qm 1 "$1" dictionary1.txt ||
echo "$1" >> dictionary1.txt
}
for wd
do checkWord "$wd"
done
If that works, you can add more structure and error checking.
You can remove your dictionary=awk... line (as mentioned it's blocking waiting for input) and simply grep your dictionary file for each argument, something like the below :
for i in "$#"
do
if ! grep -qow "$i" dictionary1.txt
then
echo "$i" >> dictionary1.txt
fi
done
With any awk in any shell on any UNIX box:
awk -v words="$*" '
BEGIN {
while ( (getline word < "dictionary1.txt") > 0 ) {
dict[word]++
}
close("dictionary1.txt")
split(words,tmp)
for (i in tmp) {
word = tmp[i]
if ( !dict[word]++ ) {
newWords = newWords word ORS
}
}
printf "%s", newWords >> "dictionary1.txt"
exit
}'

Linux "echo -n" not being flushed

I have the following code:
while ...
echo -n "some text"
done | while read; do
echo "$REPLY" >> file
done
but echo works only when used without "-n" flag.
looks like when using -n, the output is not flushed/read by next while loop
How can I make sure that "some text" will be read even when not followed by EOL?
You can't distinguish between
echo -n "some text"
and
echo -n "some t"
echo -n "ext"
so you need some kind of delimiting rule. Usually EOL is used for that. read supports custom delimiter via -d or can split based on number of chars via -n or -N. For example you can make read fire on each symbol:
echo -n qwe | while read -N 1 ch; do echo $ch; done
The workaround would be (following original example):
while ...
echo -n "some text"
done | (cat && echo) | while read; do
echo "$REPLY" >> file
done
This will append EOL to the test stream & allow read to read it.
The side effect will be an additional EOL at the end of stream.
You can start with defining your own delimiter:
while :; do
echo -n "some text"
sleep 2
done | while read -d' ' reply; do
echo "-$reply-"
done
This prints:
-some-
-textsome-
-textsome-
For an email perhaps it makes sense to use . as a delimiter, but you need to decide on some tokenization scheme.
You can make read read one char a time, but should add something for reading special characters (newlines, spaces): IFS=.
I want to show that I really capture the characters, so I will uppercase the replies.
i=0
while (( i++<5 )) ; do
echo -n "some text $i. "
sleep 1;
done | while IFS= read -rn1 reply; do
printf "%s" "${reply^^}"
done
This solution has one feature: You will not see any newlines.
When you want to see them too, you need to fix this with
i=1
while (( i++<5 )) ; do
echo -n "some text $i.
second line."
sleep 1;
done | while IFS= read -rn1 reply; do
if (( ${#reply} == 0 )); then
echo
else
printf "%s" "${reply^^}"
fi
done

How do I indirectly assign a variable in bash to take multi-line data from both Standard In, a File, and the output of execution

I have found many snippets here and in other places that answer parts of this question. I have even managed to do this in many steps in an inefficient manner. If it is possible, I would really like to find single lines of execution that will perform this task, rather than having to assign to a variable and copy it a few times to perform the task.
e.g.
executeToVar ()
{
# Takes Arg1: NAME OF VARIABLE TO STORE IN
# All Remaining Arguments Are Executed
local STORE_INvar="${1}" ; shift
eval ${STORE_INvar}=\""$( "$#" 2>&1 )"\"
}
Overall does work, i.e. $ executeToVar SOME_VAR ls -l * # will actually fill SOME_VAR with the output of the execution of the ls -l * command that is taken from the rest of the arguments. However, if the command was to output empty lines at the end, (for e.g. - echo -e -n '\n\n123\n456\n789\n\n' which should have 2 x new lines at the start and the end ) these are stripped by bash's sub-execution process. I have seen in other posts similar to this that this has been solved by adding a token 'x' to the end of the stream, e.g. turning the sub-execution into something like:
eval ${STORE_INvar}=\""$( "$#" 2>&1 ; echo -n x )"\" # <-- ( Add echo -n x )
# and then if it wasn't an indirect reference to a var:
STORE_INvar=${STORE_INvar%x}
# However no matter how much I play with:
eval "${STORE_INvar}"=\""${STORE_INvar%x}"\"
# I am unable to indirectly remove the x from the end.
Anyway, I also need 2 x other variants on this, one that assigns the STDIN stream to the var and one that assigns the contents of a file to the var which I assume will be variations of this involving $( cat ${1} ), or maybe $( cat ${1:--} ) to give me a '-' if no filename. But, none of that will work until I can sort out the removal of the x that is needed to ensure accurate assignment of multi line variables.
I have also tried (but to no avail):
IFS='' read -d '' "${STORE_INvar}" <<<"$( $# ; echo -n x )"
eval \"'${STORE_INvar}=${!STORE_INvar%x}'\"
This is close to optimal -- but drop the eval.
executeToVar() { local varName=$1; shift; printf -v "$1" %s "$("$#")"; }
The one problem this formulation still has is that $() strips trailing newlines. If you want to prevent that, you need to add your own trailing character inside the subshell, and strip it off yourself.
executeToVar() {
local varName=$1; shift;
local val="$(printf %s x; "$#"; printf %s x)"; val=${val#x}
printf -v "$varName" %s "${val%x}"
}
If you want to read all content from stdin into a variable, this is particularly easy:
# This requires bash 4.1 for automatic fd allocation
readToVar() {
if [[ $2 && $2 != "-" ]]; then
exec {read_in_fd}<"$2" # copy from named file
else
exec {read_in_fd}<&0 # copy from stdin
fi
IFS= read -r -d '' "$1" <&$read_in_fd # read from the FD
exec {read_in_fd}<&- # close that FD
}
...used as:
readToVar var < <( : "run something here to read its output byte-for-byte" )
...or...
readToVar var filename
Testing these:
bash3-3.2$ executeToVar var printf '\n\n123\n456\n789\n\n'
bash3-3.2$ declare -p var
declare -- var="
123
456
789
"
...and...
bash4-4.3$ readToVar var2 < <(printf '\n\n123\n456\n789\n\n')
bash4-4.3$ declare -p var2
declare -- var2="
123
456
789
"
what'w wrong with storing in a file:
$ stuffToFile filename $(stuff)
where "stuffToFile" tests for a. > 1 argument, b. input on a pipe
$ ... commands ... | stuffToFile filename
and
$ stuffToFile filename < another_file
where "stoffToFile" is a function:
function stuffToFile
{
[[ -f $1 ]] || { echo $1 is not a file; return 1; }
[[ $# -lt 2 ]] && { cat - > $1; return; }
echo "$*" > $1
}
so, if "stuff" has leading and trailing blank lines, then you must:
$ stuff | stuffToFile filename

Awk: loop & save different lines to different files?

I'm looping over a series of large files with a shell script:
i=0
while read line
do
# get first char of line
first=`echo "$line" | head -c 1`
# make output filename
name="$first"
if [ "$first" = "," ]; then
name='comma'
fi
if [ "$first" = "." ]; then
name='period'
fi
# save line to new file
echo "$line" >> "$2/$name.txt"
# show live counter and inc
echo -en "\rLines:\t$i"
((i++))
done <$file
The first character in each line will either be alphanumeric, or one of the above defined characters (which is why I'm renaming them for use in the output file name).
It's way too slow.
5,000 lines takes 128seconds.
At this rate I've got a solid month of processing.
Will awk be faster here?
If so, how do I fit the logic into awk?
This can certainly be done more efficiently in bash.
To give you an example: echo foo | head does a fork() call, creates a subshell, sets up a pipeline, starts the external head program... and there's no reason for it at all.
If you want the first character of a line, without any inefficient mucking with subprocesses, it's as simple as this:
c=${line:0:1}
I would also seriously consider sorting your input, so you can only re-open the output file when a new first character is seen, rather than every time through the loop.
That is -- preprocess with sort (as by replacing <$file with < <(sort "$file")) and do the following each time through the loop, reopening the output file only conditionally:
if [[ $name != "$current_name" ]] ; then
current_name="$name"
exec 4>>"$2/$name" # open the output file on FD 4
fi
...and then append to the open file descriptor:
printf '%s\n' "$line" >&4
(not using echo because it can behave undesirably if your line is, say, -e or -n).
Alternately, if the number of possible output files is small, you can just open them all on different FDs up-front (substituting other, higher numbers where I chose 4), and conditionally output to one of those pre-opened files. Opening and closing files is expensive -- each close() forces a flush to disk -- so this should be a substantial help.
A few things to speed it up:
Don't use echo/head to get the first character. You're
spawning at least two additional processes per line. Instead,
use bash's parameter expansion facilities to get the first character.
Use if-elif to avoid checking $first against all the
possibilities
each time. Even better, if you are using bash 4.0 or later, use an associative array
to store the output file names, rather than checking against
$first in a big if-statement for each line.
If you don't have a version of bash that supports associative
arrays, replace your if statements with the following.
if [[ "$first" = "," ]]; then
name='comma'
elif [[ "$first" = "." ]]; then
name='period'
else
name="$first"
fi
But the following is suggested. Note the use of $REPLY as the default variable used by read if no name is given (just FYI).
declare -A OUTPUT_FNAMES
output[","]=comma
output["."]=period
output["?"]=question_mark
output["!"]=exclamation_mark
output["-"]=hyphen
output["'"]=apostrophe
i=0
while read
do
# get first char of line
first=${REPLY:0:1}
# make output filename
name=${output[$first]:-$first}
# save line to new file
echo $REPLY >> "$name.txt"
# show live counter and inc
echo -en "\r$i"
((i++))
done <$file
#!/usr/bin/awk -f
BEGIN {
punctlist = ", . ? ! - '"
pnamelist = "comma period question_mark exclamation_mark hyphen apostrophe"
pcount = split(punctlist, puncts)
ncount = split(pnamelist, pnames)
if (pcount != ncount) {print "error: counts don't match, pcount:", pcount, "ncount:", ncount; exit}
for (i = 1; i <= pcount; i++) {
punct_lookup[puncts[i]] = pnames[i]
}
}
{
print > punct_lookup[substr($0, 1, 1)] ".txt"
printf "\r%6d", i++
}
END {
printf "\n"
}
The BEGIN block builds an associative array so you can do punct_lookup[","] and get "comma".
The main block simply does the lookups for the filenames and outputs the line to the file. In AWK, > truncates the file the first time and appends subsequently. If you have existing files that you don't want truncated, then change it to >> (but don't use >> otherwise).
Yet another take:
declare -i i=0
declare -A names
while read line; do
first=${line:0:1}
if [[ -z ${names[$first]} ]]; then
case $first in
,) names[$first]="$2/comma.txt" ;;
.) names[$first]="$2/period.txt" ;;
*) names[$first]="$2/$first.txt" ;;
esac
fi
printf "%s\n" "$line" >> "${names[$first]}"
printf "\rLine $((++i))"
done < "$file"
and
awk -v dir="$2" '
{
first = substr($0,1,1)
if (! (first in names)) {
if (first == ",") names[first] = dir "/comma.txt"
else if (first == ".") names[first] = dir "/period.txt"
else names[first] = dir "/" first ".txt"
}
print > names[first]
printf("\rLine %d", NR)
}
'

Resources