I have a text file, say, input.txt and I want to run a command and write the output to another text file, say, output.txt. I need to read values from input.txt, each value is in a line, then I need to insert them in the command then write the result in output.txt file. I tried the following and it works fine with me:
for i in `cat input.txt`; do command -m $i -b 100; echo $i; >> output.txt; done
Now, I need to make some improvements over this but I have little experience in Linux so I need some help.
What I need to do is:
1) Before each command result, I want to insert the value of i separated by comma. For example:
i1,result1
i2,result2
i3,result3
2) I need to change the second fixed value that I used in my command from a fixed value (100) to a value read from input.txt. So, the new input file which contains two values, say, newinput.txt is as the following:
i1,value1
i2,value2
i3,value3
Try this, in bash:
IFS=','
while read i val; do
echo -n "$i,"
command $i $val
done < input.txt > output.txt
Related
I'm trying to use this command:
for i in $(cat file); do echo "$whatever_text.$i">$i; done
which is for making each line a new file, I will get straight to the point here!
I want for bash to ignore the expressions such as "$" because for example if I have a line like this:
$a = 'string';
or multiple lines like that, it won't be printed like that, bash leaves only 'string'.
The correct way to iterate over the lines of a text file is to use the read command from a while loop.
while IFS= read -r i; do
echo "$whatever_text.$i" > "$i"
done < file
See Bash FAQ 001 for more details.
I have a bash script which asks for two arguments with a space between them. Now I would like to automate filling out the prompt in the command line with reading from a text file. The text file contains a list with the argument combinations.
So something like this in the command line I think;
for line in 'cat text.file' ; do script.sh ; done
Can this be done? What am I missing/doing wrong?
Thanks for the help.
A while loop is probably what you need. Put the space separated strings in the file text.file :
cat text.file
bingo yankee
bravo delta
Then write the script in question like below.
#!/bin/bash
while read -r arg1 arg2
do
/path/to/your/script.sh "$arg1" "$arg2"
done<text.file
Don't use for to read files line by line
Try something like this:
#!/bin/bash
ARGS=
while IFS= read -r line; do
ARGS="${ARGS} ${line}"
done < ./text.file
script.sh "$ARGS"
This would add each line to a variable which then is used as the arguments of your script.
'cat text.file' is a string literal, $(cat text.file) would expand to output of command however cat is useless because bash can read file using redirection, also with quotes it will be treated as a single argument and without it will split at space tab and newlines.
Bash syntax to read a file line by line, but will be slow for big files
while IFS= read -r line; do ... "$line"; done < text.file
unsetting IFS for read command preserves leading spaces
-r option preserves \
another way, to read whole file is content=$(<file), note the < inside the command substitution. so a creative way to read a file to array, each element a non-empty line:
read_to_array () {
local oldsetf=${-//[^f]} oldifs=$IFS
set -f
IFS=$'\n' array_content=($(<"$1")) IFS=$oldifs
[[ $oldsetf ]]||set +f
}
read_to_array "file"
for element in "${array_content[#]}"; do ...; done
oldsetf used to store current set -f or set +f setting
oldifs used to store current IFS
IFS=$'\n' to split on newlines (multiple newlines will be treated as one)
set -f avoid glob expansion for example in case line contains single *
note () around $() to store the result of splitting to an array
If I were to create a solution determined by the literal of what you ask for (using a for loop and parsing lines from a file) I would use iterations determined by the number of lines in the file (if it isn't too large).
Assuming each line has two strings separated by a single space (to be used as positional parameters in your script:
file="$1"
f_count="$(wc -l < $file)"
for line in $(seq 1 $f_count)
do
script.sh $(head -n $line $file | tail -n1) && wait
done
You may have a much better time using sjsam's solution however.
I am trying to cat a file.txt and loop it twice through the whole content and copy it to a new file file_new.txt. The bash command I am using is as follows:
for i in {1..3}; do cat file.txt > file_new.txt; done
The above command is just giving me the same file contents as file.txt. Hence file_new.txt is also of the same size (1 GB).
Basically, if file.txt is a 1GB file, then I want file_new.txt to be a 2GB file, double the contents of file.txt. Please, can someone help here? Thank you.
Simply apply the redirection to the for loop as a whole:
for i in {1..3}; do cat file.txt; done > file_new.txt
The advantage of this over using >> (aside from not having to open and close the file multiple times) is that you needn't ensure that a preexisting output file is truncated first.
Note that the generalization of this approach is to use a group command ({ ...; ...; }) to apply redirections to multiple commands; e.g.:
$ { echo hi; echo there; } > out.txt; cat out.txt
hi
there
Given that whole files are being output, the cost of invoking cat for each repetition will probably not matter that much, but here's a robust way to invoke cat only once:[1]
# Create an array of repetitions of filename 'file' as needed.
files=(); for ((i=0; i<3; ++i)); do files[i]='file'; done
# Pass all repetitions *at once* as arguments to `cat`.
cat "${files[#]}" > file_new.txt
[1] Note that, hypothetically, you could run into your platform's command-line length limit, as reported by getconf ARG_MAX - given that on Linux that limit is 2,097,152 bytes (2MB) that's not likely, though.
You could use the append operator, >>, instead of >. Then adjust your loop count as needed to get the output size desired.
You should adjust your code so it is as follows:
for i in {1..3}; do cat file.txt >> file_new.txt; done
The >> operator appends data to a file rather than writing over it (>)
if file.txt is a 1GB file,
cat file.txt > file_new.txt
cat file.txt >> file_new.txt
The > operator will create file_new.txt(1GB),
The >> operator will append file_new.txt(2GB).
for i in {1..3}; do cat file.txt >> file_new.txt; done
This command will make file_new.txt(3GB),because for i in {1..3} will run three times.
As others have mentioned, you can use >> to append. But, you could also just invoke cat once and have it read the file 3 times. For instance:
n=3; cat $( yes file.txt | sed ${n}q ) > file_new.txt
Note that this solution exhibits a common anti-pattern and fails to properly quote the arguments, which will cause issues if the filename contains whitespace. See mklement's solution for a more robust solution.
I am new to bash and writing a script to read variables that is stored on each line of a text file (there are thousands of these variables). So I tried to write a script that would read the lines and automatically output the solution to the screen and save into another text file.
./reader.sh > solution.text
The problem I encounter is currently I have only 1 variable store in the Sheetone.txt for testing purpose which should take about 2 seconds to output everything but it is stuck in the while loop as well as is not outputting the solution.
#!/bin/bash
file=Sheetone.txt
while IFS= read -r line
do
echo sh /usr/local/test/bin/test -ID $line -I
done
As indicated in the comments, you need to provide "something" to your while loop. The while construct is written in a way that will execute with a condition; if a file is given, it will proceed until the read exhausts.
#!/bin/bash
file=Sheetone.txt
while IFS= read -r line
do
echo sh /usr/local/test/bin/test -ID $line -I
done < "$file"
# -----^^^^^^^ a file!
Otherwise, it was like cycling without wheels...
I'm trying to write a pretty basic script in Linux shell but I'm still learning. Basically, everything is good to go except one part. I direct two outputs into the same file, e.g.:
echo `losetup -a` > partitionfile
echo "p1" >> partition final
Basically, I need to add the letter/number "p1" to the end of whatever is written in the file.
The problem is, it ends up being read (cat partitionfile) as:
/dev/loop0
p1
I need it on the same line to it reads out as:
/dev/loop0p1
There has to be a way to fix this, I just don't know it. Any help would be much appreciated!
Thanks!
I would go for:
echo "$(losetup -a)p1" > partitionfile
For an example, see the following transcript:
pax> echo "$(echo xyzzy_)p1"
xyzzy_p1
The xyzzy_ is the output of the inner echo command (which in your case would be losetup) and the outer echo command appends p1.
Hi Actually the correct option of echo to achieve this is "\c"
\c Keeps the cursor on the same line.
However you cannot use \c unless you have enabled it with
-e
Thus your code should be something like this ...
echo -e "`losetup -a` \c" > partitionfile
echo "p1" >> partition final
this will write in partitionfile as
< output of losetup -a > p1
everything on same line.
You can pass -n flag to the first echo statement to not print the trailing new line.
Ref: http://linux.die.net/man/1/echo