This question already has answers here:
Variables overwriting text problem with "echo" in Bash
(2 answers)
Closed 4 years ago.
I wrote a script to change .CSV to json.
#!/bin/bash
exec 0< example.csv
while IFS=, read name element input decrease
do
echo "${element}decrease: ${decrease}test"
done
the example.csv I paste here
name1,A,11,12
name2,B,13,14
But the output is really weird...
testrease: 12
testrease: 14
As u can see, The test rewrite Adecrease and Bdecrease, makes them to testrease.
I can't believe it!! So I tried with out exec 0< example.csv, type them in stdin, this time I got what I want
name1,A,11,12
Adecrease: 12test
So I guess maybe there are some characters in example.csv I can't see which makes this problem. I use cat -v example.csv
name1,A,11,12^M
name2,B,13,14^M
Nothing strange and I stuck here.
I am very new to shell script, so if anyone can give me some suggestions I will be really thrilled!!
Thank u, #chepner! tag wiki saves me another hour on this stupid question.
And here is the solution from wiki:
Check whether your script or data has DOS style end-of-line characters.
Use cat -v yourfile or echo "$yourvariable" | cat -v.
DOS carriage returns will show up as ^M after each line.
If you find them, delete them using dos2unix (a.k.a. fromdos) or tr -d '\r'.
Related
This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed last year.
I am new on coding in bash Linux and I have the following problem.
I'm trying to concatenate string in loop to create a path. I have a text file in which I stored some strings to use in the loop. I wrote this example just to show you the problem:
for bio in `cat /data/giordano/species_ranges/prova_bio.txt` # list of strings: "bio_01", "bio_02"...
do
echo /data/giordano/species_range/$bio.tif # concatenation
done
The result I expect would be:
/data/giordano/species_range/bio_01.tif
/data/giordano/species_range/bio_02.tif
/data/giordano/species_range/bio_03.tif
But what actually came out was:
.tifa/giordano/species_range/bio_01
.tifa/giordano/species_range/bio_02
.tifa/giordano/species_range/bio_03
/data/giordano/species_range/bio_04.tif
I really don't understand what kind of problem it is...
I suggest that awk would be simpler for this task. We use tr to remove the Cr line endings
~/tests/bash $ tr -d "\r" < data/giordano/species_range/proverbio.txt | awk '{ print "/data/giordano/species_range/" $0 ".tif"
> }'
/data/giordano/species_range/bio_1.tif
/data/giordano/species_range/bio_2.tif
/data/giordano/species_range/bio_3.tif
/data/giordano/species_range/bio_4.tif
Thank you to Charles Duffy for the improvements.
You probably have Windows line endings in your file, which contain an additional carriage return (\r). This makes the cursor go to the beginning of the line. You can remove the \rs from your file by piping to tr. Extend your first line like this:
for bio in `cat /data/giordano/species_ranges/prova_bio.txt | tr -d '\r'`
This question already has answers here:
Capturing multiple line output into a Bash variable
(7 answers)
Closed 3 years ago.
I have a small script where I appended the output of linux mpstat to a log file.
#/bin/bash
CPU_USAGE=$(mpstat)
echo $CPU_USAGE >> temp.log
The problem is that the output of mpstat on the terminal is formatted properly in 3 lines like so
However, the output to the file is all in one line.
How do I format the output like the one on the terminal?
Just quote the variable so it is not seen as several different parameters to be printed one after the other:
echo "$CPU_USAGE" >> temp.log
You could just directly pipe the output to the file:
#!/bin/bash
mpstat >> temp.log
If you must store it in a variable, then quote it like:
#!/bin/bash
CPU_USAGE=$(mpstat)
echo "$CPU_USAGE" >> temp.log
Otherwise, bash will not interpret the newlines as part of the message to echo, but the whole output as a list of short strings to output.
This question already has answers here:
How do I properly quote this bash pipeline for watch?
(2 answers)
Closed 3 years ago.
How can I echo this entire piece of code to .bashrc without leaving out a single character?
# automatic logging of terminal input/output
test "$(ps -ocommand= -p $PPID | awk '{print $1}')" == 'script' || (script -f -q /home/user/.logs/terminal/manjaro/$(date +"%Y-%m- %d_%H:%M:%S")_terminal.log)
When I attempt to enter the following into terminal:
echo "the above code" >> ~/.bashrc
I get the following appended to .bashrc which is nothing like "the above code", its short about 45 or so characters.
# automatic logging of terminal input/output
test script == 'script' || (script -f -q /home/user/.logs/terminal/manjaro/2019-05- 08_09:09:19_terminal.log)
As you can see, it's leaving out A LOT of the original code. I understand this has a lot to do with the number of different quotations and placement, but without altering my code much, or at least to the point where it can still function as its intended, how can I go about getting this to echo to the file properly?
Thank you for every nanosecond of your time.
Wrap your echo'd string with single quotes ' instead of double "
This question already has answers here:
Passing arguments to an interactive program non-interactively
(5 answers)
Closed 7 years ago.
So suppose a normal command run in terminal goes like this....
user$ thecommand
Please enter your first name:
>
and then waits for your to type your name... straightforward, but if in a bash script I try and do something like:
#! /bin/bash
echo "What is your name?"
read name
thecommand
how would I have THE SCRIPT enter "$name" in response to "thecommand" instead of having the user manually input it themselves?
you can add input by pipe like this:
echo yourname | ./yourscript
for more inputs you can use printf
printf "input1\ninput2" | ./yourscript
where \n means new line and it will be used like new input.
Run your script like:
./yourscript.sh < file.txt
where file.txt will contain the name.
now your script will look for name from the file(file.txt), in file.txt you can type the names which will act as input for read command.
read command reads on line at a time so if u have more than on read command in your script you should have multiple lines in file.txt file
For complicated cases, for example if your input depends on the output of your command, you may write an "expect" script.
To see how it works you can auto-generate such script interactively
$ autoexpect thecommand
And then run it
$ expect -f script.exp
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I am using Mac OS and Bash terminal to do some basic scripting.
I have some questions which I could not find answers when I searched online
(or may be my search keywords were bad)
Firstly, I see there are three paths: /bin, /usr/bin and /usr/local/bin. some commands like grep are found in /usr/bin, while some others like ls are found in /bin.
why we have three different bins and what criteria goes to put commands like these over there
Secondly, I want to know the difference between using ' and `.
echo `date`
Fri Jan 10 10:36:52 PST 2014
awk '{print $1}' test.txt
1
2
3
4
if I try:
awk `{print $1}` test.txt
-bash: {print: command not found
awk: syntax error at source line 1
context is
>>> test. <<< txt
awk: bailing out at source line 1
so when to use ' and `.
Lastly, the above awk with print works. but this does not work
awk '{echo $1}' test.txt
can print and echo not be used interchangebly?
P.S: I am a beginner with bash scripting, please be kind
Brief answers not necessarily exhaustive:
/bin is usually for system commands
/usr/bin is for commands for users
/usr/local/bin is for software not typically installed by a distribution or release of an OS.
Quotes:
Things in single quotes are not touched by the shell
Things in double quotes are variable expanded by the shell ($var is expanded to what $var contains)
Things in back quotes are executed as a command and the output of that command replaces what was in the back quotes. You can also use $(echo Hello) to achieve the same thing.
You cannot easily mix awk and shell inside a string passed as a script to awk. (ex: awk '{echo $1}' test.txt)
awk `{print $1}` test.txt
This fail since you are using back tics and not single quotes.
Correct:
awk '{print $1}' test.txt
It will then print first field of all line in the file.
awk '{echo $1}' test.txt
Does not work since echo is not an awk command.
Tell us what text you have and what you like to get out of it..
You need to read a book. I recommend Shell Scripting Recipes: A Problem-Solution Approach (http://www.amazon.com/Shell-Scripting-Recipes-Problem-Solution-Approach/dp/1590594711). Read it, work through the exercises, and then come back with questions if you have any (or even better ask them at the comp.unix.shell newsgroup where that book's author and all the other shell experts hang out).
print is an awk command.
echo is a shell command.
backticks and $() run shell commands.
' quotes the contents such that the shell does not expand anything inside them.
$(echo hello) is the same as: echo hello. By doing this you ask the bash interpreter to execute the result of that command which is "hello". But hello is not a command (unless if you define it yourself).
These examples are all correct:
foo=$(echo hello)
echo ${foo}
foo=(echo "echo hello")
${foo}