Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I am using Mac OS and Bash terminal to do some basic scripting.
I have some questions which I could not find answers when I searched online
(or may be my search keywords were bad)
Firstly, I see there are three paths: /bin, /usr/bin and /usr/local/bin. some commands like grep are found in /usr/bin, while some others like ls are found in /bin.
why we have three different bins and what criteria goes to put commands like these over there
Secondly, I want to know the difference between using ' and `.
echo `date`
Fri Jan 10 10:36:52 PST 2014
awk '{print $1}' test.txt
1
2
3
4
if I try:
awk `{print $1}` test.txt
-bash: {print: command not found
awk: syntax error at source line 1
context is
>>> test. <<< txt
awk: bailing out at source line 1
so when to use ' and `.
Lastly, the above awk with print works. but this does not work
awk '{echo $1}' test.txt
can print and echo not be used interchangebly?
P.S: I am a beginner with bash scripting, please be kind
Brief answers not necessarily exhaustive:
/bin is usually for system commands
/usr/bin is for commands for users
/usr/local/bin is for software not typically installed by a distribution or release of an OS.
Quotes:
Things in single quotes are not touched by the shell
Things in double quotes are variable expanded by the shell ($var is expanded to what $var contains)
Things in back quotes are executed as a command and the output of that command replaces what was in the back quotes. You can also use $(echo Hello) to achieve the same thing.
You cannot easily mix awk and shell inside a string passed as a script to awk. (ex: awk '{echo $1}' test.txt)
awk `{print $1}` test.txt
This fail since you are using back tics and not single quotes.
Correct:
awk '{print $1}' test.txt
It will then print first field of all line in the file.
awk '{echo $1}' test.txt
Does not work since echo is not an awk command.
Tell us what text you have and what you like to get out of it..
You need to read a book. I recommend Shell Scripting Recipes: A Problem-Solution Approach (http://www.amazon.com/Shell-Scripting-Recipes-Problem-Solution-Approach/dp/1590594711). Read it, work through the exercises, and then come back with questions if you have any (or even better ask them at the comp.unix.shell newsgroup where that book's author and all the other shell experts hang out).
print is an awk command.
echo is a shell command.
backticks and $() run shell commands.
' quotes the contents such that the shell does not expand anything inside them.
$(echo hello) is the same as: echo hello. By doing this you ask the bash interpreter to execute the result of that command which is "hello". But hello is not a command (unless if you define it yourself).
These examples are all correct:
foo=$(echo hello)
echo ${foo}
foo=(echo "echo hello")
${foo}
Related
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I want to search for a String by navigating to a particular line, How to do this in shell scripting?
For example,
I have
this is the first line
this is the Second line
This is the Third line
Now here i would want to look for string "Third" by going to 3rd line.
Any help is appreciated, Thank you.
Try stringing together cat, sed, and grep.
sed '3!d' filename | grep Third
The unnamed or anonymous pipe (|) and redirection (<, >) are powerful features of many shells. They allow one to combine a set of commands to perform a more complex function.
In the case of this question there were two clear steps,
1) Operate on a specific line of a file (e.g. filter a file)
2) Search the output of the filter for a specific string
Recognizing that there were two steps is a strong indicator that two commands will need to be combined. Therefore, the problem can be solved by finding a solution to each step and then combining them in to one command with pipes and redirection.
If you know about the Stream Editor (sed), it may come to your mind when thinking about how to accomplish the first step of filtering the file. If not searching for, "linux get a specific line of a file" this OS question comes up high in the search results.
$ cat tmp.txt
this is the first line
this is the Second line
This is the Third. line
$ sed '3!d' tmp.txt
This is the Third. line
Knowing that grep can be search for lines with the string of interest the next challenge is to figure out how to get the output of sed as the input to grep. The pipe (|) solves this problem.
sed '3!d' filename | grep Third
Example output:
$ sed '3!d' tmp.txt | grep Third
This is the Third. line
$
Another powerful concept in shell scripting is the exit status. The grep command will set the exit status to 0 when a match is found and 1 when a match is not found. The shell stores the exit status in a special variable named $? (for bash). Therefore, one could use the exit status to conditionally determine the next step in the shell script. The example below does not implement conditions (like if, else). The example below shows the exit status value using the echo command.
$ sed '3!d' tmp.txt | grep Third
This is the Third. line
$ echo $?
0
$ sed '3!d' tmp.txt | grep third
$ echo $?
1
$
This question already has answers here:
Variables overwriting text problem with "echo" in Bash
(2 answers)
Closed 4 years ago.
I wrote a script to change .CSV to json.
#!/bin/bash
exec 0< example.csv
while IFS=, read name element input decrease
do
echo "${element}decrease: ${decrease}test"
done
the example.csv I paste here
name1,A,11,12
name2,B,13,14
But the output is really weird...
testrease: 12
testrease: 14
As u can see, The test rewrite Adecrease and Bdecrease, makes them to testrease.
I can't believe it!! So I tried with out exec 0< example.csv, type them in stdin, this time I got what I want
name1,A,11,12
Adecrease: 12test
So I guess maybe there are some characters in example.csv I can't see which makes this problem. I use cat -v example.csv
name1,A,11,12^M
name2,B,13,14^M
Nothing strange and I stuck here.
I am very new to shell script, so if anyone can give me some suggestions I will be really thrilled!!
Thank u, #chepner! tag wiki saves me another hour on this stupid question.
And here is the solution from wiki:
Check whether your script or data has DOS style end-of-line characters.
Use cat -v yourfile or echo "$yourvariable" | cat -v.
DOS carriage returns will show up as ^M after each line.
If you find them, delete them using dos2unix (a.k.a. fromdos) or tr -d '\r'.
This question already has answers here:
How do I use a variable argument number in a bash script?
(3 answers)
Closed 6 years ago.
What's going on everyone? I have this assignment in my Linux Operating System class and im having a little trouble. It asks me to,
Write a Bash shell script called chkread that takes an unlimited number of
arguments that all represent file names.
I have come up with this so far, but i don't think it's exactly what the professor is looking for.
~$ cat MY_SCRIPT
#!/bin/bash
echo ${unlimited arguments}
~$ bash MY_SCRIPT cat dog horse
cat dog horse
You could use something like this to handle multiple arguments in a bash script.
#!/bin/bash
file_names=("$#")
for name in "${file_names[#]}"; do
echo "$name"
done
And then when you call the script:
bash chkread.sh file1 file2 file3 file4
The script will print them to output:
file1
file2
file3
file4
But this is just an example. Inside the script you can do with them whatever you need to do.
Use $*. That will give you everything. For example:
#!/bin/bash
file_names=$*
echo $file_names
Output:
jbanks#efsappdev1:~$ x.sh `ls *.sql`
current.sql goop.sql latest.sql long.sql report.sql
jbanks#efsappdev1:~$ x.sh one two three
one two three
I have a situation here.
I have lot of files like below in linux
SIPTV_FIPTV_ID00$line_T20141003195717_C0000001000_FWD148_IPV_001.DATaac
SIPTV_FIPTV_ID00$line_T20141003195717_C0000001000_FWD148_IPV_001.DATaag
I want to remove the $line and make a counter from 0001 to 6000 for my 6000 such files in its place.
Also i want to remove the trailer 3 characters after this is done for each file.
After fix file should be like
SIPTV_FIPTV_ID0000001_T20141003195717_C0000001000_FWD148_IPV_001.DAT
SIPTV_FIPTV_ID0000002_T20141003195717_C0000001000_FWD148_IPV_001.DAT
Please help.
With some assumption, I think this should do it:
1. list of the files is in a file named input.txt, one file per line
2. the code is running in the directory the files are in
3. bash is available
awk '{i++;printf "mv \x27"$0"\x27 ";printf "\x27"substr($0,1,16);printf "%05d", i;print substr($0,22,47)"\x27"}' input.txt | bash
from the command prompt give the following command
% echo *.DAT??? | awk '{
old=$0;
sub("\\$line",sprintf("%4.4d",++n));
sub("...$","");
print "mv", old, $1}'
%
and check the output, if it looks OK
% echo *.DAT??? | awk '{
old=$0;
sub("\\$line",sprintf("%4.4d",++n));
sub("...$","");
print "mv", old, $1}' | sh
%
A commentary: echo *.DAT??? is meant to give as input to awk a list of all the filenames that you want to modify, you may want something more articulated if the example names you gave aren't representative of the whole spectrum... regarding the awk script itself, I used sprintf to generate a string with the correct number of zeroes for the replacement of $line, the idiom `"\\$..." with two backslashes to quote the dollar sign is required by gawk and does no harm in mawk, and as a last remark I have to say that in similar cases I prefer to make at least a dry run before passing the commands to the shell...
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I have this script I'm studying and I would like to know what is cat doing in this section.
if cat downloaded.txt | grep "$count" >/dev/null
then
echo "File already downloaded!"
else
echo $count >> downloaded.txt
cat $count | egrep -o "http://server.*(png|jpg|gif)" | nice -n -20 wget --no-dns-cache -4 --tries=2 --keep-session-cookies --load-cookies=cookies.txt --referer=http://server.com/wallpaper/$number -i -
rm $count
fi
Like most cats, this is a useless cat.
Instead of:
if cat downloaded.txt | grep "$count" >/dev/null
It could have been written:
if grep "$count" download.txt > /dev/null
In fact, because you've eliminated the pipe, you've eliminated issues with which exit value the if statement is dealing with.
Most Unix cats you'll see are of the useless variety. However, people like cats almost as much as they like using a grep/awk pipe, or using multiple grep or sed commands instead of combining everything into a single command.
The cat command stands for concatenate which is to allow you to concatenate files. It was created to be used with the split command which splits a file into multiple parts. This was useful if you had a really big file, but had to put it on floppy drives that couldn't hold the entire file:
split -b140K -a4 my_really_big_file.txt my_smaller_files.txt.
Now, I'll have my_smaller_files.txt.aaaa and my_smaller_files.txt.aaab and so forth. I can put them on the floppies, and then on the other computer. (Heck, I might go all high tech and use UUCP on you!).
Once I get my files on the other computer, I can do this:
cat my_smaller_files.txt.* > my_really_big_file.txt
And, that's one cat that isn't useless.
cat prints out the contents of the file with the given name (to the standard output or to wherever it's redirected). The result can be piped to some other command (in this case, (e)grep to find something in the file contents). Concretely, here it tries to download the images referenced in that file, then adds the name of the file to downloaded.txt in order to not process it again (this is what the check in if was about).
http://www.linfo.org/cat.html
"cat" is a unix command that reads the contents of one or more files sequentially and by default prints out the information the user console ("stdout" or standard output).
In this case cat is being used to read the contents of the file "downloaded.txt", the pipe "|" is redirecting/feeding its output to the grep program, which is searching for whatever is in the variable "$count" to be matched with.