I know the following code creates a file core-site.xml in the /opt/hadoop/conf directory. Can some one please break it down in linux shell terms for me? Especially the << operator & CORE_EOF? How does those markers work? I kind of understand this but wanted to know better.
cat >/opt/hadoop/conf/core-site.xml <<CORE_EOF
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:8020</value>
</property>
</configuration>
CORE_EOF
A command with the << operator will do the following things :
Launch the program specified in the left of the operator, cat for instance.
Grab user input, including newlines, until what is specified on the right of the operator is met on one line, EOF for instance
Send all that have been read except the EOF value to the standard input of the program on the left.
cat << EOF
Hello
World
EOF
Will send "Hello
World"
To the standard input of cat.
It is the same as doing this:
cat < file
With file containing :
Hello
World
Cat make a new file or rewrite old with same name in this condition
and put your string into file.
When you want to add strings into file
type this:
cat >> /opt/hadoop/conf/core-site.xml << EOF
String
String
EOF
Related
I have an XML file like shown below(XML response will be always same). I need to extract sessionToken value and use it in Jenkins file.
XML file -
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<XmlResponse><httpCode>200</httpCode><httpStatus>OK</httpStatus><action>None</action><messageLevel>INFO</messageLevel><objectsList>{"sessionToken":"1234567890"}</objectsList><results/></XmlResponse>
This is the code I tried in Jenkins and didn't work -
def var=sh([returnStdout: true, script: '`cat output.xml | cut -f10 -d"\\""`'])
println ("var is" + var)
Here is the output I see in Jenkins console log -
++cat output.xml
+ 1234567890 -----> session Token is extracted in this step but for some reason it assume this as command
/workspace/script.sh: line 1: 1234567890: command not found
Answer
Depending on the version of Jenkins you are running, pipe can be an issue. Please try the following solution
def var=sh([returnStdout: true, script:'/bin/bash -c \'`cat output.xml | cut -f10 -d"\\""`\''])
I am using linux commands to generate a file "a.txt" . Now I need to read the first word of file "a.txt" and update an existing file called "b.txt". I will search a word called "/fill " in b.txt and replace it with the word read from a.txt Below is the code
bash 'example' do
code <<-EOH
cat ex.txt >> a.txt
EOH
end
test = /#{'cat /a.txt'}/
file_names = ['/b.txt']
file_names.each do |file_name|
text = File.read(file_name)
new_contents = text.gsub(/fill, test)
puts new_contents
File.open(file_name, "w") {|file| file.puts new_contents }
With the help of linux command "cat ex.txt >> a.txt " I am putting the content of ex.txt to a.txt.
After this I want to read the file a.txt with test = /#{'cat /a.txt'}/. Example a.txt contains "azure" word
Now in b.txt I want to search for a word "/fill" and replace with content read in step 2 from b.txt file ie azure
The problem is instead of replacing /fill with azure, /fill is getting replaced with cat /a.txt.
Hope its clear now
Can you please help here
It is a bit hard to follow, what you actually want to achieve. Your code has a few issues. General advice:
put your ruby code inside a ruby_block resource so that is executed during Chef's convergence phase
Use Chef::Util::FileEdit, if you want to edit files that are not entirely managed by Chef (see this question for more inspiration)).
In case you really want to write the complete file using Chef, use a file resource and specify the content based on what you've read using File.read.
As said, ruby code outside of ruby_block is executed in the compile phase (which precedes the convergence phase). If you this is too early (because the source file isn't there yet, you can use a lazy block for lazy evaluation:
file "b.txt" do
content lazy { File.read .. }
end
I would like to create a shell script that writes some configuration settings to a xml configuration file on Ubuntu. However, the settings are for an MQ cluster and I need the script to loop through a varying number of times (set by an input parameter) for each of the nodes that are being established.
The xml I would like to write to the file is:
<listeners>
<tcp-listener>
<port>1883</port>
<bind-address>10.0.0.4</bind-address>
</tcp-listener>
</listeners>
<mqtt>
<max-client-id-length>65535</max-client-id-length>
<retry-interval>10</retry-interval>
<max-queued-messages>1000</max-queued-messages>
</mqtt>
<cluster>
<enabled>true</enabled>
<transport>
<tcp>
<bind-address>10.0.0.4</bind-address>
<bind-port>7800</bind-port>
</tcp>
</transport>
<discovery>
<static>
<node>
<host>10.0.0.5</host>
<port>7800</port>
</node>
<node>
<host>10.0.0.6</host>
<port>7800</port>
</node>
<node> n times </node>
</static>
</discovery>
<failure-detection>
<heartbeat>
<enabled>true</enabled>
<interval>5000</interval>
<timeout>15000</timeout>
</heartbeat>
</failure-detection>
</cluster>
So basically, the number of <node> objects needs to reflect the variable the script takes in.
But, I am not sure how to loop through based on writing to the file. I was looking into using the tee command, but this doesn't let me loop though. I guess I could write the file up to the node object then loop through doing a write based on
Here is what I have so far that just writes static text:
tee /opt/hivemq/conf/config.xml > /dev/null <<'EOF'
<the xml goes here>
exit 0
EOF
Is there a way to loop during the write? Or do I need to write up to the looped object, stop writing then have a loop that does multiple writes based on the loop counter, and then finally write the last bit.
Any help would be greatly appreciated.
There's a command called 'seq' that you could use to help with iterating, so try something like :
#!/bin/bash
end=$1
(
echo start
for num in $(seq 1 $end)
do
node="10.0.0.$num"
echo $node
done
echo end
) > out.xml
I need to replace a particular character in a text file with another character. For example, replacing "E" with "A":
Apple ice → ApplA Ica
While executing sed 's/E/A' < apple.txt > app.txt I receive the error
function cannot be parsed
Please help! I need to automate this using Antscript.
You should terminate your sed command with a slash (/) and I guess you want to exchange all occurences of E with A? Then you have to add a g for a global substitution:
sed 's/E/A/g' app.txt
sed 's/E/A/g' app.txt. You missed the trailing / (g means all occurrences),
Since you are in Ant environment, you probably don't need to execute sed at all, but rather use Copy task with filter, or ReplaceRegExp task.
<?xml version="1.0" encoding="UTF-8"?>
<project>
<replaceregexp file="apple.txt" flags="g" match="e" replace="A"/>
</project>
This alters the file in place:
$ cat apple.txt
Apple ice
$
$ ant
Buildfile: build.xml
BUILD SUCCESSFUL
Total time: 0 seconds
$
$ cat apple.txt
ApplA icA
Your example is strange with case (a|A, e|E). I'll assume that's typo.
Follow up: To declare encoding...
<?xml version="1.0" encoding="utf-8"?>
<project>
<replaceregexp file="apple.txt" encoding="utf-8" flags="g" match="Á" replace=" "/>
</project>
I tested this successfully. Before:
ApplA icA
ApplÁs icÁs
After:
ApplA icA
Appl s ic s
When "cat > xx.txt << EOF" is entered on the command line, further input from cmdline goes to file xx.txt until EOF is written. EOF is not a sacred word here, if instead the command was cat > xx.txt << BBB, then cmdline input goes to xx.txt until BBB is written. I don't know whats the rationale behind ( << end_of_input_sequence)this. Cat man page does not explain much.
I have only seen this in scripts etc.
It's a feature of the shell, not cat - that's why you won't find it in the cat manual.
It's known as a "Here document" - see this page of the Advanced Bash-Scripting Guide for some documentation.
This is called a here document. I believe it first appeared in shells, but some programming languages such as Perl, Ruby, and PHP also implement this style.
This syntax is called Here Document (scroll a bit to find it).
It's not specific to any command, not cat anymore than any other command ; and it can be find in the man of the shell ; for instance, man bash :
3.6.6 Here Documents
This type of redirection instructs the
shell to read input from the current
source until a line containing only
word (with no trailing blanks) is
seen. All of the lines read up to that
point are then used as the standard
input for a command.
(Not a full quote -- there is more to read in the man)
BTW, It's a syntax that has been re-used in some programming languages, like PHP ;-)