So This is related to my other two posts. Im dealing with extracting text from a text file and analyzing it and I've run into some problems. For A while I've been using a method that sets all the text between two other strings as a variable, but here is the situation I have. I need to extract the speed (numbers) from the below string: "etc...,query":{"ping":47855},"cmts":...etc. The problem is that the text cmts sometimes changes to something else so really I need to extract all the numbers from this:
,query":{"ping":47855},"
One more thing that makes this difficult is that the characters }," Are all over the file. Thank you for helping me! -Lucas EDG Programmer.
Here's the full file:
{"_id":53291,"ip":"158.69.22.95","domain":"jectile.com","port":25565,"url":"","date_add":1453897770,"status":1,"scan":1,"uptime":99.53,"last_update":1485436105,"geo":{"country":"US","country_name":"United States","city":"Lake Forest"},"info":{"name":" Jectile | jectile.com [1.8-1.11]\n Shoota (Call of Duty) \/ Zambies (Zombie Survival)","type":"FML","version":"1.10","plugins":[],"players":18,"max_players":420,"players_list":[],"map":"world","software":"BungeeCord 1.8.x, 1.9.x, 1.10.x, 1.11.x","avg_player_day":24.458333,"avg_load_day":5.8234,"platform":"MINECRAFT","icon":true},"counter":{"online":47871,"offline":228,"players":{"date":"2017-01-26","total":0},"last_offline":0,"query":{"ping":47855},"cmts":1},"rating":{"main":19.24,"difference":-0.64,"content_up":0.15,"K":0},"last":{"offline":1485415702,"online":1485436105},"chart":{"14:30":14,"14:40":16,"14:50":15,"15:00":18,"15:10":12,"15:20":13,"15:30":9,"15:40":9,"15:50":11,"16:00":12,"16:10":11,"16:20":11,"16:30":18,"16:40":25,"16:50":23,"17:00":27,"17:10":27,"17:20":23,"17:30":24,"17:40":26,"17:50":33,"18:00":31,"18:10":31,"18:20":32,"18:30":37,"18:40":38,"18:50":39,"19:00":38,"19:10":34,"19:20":33,"19:30":40,"19:40":36,"19:50":37,"20:00":38,"20:10":36,"20:20":38,"20:30":37,"20:40":37,"20:50":37,"21:00":34,"21:10":32,"21:20":33,"21:30":33,"21:40":29,"21:50":28,"22:00":26,"22:10":21,"22:20":24,"22:30":29,"22:40":22,"22:50":23,"23:00":27,"23:10":24,"23:20":26,"23:30":25,"23:40":28,"23:50":27,"00:00":32,"00:10":29,"00:20":33,"00:30":32,"00:40":31,"00:50":33,"01:00":40,"01:10":40,"01:20":40,"01:30":41,"01:40":45,"01:50":48,"02:00":43,"02:10":45,"02:20":46,"02:30":46,"02:40":43,"02:50":42,"03:00":39,"03:10":36,"03:20":44,"03:30":34,"03:40":0,"03:50":32,"04:00":35,"04:10":35,"04:20":33,"04:30":43,"04:40":37,"04:50":26,"05:00":31,"05:10":31,"05:20":27,"05:30":25,"05:40":26,"05:50":18,"06:00":13,"06:10":15,"06:20":17,"06:30":18,"06:40":17,"06:50":15,"07:00":16,"07:10":17,"07:20":16,"07:30":16,"07:40":18,"07:50":19,"08:00":14,"08:10":12,"08:20":12,"08:30":13,"08:40":17,"08:50":20,"09:00":18,"09:10":0,"09:20":0,"09:30":27,"09:40":18,"09:50":20,"10:00":15,"10:10":13,"10:20":12,"10:30":10,"10:40":10,"10:50":11,"11:00":13,"11:10":13,"11:20":16,"11:30":19,"11:40":17,"11:50":13,"12:00":10,"12:10":11,"12:20":12,"12:30":16,"12:40":15,"12:50":16,"13:00":14,"13:10":10,"13:20":13,"13:30":16,"13:40":16,"13:50":17,"14:00":20,"14:10":16,"14:20":16},"query":"ping","max_stat":{"max_online":{"date":1470764061,"players":129}},"status_query":"ok"}
By the way, the reason things change is because it looks at info from different servers
Very similar to ther answer I gave you to your first question:
#Echo Off
Set/P var=<some.json
Set var=%var:*:{"ping":=%
Set var=%var:},=&:%
Echo=%var%
Timeout -1
I'm writing code in .csh, and I'm trying to change the bunit header for a FITS file from K (kelvin) to km/s. How can I do that?
I know in Python I would use new_fitsfile.header['BUNIT']='km/s', but that won't work in the current .csh code, and it's not an option to switch it to Python code.
If this is needed only once, call interactively fv or ds9, move to the header, edit the header card and save the result.
For generic batch jobs, one needs some online FITS editor like fmodhead fmodhead, fthedit, or my fedithead
sed "s:BUNIT = 'K ':BUNIT = 'km/s ':g" old.fits >new.fits
and be very careful to count the significant spaces.
we have a requirement where contents of our text files are like this:
[some-section-1]
big_msg_line1 var=random_value1
big_msg_line2 var=random_value2
big_msg_line3 var=random_value3
[some-section-2]
"lots of irrelevant data"
[some-section-3]
"lots of irrelevant data"
[some-section-4]
big_msg_line4 var=random_value4
big_msg_line5 var=random_value5
big_msg_line6 var=random_value6
big_msg_line7 var=random_value7
big_msg_line8 var=random_value8
[some-section-5]
"lots of irrelevant data"
All the lines that we want to modify starts with common charaters, like in this example all lines which we would like to modify starts with the word "big". We would like to change it to something like this:
[some-section-1]
random_value1 msg=big_msg_line1
random_value2 msg=big_msg_line2
random_value3 msg=big_msg_line3
[some-section-2]
"lots of irrelevant data"
[some-section-3]
"lots of irrelevant data"
[some-section-4]
random_value4 msg=big_msg_line4
random_value5 msg=big_msg_line5
random_value6 msg=big_msg_line6
random_value7 msg=big_msg_line7
random_value8 msg=big_msg_line8
[some-section-5]
"lots of irrelevant data"
These were for examples only. The original file contains way lot more data than these. In hundreds if not in thousands lines.
I am currently doing this using for a loop, reading each line, cutting the values, formatting them like the way I want, putting then in separate file and then replace the original file with the new file. Is there a way to achieve this using some one liners? That would really be of great help. Hope I am clear with my question.
Thanks in advance.
From what I understood, this awk one-liner would do the job :
cat a
[some-section-1]
big_msg_line1 var=random_value1
big_msg_line2 var=random_value2
big_msg_line3 var=random_value3
[some-section-2]
lots of irrelevant data
[some-section-3]
lots of irrelevant data
[some-section-4]
big_msg_line4 var=random_value4
big_msg_line5 var=random_value5
big_msg_line6 var=random_value6
big_msg_line7 var=random_value7
big_msg_line8 var=random_value8
[some-section-5]
lots of irrelevant data
This :
awk '{FS="var="; if ($1~/big/) { print $2"\tmsg="$1} else {print }}' a
Gives
[some-section-1]
random_value1 msg=big_msg_line1
random_value2 msg=big_msg_line2
random_value3 msg=big_msg_line3
[some-section-2]
lots of irrelevant data
[some-section-3]
lots of irrelevant data
[some-section-4]
random_value4 msg=big_msg_line4
random_value5 msg=big_msg_line5
random_value6 msg=big_msg_line6
random_value7 msg=big_msg_line7
random_value8 msg=big_msg_line8
[some-section-5]
lots of irrelevant data
this command should do the job
sed -e 's/\(big[^ ]*\)\([ ]*\)var=\([^ ]*\)/\3\2msg=\1/' [your file] > [output file]
EDIT: You might need to change the slahes (/) to a letter which is not used in your file
I'm trying to create a Chef recipe to append multiple lines (20-30) to a specific config file.
I'm aware the recommended pattern is to change entire config files rather than just appending to a file, but I dislike this approach for multiple reasons.
So far the only solution I found was to use a cookbook_file and then use a bash resource to do:
cat lines_to_append >> /path/configfile
Obviously this wouldn't work properly, as it'd append the file over and over, each time you run chef-client. I'd have to create a small bash script to check for a specific string first, and, if not found, append to the file.
But this seems to defeat the purpose of using Chef. There must be a better way.
One promising solution was the line cookbook from OpsCode Community. It aimed to solve this exact problem. Unfortunately the functionality is incomplete, buggy, and the code is just a quick hack. Far from being a solid solution.
Another option I evaluated was augeas. Seems pretty powerful, but it'd add yet-another layer of abstraction to the system. Overkill, in my case.
Given that this is one of the most obvious tasks for any sysadmin, is there any easy and beautiful solution with Chef that I'm not seeing?
EDIT: here's how I'm solving it so far:
cookbook_file "/tmp/parms_to_append.conf" do
source "parms_to_append.conf"
end
bash "append_to_config" do
user "root"
code <<-EOF
cat /tmp/parms_to_append.conf >> /etc/config
rm /tmp/parms_to_append.conf
EOF
not_if "grep -q MY_IDENTIFIER /etc/config"
end
It works, but not sure this is the recommended Chef pattern.
As you said yourself, the recommended Chef pattern is to manage the whole file.
If you're using Chef 11 you could probably make use of partials for what you're trying to achieve.
There's more info here and on this example cookbook.
As long as you have access to the original config template, just append <%= render "original_config.erb" %> to the top of your parms_to_append.conf template.
As said before, using templates and partials is common way of doing this, but chef allows appending files, and even changing(editing) file lines. Appendind is performed using following functions:
insert_line_after_match(regex, newline);
insert_line_if_no_match(regex, newline)
You may find and example here on stackoverflow, and the full documentation on rubydoc.info
Please use it with caution, and only when partials and templates are not appropriate.
I did something like this:
monit_overwrites/templates/default/monitrc.erb:
#---FLOWDOCK-START
set mail-format { from: monit#ourservice.com }
#---FLOWDOCK-END
In my recipe I did this:
monit_overwrites/recipes/default.rb:
execute "Clean up monitrc from earlier runs" do
user "root"
command "sed '/#---FLOWDOCK-START/,/#---FLOWDOCK-END/d' > /etc/monitrc"
end
template "/tmp/monitrc_append.conf" do
source "monitrc_append.erb"
end
execute "Setup monit to push notifications into flowdock" do
user "root"
command "cat /tmp/monitrc_append.conf >> /etc/monitrc"
end
execute "Remove monitrc_append" do
command "rm /tmp/monitrc_append.conf"
end
The easiest way to tackle this would be to create a string and pass it to content. Of course bash blocks work... but I think file resources are elegant.
lines = ""
File.open('input file') do |f|
f.lines.each do |line|
lines = lines + line + "\n"
end
end
file "file path" do
content line
end
Here is the example ruby block for inserting 2 new lines after match:
ruby_block "insert_lines" do
block do
file = Chef::Util::FileEdit.new("/etc/nginx/nginx.conf")
file.insert_line_after_match("worker_rlimit_nofile", "load_module 1")
file.insert_line_after_match("pid", "load_module 2")
file.write_file
end
end
insert_line_after_match searches for the regex/string and it will insert the value in after the match.