I have an array as follows:
$servers = [
{name => 'server1', address => '10.10.10.11', port => 8081},
{name => 'server2', address => '10.10.10.12', port => 8082},
]
and I need to programmatically convert it into a hash based on the 'name' key:
$serversMap = {
server1 => {address => '10.10.10.11', port => 8081},
server2 => {address => '10.10.10.12', port => 8082},
}
this works in puppet 3:
$serversMap = $servers.reduce({}) |$cumulate, $server| {
$key = "${server['name']}"
$value = $server.filter |$entry| { $entry[0] != 'name' }
$tmp = merge($cumulate, {"$key" => $value })
$tmp
}
however, is there an easier way?
and why I need to create the temporal variable $tmp?
without that, I get at Error: Function 'merge' must be the value of a statement error.
ps: it's obvious but I will say it anyway: the $servers variable is given to me, I cannot change its structure. and I need that same data but in the form of $serversMap. and so, I am asking how to programmatically convert from $servers to $serversMap.
Except for the $tmp variable this code is optimal.
The $tmp variable needs to be there, because the merge function is an rvalue. From the docs:
There are actually two completely different types of functions available — rvalues (which return a value) and statements (which do not). If you are writing an rvalue function, you must pass :type => :rvalue when creating the function; see the examples below.
The puppet source that produces this error is pretty self explanatory when it comes to rvalue statements:
# It is harmless to produce an ignored rvalue, the alternative is to mark functions
# as appropriate for both rvalue and statements
# Keeping the old behavior when a pblock is not present. This since it is not known
# if the lambda contains a statement or not (at least not without a costly search).
# The purpose of the check is to protect a user for producing a meaningless rvalue where the
# operation has no side effects.
#
if !pblock && Puppet::Parser::Functions.rvalue?(#name)
raise Puppet::ParseError,
"Function '#{#name}' must be the value of a statement"
end
The real reason is that Puppet is just not that sophisticated as a language. If you want real power switch to Ruby.
Related
Looking for right way to detect one keyword during board boot up message.
After keyword detected, send Enter key after one second.
Kernel is Linux.
# Serial port inisialisation is finished here.
# Read boot message
($count, $result) = $ob->read(300); # at least 300 chars coming till keyword appear
if ($result =~ m/Booting_up/) {
print "send Enter ...\n";
sleep 1;
$ob->write("\r\n");
}
Thanks for hint
It appears that you are using Win32::SerialPort module, or perhaps Device::SerialPort which
provides an object-based user interface essentially identical to the one provided by the Win32::SerialPort module.
Its method read takes the number of bytes to read and returns the number read and writes them into the given string.
You may be "missing" the phrase because it's past the 300-mark, and your code doesn't read any further. Try to loop, getting a few bytes at a time and adding them up, thus building the string in small reads.
my bytes_in = 10; # length of pattern, but it does NOT ensure anything
my ($read, $result);
while (1)
{
my ($count, $read) = $ob->read( $bytes_in );
$result = $result . $read;
if ($result =~ m/Booting_up/) { # is it "Booting_up" or "Booting up" ?
print "send Enter ...\n";
sleep 1; # is this needed?
$ob->write("\r\n");
# last; # in case this is all you need to do
}
last if $count != $bytes_in; # done reading
}
I don't put the $ob->read statement in the loop condition since the documentation isn't crystal clear on how the method works. You may also be able to simply use
while ( my ($count, $read) = $ob->read( $bytes_in ) ) {
$result = $result . $read;
if ($result =~ m/Booting_up/s) {
# ...
}
last if $count != $bytes_in;
}
We read a small number of bytes at a time to prevent problems with either polling or blocking reads, brought up in comments by BenPen. See Configuration and capability methods.
You can first read those first 300 bytes that precede the pattern in one go and then start reading a few (or one) at a time, which would also lead to the quickest identification of the phrase.
This can be tweaked further but let's first see what it does as it stands (I cannot test).
Documentation also offers a few other methods which may be useful, in particular readline and streamline. As this is all rather low level there are yet other ways but if you got all else working perhaps this will be enough to complete it.
Perhaps rather index the string?
($count, $result) = $ob->read(300); # at least 300 chars coming till keyword appear
$substring = 'Booting_up';
if (index($result, $substring) != -1) {
print "send Enter ..\n";
sleep 1;
$ob->write("\r\n");
}
I know there's no built in "line count" functionality while processing files through logstash (for various, understandable and documented reasons). But - there should be a mechanism, within any given logstash instance - to have an monotonically increasing variable / count for every parsed line.
I don't want to go the metrics route since it's a continuous polling mechanism (every n-seconds). Alternatives include pre-processing of log files which given my particular use case - is unacceptable.
Again, let me reiterate - I need the ability to generate/read a monotonically increasing variable that I can store during in a logstash filter.
Thoughts?
here's nothing built into logstash to do it.
You can build a filter to do it pretty easily
Just drop something like this into lib/logstash/filters/seq.rb
# encoding: utf-8
require "logstash/filters/base"
require "logstash/namespace"
require "set"
#
# This filter will adds a sequence number to a log entry
#
# The config looks like this:
#
# filter {
# seq {
# field => "seq"
# }
# }
#
# The `field` is the field you want added to the event.
class LogStash::Filters::Seq < LogStash::Filters::Base
config_name "seq"
milestone 1
config :field, :validate => :string, :required => false, :default => "seq"
public
def register
# Nothing
end # def register
public
def initialize(config = {})
super
#threadsafe = false
# This filter needs to keep state.
#seq=1
end # def initialize
public
def filter(event)
return unless filter?(event)
event[#field] = #seq
#seq = #seq + 1
filter_matched(event)
end # def filter
end # class LogStash::Filters::Seq
This will start at 1 every time Logstash is restarted, but for most situations, this would be ok. If you need something that is persistent across restarts, you need to do a bit more work to persist it somewhere
For anyone finding this in 2018+: logstash now has a ruby filter that makes this much simpler. Put the following in a file somewhere:
# encoding: utf-8
def register(params)
#seq = 1
end
def filter(event)
event.set("seq", #seq)
#seq += 1
return [event]
end
And then configure it like this in your logstash.conf (substitute in the filename you used):
ruby {
path => "/usr/local/lib/logstash/seq.rb"
}
It would be pretty easy to make the field name configurable from logstash.conf, but I'll leave that as an exercise for the reader.
I suspect this isn't thread-safe, so I'm running only a single logstash worker.
this is another choice to slove the problem,this work for me,thanks to the answer from the previous person about thread safe. i use seq field to sort my desc
this is my configure
logstash.conf
filter {
ruby {
code => 'event.set("seq", Time.now.strftime("%N").to_i)'
}
}
logstash.yml
pipeline.batch.size: 200
pipeline.batch.delay: 60
pipeline.workers: 1
pipeline.output.workers: 1
I've been tasked with creating a new server modification for Crysis Wars. I have ran into a particular issue that it cannot read the old ban-file (this is required in order to keep the server consistent). The Lua code itself does not seem to have any errors, but it's just not getting any of the data.
Looking at the code I'm using for this below, can you find anything wrong with it?
This is the code I'm using for this:
function rX.CheckBanlist(player)
local Root = System.GetCVar("sys_root");
local File = ""..Root.."System/Bansystem/Raptor.xml";
local FileHnd = io.open(File, "r");
for line in FileHnd:lines() do
if (not string.find(line, "User:Read")) then
System.Log("[rX] File Read Error: System/Raptor/Banfile.xml, The contents are unexpected.");
return false;
end
local Msg, Date, Reason, Type, Domain = string.match(line, "User:Read( '(.*)', { Date='(.*)'; Reason='(.*)'; Typ='(.*)'; Info='(.*)'; } );");
local rldomain = g_gameRules.game:GetDomain(player.id);
if (Domain == rldomain) then
return true;
else
return false;
end
end
end
Also, the actual file reads as this, but I can't get the " to work in Lua properly. Could this be the issue?
User:Read( "Banned", { Date="31.03.2011"; Reason="WEBSTREAM"; Typ="Inetnum"; Info="COMPUTER.SED.gg"; } );
You may prefer using Lua's [[ for multiline string when you want to include quotes inside quotes etc.
Also, you'd have to escape the ( and ) while matching:
local Msg, Date, Reason, Type, Domain = line:match([[User:Read%( "(.-)", { Date="(.+)"; Reason="(.+)"; Typ="(.+)"; Info="(.+)"; } %);]])
And the results will be as expected: http://codepad.org/gN8kSL6H
Can you think of a way to solve this problem in Puppet?
I have a custom fact with generates a string of IP addresses depending on the domain it is run on, it can resolve to have 1 to n addresses.
"10.1.29.1"
"10.1.29.1,10.1.29.5"
"10.1.29.1,10.1.29.5,10.1.29.7"
etc
I want to add these to the host file with a generated server names of servernameX for example;
10.1.29.1 myservername1
10.1.29.5 myservername2
10.1.29.7 myservername3
So how can you do this as puppet doesn't have an array iterator like "for each"?
Sadly, even if you go about and use a custom "define" to iterate over an array upon splitting your custom fact on a comma, the result will be rather not what you expect and not even close to a "for each" loop -- aside of causing you a headache, probably.
Said that, I am not sure if this is what you want to achieve, but have a look at this approach:
$fact = '1.1.1.1,2.2.2.2,3.3.3.3'
$servers = split($::fact, ',')
$count = size($servers)
$names = bracket_expansion("host[01-${count}].address")
file { '/tmp/test.txt':
content => inline_template('<%= #servers.each_with_index.map {|v,i| "#{v}\t\t#{#names[i]}\n" } %>'),
ensure => present
}
What we have there are two custom functions: size() and bracket_expansion(); which we then use values that they provide inside a hack that leverages the inline_template() function to render content of the file utilising parallel access to two arrays -- one with IP addresses from your fact and one with host names that should follow.
The result is a follows:
matti#acrux ~ $ cat | puppet apply
$fact = '1.1.1.1,2.2.2.2,3.3.3.3'
$servers = split($::fact, ',')
$count = size($servers)
$names = bracket_expansion("host[01-${count}].address")
file { '/tmp/test.txt':
content => inline_template('<%= #servers.each_with_index.map {|v,i| "#{v}\t\t#{#names[i]}\n" } %>'),
ensure => present
}
notice: /Stage[main]//File[/tmp/test.txt]/ensure: created
notice: Finished catalog run in 0.07 seconds
matti#acrux ~ $ cat /tmp/test.txt
1.1.1.1 host01.address
2.2.2.2 host02.address
3.3.3.3 host03.address
matti#acrux ~ $
Both size() and bracket_expansion() functions can be found here:
https://github.com/kwilczynski/puppet-functions/tree/master/lib/puppet/parser/functions/
I hope this helps a little :-)
Is there a way to parse thru a txt file that includes elements seperated by {}s
Here is a sample from the file:
virtual vs_devtnet_80 {
snat automap
pool pool_devtnet_80
destination 167.69.107.41:http
ip protocol tcp
profiles {
profile_http_health {}
tcp-lan-optimized {}
}
}
virtual vs_devdpp_4430 {
snat automap
pool pool_devdpp_5430
destination 167.69.107.31:https
ip protocol tcp
persist devdpp
profiles tcp-lan-optimized {}
}
virtual vs_devwww30_80 {
snat automap
pool pool_devwww30_80
destination 167.69.107.46:http
ip protocol tcp
profiles {
profile_http_health {}
tcp-lan-optimized {}
}
}
As you can see, the elements are separated, but with {}
Any help would be gladly appreciated. I was trying to use grep but it only returns one line...
I would like to be able to search by the top most element, for example searh.sh virtual vs_devtnet_80, and have it return the entire blob..furthermore perhaps be able to search for botht eh top layer and one of its sub layers, for example search.sh virtual vs_devtnet_80 pool which would return pool_devtnet_80
Something like:
cat .tmp | sed '/.*{/ ! { s/.*//g}'
this wont solve it completely, but i think it does something similar to what you want
Look at JSON pasers, they're written in all sort of languages, syntax looks similar enough to give you some ideas on how to tackle this.
Basically what you need to do is have a recursive function that calls itself whenever it encounters a '{' and returns the content whenever it encounters a '}'.
I wrote an article on a lisp-like parser that actually does just that. Check it out here for inspiration: http://www.codeproject.com/KB/linq/TinyLisp.aspx
Rgds Gert-Jan
This is Tcl syntax, so you can set up a mechanism to run it as a Tcl script that creates a data structure of itself.
#!/usr/bin/env tclsh
# create a safe slave interpreter
set slave [interp create -safe]
# set up the slave to parse the input file
$slave eval {
proc virtual {subkey body} {
eval $body
}
proc unknown {cmd args} {
upvar 1 subkey subkey ;# access the 'subkey' variable from the 'virtual' proc
if {[llength $args] == 1} {set args [lindex $args 0]}
dict set ::data $subkey $cmd $args
}
set data [dict create]
}
$slave expose source
# now do the parsing in the slave
$slave eval source [lindex $argv 0]
# fetch the data structure
set data [$slave eval set data]
# and do stuff with it.
dict for {subkey subdict} $data {
puts "$subkey destination = [dict get $subdict destination]"
}
And then, parser.tcl input_file outputs
vs_devaetnet_80 destination = 167.69.107.41:http
vs_devdpp_4430 destination = 167.69.107.31:https
vs_devwww30_80 destination = 167.69.107.46:http
This should give you the 2nd 'blob', for instance:
sed -n '/virtual vs_devdpp_4430.*/,/^}$/p' filename.txt
and pipe through grep -oP '(?<=pool ).*' to get what follows pool in that blob.
I ended up having to create a recursive function that first used strpos to find the search variable within the entire config file. Then using a recursive function pop'd on and pop'd off brackets to return the searched variable's entire body.