I have a closure to find all files with name matching a pattern and containing a given String:
def path = "path/to/logs"
def namePatten = ~/.*.log/
def contentPattern ~/.*ERROR.*/
def result = []
new File(path).eachDirRecurse { File dir ->
dir.eachFileMatch(namePattern) { File f ->
f.eachLine { String l ->
if(l.matches(contentPattern)) {
result.add(f)
return
}
}
}
But I'm pretty sure I can have something shorter (hey, else I can use plain java :) )
I have tried to find a way to write this a bit like that:
result = new File(path).eachFileRecurse.filter(filePattern).grep(contentPattern)
as I would have done using guava or similar fluent interface collection tools.
How woud you write this closure in a concise, yet still readable, manner ?
The smallest I can get it to at present is to use the File.traverse method to recursively scan the root folder:
new File( path ).traverse( nameFilter: namePattern ) { f ->
if( f.filterLine( { it ==~ contentPattern } ) as String ) result << f
}
Using filterLine returns a Writable which I convert to a String as then we can exploit the Groovy truth to see whether to add the file to result or not.
Edit:
You can also use AntBuilder to do a similar thing:
def result = new AntBuilder().fileset( dir:'path/to/logs', includes:'**/*.log' ) {
containsregexp expression:'.*ERROR.*'
}*.file
Which I tend to prefer as it generates the list in one go, rather than adding results to an already defined results list.
Related
I have a file with integers for which I want the average. Below is what I came up with and it works, but was wondering if there was a better (groovier) way..
File myFile = new File("myListOfNumbers.txt")
int total = 0
int count = 0
myFile.eachLine {line->
total = Integer.parseInt(line) + total
count++
}
println "Avg is ${total /count}"
If you are looking for more declarative approach then you could do something like this:
def avg2 = myFile.readLines()
.sum({ it.toInteger() }) / myFile.readLines().size()
The only problem is that this is not very efficient, because here we are calling readLines() method twice. Of course we could split declaration from this expression to make code more efficient and then we will end up with something like this:
def lines = myFile.readLines()
def avg2 = lines.sum({ it.toInteger() }) / lines.size()
Mixing with Java 8 Stream API
Of course you can always mix Groovy approach with Java 8 approach and use built-in .average() method from Java Stream API. In this case your code would look similar to this one:
def avg1 = Files.lines(myFile.toPath())
.mapToInt({ s -> s.toInteger() })
.average()
.getAsDouble()
I hope it helps.
Groovier:
def mean = myFile.readLines().with { sum{it as Integer} / size() }
Uses groovyisms such as with and as.
Inside the closure passed to with, one can use sum() and size() directly, the target will be the object on which .with is invoked, in our case the list of lines.
Grooviest:
def mean = myFile.mean()
Of course I cheated, and did this beforehand:
File.metaClass.mean {
delegate.readLines().with { sum{it as Integer} / size() }
}
"When you've found the treasure, stop digging!"
I'm wanting to use more functional programming in Groovy, and thought rewriting the following method would be good training. It's harder than it looks because Groovy doesn't appear to build short-circuiting into its more functional features.
Here's an imperative function to do the job:
fullyQualifiedNames = ['a/b/c/d/e', 'f/g/h/i/j', 'f/g/h/d/e']
String shortestUniqueName(String nameToShorten) {
def currentLevel = 1
String shortName = ''
def separator = '/'
while (fullyQualifiedNames.findAll { fqName ->
shortName = nameToShorten.tokenize(separator)[-currentLevel..-1].join(separator)
fqName.endsWith(shortName)
}.size() > 1) {
++currentLevel
}
return shortName
}
println shortestUniqueName('a/b/c/d/e')
Result: c/d/e
It scans a list of fully-qualified filenames and returns the shortest unique form. There are potentially hundreds of fully-qualified names.
As soon as the method finds a short name with only one match, that short name is the right answer, and the iteration can stop. There's no need to scan the rest of the name or do any more expensive list searches.
But turning to a more functional flow in Groovy, neither return nor break can drop you out of the iteration:
return simply returns from the present iteration, not from the whole .each so it doesn't short-circuit.
break isn't allowed outside of a loop, and .each {} and .eachWithIndex {} are not considered loop constructs.
I can't use .find() instead of .findAll() because my program logic requires that I scan all elements of the list, nut just stop at the first.
There are plenty of reasons not to use try..catch blocks, but the best I've read is from here:
Exceptions are basically non-local goto statements with all the
consequences of the latter. Using exceptions for flow control
violates the principle of least astonishment, make programs hard to read
(remember that programs are written for programmers first).
Some of the usual ways around this problem are detailed here including a solution based on a new flavour of .each. This is the closest to a solution I've found so far, but I need to use .eachWithIndex() for my use case (in progress.)
Here's my own poor attempt at a short-circuiting functional solution:
fullyQualifiedNames = ['a/b/c/d/e', 'f/g/h/i/j', 'f/g/h/d/e']
def shortestUniqueName(String nameToShorten) {
def found = ''
def final separator = '/'
def nameComponents = nameToShorten.tokenize(separator).reverse()
nameComponents.eachWithIndex { String _, int i ->
if (!found) {
def candidate = nameComponents[0..i].reverse().join(separator)
def matches = fullyQualifiedNames.findAll { String fqName ->
fqName.endsWith candidate
}
if (matches.size() == 1) {
found = candidate
}
}
}
return found
}
println shortestUniqueName('a/b/c/d/e')
Result: c/d/e
Please shoot me down if there is a more idiomatic way to short-circuit in Groovy that I haven't thought of. Thank you!
There's probably a cleaner looking (and easier to read) solution, but you can do this sort of thing:
String shortestUniqueName(String nameToShorten) {
// Split the name to shorten, and make a list of all sequential combinations of elements
nameToShorten.split('/').reverse().inject([]) { agg, l ->
if(agg) agg + [agg[-1] + l] else agg << [l]
}
// Starting with the smallest element
.find { elements ->
fullyQualifiedNames.findAll { name ->
name.endsWith(elements.reverse().join('/'))
}.size() == 1
}
?.reverse()
?.join('/')
?: ''
}
I have an dynamic html file that groovy is generated from. Part of this html template format is {routeId}{groovyMap} like so
USER_FORM[name:'Dean', user:randomFunction([item:'s', day:'Tuesday'])]
or something like
USER_FORM[name: 'Dean', user: user]
I made the first example more complex. Currently, I split on ':' and validate all the keys supplied. What I would like to do is take the groovy snippet and grab all the keys and validate
1. all keys are strings
2. validate the keys against some meta data I already have
I do not care about the values at all. Currently, I split on ':' but obviously that won't work for all cases. I am worried about other complex cases I may not be thinking about.
This is for a templating engine and I prefer to failfast if possible making it easier on the user when something is wrong.
I concur with others that you want to avoid parsing directly.
If you use GroovyShell, you can dope the input string with no-op methodMissing and propertyMissing handlers. In this way, even the complex example will work.
See code below, including test-cases (extracting map string from the "USER_FORMstr" format is left to the reader).
class KeyGenerator {
// these could be "final static". omitted for brevity
def shell = new GroovyShell()
def methodMissingHandler = "def methodMissing(String name, args) {}"
def propertyMissingHandler = "def propertyMissing(String name) {}"
def generateKeys(mapStr) {
def evalInput = "${methodMissingHandler} ; " +
"${propertyMissingHandler} ; " +
"${mapStr}"
def map = shell.evaluate(evalInput)
return map.keySet()
}
}
// ------- main
def keyGenerator = new KeyGenerator()
def expected = new HashSet()
expected << "name"
expected << "user"
def mapStr = "[name:'Dean', user:randomFunction([item:'s', day:'Tuesday'])]"
assert expected == keyGenerator.generateKeys(mapStr)
def mapStr2 = "[name: 'Dean', user: user]"
assert expected == keyGenerator.generateKeys(mapStr2)
If I got you right, you can use something like:
String val = "USER_FORM[name:'Dean', user:randomFunction([item:'s', day:'Tuesday'])]"
def res = []
val.eachMatch( /[\[,] ?(\w+):/ ){ res << it[ 1 ] }
assert '[name, user, item, day]' == res.toString()
all keys are strings
When using the literal syntax for creating a Map, i.e.
Map m = [foo: 'bar']
as opposed to
Map m = new HashMap()
m.put('foo', 'bar')
the keys are always strings, even if you have a variable in scope with the same name as the key. For example, in the following snippet, the key will be the string 'foo', not the integer 6
def foo = 6
Map m = [foo: 'bar']
The only way you can create a Map using the literal syntax with a key that is not a string is if you have a variable in scope with the same name as the key and you wrap the key name in parentheses. For example, in the following snippet, the key will be the integer 6, not the string 'foo'
def foo = 6
Map m = [(foo): 'bar']
Currently, I split on ':' but obviously that won't work for all cases. I am worried about other complex cases I may not be thinking about.
Parsing a map literal using regex/string splitting seems like a bad idea as you'll likely end up badly re-implementing the Groovy lexer. Something like the following seems a better option
def mapString = '[foo: "bar"]'
Map map = Eval.me(mapString)
// now you can process the map via the Map interface, e.g.
map.keySet().toList() == ['foo']
I am new to groovy and I've been facing some issues understanding the each{} and eachwithindex{} statements in groovy.
Are each and eachWithIndex actually methods? If so what are the arguments that they take?
In the groovy documentation there is this certain example:
def numbers = [ 5, 7, 9, 12 ]
numbers.eachWithIndex{ num, idx -> println "$idx: $num" } //prints each index and number
Well, I see that numbers is an array. What are num and idx in the above statement? What does the -> operator do?
I do know that $idx and $num prints the value, but how is it that idx and num are automatically being associated with the index and contents of the array? What is the logic behind this? Please help.
These are plain methods but they follow quite a specific pattern - they take a Closure as their last argument. A Closure is a piece of functionality that you can pass around and call when applicable.
For example, method eachWithIndex might look like this (roughly):
void eachWithIndex(Closure operation) {
for (int i = 0; this.hasNext(); i++) {
operation(this.next(), i); // Here closure passed as parameter is being called
}
}
This approach allows one to build generic algorithms (like iteration over items) and change the concrete processing logic at runtime by passing different closures.
Regarding the parameters part, as you see in the example above we call the closure (operation) with two parameters - the current element and current index. This means that the eachWithIndex method expects to receive not just any closure but one which would accept these two parameters. From a syntax prospective one defines the parameters during closure definition like this:
{ elem, index ->
// logic
}
So -> is used to separate arguments part of closure definition from its logic. When a closure takes only one argument, its parameter definition can be omitted and then the parameter will be accessible within the closure's scope with the name it (implicit name for the first argument). For example:
[1,2,3].each {
println it
}
It could be rewritten like this:
[1,2,3].each({ elem ->
println elem
})
As you see the Groovy language adds some syntax sugar to make such constructions look prettier.
each and eachWithIndex are, amongst many others, taking so called Closure as an argument. The closure is just a piece of Groovy code wrapped in {} braces. In the code with array:
def numbers = [ 5, 7, 9, 12 ]
numbers.eachWithIndex{ num, idx -> println "$idx: $num" }
there is only one argument (closure, or more precisely: function), please note that in Groovy () braces are sometime optional. num and idx are just an optional aliases for closure (function) arguments, when we need just one argument, this is equivalent (it is implicit name of the first closure argument, very convenient):
def numbers = [ 5, 7, 9, 12 ]
numbers.each {println "$it" }
References:
http://groovy.codehaus.org/Closures
http://en.wikipedia.org/wiki/First-class_function
Normally, if you are using a functional programing language such as Groovy, you would want to avoid using each and eachWithIndex since they encourage you to modify state within the closure or do things that have side effects.
If possible, you may want to do your operations using other groovy collection methods such as .collect or .inject or findResult etc.
However, to use these for your problem, i.e print the list elements with their index, you will need to use the withIndex method on the original collection which will transform the collection to a collection of pairs of [element, index]
For example,
println(['a', 'b', 'c'].withIndex())
EachWithIndex can be used as follows:
package json
import groovy.json.*
import com.eviware.soapui.support.XmlHolder
def project = testRunner.testCase.testSuite.project
def testCase = testRunner.testCase;
def strArray = new String[200]
//Response for a step you want the json from
def response = context.expand('${Offers#Response#$[\'Data\']}').toString()
def json = new JsonSlurper().parseText(response)
//Value you want to compare with in your array
def offername = project.getPropertyValue("Offername")
log.info(offername)
Boolean flagpresent = false
Boolean flagnotpresent = false
strArray = json.Name
def id = 0;
//To find the offername in the array of offers displayed
strArray.eachWithIndex
{
name, index ->
if("${name}" != offername)
{
flagnotpresent= false;
}
else
{
id = "${index}";
flagpresent = true;
log.info("${index}.${name}")
log.info(id)
}
}
as a tcl developer starting with groovy, I am a little bit surprised about the list and map support in groovy. Maybe I am missing something here.
I am used to convert between strings, lists and arrays/maps in tcl on the fly. In tcl, something like
"['a':2,'b':4]".each {key, value -> println key + " " + value}
would be possible, where as in groovy, the each command steps through each character of the string.
This would be much of a problem is I could easily use something like the split or tokenize command, but because a serialized list or map isn't just "a:2,b:4", it is a little bit harder to parse.
It seems that griffon developers use a stringToMap library (http://code.google.com/p/stringtomap/) but the example can't cope with the serialized maps either.
So my question is now: what's the best way to parse a map or a list in groovy?
Cheers,
Ralf
PS: it's a groovy question, but I've tagged it with grails, because I need this functionality for grails where I would like to pass maps through the URL
Update: This is still an open question for me... so here are some updates for those who have the same problem:
when you turn a Map into a String, a .toString() will result in something which can't be turned back into a map in all cases, but an .inspect() will give you a String which can be evaluated back to a map!
in Grails, there is a .encodeAsJSON() and JSON.parse(String) - both work great, but I haven't checked out yet what the parser will do with JSON functions (possible security problem)
You might want to try a few of your scenarios using evaluate, it might do what you are looking for.
def stringMap = "['a':2,'b':4]"
def map = evaluate(stringMap)
assert map.a == 2
assert map.b == 4
def stringMapNested = "['foo':'bar', baz:['alpha':'beta']]"
def map2 = evaluate(stringMapNested)
assert map2.foo == "bar"
assert map2.baz.alpha == "beta"
Not exactly native groovy, but useful for serializing to JSON:
import groovy.json.JsonBuilder
import groovy.json.JsonSlurper
def map = ['a':2,'b':4 ]
def s = new JsonBuilder(map).toString()
println s
assert map == new JsonSlurper().parseText(s)
with meta-programming:
import groovy.json.JsonBuilder
import groovy.json.JsonSlurper
Map.metaClass.toJson = { new JsonBuilder(delegate).toString() }
String.metaClass.toMap = { new JsonSlurper().parseText(delegate) }
def map = ['a':2,'b':4 ]
assert map.toJson() == '{"a":2,"b":4}'
assert map.toJson().toMap() == map
unfortunately, it's not possible to override the toString() method...
I think you are looking for a combination of ConfigObject and ConfigSlurper. Something like this would do the trick.
def foo = new ConfigObject()
foo.bar = [ 'a' : 2, 'b' : 4 ]
// we need to serialize it
new File( 'serialized.groovy' ).withWriter{ writer ->
foo.writeTo( writer )
}
def config = new ConfigSlurper().parse(new File('serialized.groovy').toURL())
// highest level structure is a map ["bar":...], that's why we need one loop more
config.each { _,v ->
v.each {key, value -> println key + " " + value}
}
If you don't want to use evaluate(), do instead:
def stringMap = "['a':2,'b':4]"
stringMap = stringMap.replaceAll('\\[|\\]','')
def newMap = [:]
stringMap.tokenize(',').each {
kvTuple = it.tokenize(':')
newMap[kvTuple[0]] = kvTuple[1]
}
println newMap
I hope this help:
foo= "['a':2,'b':4]"
Map mapResult=[:]
mapResult += foo.replaceAll('\\[|\\]', '').split(',').collectEntries { entry ->
def pair = entry.split(':')
[(pair.first().trim()): pair.last().trim()]
}