I am wondering is there is a simple way to tail a file in Groovy? I know how to read a file, but how do I read a file and then wait for more lines to be added, read them, wait, etc...
I have what I am sure is a really stupid solution:
def lNum = 0
def num= 0
def numLines = 0
def myFile = new File("foo.txt")
def origNumLines = myFile.eachLine { num++ }
def precIndex = origNumLines
while (true) {
num = 0
lNum = 0
numLines = myFile.eachLine { num++ }
if (numLines > origNumLines) {
myFile.eachLine({ line ->
if (lNum > precIndex) {
println line
}
lNum++
})
}
precIndex = numLines
Thread.sleep(5000)
}
Note that I am not really interested in invoking the Unix "tail" command. Unless it is the only solution.
I wrote a groovy class which resembles the basic tail functionality:
class TailReader {
boolean stop = false
public void stop () {
stop = true
}
public void tail (File file, Closure c) {
def runnable = {
def reader
try {
reader = file.newReader()
reader.skip(file.length())
def line
while (!stop) {
line = reader.readLine()
if (line) {
c.call(line)
}
else {
Thread.currentThread().sleep(1000)
}
}
}
finally {
reader?.close()
}
} as Runnable
def t = new Thread(runnable)
t.start()
}
}
The tail method taks a file and closure as parameters. It will run in a separate thread and will pass each new line that will be appended to the file to the given closure. The tail method will run until the stop method is called on the TailReader instance. Here's a short example of how to use the TailReader class:
def reader = new TailReader()
reader.tail(new File("/tmp/foo.log")) { println it }
// Do something else, e.g.
// Thread.currentThread().sleep(30 * 1000)
reader.stop()
In reply to Christoph :
For my use case, I've replaced the block
line = reader.readLine()
if (line) {
c.call(line)
}
else {
Thread.currentThread().sleep(1000)
}
with
while ((line = reader.readLine()) != null)
c.call(line)
Thread.currentThread().sleep(1000)`
... != null => to ensure empty lines are outputted as well
while (... => to ensure every line is read as fast as possible
Related
I'm new to groovy and SOAP UI free. I'm using a groovy script to drive my test for SOAP UI.
I want to write a script that reads a file of person IDs, removes the first one, sets a property, writes the file back out without the one I just read.
Here's my first cut at it:
List pids = new ArrayList()
new File("c:/dev/pids.csv").eachLine { line -> pids.add(line) }
String pid = pids.get(0);
testRunner.testCase.setPropertyValue( "personId", pid )
pids.remove(0)
new File("c:/dev/pids.csv").withWriter { out ->
pids.each() { aPid ->
out.writeLine(aPid)
}
}
The output gets displayed on SOAP UI and the file doesn't get touched. I'm lost.
ArrayList pids = null
PrintWriter writer = null
File f = new File("c:/temp/pids.txt")
if (f.length() > 0){
pids = new ArrayList()
f.eachLine { line -> pids.add(line) }
println("Item to be removed: " + pids.get(0))
//testRunner.testCase.setPropertyValue( "personId", pid )
pids.remove(0)
println pids
writer = new PrintWriter(f)
pids.each { id -> writer.println(id) }
writer.close()
}
else{
println "File is empty!"
}
def myFile = new File("newfile.txt")
def newFile = new File("newfile2.txt")
//testRunner.testCase.setPropertyValue( "personId", pid )
PrintWriter printWriter = new PrintWriter(newFile)
myFile.eachLine { currentLine, lineNumber ->
if(lineNumber > 1 )
printWriter.println(currentLine)
}
printWriter.close()
I have a class which is paring csv based file, but I would like to put a parameter for the token symbol.
Please let me know how can I change the function and use the function on program.
class CSVParser{
static def parseCSV(file,closure) {
def lineCount = 0
file.eachLine() { line ->
def field = line.tokenize(';')
lineCount++
closure(lineCount,field)
}
}
}
use(CSVParser.class) {
File file = new File("test.csv")
file.parseCSV { index,field ->
println "row: ${index} | ${field[0]} ${field[1]} ${field[2]}"
}
}
You'll have to add the parameter in between the file and closure parameters.
When you create a category class with static methods, the first parameter is the object the method is being called on so file must be first.
Having a closure as the last parameter allows the syntax where the open brace of the closure follows the function invocation without parentheses.
Here's how it would look:
class CSVParser{
static def parseCSV(file,separator,closure) {
def lineCount = 0
file.eachLine() { line ->
def field = line.tokenize(separator)
lineCount++
closure(lineCount,field)
}
}
}
use(CSVParser) {
File file = new File("test.csv")
file.parseCSV(',') { index,field ->
println "row: ${index} | ${field[0]} ${field[1]} ${field[2]}"
}
}
Just add the separator as the second parameter to the parseCSV method:
class CSVParser{
static def parseCSV(file, sep, closure) {
def lineCount = 0
file.eachLine() { line ->
def field = line.tokenize(sep)
closure(++lineCount, field)
}
}
}
use(CSVParser.class) {
File file = new File("test.csv")
file.parseCSV(";") { index,field ->
println "row: ${index} | ${field[0]} ${field[1]} ${field[2]}"
}
}
I just want to print files which are not located in ss
def folder = "./test-data"
// println "reading files from directory '$folder'"
def basedir = new File(folder)
basedir.traverse {
if (it.isFile()) {
def rec = it
// println it
def ss = rec.toString().substring(12)
if(!allRecords contains(ss)) {
println ss
}
}
Your question isn't exactly clear, but it looks like you're just trying to do
if (!allRecords.contains(ss)) {
println ss
}
in the last part of your code segment.
I wrote logic for testing urls using threads.
This works good for less number of urls and failing with more than 400 urls to check .
class URL extends Thread{
def valid
def url
URL( url ) {
this.url = url
}
void run() {
try {
def connection = url.toURL().openConnection()
connection.setConnectTimeout(10000)
if(connection.responseCode == 200 ){
valid = Boolean.TRUE
}else{
valid = Boolean.FALSE
}
} catch ( Exception e ) {
valid = Boolean.FALSE
}
}
}
def threads = [];
urls.each { ur ->
def reader = new URL(ur)
reader.start()
threads.add(reader);
}
while (threads.size() > 0) {
for(int i =0; i < threads.size();i++) {
def tr = threads.get(i);
if (!tr.isAlive()) {
if(tr.valid == true){
threads.remove(i);
i--;
}else{
threads.remove(i);
i--;
}
}
}
Could any one please tell me how to optimize the logic and where i was going wrong .
thanks in advance.
Have you considered using the java.util.concurrent helpers? It allows multithreaded programming at a higher level of abstraction. There's a simple interface to run parallel tasks in a thread pool, which is easier to manage and tune than just creating n threads for n tasks and hoping for the best.
Your code then ends up looking something like this, where you can tune nThreads until you get the best performance:
import java.util.concurrent.*
def nThreads = 1000
def pool = Executors.newFixedThreadPool(nThreads)
urls.each { url ->
pool.submit(url)
}
def timeout = 60
pool.awaitTermination(timeout, TimeUnit.SECONDS)
Using ataylor's suggestion, and your code, I got to this:
import java.util.concurrent.Executors
import java.util.concurrent.TimeUnit
class MyURL implements Runnable {
def valid
def url
void run() {
try {
url.toURL().openConnection().with {
connectTimeout = 10000
if( responseCode == 200 ) {
valid = true
}
else {
valid = false
}
disconnect()
}
}
catch( e ) {
valid = false
}
}
}
// A list of URLs to check
def urls = [ 'http://www.google.com',
'http://stackoverflow.com/questions/2720325/groovy-thread-for-urls',
'http://www.nonexistanturlfortesting.co.ch/whatever' ]
// How many threads to kick off
def nThreads = 3
def pool = Executors.newFixedThreadPool( nThreads )
// Construct a list of the URL objects we're running, submitted to the pool
def results = urls.inject( [] ) { list, url ->
def u = new MyURL( url:url )
pool.submit u
list << u
}
// Wait for the poolclose when all threads are completed
def timeout = 10
pool.shutdown()
pool.awaitTermination( timeout, TimeUnit.SECONDS )
// Print our results
results.each {
println "$it.url : $it.valid"
}
Which prints out this:
http://www.google.com : true
http://stackoverflow.com/questions/2720325/groovy-thread-for-urls : true
http://www.nonexistanturlfortesting.co.ch/whatever : false
I changed the classname to MyURL rather than URL as you had it, as it will more likely avoid problems when you start using the java.net.URL class
I'm newbie to groovy/grails.
How to implement thread for this code . Had 2500 urls and this was taking hours of time for checking each url.
so i decided to implement multi-thread for this :
Here is my sample code :
def urls = [
"http://www.wordpress.com",
"http://67.192.103.225/QRA.Public/" ,
"http://www.subaru.com",
"http://baldwinfilter.com/products/start.html"
]
def up = urls.collect { ur ->
try {
def url = new URL(ur)
def connection = url.openConnection()
if (connection.responseCode == 200) {
return true
} else {
return false
}
} catch (Exception e) {
return false
}
}
For this code i need to implement multi-threading .
Could any one please suggest me the code.
thanks in advance,
sri.
I would take a look at the Groovy Parallel Systems library. In particular I think that the Parallel collections section would be useful.
Looking at the docs, I believe that collectParallel is a direct drop-in replacement for collect (bearing in mind the obvious caveats about side-effects). The following works fine for me:
def urls = [
"http://www.wordpress.com",
"http://www.subaru.com",
"http://baldwinfilter.com/products/start.html"
]
Parallelizer.doParallel {
def up = urls.collectParallel { ur ->
try {
def url = new URL(ur)
def connection = url.openConnection()
if (connection.responseCode == 200) {
return true
} else {
return false
}
} catch (Exception e) {
return false
}
}
println up
}
See the Groovy docs for an example how to use an ExecutorService to do what you want.
You can use this to check the URL in a separate thread.
class URLReader implements Runnable
{
def valid
def url
URLReader( url ) {
this.url = url
}
void run() {
try {
def connection = url.toURL().openConnection()
valid = ( connection.responseCode == 200 ) as Boolean
} catch ( Exception e ) {
println e.message
valid = Boolean.FALSE
}
}
}
def reader = new URLReader( "http://www.google.com" )
new Thread( reader ).start()
while ( reader.valid == null )
{
Thread.sleep( 500 )
}
println "valid: ${reader.valid}"
Notes: The valid attribute will be either null, Boolean.TRUE or Boolean.FALSE. You'll need to wait for a while to give all the threads a chance to open the connection. Depending on the number of URLs you're checking you will eventually hit a limit of the number of threads / connections you can realistically handle, so should check URLs in batches of the appropriate size.
I think this way is very simple to achieve.
import java.util.concurrent.*
//Thread number
THREADS = 100
pool = Executors.newFixedThreadPool(THREADS)
defer = { c -> pool.submit(c as Callable) }
def urls = [
"http://www.wordpress.com",
"http://www.subaru.com",
]
def getUrl = { url ->
def connection = url.openConnection()
if (connection.responseCode == 200) {
return true
} else {
return false
}
}
def up = urls.collect { ur ->
try {
def url = new URL(ur)
defer{ getUrl(url) }.get()
} catch (Exception e) {
return false
}
}
println up
pool.shutdown()
This is how I implemented:
class ValidateLinks extends Thread{
def valid
def url
ValidateLinks( url ) {
this.url = url
}
void run() {
try {
def connection = url.toURL().openConnection()
connection.setConnectTimeout(5000)
valid = ( connection.responseCode == 200 ) as Boolean
} catch ( Exception e ) {
println url + "-" + e.message
valid = Boolean.FALSE
}
}
}
def threads = [];
urls.each { ur ->
def reader = new ValidateLinks(ur.site_url)
reader.start()
threads.add(reader);
}
while (threads.size() > 0) {
for(int i =0; i < threads.size();i++) {
def tr = threads.get(i);
if (!tr.isAlive()) {
println "URL : " + tr.url + "Valid " + tr.valid
threads.remove(i);
i--;
}
}
}