Scala Process exits but doesn't clean up threads - multithreading

I am trying to run PhantomJS from within a scala application using akka actors:
val process = Process("phantomjs --ignore-ssl-errors=yes " + myrenderscript.js + args ...)
val result = process.run(processLogger, true).exitValue() match {
case ExitCode.SUCCESS => Left(Success)
case ExitCode.TIMEOUT => Right(TimeoutError)
case ExitCode.OPEN_FAILED => Right(NetworkError)
case _ => Right(UnknownError)
}
the myrenderscript.js looks like this:
var version = "1.1";
var TIMEOUT = 30000,
EXIT_SUCCESS = 0,
EXIT_TIMEOUT = 2,
EXIT_OPEN_FAILED = 3;
if (phantom.args.length < 2) {
console.log("Usage: phantomjs render.js parentUrl output [width height]");
phantom.exit(1);
}
var url = phantom.args[0];
var output = phantom.args[1];
var width = parseInt(phantom.args[2] || 1024);
var height = parseInt(phantom.args[3] || 1024);
var clipwidth = parseInt(phantom.args[4] || 1024);
var clipheight = parseInt(phantom.args[5] || 1024);
var zoom = parseFloat(phantom.args[6] || 1.0);
var phantom_version = phantom.version.major + "." + phantom.version.minor + "." + phantom.version.patch;
var userAgentString = "PhantomJS/" + phantom_version + " screenshot-webservice/" + version;
renderUrlToFile(url, output, width, height, clipwidth, clipheight, zoom, userAgentString, function (url, file) {
console.log("Rendered '" + url + "' at size (" + width + "," + height + ") into '" + output + "'");
phantom.exit(EXIT_SUCCESS);
phantom = null;
});
setTimeout(function () {
console.error("Timeout reached (" + TIMEOUT + "ms): " + url);
phantom.exit(EXIT_TIMEOUT);
}, TIMEOUT);
function renderUrlToFile(url, file, width, height, clipwidth, clipheight, zoom, userAgentString, callback) {
console.log("renderUrlToFile start: " + url)
var page = new WebPage();
page.viewportSize = { width: width, height: height };
page.clipRect = { top: 0, left: 0, width: clipwidth, height: clipheight};
page.settings.userAgent = userAgentString;
page.zoomFactor = zoom;
page.open(url, function (status) {
console.log("renderUrlToFile open page: " + url)
if (status !== "success") {
console.log("Unable to render '" + url + "' (" + status + ")");
page.release();
page.close();
page = null;
phantom.exit(EXIT_OPEN_FAILED);
} else {
console.log("renderUrlToFile open page success and pre-render: " + url)
page.render(file);
console.log("renderUrlToFile open page post-render: " + url)
page.release();
page.close();
page = null;
callback(url, file);
}
});
}
prior to creating the process and after it finishes running, about 4 new threads are being created.
Each time the method that creates the process is called, new threads are created and started. After the process is done, the threads go back to a state of monitoring. Eventually my application takes upwards to 500+ threads (I'm capturing a large website and the internal links)
How do I get scala to clean up the threads that are created when running phantomjs?
Edit:
I've changed the scala code to do the following:
val process = Process("phantomjs --ignore-ssl-errors=yes " + myrenderscript.js + args ...).run(processLogger, connectInput)
val result = process.exitValue() match {
case ExitCode.SUCCESS => Left(Success)
case ExitCode.TIMEOUT => Right(TimeoutError)
case ExitCode.OPEN_FAILED => Right(NetworkError)
case _ => Right(UnknownError)
}
process.destroy()
Yet still the threads live on....

I figured out why it is not cleaning up the threads but I don't fully understand it. So if someone posts the true answer on here, I'll vote that answer up.
The problem was I was setting the connectInput value to true. When I set it to false, the threads get destroyed as expected. I'm not sure as to why.
When set to true, a thread dump reveals that one of the threads was blocking the others:
Thread-3#2830 daemon, prio=5, in group 'main', status: 'RUNNING'
blocks Thread-63#4131
blocks Thread-60#4127
blocks Thread-57#4125
blocks Thread-54#4121
blocks Thread-51#4103
blocks Thread-48#4092
blocks Thread-45#4072
blocks Thread-42#4061
blocks Thread-39#4054
blocks Thread-36#4048
blocks Thread-33#4038
blocks Thread-30#4036
blocks Thread-27#4008
blocks Thread-24#3996
blocks Thread-21#3975
blocks Thread-18#3952
blocks Thread-15#3939
blocks Thread-12#3905
blocks Thread-9#3885
blocks Thread-6#3850
at java.io.FileInputStream.readBytes(FileInputStream.java:-1)
at java.io.FileInputStream.read(FileInputStream.java:220)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
at java.io.BufferedInputStream.read(BufferedInputStream.java:317)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at java.io.FilterInputStream.read(FilterInputStream.java:90)
at scala.sys.process.BasicIO$.loop$1(BasicIO.scala:225)
at scala.sys.process.BasicIO$.transferFullyImpl(BasicIO.scala:233)
at scala.sys.process.BasicIO$.transferFully(BasicIO.scala:214)
at scala.sys.process.BasicIO$.connectToIn(BasicIO.scala:183)
at scala.sys.process.BasicIO$$anonfun$input$1.apply(BasicIO.scala:190)
at scala.sys.process.BasicIO$$anonfun$input$1.apply(BasicIO.scala:189)
at scala.sys.process.ProcessBuilderImpl$Simple$$anonfun$2.apply$mcV$sp(ProcessBuilderImpl.scala:72)
at scala.sys.process.ProcessImpl$Spawn$$anon$1.run(ProcessImpl.scala:22)
I initially thought it was the process logger, but that wasn't the case.
Can someone explain this to me?

Related

End all worker threads after the first one ends in Node.js

I am currently trying to find a way to end all worker threads spawned from some master thread after any of them finishes. I've looked at various schemes for communication between worker threads (e.g. cluster.worker.send(), process.send(), etc.), but none of them seem to work.
The code that I have so far is quite simple:
const cluster = require('cluster')
const os = require('os')
if ( cluster.isMaster ) {
let children = []
for ( let i = 0; i < os.cpus().length; i++ ) {
children.push(cluster.fork())
}
process.on('message', (msg) => {
if ( msg.cmd === 'shutdown' ) {
console.log('shutting down')
children.forEach(child => {
child.kill()
})
}
})
} else {
console.log('I am worker #' + cluster.worker.id)
let time = Date.now()
while ( Date.now() < time + cluster.worker.id * 1000 ) {
// do nothing for some time
}
console.log('I am worker #' + cluster.worker.id + ' and I am done')
process.send({cmd: 'shutdown'})
}
The first part creates a child process for each CPU using the cluster module. Next, the master process expects some kind of shutdown message from any of the workers. Each worker runs some process, and, after a certain amount of time, sends a shutdown message to the main thread. At this point, the master thread should trigger all the other threads to shut down, but instead the message is never received. This seems to be the problem but I haven't been able to find a way to get this message sent.
When the code runs, each process sends an 'I am done' message but the threads are not killed.
There are a couple of things you need to change:
Use cluster.on instead of process.on, as process.on is for the child process, not the parent process.
Use child.process.kill() instead of child.kill().
const cluster = require('cluster')
const os = require('os')
if ( cluster.isMaster ) {
let children = []
for ( let i = 0; i < os.cpus().length; i++ ) {
children.push(cluster.fork())
}
cluster.on('message', (worker, msg) => {
if (msg.command = 'shutdown') {
// Kill every cluster worker
children.forEach(child => {
child.process.kill()
})
}
})
} else {
console.log('I am worker #' + cluster.worker.id)
let time = Date.now()
while ( Date.now() < time + cluster.worker.id * 1000 ) {
// do nothing for some time
}
console.log('I am worker #' + cluster.worker.id + ' and I am done')
process.send({command: 'shutdown'})
}

Barcode wasn't printed as many as the Request

I was able to print raw ZPL commands from PHP directly to the printer, except that I can't print more than 1 label at once after windows update to windows-10 on the TLP 2844-Z printer and my first time when installing WebClientPrint Processor (WCPP) in windows-10. When I was trying to emulate ZPL printer in the ZPL Printer app it also happened. The only exception was when I try this on the mac Safari browser, it's doing fine.
Working request script (still working in Safari, and previously in all other browser):
for(var i=0; i<rows.length; i++){
javascript:jsWebClientPrint.print('useDefaultPrinter=' + $('#useDefaultPrinter').attr('checked') + '&printerName=' + $('#installedPrinterName').val() + '&param=' + rows[i].value);
}
What's preventing me was the permission asking:
on Chrome weren't generated as many time as the request were (which aren't the problem on Safari).
Example when request were 2:
it only ask for permission once, resulting (only 1 label printed):
when it supposed to be (2 labels printed):
I was able to reproduce the above by using the following script:
for (var i = 0; i < rows.length; i++) {
var url = ('useDefaultPrinter=' + $('#useDefaultPrinter').attr('checked') + '&printerName=' + $('#installedPrinterName').val() + '&param=' + rows[i].value);
window.open('webclientprint:' + domain + url);
}
Which aren't ideal since many tabs would be generated only to print, where previously you don't need any new tab to do the same.
Any idea how to solve this? So that it would print as many as the request ask?
What I did to solve this was to make each request on a separate tab and closed the tab once it's executed. To make it simple I make it into a separate function.
Request script changed into:
for (var i = 0; i < rows.length; i++) {
if (i > 0)
delayPrint(rows[i], i); // separate function
else
javascript: jsWebClientPrint.print('useDefaultPrinter=' + $('#useDefaultPrinter').attr('checked') + '&printerName=' + $('#installedPrinterName').val() + '&param=' + rows[i].value);
}
separate function used for delaying request and to make each request on a separate tab and closed the tab once it's executed:
function delayPrint(data, interval) {
setTimeout(function() {
var wnd = window.open('webclientprint:' + domain + ('useDefaultPrinter=' + $('#useDefaultPrinter').attr('checked') + '&printerName=' + $('#installedPrinterName').val() + '&param=' + rows[i].value));
setTimeout(function() {
wnd.close(); // close once it's done
}, 1000);
}, interval * 3000);
}

Ping error return always equals 0

Having trouble getting this code to work. I can run the script (with changes) in vbscript and it runs fine. But I cant get this version to return a ping error return value other than 0. Any help is appreciated.
This is scanning a list of remote machines with checkboxes and returning the checked values. I want to run the ping to verify the remote machine are there before continuing. But with a error return of 0 for all queries its useless.
function statuschk3(){
var checkedValue = null;
var inputElements = document.getElementsByName("comp");
for(var i=0; inputElements[i]; ++i){
if(inputElements[i].checked)
{checkedValue = inputElements[i].value;
var WshShell = new ActiveXObject("WScript.Shell");
var status = WshShell.run ("ping -n 1 -a" + checkedValue + ",0 ,true");
if(status == 0)
{ var fso = new ActiveXObject("Scripting.FileSystemObject");
var s = fso.OpenTextFile("C:/script/testfile1.txt", 8, true);
s.WriteLine(checkedValue + " is turned off or off the domain");
s.Close();
}
else`
Here's a function I use to test connectivity. I use a vbscript version of this, but I rewrote it in javascript.
function Reachable(strComputer)
{
var wmiQuery = "Select * From Win32_PingStatus Where Address = '" + strComputer + "'";
var objWMIService = GetObject("winmgmts://./root/cimv2");
var colItems = objWMIService.ExecQuery(wmiQuery);
var enumItems = new Enumerator(colItems)
for (; !enumItems.atEnd(); enumItems.moveNext())
{
var objStatus = enumItems.item();
if ((objStatus.StatusCode == null) || (objStatus.Statuscode != 0))
{
return false //if computer is unreachable, return false
}
else
{
return true //'if computer is reachable, return true
}
}
}
Usage:
vbscript:
If Reachable(strComputer) Then MsgBox "Online" Else MsgBox "Offline"
javascript:
if (Reachable(strComputer)) { alert("Online") } else { alert("Offline") }
edit:
If you'd like to adjust the timeout of this, you can add the following to this line:
var wmiQuery = "Select * From Win32_PingStatus Where Address = '" + strComputer + "' and Timeout=500";
Where 500 is 500 milliseconds.
Here's more on the Win32_PingStatus class, where it says the default Timeout is 1000 milliseconds.
another edit to address your original question:
It looks like you have some syntax issues with your original code:
var status = WshShell.run ("ping -n 1 -a" + checkedValue + ",0 ,true");
needs to be
var status = WshShell.run ("ping -n 1 -a " + checkedValue, 0,true);
Notice the location of the space after a and the quotation marks after checkedValue
Also, the logic is backwards. if(status==0) then the device is ONLINE.
For what it's worth, if langstrom's WMI solution works, I think I prefer it over my own. My purpose in answering is to help you figure out what went wrong with your script.
Firstly, ping.exe can receive error responses from the gateway saying "no route to host" or similar. But having received responses (erroneous or not), the exit status is 0. It's better to check the output of the command for valid responses to determine success / fail.
But the more immediate problem is that shellObj.run() is forking the ping process asynchronously. It's non-blocking.
Using shellObj.exec() solves both problems. Here's an example using console JScript:
// returns 0 for success, non-zero for fail
function ping(host) {
var osh = WSH.CreateObject('Wscript.Shell'),
proc = osh.exec('ping -n 1 ' + host);
while (!proc.Status) WSH.Sleep(25);
return !/\btime\b/i.test(proc.StdOut.ReadAll());
}
Unfortunately, since browser / HTA JScript doesn't understand WSH or Sleep, you have to get creative with setInterval to block until the ping executable is finished. (This is why langstrom's WMI solution would be preferable to spawning an executable. Trying to hack a non-blocking command into something that blocks is cumbersome and awkward with HTA JScript.)
// alerts true on success, false on fail
function ping(host) {
var osh = new ActiveXObject('Wscript.Shell'),
proc = osh.exec('ping -n 1 ' + host),
waitForExit = setInterval(function() {
if (proc.Status) {
clearInterval(waitForExit);
alert(/\btime\b/i.test(proc.StdOut.ReadAll()));
}
}, 25);
}
ping('localhost');

How does a NodeJS app with multiple timers scale?

My app will require up to 1000 timers at any given moment. Do timers consume a lot of resources? Is it an accepted practice to deploy multiple timers? Or should I avoid it?
By #jfriend00's suggestion, i made a sample check below, this might be not accurate (cuz of dom manipulations), but hope it gives you concept
// let's use 2 measurings, window.performance.now and console.time
// start console.time, usually it gives same result as window.perf.now
// but window.perf.now uses 10 decimal points
console.time('timer_perf_test');
var now1 = window.performance.now();
// our count, this test really lags when 100,000
// CHANGE THIS
var count = 10000;
// some dom elements for text
var elem = document.querySelector('img');
var counter = document.querySelector('#counter');
var perf = document.querySelector('#perf');
// how smooth our image gonna rotate?
var rotate = function(degree){
elem.style.transform = 'rotate(' + degree +'deg)';
counter.textContent = 'timers executed: ' + degree;
}
// create bunch of timers with different timeout
var timer = function(added_time){
setTimeout(function(){
rotate(added_time);
}, 1000 + (added_time * 10));
}
// test begins
for (var i = 0; i < count; i++) {
timer(i)
}
// check results
var now2 = window.performance.now();
perf.textContent = now2 - now1 + ' MS required to create ' + count + ' timers';
console.timeEnd('timer_perf_test');
<img src="https://km.support.apple.com/library/APPLE/APPLECARE_ALLGEOS/HT4211/HT4211-ios7-activity_icon-001-en.png" width="100" height="100">
<p id="counter"></p>
<p id="perf"></p>

NODEJS [electron io.js] send input to spawned process child.stdin.write();

I have a set of scripts in PowerShell. I built a GUI with a .HTA, but I am migrating over to a NODE.js like approach.
I have a working script, but I had to make a modification to work around the lost functionality to get input from the command line. I used to have the CMD prompt window appear where I would see all output and, if there was a prompt (Read-Host), the user could interact with it. To work around that, I am now using [Microsoft.VisualBasic.Interaction]::InputBox() to get input from the user when needed.
I have had issues trying to use child.stdin.write(); because my PowerShell process just hangs. It will not respond to my input.
I have tried every answer I have found on the net, with no success. The script that is called in production is a long running process, depending on the task selected it will take from 20 min to 3 hours.
So, the issue is, I cannot get the child.spawned process to send the input, or maybe PowerShell does not accept input from stdin.
Thanks for your help in advance.
CODE:
HTML:
<button type="button" id="btn_ExecuteReport">Run Report</button>
<button type="button" id="btn_StopReport" >Stop Report</button>
<button type="button" id="btn_SendInput" >Send this...</button>
<h2>Results:</h2>
<p id="result_id">...</p>
<br/>
JS (using Electron v 0.31.0, io.js v 3.1.0, jQuery v 1.7.2):
$(document).ready(function () {
$("#btn_ExecuteReport").click(function (event) {
event.stopPropagation();
global.$ = $;
var remote = require('remote');
// to get some paths depending on OS
var app = remote.require('app');
//clipboard
var clipboard = require('clipboard');
var shell = require('shell');
// for choosing a folder
var dialog = remote.require('dialog');
var spawn = require('child_process').spawn;
sCmd = "powershell.exe ";
sCmdArg = '&' + "'" + __dirname + "/app.ps1' " ;
// omit parameters for the purpose of this example
// + "-ODir '" + sOutputDirectory
// + "' -WDir '" + sWorkingDirectory + "'"
// + " -Report " + sReport
// + " -pSendEmail " + sSendEmail
// + " -pSendEmailHTMLBody " + sSendEmailHTMLBody
// + " -pCreateReport " + sCreateReport
// + " -pCheckReport " + sCheckReport
// + " -pEmailAction " + sEmailAction
// ;
$("#result_id").html("<p><font color='magenta'>Command: </font>"
+ sCmdConcat + "</p>");
// try again with spawn and event handlers
var psSpawn = spawn(sCmd, [sCmdArg]);
// add a 'data' event listener for the spawn instance
psSpawn.stdout.on('data', function(data) {
console.log('stdout: ' + data);
});
// add an 'end' event listener to close the writeable stream
psSpawn.stdout.on('end', function(data) {
console.log('Done with psSpawn...' );
});
psSpawn.stderr.on('data', function (data) {
console.log('stderr: ' + data);
});
// when the spawn child process exits, check if there were any errors and close the writeable stream
psSpawn.on('exit', function(code) {
if (code != 0) {
console.log('[EXIT] Failed: ' + code);
}
//Collect result from PowerShell (via clipboard)
sResultData = clipboard.readText("Text");
});
psSpawn.on('close', function(code) {
if (code != 0) {
console.log('[ISSUE] code: ' + code);
}
console.log('[CLOSE]');
});
//sending the input with button #btn_sendInput
$("#btn_SendInput").click(function (event) {
psSpawn.stdin.setEncoding('utf8');
psSpawn.stdin.write('ok');
psSpawn.stdin.write('\n');
psSpawn.stdin.write(os.EOL);
//uncomment following line to end stdin on first button press
//psSpawn.stdin.end();
});
// killing process
$("#btn_StopReport").click(function (event) {
event.stopPropagation();
psSpawn.kill('SIGTERM');
console.log('killed ' + psSpawn.pid);
});
});
});
PowerShell:
#work around to lack of Read-Host functionality
Write-Host "waiting"
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.VisualBasic') | Out-Null
$computer = [Microsoft.VisualBasic.Interaction]::InputBox("Enter a computer name", "Computer", "$env:computername")
#
Write-Host "[Done] waiting"
#workaround END works
Write-Host "Computer: [$computer]"
# START omit ----------------
#if we remove the following lines, all works but we lose the functionality to ask for input from the command line
Write-Host "Propmt for OK"
$ok = Read-Host
#
Write-Host "Prompt. Value of 'ok': [$ok]"
# END omit ----------------
#Copy new message to clipboard
"we want to see this maessage on the browser. $computer" | clip
#Exit script returning the appropriate value
[Environment]::Exit(0)
exit
My console output is:
stdout: waiting
stdout:
stdout: [Done] waiting
Computer: [COMPUTER_NAME]
Propmt for OK
The script hands no matter how many times I press Send this.... The console output is (after pressing the #buton_StopReport):
killed 2548
Done with psSpawn...
[EXIT] Failed: null
[ISSUE] code: null
[CLOSE]

Resources