MPJ Express -- runtime.MPJRuntimeException: Cannot connect to the daemon at machine <x> and port <10000> - mpj-express

i am facing problem while configuring MPJ Express in cluster mode.
i am following the guide given at
http://mpj-express.org/docs/guides/linuxguide.pdf
the environment variables are successfully Initialized as
1)
Set MPJ_HOME and PATH variables
a. export MPJ_HOME=/path/to/mpj/
b. export PATH=$PATH:$MPJ_HOME/bin
2)
the machines are also added successfully with a machines file as
mpjboot machines
it shows message
starting mpjd...
3)
Next step is to test the HelloWolrd program by
Compile:
javac -cp .:$MPJ_HOME/lib/mpj.jar HelloWorld.java
10. Execute: mpjrun.sh -np 2 –dev niodev HelloWorld
when i do that i get this error
runtime.MPJRuntimeException: Cannot connect to the daemon at machine and port <10000>
The Platform i am using is
Sun Ultra 25 workstations with Solaris 10 OS
i have made a Fast Ethernet cluster of 2 machines
ssh is enabled with root user on each machine
network is established well
any help or solution is appreciated..
thank you

Examine the cluster nodes environment variables.
Try adding vars to .bashrc in each cluster node (ssh to it from main node):
echo 'export MPJ_HOME=/home/<user>/path/to/mpj' >> ~/.bashrc
echo 'export PATH=$PATH:$MPJ_HOME/bin' >> ~/.bashrc
Alternately, turn on logging /conf/wrapper.conf, run and report your findings.

This is the first time i give an answer in stackOverflow.
I set up MPJExpress-v0_42 in my centos6.3 with jdk1.6.0_32, and finally everything seems OK.
Question 1: can not connect to daemon at machine and port
This may happens that the daemon is not running. You can give a try as follows:
you can use sudo netstat -anp |grep port to check whether the port is listening and also get the processId.
In my machine, after I started the daemon using mpjdaemon -boot localhost, I used the sudo netstat -apn |grep 4000 command to check the port and got the followings.
tcp 0 0 :::40000 :::* LISTEN 8766/java
tcp 0 0 :::40001 :::* LISTEN 8766/java
the port is configured in $MPJ_HOME/conf/wrap.conf and my configuration is as follows:
#port number for the daemon.
wrapper.app.parameter.2=40001
#Socket Server Port Number.
wrapper.app.parameter.3=40000
I also use the jps -m command to get the java daemon process, the result is as follows:
8766 MPJDaemon 40001
30850 Jps -m
Here 8766 is the processId as netstat shows, and MPJDaemon is the daemon process, 40001 is the listening port.
In your question, you cannot connect the daemon process. Do the followings:
you should check the port. If you cannot find the port using netstat, it generally means the MPJDaemon is not running.
To be sure, you can also using jps to check the process. Maybe process is running but the port is another one
Do not use the Cluster commands like mpjboot machines or mpjrun.sh -np 2 -dev niodev HelloWorld. You can use mpjdaemon -boot localhost to test current machine, if it is configured OK and you can set the machines file with one item of localhost and run Cluster commands.
Other questions that I met.
Question 2: compile failed using ant
At first I use MPJ-v0_44.zip and jdk1.6.0, but it cannot compile and give an error that ProcessBuilder has no method of "interNIO"(something like that). I analyse the source code and guess may be it is because JDK version is a litter lower. Given the fact that upgrading JDK is a litter complicated, I find another MPJ version of mpj-v0_42.zip and it compiled OK.
Question 3: mpjdaemon -boot localhost with no error message, but MPJDeamon is not running.
After I configure the environment variables like MPJ_HOME from .bash_profile to .bashrc, the question is resolved. I don't know why.
Question 4: command jps -m says "MPJDeamon is running" , but mpjdaemon -status localhost says "mpjdeamon is not running".
I use command (ssh localhost nohup 'jps -m') and it says jps command not found, but i use jps the command works fine. I guess maybe it is the same question as above. The jps's path is not configured in path environment variable of .bashrc. So after adding one line of PATH=/jpspath:$PATH in .bashrc, everything works fine.

Related

Open a port on mac for locally running spark

I am running a stand-alone Spark 3.2.1 locally, on my mac, installed via brew. This is for low-cost (free) unit testing purposes. I am starting this instance via pyspark command from terminal and able to access the instance web ui.
I am also trying to run spark-submit locally (from the same mac) to run a pyspark script on the pyspark instance described above. When specifying the --master :7077 I am getting the "connection refused" error. It does not look like the port 7077 is open on my mac.
How do I open the port 7077 on my mac such that I can access it from my mac via spark-submit, but other machines on the same network cannot?
Could someone share clear steps with explanations?
Much appreciated :)
Michael
Check your spark master process is running.
It must be like following output.
jps
$PID Master
$PID Worker
If spark process is not running,
run script $SPARK_HOME/sbin/start-master.sh in your shell first.
also $SPARK_HOME/sbin/start-worker.sh.
and then check if process listen on 7077 port with following command.
sudo lsof -nP -i:7077 | grep LISTEN

Two docker instances/daemons running on linux machine

So I have been experiencing an issue where there are two Docker instances (daemons) that are running at the same time on my Ubuntu machine.
The issue is the following:
I have been using docker for some time without problems. I have tons of images and volumes there. Now one day after restart when I try to start my project using docker-compose up I get error that port is in use:
Error starting userland proxy: listen tcp 0.0.0.0:8011: bind: address already in use.
Now the thing is there is no project apart from mine that is using this port. I checked docker ps and there are no containers up. Not even portainer that I use to manage images and containers. It seems that there is another docker-daemon running on my machine or other version of docker. It might be the case that I botched installation at the beginning and now it came back to haunt me.
What I tried:
Uninstalled snap version of docker.
Restarted using sudo systemctl restart docker
Reinstalled docker completelly- it worked for a while but I lost all containers and images and again after a while it started showing me different docker with different images and no volumes while at the same time the ports for my apps where blocked because the docker I was using previously still was up.
Is there a way to list running docker-daemons/engines/instances and choose which one to use in the system?
Are you sure that you have two dockerds? It is very possible, but very uncommon.
With an lsof -n -P|grep TCP|grep 8011 or so, you can list what is listening already on 8011. This "userland proxy" is a docker thingy: if you publish 8011 of a container, docker starts a process listening on 8011 and forwarding the connections to the container. The likely cause is that you have already running something on 8011. It does not matter, which dockerd (or another, non-docker process) is listening already on 8011.
There is no specific command to list the dockerd(s) of your system. You can list the running dockerds, for example, by a ps uxa|grep dockerd command. After that, an lsof -n -p <pid> -P will show, where are they listening.
The most likely cause of your problem is that another container is already using your 8011.
It looks like both docker daemons where listening at /var/run/docker.sock and the requests were randomly distributed causing the inconsistency.

Kong: Running Mashape Kong fails on Mac OS X

Today I installed Kong (API Management Layer) on my Mac OS X (Yosemite 10.10.5). I used the .pkg file which is available here. I followed the installation instructions and everything was successful.
I also installed Cassandra using the information given on the Cassandra home page.
But when I start Kong, using the command:
$ kong start
It throws the following error:
dnsmasq: failed to create listening socket for fe80::3e15:XXXX:XXXX%en0: Can't assign requested address
Looks like you have already something running on the kong port 8000,8001 or 9042. Try to shut down anything that might be using those ports and retry?
you can configure Kong to run on any custom ports for (proxy and admin) by changing the values in the /etc/kong/kong.yml file. see full config reference here
The problem is that dnsmasq on Kong is listening by default on port 8053 (https://github.com/Mashape/kong/blob/master/kong.yml#L29), which seems to be already taken on your system.
Make sure that there is nothing else running on that port.
You can kill the process that is running on the dnsmasq port and restart the kong.
Kill the process:
sudo kill `sudo lsof -t -i:8053`
Start Kong:
kong start
/usr/local/bin/kong start
/usr/local/bin/kong stop
/usr/local/bin/kong reload
/usr/local/bin/kong status

Remote debug with gdbServer

1) I start GDB on remote machine by gdb myAppName
2) Then i give run -p portNumber
3) Then i run java application which connect with it on above mentioned port.
Now when i am trying to remote debug via gdbserver then the scenario is
1) I run gdb server on remote machine
gdbserver host:port path to application
Now when i try to connect my java client application to the gdbserver port .It will not be able to connect.
So i am searching of alternative of run -p command of gdb in gdb server
What i was trying was remote debugging of C code via eclipse CDT
Any help would be highly appreciated.
gdbserver can be started attached to a pid on some systems according to the docs using the following syntax which is different than the normal invocation
gdbserver hostname : port --attach pid
or gdb can be started with a program that is subsequently started by the remote machine
gdbserver hostname : port program
and then you can attach to it from another remote gdb using
(gdb) target remote hostname: port
Eclipse would probably do the same thing so long at the remote gdbserver is invoked as above
and then start the remote program using a different port
(gdb) run -p someotherport
This 2nd invocation, from your comments above, I assume you have created some flag to your program that allows it to bind to someotherport

Jmeter-server hanging on linux system

When I try and run jmeter-server from a server connected via putty, it hangs with:
bash-3.2$ ./jmeter-server -t filename.jmx
Using local port: 4000
Created remote object: UnicastServerRef [liveRef: [endpoint:[172.31.120.200:4000] (local),objID:[1a15acdf:12f949f58d8:-7fff, -8305458059461378367]]]
What am I doing wrong? It just hangs. Previously it was erroring with:
bash-3.2$ ./jmeter-server
Using local port: 4000
Created remote object: UnicastServerRef [liveRef: [endpoint:[172.31.120.200:4000](local),objID:[1fa5aea9:12f94a578a6:-7fff, -8358825329658872549]]]
Problem creating registry: java.rmi.server.ExportException: Port already in use: 1099; nested exception is:
java.net.BindException: Address already in use
Continuing...
Server failed to start: java.rmi.RemoteException: Cannot start. See server log file.
What gives?
Port 1099 is default port of jmeter server, this means that you have already lauched jmeter server. I think you should first kill it and then try to start:
ps ax | grep jmeter
kill -9 PID (from previous command)
Before starting remote testing I think you should read documentation, seems you are doing it wrong: http://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf
Looks you need change your jmeter SERVER_PORT to a not "1099" number from the error information.
Guess I probably were in your situation.
It helps to me to use java8 and the next properties when starting jmeter-server
/opt/jmeter/bin/jmeter-server \
-Dserver.rmi.localport=50000 \
-Dserver_port=1099 \
-Jserver.rmi.ssl.disable=true
Also check logs in jmeter-server.log file

Resources