spark running at the port 249 - apache-spark

sbin$ start-all.sh
I input this command then I got the below message.
I use port 249 not 22.
Port 22 is prohibited.
I'm connecting to the server using putty.
How can I fix this problem?
org.apache.spark.deploy.master.Master running as process 6211. Stop it first.
localhost: ssh: connect to host localhost port 22: Connection refused

I have the same problem and have solved it.
Follow these steps maybe can solve your problem :
vi conf/spark-env.sh
add these export SPARK_SSH_OPTS="-p 249"
rerun start-all.sh

Related

How to connect to ClamAV daemon?

I'm trying to connect to ClamAV daemon clamd on localhost 3310 port via telnet comand in terminal:telnet 127.0.0.1 3310
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
nIDSESSION [pressed ENTER button on keyboard]
nPING [pressed ENTER]
But it gives error:
1: Only nCMDS\n and zCMDS\0 are accepted inside IDSESSION. ERROR
1: Error processing command. ERROR
Connection closed by foreign host.
Entered zPING\0 [pressed ENTER] or nPING\n useless, prompts that error again.
Can you suggest anything?

HAProxy 1.6 configuration Node.js ssh server child process

I am running a Node.js SSH server that spawns a child process to exec code (using require('child_process').spawn) after successful authentication.
The client server connections works fine on port 22 and connection is kept alive successfully through spawned process.
I am trying to setup up now with HAProxy 1.6, to forward port 22 to a non-privileged port on which the SSH server is listening.
However, when the child process is spawned the server either errors Error: write EPIPE or Error: read ECONNRESET.
This suggests to me there is an issue with prematurely closed stream or connection between the client -> HAProxy -> server?
I am looking at websocket configurations and ssh configurations for HAProxy and various keep alive options. However I cannot get the connection to work.
My configuration:
global
daemon
maxconn 10000
log 127.0.0.1 local0
defaults
log global
option tcplog
option logasap
timeout connect 500s
timeout client 5000s
timeout server 2h
timeout server-fin 5000s
timeout client-fin 5000s
timeout tunnel 1h
option tcpka
frontend sshd
bind *:22
default_backend ssh
timeout client 2h
backend ssh
mode tcp
server ssh2server 127.0.0.1:5000 check port 5000
Any pointers or help would be awesome. Thanks in advance.
EDIT
Runing haproxy in debug mode I have
00000000:sshd.accept(0004)=0005 from [my ip]
00000000:ssh.srvcls[0005:0006]
00000000:ssh.clicls[0005:0006]
00000000:ssh.closed[0005:0006].
On the tcplog
Oct 15 15:15:38 localhost haproxy[16036]: 128.277.13.23:51146 [15/Oct/2016:15:15:38.804] sshd ssh/ssh2server 1/0/+0 +0 -- 1/1/1/1/0 0/0

Spark High Availability

I`m using spark 1.2.1 on three nodes that run three workers with slave configuration and run daily jobs by using:
./spark-1.2.1/sbin/start-all.sh
//crontab configuration:
./spark-1.2.1/bin/spark-submit --master spark://11.11.11.11:7077 --driver-class-path home/ubuntu/spark-cassandra-connector-java-assembly-1.2.1-FAT.jar --class "$class" "$jar"
I want to keep spark master and slave workers available at all times, and even if it fail I need it to be restarted like a service (like cassandra does).
Is there any way to do it?
EDIT:
I looked into start-all.sh script and it is only contains the setup for start-master.sh script and start-slaves.sh script.
I tried to create a supervisor configuration file for it and only get the below errors:
11.11.11.11: ssh: connect to host 11.11.11.12 port 22: No route to host
11.11.11.13: org.apache.spark.deploy.worker.Worker running as process 14627. Stop it first.
11.11.11.11: ssh: connect to host 11.11.11.12 port 22: No route to host
11.11.11.12: ssh: connect to host 11.11.11.13 port 22: No route to host
11.11.11.11: org.apache.spark.deploy.worker.Worker running as process 14627. Stop it first.
11.11.11.12: ssh: connect to host 11.11.11.12 port 22: No route to host
11.11.11.13: ssh: connect to host 11.11.11.13 port 22: No route to host
11.11.11.11: org.apache.spark.deploy.worker.Worker running as process 14627. Stop it first.
There are tools like monit and supervisor (or even systemd) that can monitor and restart failed processes.

Unsuccessful connection on ssh to the subordinate nodes of a cluster

Executing start of services on the subordinate nodes of a cluster by means of the following command:
hadoop#one:/export/hadoop-1.0.1/bin$. ./start-all.sh
not the first time I receive result
starting namenode, logging to /export/hadoop-1.0.1/libexec/../logs/hadoop--namenode-one.out
192.168.1.10: starting datanode, logging to /export/hadoop-1.0.1/libexec/../logs/hadoop-hadoop-datanode-myhost2.out
192.168.1.11: ssh: connect to host 192.168.1.11 port 22: Connection timed out
192.168.1.5: starting secondarynamenode, logging to /export/hadoop-1.0.1/libexec/../logs/hadoop-hadoop-secondarynamenode-one.out
starting jobtracker, logging to /export/hadoop-1.0.1/libexec/../logs/hadoop--jobtracker-one.out
192.168.1.10: starting tasktracker, logging to /export/hadoop-1.0.1/libexec/../logs/hadoop-hadoop-myhost2.out
192.168.1.11: ssh: connect to host 192.168.1.11 port 22: Connection timed out
How do I fix the error below:
ssh: connect to host port 22: Connection timed out
hadoop#one: /export/hadoop-1.0.1/bin$ ssh -vvv 192.168.1.10
Sun_SSH_1.5, SSH protocols 1.5/2.0, OpenSSL 0x1000004f
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Rhosts Authentication disabled, originating port will not be trusted.
debug1: ssh_connect: needpriv 0
debug1: Connecting to 192.168.1.10 [192.168.1.10] port 22
debug1: connect to address 192.168.1.10 port 22: Connection timed out
ssh: connect to host 192.168.1.10 port 22: Connection timed out
What to do to correct an error?
Do you have sshd daemon running on your machine? Your OS might come with ssh, but in order to start sshd daemon, you need to install ssh completely. By complete I mean :
ssh : The command we use to connect to remote machines - the client.
sshd : The daemon that runs on the server and allows clients to
connect to this server.
Also, make sure there is no issue with port 22. If you still face some issue try ssh with -v switch to get the complete trace.
ssh -v myhost2
You can go here for a detailed explanation of ssh.

Cassandra nodetool in standalone mode

I've got Cassandra 0.7 running in standalone mode and I'm tryin to run nodetool but I'm getting JMX exceptions. Isn't the JMX configuration required on accessing a remote server? I'm accessing my local machine.
Also why is nodetool looking for 63.251.179.13?
[rav#ubix bin]$ ./nodetool -h 127.0.0.1 flush
Error connection to remote JMX agent!
java.rmi.ConnectException: Connection refused to host: 63.251.179.13; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:128)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown Source)
at javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2343)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:296)
at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:267)
at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:144)
at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:114)
at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:621)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at java.net.Socket.<init>(Socket.java:392)
at java.net.Socket.<init>(Socket.java:206)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 10 more
Thanks,
Try nodetool with -h or --host and -p or --port as per the instructions:
-h,--host <arg> node hostname or ip address
-p,--port <arg> remote jmx agent port number
When Cassandra is offline, check the ports in use to see if another process is using the default port that Cassandra binds to. You can find the default in conf/cassandra-env.sh
Once you know the port, you can see if another process is bound to it with netstat -an
If nothing is running on the port, and you start up cassandra, verify that it is running on the correct port and try to connect again with the -p or --port arguments. More information can be found here: http://wiki.apache.org/cassandra/GettingStarted
Is the machine unix or windows? do you have a bad entry in /etc/hosts indicating that 127.0.0.1 maps to another hostname or IP address, namely 63.251.179.13
I had a similar issue running nodetool on an instance of Cassandra running locally on my machine. When trying to run nodetool -h 127.0.0.1 nodetool was issuing an exception relating to JMX that looked like this (where there was an unknown - to me - IP Address).
Error connecting to remote JMX agent!
java.rmi.ConnectIOException: Exception creating connection to: ; nested exception is:
java.net.SocketException: Host is down
Douglas Muth posted a similar issue here, and from this, I found out that Cassandra seems to be recording the hostname at startup. Unfortunately, by the time I ran nodetool the hostname had become stale (my IP address is allocated dynamically).
My solution then, was to restart cassandra, which updated the IP and rerun nodetool. No more JMX errors, no more strange IP address. This worked a treat for me as I'm running a local instance of Cassandra on localhost and don't mind the restart but it's not a very satisfactory solution.

Resources