I'm trying to start up a docker image that runs cassandra. I need to use thrift to communicate with cassandra, but it looks like that's disabled by default. Checking out the cassandra logs shows:
INFO 21:10:35 Not starting RPC server as requested.
Use JMX (StorageService->startRPCServer()) or nodetool (enablethrift) to start it
My question is: how can I enable thrift when starting this cassandra container?
I've tried to set various environment variables to no avail:
docker run --name cs1 -d -e "start_rpc=true" cassandra
docker run --name cs1 -d -e "CASSANDRA_START_RPC=true" cassandra
docker run --name cs1 -d -e "enablethrift=true" cassandra
The sed workaround (and subsequent custom Dockerfiles that enable only this behavior) is no longer necessary.
Newer official Docker containers support a CASSANDRA_START_RPC environment variable using the -e flag. For example:
docker run --name cassandra1 -d -e CASSANDRA_START_RPC=true -p 9160:9160 -p 9042:9042 -p 7199:7199 -p 7001:7001 -p 7000:7000 cassandra
I've been having the same problem with the Docker Cassandra image. You can use my docker container on Github or on Docker hub instead of the default Cassandra image.
The problem is that the cassandra.yaml file has start_rpc set to false. We need to change that. To do that we can use the following Dockerfile (which is what my image does):
FROM cassandra
RUN sed -i 's/^start_rpc.*$/start_rpc: true/' /etc/cassandra/cassandra.yaml
Don't forget to expose the thrift client API port with the run command to be able to access the container from outside like:
docker run --name cs1 -d .... -p 9160:9160 cassandra
You might also want to expose more ports, like for CQL port 9042, port 7199 for JMX, port 7000 and 7001 for internode communication.
Related
I have the following container:
admin#PC:/$docker ps -a
returns
CONTAINER ID
IMAGE
COMMAND
CREATED
STATUS
PORTS
NAMES
9c0adfffff
hg/sample:1.1
"/usr/sbin/init"
8 days ago
Up 7 days
0.0.0.0:80->80/tcp, :::80->80/tcp
agitated_euclid
This container is a springboot webapp, that maps the application on 80:80. So, the problem is how to access the postgresql that is used by this application inside the same docker container to be accessible from:
the host linux machine containing the docker with this container? and,
any computer with pgadmin interface to connect to this docker postgresql?
Currently I'm using sudo docker exec -it 9c0adfffff bash command to connect to the docker terminal and accessing database using psql, but that doesn't satisfy my current requirement. (like this)
I also tried docker run --name some-postgres -e POSTGRES_PASSWORD=mysecretpassword -d -p 5432:5432 postgres from this answer, but I think this fires up the new container, and also that is not what I need. I need to access database of existing container, whose webapp is running on port 80 currently.
Could you please help me how to install a local standalone pulsar cluster using windows docker.i have followed the below options.but i couldn't able to access the pulsar UI
8080 port is already allocated for some other process.so here i'm using 8081 port.
Option 1:
docker run -it -p 6650:6650 -p 8081:8081 --mount source=pulsardata,target=/pulsar/data --mount source=pulsarconf,target=/pulsar/conf apachepulsar/pulsar:2.5.2 bin/pulsar standalone
Option 2:
docker run -it -p 6650:6650 -p 8081:8081 -v "$PWD/data:/pulsar/data".ToLower() apachepulsar/pulsar:2.5.2 bin/pulsar standalone
Using the above two options, i couldn't able to see the INFO - [main:WebService] - Web Service started at http://127.0.0.1:8081.Also i'm not able to access the following url in the system.
pulsar://localhost:6650
http://localhost:8081
Thanks
The problem is the mapping between the ports. It is clear that you cannot use 8080 on your side, but the port 8080 should be still used within the container, because this port is used by the service. The correct command is:
docker run -it -p 6650:6650 -p 8081:8080 apachepulsar/pulsar:2.5.2 bin/pulsar standalone
It makes sense to try it out without the volumes first and add them later.
I have a feeling the question I'm about to ask is silly but I can't find the solution to my issue and I've been on this problem for a while now.
I am trying to run a docker container for a node application with a command that looks similar to the following:
$ docker run --rm -d -p 3000:3000 <username>/<project>
The above command is working fine. However, when I attempt to map my ports to something different like so:
$ docker run --rm -d -p 3000:8080 <username>/<project>
...The program doesn't work anymore
EDIT: To answer the questions in the comments. I've also tried ports 5000 and 7000 and I'm sure their not in use.
I think you're attempting to change the wrong port in the mapping:
docker run --publish=${HOST_PORT}:${CONTAINER_PORT} <username>/<project>
Maps the host's ${HOST_PORT} to the container's ${CONTAINER_PORT}.
Unless you change the container image's configuration, you're more likely to be choosing a host port.
What happens if you:
docker run --rm -d -p 8080:3000 <username>/<project>
And the try (from the host), e.g. curl localhost:8080?
I'm trying to run a gameserver inside a docker container on my server but I'm having troubles connecting to it.
I created my container and started my gameserver (which is using port 7777) inside it.
I'm running the container with this command:
docker run -p 7777:7777 -v /home/gameserver/:/home -c=1024 -m=1024m -d --name my_gameserver game
I published the ports 7777 with the -p parameter but I can't connect to my gameserver, even though logs show that it is started.
I think I should bind my IP in some way but I have no idea what to do.
What I found so far is that docker inspect my_gameserver | grep IPAddress returns 172.17.0.24.
The problem was coming from the fact that I didn't expose the UDP port.
Correct command was:
docker run -p 7777:7777 -p 7777:7777/udp -v -d --name my_gameserver game
Can’t run Apache Spark on Docker.
When I try to communicate from my driver to spark master I receive next error:
15/04/03 13:08:28 WARN TaskSchedulerImpl: Initial job has not accepted
any resources; check your cluster UI to ensure that workers are
registered and have sufficient resources
This error sounds like the workers have not registered with the master.
This can be checked at the master's spark web stool http://<masterip>:8080
You could also simply use a different docker image, or compare docker images with one that works and see what is different.
I have dockerized a spark master and spark worker.
If you have a Linux machine sitting behind a NAT router, like a home firewall, that allocates addresses in the private 192.168.1.* network to the machines, this script will download a spark 1.3.1 master and a worker to run in separate docker containers with addresses 192.168.1.10 and .11 respectively. You may need to tweak the addresses if 192.168.1.10 and 192.168.1.11 are already used on your LAN.
pipework is a utility for bridging the LAN to the container instead of using the internal docker bridge.
Spark requires all of the machines to be able to communicate with each other. As far as I can tell, spark is not hierarchical, I've seen the workers try to open ports to each other. So in the shell script I expose all the ports, which is OK if the machines are otherwise firewalled, such as behind a home NAT router.
./run-docker-spark
#!/bin/bash
sudo -v
MASTER=$(docker run --name="master" -h master --add-host master:192.168.1.10 --add-host spark1:192.168.1.11 --add-host spark2:192.168.1.12 --add-host spark3:192.168.1.13 --add-host spark4:192.168.1.14 --expose=1-65535 --env SPARK_MASTER_IP=192.168.1.10 -d drpaulbrewer/spark-master:latest)
sudo pipework eth0 $MASTER 192.168.1.10/24#192.168.1.1
SPARK1=$(docker run --name="spark1" -h spark1 --add-host home:192.168.1.8 --add-host master:192.168.1.10 --add-host spark1:192.168.1.11 --add-host spark2:192.168.1.12 --add-host spark3:192.168.1.13 --add-host spark4:192.168.1.14 --expose=1-65535 --env mem=10G --env master=spark://192.168.1.10:7077 -v /data:/data -v /tmp:/tmp -d drpaulbrewer/spark-worker:latest)
sudo pipework eth0 $SPARK1 192.168.1.11/24#192.168.1.1
After running this script I can see the master web report at 192.168.1.10:8080, or go to another machine on my LAN that has a spark distribution, and run ./spark-shell --master spark://192.168.1.10:7077 and it will bring up an interactive scala shell.
Second is more common reason for docker case. You should check, that you
Expose all necessary ports
Set correct spark.broadcast.factory
Handle docker aliases
Without handling all 3 issues spark cluster parts(master, worker, driver) can't communicate. You can read closely on every issue on http://sometechshit.blogspot.ru/2015/04/running-spark-standalone-cluster-in.html or use container ready for spark from https://registry.hub.docker.com/u/epahomov/docker-spark/
If problem in resources, try to allocate less resources(number of executors, memory, cores) with flags from https://spark.apache.org/docs/latest/configuration.html. Check how much resources do you have on spark master UI page, which is http://localhost:8080 by default.
You need to get the master pod ip ... 127.0.0.x
then launch the workers using docker run the following command to launch the worker
docker run -i -t -p 8081:8081 XXImage /bin/bash -c "
cd /opt/spark/bin && ./spark-class org.apache.spark.deploy.worker.Worker spark://172.17.0.x:7077 --port 7000 --webui-port 8081"
the worker should connect to the master spark://172.17.0.x:7077
You can have spark cluster on docker but your may need to have hadoop on your image also but I think with kubernetes you can just have zookeeper Yaml file and don’t need YARN setup.
But ultimately it is best run in minikube with an ingress access point