Docker service exits on deploy-over-SSH from Jenkins but manually succeeds - cron

I have a strange issue. I have configured a SSH_USER on Jenkins and trying to deploy a simple docker-service with "deploy-over-SSH".
Every time I deploy it Exits as below, and logs just says "Terminated"
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
bea48e1ee755 localhost/my-image:latest /bin/sh -c npm ru... 13 seconds ago Exited (143) 13 seconds ago 0.0.0.0:6007->3000/tcp my-cont
$ docker logs my-cont
Terminated
But if I try running manually on the same server with same SSH_USER, I am able to run docker container successfully.
docker run -d -it -p 6007:3000 --name my-cont my-image
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
nce48e1ee721 localhost/my-image:latest /bin/sh -c npm ru... 21 minutes ago 21 minutes ago 0.0.0.0:6007->3000/tcp my-cont
The script I am running over-SSH is very simple from Jenkins, I am passing the Port from Jenkins,
$ bash ./run.sh $Port
docker run -d -it -p $1:3000 --name my-cont my-image
Not sure , what is causing the issue.

As I mentioned in the comment, upgrading the version 'podman version 2.0.5' worked for me.
The error or docker logs never gave me any hint, clue on version.
But that was the solution. Thanks for your comments.

Related

How to stop a running Docker container from within the same terminal?

I've built a Dockerfile which ends with compiling my (Golang) code and then running the compiled binary:
# Compile my Go code
RUN go build -o bin *.go
# Run the binary
CMD /root/src/whisky/bin
After building the image (docker build -t whisky .) I run the image (docker run --name whisky whisky_image) and the program starts to give output in the terminal.
When I run the program from my laptop I can always stop it using CTRL+C, but now it's running inside the container and hitting CTRL+C doesn't do anything. The container simply keeps giving output and nothing stops.
I can still stop the container from another terminal docker stop whisky, but I guess there should be a nicer way.
Does anybody know the recommended way to stop the command in the container or stop the whole container from the same terminal?
[EDIT]
From the answers given I learned I can run the container in detached mode using -d and then tail the container logs using docker logs container_id --follow. Although that works I prefer to run the container directly (in non-detached mode) because then I can't make the mistake of running it in the background and forgetting about it.
Does anybody know how I can run the container in a way that I can stop it from the same terminal by hitting CTRL+C?
you can use '-d' --> in detached mode
[EDIT]
To add. After that you can tail to logs using
docker logs container_id --follow
Allocate a pseudo-tty (-t) and then you can control output by hitting CTRL+C:
$ docker run --name whisky -t whisky_image
^C
But container will be still running, so add also stdin (-i):
$ docker run --name whisky -ti whisky_image
^C
But container will be still there (see docker ps -a - in exited status), so you may add cleanup flag --rm:
$ docker run --name whisky --rm -ti whisky_image
^C
See Docker run doc for more details.
you need to run your container using init
docker run --init --name whisky whisky_image
then kill it using Ctrl+C
the initwill start your program as PID 1
As mentionned ealier, there is 2 possibilities, the first, detaching the process (-d option) or use a TTY with Interactive mode (-ti option).
The detached version will allow you to work and manage the instance, but won't let you down it by pressing CTRL+C
The TTY version with an Interactive mode (-i) will.
So, docker run -it should do the trick
Docker Run reference and Docker Container process behaviors can also help
Try this:
# Compile my Go code
RUN go build -o bin *.go
# Run the binary
CMD ["/root/src/whisky/bin"]
This is the exec form. in this form your executable is being called directly.
What you used is the shell form which starts your process with /bin/sh -c. This form does not automatically pass signals to the child processes.
Dockerfile
FROM golang
WORKDIR /go/src/app
ADD . .
# Compile my Go code
RUN go build -o bin *.go
# Run the binary
CMD /go/src/app/bin
Go program
package main
import (
"fmt"
"time"
)
func main() {
for {
fmt.Println("\nWHISKY TIME: ", time.Now().String())
time.Sleep(time.Second)
}
}
Building the Container
╭─exadra37#exadra37-Vostro-470 ~/Developer/DevNull/stackoverflow
╰─➤ sudo docker build -t whisky .
Sending build context to Docker daemon 3.072kB
Step 1/5 : FROM golang
---> c4d6fdf2260a
Step 2/5 : WORKDIR /go/src/app
---> Using cache
---> ab35c58a3608
Step 3/5 : ADD . .
---> 320a51d3e579
Step 4/5 : RUN go build -o bin *.go
---> Running in 5db9f4a9cbe1
Removing intermediate container 5db9f4a9cbe1
---> e16b272606da
Step 5/5 : CMD /go/src/app/bin
---> Running in f66039aeb53a
Removing intermediate container f66039aeb53a
---> d453f9e10c49
Successfully built d453f9e10c49
Successfully tagged whisky:latest
Running the Container
╭─exadra37#exadra37-Vostro-470 ~/Developer/DevNull/stackoverflow
╰─➤ sudo docker run -it --name whisky whisky 137 ↵
WHISKY TIME: 2019-11-25 16:19:26.397705029 +0000 UTC m=+0.000041700
WHISKY TIME: 2019-11-25 16:19:27.399189969 +0000 UTC m=+1.001526690
WHISKY TIME: 2019-11-25 16:19:28.399392173 +0000 UTC m=+2.001728894
WHISKY TIME: 2019-11-25 16:19:29.399566682 +0000 UTC m=+3.001903403
^C%
As you can see I was able o stop it with CTRL + C, and I just needed to use the flags -it.
The flags used:
╭─exadra37#exadra37-Vostro-470 ~/Developer/DevNull/stackoverflow
╰─➤ docker run --help | grep "\-t, \-\-tty" - 2 ↵
-t, --tty Allocate a pseudo-TTY
╭─exadra37#exadra37-Vostro-470 ~/Developer/DevNull/stackoverflow
╰─➤ docker run --help | grep "\-i, \-\-interactive" -
-i, --interactive Keep STDIN open even if not attached
Restarting the container
╭─exadra37#exadra37-Vostro-470 ~/Developer/DevNull/stackoverflow
╰─➤ sudo docker start -i whisky
WHISKY TIME: 2019-11-25 16:21:03.159420042 +0000 UTC m=+0.000047654
WHISKY TIME: 2019-11-25 16:21:04.161264182 +0000 UTC m=+1.001891819
WHISKY TIME: 2019-11-25 16:21:05.161470301 +0000 UTC m=+2.002097939
^C%
When restarting the container we just need to use the flag -i, and you are still able to stop it with CTRL + C.
The flags used:
╭─exadra37#exadra37-Vostro-470 ~/Developer/DevNull/stackoverflow
╰─➤ docker start --help | grep "\-i, \-\-interactive" -
-i, --interactive Attach container's STDIN
To stop the whole container please use below commands.
$ docker stop <container>
The docker stop command stops running Docker containers.
It sends the SIGTERM signal to the running process inside the docker container and requests it to terminate.

Dockerized nginx shuts down after a few seconds

I'm working with Ubuntu 18 and I´m trying to run a dockerized nginx with a shared file between the host machine and the container: /home/ric/wrkspc/djangodocker/djangodocker/nginx.conf
I do so by running the following command, after which I'm prompted with container's ID:
$ sudo docker container run -v /home/ric/wrkspc/djangodocker/djangodocker/nginx.conf:/etc/nginx/nginx.conf:ro -d nginx
facc4d32db31de85d6f360e581bc7d36f257ff66953814e985ce6bdf708c3ad0
Now, if I try to list all the running containers, the nginx one doesn't appear listed:
(env) ric#x:~/wrkspc/djangodocker/djangodocker$ sudo docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
36090ff0759c mysql "docker-entrypoint.s…" 3 days ago Up 3 days 0.0.0.0:3306->3306/tcp, 33060/tcp boring_panini
Sometimes, if I run the docker ls command fast enough, I can see the nginx container listed for just a few seconds and then it disappears.
Why is the nginx container not being listed?
I think container immediately exits after started.
can you troubleshoot by looking into docker logs using the command
docker logs containerID
Also, you can try running the container interactively to identify the error without using -d option

Container error when creating channel during running fabcar (fabric-sample)

Getting up to speed with Hyperledger and trying to run through the Hyperledger-Fabric tutorial fabcar, but hitting an error every time that I try to create the channel in the startFabric.sh script.
Here's the error:
Error response from daemon: Container 3640f4fca98aef120a2069292a3fc613954a0fbe7c625a31c2843ec643462 is not running
Ran all the pre-requisites and the commands listed, cloned the latest fabric-samples, updated node, tried longer start times. But still have this error. If anyone knows where I am going wrong would really appreciate some help to resolve. Thanks in advance.
Perhaps worth mentioning that I am running on Windows 7 and using Docker Toolbox.
startFabric.sh output is shown below.
$ ./startFabric.sh node
# don't rewrite paths for Windows Git Bash users
export MSYS_NO_PATHCONV=1
docker-compose -f docker-compose.yml down
Stopping ca.example.com ... done
Stopping couchdb ... done
Removing peer0.org1.example.com ... done
Removing orderer.example.com ... done
Removing ca.example.com ... done
Removing couchdb ... done
Removing network net_basic
docker-compose -f docker-compose.yml up -d ca.example.com orderer.example.com peer0.org1.example.com couchdb
Creating network "net_basic" with the default driver
Creating ca.example.com ... done
Creating couchdb ... done
Creating orderer.example.com ... done
Creating peer0.org1.example.com ... done
# wait for Hyperledger Fabric to start
# incase of errors when running later commands, issue export FABRIC_START_TIMEOUT=<larger number>
export FABRIC_START_TIMEOUT=10
#echo ${FABRIC_START_TIMEOUT}
sleep ${FABRIC_START_TIMEOUT}
# Create the channel
docker exec -e "CORE_PEER_LOCALMSPID=Org1MSP" -e "CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/msp/users/Admin#org1.example.com/msp" peer0.org1.example.com peer channel create -o orderer.example.com:7050 -c mychannel -f /etc/hyperledger/configtx/channel.tx
Error response from daemon: Container 4ebfce361f3e71dd2d678efca1dbf1853cc5387b491f706917b8c54013ec6a80 is not running
docker ps output:
$ docker ps
CONTAINER ID IMAGE COMMAND CRED STATUS PORTS AMES
2d93296f3cb1 hyperledger/fabric-couchdb "tini -- /docker-entâ¦" 13nutes ago Up 13 minutes 4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcpcouchdb
6b8638d0ecaf hyperledger/fabric-ca "sh -c 'fabric-ca-seâ¦" 13nutes ago Up 13 minutes 0.0.0.0:7054->7054/tcp ca.example.com
docker ps -a output:
$ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
4ebfce361f3e hyperledger/fabric-peer "peer node start" 15 minutes ago Exited (1) 15 minutes ago peer0.org1.example.com
1187120cdcd0 hyperledger/fabric-orderer "orderer" 15 minutes ago Exited (1) 15 minutes ago orderer.example.com
2d93296f3cb1 hyperledger/fabric-couchdb "tini -- /docker-entâ¦" 15 minutes ago Up 15 minutes 4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcp couchdb
6b8638d0ecaf hyperledger/fabric-ca "sh -c 'fabric-ca-seâ¦" 15 minutes ago Up 15 minutes 0.0.0.0:7054->7054/tcp ca.example.com
For me this issue was caused by the volume handoff from WSL (Windows Subsystem for Linux - available on Windows 10) to Docker. Use Kitematic to view your docker containers (See screenshot below and note how the folder path is messed up). Click the container and go to "Settings" and "Volumes". Changing the volume manually made it run properly.
In my case, docker automatically is changing \c to C:
I was able to resolve it with this trick: https://superuser.com/questions/1195215/docker-client-on-wsl-creates-volume-path-problems/1203233
WSL mounts the windows drives at /mnt/ (so /mnt/c/ = C:). If we create a /c/ bind point, this allows Docker to properly interpret this as C:\
Here's the process:
Bind the drive and download fabric to your windows user directory by using the following set of commands (one at a time)
sudo mkdir /c
sudo mount --bind /mnt/c /c
cd /c/Users/YOUR_WINDOWS_USERNAME #(go to C:\Users to see what its called)
mkdir blockchain #(or whatever you want to call it)
cd blockchain
curl -O https://raw.githubusercontent.com/hyperledger/composer-tools/master/packages/fabric-dev-servers/fabric-dev-servers.tar.gz
tar -xvf fabric-dev-servers.tar.gz
cd ~/fabric-dev-servers
export FABRIC_VERSION=hlfv12
./downloadFabric.sh
Now you should be able to successfully run Fabric
./startFabric.sh
It may ask you to share C drive with docker. You should do so. You can check the volume mount points and containers in Kitematic, like below
Try to do:
Remove all containers - docker rm -f $(docker ps -aq)
Remove network - docker network prune
Remove the chaincode image (if existing) - docker rmi dev-peer0.org1.example.com
Restart docker
Run startFabric.sh again.
Hope help you.
The actual problem is that the location /etc/hyperledger/msp/peer/signcerts is not accessible. For some reason it seems Docker Toolbox looks for this location under the home directory. So, check the home directory in the Docker VM using
echo $HOME (in my case it is C:/Users/knwny) and then place the fabric-samples folder in that location.
This fixed the problem for me.
Got this working by re-running through the fabric installation guidelines: http://hyperledger-fabric.readthedocs.io/en/latest/prereqs.html
The following procedure was entirely performed in a windows account with admin rights
Initial steps:
Set the GOPATH environment variable
Checked that Go is building executables correctly
All next steps in powershell in Admin mode
Ran npm install npm#5.6.0 -g
sudo apt-get install python (note that for me This command failed, but had python anyway and it seemed to work OK as it couldn't find sudo)
Checked that python 2.7 is installed correctly
Set git config --global core.autocrlf false
Set git config --global core.longpaths true
Ran npm install --global windows-build-tools
Ran npm install --global grpc
Ran the following in Git Bash because cURL didnt work in powershell:
curl -sSL http:/ /bit.ly/2ysbOFE | bash -s 1.2.0
Note that for the above command to work successfully I needed to have Docker running in a shell, i.e. running the Docker Toolbox installed start.sh script (I ran in Git Bash and worked OK).
Finally, in registerUser.js and enrollAdmin.js changed the local host, to be the IP address stated on docker on startup.
fabric_ca_client = new Fabric_CA_Client('http://localhost:7054', null , '', crypto_suite);
After doing this I re-ran the Fabric-samples 1) first-network and 2) fabcar examples and they worked as expected!! I think that I had missed out the GoPath and hadn't successfully installed the cURL command first time I tried.
Thanks to Nhat Duy and knwny for your help to resolve this issue.

Ubuntu 16.04 Cron to run Docker container

I have a virtual machine running Ubuntu 16.04 that I want to run cron-scheduled Docker containers on. I have configured the Docker host and my image repository such that they are accessible by the VM, and images run fine on the machine.
My issue is that when I create a cron-schedule using crontab -e in /var/spool/cron, the containers do not show up. I should be able to view any stopped containers using $ docker ps -a, but it does not show them. Running sudo grep CRON -i /var/log/syslog shows the output:
Mar 20 16:22:01 SpacyVM CRON[121879]: (bdsadmin) CMD (docker run -d
bdsdev.azurecr.io/crawler-public)
Mar 20 16:22:05 SpacyVM CRON[121878]: (CRON) info (No MTA installed,
discarding output)
Mar 20 16:24:01 SpacyVM CRON[124254]: (bdsadmin) CMD (docker run -d
bdsdev.azurecr.io/crawler-public)
Mar 20 16:24:02 SpacyVM CRON[124253]: (CRON) info (No MTA installed,
discarding output)
Now, this does run every 2 minutes, per the schedule for debugging purposes, but docker ps -a shows nothing, even though the containers should have exited and thus should be idle until I use docker rm $(docker ps -aq). Has anyone seen this problem before?
Here is my job configured in crontab:
*/2 * * * * docker run -d bdsdev.azurecr.io/crawler-public
Note: I have not created a mailing setup for the cronjobs to output to, hence the No MTA... error, and I am running this in an Ubuntu VM, not on an Ubuntu docker base image.
Found my problem. When I was originally troubleshooting using grep, my search was omitting a fairly important piece of the log stream. So by using vim to view /var/log/syslog:
Mar 20 06:34:01 SpacyVM dockerd[1469]: time="2018-03-20T06:34:01.634861659Z" level=info msg="Attempting next endpoint for pull after error: manifest unknown: manifest unknown"
I had made a spelling error in my crontab file, so the image is not able to be located by the docker daemon, causing this problem.
Chalk it up to amateur hour

Docker not showing default page

I am using Docker, freshly installed on my Ubuntu. I tried to run the hello-world container and worked fine.
I am trying to use a tomcat server on running :
sudo docker run -it --rm -p 8888:8080 tomcat:8.0
But I can't see any webpage on my localhost.
Otherwise, I see it works when I launch the ps command :
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ce51e081bb36 tomcat:8.0 "catalina.sh run" 2 minutes ago Up 2 minutes 0.0.0.0:8888->8080/tcp berserk_pare
I don't understand why it is not working. It's my very first time on Docker and I don't understand it very well.

Resources