I'm trying to write a single command in my makefile to get the current folder and remove all "." from the name.
I can get the current folder with $${PWD##*/} and I can remove the "."s with $${PWD//.} but I can't figure out how to combine these two into one.
The reason I need this is to kill my docker containers based on name of project. This is my command:
docker ps -q --filter name="mycontainer" | xargs -r docker stop
and i was hoping I could inject the project name before my container name like this:
docker ps -q --filter name=$${PWD##*/}"_mycontainer" | xargs -r docker stop
You can try:
var=$(echo ${PWD##*/} | sed "s/\.//")
or:
var=$(tmp=${PWD##*/} && printf "${tmp//./}")
In your use case will be something like:
docker ps -q --filter name=$(tmp=${PWD##*/} && printf "%s_mycontainer" "${tmp//./}") | xargs -r docker stop
Note that there are more ways to do that (even more efficient).
Related
So I am trying to get the combined output of all the container logs with its container name in a log file
docker logs --tail -3 with container name >> logst.log file.
docker logs takes a single command so you would have to run them for each container. I guess I could do something along:
docker ps --format='{{.Names}}' | xargs -P0 -d '\n' -n1 sh -c 'docker logs "$1" | sed "s/^/$1: /"' _
docker ps --format='{{.Names}}' - print container names
xargs - for input
-d '\n' - for each line
-P0 - execute in parallel with any count of parallel jobs
remove this option if you don't intent to do docker logs --follow
it may cause problems, consider adding stdbuf -oL and sed -u to unbuffer the streams
-n1 - pass one argument to the underyling process
sh -c 'script' _ - execute script for each line with line passed as first positional argument
docker logs "$1" - get the logs
sed 's/^/$1: /' - prepend the name of the docker name to each log line
But a way better and industrial grade solution would be to forward docker logs to journalctl or other logging solution and use that utility to aggregate and filter logs.
Got it.
for i in docker ps -a -q --format "table {{.ID}}"; do { docker ps -a -q --format "table {{.ID}}\t{{.Names}}\n" | grep "$i" & docker logs --timestamps --tail 1 "$i"; } >> logs.log; done
logs.log is a generic file name.
I have a bunch of Docker containers running on a server and I used the "latest" tag or no tag at all for all of them. Now I want to freeze the image versions, but I have no idea when I pulled these images, so I can't tell which version "latest" is referring to. docker ps is just showing me that the containers use the "latest" or no tag, like this:
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
371d6675888b node:latest "npm start" 6 days ago Up 2 hours project_xyz_1
ca5a75425a34 selenium/node-chrome "/usr/bin/nohup go..." 6 days ago Up 2 hours project_xyz-chrome_1
...
All images that I use are public images from the docker hub.
I thought maybe I could use the hex ID that docker ps shows for all the containers, but then I realized that the IDs are container IDs and not image IDs.
Is it maybe possible to get the image IDs/hashes of all running containers and then scan for all matching tags or something like that?
Docker version: 18.09.1, build 4c52b90
Edit:
So there have been some answers showing how to get the IDs (digests) of the images, but I need to somehow find the actual tags of those images.
After doing some research, I found that the docker hub has an API and that there is a way to get all tags for a given image and there is a way to get the digest for a given image+tag. After looking at the API and a lot of examples from stackoverflow, I came up with this:
(It also includes the code required to get the digest of local images, taken form the answers below)
function getDigestByImageNameWithTag () {
TARGET_IMAGE_NAME_WITH_TAG="$1" # works with and without tag
docker image inspect --format '{{index .RepoDigests 0}}' "$TARGET_IMAGE_NAME_WITH_TAG" | cut -d '#' -f2
}
function getTagsByDigest () {
TARGET_IMAGE_NAME="$1"
TARGET_DIGEST="$2"
# prepend the image name with "library/" if it doesn't contain a slash
if [[ $TARGET_IMAGE_NAME != *"/"* ]]; then
TARGET_IMAGE_NAME="library/$TARGET_IMAGE_NAME"
fi
# get authorization token for the given image name
TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$TARGET_IMAGE_NAME:pull" | jq -r .token)
# find all tags for the given image name
ALL_TAGS=$(curl -s -H "Authorization: Bearer $TOKEN" https://index.docker.io/v2/$TARGET_IMAGE_NAME/tags/list | jq -r .tags[])
# itate over all these tags
for TAG in ${ALL_TAGS[#]}; do
# get image digest
DIGEST=$(curl -s -D - -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.docker.distribution.manifest.v2+json" https://index.docker.io/v2/$TARGET_IMAGE_NAME/manifests/$TAG | grep Docker-Content-Digest | cut -d ' ' -f 2)
# if the tag matches the given digest
if [[ $TARGET_DIGEST = $DIGEST ]]; then
# "return" the tag
echo "$TAG"
fi
done
}
function getContainerImageNames () {
docker inspect $(docker ps | awk '{print $2}' | grep -v ID) | jq .[].RepoTags | grep -v "\[" | grep -v "\]" | grep " " | cut -d '"' -f2 | cut -d '/' -f2-
}
# get all image names of all local containers
IMGS_WITH_TAG=$(getContainerImageNames)
# iterate of those image names
for IMAGE_NAME_WITH_TAG in ${IMGS_WITH_TAG[#]}; do
# get the digest of the current iteration's IMAGE_NAME_WITH_TAG
DIGEST=$(getDigestByImageNameWithTag $IMAGE_NAME_WITH_TAG)
echo "TARGET_DIGEST: $DIGEST"
# get the raw image name without the tag
IMAGE_NAME=$(echo "$IMAGE_NAME_WITH_TAG" | cut -d ':' -f1)
# find all tags for this image that have the same digest
MATCHING_TAGS=$(getTagsByDigest $IMAGE_NAME $DIGEST)
echo "Image: $IMAGE_NAME_WITH_TAG"
echo "Image digest: $IMAGE_NAME"
echo "Image tags with same digest: "
echo "$MATCHING_TAGS"
echo "-----------------------------"
done
Unfortunately it seems to take forever to finish. I'm not sure if I'm doing something wrong, but that's the best thing I could come up with.
Any ideas on how to make this work properly?
I think this is a better approach without inspecting the container, as docker ps already printing the docker image tag form which the container is created.
docker inspect $(docker ps | awk '{print $2}' | grep -v ID) | jq .[].RepoTags
So first this gets the list of running containers, then inspect each image being used by running container and using jq get all repo tags of that image.
Here is the output.
Updated:
Here is you go using skopeo , you can do using API but will do the effort, so why if you have skopeo
You do not need to install skopeo you can run the container and then or remove once get the result, or you can install, script support both
running_container=$(docker ps | awk '{print $2}' | grep -v ID)
echo "running container: $running_container"
for image in $running_container
do
local_tag=$(echo "$image" | awk -F":" '{print $2}')
if [ -z $local_tag ]; then
# if tag is empty then tag is latest
local_tag="latest"
image="$image":"$local_tag"
fi
local_digest=$(docker inspect $image | jq '.[].RepoDigests[]' | awk -F"#" '{print $2}' | tr -d '"')
echo "Local digest is:" $local_digest
remote_digest=$(docker run --rm --env image=$image alexeiled/skopeo:latest ash -c "skopeo inspect docker://docker.io/$image" | jq '.Digest' | tr -d '"' )
echo $remote_digest
# option2 install the skopeo on your local system
# remote_digest=$(skopeo inspect docker://docker.io/$image | jq '.Digest' | tr -d '"')
echo "Remote digest is : "$remote_digest
if [ "${local_digest}" == "${remote_digest}" ]; then
echo "local image is up to date with remote"
else
echo "Remote image is updated; please run docker pull $image"
fi
done
The RepoDigest field in the image inspect will have a sha256 reference if you pulled the image from a registry:
docker ps --format '{{.Image}}' | xargs \
docker image inspect --format '{{if .RepoDigests}}{{index .RepoDigests 0}}{{end}}'
For a single image like node:latest on your host, that looks like:
docker image inspect --format '{{index .RepoDigests 0}}' node:latest
That digest cannot be changed by a push to the registry of the same tag name. When you pull the updated tag from the registry, you will see this digest update.
The docker inspect command can be used for this.
You can take a look at the answer here https://stackoverflow.com/a/54075889/8113039
Docker images and containers are identified by an ID and for a running container you can get the Id of its image and then pull the image corresponding to the given ID.
First you need to use docker inspect on all your running containers in order to get the sha256 Id the image on which the container is based.
docker inspect returns the image ID under "Image" :
{
"Id": "6de053a2afa4499471c5e5c2afe0b0d83c9c7e50fc7e687fb63a7ebfd2bff320",
...
},
"Image": "sha256:26eb6780e26887a6838684a549562c0404fd85c55f71e0af6c79a4da5505d2a7",
....
}
Then you simply have to pull those images by digest (immutable identifier)
$ docker pull node#sha256:the-image-digest-here
or
$ docker pull node#sha256:26eb6780e26887a6838684a549562c0404fd85c55f71e0af6c79a4da5505d2a7
If you are lucky images corresponding to those digests are still available into the docker hub.
After that, is you are still facing latest images I will suggest you to rename those images with a proper name and tag and pull them in your own docker hub repository to be able to use them directly...
I need to check if container is already present with specific name.
Using if statement for same, however container is present with same name but i am not sure why if statement is not running successfully.
if [`echo password | sudo -S docker ps -all|grep test|cut -d' ' -f1`]
then
statements
else
statements
container with test name is present but its always going inside the else statement. Could you help me on this.
You need blanks between [, ] and `, i.e. :
if [ `echo password | sudo -S docker ps -all|grep test|cut -d' ' -f1` ]
Also, you need fi after the last line.
in addition to #ÔHARA Kazutaka information.
-all does not exists, try --all instead.
The --quiet --filter name=test options are going to make docker filters output using the container's name, and make it prints to the standard output the CONTAINER ID if a container exists.
Give a try to this:
container_name=test
container_id=$(echo password | sudo -S docker ps --all --quiet --filter name="${container_name}")
if [ ! -z "${container_id}" ] ; then
printf "%s is the id of the container with name '%s'\n" "${container_id}" "${container_name}"
else
printf "no docker container with name '%s' has been found.\n" "${container_name}"
fi
I am running this command in centos7 termnial:
docker pull www.someRepository.com/authorization:latest
Now, I want to run the "docker run" command, but I need to know the id of the image that was created
$id=commandThatParsesTheId
Is there a command that gets the id back from the "docker images" list?
You can use the name of image and its' tag
Or you can use
docker images -q | grep yourimagename
docker images | grep yourimagename | awk {'print $3'}
You might be able to clean this up a bit, but the following should work:
docker pull <someimage> | grep "Digest:" | cut -f2 -d " " > container_digest
docker images --digests | grep $(cat container_digest) | sed -Ee 's/\s+/ /g' | cut -f4 -d " "
It maps the digest you receive when you pull an image to the image ID.
you can run the "docker run" command on images name also instead of image id
docker pull www.someRepository.com/authorization:latest
docker run -i -t www.someRepository.com/authorization:latest "/bin/bash"
Above one is to run the container for interactive mode.
you can get image id also to run the docker run command
docker images
docker run -i -t dockerid "/bin/bash"
I use fish shell on my desktop.
We use many servers running nginx within docker. I've tried to create a function so I can ssh to the servers and then log into the docker.
The problem is fish is complaining about the $ in the command, but the command is the one to be executed on the remote server (running bash), not on my machine running fish. I've simplified the script to make it easier to see.
config.fish snippet
function ssh-docker-nginx
ssh -t sysadmin#10.10.10.10 "sudo bash && docker exec -it $(docker ps | grep -i nginx | awk '{print $1}') bash"
end
Fish error:
$(...) is not supported. In fish, please use '(docker)'.
~/.config/fish/config.fish (line 59): ssh -t sysadmin#10.10.10.10 "sudo bash && docker exec -it $(docker ps | grep -i nginx | awk '{print $1}') bash"
^
from sourcing file ~/.config/fish/config.fish
called during startup
Is there a way to get fish to ignore this?
You'll want to single-quote that argument.
In double-quotes (") fish will try to expand everything that starts with a $, so it will see that $( and then print the error for it. But it will also see the $1 in your arguments to awk and expand that.
And when you want single-quotes to go to the called command (like here, where you want the argument to awk to be single-quoted because this'll go through bash's expansion), you need to escape the quotes with \.
Try
ssh -t sysadmin#10.10.10.10 'sudo bash && docker exec -it $(docker ps | grep -i nginx | awk \'{print $1}\') bash'
Thanks for the great advice and tip above about the single/double quotes. Unfortunately the escaped quotes in awk did not play nicely being passed to ssh.
After various options, I settled with this approach (which needed force tty):
function ssh-docker-nginx
cat docker-bash.sh | ssh -t -t sysadmin#10.10.10.10
end
# docker-bash.sh
#!/bin/bash
sudo chmod 777 /var/run/docker.sock
sudo docker exec -it $(docker ps | grep -i nginx | awk '{print $1}') bash