Is there a command to update (pull) all the downloaded Docker images at once in the terminal ?
No, there is no built-in command to pull all docker images at once.
But you can try this (multiline) bash using docker --format :
for image in $(docker images --format "{{.Repository}}:{{.Tag}}" | grep -v '<none>')
do
docker pull $image
done
Or in one line:
for image in $(docker images --format "{{.Repository}}:{{.Tag}}" | grep -v '<none>'); do docker pull $image; done;
You can use this :
docker images | awk '{print $1":"$2}' | grep -v REPOSITORY | xargs -L1 docker pull
The most "dockerist" way to do it is:
docker images --format "{{.Repository}}:{{.Tag}}" | xargs -L1 docker pull
Related
Problem : I'm using node-machine-id library inside container
This library should get the default machine-id exist in one of files /var/lib/dbus/machine-id , /etc/machine-id or create new one
During updating my docker node image from 14.19.0 to 14.19.1 , I found that docker have different behavior for machine-id ?
My Notice :
Same OS version is used , so there is no difference from os behavior
In the Image for Node version 14.19.1, I found that the file /etc/machine-id does not exist during running the container.
In the Image for Node version 14.19.0, I found that the file /etc/machine-id exists during running the container.
My Code :
Dockerfile :
FROM node:14.19.1
CMD uname -a && node --version && cat /etc/machine-id /var/lib/dbus/machine-id
Command I use to test ( to build image , run container , delete the container ,and delete the image )
temp_image_name=nodeimageupdate ; docker rm $(docker ps --filter status=exited | grep $temp_image_name) 2>/dev/null ; docker build -t $temp_image_name . ; docker run -d -it $temp_image_name ; sleep 1 ; docker logs $(docker ps -a | grep $temp_image_name | awk '{print $1;}' | xargs echo ) ; docker rm $(docker ps --filter status=exited | grep $temp_image_name | awk '{print $1;}' | xargs echo); docker rmi -f $(docker images | grep $temp_image_name | awk '{print $3;}' | xargs echo ) ;
Result for node 14.19.1 ( no machine-id ) , So running twice will have non statice machine id
Result for node 14.19.0 ( machine-id file exist ) , So running twice will have same statice machine id
I cannot recognize why that happens , Any help ?
I have a bunch of Docker containers running on a server and I used the "latest" tag or no tag at all for all of them. Now I want to freeze the image versions, but I have no idea when I pulled these images, so I can't tell which version "latest" is referring to. docker ps is just showing me that the containers use the "latest" or no tag, like this:
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
371d6675888b node:latest "npm start" 6 days ago Up 2 hours project_xyz_1
ca5a75425a34 selenium/node-chrome "/usr/bin/nohup go..." 6 days ago Up 2 hours project_xyz-chrome_1
...
All images that I use are public images from the docker hub.
I thought maybe I could use the hex ID that docker ps shows for all the containers, but then I realized that the IDs are container IDs and not image IDs.
Is it maybe possible to get the image IDs/hashes of all running containers and then scan for all matching tags or something like that?
Docker version: 18.09.1, build 4c52b90
Edit:
So there have been some answers showing how to get the IDs (digests) of the images, but I need to somehow find the actual tags of those images.
After doing some research, I found that the docker hub has an API and that there is a way to get all tags for a given image and there is a way to get the digest for a given image+tag. After looking at the API and a lot of examples from stackoverflow, I came up with this:
(It also includes the code required to get the digest of local images, taken form the answers below)
function getDigestByImageNameWithTag () {
TARGET_IMAGE_NAME_WITH_TAG="$1" # works with and without tag
docker image inspect --format '{{index .RepoDigests 0}}' "$TARGET_IMAGE_NAME_WITH_TAG" | cut -d '#' -f2
}
function getTagsByDigest () {
TARGET_IMAGE_NAME="$1"
TARGET_DIGEST="$2"
# prepend the image name with "library/" if it doesn't contain a slash
if [[ $TARGET_IMAGE_NAME != *"/"* ]]; then
TARGET_IMAGE_NAME="library/$TARGET_IMAGE_NAME"
fi
# get authorization token for the given image name
TOKEN=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$TARGET_IMAGE_NAME:pull" | jq -r .token)
# find all tags for the given image name
ALL_TAGS=$(curl -s -H "Authorization: Bearer $TOKEN" https://index.docker.io/v2/$TARGET_IMAGE_NAME/tags/list | jq -r .tags[])
# itate over all these tags
for TAG in ${ALL_TAGS[#]}; do
# get image digest
DIGEST=$(curl -s -D - -H "Authorization: Bearer $TOKEN" -H "Accept: application/vnd.docker.distribution.manifest.v2+json" https://index.docker.io/v2/$TARGET_IMAGE_NAME/manifests/$TAG | grep Docker-Content-Digest | cut -d ' ' -f 2)
# if the tag matches the given digest
if [[ $TARGET_DIGEST = $DIGEST ]]; then
# "return" the tag
echo "$TAG"
fi
done
}
function getContainerImageNames () {
docker inspect $(docker ps | awk '{print $2}' | grep -v ID) | jq .[].RepoTags | grep -v "\[" | grep -v "\]" | grep " " | cut -d '"' -f2 | cut -d '/' -f2-
}
# get all image names of all local containers
IMGS_WITH_TAG=$(getContainerImageNames)
# iterate of those image names
for IMAGE_NAME_WITH_TAG in ${IMGS_WITH_TAG[#]}; do
# get the digest of the current iteration's IMAGE_NAME_WITH_TAG
DIGEST=$(getDigestByImageNameWithTag $IMAGE_NAME_WITH_TAG)
echo "TARGET_DIGEST: $DIGEST"
# get the raw image name without the tag
IMAGE_NAME=$(echo "$IMAGE_NAME_WITH_TAG" | cut -d ':' -f1)
# find all tags for this image that have the same digest
MATCHING_TAGS=$(getTagsByDigest $IMAGE_NAME $DIGEST)
echo "Image: $IMAGE_NAME_WITH_TAG"
echo "Image digest: $IMAGE_NAME"
echo "Image tags with same digest: "
echo "$MATCHING_TAGS"
echo "-----------------------------"
done
Unfortunately it seems to take forever to finish. I'm not sure if I'm doing something wrong, but that's the best thing I could come up with.
Any ideas on how to make this work properly?
I think this is a better approach without inspecting the container, as docker ps already printing the docker image tag form which the container is created.
docker inspect $(docker ps | awk '{print $2}' | grep -v ID) | jq .[].RepoTags
So first this gets the list of running containers, then inspect each image being used by running container and using jq get all repo tags of that image.
Here is the output.
Updated:
Here is you go using skopeo , you can do using API but will do the effort, so why if you have skopeo
You do not need to install skopeo you can run the container and then or remove once get the result, or you can install, script support both
running_container=$(docker ps | awk '{print $2}' | grep -v ID)
echo "running container: $running_container"
for image in $running_container
do
local_tag=$(echo "$image" | awk -F":" '{print $2}')
if [ -z $local_tag ]; then
# if tag is empty then tag is latest
local_tag="latest"
image="$image":"$local_tag"
fi
local_digest=$(docker inspect $image | jq '.[].RepoDigests[]' | awk -F"#" '{print $2}' | tr -d '"')
echo "Local digest is:" $local_digest
remote_digest=$(docker run --rm --env image=$image alexeiled/skopeo:latest ash -c "skopeo inspect docker://docker.io/$image" | jq '.Digest' | tr -d '"' )
echo $remote_digest
# option2 install the skopeo on your local system
# remote_digest=$(skopeo inspect docker://docker.io/$image | jq '.Digest' | tr -d '"')
echo "Remote digest is : "$remote_digest
if [ "${local_digest}" == "${remote_digest}" ]; then
echo "local image is up to date with remote"
else
echo "Remote image is updated; please run docker pull $image"
fi
done
The RepoDigest field in the image inspect will have a sha256 reference if you pulled the image from a registry:
docker ps --format '{{.Image}}' | xargs \
docker image inspect --format '{{if .RepoDigests}}{{index .RepoDigests 0}}{{end}}'
For a single image like node:latest on your host, that looks like:
docker image inspect --format '{{index .RepoDigests 0}}' node:latest
That digest cannot be changed by a push to the registry of the same tag name. When you pull the updated tag from the registry, you will see this digest update.
The docker inspect command can be used for this.
You can take a look at the answer here https://stackoverflow.com/a/54075889/8113039
Docker images and containers are identified by an ID and for a running container you can get the Id of its image and then pull the image corresponding to the given ID.
First you need to use docker inspect on all your running containers in order to get the sha256 Id the image on which the container is based.
docker inspect returns the image ID under "Image" :
{
"Id": "6de053a2afa4499471c5e5c2afe0b0d83c9c7e50fc7e687fb63a7ebfd2bff320",
...
},
"Image": "sha256:26eb6780e26887a6838684a549562c0404fd85c55f71e0af6c79a4da5505d2a7",
....
}
Then you simply have to pull those images by digest (immutable identifier)
$ docker pull node#sha256:the-image-digest-here
or
$ docker pull node#sha256:26eb6780e26887a6838684a549562c0404fd85c55f71e0af6c79a4da5505d2a7
If you are lucky images corresponding to those digests are still available into the docker hub.
After that, is you are still facing latest images I will suggest you to rename those images with a proper name and tag and pull them in your own docker hub repository to be able to use them directly...
I've installed a Kubernetes cluster using Rancher on 5 different CentOS nodes (let's say node1, node2, ..., node5). For our CI run, we need to clean up stale docker images before each run. I created a script that runs on node1, and password-less ssh is enabled from node1 to rest of the nodes. The relevant section of the script looks something like below:
#!/bin/bash
helm ls --short --all | xargs -L1 helm delete --purge
echo "Deleting old data and docker images from Rancher host node."
rm -rf /var/lib/hadoop/* /opt/ci/*
docker images | grep localhost | awk '{print $3}' | xargs docker rmi -f
hosts=(node2 node3 node4 node5)
for host in ${hosts[*]}
do
echo "Deleting old data and docker images from ${host}"
ssh root#${host} docker images | grep localhost | awk '{print $3}' | xargs docker rmi -f
ssh root#${host} rm -rf /var/lib/hadoop/* /opt/ci/*
done
echo "All deletions are complete! Proceeding with installation."
sleep 2m
Problem is, while the docker rmi command inside the for loop runs for all other 4 nodes, I get the error Error: No such image: <image-id> for each of the images. But if I execute same command on that node, it succeeds. I'm not sure what's the issue here. Any help is appreciated.
The problem is that the only the first command in the ssh pipe is executed remotely:
ssh root#${host} docker images | grep localhost | awk '{print $3}' | xargs docker rmi -f
Shell understand that it is
ssh ssh-arguments | grep grep-arguments | awk awk-arguments | xarg xarg-arguments
And the result is that the only docker images is executed remotely. Then the output from the remote docker images is transferred to the local machine where it is filtered by grep and awk and then docker rmi is executed on local machine.
It is necessary to add quotes to inform shell that everything at command line is a ssh argument:
ssh root#${host} "docker images | grep localhost | awk '{print $3}' | xargs docker rmi -f"
I am running this command in centos7 termnial:
docker pull www.someRepository.com/authorization:latest
Now, I want to run the "docker run" command, but I need to know the id of the image that was created
$id=commandThatParsesTheId
Is there a command that gets the id back from the "docker images" list?
You can use the name of image and its' tag
Or you can use
docker images -q | grep yourimagename
docker images | grep yourimagename | awk {'print $3'}
You might be able to clean this up a bit, but the following should work:
docker pull <someimage> | grep "Digest:" | cut -f2 -d " " > container_digest
docker images --digests | grep $(cat container_digest) | sed -Ee 's/\s+/ /g' | cut -f4 -d " "
It maps the digest you receive when you pull an image to the image ID.
you can run the "docker run" command on images name also instead of image id
docker pull www.someRepository.com/authorization:latest
docker run -i -t www.someRepository.com/authorization:latest "/bin/bash"
Above one is to run the container for interactive mode.
you can get image id also to run the docker run command
docker images
docker run -i -t dockerid "/bin/bash"
I'm trying to write a single command in my makefile to get the current folder and remove all "." from the name.
I can get the current folder with $${PWD##*/} and I can remove the "."s with $${PWD//.} but I can't figure out how to combine these two into one.
The reason I need this is to kill my docker containers based on name of project. This is my command:
docker ps -q --filter name="mycontainer" | xargs -r docker stop
and i was hoping I could inject the project name before my container name like this:
docker ps -q --filter name=$${PWD##*/}"_mycontainer" | xargs -r docker stop
You can try:
var=$(echo ${PWD##*/} | sed "s/\.//")
or:
var=$(tmp=${PWD##*/} && printf "${tmp//./}")
In your use case will be something like:
docker ps -q --filter name=$(tmp=${PWD##*/} && printf "%s_mycontainer" "${tmp//./}") | xargs -r docker stop
Note that there are more ways to do that (even more efficient).