enter image description here
nvidia gpu is not available for python3.6, it's only available for python2.7 version.
I'm working on ubuntu, cuda 10.1, and all drivers working properly.
I also tried by creating new python environment but still gpu is available for python, not python3.
what should i do to use the gpu for runing python3 script?
please note that, tensorflow-gpu, torch is already installed and they work on python2, the only problem is that python3 doesn't use the gpu, including jupyter notebook(uses python3).
i use ssh to connect to the server, nvidia docker is already installed.
I was able to solve the issue by using the nvidia docker images
Look for your docker image:
$ docker images
Run the docker image with specified IMAGE ID:
$ docker run --rm -it --runtime=nvidia --net=host -v /<local dir>/:/<destination dir> <IMAGE ID>
$ docker run --rm -it --runtime=nvidia --net=host -v /my_sever_dir/:/notebook 8d78dd1e1q
To access the jupyter notebook, copy the token and paste it on the browser(in the below example, provide your correct ip address and token. The port:8888 could be different if its already taken, for instance try:8889):
http://YOUR_IP:8888/?token=YOUR_TOKEN_FROM_STEP_2_ABOVE
Related
I'm running
Ubuntu 20.04.5 LTS
Docker version 23.0.1, build a5ee5b1
Running the command
docker build -t some:other Dockerfile
Produces the following output:
unknown shorthand flag: 't' in -t
And
docker build
The following:
docker: 'buildx' is not a docker command.
I installed Docker as recommended from the repo: instructions
Other plugins don't work either (docker compose is not recognized either). Even then, docker info shows
buildx: Docker Buildx (Docker Inc.)
Version: v0.10.2
Path: /home/jpartanen/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.16.0
Path: /home/jpartanen/.docker/cli-plugins/docker-compose
scan: Docker Scan (Docker Inc.)
Docker runs without sudo with the help of the docker user group, as explained in linux-postinstall. I want to run plugins without sudo as well.
I've reinstalled Docker and rebooted the machine without any change.
What could be the problem?
Make the plugins runnable for docker by creating a link:
ln -s /usr/libexec/docker/cli-plugins/ ~/.docker/cli-plugins
The command not being recognized by Docker is extra confusing because of the mismatch in commands, build vs buildx. This is because Docker Engine 23.0 set Buildx and BuildKit as the default builder on Linux. docker build is aliased to docker buildx build.
As for running without sudo, the problem is possibly caused by the plugins being installed in the wrong place. On my machine, running the command
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
installs the plugins in /usr/libexec/docker/cli-plugins/, whereas as laid out here, the plugins are usable from $HOME/.docker/cli-plugins (without sudoing).
A somewhat robust solution is to create a link as laid out above.
I am trying to install linux header for my ubuntu 18.04 docker image (ubuntu:18.04). Usually I will do sudo apt-get install linux-headers-$(uname -r) in my VM to get the current linux header packages.
But the docker image return the following when I run uname -r
root#0c4e24cca819:/# uname -r
4.19.76-linuxkit
Just wonder which linux header image I should use for ubuntu:18.04 docker image?
Docker by definition runs your current kernel. If you are on a machine whose kernel has not been packaged for Ubuntu then there is no package you can install to get its headers.
Looks like you're on a Mac, so definitely that is the case here. Perhaps you could ask the Docker for Mac maintainers to provide headers for some popular platforms for their kernel, but I suspect they don't want to take on that responsibility.
As a workaround, maybe run Docker inside Linux on e.g. Virtualbox.
I am new to docker and this is just a fascinating tool. However, I can't understand one thing about it. Simple Dockerfile usually begins with OS name and version, like:
FROM ubuntu:xenial
....
But which Linux OS will be used for Dockerfile like
FROM perl
....
or
FROM python:3.6
....
Of course I can find this out by running a container from this image and printing out the OS info, like:
docker run -it --rm perl bash
# cat /etc/*-release
or
docker run -it --rm python:3.6 bash
# cat /etc/*-release
BTW, In both cases the OS is "Debian GNU/Linux 10 (buster)".
So, my questions are:
How do I find out which OS will be run for a specific docker image without actually creating a docker container from it (the docker inspect command does not provide this info: docker inspect perl | grep -i Debian)
How do I change the OS type for existing docker image. For example, I have an image that uses Ubuntu 14.04, and I want to change it to Ubuntu 18.04..
Thank you for your help:)
A docker image doesn't need an OS. There's a possibility of extending the scratch image which is purposely empty and the container may only contain one binary or some volume.
Having an entire OS is possible but also misleading: The host shares its kernel with the container. (This is not a virtual machine.)
That means that no matter what "OS" you are running, the same kernel in the container is found:
Both:
docker run --rm -it python:3.6 uname -a
docker run --rm -it python:3.6-alpine uname -a
will report the same kernel of your host machine.
So you have to look into different ways:
docker run --rm -it python:3.6 cat /etc/os-release
or
lsb_release -sirc
or for Cent OS:
cat /etc/issue
In stead of scratch, a lot of images are also alpine-based to avoid the size overhead. An ubuntu base image can easily have 500MB fingerprint whereas alpine uses around 5MB; so I rather check for that as well.
Also avoid the trap of manually installing everything onto one Ubuntu image inside one big Dockerfile. Docker works best if each service is its own container that you link together. (For that check out docker-compose.)
In the end, you as an user shouldn't care about the OS of an image, but rather its size. Only as a developer of the Dockerfile is it relevant to know the OS and that you'll find out either by looking into the Dockerfile the image was built (if it's on docker hub you can read it there).
You basically have to look what was used to create your image an use the appropriate tools for the job. (Debian-based images use apt-get, alpine uses apk, and Fedora uses yum.)
How do I find out which OS will be run for a specific docker image without actually creating a docker container from it
The only way to determine what os is being used is as you have described: spawn a container and print the os information. There is no metadata that says "this image was build using <x>".
In many (but not all) situations, this information may not be especially important.
How do I change the OS type for existing docker image. For example, I have an image that uses Ubuntu 14.04, and I want to change it to Ubuntu 18.04..
If you have access to the Dockerfile used to build the image, you can of course change the base image (the image named in the FROM line) and build a new one, but you may find that this requires a number of other changes due to different software versions in your updated image.
You can use "docker cp" to extract the "/etc/os-release" file without starting the container:
$ docker pull ubuntu:latest
Status: Image is up to date for ubuntu:latest
$ docker create ubuntu:latest
2e5da8bf02312870acd0436e0cc4eb28fbcc998f766cd9639c37101f65739553
$ docker cp -L 2e5da8bf02312870acd0436e0cc4eb28fbcc998f766cd9639c37101f65739553:/etc/os-release .
$ docker rm 2e5da8bf02312870acd0436e0cc4eb28fbcc998f766cd9639c37101f65739553
$ cat ./os-release
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Note: I had to use "docker cp -L" because /etc/os-release is a symlink on ubuntu:latest.
Honestly, I find this to be a lot of trouble just to avoid starting the container, and it requires the "/etc/os-release" file to be present. If you're willing to (very) briefly run the container, I find this more convenient, and a little more robust. Note: it's very important to specify --entrypoint="", otherwise the container will start invoking its normal startup routine!
$ docker run --rm -i -a STDOUT --entrypoint="" \
ubuntu:latest sh -c 'head -n 1000 /etc/hostname /etc/*[Rr][Ee][Ll]*'
==> /etc/hostname <==
b243ff33e245
==> /etc/lsb-release <==
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.2 LTS"
==> /etc/os-release <==
NAME="Ubuntu"
VERSION="20.04.2 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Here's the same command against "alpine:latest":
docker run --rm -i -a STDOUT --entrypoint="" \
alpine:latest 'sh' '-c' 'head -n 1000 /etc/hostname /etc/*[Rr][Ee][Ll]*'
==> /etc/hostname <==
a8521c768aeb
==> /etc/alpine-release <==
3.13.4
==> /etc/os-release <==
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.13.4
PRETTY_NAME="Alpine Linux v3.13"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
Note: I add "/etc/hostname" to the list of files to "head" to make sure it finds 2 or more files, to ensure "head" to uses its "==> file <==" output style. Whereas if it only runs against a single file it doesn't print the filename.
I'm trying to expose an Arduino that's plugged into my mac to a linux instance I'm running in Docker for Mac (no vm).
The Arduino exposes itself as /dev/tty.usbserialXXX. I'm using the node docker image which is based upon ubuntu.
The command I'm running is
$ docker run --rm -it -v `pwd`:/app --device /dev/tty.usbmodem1421 node bash
docker: Error response from daemon: linux runtime spec devices: error gathering device information while adding custom device "/dev/tty.usbmodem1421": lstat /dev/tty.usbmodem1421: no such file or directory.
If I try to use --privileged
$ docker run --rm -it -v `pwd`:/app --device /dev/tty.usbmodem1421 --privileged node bash
root#8f18fdbcf64d:/# ls /dev/tty.*
ls: cannot access /dev/tty.*: No such file or directory
Nothing is exposed!
I'm using this to expose serial devices to test serial drivers in linux.
The problem here is largely that you're not running Docker on your mac. You're running a Linux VM on your Mac, inside which you're running Docker. This means that it's easy to expose the /dev tree inside the Linux VM to Docker, but less easy to expose devices from your Mac, absent some kind of support from the hypervisor.
Using the legacy "Docker Toolbox" for Mac, which is built around VirtualBox, it ought to be possible to assign a USB device to the VirtualBox host running Docker (which would in turn allow you to expose it to your Docker containers).
This GitHub issue talks about this particular situation and has links to helpful documentation.
I don't know if this sort of feature is currently available with the hypervisor used in the newer "Docker for Mac" package.
The Arduino devise that is listed at /dev/tty.usbserialXXX could be a symlink to the device, and not the actual path. To read the symlink path try using
docker run --rm -it -v `pwd`:/app --device=/dev/$(readlink /dev/tty.usbmodem1421) node bash
There was an issue open for this some time back. Do check if it solves your problem
I'm having trouble installing docker inside docker.
Outer docker works just fine but inner docker behavior is doubtful. I've followed all steps in official tutorial but after installation when I call status it still shouts error:
$ > sudo service docker status
/etc/init.d/docker: invalid arguments
* could not access PID file for Docker
BTW starting service works fine:
$ > sudo service docker start
* Starting Docker: docker [ OK ]
But docker itself seems to be working properly. hello-world is working and all other experiment I'm doing are also working so far. But I'm not sure I won't stumble upon some other issue later on.
I tried using several OS combinations. For outer OS i used Centos 7 and LinuxMint 17.3 and inner OS ubuntu:14.04 and ubuntu:16.04 but result is always the same (so I'm guessing it has to do something with ubuntu).
Outer docker is running docker with --privileged flag.
So it looks like this:
[outer OS Centos 7] docker run -ti --privileged ubuntu:14.04 bash
[inner OS Ubuntu] sudo service docker status
Thanks for the help,
x3mSpeedy