I am using google colab and PyTorch. I set my hardware accelerator to TPU.
This line of code shows that no cuda device is being detected:
device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
print(device)
TPUs are not CUDA devices. Please take a look at this Colabthat shows how to run some PyTorch code on TPUs.
Related
I want to use a specific GPU. I followed the instructions of some posts including (1, 2) but they don't work for me. More specifically, I when I run the code for debugging with vscode, even when I select the GPU using: dev = torch.device('cuda:1' if torch.cuda.is_available() else 'cpu') the program is run on the device 0 and device 2. How can I fix that?
This is the code I use:
dev = torch.device('cuda:1' if torch.cuda.is_available() else 'cpu')
model = model.to(dev)
I am running code on server. There are 2 GPUs there, and the 1st one is busy. Yet, I can't find a way to switch between them. I am using pytorch if that is important. Following lines of code should be modified:
device = 'cuda' if torch.cuda.is_available() else 'cpu'
Modification may be stated only here.
Thanks.
cuda by defaults chooses cuda:0, switching to the other GPU may be done through cuda:1
So, your line becomes:
device = 'cuda:1' if torch.cuda.is_available() else 'cpu'
You can read more about CUDA semantics.
Here is the way I'm doing it while using FastAI and pre-trained model for inference.
First, while model definition with fai (import fastai.vision.all as fai) I obtain the model instance and put it to specified GPU (say with gpu_id=3):
model = fai.nn.Sequential(body, head)
model.cuda(device=gpu_id)
Then, while loading model weights I also specify which device to use (otherwise it creates the copy of a model in GPU 0):
model.load_state_dict(torch.load(your_model_state_filepath, torch.device(gpu_id)))
everyone. I have a problem when I use Keras with backend cntk-gpu version 2.5.1.
For some reason, I have to use cntk-gpu 2.5.1 as Keras backend and I have a piece of code with the core code as follows (really simple code for prediction):
# test_x: test data from dataset imagenet
predict_model = keras.models.load_model(model_path,custom_objects=ModelUtils.custom_objects())
predict_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
res = predict_model.predict(test_x,batch_size=4)
I found that the cntk-gpu2.7 version only takes 50 seconds, but the version 2.5.1 takes 20 minutes. The console shows that cntk does use gpu because it prints my GPU information like:
Selected GPU[0] GeForce GTX 1080 Ti as the process wide default device
I have tested both on ubuntu18.04 cuda10.0 and ubuntu16.04 cuda9.1. Version 2.4 and below will also encounter this problem
I don't know what the reason is and how to solve the problem.
Have you ever encountered a situation where the cntk-gpu 2.5 version is running very slowly?
I hope to hear from you. Thank you in advance.
My Google colab crashes immediately as soon as it starts training on tiny-imagenet with 0.1 million images and 200 classes of size 64*64
Colab log shows
WARNING:root:kernel 1fe0be22-c98a-4519-a16a-69c9fb4be1da restarted
KernelRestarter: restarting kernel (1/5), keep random ports
tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library libcublas.so.10.0 locally
tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10754 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
I am using model.fit_generator with batch size (tried from 32 till 1024) and size of image ( tried from 16 till 64) but nothing works.
I tried resnet-18 architecture with (1.8*10^9 params) as well as custom model too with 0.8 million params but in vain.
I am pasting the link to my colab in case anybody needs some other info
https://colab.research.google.com/drive/1QG1mg1zOn6gZaaSv4rrI4F6erdxsxQ8V#scrollTo=Uy0M-VDHivOX
I've used YOLO detection with trained model using my GPU - Nvidia 1060 3Gb, and everything worked fine.
Now I am trying to generate my own model, with param --gpu 1.0. Tensorflow can see my gpu, as I can read at start those communicates:
"name: GeForce GTX 1060 major: 6 minor: 1 memoryClockRate(GHz): 1.6705"
"totalMemory: 3.00GiB freeMemory: 2.43GiB"
Anyway, later on, when program loads data, and is trying to start learning i got following error:
"failed to allocate 832.51M (872952320 bytes) from device: CUDA_ERROR_OUT_OF_MEMORY"
I've checked if it tries to use my other gpu (Intel 630) , but it doesn't.
As i run the train process without "--gpu" mode option, it works fine, but slowly.
( I've tried also --gpu 0.8, 0.4 ect.)
Any idea how to fix it?
Problem solved. Changing batch size and image size in config file didn't seem to help as they didn't load correctly. I had to go to defaults.py file and change them up there to lower, to make it possible for my GPU to calculate the steps.
Look like your custom model use to much memory and the graphic card cannot support it. You only need to use the --batch option to control the size of memory.