I am using keras wrapper with tensorflow as backend. I have a huge image which I subtile into smaller images and run a script which generates a heatmap individually on them. As its using tensorflow as backend it uses GPU and optimizes the computation for that particular heat map.
However I am trying to run multiple instances of the script at the same time on a single GPU using CUDA.
How do I calculate the % of GPU usage each time the script runs? I am planning to use the remaining part of GPU to parallelize the process.
Related
I perform some preprocessing issues which takes approx. 14 min for CPU runtime in Google Colab. But when I switch to GPU or TPU the runtime period doesn't change, though it shows:
Found TPU at: grpc://10.113.22.130:8470
Is there any specific way to utilize the resource of TPU to reduce the runtime period?
If you want to actually utilize the GPU/TPU in Colab then you generally do need to add additional code (the runtime doesn't automatically detect the hardware, at least for TPU).
Here is a Colab example you can follow to utilize the TPU. However I will note that generally data preprocessing runs on the CPU anyways regardless if running on CPU or GPU and the magic of the accelerator kicks in when you accelerate actual training.
If you want to speed up your preprocessing, the best practice is to use tf.data. You can learn more about how TF data is used to improve performance here.
I am currently trying to use the vgg16 model from keras library but whenever I create an object of the VGG16 model by doing
from keras.applications.vgg16 import VGG16
model = VGG16()
I get the following message 3 times.
tensorflow/core/framework/allocator.cc.124 allocation of 449576960 exceeds 10% of system memory
following this, my computer freezes. I am using a 64-bit, 4gb RAM with linux mint 18 and I have no access to GPU.
Is this problem has to do something with my RAM?
As a temporary solution I am running my python scripts from command line because my computer freezes less there compared to any IDE. Also, this does not happen when I use any alternate model like InceptionV3.
I have tried the solution provided here
but it didn't work
Any help is appreciated.
You are most likely running out of memory (RAM).
Try running top (or htop) in parallel and see your memory utilization.
In general, VGG models are rather big and require a decent amount of RAM. That said, the actual requirement depends on batch size. Smaller batch means smaller activation layer.
For example, a 6 image batch would consume about a gig of ram (reference). As a test you could lower your batch size to 1 and see it that fits in your memory.
When I'm running my Python code on the most powerfull AWS GPU instances (with 1 or 8 x Tesla v100 16mb aka. P3.x2large or P3.16xlarge) they are both only 2-3 times faster than my DELL XPS Geforce 1050-Ti laptop?
I'm using Windows, Keras, Cuda 9, Tensorflow 1.12 and the newest Nvidia drivers.
When I check the GPU load via GZU the GPU max. run at 43% load for a very short period - each time. The controller runs at max. 100%...
The dataset I use is matrices in JSON format and the files are located on a Nitro drive at 10TB with MAX 64.000 IOPS. No matter if the folder contains 10TB, 1TB or 100mb...the training is still very very slow per iteration?
All advises are more than welcome!
UPDATE 1:
From the Tensorflow docs:
"To start an input pipeline, you must define a source. For example, to construct a Dataset from some tensors in memory, you can use tf.data.Dataset.from_tensors() or tf.data.Dataset.from_tensor_slices(). Alternatively, if your input data are on disk in the recommended TFRecord format, you can construct a tf.data.TFRecordDataset."
Before I had matrices stored in JSON format (Made by Node). My TF runs in Python.
I will now only save the coordinates in Node and save it in JSON format.
The question is now: In Python what is the best solution to load data? Can TF use the coordinates only or do I have to make the coordinates back to matrices again or what?
The performance of any machine learning model depends on many things. Including but not limited to: How much pre-processing you do, how much data you copy from CPU to GPU, Op bottlenecks, and many more. Check out the tensorflow performance guide as a first step. There are also a few videos from the tensorflow dev summit 2018 that talk about performance. How to properly use tf.data, and how to debug performance are two that I recommend.
The only thing I can say for sure is that JSON is a bad format for this purpose. You should switch to tfrecord format, which uses protobuf (better than JSON).
Unfortunately performance and optimisation of any system takes a lot of effort and time, and can be a rabbit hole that just keeps going down.
First off, you should be having a really good reason to go for an increased computational overhead with Windows-based AMI.
If your CPU is at ~100%, while GPU is <100%, then your CPU is likely the bottleneck. If you are on cloud, consider moving to instances with larger CPU-count (CPU is cheap, GPU is scarce). If you can't increase CPU count, moving some parts of your graph to GPU is an option. However, tf.data-based input pipeline is run entirely on CPU (but highly scalable due to C++ implementation). Prefetching to GPUs might also help here, but the cost of spawning another background thread to populate the buffer for downstream might damp this effect. Another option is to do some or all pre-processing steps offline (i.e. prior to training).
A word of caution on using Keras as the input pipeline. Keras relies on Python´s multithreading (and optionally multiprocessing) libraries, which may both lack performance (when doing heavy I/O or augmentations on-the-fly) and scalability (when running on multiple CPUs) compared to GIL-free implementations. Consider performing preprocessing offline, pre-loading input data, or using alternative input pipelines (as the aforementioned TF native tf.data, or 3rd party ones, like Tensorpack).
I have installed Tensorflow and Keras with an Anaconda installation on Windows 10. I´m using an Intel i7 processor. It takes 40 minutes to train 4000 data samples of a CSV file and I´m trying to perform a LSTM RNN predictive analytics on this data.
Is this an expected compile time using CPU? Can we make it faster using cpu or switching to GPU?
Yes, this does seem like a reasonable amount of time for your code to run when you're training using only a CPU. If you used a NVIDIA GPU it would run much faster.
However, you might not be using every core on your CPU; if you are, it might run faster. You can change the number of threads that Tensorflow uses by running
sess = tf.Session(config=tf.ConfigProto(intra_op_parallelism_threads=NUM_THREADS))
If you set the number of threads equal to those provided by your CPU, it should run faster.
I am trying to use Tensorflow for my work on a classification problem.
Before feeding the input in the form of images, I want to do some pre-processing on the images. I would like to carry out this pre-processing on multiple CPU cores in parallel and feed them to the TensorFlow graph which I want to run in a Multi-GPU setting (I have 2 TitanX GPUs).
The reasoning I want this setup is, so that while the GPU is performing the training, the CPUs keep on doing their job of pre-processing and hence after each iteration, the GPU does not remain idle. I have been looking into the TensorFlow API for this, but could not locate something which specifically addresses such a scenario.
So, multiple CPU cores should keep on pre-processing a list of files and fill in a queue from which TensorFlow extracts its batch of data. Whenever this queue is full, CPU cores should wait and again start processing when the queue (or part of it) is vacated due to feeding of examples to TensorFlow graph.
I have two questions specifically :
How may I achieve this setup ?
Is it a good idea to have this setup ?
A clear example would be a great help.