Import tensorflow module is slow in tensorflow 2 - python-3.x

Related: Import TensorFlow contrib module is slow in TensorFlow 1.2.1 also: What can cause the TensorFlow import to be so slow?
I am using an ssd and importing TensorFlow. I have 4 ghz 8 core pc with 16 gb ram (Processor AMD FX(tm)-8350 Eight-Core Processor, 4000 Mhz, 4 Core(s), 8 Logical Processor(s)). TensorFlow takes 10-12 seconds to import.
Is there any way to selectively import parts of TensorFlow?
Would a RAM disk help?
Is there any more work being done on stuff like this or: Slow to import tensorflow.contrib with Python 3 because inspect.stack is slow #11829?
Edit: Python 3.6.8 :: Anaconda, Inc. on windoze 8.1.
Dos box, cygwin bash are slow at 12 seconds.
Vs code bash/power shell is fastest at 8 seconds.
Importing in a .py file like: import tensorflow as tf.
Not sure what environments are.
Edit 2:
PS D:\ray\dev\ml\ss1> conda info --envs
# conda environments:
#
base * D:\Anaconda3
first D:\Anaconda3\envs\first
d:\Anaconda3
first d:\Anaconda3\envs\first
Edit 3: Using the code below, i get 9-10 seconds in a command prompt:
(tf2) D:\ray\dev\ml\ss1>python timeimport.py
import tensorflow: 1 units, 9.796 seconds. 0.1 units/second.
version: 2.0.0
(tf2) D:\ray\dev\ml\ss1>python timeimport.py
import tensorflow: 1 units, 9.448 seconds. 0.11 units/second.
version: 2.0.0
(tf2) D:\ray\dev\ml\ss1>python timeimport.py
import tensorflow: 1 units, 9.421 seconds. 0.11 units/second.
version: 2.0.0
from __future__ import absolute_import, division, print_function, unicode_literals
from contextlib import contextmanager
from timeit import default_timer as timer
#contextmanager
def timing(description: str,units=1,title="",before="") -> None:
if before!="":
print(before,flush=True)
start = timer()
yield
dt = timer() - start
frequency=0 if units is None else (units/dt)
if units is None:
if title is None: print(f"{description}: {dt} seconds.",flush=True)
else: print(f"{title} {description}: {dt} seconds.",flush=True)
else:
#"{0:.2f}".format(a)
dt=round(dt,3)
frequency=round(frequency,2)
print(f"{title} {description}: {str(units)} units, {dt} seconds. {str(frequency)} units/second.",flush=True)
return dt
with timing("import tensorflow",1):
import tensorflow as tf
print("version:",tf.__version__)
Edit 4: turning of windows degender, i get 8-9 seconds instead of 9-10 seconds.
Edit 5: i found a workaround:
make a notebook with:
import tensorflow as tf
print(tf.__version__)
import tensorflow_datasets as tfds
import code.py
then in your code.py:
print("enter imported code")
import tensorflow as tf
print(tf.__version__)
# !pip install -q tensorflow-datasets
import tensorflow_datasets as tfds
import matplotlib.pyplot as plt
import numpy as np
#tfds.list_builders()
ds = tfds.load('mnist', split='train', shuffle_files=True)
...
so you run the notebook once and it takes 10 seconds. next time it goes like the wind.

I want to start off by saying that I'm using a 3 Ghz quad core and it does not take me any where near ten seconds to import TensorFlow in Python. Could you elaborate on what environment you're having issues importing it with (i.e. Windows/Mac/Linux in terminal/console/command prompt/Anaconda etc.)?
You didn't specify how you are trying to import Tensorflow, but considering you tagged it with python-3.x I'm assuming you are importing Tensorflow with Python. I'm sure this won't be a popular answer but maybe consider using Tensorflow with a compiled language like c++. It is well known that interpreted languages such as Python are considerably slower than compiled languages and if speed is paramount then it would seem obvious to use TensorFlow in its native language.

When I import tensorflow from a Thinkpad T560(i7 6600U with integrated GPU) in my office, it takes over 10 secs(about 15 secs).
But recently I got a new Laptop Lenovo Y7000 at home(i7 10750H and GTX 1650 Card, SAMSUNG SSD), and I installed CUDA dependencies and able to run TensorFlow with CUDA successfully.
I would say now I feel like I switch from an old carriage to a bullet train.

Related

How to get AWS GPU instance details using Python-tensorflow

I created an AWS GPU instance with g4dn.xlarge instance type.
I installed Python and Jupyter-notebook as well.
When I am trying to load the GPU details in the Jupyter notebook with the below code:
import tensorflow as tf
tf.config.list_physical_devices()
Output:
[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]
I tried few other methods as well
1
import tensorflow as tf
gpus = tf.config.list_physical_devices('GPU')
for gpu in gpus:
print("Name:", gpu.name, " Type:", gpu.device_type)
2
from tensorflow.python.client import device_lib
device_lib.list_local_devices()
3
from tensorflow.python.client import device_lib
def get_available_gpus():
local_device_protos = device_lib.list_local_devices()
return [x.name for x in local_device_protos if x.device_type == 'GPU']
get_available_gpus()
All the codes will respond with no GPU.
what could be the possible options to access the GPU?
Since you said you installed Python yourself, it's likely you didn't start with a deep learning AMI with all the drivers installed, so you'd have to install Nvidia drivers, CUDA, and cudnn. But trying to install Nvidia drivers on an AWS EC2 instance can be tough...
Solution: start with the deep learning AMIs:
https://aws.amazon.com/machine-learning/amis/

After downgrading Tensorflow 2.0 to 1.5 results changed and results reproduction is not available

Would you help me to achieve reproducible results with Tensorflow 1.15 without restarting Python kernel. And why the output results in TF 2.0 and TF 1.5 are different with absolutely identical parameters and dataset? Is it possible to achieve identical output?
More details:
I tried to interpret model results in TF 2.0 by:
import shap
background = df3.iloc[np.random.choice(df3.shape[0], 100, replace=False)]
explainer = shap.DeepExplainer(model, background)
I recieved an error:
`get_session` is not available when using TensorFlow 2.0.`get_session` is not available when using TensorFlow 2.0.
According to the SO topic, I tried to setup TF 2.0 compatibility with TF 1 by using in the front of my code:
import tensorflow.compat.v1 as tf
But the error appeared again.
Following advice by many users, I downgraded TF2 to TF 1.15 it solved the problem, and shap module interprets the results but:
1) to make results reproducible now I have to change tf.random.set_seed(7) on tf.random.set_random_seed(7) and restart the Python kernel every time! In TF2 I didn't have to restart the kernel.
2) prediction results has been changed, especially, Economical efficiency (that is, TF1.5. wrongly classifies more important samples than TF2.0).
TF 2:
Accuracy: 94.95%, Economical efficiency = 64%
TF 1:
Accuracy: 94.85%, Economical efficiency = 56%
The code of the model is here
First, results differ from each other not only in TF1 and TF2 versions, but also in TF2.0 and TF2.2 versions. Probably, it depends on diffenent internal parameters in the packages.
Second, TensorFlow2 works with DeepExplainer in the following versions:
import tensorflow
import pandas as pd
import keras
import xgboost
import numpy
import shap
print(tensorflow.__version__)
print(pd.__version__)
print(keras.__version__)
print(xgboost.__version__)
print(numpy.__version__)
print(shap.__version__)
output:
2.2.0
0.24.2
2.3.1
0.90
1.17.5
0.35.0
But you will face some difficulties in updating the libraries.
In Python 3.5, running TF2.2, you will face the error 'DLL load failed: The specified module could not be found'.
It 100% can be solved by installing newer C++ package. See this:https://github.com/tensorflow/tensorflow/issues/22794#issuecomment-573297027
Link to download the package:https://support.microsoft.com/ru-ru/help/2977003/the-latest-supported-visual-c-downloads
In Python 3.7 you will not find the shap 0.35.0 version with whl extention. Only tar.gz extension which gives the error: "Install visual c++ package". But installation doesn't help.
Then download shap 0.35.0 for Python 3.7 here: https://anaconda.org/conda-forge/shap/files. Run Anaconda shell. Type: conda install -c conda-forge C:\shap-0.35.0-py37h3bbf574_0.tar.bz2.

running tensorboard for the code in "Hands-On Tensorboard for Developers" by Joe Laba

I am attempting to run the code in Video 1.3 of this series on a Windows 10 machine. I am using the VScode IDE with Python 3.7 64 bit IDE from the Miniconda3 distribution. I have solved many config errors between versions of tensorflow, tensorboard, and Cuda and have generated a run file. Running tensorboard --logfile=./runfile from the directory now runs without and error but it does nothing. Does not output a URL and when I try to open localhost:6006 from my Firefox browser, it can't find it. I love puzzles but am now frustrated.
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
#import numpy as np #Change all calls to numpy instead of np or set np = numpy
import os
import sys
import argparse
"""
This code implements the tensor board programming from
the book "Hands-On TensorBoard for PyTorch Developers [Video]" by joe Laba
The book/videos use Tensorboard 2.0.2, Torch 1.3.1.
It requires the use of the Python 3.7 64 bit environment which comes from
the Miniconda3 folder. This upgrades to the following packages;
numpy (1.17.2)
pillow (7.1.1)
python (3.7.1)
pytorch (1.2.0)
scipy(1.3.1)
tensorboard(2.1.1)
tensorflow(2.1.0)
tensorflow-gpu (2.1.0)
theano(1.0.1)
torchvision (0.4.0)
Which compiles with no errors
NVidia driver 441.22
cuDnn 7.6.5
Cuda 10.2.95
"""
###################################################################
# Variables #
# When launching project or scripts from Visual Studio, #
# input_dir and output_dir are passed as arguments automatically. #
# Users could set them from the project setting page. #
###################################################################
input_dir = None
output_dir = None
log_dir = None
#################################################################################
# PyTorch imports. #
#################################################################################
import tensorflow
import tensorboard
#from tensorflow.python.keras.callbaccks import Tensorboard
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.tensorboard import SummaryWriter
def main():
print("I am now in main")
# Set up Tensorboard
#Writer will output to ./runs/ directory by default
writer = SummaryWriter()
for x in range(5):
y=100*x
writer.add_scalar ('y',y,x)
writer.close()
print("done with writer")
print("more done with writer")
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--input_dir", type=str,
default=None,
help="Input directory where where training dataset and metadata are saved",
required=False
)
parser.add_argument("--output_dir", type=str,
default=None,
help="Input directory where where logs and models are saved",
required=False
)
args, unknown = parser.parse_known_args()
input_dir = args.input_dir
main() #This causes main to run
After researching the source docs for VS2019 environments and PED-514
I removed multiple unused environments.
then I did pip3 uninstall ... of multiple versions of tensorflow, tensorboard and numpy.
then I did pip install of compatible versions of tensorflow (2.1.0), tensorboard (2.1.1) and numpy (1.18.3). Now things look great.

import of keras and tensorflow seems to break pycharm

I'm facing a weird issue.
I'm working in Pycharm IDE and I want to use keras with tensorflow backend.
When I try to import keras or tensorflow pycharm process finished with exit code -1073741819 (0xC0000005).
For example
import tensorflow as tf
n_input = 2
print (n_input)
returns message described above without printing anything.
If i comment the tf import it works.
Keras and tensorflow are correctly installed under the python interpreter of my choice.
I'm on windows machine, Python 3.6.8
Thanks

Problems setting up FastAi working on Colab

I'm trying to work through a tutorial on CycleGANs using the Colab platform but I'm struggling to find a way through the 'simple' part of just importing libraries.
I'm just trying to import the following:
from fastai.conv_learner import *
from fastai.dataset import *
from cgan.options.train_options import *
from sklearn.model_selection import train_test_split
from cgan.options.train_options import TrainOptions
from cgan.data.data_loader import CreateDataLoader
from cgan.models.models import create_model
from cgan.util.visualizer import Visualizer
from google_images_download import google_images_download
I'm currently stuck with an error on the first line:
----> 7 class IntermediateLayerGetter(nn.ModuleDict):
8 """
9 Module wrapper that returns intermediate layers from a model
AttributeError: module 'torch.nn' has no attribute 'ModuleDict'
The code can be found online: https://colab.research.google.com/drive/1dMQWStzLfAHDTGuKaUNQn1aOBWeJw4FN
Did you check this issue it sorts partially the problem.
Afterthat I received and this error (Detected that PyTorch and torchvision were compiled with different CUDA versions. PyTorch has CUDA Version=9.0 and torchvision has CUDA Version=10.0. Please reinstall the torchvision that matches your PyTorch install.) and solve it with
pip install torch==1.0.1 -f https://download.pytorch.org/whl/cu100/stable
I hope it helps :)

Resources