I've just found that PyTorch docs expose something that is called Torch Scripts. However, I do not know:
When they should be used?
How they should be used?
What are their benefits?
Torch Script is one of two modes of using the PyTorch just in time compiler, the other being tracing. The benefits are explained in the linked documentation:
Torch Script is a way to create serializable and optimizable models from PyTorch code. Any code written in Torch Script can be saved from your Python process and loaded in a process where there is no Python dependency.
The above quote is actually true both of scripting and tracing. So
You gain the ability to serialize your models and later run them outside of Python, via LibTorch, a C++ native module. This allows you to embed your DL models in various production environments like mobile or IoT. There is an official guide on exporting models to C++ here.
PyTorch can compile your jit-able modules rather than running them as an interpreter, allowing for various optimizations and improving performance, both during training and inference. This is equally helpful for development and production.
Regarding Torch Script specifically, in comparison to tracing, it is a subset of Python, specified in detail here, which, when adhered to, can be compiled by PyTorch. It is more laborious to write Torch Script modules instead of tracing regular nn.Module subclasses, but it allows for some extra features over tracing, most notably flow control like if statements or for loops. Tracing treats such flow control as "constant" - in other words, if you have an if model.training clause in your module and trace it with training=True, it will always behave this way, even if you change the training variable to False later on.
To answer your first question, you need to use jit if you want to deploy your models outside Python and otherwise you should use jit if you want to gain some execution performance at the price of extra development effort (as not every model can be straightforwardly made compliant with jit). In particular, you should use Torch Script if your code cannot be jited with tracing alone because it relies on some features such as if statements. For maximum ergonomy, you probably want to mix the two on a case-by-case basis.
Finally, for how they should be used, please refer to all the documentation and tutorial links.
Related
I am doing a meta learning research and am using the MAML optimization provided by learn2learn. However as one of the baseline, I would like to test a non-meta-learning approach, i.e. the traditional training + testing.
Due to the lightning's internal usage of optimizer it seems that it is difficult to make the MAML work with learn2learn in lightning, so I couldn't use lightning in my meta-learning setup, however for my baseline, I really like to use lightning in that it provides many handy functionalities like deepspeed or ddp out of the box.
Here is my question, other than setting up two separate folders/repos, how could I mix the vanilia pytorch (learn2learn) with pytorch lightning (baseline)? What is the best practice?
Thanks!
Decided to answer my question. So I ended up using the torch lightning's manual optimization so that I can customize the optimization step. This would make both approaches using the same framework, and I think is better than maintaining 2 separate repos.
Torchscript provides torch.jit.trace and torch.jit.script to convert pytorch code from eager mode to script model. From the documentation, I can understand torch.jit.trace cannot handle control flows and other data structures present in the python. Hence torch.jit.script was developed to overcome the problems in torch.jit.trace.
But it looks like torch.jit.script works for all the cases, then why do we need torch.jit.trace?
Please help me understand the difference between these two methods
If torch.jit.script works for your code, then that's all you should need. Code that uses dynamic behavior such as polymorphism isn't supported by the compiler torch.jit.script uses, so for cases like that, you would need to use torch.jit.trace.
I made my own model for application and saved this in Keras as .h5 file. and I made GUI Application using PyQt5 and this application uses this model. I'm trying to deploy this application without any information about deep learning model.
I have some questions about this situation.
Can I hide or encrypt my model to prevent its architecture and weight exposure?
If Keras doesn't support encrypting model, are there any other libraries(like PyTorch) that support this function?
I'm looking forward to hearing any advice. Thank you for your answer.
Model encryption is not officially part of either keras nor pytorch.
I think Python is a big problem if you want to hide something. AFAIK it's not really possible to hide your solution well enough using it, I will outline what I would do to "protect" the model (those are quite lengthy, so make sure you really need this protection [or what level of protection exactly]).
Provided Python solutions
There exists PySyft which handles both PyTorch and Keras but it's used for Secure Multi-Party Computation. As users have access to your Python code (you mentioned PyQT5) and all the sensible data (model in this case) they would be able to recover it quite easily.
Possible solution
If I were you I would go for simple password-protected archive (AES or .zip). For the first case I've found this post and related TFSecured repository, which does AES encryption of tensorflow model via Python and allows you to load saved encrypted protobuf model file in C++ (which should be your way to go, reasons below).
Leave Python alone
Is you want to seriously secure your model (not some kind of mere obfuscation) you shouldn't use Python at the user's side at all.
There is no way to compile Python's code, especially the one using heavy ML libraries like Keras, Tensorflow or PyTorch. Although there are programs like PyInstaller it's notoriously hard to make it work with complex dependencies. Even if you do, users will still be able to get to the code albeit it might be a little harder (PyInstaller just bundles Python, your dependencies and app as a single archive which is later unzipped).
You could further obfuscate the code using pyarmor or a-like but it's all quite easily reversible if someone's determined.
C++
Whether you go for keras/tensorflow or pytorch you can go lower level and use C++ to load your network.
As it is a compiled language all you have to do is to provide a binary file (if linking statically) or binary file with shared libraries. Inside C++ source code you keep your AES/zip key as shown by blog post about TFSecured:
#include <GraphDefDecryptor.hpp>
........
tensorflow::GraphDef graph;
// Decryption:
const std::string key = "JHEW8F7FE6F8E76W8F687WE6F8W8EF5";
auto status = tfsecured::GraphDefDecryptAES(path, // path to *.pb file (encrypted graph)
graph,
key); // your key
if (!status.ok()) {
std::cout << status.error_message() << std::endl;
return;
}
// Create session :
std::unique_ptr<Session> session(NewSession(options));
status = session->Create(graph);
It would be much harder to reverse engineer compiled C++ code to get to key buried inside. Similar procedure could be done for PyTorch as well via some third party tools/libraries. On the other hand you would have to rewrite your PyQt5 app in C++ with Qt5.
i just feel you need one complied model file where your client shouldn't be aware of the model architecture.
So in that case you can have a look at this
It supports all framework that too with privileges' of using python
Have a look here
I'm trying to get some insights into reinforcement learning while using openAI gym as a learning environment. I do this by reading the book Hands-on reinforcement learning with Python. In this book, some code is provided. Often, the code doesn't work, because I have to unwrap it first, as shown in: openai gym env.P, AttributeError 'TimeLimit' object has no attribute 'P'
However, I personally am still interested in the WHY of this unwrapping. Why do you need to unwrap? What does this do exactly? And why isn't it coded like that in the book? Is it outdated software as Giuliov assumed?
Thanks in advance.
Open AI Gym offers many different environments. Each of them with their own set of parameters and methods. Nevertheless they generally are wrapped by a single Class (like an interface on real OOPLs) called Env. This class exposes the common most essential methods of any environment, like step, reset and seed. Having this “interface” class is great, because it allows your code to be environment agnostic. It is also makes things easier if you want to test a single agent on different environments.
However, if you want to access the behind-the.scenes dynamics of a specific environment, then you use the unwrapped property.
So tf.image for example has some elementary image processing methods already implemented which i'd assumed are optimized. The question is as I'm iterating through a large dataset of images what/how is the recommended way of implementing a more complex function on every image, in batches of course, (for example a a patch 2-D DCT) for it to go as best as possible with the whole tf.data framework.
Thanks in advance.
p.s. of course I could use the "Map" method but i'm asking beyond that. like if I'm passing a pure numpy written function to pass to "map", it wouldn't help as much.
The current best approach (short of writing custom ops in C++/CUDA) is probably to use https://www.tensorflow.org/api_docs/python/tf/contrib/eager/py_func. This allows you to write any TF eager code and use Python control flow statements. With this you should be able to do most of the things you can do with numpy. The added benefit is that you can use your GPU and the tensors you produce in tfe.py_func will be immediately usable in your regular TF code - no copies are needed.