I'm looking at the PyTorch SageMaker Neo compilation examples. They seem to place the required neo_preprocess and neo_postprocess functions inside the train.py file, rather than a serve.py that also encorporates model_fn and predict_fn.
I was wondering, is it possible to instead have a custom serve.py with my own model_fn and predict_fn's, while still using Neo compilation?
neo_preprocess and neo_postprocess are required because our container will use these two functions. However, you can still use your own serve.py with your own model_fn and just make sure you keep the neo_preprocess and neo_postprocess.
Related
Torchscript provides torch.jit.trace and torch.jit.script to convert pytorch code from eager mode to script model. From the documentation, I can understand torch.jit.trace cannot handle control flows and other data structures present in the python. Hence torch.jit.script was developed to overcome the problems in torch.jit.trace.
But it looks like torch.jit.script works for all the cases, then why do we need torch.jit.trace?
Please help me understand the difference between these two methods
If torch.jit.script works for your code, then that's all you should need. Code that uses dynamic behavior such as polymorphism isn't supported by the compiler torch.jit.script uses, so for cases like that, you would need to use torch.jit.trace.
I made my own model for application and saved this in Keras as .h5 file. and I made GUI Application using PyQt5 and this application uses this model. I'm trying to deploy this application without any information about deep learning model.
I have some questions about this situation.
Can I hide or encrypt my model to prevent its architecture and weight exposure?
If Keras doesn't support encrypting model, are there any other libraries(like PyTorch) that support this function?
I'm looking forward to hearing any advice. Thank you for your answer.
Model encryption is not officially part of either keras nor pytorch.
I think Python is a big problem if you want to hide something. AFAIK it's not really possible to hide your solution well enough using it, I will outline what I would do to "protect" the model (those are quite lengthy, so make sure you really need this protection [or what level of protection exactly]).
Provided Python solutions
There exists PySyft which handles both PyTorch and Keras but it's used for Secure Multi-Party Computation. As users have access to your Python code (you mentioned PyQT5) and all the sensible data (model in this case) they would be able to recover it quite easily.
Possible solution
If I were you I would go for simple password-protected archive (AES or .zip). For the first case I've found this post and related TFSecured repository, which does AES encryption of tensorflow model via Python and allows you to load saved encrypted protobuf model file in C++ (which should be your way to go, reasons below).
Leave Python alone
Is you want to seriously secure your model (not some kind of mere obfuscation) you shouldn't use Python at the user's side at all.
There is no way to compile Python's code, especially the one using heavy ML libraries like Keras, Tensorflow or PyTorch. Although there are programs like PyInstaller it's notoriously hard to make it work with complex dependencies. Even if you do, users will still be able to get to the code albeit it might be a little harder (PyInstaller just bundles Python, your dependencies and app as a single archive which is later unzipped).
You could further obfuscate the code using pyarmor or a-like but it's all quite easily reversible if someone's determined.
C++
Whether you go for keras/tensorflow or pytorch you can go lower level and use C++ to load your network.
As it is a compiled language all you have to do is to provide a binary file (if linking statically) or binary file with shared libraries. Inside C++ source code you keep your AES/zip key as shown by blog post about TFSecured:
#include <GraphDefDecryptor.hpp>
........
tensorflow::GraphDef graph;
// Decryption:
const std::string key = "JHEW8F7FE6F8E76W8F687WE6F8W8EF5";
auto status = tfsecured::GraphDefDecryptAES(path, // path to *.pb file (encrypted graph)
graph,
key); // your key
if (!status.ok()) {
std::cout << status.error_message() << std::endl;
return;
}
// Create session :
std::unique_ptr<Session> session(NewSession(options));
status = session->Create(graph);
It would be much harder to reverse engineer compiled C++ code to get to key buried inside. Similar procedure could be done for PyTorch as well via some third party tools/libraries. On the other hand you would have to rewrite your PyQt5 app in C++ with Qt5.
i just feel you need one complied model file where your client shouldn't be aware of the model architecture.
So in that case you can have a look at this
It supports all framework that too with privileges' of using python
Have a look here
I am trying to do research on batch normalization, and had to make some modifications for the pytorch BN code. I dig into the pytorch code and got stuck with torch.nn.functional.batch_norm, which references torch.batch_norm.
The problem is that torch.batch_norm cannot be further found in the torch library. Is there any way I can find the source code of this built-in function and re-implement it? Thanks!
It's there, but it's not defined in Python. They're defined in C++ in the aten/ directories.
For CPU, the implementation (one of them, it depends on whether or not the input is contiguous) is here: https://github.com/pytorch/pytorch/blob/420b37f3c67950ed93cd8aa7a12e673fcfc5567b/aten/src/ATen/native/Normalization.cpp#L61-L126
For CUDA, the implementation is here: https://github.com/pytorch/pytorch/blob/7aae51cdedcbf0df5a7a8bf50a947237ac4b3ee8/aten/src/ATen/native/cudnn/BatchNorm.cpp#L52-L143
I've just found that PyTorch docs expose something that is called Torch Scripts. However, I do not know:
When they should be used?
How they should be used?
What are their benefits?
Torch Script is one of two modes of using the PyTorch just in time compiler, the other being tracing. The benefits are explained in the linked documentation:
Torch Script is a way to create serializable and optimizable models from PyTorch code. Any code written in Torch Script can be saved from your Python process and loaded in a process where there is no Python dependency.
The above quote is actually true both of scripting and tracing. So
You gain the ability to serialize your models and later run them outside of Python, via LibTorch, a C++ native module. This allows you to embed your DL models in various production environments like mobile or IoT. There is an official guide on exporting models to C++ here.
PyTorch can compile your jit-able modules rather than running them as an interpreter, allowing for various optimizations and improving performance, both during training and inference. This is equally helpful for development and production.
Regarding Torch Script specifically, in comparison to tracing, it is a subset of Python, specified in detail here, which, when adhered to, can be compiled by PyTorch. It is more laborious to write Torch Script modules instead of tracing regular nn.Module subclasses, but it allows for some extra features over tracing, most notably flow control like if statements or for loops. Tracing treats such flow control as "constant" - in other words, if you have an if model.training clause in your module and trace it with training=True, it will always behave this way, even if you change the training variable to False later on.
To answer your first question, you need to use jit if you want to deploy your models outside Python and otherwise you should use jit if you want to gain some execution performance at the price of extra development effort (as not every model can be straightforwardly made compliant with jit). In particular, you should use Torch Script if your code cannot be jited with tracing alone because it relies on some features such as if statements. For maximum ergonomy, you probably want to mix the two on a case-by-case basis.
Finally, for how they should be used, please refer to all the documentation and tutorial links.
I'm trying to get some insights into reinforcement learning while using openAI gym as a learning environment. I do this by reading the book Hands-on reinforcement learning with Python. In this book, some code is provided. Often, the code doesn't work, because I have to unwrap it first, as shown in: openai gym env.P, AttributeError 'TimeLimit' object has no attribute 'P'
However, I personally am still interested in the WHY of this unwrapping. Why do you need to unwrap? What does this do exactly? And why isn't it coded like that in the book? Is it outdated software as Giuliov assumed?
Thanks in advance.
Open AI Gym offers many different environments. Each of them with their own set of parameters and methods. Nevertheless they generally are wrapped by a single Class (like an interface on real OOPLs) called Env. This class exposes the common most essential methods of any environment, like step, reset and seed. Having this “interface” class is great, because it allows your code to be environment agnostic. It is also makes things easier if you want to test a single agent on different environments.
However, if you want to access the behind-the.scenes dynamics of a specific environment, then you use the unwrapped property.