I am a beginner to Rust and PyO3, I read the PyO3 User Guide and got some information as shown below:
When Rust calls Python, the memory of Python part is allocated in Python heap and is taken care by the Python garbage collector.
When Python calls rust, the arguments of rust functions can be either rust type or python native type, the former will incur a conversion but it's faster, the later is almost zero-cost.
Rust can access python heap through reference, PyCell is always allocated in the Python heap, Rust doesn't have ownership of it, but Rust can benefit from it's interior mutability pattern.
But I still have the following questions
How is memory passed/copied in multiple data interaction scenarios?
a. When Python calls Rust
i. Python passes some arguments to Rust func
ii. Rust func creates some data and return to python
iii. rust accepts arguments from python, do some calculation and return to Python
b. When Rust calls Python
i. Rust passes some arguments to Python func
ii. Python func creates some data and return to python
iii. Python accepts arguments from Rust, do some calculation and return to Rust
If I create a PyCell with a Rust value which implements Into< PyClassInitializer >, will there be a memory copy?
When Python calls Rust, does the conversion from Python Native Type to Rust Type mean that there is a memory copy from Python heap to rust heap？
Related
Rust code can be debugged using LLDB. The representation of variables used by tools like CodeLLDB, though, is simply a breakdown of the in-memory contents of the variable and does not show any information from the standard Debug trait.
Is there any way using LLDB I can invoke the actual Debug representation of a variable at runtime? This often has a significant amount of useful information which is not clear from a simple memory snapshot.
The "Debug" trait in Rust looks like a pretty close equivalent to the ObjC and Swift object description methods: it provides a to-string method that prints a developer-friendly view of the object. From what I can tell this trait cooperates with the standard formatted printing machinery in Rust.
If that's right, the natural way to give access to the Rust Debug Trait in lldb would be to implement the "Object Description" part of lldb's "Rust LanguageRuntime" and call the print function under the covers. This would be accessed by the po or print-object command in lldb.
Unfortunately, Rust doesn't have a "Rust LanguageRuntime" in lldb or really much of any support currently. The only mentions of Rust in the lldb sources are a recognizer for the Rust mangling scheme and a define that says "other than mangling, pretend Rust is C++". So that isn't a viable option at present.
You could also try calling Rust's print directly in the expression evaluator, but YMMV as calling Rust code in lldb doesn't always work: as it turns out, Rust is not C++...
I'm currently in the progress of learning Rust. I'm mainly using The Rust Programming Language book and this nice reference which relates Rust features/syntax to C++ equivalents.
I'm having a hard time understanding where the core language stops and the standard library starts. I've encountered a lot of operators and/or traits which seems to have a special relationship with the compiler. For example, Rust has a trait (which from what I understand is like an interface) called Deref which let's a type implementing it be de-referenced using the * operator:
fn main() {
let x = 5;
let y = Box::new(x);
assert_eq!(5, x);
assert_eq!(5, *y);
}
Another example is the ? operator, which seems to depend on the Result and Option types.
Can code that uses those operators can be compiled without the standard library? And if not, what parts of the Rust language are depending on the standard library? Is it even possible to compile any Rust code without it?
The Rust standard library is in fact separated into three distinct crates:
core, which is the glue between the language and the standard library. All types, traits and functions required by the language are found in this crate. This includes operator traits (found in core::ops), the Future trait (used by async fn), and compiler intrinsics. The core crate does not have any dependencies, so you can always use it.
alloc, which contains types and traits related to or requiring dynamic memory allocation. This includes dynamically allocated types such as Box<T>, Vec<T> and String.
std, which contains the whole standard library, including things from core and alloc but also things with further requirements, such as file system access, networking, etc.
If your environment does not provide the functionality required by the std crate, you can choose to compile without it. If your environment also does not provide dynamic memory allocation, you can choose to compile without the alloc crate as well. This option is useful for targets such as embedded systems or writing operating systems, where you usually won't have all of the things that the standard library usually requires.
You can use the #![no_std] attribute in the root of your crate to tell the compiler to compile without the standard library (only core). Many libraries also usually support "no-std" compilation (e.g. base64 and futures), where functionality may be restricted but it will work when compiling without the std crate.
DISCLAIMER: This is likely not the answer you're looking for. Consider reading the other answers about no_std, if you're trying to solve a problem. I suggest you only read on, if you're interested in trivia about the inner workings of Rust.
If you really want full control over the environment you use, it is possible to use Rust without the core library using the no_core attribute.
If you decide to do so, you will run into some problems, because the compiler is integrated with some items defined in core.
This integration works by applying the #[lang = "..."] attribute to those items, making them so called "lang items".
If you use no_core, you'll have to define your own lang items for the parts of the language you'll actually use.
For more information I suggest the following blog posts, which go into more detail on the topic of lang items and no_core:
Rust Tidbits: What Is a Lang Item?
Oxidizing the technical interview
So yes, in theory it is possible to run Rust code without any sort of standard library and supplied types, but only if you then supply the required types yourself.
Also this is not stable and will likely never be stabilized and it is generally not a recommended way of using Rust.
When you're not using std, you rely on core, which is a subset of the std library which is always (?) available. This is what's called a no_std environment, which is commonly used for some types of "embedded" programming. You can find more about no_std in the Rust Embedded book, including some guidance on how to get started with no_std programming.
I've been searching, and while this seems to be a much-wanted feature, all search results seem to be at least one year old.
What is the current state of this? Is there a good solution to evaluating arbitrary Rust code at runtime (like Haskell's hint)?
Maybe something can be done with Miri?
Miri (short for MIR Interpreter) is the de-facto interpreter of Rust code. It is what powers the compile-time function evaluation inside of rustc, the Rust compiler, but Miri is more featureful than what is currently used by compiler.
For experimentation purposes, Miri is also available in the Rust playground. It can be used to evaluate a particular run of a program, detecting if certain types of undefined behavior exist.
Miri does not provide a Rust REPL, but it may be part of creating such a tool.
I want to create a function (for a library) which will output a struct for any CSV which contains all the columns and their data. This means that the column names (unless explicitly provided by the user) will not be known until runtime.
Is it possible to create a struct definition at runtime or mutate an existing struct? If so, how?
For example, how can I mutate the following struct structure:
struct Point {
x: String,
y: String,
}
To the following (in memory only):
struct Point {
x: String,
y: String,
z: String,
}
This behaviour is possible in languages such as Python, but I am not sure if it is possible in compiled languages such as Rust.
No, it is not possible.
Simplified, at compile time, the layout (ordering, offset, padding, etc.) of every struct is computed, allowing the size of the struct to be known. When the code is generated, all of this high-level information is thrown away and the machine code knows to jump X bytes in to access field foo.
None of this machinery to convert source code to machine code is present in a Rust executable. If it was, every Rust executable would probably gain several hundred megabytes (the current Rust toolchain weighs in at 300+MB).
Other languages work around this by having a runtime or interpreter that is shared. You cannot take a Python source file and run it without first installing a shared Python interpreter, for example.
Additionally, Rust is a statically typed language. When you have a value, you know exactly what fields and methods are available. There is no way to do this with dynamically-generated structs — there's no way to tell if a field/method actually exists when you write the code that attempts to use it.
As pointed out in the comments, dynamic data needs a dynamic data structure, such as a HashMap.
I'm working with raw pointers in Rust and I've got a couple of lines which are really hard to read because I can't find an operator in Rust that has a similar function to -> in C++. Does one exist?
Here's a snippet of my code in its currently hard to read state:
(*(*block).next().unwrap()).split(ExcHeader(30));
No. Raw pointers in Rust must be dereferenced manually, because doing so in an unsafe operation.
If you're repeatedly dereferencing the same pointer over and over again, you can just convert it into a reference (let block = &*block;). Sadly, there is no postfix dereference operator, so there's not much you can do about the result of next being a pointer.