I am writing a library which will leverage LLVM (via inkwell) to JIT compile some functions. The functions need to be able to call back into some rust functions in my code.
I have it working, but my unit tests don't work, because it seems that the callback functions are optimized away when building the tests. These callback functions are not called by the Rust code itself - only by the dynamically generated JIT functions - so I guess the linker thinks they are unused and drops them.
If I call them from the rust code, within unit test, then they are not removed - but that is not a desirable workaround. Also note that the functions are not removed when the package is built as a library, only when building the tests.
Below is an MVCE.
// lib.rs
use inkwell::{OptimizationLevel, context::Context};
use inkwell::execution_engine::JitFunction;
#[no_mangle]
pub extern "C" fn my_callback(x:i64) {
println!("Called Back x={}", x);
}
type FuncType = unsafe extern "C" fn();
pub fn compile_fn() {
let context = Context::create();
let module = context.create_module("test");
let execution_engine = module.create_jit_execution_engine(OptimizationLevel::None).unwrap();
let builder = context.create_builder();
let func_type = context.void_type().fn_type(&[], false);
let function = module.add_function("test", func_type, None);
let basic_block = context.append_basic_block(function, "entry");
builder.position_at_end(basic_block);
let cb_fn_type = context.void_type().fn_type(&[context.i64_type().into()], false);
let cb_fn = module.add_function("my_callback", cb_fn_type, None);
let x = context.i64_type().const_int(42, false);
builder.build_call(cb_fn, &[x.into()], "callback");
builder.build_return(None);
function.print_to_stderr();
let jit_func:JitFunction<FuncType> = unsafe { execution_engine.get_function("test").unwrap() };
unsafe { jit_func.call() };
}
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
// If I uncomment this line, it works, otherwise it segfaults
//super::my_callback(1);
super::compile_fn();
}
}
# Cargo.toml
[package]
name = "so_example_lib"
version = "0.1.0"
authors = ["harmic <harmic#no-reply.com>"]
edition = "2018"
[dependencies]
inkwell = { git = "https://github.com/TheDan64/inkwell", branch = "master", features = ["llvm7-0"] }
// build.rs
fn main() {
println!("cargo:rustc-link-search=/usr/lib64/llvm7.0/lib");
println!("cargo:rustc-link-lib=LLVM-7");
}
You can tell the function has been removed by running nm on the resulting test binary. When you run the test, it segfaults. If you run it in gdb, you can see it is trying to call a function at 0x0000000000000000.
#0 0x0000000000000000 in ?? ()
#1 0x00007ffff7ff201f in test ()
How can I instruct Rust not to drop these functions?
Related
I'm trying to replicate an example from reqwest documentation. Here is the example:
let body = reqwest::get("https://www.rust-lang.org")?
.text()?;
After I have added the reqwest = "0.10.10" line in the file Cargo.toml, I add the following code in my main.rs file:
extern crate reqwest;
fn main() {
let body = reqwest::get("https://www.rust-lang.org")?.text()?;
println!("body = {:?}", body);
}
This code don't compile and returns the following error:
cannot use the `?` operator in a function that returns `()`
I'm a bit surprised with this behaviour since my code is almost ipsis litteris the documentation code.
I think that ? works only on Response objects, so I checked which object the get method is returning:
extern crate reqwest;
fn print_type_of<T>(_: &T) {
println!("{}", std::any::type_name::<T>())
}
fn main() {
let body = reqwest::get("https://www.rust-lang.org");
print_type_of(&body);
}
output:
core::future::from_generator::GenFuture<reqwest::get<&str>::{{closure}}
I mean, why I'm not getting an Response object instead, just like the documentation?
There are two separate issues here tripping you up.
The documentation that you've linked to is for reqwest version 0.9.18, but you've installed version 0.10.10. If you look at the docs for 0.10.10, you will see that the snippet you use is
let body = reqwest::get("https://www.rust-lang.org")
.await?
.text()
.await?;
println!("body = {:?}", body);
or more likely in your case, since you have not mentioned having an async runtime, the blocking docs
let body = reqwest::blocking::get("https://www.rust-lang.org")?
.text()?;
println!("body = {:?}", body);
Note that when you try to use this one, you will still get that
cannot use the ? operator in a function that returns ()
and you will need to set a return type on main like
fn main() -> Result<(), reqwest::Error> {
See Why do try!() and ? not compile when used in a function that doesn't return Option or Result? for more info.
You might want to read this section of the Rust docs: https://doc.rust-lang.org/edition-guide/rust-2018/error-handling-and-panics/index.html
In short, ? simply passes the error from a Result further up the call stack via an implicit return. As such the function in which you use the ? operator must return the same type as the function on which ? was used.
Each response is wrapped in a Result type which you can unwrap.
use reqwest;
#[tokio::main]
async fn main() {
let response = reqwest::get("https://www.rust-lang.org")
.await
.unwrap()
.text()
.await;
println!("{:?}", response);
}
Cargo.toml:
[package]
name = "request"
version = "0.1.0"
edition = "2021"
reqwest = "0.10.10"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
reqwest = { version = "0.11", features = ["json"] } # reqwest with JSON parsing support
futures = "0.3" # for our async / await blocks
tokio = { version = "1.12.0", features = ["full"] } # for async runtime
I'm making a mini web browser with gtk. This code works fine, but I want to avoid all the cloning that is using.
I can't just use the web_view object multiple times because of ownership errors on the closures.
extern crate gio;
extern crate gtk;
extern crate webkit2gtk;
use gio::prelude::*;
use gtk::prelude::*;
use gtk::{Application, ApplicationWindow, Button, Entry};
use webkit2gtk::{WebView, WebViewExt};
fn main() {
let application =
Application::new(Some("com.github.gtk-rs.examples.basic"), Default::default())
.expect("failed to initialize GTK application");
application.connect_activate(|app| {
let window = ApplicationWindow::new(app);
window.set_title("Web Browser");
window.set_default_size(1024, 768);
let button_back = Button::new_with_label("Back");
let button_next = Button::new_with_label("Next");
let button_reload = Button::new_with_label("Reload");
let url_entry = Entry::new();
url_entry.set_text("https://www.google.com");
let button_go = Button::new_with_label("Go");
let web_view = WebView::new();
web_view.load_uri("https://www.google.com");
let clone1: WebView = web_view.clone();
let clone2: WebView = web_view.clone();
let clone3: WebView = web_view.clone();
let clone4: WebView = web_view.clone();
button_back.connect_clicked(move |_| {
clone1.go_back();
});
button_next.connect_clicked(move |_| {
clone2.go_forward();
});
button_reload.connect_clicked(move |_| {
clone3.reload();
});
button_go.connect_clicked(move |_| {
clone4.load_uri(&url_entry.get_buffer().get_text());
});
window.show_all();
});
application.run(&[]);
}
These are my dependency versions:
[dependencies]
gtk = "0.8.1"
gio = "0.8.1"
webkit2gtk = "0.9.2"
The signature for the method connect_clicked of the trait gtk::ButtonExt is
fn connect_clicked<F: Fn(&Self) + 'static>(&self, f: F) -> SignalHandlerId
No, there isn't.
'static on the callback definition forbids it from referencing any temporary variables, and forces it to move and own any values it uses. That's deliberate, because the callback will outlive the function that creates it.
gtk's WebView behaves like Rc, so the clone is very cheap, and doesn't make a copy of anything. It only performs bookkeeping to help the callbacks agree which one of them will be the last to be destroyed, and therefore the one to free the underlying WebView data.
There's a clone! macro in glib that helps making these clones without having to name extra variables by hand.
There are several ways of doing something in my crate, some result in fast execution, some in low binary size, some have other advantages, so I provide the user interfaces to all of them. Unused functions will be optimized away by the compiler. Internal functions in my crate have to use these interfaces as well, and I would like them to respect the user choice at compile time.
There are conditional compilation attributes like target_os, which store a value like linux or windows. How can I create such an attribute, for example prefer_method, so I and the user can use it somewhat like in the following code snippets?
My crate:
#[cfg(not(any(
not(prefer_method),
prefer_method = "fast",
prefer_method = "small"
)))]
compile_error("invalid `prefer_method` value");
pub fn bla() {
#[cfg(prefer_method = "fast")]
foo_fast();
#[cfg(prefer_method = "small")]
foo_small();
#[cfg(not(prefer_method))]
foo_default();
}
pub fn foo_fast() {
// Fast execution.
}
pub fn foo_small() {
// Small binary file.
}
pub fn foo_default() {
// Medium size, medium fast.
}
The user crate:
#[prefer_method = "small"]
extern crate my_crate;
fn f() {
// Uses the `foo_small` function, the other `foo_*` functions will not end up in the binary.
my_crate::bla();
// But the user can also call any function, which of course will also end up in the binary.
my_crate::foo_default();
}
I know there are --cfg attributes, but AFAIK these only represent boolean flags, not enumeration values, which allow setting multiple flags when only one enumeration value is valid.
Firstly, the --cfg flag supports key-value pairs using the syntax --cfg 'prefer_method="fast"'. This will allow you to write code like:
#[cfg(prefer_method = "fast")]
fn foo_fast() { }
You can also set these cfg options from a build script. For example:
// build.rs
fn main() {
println!("cargo:rustc-cfg=prefer_method=\"method_a\"");
}
// src/main.rs
#[cfg(prefer_method = "method_a")]
fn main() {
println!("It's A");
}
#[cfg(prefer_method = "method_b")]
fn main() {
println!("It's B");
}
#[cfg(not(any(prefer_method = "method_a", prefer_method = "method_b")))]
fn main() {
println!("No preferred method");
}
The above code will result in an executable that prints "It's A".
There's no syntax like the one you suggest to specify cfg settings. The best thing to expose these options to your crates' users is through Cargo features.
For example:
# Library Cargo.toml
# ...
[features]
method_a = []
method_b = []
// build.rs
fn main() {
// prefer method A if both method A and B are selected
if cfg!(feature = "method_a") {
println!("cargo:rustc-cfg=prefer_method=\"method_a\"");
} else if cfg!(feature = "method_b") {
println!("cargo:rustc-cfg=prefer_method=\"method_b\"");
}
}
# User Cargo.toml
# ...
[dependencies.my_crate]
version = "..."
features = ["method_a"]
However, in this case, I'd recommend just using the Cargo features directly in your code (i.e. #[cfg(feature = "fast")]) rather than adding the build script since there's a one-to-one correspondence between the cargo feature and the rustc-cfg being added.
In Rust, it's possible to perform conditional compilation as follows.
#[cfg(rust_version = "1.10")]
fn my_func() {}
Is it possible to define variables for cfg to check within the same source file?
For example:
// leave off, just a quick test to enable when troubleshooting.
#define use_counter 1 // C style (not valid Rust)
#[cfg(use_counter == "1")]
static mut fn_counter: usize = 0;
fn my_func() {
#[cfg(use_counter = "1")]
unsafe { fn_counter += 1; }
}
main () {
// code calling 'my_func'
// print how many times the function is called.
#[cfg(use_counter = "1")]
unsafe { println!("Function count {}", fn_counter); }
}
I'm not asking how to write a function counter, it's just an example of optionally inserting logic into a source file.
Yes, this is written as #[cfg(use_counter)]. Such flags can be enabled or disabled on the command line at compile time and are not exposed in Cargo.toml.
fn main() {
#[cfg(use_counter)]
println!("counter is enabled");
}
Using Cargo, run with the feature disabled:
$ cargo run
Using Cargo, run with the feature enabled:
$ RUSTFLAGS="--cfg use_counter" cargo run
Compile directly with the feature disabled:
$ rustc src/main.rs
Compile with the feature enabled:
$ rustc src/main.rs --cfg use_counter
I'm trying to do some dynamic library loading in Rust. I'm getting a segmentation fault when passing a large Vec from a dynamically loaded library function. It's a basic function that creates a Vec<i32> of a specified size. If the Vec gets much bigger than 8MB, the program hits a segfault on OSX. I haven't had the same problem when running on linux, can anyone look at this and tell me if I'm doing something wrong here? I'm running this with:
$ cargo build --release
$ ./target/release/linkfoo
8281
[1] 84253 segmentation fault ./target/release/linkfoo
Cargo.toml
[package]
name = "linkfoo"
version = "0.1.0"
authors = ["Nate Mara <nathan.mara#kroger.com>"]
[dependencies]
libloading = "0.3.0"
[lib]
name = "foo"
crate-type = ["dylib"]
main.rs
extern crate libloading as lib;
fn main() {
let dylib = lib::Library::new("./target/release/libfoo.dylib").expect("Failed to load library");
let func = unsafe {
let wrapped_func: lib::Symbol<fn(&[i32]) -> Vec<i32>> = dylib.get(b"alloc")
.expect("Failed to load function");
wrapped_func.into_raw()
};
let args = vec![8182];
println!("{}", func(&args).len());
}
lib.rs
#[no_mangle]
pub fn alloc(args: &[i32]) -> Vec<i32> {
let size = args[0] as usize;
let mut mat = Vec::with_capacity(size);
for _ in 0..size {
mat.push(0);
}
mat
}
Rust uses the system allocator for dynamic libraries, and jemalloc for all other code. This difference in loaders was causing the segfault, and I was able to fix it by adding this to the top of main.rs:
#![feature(alloc_system)]
extern crate alloc_system;