Rust linker errors: using "-Wl,--as-needed" when linking system libraries - rust

I'm trying to write an application primarily in rust that uses a gtk-based frontend written in C++. I've gotten pretty far in getting the build setup in Cargo, but it's failing in the linking stage.
I've created a minimal reproducible example.
// build.rs
extern crate cc;
extern crate pkg_config;
fn main() {
let gtk = pkg_config::probe_library("gtk+-3.0").unwrap();
cc::Build::new()
.cpp(true)
.file("src/gui.cc")
.includes(gtk.include_paths)
.compile("gui");
}
// src/gui.cc
// This is essentially the first example from https://docs.gtk.org/gtk3/getting_started.html
#include <gtk/gtk.h>
extern "C" {
int run_gui();
}
static void activate(GtkApplication* app, gpointer) {
GtkWidget* window = gtk_application_window_new (app);
gtk_window_set_title (GTK_WINDOW (window), "Window");
gtk_window_set_default_size (GTK_WINDOW (window), 200, 200);
gtk_widget_show_all (window);
}
int run_gui() {
GtkApplication* app = gtk_application_new ("org.gtk.example", G_APPLICATION_DEFAULT_FLAGS);
g_signal_connect (app, "activate", G_CALLBACK (activate), nullptr);
int status = g_application_run (G_APPLICATION (app), 0, nullptr);
g_object_unref (app);
return status;
}
// src/main.rs
extern "C" {
fn run_gui() -> core::ffi::c_int;
}
fn main() {
unsafe { run_gui(); }
}
When I cargo build the package, compiling the c++ and rust files seems to work fine, but then I get undefined reference errors on the link stage.
/usr/bin/ld: /home/pete/workspaces/csvtk/so-minimal-gtk/target/debug/build/so-minimal-gtk-75c7a9a61d32d2e6/out/libgui.a(gui.o): in function `activate(_GtkApplication*, void*)':
/home/pete/workspaces/csvtk/so-minimal-gtk/src/gui.cc:9: undefined reference to `gtk_application_window_new'
/usr/bin/ld: /home/pete/workspaces/csvtk/so-minimal-gtk/src/gui.cc:10: undefined reference to `gtk_window_get_type'
...
The link line, which is quite long, is also logged before that error. It includes all the required gtk libraries, -lgtk-3, -lcairo, etc. When I run it outside of cargo, it fails with the same errors, but, when I remove the -Wl,--as-needed flag from the link line, it links correctly and the program runs fine.
Why did rustc add -Wl,--as-needed? Is there a reason those gtk symbols aren't "needed" from the perspective of the linker? Any tips on how to fix this problem elegantly?

In general, using --as-needed as a flag to the linker is extremely helpful because it prevents your binary from being linked to libraries it doesn't use directly. As an example of a reason why this is beneficial, if you depend on libfoo 1.0, which depends on libbar 1.0, and later on libfoo 1.1 updates to libbar 2.0, then as long as you don't use libbar directly, your code will continue to work with --as-needed, but won't work without it (unless libbar has symbol versioning).
The kind of impact is visible here when you use objdump -x on the binary after making the change I suggest below:
NEEDED libgtk-3.so.0
NEEDED libgio-2.0.so.0
NEEDED libgobject-2.0.so.0
NEEDED libgcc_s.so.1
NEEDED libc.so.6
NEEDED ld-linux-x86-64.so.2
ldd lists 70 libraries linked into your program, and you're only directly requiring to 6, including the dynamic linker.
You can usually rely on other crates to specify their link dependencies correctly. However, you need to be careful because when you're creating your own dependency on a library using C or C++, you have to specify the libraries to link with yourself. This can be seen if you use cargo build -v, where you can notice that your gui library is linked after the dependencies of libgtk-3:
Running `rustc --crate-name test_repo --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=c60f7e6ab8348a08 -C extra-filename=-c60f7e6ab8348a08 --out-dir /tmp/user/1000/test-repo/target/debug/deps -C incremental=/tmp/user/1000/test-repo/target/debug/incremental -L dependency=/tmp/user/1000/test-repo/target/debug/deps -L native=/usr/lib/x86_64-linux-gnu -L native=/tmp/user/1000/test-repo/target/debug/build/test-repo-c89dae0927f2103a/out -l gtk-3 -l gdk-3 -l z -l pangocairo-1.0 -l pango-1.0 -l harfbuzz -l atk-1.0 -l cairo-gobject -l cairo -l gdk_pixbuf-2.0 -l gio-2.0 -l gobject-2.0 -l glib-2.0 -l static=gui -l stdc++`
So you'd want to do something more like this in your build.rs:
// build.rs
extern crate cc;
extern crate pkg_config;
fn main() {
let gtk = pkg_config::probe_library("gtk+-3.0").unwrap();
cc::Build::new()
.cpp(true)
.file("src/gui.cc")
.includes(gtk.include_paths)
.compile("gui");
for path in gtk.link_paths {
println!("cargo:rustc-link-search={}", path.display());
}
for lib in gtk.libs {
println!("cargo:rustc-link-lib={}", lib);
}
}

Related

I'm unable to run rust winit application on Alpine (Wayland)

I'm following this tutorial to create a winit window with Rust on Alpine Linux.
When I started the demo application described in the tutorial using cargo run it not compile. But after installing build-base cmake musl-dev libpng-dev freetype-dev fontconfig-dev it was compiling. However, it fails to run, and it throws the following error:
thread 'main' panicked at 'Failed to initialize any backend! Wayland status: NoWaylandLib X11 status: LibraryOpenError(OpenError { kind: Library, detail: "opening library failed (Dynamic loading not supported); opening library failed (Dynamic loading not supported)" })', /home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/winit-0.27.5/src/platform_impl/linux/mod.rs:719:9
It seems to me that it is unable to use a dynamic library because it's MUSL. I don't know how to fix this. (It does compile, but does not run!)
System info:
OS: Linux 5.15.80-0-lts, Alpine v3.17 x86_64
WM: sway (wayland)
Rust: Toolchain stable-x86_64-unknown-linux-musl, rustc 1.65.0
Shell: ash
What I've tried:
I have tried to install some Xorg and Gui dev libraries like libxtst-dev libxext-dev libxrender-dev freetype-dev fontconfig-dev libxslt glib-dev musl-dev libxcursor-dev libxi-dev libx11-dev glu-dev glew-dev mesa-dev libxcb-dev libxkbcommon-dev libx11-dev xproto lbxft-dev libxext-dev libxcb-dev libxkbcommon-dev just to be sure. That did not work
I tried to run the application using RUSTFLAGS="-C target-feature=-crt-static" but that did not work
I've tried to run the application using env var RUSTFLAGS="-C target-feature=+crt-static" to compile it static instead of linked (I don't know if this was correct) but that throws an exception
I've tried to run the application using env var RUSTFLAGS="-C target-feature=-crt-static". That did compile and run, but no window shows up. And I'm not sure if that is the way to go. Also that bugs cargo to recompile all dependencies every time I've tried to change something in the code.
I've found no documentation or anything on the internet that could help me out, telling how to run this on Alpine or musl.
I was expecting that with the correct dependencies the program compiles and a window shows up.
Can anybody help me out?
This is my cargo.toml:
[package]
name = "gpu-programming"
version = "0.1.0"
edition = "2021"
[dependencies]
wgpu = ">=0.14"
winit = ">=0.27"
env_logger = ">=0.10"
log = ">=0.4"
And this is the code I'm talking about (main.rs)
use winit::{
event::{ElementState, Event, KeyboardInput, VirtualKeyCode, WindowEvent},
event_loop::{ControlFlow, EventLoop},
window::WindowBuilder,
};
pub fn main() {
env_logger::init(); // Make sure WGPU errors are printed to console. Else it will fail silently!
// Create event loop and window
let event_loop = EventLoop::new();
let window = WindowBuilder::new()
.build(&event_loop)
.expect("Failed to create window");
println!("Window created! {:?}", window.id());
window.set_visible(true);
println!("Visible {:?}", window.is_visible());
println!("Monitor: {:?}", window.current_monitor());
event_loop.run(move |event, _, control_flow| match event {
Event::WindowEvent {
ref event,
window_id,
} if window_id == window.id() => match event {
WindowEvent::CloseRequested
| WindowEvent::KeyboardInput {
input:
KeyboardInput {
state: ElementState::Pressed,
virtual_keycode: Some(VirtualKeyCode::Escape),
..
},
..
} => *control_flow = ControlFlow::Exit,
_ => {}
},
_ => {}
});
}

PyO3 linker error when using "extension-module" feature

When I try to build very simple a PyO3 module, the compilation stops at the link stage and returns a very long error message. I compile this code on Linux.
The Rust project looks like this:
├ Cargo.toml
└ src/
└ lib.rs
Cargo.toml:
[package]
name = "dytest"
version = "0.1.0"
edition = "2021"
[lib]
name = "dytest"
crate-type = ["cdylib"]
[dependencies]
pyo3 = { version = "0.16", features = ["extension-module"] }
lib.rs:
use pyo3::prelude::*;
#[test]
fn test_print() {
pyo3::prepare_freethreaded_python();
Python::with_gil(|py| py.run( "print('Hello World')", None, None ) );
}
When I now run cargo test, I get a linker error:
error: linking with `cc` failed: exit status: 1
|
= note: "cc" "-m64" [...many, many paths...]
= note: /usr/bin/ld: /home/user/dytest/target/debug/deps/libpyo3-c98e45c2daa36b66.rlib(pyo3-c98e45c2daa36b66.pyo3.b04632a8-cgu.2.rcgu.o): in function `pyo3_ffi::object::Py_DECREF':
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-ffi-0.16.2/src/object.rs:407: undefined reference to `_Py_Dealloc'
/usr/bin/ld: /home/user/dytest/target/debug/deps/libpyo3-c98e45c2daa36b66.rlib(pyo3-c98e45c2daa36b66.pyo3.b04632a8-cgu.2.rcgu.o): in function `pyo3_ffi::object::Py_None':
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-ffi-0.16.2/src/object.rs:472: undefined reference to `_Py_NoneStruct'
/usr/bin/ld: /home/user/dytest/target/debug/deps/libpyo3-c98e45c2daa36b66.rlib(pyo3-c98e45c2daa36b66.pyo3.b04632a8-cgu.2.rcgu.o): in function `pyo3::err::PyErr::take':
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-0.16.2/src/err/mod.rs:264: undefined reference to `PyErr_Fetch'
[...many, many error messages...]
/usr/bin/ld: /home/user/dytest/target/debug/deps/libpyo3-c98e45c2daa36b66.rlib(pyo3-c98e45c2daa36b66.pyo3.b04632a8-cgu.15.rcgu.o): in function `pyo3::types::string::PyString::to_string_lossy':
/home/user/.cargo/registry/src/github.com-1ecc6299db9ec823/pyo3-0.16.2/src/types/string.rs:199: undefined reference to `PyUnicode_AsEncodedString'
collect2: error: ld returned 1 exit status
= help: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
= note: use the `-l` flag to specify native libraries to link
= note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)
error: could not compile `dytest` due to previous error
If I replace
pyo3 = { version = "0.16", features = ["extension-module"] }
with
pyo3 = { version = "0.16" }
everything works, but the PyO3 User Guide explicitly states this feature.
Trying this code on a Windows machine, works.
I assume I am missing some kind of dependency, but I can't figure out, what is missing.
It looks like this is a known PyO3 issue. It is mentioned in their FAQ:
Cargo.toml:
[dependencies.pyo3]
version = "0.16.3"
[features]
extension-module = ["pyo3/extension-module"]
default = ["extension-module"]
cargo test fails with linking errors when the extension-module feature is activated. A workaround is to make the extension-module feature optional and running the tests with
cargo test --no-default-features.

Undefined symbols in hello world C++ kernel module

I have added C++ support to the Linux kernel version 4.14.41, compiled it and booted using the kernel successfully. I can check the correctness of the C++ module by inserting a LKM. This is the module that I am trying to load:
#include<c++/begin_include.h>
#include<linux/module.h>
#include<linux/kernel.h>
#include<c++/end_include.h>
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("LKM in c++");
MODULE_AUTHOR("MOOL");
class hello
{
public:
hello();
void hi();
};
void hello::hi()
{
printk("Hello world!! \n");
}
hello::hello()
{
printk("Constructor is being called \n");
}
extern "C"
{
static int __init test_classes_init()
{
class hello obj;
obj.hi();
printk("Module inserted:\n");
return 0;
}
static void __exit test_classes_fini()
{
printk("Module removed:\n");
}
module_init(test_classes_init);
module_exit(test_classes_fini);
}
The Makefile:
obj-m = helloworld.o
KVERSION=$(shell uname -r)
all:
make -C /lib/modules/$(KVERSION)/build M=$(PWD) modules
clean:
make -C /lib/modules/$(KVERSION)/build M=$(PWD) clean
When I enter the make command, the helloworld.ko is generated with the warnings
WARNING: "begin_fini" [/home/jai/Downloads/helloworld/helloworld.ko] undefined !
WARNING: "end_init" [/home/jai/Downloads/helloworld/helloworld.ko] undefined !
WARNING: "begin_init" [/home/jai/Downloads/helloworld/helloworld.ko] undefined !
But when I try to insert it using insmod helloworld.ko, the undefined symbol error occurs.
dmesg:
loading out-of-tree module taints kernel
Unknown symbol begin_init (err 0)
Unknown symbol end_init (err 0)
Unknown symbol begin_fini (err 0)
These begin_init, end_init and begin_fini are defined in lib/gcc/crtstuff.c (which was ported into the kernel). These functions are declared as extern in both crtstuff.c and linux/module.h. This module.h is being included in the helloworld module above, but still, those symbols become undefined. So, How can I make those functions defined?
Your kernel C++ implementation is incomplete. You will have to implement global constructor and destructor support (processing of .init_array and .fini_array sections properly), or stop using these C++ features in the source code. This needs cooperation from the kernel module loader. Changes to the startup code will not work because the startup code is not linked into kernel modules.

Running LLVM passes on Windows 10 gives no output in terminal?

I've the sample pass code from LLVM.org:
#include "llvm/Pass.h"
#include "llvm/IR/Function.h"
#include "llvm/Support/raw_ostream.h"
using namespace llvm;
namespace {
struct Hello : public FunctionPass {
static char ID;
Hello() : FunctionPass(ID) {}
bool runOnFunction(Function &F) override {
errs() << "Hello: ";
errs().write_escaped(F.getName()) << '\n';
return false;
}
}; // end of struct Hello
} // end of anonymous namespace
char Hello::ID = 0;
static RegisterPass<Hello> X("hello", "Hello World Pass",
false /* Only looks at CFG */,
false /* Analysis Pass */);
The project builds fine and creates a SkeletonPass.dll.
When I execute the command:
C:\Users\nlykkei\Projects\llvm-pass-tutorial\build>opt -load skeleton\Debug\SkeletonPass.dll -hello foo.bc
opt: Unknown command line argument '-hello'. Try: 'opt -help'
opt: Did you mean '-help'?
opt doesn't recognize -hello option, even thus everything works fine on Ubuntu 16.04.
In addition, if I execute:
clang -Xclang -load -Xclang skeleton\Debug\SkeletonPass.dll foo.bc
nothing is printed out on Visual Studio terminal (Native Tools Command Prompt x86). On Linux, the function names are printed nicely for the same bitcode file.
What can be the reason for my experience? I do exactly the same on Windows 10 as I do on Ubuntu, but very different results.
Plugins are special beasts on Windows, because the latter does not support proper dynamic linking, so, your pass simply does not register itself in the PassRegistry. So you'd either need to compile all the LLVM into .dll or link your pass statically into opt / clang.

Create a shared lib using another shared lib

I have a shared library "libwiston.so". I am using this to create another shared library called "libAnimation.so", which will be used by another project. Now, the second library "libAnimation.so" can't be used in test code correctly. So I doubt that the creation of the second lib "libAnimation.so" is right. The gcc command to create this lib is
g++ -g -shared -Wl,-soname,libwiston.so -o libAnimation.so $(objs) -lc".
Has someone come across this problem?
That looks like a weird link line - you are creating libAnimation.so, but its internal DT_SONAME name is libwiston.so.
I don't think that what you wanted to do. Don't you want to link libAnimation.so against libwiston.so (-lwiston)?
g++ -g -shared -o libAnimation.so $(objs) -lc -lwiston
I think it would be easier to wrap your build in automake/autoconf and rely on libtool to get the shared library creation correct.
I'll do a humble review on the process of creating shared libraries.
Let's begin by creating libwiston.so. First we implement the function we would like to export and then define it on a header so other programs knows how to call it.
/* file libwiston.cpp
* Implementation of hello_wiston(), called by libAnimation.so
*/
#include "libwiston.h"
#include <iostream>
int hello_wiston(std::string& msg)
{
std::cout << msg << std::endl;
return 0;
}
and:
/* file libwiston.h
* Exports hello_wiston() as a C symbol.
*/
#include <string>
extern "C" {
int hello_wiston(std::string& msg);
};
This code can be compiled with: g++ libwiston.cpp -o libwiston.so -shared
Now we implement the second shared library, named libAnimation.so that calls the function exported by the first library.
/* file libAnimation.cpp
* Implementation of call_wiston().
* This function is a simple wrapper around hello_wiston().
*/
#include "libAnimation.h"
#include "libwiston.h"
#include <iostream>
int call_wiston(std::string& param)
{
hello_wiston(param);
return 0;
}
and header:
/* file libAnimation.h
* Exports call_wiston() as a C symbol.
*/
#include <string>
extern "C" {
int call_wiston(std::string& param);
};
Compile it with: g++ libAnimation.cpp -o libAnimation.so -shared -L. -lwiston
Finally, we create a small application to test libAnimation.
/* file demo.cpp
* Implementation of the test application.
*/
#include "libAnimation.h"
int main()
{
std::string msg = "hello stackoverflow!";
call_wiston(msg);
}
And compile it with: g++ demo.cpp -o demo -L. -lAnimation
There's an interesting tool named nm that you can use to list the symbols exported by your shared library. Using these examples, you could execute the following commands to check for the symbols:
nm libAnimation.so | grep call_wiston
outputs:
00000634 t _GLOBAL__I_call_wiston
000005dc T call_wiston
and also:
nm libwiston.so | grep hello_wiston
outputs:
0000076c t _GLOBAL__I_hello_wiston
000006fc T hello_wiston

Resources