I'm trying to declare a global variable, who's type is a struct with a function pointer and a char pointer element { i64 ()*, i8* }, and then set the fields to null during main, but I'm getting an assertion error using a debug version of LLVM.
/media/work/contrib/llvm-project/llvm/lib/IR/ConstantsContext.h:745: void llvm::ConstantUniqueMap<ConstantClass>::remove(ConstantClass*) [with ConstantClass = llvm::ConstantExpr]: Assertion `I != Map.end() && "Constant not found in constant table!"' failed.
I believe this problem is causing another issue during optimization when compiling something a bit more complicated. The error itself occurs when disposing of the module at the end. A distilled runnable example in rust is:
use std::ffi::CString;
extern crate llvm_sys;
pub use self::llvm_sys::prelude::{ LLVMValueRef };
use self::llvm_sys::*;
use self::llvm_sys::prelude::*;
use self::llvm_sys::core::*;
use self::llvm_sys::target::*;
use self::llvm_sys::target_machine::*;
use self::llvm_sys::transforms::pass_manager_builder::*;
fn main() {
unsafe {
let context = LLVMContextCreate();
let module = LLVMModuleCreateWithName(cstr("module"));
build(module, context);
println!("{}", emit_module(module));
LLVMDisposeModule(module);
LLVMContextDispose(context);
}
}
pub fn cstr(string: &str) -> *mut i8 {
CString::new(string).unwrap().into_raw()
}
pub unsafe fn build(module: LLVMModuleRef, context: LLVMContextRef) {
let builder = LLVMCreateBuilderInContext(context);
let mut argtypes = vec!();
let func_type = LLVMFunctionType(LLVMInt64TypeInContext(context), argtypes.as_mut_ptr(), argtypes.len() as u32, false as i32);
let fptr_type = LLVMPointerType(func_type, 0);
let context_type = LLVMPointerType(LLVMInt8TypeInContext(context), 0);
let mut structfields = vec!(fptr_type, context_type);
let struct_type = LLVMStructType(structfields.as_mut_ptr(), structfields.len() as u32, false as i32);
let initializer = LLVMConstNull(struct_type);
let global = LLVMAddGlobal(module, struct_type, cstr("function"));
LLVMSetInitializer(global, initializer);
LLVMSetLinkage(global, LLVMLinkage::LLVMExternalLinkage);
let mut argtypes = vec!();
let main_type = LLVMFunctionType(LLVMInt64TypeInContext(context), argtypes.as_mut_ptr(), argtypes.len() as u32, false as i32);
let function = LLVMAddFunction(module, cstr("main"), main_type);
let bb = LLVMAppendBasicBlockInContext(context, function, cstr("entry"));
LLVMPositionBuilderAtEnd(builder, bb);
let mut indices = vec!(LLVMConstInt(LLVMInt32TypeInContext(context), 0, 0), LLVMConstInt(LLVMInt32TypeInContext(context), 0, 0));
let field = LLVMBuildGEP(builder, global, indices.as_mut_ptr(), indices.len() as u32, cstr(""));
LLVMBuildStore(builder, LLVMConstNull(fptr_type), field);
LLVMBuildRet(builder, LLVMConstInt(LLVMInt64TypeInContext(context), 0, 0));
LLVMDisposeBuilder(builder);
}
pub fn emit_module(module: LLVMModuleRef) -> String {
unsafe { CString::from_raw(LLVMPrintModuleToString(module)).into_string().unwrap() }
}
The full output is:
; ModuleID = 'module'
source_filename = "module"
#function = global { i64 ()*, i8* } zeroinitializer
define i64 #main() {
entry:
store i64 ()* null, i64 ()** getelementptr inbounds ({ i64 ()*, i8* }, { i64 ()*, i8* }* #function, i32 0, i32 0), align 8
ret i64 0
}
/media/work/contrib/llvm-project/llvm/lib/IR/ConstantsContext.h:745: void llvm::ConstantUniqueMap<ConstantClass>::remove(ConstantClass*) [with ConstantClass = llvm::ConstantExpr]: Assertion `I != Map.end() && "Constant not found in constant table!"' failed.
Aborted (core dumped)
Any help or suggestions would be most appreciated. Thanks
I finally figured out what the issue is. In the example above, the function LLVMStructType() also has an alternate version LLVMStructTypeInContext(), which if used instead will fix the assertion error. There is also another function LLVMModuleCreateWithNameInContext, which should probably be used instead, but in the example above, it will work without fixing this.
It's also possible to replace all the InContext versions with their non-context versions to fix the problem in the example
In my actual program, removing the InContext versions didn't work for some reason, but the two non-context functions mentioned above were being used, as well a couple uses of the non-context LLVMAppendBasicBlock. Replacing these with the InContext version fixed all the assertion errors, include the one that started this:
void llvm::Value::doRAUW(llvm::Value*, llvm::Value::ReplaceMetadataUses): Assertion `New->getType() == getType() && "replaceAllUses of value with new value of different type!"' failed.
The type pointers weren't the same, so it must have been because they were defined in different contexts.
Related
The following is only an example. If there's a native solution for this exact problem with reading bytes - cool, but my goal is to learn how to do it by myself, for any other purpose as well.
I'd like to do something like this: (pseudo-code below)
let mut reader = Reader::new(bytesArr);
let int32: i32 = reader.read(); // separate implementation to read 4 bits and convert into int32
let int64: i64 = reader.read(); // separate implementation to read 8 bits and convert into int64
I imagine it looking like this: (pseudo-code again)
impl Reader {
read<T>(&mut self) -> T {
// if T is i32 ... else if ...
}
}
or like this:
impl Reader {
read(&mut self) -> i32 {
// ...
}
read(&mut self) -> i64 {
// ...
}
}
But haven't found anything relatable yet.
(I actually have, for the first case (if T is i32 ...), but it looked really unreadable and inconvenient)
You could do this by having a Readable trait which you implement on i32 and i64, which does the operation. Then on Reader you could have a generic function which takes any type that is Readable and return it, for example:
struct Reader {
n: u8,
}
trait Readable {
fn read_from_reader(reader: &mut Reader) -> Self;
}
impl Readable for i32 {
fn read_from_reader(reader: &mut Reader) -> i32 {
reader.n += 1;
reader.n as i32
}
}
impl Readable for i64 {
fn read_from_reader(reader: &mut Reader) -> i64 {
reader.n += 1;
reader.n as i64
}
}
impl Reader {
fn read<T: Readable>(&mut self) -> T {
T::read_from_reader(self)
}
}
fn main() {
let mut r = Reader { n: 0 };
let int32: i32 = r.read();
let int64: i64 = r.read();
println!("{} {}", int32, int64);
}
You can try it on the playground
After some trials and searches, I found that implementing them in current Rust seems a bit difficult, but not impossible.
Here is the code, I'll explain it afterwards:
#![feature(generic_const_exprs)]
use std::{
mem::{self, MaybeUninit},
ptr,
};
static DATA: [u8; 8] = [
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
u8::MAX,
];
struct Reader;
impl Reader {
fn read<T: Copy + Sized>(&self) -> T
where
[(); mem::size_of::<T>()]: ,
{
let mut buf = [unsafe { MaybeUninit::uninit().assume_init() }; mem::size_of::<T>()];
unsafe {
ptr::copy_nonoverlapping(DATA.as_ptr(), buf.as_mut_ptr(), buf.len());
mem::transmute_copy(&buf)
}
}
}
fn main() {
let reader = Reader;
let v_u8: u8 = reader.read();
dbg!(v_u8);
let v_u16: u16 = reader.read();
dbg!(v_u16);
let v_u32: u32 = reader.read();
dbg!(v_u32);
let v_u64: u64 = reader.read();
dbg!(v_u64);
}
Suppose the global static variable DATA is the target data you want to read.
In current Rust, we cannot directly use the size of a generic parameter as the length of an array. This does not work:
fn example<T: Copy + Sized>() {
let mut _buf = [0_u8; mem::size_of::<T>()];
}
The compiler gives a weird error:
error: unconstrained generic constant
--> src\main.rs:34:31
|
34 | let mut _buf = [0_u8; mem::size_of::<T>()];
| ^^^^^^^^^^^^^^^^^^^
|
= help: try adding a `where` bound using this expression: `where [(); mem::size_of::<T>()]:`
There is an issue that is tracking it, if you want to go deeper into this error you can take a look.
We just follow the compiler's suggestion to add a where bound. This requires feature generic_const_exprs to be enabled.
Next, unsafe { MaybeUninit::uninit().assume_init() } is optional, which drops the overhead of initializing this array, since we will eventually overwrite it completely. You can replace it with 0_u8 if you don't like it.
Finally, copy the data you need and transmute this array to your generic type, return.
I think you will see the output you expect:
[src\main.rs:38] v_u8 = 255
[src\main.rs:41] v_u16 = 65535
[src\main.rs:44] v_u32 = 4294967295
[src\main.rs:47] v_u64 = 18446744073709551615
I am currently experimenting with the FFI functionality of Rust and implemented a simble HTTP request using libcurl as an exercise. Consider the following self-contained example:
use std::ffi::c_void;
#[repr(C)]
struct CURL {
_private: [u8; 0],
}
// Global CURL codes
const CURL_GLOBAL_DEFAULT: i64 = 3;
const CURLOPT_WRITEDATA: i64 = 10001;
const CURLOPT_URL: i64 = 10002;
const CURLOPT_WRITEFUNCTION: i64 = 20011;
// Curl types
type CURLcode = i64;
type CURLoption = i64;
// Curl function bindings
#[link(name = "curl")]
extern "C" {
fn curl_easy_init() -> *mut CURL;
fn curl_easy_setopt(handle: *mut CURL, option: CURLoption, value: *mut c_void) -> CURLcode;
fn curl_easy_perform(handle: *mut CURL) -> CURLcode;
fn curl_global_init(flags: i64) -> CURLcode;
}
// Curl callback for data retrieving
extern "C" fn callback_writefunction(
data: *mut u8,
size: usize,
nmemb: usize,
user_data: *mut c_void,
) -> usize {
let slice = unsafe { std::slice::from_raw_parts(data, size * nmemb) };
let mut vec = unsafe { Box::from_raw(user_data as *mut Vec<u8>) };
vec.extend_from_slice(slice);
Box::into_raw(vec);
nmemb * size
}
type Result<T> = std::result::Result<T, CURLcode>;
// Our own curl handle
pub struct Curl {
handle: *mut CURL,
data_ptr: *mut Vec<u8>,
}
impl Curl {
pub fn new() -> std::result::Result<Curl, CURLcode> {
let ret = unsafe { curl_global_init(CURL_GLOBAL_DEFAULT) };
if ret != 0 {
return Err(ret);
}
let handle = unsafe { curl_easy_init() };
if handle.is_null() {
return Err(2); // CURLE_FAILED_INIT according to libcurl-errors(3)
}
// Set data callback
let ret = unsafe {
curl_easy_setopt(
handle,
CURLOPT_WRITEFUNCTION,
callback_writefunction as *mut c_void,
)
};
if ret != 0 {
return Err(2);
}
// Set data pointer
let data_buf = Box::new(Vec::new());
let data_ptr = Box::into_raw(data_buf);
let ret = unsafe {
curl_easy_setopt(handle, CURLOPT_WRITEDATA, data_ptr as *mut std::ffi::c_void)
};
match ret {
0 => Ok(Curl { handle, data_ptr }),
_ => Err(2),
}
}
pub fn set_url(&self, url: &str) -> Result<()> {
let url_cstr = std::ffi::CString::new(url.as_bytes()).unwrap();
let ret = unsafe {
curl_easy_setopt(
self.handle,
CURLOPT_URL,
url_cstr.as_ptr() as *mut std::ffi::c_void,
)
};
match ret {
0 => Ok(()),
x => Err(x),
}
}
pub fn perform(&self) -> Result<String> {
let ret = unsafe { curl_easy_perform(self.handle) };
if ret == 0 {
let b = unsafe { Box::from_raw(self.data_ptr) };
let data = (*b).clone();
Box::into_raw(b);
Ok(String::from_utf8(data).unwrap())
} else {
Err(ret)
}
}
}
fn main() -> Result<()> {
let my_curl = Curl::new().unwrap();
my_curl.set_url("https://www.example.com")?;
my_curl.perform().and_then(|data| Ok(println!("{}", data)))
// No cleanup code in this example for the sake of brevity.
}
While this works, I found it surprising that my_curl does not need to be declared mut, since none of the methods use &mut self, even though they pass a mut* pointer to the FFI function
s.
Should I change the declaration of perform to use &mut self instead of &self (for safety), since the internal buffer gets modified? Rust does not enforce this, but of course Rust does not know that the buffer gets modified by libcurl.
This small example runs fine, but I am unsure if I would be facing any kind of issues in larger programs, when the compiler might optimize for non-mutable access on the Curl struct, even though the instance of the struct is getting modified - or at least the data the pointer is pointing to.
Contrary to popular belief, there is absolutely no borrowchecker-induced restriction in Rust on passing *const/*mut pointers. There doesn't need to be, because dereferencing pointers is inherently unsafe, and can only be done in such blocks, with the programmer verifying all necessary invariants manually. In your case, you need to tell the compiler that is a mutable reference, as you already suspected.
The interested reader should definitely give the ffi section of the nomicon a read, to find out about some interesting ways to shoot yourself in the foot with it.
I have been trying to use the ORC JIT compiler from the LLVM C bindings in Rust, but I keep running into the problem that LLVMOrcGetSymbolAddress is not able to find the symbol of my function run in the module I provide it. The code below combines the most important parts of my code that unfortunately doesn't work. All goes well until the last part of foo, where an error is returned because the function cannot be found. The function is definitely in the module, as LLVMGetNamedFunction is able to find it, but somehow the ORC engine cannot see it. Can anyone see what I am doing wrong? I am using LLVM 6.0 and the llvm-sys Rust bindings. Everything works fine if I use MCJIT but I need ORC for lazy compilation.
fn foo(module: LLVMModuleRef) -> Result<I64Func, LlvmError> {
let def_triple = LLVMGetDefaultTargetTriple();
let mut target_ref = ptr::null_mut();
let mut error_str = ptr::null_mut();
// Get target from default triple
if LLVMGetTargetFromTriple(def_triple, &mut target_ref, &mut error_str) != 0 {
let msg = format!("Creating target from triple failed: {}", CStr::from_ptr(error_str).to_str().unwrap());
LLVMDisposeMessage(def_triple);
LLVMDisposeMessage(error_str);
return Err(LlvmError(msg));
}
// Check if JIT is available
if LLVMTargetHasJIT(target_ref) == 0 {
let msg = format!("Cannot do JIT on this platform");
LLVMDisposeMessage(def_triple);
return Err(LlvmError(msg));
}
// Create target machine
let tm_ref = LLVMCreateTargetMachine(target_ref,
def_triple,
CString::new("").unwrap().as_ptr(),
CString::new("").unwrap().as_ptr(),
llvm_opt_level(optimization_level)?,
LLVMRelocMode::LLVMRelocDefault,
LLVMCodeModel::LLVMCodeModelJITDefault);
LLVMDisposeMessage(def_triple);
let engine = LLVMOrcCreateInstance(tm_ref);
// Add eagerly compiled IR
let mut handle = LLVMOrcModuleHandle::default();
let shared_module = LLVMOrcMakeSharedModule(module);
let ctx = engine as *mut libc::c_void;
map_orc_err(engine, LLVMOrcAddEagerlyCompiledIR(engine,
&mut handle,
shared_module,
symbol_resolver_callback,
ctx))?;
// Find function named 'run'
let c_name = CString::new("run").unwrap().as_ptr();
let mut func_addr = LLVMOrcTargetAddress::default();
map_orc_err(engine, LLVMOrcGetSymbolAddress(engine, &mut func_addr, c_name))?;
if func_addr == 0 {
// This errors always gets thrown
return Err(LlvmError(format!("No function named {} in module", name)));
}
let function: I64Func = mem::transmute(func_addr);
Ok(function)
}
extern "C" fn symbol_resolver_callback(symbol: *const libc::c_char, ctx: *mut libc::c_void) -> LLVMOrcTargetAddress {
let mut address = LLVMOrcTargetAddress::default();
let engine: LLVMOrcJITStackRef = ctx as LLVMOrcJITStackRef;
unsafe { LLVMOrcGetSymbolAddress(engine, &mut address, symbol) };
address
}
unsafe fn map_orc_err(engine: LLVMOrcJITStackRef, error_code: LLVMOrcErrorCode) -> Result<(), LlvmError> {
match error_code {
LLVMOrcErrorCode::LLVMOrcErrSuccess => Ok(()),
LLVMOrcErrorCode::LLVMOrcErrGeneric => {
let c_str: &CStr = CStr::from_ptr(LLVMOrcGetErrorMsg(engine));
let str_slice: &str = c_str.to_str().unwrap();
let str_buf: String = str_slice.to_owned();
Err(LlvmError(str_buf))
}
}
}
EDIT: I tried downgrading to LLVM 4.0 just to see what effect that might have. I still cannot resolve the function, but now I'm getting an assertion error:
Assertion failed: (!Name.empty() && "getNameWithPrefix requires non-empty name"), function getNameWithPrefixImpl, file /tmp/llvm-4.0-20180412-49671-1pw0nxu/llvm-4.0.1.src/lib/IR/Mangler.cpp, line 37.
EDIT 2: Below is some basic IR for which the engine fails to find the function address:
define i64 #bar(i64 %arg) {
%1 = add i64 %arg, 1
ret i64 %1
}
define i64 #run(i64 %arg) {
%1 = add i64 %arg, 1
%2 = call i64 #bar(i64 %1)
ret i64 %2
}
In pseudo-code, I'm trying to do the following:
my_array = [[1,2,3,4],[5,6,7,8]]
my_array = array_modify_fn(my_array)
fn array_modify_fn(array) {
for i in array {
array[i] = some_operation
}
}
Having read this question about the type mismatch this kind of loop/function would cause in Rust, I'm still confused as to how to actually implement what I want to implement here, but in Rust.
Am I just going about the problem in the wrong way? (For Rust at least; this is how I would do it in Python.)
My Rust at the moment looks like this:
let mut life_array = [[false; SIZE]; SIZE];
life_array = random_init(&mut life_array); // in main function
fn random_init(arr: &mut [[bool; SIZE]; SIZE]) -> [[bool; SIZE]; SIZE] {
for i in 0 .. (SIZE*SIZE) {
arr[i/SIZE][i%SIZE] = rand::random()
}
}
and this returns the type mismatch: expected type '[[bool; SIZE]; SIZE]' found type '()'
You've defined random_init with a return type, yet your function doesn't return anything (strictly speaking, it returns ()). Since you're mutating the array in-place, your function doesn't have to return anything, so you should just omit the return type.
const SIZE: usize = 4;
extern crate rand;
fn main() {
let mut life_array = [[false; SIZE]; SIZE];
random_init(&mut life_array);
}
fn random_init(arr: &mut [[bool; SIZE]; SIZE]) {
for i in 0..(SIZE * SIZE) {
arr[i / SIZE][i % SIZE] = rand::random()
}
}
extern crate llvm_sys;
use llvm_sys::*;
use llvm_sys::prelude::*;
use llvm_sys::core::*;
pub fn emit(module: LLVMModuleRef) {
unsafe {
use llvm_sys::target::*;
use llvm_sys::target_machine::*;
let triple = LLVMGetDefaultTargetTriple();
LLVM_InitializeNativeTarget();
let target = LLVMGetFirstTarget();
let cpu = "x86-64\0".as_ptr() as *const i8;
let feature = "\0".as_ptr() as *const i8;
let opt_level = LLVMCodeGenOptLevel::LLVMCodeGenLevelNone;
let reloc_mode = LLVMRelocMode::LLVMRelocDefault;
let code_model = LLVMCodeModel::LLVMCodeModelDefault;
let target_machine = LLVMCreateTargetMachine(target, triple, cpu, feature, opt_level, reloc_mode, code_model);
let file_type = LLVMCodeGenFileType::LLVMObjectFile;
LLVMTargetMachineEmitToFile(target_machine, module, "/Users/andyshiue/Desktop/main.o\0".as_ptr() as *mut i8, file_type, ["Cannot generate file.\0".as_ptr()].as_mut_ptr() as *mut *mut i8);
}
}
I'm writing a toy compiler and I want to generate object files, but the file LLVM outputs is empty.
I found that LLVMTargetMachineEmitToFile returns 1, which means something I'm doing is wrong, but what am I doing wrong?
It would be better if I can know how I can know what is wrong. Is there any way I can get some error message? I don't have any experience in C/C++.
As commenters have already said, to do what you want to do (write a compiler using LLVM), you are going to need to be able to read (and probably write) at the very least C and maybe C++.
Even though you are compiling code with the Rust compiler, you aren't really writing any Rust yet. Your entire program is wrapped in unsafe blocks because you are calling the C functions exposed by LLVM (which is written in C++). This may be why some commenters are asking if you have gotten your code to work in C first.
As in your other question, you are still calling the LLVM methods incorrectly. In this case, review the documentation for LLVMTargetMachineEmitToFile:
LLVMBool LLVMTargetMachineEmitToFile(LLVMTargetMachineRef T,
LLVMModuleRef M,
char *Filename,
LLVMCodeGenFileType codegen,
char **ErrorMessage)
Returns any error in ErrorMessage. Use LLVMDisposeMessage to dispose the message.
The method itself will tell you what is wrong, but you have to give it a place to store the error message. You should not provide an error string to it. I'm pretty sure that the current code is likely to generate some exciting memory errors when it tries to write to the string literal.
If I rewrite your code to use the error message:
extern crate llvm_sys;
use llvm_sys::*;
use llvm_sys::prelude::*;
use llvm_sys::core::*;
use std::ptr;
use std::ffi::{CStr, CString};
pub fn emit(module: LLVMModuleRef) {
let cpu = CString::new("x86-64").expect("invalid cpu");
let feature = CString::new("").expect("invalid feature");
let output_file = CString::new("/tmp/output.o").expect("invalid file");
unsafe {
use llvm_sys::target::*;
use llvm_sys::target_machine::*;
let triple = LLVMGetDefaultTargetTriple();
LLVM_InitializeNativeTarget();
let target = LLVMGetFirstTarget();
let opt_level = LLVMCodeGenOptLevel::LLVMCodeGenLevelNone;
let reloc_mode = LLVMRelocMode::LLVMRelocDefault;
let code_model = LLVMCodeModel::LLVMCodeModelDefault;
let target_machine = LLVMCreateTargetMachine(target, triple, cpu.as_ptr(), feature.as_ptr(), opt_level, reloc_mode, code_model);
let file_type = LLVMCodeGenFileType::LLVMObjectFile;
let mut error_str = ptr::null_mut();
let res = LLVMTargetMachineEmitToFile(target_machine, module, output_file.as_ptr() as *mut i8, file_type, &mut error_str);
if res == 1 {
let x = CStr::from_ptr(error_str);
panic!("It failed! {:?}", x);
// TODO: Use LLVMDisposeMessage here
}
}
}
fn main() {
unsafe {
let module = LLVMModuleCreateWithName("Main\0".as_ptr() as *const i8);
emit(module);
}
}
TargetMachine can't emit a file of this type
So that's your problem.
Rust-wise, you may want to wrap up the work needed to handle the silly LLVMBool so you can reuse it. One way would be:
fn llvm_bool<F>(f: F) -> Result<(), String>
where F: FnOnce(&mut *mut i8) -> i32
{
let mut error_str = ptr::null_mut();
let res = f(&mut error_str);
if res == 1 {
let err = unsafe { CStr::from_ptr(error_str) };
Err(err.to_string_lossy().into_owned())
//LLVMDisposeMessage(error_str);
} else {
Ok(())
}
}
// later
llvm_bool(|error_str| LLVMTargetMachineEmitToFile(target_machine, module, output_file.as_ptr() as *mut i8, file_type, error_str)).expect("Couldn't output");