Is this interface binary compatible between MSVC and mingw? - visual-c++

I am working on a library that allows its users (other libraries residing in the same process) to exchange data buffers and streams. The library has to be usable from both MSVC and mingw code (more compatibility doesn't hurt, but is not strictly necessary). In order to achieve this, the core functionality should be provided from a small, compiler-compatible interface that can be hidden later by a convenience layer that is compiled with the client code.
A challenging aspect of the library is that it has to be extensible, so that clients can provide their own buffer and stream implementations, but the core library interface must remain stable once it is released. If you are interested in further background, you can read about it in the forum thread discussion.
I have tried to learn about problems of binary compatibility between compilers, but since I am new to this topic I would be interested in comments on my result. I am not interested in standards-defined behaviour here (the structs probably fail that test), only in compatibility between mingw and MSVC and maybe other compilers if conveniently possible.
In particular, will the structs be compatible? They uniformly consist of function pointers, so I don't think padding will be an issue. Additionally, is the stdcall convention necessary here, or would cdecl work just as well? Could I leave it unspecified then since both compilers will default to cdecl? Should I? Here is what I have right now:
#include <stdint.h>
typedef struct {
uint32_t (__stdcall *read)(void*, uint8_t*, uint32_t);
void (__stdcall *write)(void*, const uint8_t*, uint32_t);
uint32_t (__stdcall *getBytesLeft)(void*);
uint8_t (__stdcall *destroy)(void*);
} SharedStreamInterface;
typedef struct {
uint32_t (__stdcall *read)(void*, uint8_t*, uint32_t);
void (__stdcall *write)(void*, const uint8_t*, uint32_t);
uint32_t (__stdcall *getBytesLeft)(void*);
uint8_t (__stdcall *destroy)(void*);
uint32_t (__stdcall *getreadpos)(void*);
uint32_t (__stdcall *getwritepos)(void*);
uint32_t (__stdcall *getlength)(void*);
void (__stdcall *setreadpos)(void*, uint32_t);
void (__stdcall *setwritepos)(void*, uint32_t);
void (__stdcall *setlength)(void*, uint32_t);
} SharedBufferInterface;
extern "C" {
// Functions applicable for both buffers and streams
__stdcall uint32_t readData(uint32_t id, uint8_t* data, uint32_t size);
__stdcall void writeData(uint32_t id, const uint8_t* data, uint32_t size);
__stdcall uint32_t getBytesLeft(uint32_t id);
__stdcall void destroyStreamOrBuffer(uint32_t id);
__stdcall uint8_t streamOrBufferExists(uint32_t id);
// Functions only applicable for buffers
__stdcall uint32_t getReadPos(uint32_t id);
__stdcall uint32_t getWritePos(uint32_t id);
__stdcall uint32_t getLength(uint32_t id);
__stdcall void setReadPos(uint32_t id, uint32_t pos);
__stdcall void setWritePos(uint32_t id, uint32_t pos);
__stdcall void setLength(uint32_t id, uint32_t length);
__stdcall uint8_t bufferExists(uint32_t id);
// Adding new buffers/Streams
__stdcall uint32_t addStream(SharedStreamInterface *interface, void *stream);
__stdcall uint32_t addBuffer(SharedBufferInterface *interface, void *buffer);
}
Edit: The project this was meant for has been on hold for a while now and probably needs a lot of rethinking if it's ever unshelved again. I'm leaving the question up though, because I'm still interested in the answer.

Yes, they will be compatible. That's the beauty with structs. As long as you don't introduce padding issues (which indeed wouldn't be the case here as you correctly pointed out), or in C++ add functionality to the structs that will result in - compiler-specific - vtable layouts, this will be always compatible.
You will also notice that from the Windows headers the C declarations of COM interfaces use structs in much the same way you do.
Side-note: the SharedStreamInterface::destroy member begs the question whether there is also one to "create" such a stream. You may want to share that as well. But your mileage may vary ...
As for the question of the calling convention, both __cdecl and __stdcall should work across the binaries, however I would always prefer __stdcall for another reason: it is compatible with more "languages" (i.e. tools) than __cdecl.
For style: use a #define to declare the calling convention explicitly (as you do) similar to WINAPI from the Windows headers.

Related

LPNMITEMACTIVATE and code analysis (C26462)

Why is it that in the source code in the SDK for LPNMITEMACTIVATE it is defined with the asterix to the left?
typedef struct tagNMITEMACTIVATE
{
NMHDR hdr;
int iItem;
int iSubItem;
UINT uNewState;
UINT uOldState;
UINT uChanged;
POINT ptAction;
LPARAM lParam;
UINT uKeyFlags;
} NMITEMACTIVATE, *LPNMITEMACTIVATE;
I am always used to the pointer being on the right. Either way, code like:
const LPNMITEMACTIVATE pNMItem = reinterpret_cast<LPNMITEMACTIVATE>(pNMHDR);
Will still flag a const (C26462) warning:
If I change the code to:
const NMITEMACTIVATE* pNMItem = reinterpret_cast<LPNMITEMACTIVATE>(pNMHDR);
The warning will go away.
I tried this with Visual Studio 2022, first of all, warning C26462 was not enabled by default. Perhaps you are using an earlier release, or there is something odd with my installation.
After manually enabling the warning, I could make that warning go away by assigning pNMItem more than once:
LPNMITEMACTIVATE pNMItem = nullptr;
pNMItem = reinterpret_cast<LPNMITEMACTIVATE>(pNMHDR);
How is this useful?
Or it can be fixed as suggested in other answers. But you may have additional problem because pNMHDR was probably declared as LPNMHDR, so you have to rewrite more lines:
NMHDR hdr = { 0 };
const NMHDR* pNMHDR = reinterpret_cast<NMHDR*>(&hdr);
const NMITEMACTIVATE* pNMItem = reinterpret_cast<const NMITEMACTIVATE*>(pNMHDR);
This can be a big waste of time. Note, the extra compliance is recommended if you are writing code that's supposed to run on any system. But MFC is tied to Windows so this isn't really an issue. MFC and Windows are still using that "long pointer" crap that's left over from 16-bit Windows, they are not compliant themselves, so consider turning off some of these warnings.
This is standard C/C++
Like in this (not runnable) code snippet:
typedef int *LPINT;
// typedef int* LPINT; // you could write this, it's exactly the
// the same as above
int main()
{
LPINT pint;
int* pint2;
*pint = *pint2;
}
pint and pint2 are both pointers to int. BTW this is hiding a pointer type behind a typedef, which is a bad idea (but was considered as a good idea in old MS days), but lots of Microsoft headers still have these typedef sometype *LPsometype; typedefs for compatibility reasons.
Another example which is closer to the MS header you're refering to:
This:
typedef struct tagNMITEMACTIVATE
{
int hdr;
int iItem;
} NMITEMACTIVATE, *LPNMITEMACTIVATE;
is equivalent to this:
typedef struct tagNMITEMACTIVATE
{
int hdr;
int iItem;
} NMITEMACTIVATE;
typedef struct tagNMITEMACTIVATE *LPNMITEMACTIVATE;
For pointer const can be applied to the type the pointer points at:
const NMITEMACTIVATE* p;
or
NMITEMACTIVATE const* p;
Or it can be applied to the pointer variable itself:
NMITEMACTIVATE* const p;
Now if you have typedef:
typedef NMITEMACTIVATE *PNMITEMACTIVATE;
The const would not apply to the type being pointed at. Either way it is the pointer itself is constant:
const PNMITEMACTIVATE p;
PNMITEMACTIVATE const p;
To avoid this confusion, prefer not to use raw pointer typedefs (and not to define them).

Why does the `of_find_compatible_node` function have two definitions in `/kernel/include/linux/of.h`, and how is the definition chosen?

The function of_find_compatible_node has two definition. One is
static online struct device_node *of_find_compatible_node(struct device_node *from,const char *type, const char *compatible)
another is
extern struct device_node *of_find_compatible_node(struct device_node *from,const char *type, const char *compatible)
These functions are defined the same.I know the keyword static online and extern,but I don't understand :
When I use the function,what does the compiler choose?
Why are two functions not redefined?
In base.c and include<of.h>, there appears to be a re-definition of struct_device_node *of_find_comatible_node(struct device_node *from,const char *type,const char *compatible). Why? I think it causes redefinition when linking.
This is a very common pattern in kernel headers (and not just there). If you take a closer look at the file (or use an editor that does code folding), you can notice that both lines are inside conditional compilation blocks, like this:
#ifdef CONFIG_OF
extern struct device_node *of_find_compatible_node(struct device_node *from,
const char *type, const char *compat);
#else /* CONFIG_OF */
static inline struct device_node *of_find_compatible_node(
struct device_node *from,
const char *type,
const char *compat)
{
return NULL;
}
#endif
Thus if CONFIG_OF is enabled, the former part gets compiled in and serves as a declaration of the function, whose real definition is placed in some .c file. If the kernel is configured without CONFIG_OF, the #else branch will be used, supplying a dummy no-op implementation of the function so that the rest of the kernel still compiles correctly.
The dummy implementation is also static inline, allowing the compiler to just replace calls to it by a constant NULL value and likely optimize whole code paths away, decreasing code size and eliminating runtime overhead.

LARGE_INTEGER and BYTE types in Linux?

I am studying one Windows code which converting to Linux and this answer here.
It is possible that there are no injective datatypes between LARGE_INTEGER and BYTE in Linux so special situations may need to be considered.
The two data types could possibly replaced by the following two in Linux
uint64_t
unsigned char
where the first one is a part of stdint.h.
I would propose replace both LARGE_INTEGER and BYTE by uint64_t but not certain.
Which Linux datatypes can use in place of LARGE_INTEGER and BYTE?
The following file is a case example of National Instruments' driver for MCA 8000A described here.
MictoTime.h
//////////////////////////////////////////////////////////////////////
//
// MicroTime.h: interface for the MicroTime class.
//
// This header file is a part of the PMCA COM package
//
// Amptek Inc. 2000
//
//////////////////////////////////////////////////////////////////////
#if !defined(AFX_MICROTIME_H__495147C7_F0B7_11D1_B62F_9CFF0CC10000__INCLUDED_)
#define AFX_MICROTIME_H__495147C7_F0B7_11D1_B62F_9CFF0CC10000__INCLUDED_
#if _MSC_VER >= 1000
#pragma once
#endif // _MSC_VER >= 1000
class MicroTimeType
{
LARGE_INTEGER m_startTime;
double m_microScale;
BOOL m_waiting;
public:
MicroTimeType();
LARGE_INTEGER GetCounter();
double Get(void);
double Wait(double microSec);
void CancelWait(void);
};
#endif // !defined(AFX_MICROTIME_H__495147C7_F0B7_11D1_B62F_9CFF0CC10000__INCLUDED_)
Do you need to add here new types of this answer?
Do you need to define uint8_t anywhere else?
You can see the reference for data types in C supported by Windows API and toolchain.
LARGE_INTEGER is actually a union representing signed 64-bit integers which has been historically used in some APIs. Simple typedef int64_t LARGE_INTEGER; won't work as expected.
You will need something like this:
#include <stdint.h>
typedef uint8_t BYTE;
typedef uint32_t DWORD;
typedef int32_t LONG;
typedef int64_t LONGLONG;
typedef union _LARGE_INTEGER {
struct {
DWORD LowPart;
LONG HighPart;
};
struct {
DWORD LowPart;
LONG HighPart;
} u;
LONGLONG QuadPart;
} LARGE_INTEGER, *PLARGE_INTEGER;

Intercepting a system call [duplicate]

I'm trying to write some simple test code as a demonstration of hooking the system call table.
"sys_call_table" is no longer exported in 2.6, so I'm just grabbing the address from the System.map file, and I can see it is correct (Looking through the memory at the address I found, I can see the pointers to the system calls).
However, when I try to modify this table, the kernel gives an "Oops" with "unable to handle kernel paging request at virtual address c061e4f4" and the machine reboots.
This is CentOS 5.4 running 2.6.18-164.10.1.el5. Is there some sort of protection or do I just have a bug? I know it comes with SELinux, and I've tried putting it in to permissive mode, but it doesn't make a difference
Here's my code:
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/unistd.h>
void **sys_call_table;
asmlinkage int (*original_call) (const char*, int, int);
asmlinkage int our_sys_open(const char* file, int flags, int mode)
{
printk("A file was opened\n");
return original_call(file, flags, mode);
}
int init_module()
{
// sys_call_table address in System.map
sys_call_table = (void*)0xc061e4e0;
original_call = sys_call_table[__NR_open];
// Hook: Crashes here
sys_call_table[__NR_open] = our_sys_open;
}
void cleanup_module()
{
// Restore the original call
sys_call_table[__NR_open] = original_call;
}
I finally found the answer myself.
http://www.linuxforums.org/forum/linux-kernel/133982-cannot-modify-sys_call_table.html
The kernel was changed at some point so that the system call table is read only.
cypherpunk:
Even if it is late but the Solution
may interest others too: In the
entry.S file you will find: Code:
.section .rodata,"a"
#include "syscall_table_32.S"
sys_call_table -> ReadOnly You have to
compile the Kernel new if you want to
"hack" around with sys_call_table...
The link also has an example of changing the memory to be writable.
nasekomoe:
Hi everybody. Thanks for replies. I
solved the problem long ago by
modifying access to memory pages. I
have implemented two functions that do
it for my upper level code:
#include <asm/cacheflush.h>
#ifdef KERN_2_6_24
#include <asm/semaphore.h>
int set_page_rw(long unsigned int _addr)
{
struct page *pg;
pgprot_t prot;
pg = virt_to_page(_addr);
prot.pgprot = VM_READ | VM_WRITE;
return change_page_attr(pg, 1, prot);
}
int set_page_ro(long unsigned int _addr)
{
struct page *pg;
pgprot_t prot;
pg = virt_to_page(_addr);
prot.pgprot = VM_READ;
return change_page_attr(pg, 1, prot);
}
#else
#include <linux/semaphore.h>
int set_page_rw(long unsigned int _addr)
{
return set_memory_rw(_addr, 1);
}
int set_page_ro(long unsigned int _addr)
{
return set_memory_ro(_addr, 1);
}
#endif // KERN_2_6_24
Here's a modified version of the original code that works for me.
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/moduleparam.h>
#include <linux/unistd.h>
#include <asm/semaphore.h>
#include <asm/cacheflush.h>
void **sys_call_table;
asmlinkage int (*original_call) (const char*, int, int);
asmlinkage int our_sys_open(const char* file, int flags, int mode)
{
printk("A file was opened\n");
return original_call(file, flags, mode);
}
int set_page_rw(long unsigned int _addr)
{
struct page *pg;
pgprot_t prot;
pg = virt_to_page(_addr);
prot.pgprot = VM_READ | VM_WRITE;
return change_page_attr(pg, 1, prot);
}
int init_module()
{
// sys_call_table address in System.map
sys_call_table = (void*)0xc061e4e0;
original_call = sys_call_table[__NR_open];
set_page_rw(sys_call_table);
sys_call_table[__NR_open] = our_sys_open;
}
void cleanup_module()
{
// Restore the original call
sys_call_table[__NR_open] = original_call;
}
Thanks Stephen, your research here was helpful to me. I had a few problems, though, as I was trying this on a 2.6.32 kernel, and getting WARNING: at arch/x86/mm/pageattr.c:877 change_page_attr_set_clr+0x343/0x530() (Not tainted) followed by a kernel OOPS about not being able to write to the memory address.
The comment above the mentioned line states:
// People should not be passing in unaligned addresses
The following modified code works:
int set_page_rw(long unsigned int _addr)
{
return set_memory_rw(PAGE_ALIGN(_addr) - PAGE_SIZE, 1);
}
int set_page_ro(long unsigned int _addr)
{
return set_memory_ro(PAGE_ALIGN(_addr) - PAGE_SIZE, 1);
}
Note that this still doesn't actually set the page as read/write in some situations. The static_protections() function, which is called inside of set_memory_rw(), removes the _PAGE_RW flag if:
It's in the BIOS area
The address is inside .rodata
CONFIG_DEBUG_RODATA is set and the kernel is set to read-only
I found this out after debugging why I still got "unable to handle kernel paging request" when trying to modify the address of kernel functions. I was eventually able to solve that problem by finding the page table entry for the address myself and manually setting it to writable. Thankfully, the lookup_address() function is exported in version 2.6.26+. Here is the code I wrote to do that:
void set_addr_rw(unsigned long addr) {
unsigned int level;
pte_t *pte = lookup_address(addr, &level);
if (pte->pte &~ _PAGE_RW) pte->pte |= _PAGE_RW;
}
void set_addr_ro(unsigned long addr) {
unsigned int level;
pte_t *pte = lookup_address(addr, &level);
pte->pte = pte->pte &~_PAGE_RW;
}
Finally, while Mark's answer is technically correct, it'll case problem when ran inside Xen. If you want to disable write-protect, use the read/write cr0 functions. I macro them like this:
#define GPF_DISABLE write_cr0(read_cr0() & (~ 0x10000))
#define GPF_ENABLE write_cr0(read_cr0() | 0x10000)
Hope this helps anyone else who stumbles upon this question.
Note that the following will also work instead of using change_page_attr and cannot be depreciated:
static void disable_page_protection(void) {
unsigned long value;
asm volatile("mov %%cr0,%0" : "=r" (value));
if (value & 0x00010000) {
value &= ~0x00010000;
asm volatile("mov %0,%%cr0": : "r" (value));
}
}
static void enable_page_protection(void) {
unsigned long value;
asm volatile("mov %%cr0,%0" : "=r" (value));
if (!(value & 0x00010000)) {
value |= 0x00010000;
asm volatile("mov %0,%%cr0": : "r" (value));
}
}
If you are dealing with kernel 3.4 and later (it can also work with earlier kernels, I didn't test it) I would recommend a smarter way to acquire the system callы table location.
For example
#include <linux/module.h>
#include <linux/kallsyms.h>
static unsigned long **p_sys_call_table;
/* Aquire system calls table address */
p_sys_call_table = (void *) kallsyms_lookup_name("sys_call_table");
That's it. No addresses, it works fine with every kernel I've tested.
The same way you can use a not exported Kernel function from your module:
static int (*ref_access_remote_vm)(struct mm_struct *mm, unsigned long addr,
void *buf, int len, int write);
ref_access_remote_vm = (void *)kallsyms_lookup_name("access_remote_vm");
Enjoy!
As others have hinted, the whole story is a bit different now on modern kernels. I'll be covering x86-64 here, for syscall hijacking on modern arm64 refer to this other answer of mine. Also NOTE: this is plain and simple syscall hijacking. Non-invasive hooking can be done in a much nicer way using kprobes.
Since Linux v4.17, x86 (both 64 and 32 bit) now uses syscall wrappers that take a struct pt_regs * as the only argument (see commit 1, commit 2). You can see arch/x86/include/asm/syscall.h for the definitions.
Additionally, as others have described already in different answers, the simplest way to modify sys_call_table is to temporarily disable CR0 WP (Write-Protect) bit, which could be done using read_cr0() and write_cr0(). However, since Linux v5.3, [native_]write_cr0 will check sensitive bits that should never change (like WP) and refuse to change them (commit). In order to work around this, we need to write CR0 manually using inline assembly.
Here is a working kernel module (tested on Linux 5.10 and 5.18) that does syscall hijacking on modern Linux x86-64 considering the above caveats and assuming that you already know the address of sys_call_table (if you also want to find that in the module, see Proper way of getting the address of non-exported kernel symbols in a Linux kernel module):
// SPDX-License-Identifier: (GPL-2.0 OR MIT)
/**
* Test syscall table hijacking on x86-64. This module will replace the `read`
* syscall with a simple wrapper which logs every invocation of `read` using
* printk().
*
* Tested on Linux x86-64 v5.10, v5.18.
*
* Usage:
*
* sudo cat /proc/kallsyms | grep sys_call_table # grab address
* sudo insmod syscall_hijack.ko sys_call_table_addr=0x<address_here>
*/
#include <linux/init.h> // module_{init,exit}()
#include <linux/module.h> // THIS_MODULE, MODULE_VERSION, ...
#include <linux/kernel.h> // printk(), pr_*()
#include <asm/special_insns.h> // {read,write}_cr0()
#include <asm/processor-flags.h> // X86_CR0_WP
#include <asm/unistd.h> // __NR_*
#ifdef pr_fmt
#undef pr_fmt
#endif
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
typedef long (*sys_call_ptr_t)(const struct pt_regs *);
static sys_call_ptr_t *real_sys_call_table;
static sys_call_ptr_t original_read;
static unsigned long sys_call_table_addr;
module_param(sys_call_table_addr, ulong, 0);
MODULE_PARM_DESC(sys_call_table_addr, "Address of sys_call_table");
// Since Linux v5.3 [native_]write_cr0 won't change "sensitive" CR0 bits, need
// to re-implement this ourselves.
static void write_cr0_unsafe(unsigned long val)
{
asm volatile("mov %0,%%cr0": "+r" (val) : : "memory");
}
static long myread(const struct pt_regs *regs)
{
pr_info("read(%ld, 0x%lx, %lx)\n", regs->di, regs->si, regs->dx);
return original_read(regs);
}
static int __init modinit(void)
{
unsigned long old_cr0;
real_sys_call_table = (typeof(real_sys_call_table))sys_call_table_addr;
pr_info("init\n");
// Temporarily disable CR0 WP to be able to write to read-only pages
old_cr0 = read_cr0();
write_cr0_unsafe(old_cr0 & ~(X86_CR0_WP));
// Overwrite syscall and save original to be restored later
original_read = real_sys_call_table[__NR_read];
real_sys_call_table[__NR_read] = myread;
// Restore CR0 WP
write_cr0_unsafe(old_cr0);
pr_info("init done\n");
return 0;
}
static void __exit modexit(void)
{
unsigned long old_cr0;
pr_info("exit\n");
old_cr0 = read_cr0();
write_cr0_unsafe(old_cr0 & ~(X86_CR0_WP));
// Restore original syscall
real_sys_call_table[__NR_read] = original_read;
write_cr0_unsafe(old_cr0);
pr_info("goodbye\n");
}
module_init(modinit);
module_exit(modexit);
MODULE_VERSION("0.1");
MODULE_DESCRIPTION("Test syscall table hijacking on x86-64.");
MODULE_AUTHOR("Marco Bonelli");
MODULE_LICENSE("Dual MIT/GPL");

Question about pthreads & pointers

Here is an example of thread creation code that is often seen. pthread_create uses a lot of pointers/addresses and I was wondering why this is so.
pthread_t threads[NUM_THREADS];
long t;
for(t=0; t<NUM_THREADS; t++){
rc = pthread_create(&threads[t], NULL, &someMethod, (void *)t);
}
Is there a major advantage or difference for using the '&' to refer to the variable array 'threads' as well as 'someMethod' (as opposed to just 'threads' and just 'someMethod')? And also, why is 't' usually passed as a void pointer instead of just 't'?
int pthread_create(pthread_t *thread, const pthread_attr_t *attr,
void *(*start_routine)(void*), void *arg);
You need to pass a pointer to a pthread_t variable to pthread_create. &threads[t] and threads+t achieve this. threads[t] does not. pthread_create requires a pointer so it can return a value through it.
someMethod is a suitable expression for the third argument, since it's the address of the function. I think &someMethod is redundantly equivalent, but I'm not sure.
You are casting t to void * in order to jam a long into a void *. I don't think a long is guaranteed to fit in a void *. It's definitely a suboptimal solution even if the guarantee exists. You should be passing a pointer to t (&t, no cast required) for clarity and to ensure compatibility with the expected void *. Don't forget to adjust someMethod accordingly.
pthread_t threads[NUM_THREADS];
long t;
for (t=0; t<NUM_THREADS; t++) {
rc = pthread_create(&threads[t], NULL, someMethod, &t);
}

Resources