Kernel make generates “ld: arch/x86/entry/syscall_64.o:(.rodata+0xdc0): undefined reference to `__x64_sys_s_enable'” - linux

Question

OS is Ubuntu 20.10 Kernel Source is linux_5.8.0-59.66
I am porting kernel modifications from Centos 7 Rhel 7.9 to Ubuntu.
The original unmodified Ubuntu kernel source compiles and runs cleanly on this machine. The compiler set up seems to be functioning properly.
My current problem is related to a system call I've added. The error generated is -
LD .tmp_vmlinux.btf
ld: arch/x86/entry/syscall_64.o:(.rodata+0xdc0): undefined reference to `__x64_sys_s_enable'
BTF .btf.vmlinux.bin.o
Segmentation fault (core dumped)
LD .tmp_vmlinux.kallsyms1
.btf.vmlinux.bin.o: file not recognized: file format not recognized
make: *** [Makefile:1163: vmlinux] Error 1
I have searched and googled this original error "undefined reference", found possible fixes which have not worked.
Here are the steps I used to add the system call, which originally worked on Centos 7 and RHEL 7.9.
Modified /SOURCE-DIRECTORY/include/linux/syscalls.h commentng out the original line and adding the reference to __64 (including a blank line above it)-
asmlinkage long __64_sys_s_enable(int s_enable_flag);
//asmlinkage long sys_s_enable(int s_enable_flag);
Modified /SOURCE-DIRECTORY/arch/x86/include/asm/syscalls.h adding -
440 64 s_enable sys_s_enable
The fields are delimited by TAB, and I did not add any blank lines.
Created the source directory and files - /SOURCE-DIRECTORY/s_enable containing s_enable.c. s_enable.c in it's entirety is
#include <linux/kernel.h>
extern int s_enable_flag;
asmlinkage long sys_s_enable(int i)
{
// printk(KERN_INFO "In ORIGINAL SYSCALL s_enable\n");
s_enable_flag = i;
return 0;
}
And added the appropriate syscall directory to the Makefile.
core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/ s_enable/
And ran "sudo make".
I'm not sure what I might be doing wrong in that the "make" works with the original kernel source, and the system call I am trying to add has worked on the other mentioned distros.
Thanks for any input you can provide.
UPDATE 07-18-2021
I made the following changes on 07-17-2021 in order to use SYSCALL_DEFINE1.
SOURCEDIR/include/linux/syscalls.h
The reference to sys_s_enable has been commented out.
//asmlinkage long sys_s_enable(int s_enable_flag);
SOURCEDIR/arch/x86/entry/syscalls/syscall_64.tbl
"64" changed to "common"
440 common s_enable sys_s_enable
SOURCEDIR/Makefile has been edited to remove SOURCEDIR/s_enable from core-y
core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/
#core-y += kernel/ certs/ mm/ fs/ ipc/ security/ crypto/ block/ s_enable/
Copied/edited the original s_enable.c into SOURCEDIR/kernel/sys.c using SYSCALL_DEFINE1
SYSCALL_DEFINE1(su_enable, int, i)
{
extern int s_enable_flag;
s_enable_flag = i;
return 0;
}
The compile command was sudo make -j4 and took 12-15 hours which is somewhat normal.
The error was
LD .tmp_vmlinux.btf
ld: arch/x86/entry/syscall_64.o:(.rodata+0xdc0): undefined reference to `__x64_sys_s_enable'
Thanks - Roger

Answer 1

If we want to create our own system call a newer version of Linux __x64_sys_
Here is the comment from arch/x86/entry/syscalls/syscall_64.tbl in the begin
The x64_sys*() stubs are created on-the-fly for sys*() system calls
so that our system call function name might start the prefix with __x64_sys_, here is the sample code for your own function.
asmlinkage long __x64_sys_s_enable(int i)
{
// printk(KERN_INFO "In ORIGINAL SYSCALL s_enable\n");
s_enable_flag = i;
return 0;
}
Then the include/linux/syscalls.h file might need to add this prefix name which aligns with the function name
asmlinkage long __x64_sys_s_enable(int i);
the system-call entry we can just use your expectation function name arch/x86/entry/syscalls/syscall_64.tbl
440 common s_enable sys_s_enable
We can recompile your kernel if we follow those steps, and we might get a successful build.