needs corrections in writing a LL(1) math expression parser - antlr4

I tried to write a LL(1) math expression parser.
I tried to eliminate left-recursive and left common factor, and my grammar looks like the following:
grammar test;
NUM:
LEFT_PAR? NEG? INT (DOT INT)? EXP? RIGHT_PAR? ;
fragment INT: [1-9][0-9]* | '0' ;
fragment EXP: [eE][+\-]? INT ;
fragment NEG: '-' ;
fragment DOT: '.' ;
FACTORIAL: '^' ;
ADD: '+' ;
SUB: '-' ;
MUL: '*' ;
DIV: '/' ;
MODULUS: '%' ;
LOGICAL_NEG: '!' ;
LEFT_PAR: '(' ;
RIGHT_PAR: ')' ;
COMMA: ',' ;
VAR_NAME: [A-Za-z0-9_.]+ ;
CEIL : 'CEIL' ;
FLOOR: 'FLOOR' ;
TRUNC: 'TRUNC' ;
SQRT: 'SQRT' ;
ABS: 'ABS' ;
EXP_FUNC: 'EXP' ;
LOG: 'LOG' ;
SIN: 'SIN' ;
COS: 'COS' ;
TAN: 'TAN' ;
MIN: 'MIN' ;
MAX: 'MAX' ;
WS: [ \t\r\n]+ -> channel(HIDDEN) ;
math_expression:
math_expression_t math_expression_e_dash ;
math_expression_t:
math_expression_f math_expression_t_dash ;
math_expression_e_dash :
( ADD | SUB ) math_expression_t math_expression_e_dash | /*epsilon*/ ;
math_expression_f:
math_expression_n math_expression_f_dash ;
math_expression_t_dash:
( MUL | DIV | MODULUS ) math_expression_f math_expression_t_dash | /*epsilon*/ ;
math_expression_n:
LEFT_PAR math_expression RIGHT_PAR | LOGICAL_NEG math_expression | NUM | SUB? VAR_NAME ;
math_expression_f_dash:
FACTORIAL math_expression_f | /*epsilon*/ ;
I tried to figure it out whether it's LL(1), by typing antlr4 -Xlog option in terminal.But the result from the log file didn't imply my grammar is good.
The result from the log file is shown as follows:
2022-11-08 16:57:33:896 semantics LogManager.java:25 tokens={EOF=-1, NUM=1, FACTORIAL=2, ADD=3, SUB=4, MUL=5, DIV=6, MODULUS=7, LOGICAL_NEG=8, LEFT_PAR=9, RIGHT_PAR=10, COMMA=11, VAR_NAME=12, CEIL=13, FLOOR=14, TRUNC=15, SQRT=16, ABS=17, EXP_FUNC=18, LOG=19, SIN=20, COS=21, TAN=22, MIN=23, MAX=24, WS=25}
2022-11-08 16:57:33:896 semantics LogManager.java:25 strings={'^'=2, '+'=3, '-'=4, '*'=5, '/'=6, '%'=7, '!'=8, '('=9, ')'=10, ','=11, 'CEIL'=13, 'FLOOR'=14, 'TRUNC'=15, 'SQRT'=16, 'ABS'=17, 'EXP'=18, 'LOG'=19, 'SIN'=20, 'COS'=21, 'TAN'=22, 'MIN'=23, 'MAX'=24}
2022-11-08 16:57:33:899 LL1 LogManager.java:25
DECISION 0 in rule math_expression_e_dash
2022-11-08 16:57:33:899 LL1 LogManager.java:25 look=[{3..4}, {2..7, 10}]
2022-11-08 16:57:33:899 LL1 LogManager.java:25 LL(1)? false
2022-11-08 16:57:33:899 LL1 LogManager.java:25
DECISION 1 in rule math_expression_t_dash
2022-11-08 16:57:33:900 LL1 LogManager.java:25 look=[{5..7}, {2..7, 10}]
2022-11-08 16:57:33:900 LL1 LogManager.java:25 LL(1)? false
2022-11-08 16:57:33:900 LL1 LogManager.java:25
DECISION 2 in rule math_expression_n
2022-11-08 16:57:33:900 LL1 LogManager.java:25 look=[4, 12]
2022-11-08 16:57:33:900 LL1 LogManager.java:25 LL(1)? true
2022-11-08 16:57:33:900 LL1 LogManager.java:25
DECISION 3 in rule math_expression_n
2022-11-08 16:57:33:900 LL1 LogManager.java:25 look=[9, 8, 1, {4, 12}]
2022-11-08 16:57:33:900 LL1 LogManager.java:25 LL(1)? true
2022-11-08 16:57:33:900 LL1 LogManager.java:25
DECISION 4 in rule math_expression_f_dash
2022-11-08 16:57:33:900 LL1 LogManager.java:25 look=[2, {2..7, 10}]
2022-11-08 16:57:33:900 LL1 LogManager.java:25 LL(1)? false
The log above seems to show that e_dash,t_dash,and f_dash have problems in look ahead symbol and can't make it LL(1).
I wonder how can I correct my grammar to make it right in LL(1),thank you.

Related

How to use spi-loopback-test linux kernel module?

I do not understand how to use the linux kernel module spi-loopback-test.
This module is compiled from spi-loopback-test.c, and is still maintained.
It was created for SPI regression test purpose (see initial commit log), but there is not documentation nor instruction anywhere on the net.
I am able to compile and load the module, but nothing happens when it is loaded.
No message in dmesg, no activity on the physical SPI buses. lsmod shows the module is loaded.
It seems like the module's spi_loopback_test_probe function is never called.
I think I am missing some "glue" (DT overlay??) in the Device Tree, to assign a physical SPI port to the module.
How to assign a SPI port to this module?
Should I create a DT overlay to add a spi-loopback-test driver node under the existing spi0 peripheral node?
Also, I noticed the module parameter compatible that can changed before loading.
What value could be set here?
My target system is a Raspberry Pi 4B (but I guess the issue is generalisable to all linux systems).
Any help/pointer is appreciated!
Thanks
Okay, I finally found one way to run the spi-loopback-test module.
The trick here was to use anyspi Device-Tree overlay, to avoid the dependency of spidev module.
I got my inspiration from this commit.
First of all, the kernel must be recompiled to include the test feature “spi-loopback-test”.
I followed this guide, cross-compiling from a Ubuntu VM.
Enable SPI loopback module via “menuconfig” before compiling: CONFIG_SPI_DEBUG=y and CONFIG_SPI_LOOPBACK_TEST=m
Load the kernel image and its device-tree onto the target system (RPi4).
Run the following commands on the RPi4:
$ sudo dmesg -n 8
$ lsmod | grep spi
$ sudo dtoverlay anyspi spi0-1 dev="linux,spi-loopback-test" speed=125000000
$ lsmod | grep spi
spi_loopback_test 28672 0
spidev 20480 0
spi_bcm2835 24576 0
$ dmesg | grep spi > spi-loopback-test.log
$ $ head spi-loopback-test.log -n 15
[ 48.170758] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/spi#7e204000/spidev#1/status
[ 48.170785] OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/spi#7e204000/status
[ 48.182921] spi-bcm2835 fe204000.spi: registered master spi0
[ 48.183056] spi spi0.1: setup: forcing CS_HIGH (use_gpio_descriptors)
[ 48.183075] spi spi0.1: setup mode 0, cs_high, 8 bits/w, 125000000 Hz max --> 0
[ 48.183187] spi-bcm2835 fe204000.spi: registered child spi0.1
[ 48.183217] spi spi0.0: setup: forcing CS_HIGH (use_gpio_descriptors)
[ 48.183232] spi spi0.0: setup mode 0, cs_high, 8 bits/w, 125000000 Hz max --> 0
[ 48.183327] spi-bcm2835 fe204000.spi: registered child spi0.0
[ 48.269732] spi-loopback-test spi0.1: Executing spi-loopback-tests
[ 48.269946] spi-loopback-test spi0.1: Running test tx/rx-transfer - start of page
[ 48.270204] spi-loopback-test spi0.1: with iteration values: len = 0, tx_off = 0, rx_off = 1
[ 48.270431] spi-loopback-test spi0.1: with iteration values: len = 0, tx_off = 0, rx_off = 2
[ 48.270657] spi-loopback-test spi0.1: with iteration values: len = 0, tx_off = 0, rx_off = 3
[ 48.270883] spi-loopback-test spi0.1: with iteration values: len = 0, tx_off = 1, rx_off = 0
$ tail spi-loopback-test.log
[ 51.878516] spi-loopback-test spi0.1: with iteration values: len = 1024, tx_off = 0, rx_off = 0
[ 51.881159] spi-loopback-test spi0.1: with iteration values: len = 1031, tx_off = 0, rx_off = 0
[ 51.883826] spi-loopback-test spi0.1: with iteration values: len = 4093, tx_off = 0, rx_off = 0
[ 51.886892] spi-loopback-test spi0.1: with iteration values: len = 4096, tx_off = 0, rx_off = 0
[ 51.889950] spi-loopback-test spi0.1: with iteration values: len = 4099, tx_off = 0, rx_off = 0
[ 51.893862] spi-loopback-test spi0.1: with iteration values: len = 65536, tx_off = 0, rx_off = 0
[ 51.905280] spi-loopback-test spi0.1: with iteration values: len = 65537, tx_off = 0, rx_off = 0
[ 51.916708] spi-loopback-test spi0.1: with iteration values: len = 131071, tx_off = 0, rx_off = 0
[ 51.937417] spi-loopback-test spi0.1: with iteration values: len = 131072, tx_off = 0, rx_off = 0
[ 52.057851] spi-loopback-test spi0.1: Finished spi-loopback-tests with return: 0
So it looks like the SPI tests succeeded, as expected. Now I know a bit better how to interface the kernel modules to the hardware.
I hope this can be useful to somebody else too.
However, I wonder if there is a simpler solution to load and run this kernel module??
Can it be done without relying on anyspi DT overlay, by only setting the compatible parameter of spi-loopback-mode module?

What are the meanings of columns shown by readelf, while inspecting Section Headers?

Can someone explain the meaning of these columns?
I use readelf to read a ELF file and can't find any rellevant info (like for objdump for example) about Section Headers columns.
For example what are 'ES', 'Lk' and 'Info' ?
What are all available flags in 'Flg' ?
'Al' means alignment ?
Section Headers:
[Nr] Name Type Addr Off Size ES Flg Lk Inf Al
[ 0] NULL 00000000 000000 000000 00 0 0 0
[ 1] .data PROGBITS 3ffe8000 000120 0004fa 00 WA 0 0 16
[ 2] .rodata PROGBITS 3ffe8500 000620 000ea4 00 A 0 0 16
[ 3] .bss NOBITS 3ffe93a8 0014c8 0089d0 00 WA 0 0 16
...
...
Thanks in advance,
Here is a link to the System V Application Binary Interface where is a section header defined:
typedef struct {
Elf64_Word sh_name;
Elf64_Word sh_type;
Elf64_Xword sh_flags;
Elf64_Addr sh_addr;
Elf64_Off sh_offset;
Elf64_Xword sh_size;
Elf64_Word sh_link;
Elf64_Word sh_info;
Elf64_Xword sh_addralign;
Elf64_Xword sh_entsize;
} Elf64_Shdr;
es is actually sh_entsize, it specifies the size of an entry that the header contains (example: size of the symbol forSHT_SYMTAB and SHT_DYNSYN sections)
lk is sh_link, info is sh_info and their meaning depends on the section type
(example: sh_link contains section header index with associated string table for SHT_SYMTAB and SHT_DYNSYN sections, sh_info is usually also just header table index)
These are flags defined by the standard:
SHF_WRITE 0x1
SHF_ALLOC 0x2
SHF_EXECINSTR 0x4
SHF_MERGE 0x10
SHF_STRINGS 0x20
SHF_INFO_LINK 0x40
SHF_LINK_ORDER 0x80
SHF_OS_NONCONFORMING 0x100
SHF_GROUP 0x200
SHF_TLS 0x400
SHF_COMPRESSED 0x800
and as you wrote al is sh_align.
If you want precise description of the structure check the link.

Why is the compiler adding an extra 'sxtw' instruction (resulting further in a kernel panic)?

Issue/Symptom:
At the end of a function return, the compiler adds an sxtw instruction as seen in the disassembly, resulting in a return address of only 32 bits instead of 64 bits, resulting in a kernel panic:
Unable to handle kernel paging request at virtual address xxxx
Build Environment:
Platform : ARMV7LE
gcc, linux-4.4.60
Archictecture : arm64
gdb : aarch64-5.3-glibc-2.22/usr/bin/aarch64-linux-gdb
Details:
Here's the simplified project structure. It's been taken care of correctly in the corresponding makefile. Also note that file1.c and file2.c are part of same module.
../src/file1.c /* It has func1() defined as well as called /
../src/file2.c
../inc/files.h / There's no func1() declared in the header */
Cause of the issue:
A call to the func1() was added from the file2.c w/o func1 declaration in files.h or file2.c. (Basically the inclusion of func1 was accidentally missed in the files.h.)
Code compiled with no errors, but a warning as expected -- Implicit declaration of function func1.
At run time though, right after returning from func1 inside file2, the system crashed as it tried de-referencing the returned address from func1.
Further analysis showed that at the end of a function return, the compiler added an sxtw instruction as seen in the disassembly, resulting in a return address of only 32 bits instead of 64 bits, resulting in a kernel panic.
Unable to handle kernel paging request at virtual address xxxx
Note that x19 is of 64 bit while w0 is of 32 bit.
Note that x0 LS word matches with that of x19.
System crashed while de-referencing x19.
sxtw x19, w0 /* This was added by compiler as extra instruction /
ldp x1, x0, [x19,#304] / System crashed here */
Registers:
[ 91.388130] pc : [<ffffff80016c9074>] lr : [<ffffff80016c906c>] pstate: 80000145
[ 91.462090] sp : ffffff80094333b0
[ 91.552708] x29: ffffff80094333d0 x28: ffffffc06995408a
[ 91.652701] x27: ffffffc06c400a00 x26: 0000000000000000
[ 91.716243] x25: 0000000000000000 x24: ffffffc069958000
[ 91.779784] x23: ffffffc076e00000 x22: ffffffc06c400a00
[ 91.843326] x21: 0000000000000031 x20: ffffffc073060000
[ 91.906867] x19: 0000000066bfc780 x18: ffffff8009436888
[ 91.970409] x17: 0000000000000000 x16: ffffff8008193074
[ 92.033952] x15: 00000000000a8c06 x14: 2c30323030387830
[ 92.097492] x13: 3d7367616c66202c x12: 3038653030303030
[ 92.161034] x11: 3038666666666666 x10: 78303d646e65202c
[ 92.224576] x9 : 3063303030303030 x8 : 3030303030303030
[ 92.288117] x7 : 0000000000000880 x6 : 0000000000000000
[ 92.351659] x5 : ffffffc07fd10ad8 x4 : 0000000000000001
[ 92.415202] x3 : 0000000000000007 x2 : cb88537fdc8ba63c
[ 92.478743] x1 : 0000000000000000 x0 : ffffffc066bfc780
After adding the declaration of func1 in the files.h, the extra instruction and hence the crash was not seen.
Can someone please explain why the compiler added sxtw in this case?
You should have received at least two warnings, one about the missing function declaration and another one about the the implicit conversion from int to a pointer type.
The reason is that implicitly declared functions have a return type of int. Casting this int value to a 64-bit pointer throws away 32 bit of the result. This is the expected GNU C behavior, based on what C compilers for early 64-bit targets did. The sxtw instruction is required to implement this behavior. (Current C standards no longer have implicit function declarations, but GCC still has to support them for backwards compatibility with existing autoconf tests.)
Note that your platform is obviously Aarch64 (with 64-bit registers), not 32-bit ARMv7.

usb_control_msg returns -EAGAIN

I have a Customized board which interfaces over USB.. I am writing USB Linux driver.
Everything is working fine when I test it on my Virtual Machine.. But when I switch to the real hardware and use Yocto on hardware and run the following code.. It fails with -EAGAIN..
retval = usb_control_msg(serial->dev,
usb_rcvctrlpipe(serial->dev, 0),
CP210X_GET_MDMSTS,
USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE,
0,
i,
(void *)&modem_status,
1,
USB_CTRL_SET_TIMEOUT);
retval returns with -11. I am not sure why this is not happening on the Virtual Machine, as the only difference is that the PIC Board gets connected to X86 Customized board and runs linux..
dmesg output:
transfer buffer not dma capable
------------[ cut here ]------------
WARNING: CPU: 1 PID: 754 at /usr/src/kernel/drivers/usb/core/hcd.c:1595 usb_hcd_map_urb_for_dma+0x3e6/0x5b0
Modules linked in: cp2108(O)
CPU: 1 PID: 754 Comm: test_quad Tainted: G O 4.14.68-intel-pk-standard #1
task: ffff9a33b7d4a4c0 task.stack: ffff9ce5c0130000
RIP: 0010:usb_hcd_map_urb_for_dma+0x3e6/0x5b0
RSP: 0018:ffff9ce5c0133b08 EFLAGS: 00010296
RAX: 000000000000001f RBX: ffff9a33b7d89780 RCX: 0000000000000000
RDX: ffff9a33bfc9d680 RSI: ffff9a33bfc95598 RDI: ffff9a33bfc95598
RBP: ffff9ce5c0133b28 R08: 0000000000000001 R09: 0000000000000328
R10: ffff9a33ba840068 R11: 0000000000000000 R12: ffff9a33ba2ea000
R13: 00000000fffffff5 R14: 0000000001400000 R15: 0000000000000200
FS: 00007fac9eeed4c0(0000) GS:ffff9a33bfc80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000003c334e3cb0 CR3: 0000000179498000 CR4: 00000000003406e0
Call Trace:
usb_hcd_submit_urb+0x420/0xa00
? tty_port_open+0xa7/0xd0
? tty_ldisc_unlock+0x1a/0x20
? tty_open_proc_set_tty+0x7f/0xb0
? tty_unlock+0x29/0x40
? tty_open+0x38e/0x450
usb_submit_urb+0x364/0x550
usb_start_wait_urb+0x5f/0xe0
usb_control_msg+0xc5/0x110
cp210x_ioctl+0x4d2/0xe20 [cp2108]
? filemap_map_pages+0x129/0x290
? do_filp_open+0xa0/0xf0
serial_ioctl+0x46/0x50
tty_ioctl+0xe7/0x870
do_vfs_ioctl+0x99/0x5e0
? putname+0x4c/0x60
SyS_ioctl+0x79/0x90
Can you guys please provide a hint for me to try.. Appreciate your time and efforts.
I got the solution Kernel >= 4.9 no longer accepts any statically allocated buffer.
Modified the code to use dynamic memory and it worked.
modem_status = kmalloc(sizeof(unsigned long), GFP_KERNEL);
if (!modem_status)
return -ENOMEM;
for (i = 0; i < MAX_CP210x_INTERFACE_NUM; i++) {
retval = usb_control_msg(serial->dev,
usb_rcvctrlpipe(serial->dev, 0),
CP210X_GET_MDMSTS,
USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_INTERFACE,
0,
i,
(void *)modem_status,
1,
USB_CTRL_SET_TIMEOUT);

SO_ATTACH_REUSEPORT_CBPF socket option unexpected behavior

I am trying use a port from two applications and have each of them receive the packet from a different set of IP addresses. In order to achieve this, I use the SO_REUSEPORT and SO_ATTACH_REUSEPORT_CBPF socket options. My code is as follows:
parentfd = socket(AF_INET, SOCK_STREAM, 0);
if (parentfd < 0)
error( "ERROR opening socket");
struct sock_filter code[]={
{ 0x28, 0, 0, 0x0000000c },
{ 0x15, 0, 3, 0x00000800 },
{ 0x20, 0, 0, 0x0000001a },
{ 0x15, 2, 0, 0xc0a8ff01 },
{ 0x6, 0, 0, 0x00000000 },
{ 0x6, 0, 0, 0x00040000 },
{ 0x6, 0, 0, 0x00000001 },
};
struct sock_fprog bpf = {
.len = ARRAY_SIZE(code),
.filter = code,
};
if (setsockopt(parentfd, SOL_SOCKET, SO_REUSEPORT, (const void *)&optval,sizeof(optval)))
error("ERROR setting SO_REUSEPORT");
if (setsockopt(parentfd, SOL_SOCKET, SO_ATTACH_REUSEPORT_CBPF, (const void *)&bpf, sizeof(bpf)))
error("ERROR setting SO_ATTACH_REUSEPORT_CBPF);
I also have a different process that listens to the same port using only the SO_REUSEPORT flag. From a machine with IP 192.168.255.1 I am running echo 1234 | ncat 192.168.255.150 1234. Based on my filter I would expect all traffic from that IP address to be received by the second process. However, it is all received by the first. When I change the filter to a simple:
struct sock_filter code[]={
{ 0x6, 0, 0, 0x00000001 },
};
It works as expected and all packets are received by the second process. Any idea why this might be happening?
I found out what the problem was. The filter is applied to all packets, even the TCP handshake packets. Also, the base pointer points to the first byte of the packet payload, not headers. Hence, when it executes
ldh[12]
it gets out of the limits of the packet (SYN packet has 0 bytes of payload) and the default behavior is to return 0.
The non-working code is:
l0: ldh [12] /* read EtherType (2 bytes), which is found at offset 12 (decimal) */
l1: jeq #0x800, l2, l5 /* if EtherType == `0x800` (IPv4), jump to `l2`, otherwise jump to `l5` */
l2: ld [26] /* read source IP address (4 bytes) */
l3: jeq #0xc0a8ff01, l6, l4 /* if source IP address == 192.168.255.1, jump to l6 (return 1), else jump to l4 (return 0) */
l4: ret #0
l5: ret #0x40000
l6: ret #0x1
The working code is:
ret #0x1
socket (7) says:
The BPF program must return an index between 0 and N-1 representing the socket which should receive the packet (where N is the number of sockets in the group). If the BPF program returns an invalid index, socket selection will fall back to the plain SO_REUSEPORT mechanism.
On my machine tcpdump -i lo -ddd 'src host 192.168.255.1' produces
10
40 0 0 12
21 0 2 2048
32 0 0 26
21 4 5 3232300801
21 1 0 2054
21 0 3 32821
32 0 0 28
21 0 1 3232300801
6 0 0 262144
6 0 0 0
Which is
l0: ldh [12]
l1: jeq #0x800, l2, l4
l2: ld [26]
l3: jeq #0xc0a8ff01, l8, l9
l4: jeq #0x806, l6, l5
l5: jeq #0x8035, l6, l9
l6: ld [28]
l7: jeq #0xc0a8ff01, l8, l9
l8: ret #0x40000
l9: ret #0
I fail to see anything that's obviously wrong with your code.
Have you tried running tcpdump on the server? Perhaps you've forgotten to remove an extra IP address on the client, or there's a forgotten SNAT rule somewhere?
Which kernel version are you running? Could you post a minimal C application that reproduces the problem?

Resources