flex/bison fixing memory leaks with unexpected tokens - memory-leaks

I have a flex bison application. For a few of my tokens, I copy out the yytext from flex using strdup. This works great except when there is an error of an unexpected token.
simple example
flex.l:
...
[a-zA-Z0-9]+ { lval.string = strdup(yytext); return IDENT };
[\{\}] { return yytext[0] };
...
and
parse.y
...
%destructor { free($$); } IDENT
%destructor { free($$->name); free($$->type); free($$); } tag
...
tag: IDENT '{' IDENT '}'
{
struct tag *mytag = malloc(sizeof(struct tag));
mytag->name = $1;
mytag->type = $3;
$<tag>$ = mytag;
}
...
Now suppose I hand it the input:
blah blah blah
The lexer will send up the first IDENT token, which gets pushed onto the stack. After the first token it's expecting a bracket token, but instead gets another IDENT token. This is a syntax error. The destructor will be called on the first IDENT token, but not on the second one (the unexpected one). I haven't been able to find a way to destruct the unexpected token. Does anyone know how I should do it?

I found that appropriate use of the 'error' token in flex prompts it to correctly call the destructor function. Go me!
parse.y
...
%destructor { free($$); } IDENT
%destructor { free($$->name); free($$->type); free($$); } tag
...
tags: tag tags | error tags | ;
tag: IDENT '{' IDENT '}'
{
struct tag *mytag = malloc(sizeof(struct tag));
mytag->name = $1;
mytag->type = $3;
$<tag>$ = mytag;
}
...

Related

Having problems with ANTLR4 in Python and getting an ErrorListener to work

I am declaring my error listener like this:
class GeneratorErrorListener(ErrorListener):
def __init__(self, listener):
super().__init__()
self.listener = listener
def systaxError(self, recognizer, offendingSymbol, line, col, msg, e):
log_it("Syntax error at line {} col {}: {}".format(line, col, msg))
I am not yet making use of the listener passed in, but will when I get it working.
and setting it up like this:
...
# Set up new error listener
parser.removeErrorListeners()
parser.addErrorListener(GeneratorErrorListener(listener))
tree = parser.protocol()
...
walker.walk(listener, tree)
Then I am testing it with some input that has a syntax error (AFAICS):
The grammar fragment is:
enumEltDecl : INT '=' ID ( ':' STRING)?
| 'default' '=' STRING
;
enumDecl: 'enum' ID ( ':' ID )? '{' enumEltDecl (',' enumEltDecl )*
(',')? '}' ;
and I can parse those things fine. However, the following input which I think should be a syntax error, and does cause parsing to stop, does not invoke the error listener:
emum some_emum:uint8 {
};
It should have at least one enumEltDecl.
Any thoughts on what I have done wrong? I have looked at the runtime code for the ErrorListener class and it seems straightforward.
More Information
The code is here: https://gitlab.com/realrichardsharpe/wireshark-generator-python
Use the following steps to see the issue:
cd src
./GenTool.py -t C ../test-data/syntax-error.proto
You will see the following output:
#include "config.h"
#include <epan/packet.h>
#include <epan/expert.h>
//Generating code for enum cmd_enum
enum cmd_enum {
CMD1 = 0x14;
CMD2 = 0x15;
CMD3 = 0x28;
CMD4 = 0x29;
CMD5 = 0x3C;
CMD5 = 0x3D;
};
//We have a uint8
static const range_string cmd_enum_rvals[] = {
{ 0, 19, "Reserved", }
{ 0x14, 0x14, "cmd1" },
{ 0x15, 0x15, "cmd2" },
{ 22, 39, "Reserved", }
{ 0x28, 0x28, "cmd3" },
{ 0x29, 0x29, "cmd4" },
{ 42, 59, "Reserved", }
{ 0x3C, 0x3C, "cmd5" },
{ 0x3D, 0x3D, "cmd6" },
{ 62. 255, "Reserved" },
};
And it stops without my ErrorListener being called. The ErrorListener is in GenTool.py.
Strangely, with a little rearrangement of the code and after lots of debugging it now seems to be working because I get the following errors with a different set of input data:
./GenTool.py -t C xxx.proto
line 3:0 mismatched input '}' expecting {'default', INT}
line 4:0 missing ';' at '<EOF>'
Syntax error on line 1
The first two lines are generated by my error listener.
UPDATE: With a little comparison, I discovered that my test case had a symbol in it that is not recognized by the grammar and things went off the rails at that point.
The real problem was that my grammar was incorrect. The first line should have been:
protocol : protoDecl+ EOF ;
In my original grammer the EOF was missing which caused the parser to stop when it hit something that did not match the grammer.

Pass a static string to a macro_rule in rust

I wrote the following code :
macro_rules! my_macro{
("A") => {
println!("Macro called !")
}
}
fn main(){
static test: &'static str = "A";
my_macro!(test);
}
but I have the following error :
error: no rules expected the token `test`
--> test.rt:9:19
|
1 | macro_rules! my_macro{
| --------------------- when calling this macro
...
9 | my_macro!(test);
| ^^^^ no rules expected this token in macro call
error: aborting due to previous error
However, it works fine if I directly call my_macro("A"). Is it possible to fix this ?
Is it possible to fix this ?
No. Macros are expanded at compile time before item names are resolved, therefore your macro has no idea what the value of test is (and would have no idea even if it were a const rather than a static).
so the first problem here is that you macro expects a pattern of "A" not a variable that contains "A"
when you create macros you define certain patterns and follow those patterns in your case your macro must always have "A" in it but it is not a string a it is a pattern of double quote followed by capital a followed by another double quote
If you want to pass a value you should use variable syntax and define what it should expect such as ($a:expr)=>{...}
here you can see all magic tokens possible just scroll down a bit on that docs there are a lot of great examples
PS. here is a macro I use for responding from my endpoints
macro_rules! resp {
(ok) => {
|_| actix_web::HttpResponse::Ok().body(r#"{"success":true}"#)
};
(ok,$data:expr) => {
|_| actix_web::HttpResponse::Ok().json(serde_json::json!({"success":true,"data":$data}))
};
(ok,) => {
|d| actix_web::HttpResponse::Ok().json(serde_json::json!({"success":true,"data":d}))
};
}

How can I parse nested source files with ANTLR4 - Trying one more time

I found the code (reproduced below) in an article from Terrence Parr showing how INCLUDE files could be handled in ANTLR3 for Java. I tried to add this to a grammar I use with ANTLR4 (with a C++ target) but when I tried to generate a parser, I got the errors
error(50): : syntax error: '^' came as a complete surprise to me
error(50): : syntax error: mismatched input '->' expecting SEMI while matching a rule
error(50): : syntax error: '^' came as a complete surprise to me
error(50): : syntax error: '(' came as a complete surprise to me while matching rule preamble
and I have no idea what these error means. Can anyone explain and perhaps show me the way forward?
(NB: I'm not wild about polluting the grammar file with code, I'm using the visitor pattern but I'll take it if I can!)
Thanks
include_filename :
('a'..'z' | 'A'..'Z' | '.' | '_')+
;
include_statement
#init { CommonTree includetree = null; }
:
'include' include_filename ';' {
try {
CharStream inputstream = null;
inputstream = new ANTLRFileStream($include_filename.text);
gramLexer innerlexer = new gramLexer(inputstream);
gramParser innerparser = new gramParser(new CommonTokenStream(innerlexer));
includetree = (CommonTree)(innerparser.program().getTree());
} catch (Exception fnf) {
;
}
}
-> ^('include' include_filename ^({includetree}))
;
Starting with ANTLR4 it is no longer possible to manipulate the generated parse tree with grammar rules. In fact ANTLR3 generated an AST (abstract syntax tree), which is a subset of a parse tree (as generated by ANTLR4). That in turn means you cannot keep the tree rewrite syntax (the part starting with ->). Hence you should change the code to:
include_statement
#init { CommonTree includetree = null; }
:
'include' Include_filename ';' {
try {
CharStream inputstream = null;
inputstream = new ANTLRFileStream($include_filename.text);
gramLexer innerlexer = new gramLexer(inputstream);
gramParser innerparser = new gramParser(new CommonTokenStream(innerlexer));
includetree = (CommonTree)(innerparser.program().getTree());
} catch (Exception fnf) {
;
}
}
;

Class parameter syntax errors

I am trying to learn to write puppet modules in a good way, so I've started looking around for tutorials and howto.
I've seen that users suggest writing the main class in the following way, but It's actually failing for me.
I am honestly a bit confused how the 2 blocks between brackets are actually connected, and so I might be not seeing an obvious error or real missing comma.
I am running Puppet 3.8 by the way
class icinga2 {
$version = 'present'
$enable = true
$start = true
} {
class{'icinga2::install': } ->
class{'icinga2::config': } ~>
class{'icinga2::service': } ->
Class["icinga2"]
}
Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Syntax error at '{'; expected '}' at /etc/puppet/modules/icinga2/manifests/init.pp:5
Your problem here is that your parameters must be surrounded by (), not {}. Also, they should be commas separated.
class icinga2 (
$version = 'present',
$enable = true,
$start = true,
) {
class{'icinga2::install': } ->
class{'icinga2::config': } ~>
class{'icinga2::service': } ->
Class["icinga2"]
}

Error 'cannot move out of dereference' when trying to match strings from vector

I am very new to rust and trying to write a command line utility as a way to learn.
I am getting the list of args and trying to match on them
let args = os::args()
//some more code
match args[1].into_ascii_lower().as_slice() {
"?" | "help" => { //show help },
"add" => { //do other stuff },
_ => { //do default stuff }
}
this causes this error
cannot move out of dereference (dereference is implicit, due to indexing)
match args[1].into_ascii_lower().as_slice() {
^~~~~~~
I have no idea what that means, but searching yield this which I didn't completely get, but changing the args[1] to args.get(1) gives me another error
error: cannot move out of dereference of `&`-pointer
match args.get(1).into_ascii_lower().as_slice() {
^~~~~~~~~~~
what's going on?
As you can see in the documentation, the type of into_ascii_lower() is (see here) :
fn into_ascii_upper(self) -> Self;
It takes self directly, not as a reference. Meaning it actually consumes the String and return an other one.
So, when you do args[1].into_ascii_lower(), you try to directly consume one of the elements of args, which is forbidden. You probably want to make a copy of this string, and call into_ascii_lower() on this copy, like this :
match args[1].clone().into_ascii_lower().as_slice() {
/* ... */
}

Resources