SCons generates an incomplete dependency tree if I use a custom decider - scons

Question

I'm trying to build a flex-bison parser with SCons, using a custom decider. (You can learn more about the decider in this question, although, the details probably aren't too relevant to the current problem.)
Files:
My SConstruct:
env = DefaultEnvironment()
deciderEnv = env.Clone()
def source_and_target_decider(dependency, target, prev_ni, repo_node=None):
src_old_csig = prev_ni.csig if hasattr(prev_ni, 'csig') else None
src_new_csig = dependency.get_csig()
tgt_stored_info = target.get_stored_info()
tgt_old_csig = tgt_stored_info.ninfo.csig if hasattr(tgt_stored_info.ninfo, 'csig') else None
tgt_new_csig = target.get_csig()
return src_new_csig != src_old_csig or tgt_new_csig != tgt_old_csig
deciderEnv.Decider(source_and_target_decider)
deciderEnv['YACCFLAGS'] = ['-d']
deciderEnv['YACCHXXFILESUFFIX'] = '.hh'
bison_source = deciderEnv.CXXFile('src/test.yy.cc', 'src/test.yy')[0]
deciderEnv['LEXFLAGS'] = [f'--header-file=${{SOURCE}}.hh']
flex_source = deciderEnv.CXXFile('src/test.ll.cc', 'src/test.ll')[0]
Program('test', [bison_source, flex_source])
src/test.yy:
%language "c++"
%code {
int yylex(int*);
}
%token X
%%
everything: X {}
and src/test.ll:
%{
#include "./test.yy.hh"
%}
%option c++
%%
. { return yy::parser::token::X; }
Result:
I run a command scons ./src/test.ll.o.
The first time you run it, it works correctly.
flex --header-file=src/test.ll.hh -t src/test.ll > src/test.ll.cc
bison -d -o src/test.yy.cc src/test.yy
g++ -o src/test.ll.o -c src/test.ll.cc
SCons seems to understand that it needs to generate src/test.yy.hh using bison before it compiles the src/test.ll.o because src/test.ll.cc #includes that header.
However, if I then remove .sconsign.dblite (leaving the rest of the files intact), SCons seems to lose that understanding for the duration of the next build.
Running scons ./src/test.ll.o yields:
flex --header-file=src/test.ll.hh -t src/test.ll > src/test.ll.cc
g++ -o src/test.ll.o -c src/test.ll.cc
and only after calling it a second time it decides to build the src/test.yy.hh:
bison -d -o src/test.yy.cc src/test.yy
g++ -o src/test.ll.o -c src/test.ll.cc
Dependency tree:
Normally the dependency tree (scons --tree=all src/test.ll.o) looks like this:
+-src/test.ll.o
+-src/test.ll.cc
| +-src/test.ll
| +-/bin/flex
+-src/test.yy.hh
| +-src/test.yy
| +-/bin/bison
+-/bin/g++
However, right after deleting the .sconsign.dblite it is incomplete:
+-src/test.ll.o
+-src/test.ll.cc
| +-src/test.ll
| +-/bin/flex
+-/bin/g++
Further observations:
The last line of the custom decider seems to be the culprit.
Specifically, the part or tgt_new_csig != tgt_old_csig. If you remove this part, the problem no longer occurs.
However, I have no clue why this particular piece of code breaks the build.
Any idea how to fix this?

Answer 1

The sconsign is where SCons stores info about previous builds.
If you remove it, then SCons will not have that info and will rebuild everything.
Because in your decider you are setting:
tgt_old_csig = tgt_stored_info.ninfo.csig if hasattr(tgt_stored_info.ninfo, 'csig') else None
and then comparing None to the current csig, which will not be equal and require a rebuild.
You'd likely need to change as follows to work when you're explicitly removing the source of the tgt_stored_info (the sconsign file).
def source_and_target_decider(dependency, target, prev_ni, repo_node=None):
src_old_csig = prev_ni.csig if hasattr(prev_ni, 'csig') else None
src_new_csig = dependency.get_csig()
tgt_stored_info = target.get_stored_info()
tgt_old_csig = tgt_stored_info.ninfo.csig if hasattr(tgt_stored_info.ninfo, 'csig') else None
tgt_new_csig = target.get_csig()
if tgt_old_csig:
return src_new_csig != src_old_csig or tgt_new_csig != tgt_old_csig
else:
return src_new_csig != src_old_csig
Why are you removing .sconsign.dblite?