I have some issues with the behavior of the wildcard function of GNU Make with respect to terminating slashes in the pattern and the output.
Consider the following simple directory structure:
dir
|
+-- file
|
+-- subdir
On Linux,
$(wildcard dir/*/) # (1)
evaluates with GNU Make 4.1 to
dir/subdir/ dir/file
but with GNU Make 4.3 to
dir/subdir/
One could argue whether including the regular file filein the former case is a bug or a feature (names of directories but not those of regular files are terminated with a slash). However, both versions of GNU Make evaluate
$(wildcard $(addsuffix /,$(wildcard dir/*))) # (2)
to
dir/file dir/subdir/
(subject to sorting). In particular, $(wildcard dir/file/) evaluates to dir/file. This is more in the spirit of the above GNU Make 4.1 feature but seems to be somewhat inconsistent with respect to GNU Make 4.3.
What can I assume from the wildcard function regarding a terminating slash in the pattern?
I would like to determine the content of a directory such that the names of subdirectories are terminated by a slash while the names of regular files are not. In GNU Make 4.1 I used approach 1 which broke my build with GNU Make 4.3. In both cases I could use approach 2. But is this feasible or do I rely on undefined behavior here? If so, what would be the correct (and efficient) way to do what I want?
The problem is not simple. The short answer is that the behavior of GNU make 4.3 is correct for the expansion of dir/*/ and the behavior of earlier versions of make that don't agree with that, are wrong.
As for the behavior of dir/file/ that seems to me to be wrong in all versions of GNU make; that is, it should return the empty string.
However, GNU make doesn't actually implement its own file globbing, at least not on systems that provide the GNU libc C runtime library, which is most Linux systems. It simply calls the system-provided glob(3) function. I wrote a small test program that simply calls GNU libc's glob(3) function directly and it gives the same behavior as GNU make 4.3:
dir/*/ -> dir/subdir/
dir/file/ -> dir/file/
In my opinion this is a bug in GNU libc's glob(3) but perhaps I'm missing some subtlety here.
In any event, if what you really want is just directories then the best/safest/works everywhere solution is to use this:
$(wildcard dir/*/.)
then you don't have to worry about magical behaviors related to trailing slashes.
The function wildcard-rec in the GNUmake table toolkit does exactly what you want. It distinguishes between files and directories via a obvious feature: if the given glob ends in / then you want directories, if the / is absent you want files.
include gmtt.mk
$(info $(call wildcard-rec,**.c)) # all C source files in the tree
$(info $(call wildcard-rec,**.c **.h)) # a C source and header files
$(info $(call wildcard-rec,drivers/**.c)) # only sources for the `drivers` tree
$(info $(call wildcard-rec,drivers/**/test/)) # all test subdirectories in the `drivers` tree
$(info $(call wildcard-rec,drivers/**/test/*.cfg)) # config files in all test subdirectories in the `drivers` tree
Related
I have a problem when including dependency folder as this isn't looking for headers recursively.
FOLDER STRUCTURE:
- main.cpp
- dependency
- sub1
- header1.h
- sub2
- header2.h
- root-header.h
main.cpp
#include "root-header.h"
#include "header1.h"
#include "header2.h"
int main() {
}
Command:
clang main.cpp -I"dependency"
Error:
fatal error: 'header1.h' file not found
The command only detects header.h inside dependency folder to one level, how to make the clang to recursively lookup for all headers inside dependency folder. Is there any compiler arguments to be added?
Thanks
The ISO/IEC 9899:2011 standard in section §6.10.2 explains the expected behavior of clang and other compilers:
# include <h-char-sequence> new-line
searches a sequence of implementation-defined places for a header identified uniquely by the specified sequence between the < and > delimiters, and causes the replacement of that directive by the entire contents of the header. How the places are specified or the header identified is implementation-defined.
You can modify the defined places by adding additional with the -I option, but a compiler should not search sub-directories.
You can work around this limitation in the spec by using make to compile a list of additional -I locations to add to you clang command. This is covered in #DanBonachea answer.
Instead, I'd advise you to change the includes to be compliant to the specification:
#include "sub1/header1.h"
#include "sub2/header2.h"
The conventional solutions are one of the following:
1. Change the include directives in the source code
This solution compiles with clang++ -Idependency main.cpp but modifies #include directives to include headers by subdirectory, eg:
#include "sub1/header1.h"
#include "sub2/header2.h"
This is obviously a modification to the code, so usually only makes sense if sub1 and sub2 are meaningful within the larger structure of the software (e.g. package names that are always the same). Or...
2. Use shell tools to traverse the directory and build the include path
This solution uses find to inject subdirectories on the include path, eg:
$ clang++ `find ./dependency -type d -exec echo -I'{}' \;` main.cpp
which scans to identify the subdirectories and adds them to the preprocessor include path.
Discussion
Both of these approaches should work with few changes with basically any C/C++ compiler on UNIX (incl Linux, macOS, WSL, etc).
Note the second approach above will involve some additional filesystem churn on every compilation, which might be noticeable if the number of subdirectories is very large. To be fair this cost is fundamental to that use case, and even if built-in support for recursive include existed in the compiler frontend, it would still need to perform a similarly expensive recursive directory traversal on every compilation to find all the files.
3. Amortize directory traversal
However we can improve upon the second solution if we assume all the headers that will be included from this directory structure have unique names. This is a reasonable assumption, because otherwise the unqualified #include directives inside the source files will be ambiguous, leading to orthogonal problems. With this assumption in hand, we can create a cache to amortize the cost of the dependency directory traversal as follows:
$ mkdir allheaders ; cd allheaders
$ find ../dependency -type f -exec ln -s '{}' . \;
Then compilation simply becomes:
$ clang++ -Iallheaders main.cpp
Or, if you additionally want to support a mix of option 1 and option 3 #include directives, then:
$ clang++ -Idependency -Iallheaders main.cpp
This approach could greatly accelerate compilation, because the preprocessor only needs to open one user directory and open the files by basename. The fact that the directory may contain a large number of headers (with some fraction potentially unused) should not significantly degrade performance, thanks to how filesystems work.
If we further assume the file names in the dependency directory change infrequently or never, then we only need to execute the directory traversal step once, and can amortize that cost against repeated compilation using the allheaders cache directory.
Currently in my Makefile I have:
prefix ?= /usr/local
So that I can override prefix value when calling make, like in the following:
make prefix="/new_path"
My question is: does a convention exist for naming this variable inside a Makefile, especially do I have to call it prefix or PREFIX?
This matter since lower or upper case matters in this situation!
Note that I do not use autotools, just a "simple" Makefile
The variables for installation directories section of the GNU make manual discusses prefix.
I believe the autotools use a similarly-cased prefix variable/configure flag.
Compare to DESTDIR however.
I have a very simple Makefile that isn't doing what I expect it would do. The ultimate goal is that it should call itself recursively, including the appropriate file each time, resulting in a build specific to what was included (I'm building several projects that all share the same code base, but utilize different combinations of the source files). I've never really dealt with recursive calls to make, so I must be missing something obvious. At the moment, I only have one .mk file in the same folder as my Makefile. It's a simple one-liner just for the purposes of this test. It will eventually contain various per-project settings.
Makefile:
SHELL = /bin/sh
ifdef MYFILE
include $(MYFILE)
PROGRAM = $(basename $(MYFILE))
endif
all: $(wildcard *.mk)
dummy:
#echo -- Entering dummy stub ... why do I need this?
%.mk: dummy
#echo Calling $(MAKE) MYFILE=$# $*
$(MAKE) MYFILE=$# $*
$(PROGRAM): objs
#echo Time to link!
objs:
#echo Building objs!
test.mk
SOMEVAR = SomeValue
I have the following two problems:
Problem 1
If I remove the dummy prerequisite from my pattern rule, the pattern rule never gets called (I get the dreaded 'Nothing to be done for all' error). Is there a way I can get the recipes under the %.mk rule to run without needing that dummy prerequisite?
Problem 2
Given the two aforementioned files, I would expect make to do the following:
make[1] starts and hit the all rule
make[1] jumps down to the %.mk pattern rule
make[1] calls itself recursively (the call would look like make MYFILE=test.mk test)
make[2] starts, includes the test.mk file, and sets up the PROGRAM variable
make[2] jumps down to the $(PROGRAM) rule (since we were explicitly called with that target)
make[2] jumps to the objs rule, runs the recipes, and returns back up the chain
In actuality, make gets stuck on the %.mk pattern rule and enters an infinite loop. I don't understand why it's insisting on hitting the pattern rule, when I explicitly told it to build test in my first recursive call (which should correspond to the $(PROGRAM) target). What am I missing here?
Problem 0:
This is overdesigned. You don't need to use recursive Make here.
Problem 1:
The reason Make doesn't try to rebuild test.mk (without a dummy preq) is that test.mk is up to date. A better approach is to switch to a static pattern rule and use PHONY:
MKS = $(wildcard *.mk)
.PHONY: $(MKS)
$(MKS): %.mk:
#echo Calling $(MAKE) MYFILE=$# $*
$(MAKE) MYFILE=$# $*
An even better approach is not to use the name of a real file as a target of a rule that doesn't rebuild (or even "touch") that file.
Problem 2:
In make[2], the makefile includes test.mk. If a makefile includes another file, Make will attempt to rebuild that file before doing anything else. If there is a rule for that file (which there is) and if it succeeds (which it does) Make then reinvokes itself.
You should reconsider this design from the ground up. There are many ways to get the behavior you're looking for, depending on the specifics (how many variable will be defined in a foo.mk? do you really want to manage the build by manually moving those files around? and so on).
P.S. Here's one kludge that springs to mind. Whether it suits your case depends on the specifics:
makefile:
# includes nothing
%.mk: dummy
#echo Calling $(MAKE) MYFILE=$# -f $# $*
$(MAKE) MYFILE=$# -f $# $*
test.mk:
SOMEVAR = SomeValue
include makefile
As you know the getch() and getche() functions don't work with the cygwin, a linux oriented one.
But can I include the conio.h header file of borland c and call the functions getch in my makefiles?
Will it work and can anyone tell me how to include the header files from different directories in cywgin.
I have a header file strcal.h in directory c:/makk/string/.
How do I include that header file in my makefile?
gcc -I/string small.c
It is not working and my current directory is makk.
In stdio.h, there is a getchar() function which is what you need. You can't just bring across the Borland header file since that just declares the function, it doesn't define it. Standard C has no need for getch().
To include header files in different areas, you use the -I directives of gcc to set up search paths.
So, if you have a /xyz/myheader.h file, you can do something like:
gcc -I /xyz myprogram.c
To get at c:/makk/string/strcal.h, you may have to use gcc -I /cygdrive/c/makk/string or, if you know you're actually in that makk directory, you can use -I string (note the lack of leading / since you want a relative path, not an absolute one).
According to g++ -print-search-dirs my C++ compiler is searching for libraries in many directories, including ...
/lib/../lib/:
/usr/lib/../lib/:
/lib/:
/usr/lib/
Naively, /lib/../lib/ would appear to be the same directory as /lib/ — lib's parent will have a child named lib, "that man's father's son is my father's son's son" and all that. The same holds for /usr/lib/../lib/ and /usr/lib/
Is there some reason, perhaps having to do with symbolic links, that g++ ought to be configured to search both /lib/../lib/ and /lib/?
If this is unnecessary redundancy, how would one go about fixing it?
If it matters, this was observed on an unmodified install of Ubuntu 9.04.
Edit: More information.
The results are from executing g++ -print-search-dirs with no other switches, from a bash shell.
Neither LIBRARY_PATH nor LPATH are output from printenv, and both echo $LPATH and echo LIBRARY_PATH return blank lines.
An attempt at an answer (which I gathered from a few minutes of looking at the gcc.c driver source and the Makefile environment).
These paths are constructed in runtime from:
GCC exec prefix (see GCC documentation on GCC_EXEC_PREFIX)
The $LIBRARY_PATH environment variable
The $LPATH environment variable (which is treated like $LIBRARY_PATH)
Any values passed to -B command-line switch
Standard executable prefixes (as specified during compilation time)
Tooldir prefix
The last one (tooldir prefix) is usually defined to be a relative path:
From gcc's Makefile.in
# Directory in which the compiler finds libraries etc.
libsubdir = $(libdir)/gcc/$(target_noncanonical)/$(version)
# Directory in which the compiler finds executables
libexecsubdir = $(libexecdir)/gcc/$(target_noncanonical)/$(version)
# Used to produce a relative $(gcc_tooldir) in gcc.o
unlibsubdir = ../../..
....
# These go as compilation flags, so they define the tooldir base prefix
# as ../../../../, and the one of the library search prefixes as ../../../
# These get PREFIX appended, and then machine for which gcc is built
# i.e i484-linux-gnu, to get something like:
# /usr/lib/gcc/i486-linux-gnu/4.2.3/../../../../i486-linux-gnu/lib/../lib/
DRIVER_DEFINES = \
-DSTANDARD_STARTFILE_PREFIX=\"$(unlibsubdir)/\" \
-DTOOLDIR_BASE_PREFIX=\"$(unlibsubdir)/../\" \
However, these are for compiler-version specific paths. Your examples are likely affected by the environment variables that I've listed above (LIBRARY_PATH, LPATH)
Well, theoretically, if /lib was a symlink to /drive2/foo, then /lib/../lib would point to /drive2/lib if I'm not mistaken. Theoretically...
Edit: I just tested and it's not the case - it comes back to /lib. Hrm :(