Using:
Linux as build host (kubuntu 14.04)
Gnu make 3.81
Compiling some C/C++ projects
I have a directory tree like this:
Repository/
Framework/
Source/
Subdir1/
Subdir2/
Subdir3/
et cetera
something more
Projects/
Project1/
Source/
SubdirA/
SubdirB/
SubdirX/
et cetera
Out/ ← subdirs structure below Out/ is for example
Source/
SubdirA/
SubdirB/
SubdirX/
Framework/
Source/
Subdir1/
Subdir2/
Subdir3/
Makefile
Project2/
Source/
...
Out/
...
Makefile
et cetera
The content of the Framework/Source directory is just a connection of some general purpose source files which will be used in a couple of projects. The Framework directory has no own makefile and e.g. won't build a lib. It's just, that the projects uses some of the sourcecode from the Framework directory.
When building Project1, I first cd into the related projects dir and then calling make, e.g.:
cd ~/Repository/Projects/Project1
make
And all build output has to be put into Project1/Out directory.
Below the Out directory, my makefile mirrors the directory hierarchy from the source tree(s).
Within my makefile, I do something like this:
SRCS += $(shell find Source -name "*.c")
SRCS += $(shell find ../../Framework/Source -name "*.c")
BUILDDIR := Out
OBJS = $(addprefix $(BUILDDIR)/, $(addsuffix .o, $(basename $(SRCS))))
$(BUILDDIR)/%.o : %.c
$(CC) $(CFLAGS) -c $< -o $#
and this causes the problem that during the pattern rule and the ../.. part, some of the generated object files will be placed outside of my build output directory, e.g.:
gcc -c ../../Framework/Source/*.c -o Out/../../Framework/Source/*.o
My first approach was, to force all source filenames to absolute path names like this:
OBJS = $(addprefix $(BUILDDIR)/, $(addsuffix .o, $(basename $(abspath $(SRCS)))))
This works (even with realpath), but it makes my build output quite ugly because of the long path + filename outputs.
I have a second approach, were I just substitute all ../ parts of the .o files with something like up/ and it looks like this:
OBJS = $(addprefix $(BUILDDIR)/, $(addsuffix .o, $(subst ../,up/,$(basename $(SRCS)))))
but then it unfortunately seems. that I have to duplicate all my pattern rules like:
$(BUILDDIR)/%.o : %.c
$(CC) $(CFLAGS) -c $< -o $#
$(BUILDDIR)/up/up/%.o : ../../%.c
$(CC) $(CFLAGS) -c $< -o $#
So, I'm still looking for a slightly nicer solution for this problem.
For some reasons, I have the following restrictions:
no symlinks
no recursive make calls
no use of vpath
no libs
no git submodules ...
Any suggestions for a bit more elegant approach?
Here is the solution:
COLLAPSED_SRC_PREFIXES_IN_BLD := ../../
SRCS += $(shell find Source -name "*.c")
SRCS += $(shell find ../../Framework/Source -name "*.c")
BUILDDIR := Out
.PRECIOUS: %/.
%/.:
mkdir -p $#
define BLD_FROM_TO
$2/%.o: $1%.c Makefile | $$$$(#D)/.
$(CC) $(CFLAGS) $$< $$#
endef
.SECONDEXPANSION:
$(foreach prefix, $(COLLAPSED_SRC_PREFIXES_IN_BLD), $(eval $(call BLD_FROM_TO,$(prefix), $(BUILDDIR))))
$(eval $(call BLD_FROM_TO,, $(BUILDDIR)))
My original answer is preserved below for reference, but the consensus of discussion seems to be that my answer is wrong. When you have been around long enough, as I have, there is a certain "ring of truth" you can hear in certain criticism. One just ignores criticism that lacks the ring of truth, but this criticism has that ring.
The odd thing is that I have been using recursive Make for years in my own work, in a properly refactored manner (so that I do not, in effect, keep multiple instances of more or less the same rule) with the help of include and symlinks to makefiles (and of course by letting my compiler automatically emit the header dependencies), and it all seems to work fine for me. It is not easy to set up, and there is some hassle whenever I open a new source subdirectory; but then I had believed that Make with multiple source directories just wasn't going to be easy to set up, so there I am.
I do not know whether the OP has learned what he had sought to learn today, but I seem to have learned something.
I withdraw my answer. Thank you for having read my answer and having commented on it.
ORIGINAL ANSWER
Makefiles always seem to be hard. There is no definite, unalterable rule, but there are some practices that help.
In general, a file is best made by a makefile, not in the directory of the file's prerequisites, but in the directory in which the file is to be made. Thus, you can use several makefiles, one in each directory in which files are to be made. In your example, for instance, you can put a makefile in the Out/ directory, then move the $(BUILDDIR)/%.o: rule over to that makefile.
If you do not know it already, learn Make's -C option, and start using it extensively and consistently. There is nothing wrong with your cd ~/Repository/Projects/Project1 before make, exactly, unless the reason for the cd is that you didn't think to use -C, instead. Consistent use of -C, both on the command line and within makefile rules, eventually leads you to think in a more productive way about makefiles. It gradually clarifies your perception in the matter, which is good.
Also if you do not know it already, learn the use of Make's $(MAKE), for recursive invocation of Make.
Use $(MAKE) and -C together in your rules.
How exactly you assemble these elements into a solution that works for you depends on the details of what you want to do, but the above advice will tend to keep you from fighting Make's design philosophy. Not fighting will make things easier for you. Since Make's manual explains features, but does not explain the design philosophy underlying the features very well, this is important.
One could write whole chapters in the matter, so I'll stop here; but if there is a specific point upon which you would like me to expand, feel free to let me know.
Related
I'm creating a Rules.make file, similar to Linux's 2.0 version, which contains all kinds of targets - including .so files. My goal is to then only need to make minimalistic Makefiles like so:
include $(DIR_TOP)/Rules.make
in the directories that contain any source files I need compiled. The rules also enable me to create targets like so in the "main" Makefile:
something: something_else lib.so
, so that something_else is done first, and then lib.so is built.
Everything has been going smoothly, until I decided to add dependencies to the aforementioned shared library target. I figured something like the following would do the trick:
${DIR_OUT}/%.so: $(shell find $(dir $#) -name *.o)
$(CC) $(CFLAGS) -o $# $(CLIBS) -shared $^
However, to my demise, $(dir $#) apparently expands to $(dir ${DIR_OUT}/%.so), which then results in simply ${DIR_OUT}, which is exactly not what I need. DIR_OUT is simply the top-level directory string, but the target may be invoked from any sub-directories, or simply like target: $(DIR_OUT)/path/to/lib.so. I was hoping that % could match not only file names, but also any directories (which it does), and then have that expanded to $# once it's already decided what the full path is. Doesn't work like that. With this solution, not only the object files I need are included in the building process, but also any other object files that are there in the output folder, and that then produces errors of kind multiple definition of x y z etc.
Is there any other way to get the list of dependencies for the shared library I want to build? Ideally a purely Makefile based solution, but if there isn't one, I'm fond of some bash scripting too.
The solution turns out to be secondary expansion:
.SECONDEXPANSION:
${DIR_OUT}/%.so: $$(shell find $$(dir $$#) -name *.o)
$(CC) $(CFLAGS) -o $# $(CLIBS) -shared $^
I am wondering why gcc/g++ doesn't have an option to place the generated object files into a specified directory.
For example:
mkdir builddir
mkdir builddir/objdir
cd srcdir
gcc -c file1.c file2.c file3.c **--outdir=**../builddir/objdir
I know that it's possible to achive this with separate -o options given to the compiler, e.g.:
gcc -c file1.c -o ../builddir/objdir/file1.o
gcc -c file2.c -o ../builddir/objdir/file2.o
gcc -c file3.c -o ../builddir/objdir/file3.o
... and I know that I can write Makefiles via VPATH and vpath directives to simplify this.
But that's a lot of work in a complex build environment.
I could also use
gcc -c file1.c file2.c file3.c
But when I use this approach my srcdir is full of .o garbage afterwards.
So I think that an option with the semantics of --outdir would be very useful.
What is your opinion?
EDIT: our Makefiles are written in such a way that .o files actually placed into builddir/obj. But I am simply wondering if there might be a better approach.
EDIT: There are several approaches which place the burden to achieve the desired behavior to the build system (aka Make, CMake etc.). But I consider them all as being workarounds for a weakness of gcc (and other compilers too).
This is the chopped down makefile for one of my projects, which compiles the sources in 'src' and places the .o files in the directory "obj". The key bit is the the use of the patsubst() function - see the GNU make manual (which is actually a pretty good read) for details:
OUT = lib/alib.a
CC = g++
ODIR = obj
SDIR = src
INC = -Iinc
_OBJS = a_chsrc.o a_csv.o a_enc.o a_env.o a_except.o \
a_date.o a_range.o a_opsys.o
OBJS = $(patsubst %,$(ODIR)/%,$(_OBJS))
$(ODIR)/%.o: $(SDIR)/%.cpp
$(CC) -c $(INC) -o $# $< $(CFLAGS)
$(OUT): $(OBJS)
ar rvs $(OUT) $^
.PHONY: clean
clean:
rm -f $(ODIR)/*.o $(OUT)
How about changing to the directory and running the compile from there:
cd builddir/objdir
gcc ../../srcdir/file1.c ../../srcdir/file2.c ../../srcdir/file3.c
That's it. gcc will interpret includes of the form #include "path/to/header.h" as starting in the directory the file exists so you don't need to modify anything.
A trivial but effective workaround is to add the following right after the gcc call in your Makefile:
mv *.o ../builddir/objdir
or even a soft-clean (possibly recursive) after the compilation is done, like
rm -f *.o
or
find . -name \*.o -exec rm {} \;
You can use a simple wrapper around gcc that will generate the necessary -o options and call gcc:
$ ./gcc-wrap -c file1.c file2.c file3.c --outdir=obj
gcc -o obj/file1.o -c file1.c
gcc -o obj/file2.o -c file2.c
gcc -o obj/file3.o -c file3.c
Here is such a gcc_wrap script in its simplest form:
#!/usr/bin/perl -w
use File::Spec;
use File::Basename;
use Getopt::Long;
Getopt::Long::Configure(pass_through);
my $GCC = "gcc";
my $outdir = ".";
GetOptions("outdir=s" => \$outdir)
or die("Options error");
my #c_files;
while(-f $ARGV[-1]){
push #c_files, pop #ARGV;
}
die("No input files") if(scalar #c_files == 0);
foreach my $c_file (reverse #c_files){
my($filename, $c_path, $suffix) = fileparse($c_file, ".c");
my $o_file = File::Spec->catfile($outdir, "$filename.o");
my $cmd = "$GCC -o $o_file #ARGV $c_file";
print STDERR "$cmd\n";
system($cmd) == 0 or die("Could not execute $cmd: $!");
}
Of course, the standard way is to solve the problem with Makefiles, or simpler, with CMake or bakefile, but you specifically asked for a solution that adds the functionality to gcc, and I think the only way is to write such a wrapper. Of course, you could also patch the gcc sources to include the new option, but that might be hard.
I believe you got the concept backwards...?!
The idea behind Makefiles is that they only process the files that have been updated since the last build, to cut down on (re-)compilation times. If you bunch multiple files together in one compiler run, you basically defeat that purpose.
Your example:
gcc -c file1.c file2.c file3.c **--outdir=**../builddir/objdir
You didn't give the 'make' rule that goes with this command line; but if any of the three files has been updated, you have to run this line, and recompile all three files, which might not be necessary at all. It also keeps 'make' from spawning a seperate compilation process for each source file, as it would do for seperate compilation (when using the '-j' option, as I would strongly suggest).
I wrote a Makefile tutorial elsewhere, which goes into some extra detail (such as auto-detecting your source files instead of having them hard-coded in the Makefile, auto-determining include dependencies, and inline testing).
All you would have to do to get your seperate object directory would be to add the appropriate directory information to the OBJFILES := line and the %.o: %.c Makefile rule from that tutorial. Neil Butterworth's answer has a nice example of how to add the directory information.
(If you want to use DEPFILES or TESTFILES as described in the tutorial, you'd have to adapt the DEPFILES := and TSTFILES := lines plus the %.t: %.c Makefile pdclib.a
rule, too.)
Meanwhile I found a "half-way" solution by using the -combine option.
Example:
mkdir builddir
mkdir builddir/objdir
cd srcdir
gcc -combine -c file1.c file2.c file3.c -o ../builddir/objdir/all-in-one.o
this "combines" all source files into one single object file.
However, this is still "half-way" because it needs to recompile everything when only one source file changes.
I think that telling pass gcc doesn't have an separate option to say where to put object file, since it already has it. It's "-c" - it says in what directory to put object.
Having additional flag for directory only must change meening of "-c".
For example:
gcc -c file.c -o /a/b/c/file.o --put-object-in-dir-non-existing-option /a1/a2/a3
You can not put /a/b/c/file.o under /a1/a2/a3, since both paths are absolute. Thus "-c" should be changed to name object file only.
I advise you to consider a replacement of makefile, like cmake, scons and other.
This will enable to implement build system as for for simple project as well as for bigger one too.
See for example how it's easy to compile using cmake your example.
Just create file CMakeList.txt in srcdir/:
cmake_minimum_required(VERSION 2.6)
project(test)
add_library(test file1.c file2c file3.c)
And now type:
mkdir -p builddir/objdir
cd builddir/objdir
cmake ../../srcdir
make
That's all, object files will reside somewhere under builddir/objdir.
I personaly use cmake and find it very convinient. It automatically generates dependencies and has other goodies.
I am trying to figure out the same thing. For me this worked
CC = g++
CFLAGS = -g -Wall -Iinclude
CV4LIBS = `pkg-config --libs opencv4`
CV4FLAGS = `pkg-config --cflags opencv4`
default: track
track: main.o
$(CC) -o track $(CV4LIBS) ./obj/main.o
ALLFLAGS = $(CFLAGS) $(CV4FLAGS)
main.o: ./src/main.cpp ./include/main.hpp
$(CC) $(ALLFLAGS) -c ./src/main.cpp $(CV4LIBS) -o ./obj/main.o
``
This is among the problems autoconf solves.
If you've ever done ./configure && make you know what autoconf is: it's the tool that generates those nice configure scripts. What not everyone knows is that you can instead do mkdir mybuild && cd mybuild && ../configure && make and that will magically work, because autoconf is awesome that way.
The configure script generates Makefiles in the build directory. Then the entire build process happens there. So all the build files naturally appear there, not in the source tree.
If you have source files doing #include "../banana/peel.h" and you can't change them, then it's a pain to make this work right (you have to copy or symlink all the header files into the build directory). If you can change the source files to say #include "libfood/comedy/banana/peel.h" instead, then you're all set.
autoconf is not exactly easy, especially for a large existing project. But it has its advantages.
Personally for single files I do this,
rm -rf temps; mkdir temps; cd temps/ ; gcc -Wall -v --save-temps ../thisfile.c ; cd ../ ; geany thisfile.c temps/thisfile.s temps/thisfile.i
temps folder will keep all the object, preprocessed and assembly files.
This is a crude way of doing things and I would prefer above answers using Makefiles.
I am trying to build the Atmel BitCloud v3.2 sample application Blink with the makefile in Linux and Mac. Everything works fine in Windows. But in Posix-like systems the following lines are not working:
all: directories images root_files size
$(OBJ_PATH)/%.o: $(SRCS)
$(CC) $(CFLAGS) $(filter %/$(subst .o,.c,$(notdir $#)), $(SRCS)) -o $#
Therefore, the compiler is not being executed and object files are not being created. As the result the linker displays error messages such as
avr-gcc: error: All_MegaRf_Atmega256rfr2_8Mhz_Gcc/Obj/blink.o: No such file or directory
The line $(OBJ_PATH)/%.o: $(SRCS) is the source of the problem.
If I substitute it with smth like $(OBJ_PATH)/blink.o: ../../src/blink.c the corresponding object-file is being successfully created.
I was even able to build the whole application by manually setting build targets as follows:
$(OBJ_PATH)/blink.o: ../../src/blink.c
$(CC) $(CFLAGS) $(filter %/$(subst .o,.c,$(notdir $#)), $(SRCS)) -o $#
$(OBJ_PATH)/stdPdsEvents.o: ../../../../BitCloud/Components/PersistDataServer/std/src/stdPdsEvents.c
$(CC) $(CFLAGS) $(filter %/$(subst .o,.c,$(notdir $#)), $(SRCS)) -o $#
for all the .c files in the project.
The list of sources on the other hand is defined like this:
SRCS = \
../../src/blink.c \
../../../../BitCloud/Components/PersistDataServer/std/src/stdPdsMemAccess.c \
../../../../BitCloud/Components/PersistDataServer/std/src/stdPdsTimer.c \
Can anyone help me to figure out why is the pattern matching not working and how to recover it.
Note: Similar topic has already been open here, but the solution found by th author himself wasn't explained well in my opinion, so I couldn't solve my problem.
The construct to have each object file depend on all source files and then have the compilation step fish the actually useful file out of the list of sources is quite dubious. A better approach would be to generate a Makefile snippet and include it, something like this:
makefile.d:
echo "$(SRCS)" \
| tr ' ' '\n' \
| sed -n 's%\(.*\)/\([^/]*\)\.c$$%$(OBJ_PATH)/\2.o: \1/\2.c%p' >$#
include makefile.d
All details of sed are not properly standardized, so you may be better off rewriting the substitution in Perl or something. The beef here is how we capture the path and the base name and generate a specific rule for each dependency.
Managed to solve it finally.
In order to make it compile both in Windows and Linux one should change the makefile as shown below:
objects := $(patsubst %.c,$(OBJ_PATH)/%.o,$(notdir $(SRCS)))
$(objects): $(SRCS)
#$(OBJ_PATH)/%.o: $(SRCS)
$(CC) $(CFLAGS) $(filter %/$(subst .o,.c,$(notdir $#)), $(SRCS)) -o $#
where $(OBJ_PATH)/%.o: $(SRCS) shall be commented or removed.
After having done this way the project can be built under linux or mac and under windows both in Atmel Studio 6.2 and Atmel Studio 7. The latter was not able to build just like it was not possible in linux or mac.
Following is my directory structure
calculator
|
|---src
(multiply.cpp sum.cpp)
|---lib
I am building a static library calc.a using following
ar -rcs calc.a multiply.o sum.o
calc.a is builded in current directory.
I trying calc.a to be put into lib folder not in current directory (i.e. src)
I searched in internet and man page but couldn't find anything.
Any idea?
Since you are using the command line, the easiest way to write the library into the lib directory is to do it explicitly:
ar -rcs lib/libcalc.a multiply.o sum.o
Using a Makefile you can do more sophisticated things, but even then, it boils down to the same thing, e.g. adding the path to the front of the library name.
Since you asked for additional info on your comment, I add here a simple Makefile, which may be helpful to get you started:
CC=g++
CFLAGS=-c -Wall
LDFLAGS=
SOURCES=src/main.cpp src/sum.cpp src/multiply.cpp
OBJECTS=$(SOURCES:.cpp=.o)
LIBRARY=lib/libcalc.a
EXECUTABLE=main
all: $(SOURCES) $(LIBRARY) $(EXECUTABLE)
$(LIBRARY): $(OBJECTS)
<tab>mkdir -p lib
<tab>ar -rcs $# $<
$(EXECUTABLE): $(OBJECTS)
<tab>$(CC) $(LDFLAGS) $(OBJECTS) -o $#
.cpp.o:
<tab>$(CC) $(CFLAGS) $< -o $#
clean:
<tab>rm -rf main lib/ src/*.o
Important: All <tab>s must be replaced with real tabs, as required by the Makefile syntax!
Makefiles are very very flexible, so they can be very simple and specific to your problem, or as general and/or complex to build many libraries and binaries, based on millions and millions of source code. I suggest you to search for Makefile documentation for more information.
As a final note, I also suggest you to rename your library as 'libcalc.a', since the 'lib' prefix is standard in Unix. Other similar standards apply for other environments.
I have a makefile similar to the following:
SRCS = a.c b.cpp
OBJS = objs/a.o objs/b.o
all: $(OBJS)
objs/%.o: %.c
gcc -c $< -o $#
objs/%.o: %.cpp
gcc -c $< -o $#
It seems to work. But I don't really understand why.
Why doesn't it try to generate a.cpp and b.c?
as I read it : a.cpp is a prerequisite for objs/a.o and it should try to generate it. And because it doesn't find a matching rule for it - it should fail
Where am I wrong?
P.S - I execute my makefile using -r -R to avoid builtin rules
Make does not combine the prerequisite lists of different pattern rules.
When Make is looking for a way to build objs/a.o, it finds that the first pattern rule matches the target, and the prerequisite (a.c) exists. The second pattern rule matches the target, but the prerequisite (a.cpp) does not exist and cannot be built, so Make uses the first rule. Likewise, Make chooses the second rule over the first when looking for a way to build objs/b.o.
Make would try to generate a.cpp and b.c if these files depended on something else. However it is not the case here, these two files are leaves in the dependency tree, so Make has no reason to try to generate them.