I'm using Visual Studio 2005 nmake, and I have a test makefile like this:
sometarget:
-mkdir c:\testdir
I want to always create the directory, without having to specify 'sometarget'. For example, I can do this:
!if [if not exist c:\testdir\$(null) mkdir c:\testdir]
!endif
But that requires two lines, where I really only want to do the "-mkdir c:\testdir". If I just replace it with "-mkdir c:\testdir" I get an error from nmake - "fatal error U1034: syntax error : separator missing".
How can I always execute the mkdir, without messing about with !if [] stuff?
I think this will work:
-# if NOT EXIST "dir" mkdir "dir"
Make always wants to do things based on targets. It's not a general scripting tool. It looks at the targets and checks to see if they exist. If the target does not exist it executes the commands for that target.
The usual way to do this is to have a dummy target that is never going to be generated by the make scripts, so every time make runs it has to execute the relevant commands.
Or, you could add the command to a batch file that then calls your make file.
I'm not sure if there is an equivalent in Windows with nmake, but I managed to create a directory without using targets on Linux. I used the make function "shell". For example:
# Find where we are
TOPDIR := $(shell pwd)
# Define destination directory
ROOTFS := $(TOPDIR)/rootfs
# Make sure destination directory exists before invoking any tags
$(shell [ -d "$(ROOTFS)" ] || mkdir -p $(ROOTFS))
all:
#if [ -d "$(ROOTFS)" ]; then echo "Cool!"; else echo "Darn!"; fi
I hope Windows has the equivalent.
$(DIRNAME):
#[ -d $# ] || mkdir -p $#
Try using this:
-mkdir -p c:\testdir
Related
I'm often creating png files out of dot (graphviz format) files. The command to do so is the following:
$ dot my_graph.dot -o my_graph.png -Tpng
However, I would like to be able to have a shorter command format like $ make my_graph.dot to automatically generate my png file.
For the moment, I'm using a Makefile in which I've defined the following rule, but the recipe is only available in the directory containing the Makefile
%.eps: %.dot
dot $< -o $# -Teps
Is it possible to define custom implicit GNU Make recipes ? Which would allow the above recipe to be available system-wide
If not, what solution do you use to solve those kind of problem ?
Setup:
Fedora Linux with ZSH/Bash
You could define shell functions in your shell's startup files, e.g.
dotpng()
{
echo dot ${1%.dot}.dot -o ${1%.dot}.png -Tpng;
}
This function can be called like
dotpng my_graph.dot
or
dotpng my_graph
The code ${1%.dot}.dot strips .dot from the file name if present and appends it (again) to allow both my_graph.dot and my_graph as function argument.
Is it possible to define custom implicit GNU Make recipes ?
Not without modifying the source code of GNU Make.
If not, what solution do you use to solve those kind of problem ?
I wouldn't be a fan o modyfying the system globally, but you could do:
Create a file /usr/local/lib/make/myimplicitrules.make with the content
%.eps: %.dot
dot $< -o $# -Teps
Use include /usr/local/lib/make/myimplicitrules.make in your Makefile.
I would rather use a git submodule or similar to share common configuration between projects, rather than depending on global configuration. Depending on global environment will make your program hard to test and non-portable.
I would rather go with a shell function, something along:
mymake() {
make -f <(cat <<'EOF'
%.eps: %.dot
dot $< -o $# -Teps
EOF
) "$#"
}
mymake my_graph.dot
GNU Make lets you specify extra makefiles to read using the MAKEFILES
environment variable. Quoting from info '(make)MAKEFILES Variable':
the default goal is never taken from one of these makefiles (or any
makefile included by them) and it is not an error if the files listed
in 'MAKEFILES' are not found
if you are running 'make' without a specific makefile, a makefile
in 'MAKEFILES' can do useful things to help the built-in implicit
rules work better
As an example, with no makefile in the current directory and the
following .mk files in make's include path (e.g. via
MAKEFLAGS=--include-dir="$HOME"/.local/lib/make/) you can create
subdir gen/ and convert my_graph.dot or dot/my_graph.dot by
running:
MAKEFILES=dot.mk make gen/my_graph.png
To further save some typing it's tempting to add MAKEFILES=dot.mk
to a session environment but defining MAKEFILES in startup files
can make things completely nontransparent. For that reason I prefer
seeing MAKEFILES=… on the command line.
File: dot.mk
include common.mk
genDir ?= gen/
dotDir ?= dot/
dotFlags ?= $(if $(DEBUG),-v)
Tvariant ?= :cairo:cairo
vpath %.dot $(dotDir)
$(genDir)%.png $(genDir)%.svg $(genDir)%.eps : %.dot | $(genDir).
dot $(dotFlags) $< -o $# -T'$(patsubst .%,%,$(suffix $#))$(Tvariant)'
The included common.mk is where you'd store general definitions to
manage directory creation, diagnostics etc., e.g.
.PRECIOUS: %/. ## preempt 'unlink: ...: Is a directory'
%/. : ; $(if $(wildcard $#),,mkdir -p -- $(#D))
References:
?= = := … - info '(make)Reading Makefiles'
vpath - info '(make)Selective Search'
order-only prerequisites (e.g. | $(genDir).) - info '(make)Prerequisite Types'
.PRECIOUS - info '(make)Chained Rules'
I've got (what feels like) a fairly simple problem but my complete lack of experience in bash has left me stumped. I've spent all day trying to synthesize a script from many different SO threads explaining how to do specific things with unintuitive commands, but I can't figure out how to make them work together for the life of me.
Here is my situation: I've got a directory full of nested folders each containing a file with extension .7 and another file with extension .pc, plus a whole bunch of unrelated stuff. It looks like this:
Folder A
Folder 1
Folder x
data_01.7
helper_01.pc
...
Folder y
data_02.7
helper_02.pc
...
...
Folder 2
Folder z
data_03.7
helper_03.pc
...
...
Folder B
...
I've got a script that I need to run in each of these folders that takes in the name of the .7 file as an input.
pc_script -f data.7 -flag1 -other_flags
The current working directory needs to be the folder with the .7 file when running the script and the helper.pc file also needs to be present in it. After the script is finished running, there are a ton of new files and directories. However, I need to take just one of those output files, result.h5, and copy it to a new directory maintaining the same folder structure but with a new name:
Result Folder/Folder A/Folder 1/Folder x/new_result1.h5
I then need to run the same script again with a different flag, flag2, and copy the new version of that output file to the same result directory with a different name, new_result2.h5.
The folders all have pretty arbitrary names, though there aren't any spaces or special characters beyond underscores.
Here is an example of what I've tried:
#!/bin/bash
DIR=".../project/data"
for d in */ ; do
for e in */ ; do
for f in */ ; do
for PFILE in *.7 ; do
echo "$d/$e/$f/$PFILE"
cd "$DIR/$d/$e/$f"
echo "Performing operation 1"
pc_script -f "$PFILE" -flag1
mkdir -p ".../results/$d/$e/$f"
mv "results.h5" ".../project/results/$d/$e/$f/new_results1.h5"
echo "Performing operation 2"
pc_script -f "$PFILE" -flag 2
mv "results.h5" ".../project/results/$d/$e/$f/new_results2.h5"
done
done
done
done
Obviously, this didn't work. I've also tried using find with -execdir but then I couldn't figure out how to insert the name of the file into the script flag. I'd appreciate any help or suggestions on how to carry this out.
Another, perhaps more flexible, approach to the problem is to use the find command with the -exec option to run a short "helper-script" for each file found below a directory path that ends in ".7". The -name option allows find to locate all files ending in ".7" below a given directory using simple file-globbing (wildcards). The helper-script then performs the same operation on each file found by find and handles moving the result.h5 to the proper directory.
The form of the command will be:
find /path/to/search -type f -name "*.7" -exec /path/to/helper-script '{}` \;
Where the -f option tells find to only return files (not directories) ending in ".7". Your helper-script needs to be executable (e.g. chmod +x helper-script) and unless it is in your PATH, you must provide the full path to the script in the find command. The '{}' will be replaced by the filename (including relative path) and passed as an argument to your helper-script. The \; simply terminates the command executed by -exec.
(note there is another form for -exec called -execdir and another terminator '+' that can be used to process the command on all files in a given directory -- that is a bit safer, but has additional PATH requirements for the command being run. Since you have only one ".7" file per-directory -- there isn't much benefit here)
The helper-script just does what you need to do in each directory. Based on your description it could be something like the following:
#!/bin/bash
dir="${1%/*}" ## trim file.7 from end of path
cd "$dir" || { ## change to directory or handle error
printf "unable to change to directory %s\n" "$dir" >&2
exit 1
}
destdir="/Result_Folder/$dir" ## set destination dir for result.h5
mkdir -p "$destdir" || { ## create with all parent dirs or exit
printf "unable to create directory %s\n" "$dir" >&2
exit 1
}
ls *.pc 2>/dev/null || exit 1 ## check .pc file exists or exit
file7="${1##*/}" ## trim path from file.7 name
pc_script -f "$file7" -flags1 -other_flags ## first run
## check result.h5 exists and non-empty and copy to destdir
[ -s "result.h5" ] && cp -a "result.h5" "$destdir/new_result1.h5"
pc_script -f "$file7" -flags2 -other_flags ## second run
## check result.h5 exists and non-empty and copy to destdir
[ -s "result.h5" ] && cp -a "result.h5" "$destdir/new_result2.h5"
Which essentially stores the path part of the file.7 argument in dir and changes to that directory. If unable to change to the directory (due to read-permissions, etc..) the error is handled and the script exits. Next the full directory structure is created below your Result_Folder with mkdir -p with the same error handling if the directory cannot be created.
ls is used as a simple check to verify that a file ending in ".pc" exits in that directory. There are other ways to do this by piping the results to wc -l, but that spawns additional subshells that are best avoided.
(also note that Linux and Mac have files ending in ".pc" for use by pkg-config used when building programs from source -- they should not conflict with your files -- but be aware they exists in case you start chasing why weird ".pc" files are found)
After all tests are performed, the path is trimmed from the current ".7" filename storing just the filename in file7. The file7 variabli is then used in your pc_script command (which should also include the full path to the script if not in you PATH). After the pc_script is run [ -s "result.h5" ] is used to verify that result.h5 exists and is non-empty before moving that file to your Result_Folder location.
That should get you started. Using find to locate all .7 files is a simple way to let the tool designed to find the files for you do its job -- rather than trying to hand-roll your own solution. That way you only have to concentrate on what should be done for each file found. (note: I don't have pc_script or the files, so I have not testes this end-to-end, but it should be very close if not right-on-the-money)
There is nothing wrong in writing your own routine, but using find eliminates a lot of area where bugs can hide in your own solution.
Let me know if you have further questions.
I have a symlink to an executable, which I've created as follows:
$ ln -s /home/x/app/wps_office/wps
If on the commandline I type:
$ /home/x/app/wps_office/wps
Then my application launches correctly, but if I try to launch my application through the symlink, then I get the following error:
$ wps
wps does not exist!
Just to make sure if the symlink is correct;
$ readlink wps
/home/x/app/wps_office/wps
The folder /home/x/bin is where I've created the symlink, this folder is included in my $PATH variable.
I don't see what is going wrong here, why doesn't my application execute when I use the symlink?
Quick update;
I've just quickly looked trough the contents of the file where the symlink is pointing to, it looks like the message wps does not exist is actually coming from the application, meaning the symlink is actually correct. I don't know the exact reason why, as I find it strange that everything works correctly when I don't use the symlink. I need to look more thorougly to the code to find that out.
The code of the file where the symlink is pointing to:
#!/bin/bash
gOpt=
gTemplateExt=("wpt" "dot" "dotx")
gBinPath=$(dirname "$0")
if [ -d "${gBinPath}/office6" ]; then
gInstallPath=${gBinPath}
else
gInstallPath=/opt/kingsoft/wps-office
fi
gApp=wps
function parse_arg()
{
if [ $# -eq 1 ] ; then
ext="${1##*.}"
if [ "" = "${ext}" ] ; then
return 0
fi
for i in ${gTemplateExt}
do
if [ "${ext}" = "${i}" ] ; then
gOpt=-t
fi
done
fi
}
function run()
{
oldPwd="${PWD}"
if [ -e "${gInstallPath}/office6/${gApp}" ] ; then
if [ -d /usr/lib32/gtk-2.0 ]; then
export GTK_PATH=/usr/lib32/gtk-2.0
fi
${gInstallPath}/office6/${gApp} ${gOpt} "$#" || ${gBinPath}/wps_error_check.sh ${gInstallPath}/office6/${gApp}
else
echo "${gApp} does not exist!"
fi
}
function main()
{
parse_arg "$#"
run "$#"
}
main "$#"
Note the line where it says echo "${gApp} does not exist!", this is where my error is coming from.
Commands will only be executed without any path elements if they're part of the shell, or if they're in the PATH environment variable. Try
./wps
in the directory where the symlink is. Also confirm that the permissions are correct.
Change the line
gInstallPath=/opt/kingsoft/wps-office
in the script to
gInstallPath=/home/x/app/wps_office
The file where the symlink was pointing to, takes the current directory to launch a different file. This is the file actually being launched. The issue can be solved by simply creating a symlink to this file, which means a symlink to /home/x/app/wps_office/office6/wps
Another option is to edit the source file itself, as explained by #Pixelchemist. However as it concerns an application which I've downloaded and which I will probably update in the future, I think in this case that is not a preferred option.
My issue is that in my make file I want to create the directory for all object code before building it and I cannot find a way to do this whithout having it impact the output of make. Also, i want to do it in ONE place and not for each and every object file.
I have tried this:
build: setup exe.i686
setup:
mkdir -p obj
exe.i686: $(OBJS)
#echo 'Building target: $#'
#echo 'Invoking: GCC C++ Linker'
${GCC_LOCATION}/Linux/i686/bin/i686-pc-linux-gnu-g++ $(OBJS) -o "exe.i686" -L...
#echo 'Finished building target: $#'
#echo ' '
where OBJS is all my object code. If I don't run setup first and there is no ./obj directory present the build will fail because of the missing directory when trying to build the object code.
Just to be clear, here is an example of how one object file is built:
./obj/Cell.o: ${CELL_ROOT}/main/cell/Cell/src/Cell.cc
#echo 'Building file: $<'
#echo 'Invoking: GCC C++ Compiler'
$(GCC_C++_BUILD) -MF"$(#:%.o=%.d)" -MT"$(#:%.o=%.d)" -o "$#" "$<"
#echo 'Finished building: $<'
#echo ' '
I don't want to build the object code in the same directory I have my make file.
Now to the actual problem. When I build tarket "build" the first time like this everything works fine:
make build
But if I build it again and there are no updates in my source code (object code is up-to-date) it behaves like this:
[7:37] [user1] [/local/repo/user1/project_x/test/bin] $ make build
mkdir -p obj
[7:37] [user1] [/local/repo/user1/project_x/test/bin] $
In this situation I would have liked make to return the nothing-to-be-done string like this:
[7:37] [user1] [/local/repo/user1/project_x/test/bin] $ make build
make: Nothing to be done for `build'.
[7:41] [user1] [/local/repo/user1/project_x/test/bin] $
This would have been the case if I didn't have to depend on the setup target (which is added to .PHONY). Is the only alternative to do it for each object code file? That would be possible I suppose but I wanted to avoid that if possible as it clutters the make file quite a bit. At least in my opinion.
Thanks
Your target setup is always executed because it has no dependency.
You have two choices:
(1) Make a stamp or use some other mechanism so that there is an actual dependency. The directory itself can be the target but that may incur some wrinkles: see http://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html for an example.
Modified relevant extract from the linked article:
$(OBJS): | obj
obj:
mkdir $(OBJDIR)
(2) Suppress the output by testing for existence:
setup:
test -d obj || mkdir -p obj
This is similar to the earlier answer, just to make it easier to read:
build: exe.i686
$(OBJS): | obj
obj:
#if [ ! -d obj ]; then mkdir -p obj; fi
exe.i686: $(OBJS)
#echo 'Building target: $#'
...
SO
I can't find out why these lines are not called for the first time I run 'make' but are called the next time:
sb_path = sb
sb_src := $(sb_path)/src
sb_build := $(sb_path)/build
ifndef DO_NOT_GENERATE_COMMIT_INFO
commit_sb: | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
$(sb_src)/last_git_commit_info.h: | commit_sb ;
endif
I'm just curious because there is no file generate-commit-info file and make crashes when I call it for the second time, but it compiles my program ok for the first try.
I use script on my local machine to copy sources over ssh to another machine and to run compile.sh script there:
...
scp -r $sbfolder/build $sbfolder/Makefile "$buildserver:$root/$curdate"
check_retcode
scp -r $sbfolder/sb/Makefile "$buildserver:$root/$curdate/sb/"
...
ssh $buildserver "$root/compile.sh $curdate $debug"
compile.sh:
# fix Makefile: we don't have git installed here
#DO_NOT_GENERATE_COMMIT_INFO=true
#now we can compile sb
curdir="/home/tmp/kamyshev/sb_new/$1"
cd $curdir
check_retcode
t_path=$curdir
debug=$2
config=RELEASE
if [[ debug -eq 1 ]]; then
config=DEBUG
fi
echo "building sb... CONFIG=$config"
make -j2 CONFIG=$config
check_retcode
As you see DO_NOT_GENERATE_COMMIT_INFO=true is commented out. So I just don't see a reason why the code is not run when I call a make or the script for the first time (either from the remote script or myselft from command line).
Do you have any clues?
UPDATE on Etan Reisner comment:
commit_sb target is checked, it does not exist, so it's rule is being run and it updates last_git_commit_info.h. Thus it forces to update the .h file. It also gives me a .PHONY target commit_sb so I could do it directly by calling make commit_sb.
The generate-commit-info also creates a file in a $(sb_bin) folder.
My another guess is that you are talking about a better way to organize this code.
I can update last_git_commit_info.h directly with a such rule:
commit_sb $(sb_src)/last_git_commit_info.h: FORCE | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
FORCE:
Thanks to the commenters on my question I've done some additional research: I've tried to make a minimal complete example. And this led me to the answer.
My code generates dependency files (look at -MMD command in SB_CXXFLAGS):
# just example - in real Makefile these are calculated on the fly
sb_deps := file1.d file2.d [...]
# rules with dependances of .o files against .h files
-include $(sb_deps)
SB_CXXFLAGS = $(CXXFLAGS) [...] -MMD
# compile and generate dependency info;
$(sb_obj)/%.o:$(sb_src)/%.cpp
$(CXX) $(SB_CXXFLAGS) $< -o $#
And when I run make for the first time there no *.d files, so no *.cpp depends on last_git_commit_info.h file and the rule is not applied.
On the subsequent runs the dependency rule appears in one of *.d files, the rule is executed and I get the error.
UPDATE: This does not concern the question directly, but this is the better way to write these rules:
ifndef DO_NOT_GENERATE_COMMIT_INFO
commit_sb $(sb_src)/last_git_commit_info.h: FORCE | $(sb_bin)
#$(sb_build)/generate-commit-info $(sb_path)
FORCE:
endif