I encountered a build issue with visual studio's 'link.exe' tool, and at some point, which I would have normally investigated by passing my build command as an argument to linux's strace -efile tool to get a dump of all the files considered by the linker, which couldn't be found and which were selected. But of course, we're on Windows this time, and strace is not an option.
the problem was indeed the linker picking the wrong file when I said ntoskrnl.lib on the command line
the root cause was an invalid %LIB% contents that wasn't matching the include paths passed to cl.exe calls
Question: how could I force link.exe to reveal the full path of the files it is linking (including .lib) ? (and save myself this hassle next time)
Related
I have a rather complex CMake configuration, which contains several execute_process commands which create files during the configuration stage. Sometimes after some changes in the CMake configuration the generation stage deletes these files. I can reproduce that.
I've checked that the files exist after the configuration stage but are gone after the Makefile generation and before actually calling make.
The files are created in some cases simply by ${CMAKE_COMMAND} -E copy and in other cases by calling a script with ${CMAKE_COMMAND} -P which contains a configure_file call to replace some placeholders in a template.
The files are created in the source tree. Their purpose is to provide the developer with some initial code. When the developer has edited the files, they should be committed to version control and not be re-created again unless they are missing. I have add_custom_commands to recreate the files if they're missing, but those are not the culprit.
I know, you'd prefer a simple test example, but unfortunately this is not so easy to create, so my question is:
What might be the cause and how can I debug that?
Unfortunately, the --trace options of cmake do not give any log data about the generation stage.
Versions
OS: Ubuntu 16.04
CMake 3.5.1 (belonging to Ubuntu 16.04)
Update
I've compiled CMake itself with the current master commit (696b2d4) and the behavior is still the same.
By running CMake under a debugger I've discovered that the line
cmSystemTools::RemoveFile(fname);
in function cmGlobalGenerator::CheckRuleHashes(std::string const& pfile, std::string const& home) actually deletes the files. It is called from cmGlobalGenerator::Generate().
With further debugging (see update in the question) I've found the cause of my problem.
CMake's behavior
CMake's global generator creates the file ${CMAKE_BINARY_DIR}/CMakeFiles/CMakeRuleHashes.txt. It contains a line for every custom command with outputs. Each line contains a hash and the path to the first output of the custom command.
The hash is made over the current binary dir of the custom command and the entire content of the COMMAND arguments of add_custom_command.
On generation CMake executes cmGlobalGenerator::CheckRuleHashes to check if the hashes are still up to date. If a hash is not up to date the corresponding output file will be removed, obviously to trigger re-execution of the custom command in the build stage.
Cause of my problem
As mentioned in the question I execute the file creation command with execute_process during configuration and call add_custom_command with the same file creation command to trigger re-creation of the file in case it was missing.
So, CMake creates a hash line in its CMakeRuleHashes.txt for the file to be created.
Some of my add_custom_command calls are contained in a CMake file included from different CMakeLists.txt belonging to different sub-directories of the source. Depending on my actual configuration only specific sub-directories are added with add_subdirectory.
Thus, because the sub-directory is part of the hash of the custom command, the hash changes depending on the sub-directory added which includes the custom command. The changed hash then causes the deletion of the output file.
Conclusion
I have to redesign my CMake configuration to resolve the dependency of the custom command on the CMake binary directory.
Debugging generation stage
This could be done by compiling CMake itself as Debug build type and run it under a debugger.
I'm trying to gdb a program under Linux. It's a compiler of a language that uses some include files (this language include files, not C/C++ includes) that get properly loaded when running the app normally. When run under gdb, the app complains that it couldn't find these include files. I've checked gdb's internal pwd, and it is exactly the same as when run stand-alone. What could be the problem? How can I fix it?
Note that this compiler doesn't use any environment variables, it gets its includes from current directory, it seems.
you can set the source directories path in gdb..
go through the source path in gdb document.
https://sourceware.org/gdb/download/onlinedocs/gdb/Source-Path.html#Source-Path
I upgraded Visual Studio to VS Professional 2012.
Building old Visual C++ projects resulted in the error:
LINK : fatal error LNK1158: cannot run 'mt.exe'
Putting the folder with mt.exe into the path did not help.
Currently I am using mt.exe in the folder C:\Program Files (x86)\Microsoft SDKs\Windows\v7.1A\Bin\x64
Set "Generate manifest" as No in Linker. This helped
As mentioned at fatal error LNK1158: cannot run 'mt.exe', there are times where turning off manifest generation is not an option, because one is actually using manifests.
At which point, one comes back to fixing the actual problem.
One cause of the actual problem is a bad executable search path.
I personally once had a setting for $(ExecutablePath) in the project file (VC++ Directories → Executable directories in the project settings) that was yielding an executable search path one of whose pathnames wasn't a valid directory name.
The pathname had a colon in one of its directory name components (caused by prefixing ..\..\ to another variable, which happened to contain an absolute pathname with a drive letter, rather than a relative pathname as expected).
In a sensible world, errors in path searches caused by merely bad/missing directories cause the pathname to just get skipped over, and the search to proceed to the next pathname in the search path.
link, in contrast, fails with this error (as indeed does midl when it tries to find cl.exe).
This is very confusing when one looks at the search path (with the build output level set to "detailed") and sees that the SDK directory containing mt.exe is right there in it, plain as day.
It seems that link does not live in a sensible world. ☺
The fix was to correct the bad project setting, of course.
I previously had some troubles updating old code that still needed a not supported compiler and expensive libraries to a version with gfortran in Eclipse on Windows. I let it rest for a while and recently I took a whole other approach, rebuilding the program from scratch, developping on a ubuntu machine, but now I want to bring it back to a windows machine so that my co-workers can contribute on it.
The status:
Program compiles, runs and gives good results on an ubuntu machine with the GCC GNU compiler
Windows 7 machine, 64bit
Cygwin installation (for gnu fortran) with lapack and liblapack-devel (however, I don't use these, because I compile blas and lapack manually)
(C:/cygwin/lib added to windows Path)
Original Issue:
The program compiles in cygwin (by calling the make-command, calling the make command with the makefile situated here: http://thijsvandenbrande.be/phd/hamfemInstall/makefile
This returns the file hamfem.exe which returns the following error when runned by double-clicking on it in windows: The program can't start because cyglapack-0.dll is missing from your computer. Try reinstalling the program to fix this problem.
When running the executable from cygwin, by calling the ./hamfem.exe command the executable starts to run. However, I want a solution so that I can give this executable to my co-workers so that they can change the input files (located in a folder in that has a relative path to the executable).
Going further on the comments below, I tried the next things:
Adding the exact path to the C:\cygwin\lib\lapack\cyglapack-0.dll file in windows path and even rebooting afterwards doesn't help.
adding a -static to the makefile before calling the library, resulting in dependency errors because I use two commands of the lapack library that depend on quite a lot of other commands (DPBTRF and DPBTRS). These commands are used in the mainprog.f90 module. The error: /usr/lib/gcc/i686-pc-cygwin/4.7.3/../../../liblapack.a(dpbtrf.f.o): In function 'dpbtrf':
/usr/src/debug/lapack-3.4.2-1/SRC/dpbtrf.f:277: undefined reference to 'dtrsm_'
and a couple of more lines stating the dependencies.
add the liblapack.a file to the src folder, but compiler always goes back to the lapack in cygwin
On the website of lapack you can normally download the functions with their dependencies (example DPBTRF), but these are not available anymore. Does anyone have another idea how to include these two functions and their dependencies in a static library-file that I can compile beforehand and add to the src-folder?
Current (semi-)Fix
The next thing worked (a bit) for me: following the instructions on http://gcc.gnu.org/wiki/GfortranBuild to manually build libblas.a and liblapack.a in the /usr/src folder of Cygwin and refering to this folder in the makefile. The updated makefile can be found here: http://thijsvandenbrande.be/phd/hamfemInstall/makefileNew
The code compiles nicely on Windows by running the make command from cygwin (next step in the process, running it out of Eclipse) and i get a .exe file that can be run by double clicking it and that keeps running if I move it with its folder to another location. Because this process is quite labour intensive, figuring it all out, I added the answer here below, stating the commands you have to parse to cygwin in order to make it work.
For your information: my file structure looks like this (after the build, I move the .exe file one folder up, both in the linux version as the windows version):
hamfem.exe
in
input.txt
NGCR_building01.txt
out
(empty folder for output files of the routine)
src
hamfem.f90 (main file)
mainprog.f90 (file that contains the commands from lapack)
...(a bunch of other modules)
makefile
I figured things out myself, with some pointers from all over stackoverflow. In order for others to help them resolve similar issues, I would like my work method here so that the question is fully documented.
The issue can be resolved by clean building the Lapack library !and the Blas library on your local machine in cygwin and pasting the liblapack.a and libblas.a file to the library folder that you refer to in the makefile. The errors that were casted by calling Lapack staticly where a result of some routines of Blas used in the two commands.
These are the steps I followed:
download the lapack.tgz and blas.tgz files from the website and past them in the C:\Cygwin\usr\src folder
Extract these files with the following commands in cygwin:
cd /usr/src
tar -xvzf lapack.tgz
tar -xvzf blas.tgz
Build the two library files with the commands shown below in Cygwin. Compiling Lapack can take a while and will result in some errors in the end because of some missing links in the test files. These tests are run for accuracy tools. A more detailed look into the make.inc file is needed to resolve these issues.
cd $HOME
cd /usr/src/BLAS
make
mv blas_LINUX.a ../libblas.a
cd ../lapack-3.4.2
mv make.inc.example make.inc
make
mv liblapack.a ../liblapack.a
check the makefile included in this repository for the correct linking to the libraries. These should say /usr/src and -static -llapack -lblas, the other options are for the linux compiler.
So I have scons working with an alternate build location (build/ for my output files, src/ for my input files) and it works great. Except for one thing.
I'm using an IDE (TI Code Composer 4) to debug my program. I point the IDE at the output executable to run it, and what it uses for the source files for debugging is the build/ directory. I don't really care, except when I go to edit the file in the IDE, for example main.cpp, the file is really build/main.cpp which gets clobbered as soon as I run scons again. I have to remember to edit src/main.cpp instead. I am aware of this issue and yet I make the same mistake often.
Is there a way to have scons make the source files it copies into the build path read-only? (I'd ask how to get TI CCS4 to use the right source files when it is debugging an executable, but I doubt I'd get any answers.)
This page has information about wrapping InstallTargets with a chmod call.
FYI, the scons user list is quite active with many knowledgeable people and you can get answers pretty quickly.
You could always tell scons not to duplicate source files in the build directory:
SConscript('src/SConscript', variant_dir='build', duplicate=0)