Confusing flags passed to MSVC through NVCC with CMake - visual-c++

Question

I have a CMake file which I'm using to build some CUDA on Windows (NVCC/MSVC). I'm trying to set the MSVC warning level to /W4, using:
add_compile_options("$<$<COMPILE_LANGUAGE:CUDA>:--compiler-options=/W4>")
Building with CMake 3.9, I get this warning:
(CudaBuildCore target) ->
cl : Command line warning D9025: overriding '/W4' with '/W3'
In CMake 3.15, the policy was changed to not automatically set /W3 in the CUDA flags, but with that version I get:
(CudaBuildCore target) ->
cl : Command line warning D9025: overriding '/W4' with '/W1'
If I do the build step with --verbose I see the following with 3.9:
"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_30,code=\"compute_30,compute_30\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64" -x cu -IC:\Users\eddi\Documents\temp\thur\sw\include -IC:\Users\eddi\Documents\temp\thur\sw\shared\common\include -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" --keep-dir x64\Debug -maxrregcount=0 --machine 64 --compile -cudart static --compiler-options=/W4 --compiler-options= -Xcompiler="/EHsc -Zi -Ob0" -g -D_WINDOWS -D"FISH_BUILD_TYPE=\"DEBUG\"" -D"CMAKE_INTDIR=\"Debug\"" -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -Xcompiler "/EHsc /W3 /nologo /Od /Fdfish.dir\Debug\fish.pdb /FS /Zi /RTC1 /MDd /GR" -o fish.dir\Debug\fish_cuda.obj "C:\Users\eddi\Documents\temp\thur\sw\fish_cuda.cpp"
and with 3.15:
"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\bin\nvcc.exe" -gencode=arch=compute_30,code=\"compute_30,compute_30\" --use-local-env -ccbin "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\VC\Tools\MSVC\14.16.27023\bin\HostX86\x64" -x cu -IC:\Users\eddi\Documents\temp\thur\sw\include -IC:\Users\eddi\Documents\temp\thur\sw\shared\common\include -I"C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v10.1\include" --keep-dir x64\Debug -maxrregcount=0 --machine 64 --compile -cudart static --compiler-options=/W4 --compiler-options= -Xcompiler="/EHsc -Zi -Ob0" -g -D_WINDOWS -D"FISH_BUILD_TYPE=\"DEBUG\"" -D"CMAKE_INTDIR=\"Debug\"" -D"CMAKE_INTDIR=\"Debug\"" -D_MBCS -Xcompiler "/EHsc /W1 /nologo /Od /Fdfish.dir\Debug\fish.pdb /FS /Zi /RTC1 /MDd /GR" -o fish.dir\Debug\fish_cuda.obj "C:\Users\eddi\Documents\temp\thur\sw\fish_cuda.cpp"
Spoiler: these are identical, except for the -Xcompiler "/EHsc /W3 /nologo /Od /Fdfish.dir\Debug\fish.pdb /FS /Zi /RTC1 /MDd /GR" portion.
Try as I might, I cannot find where this set of flags is being introduced (by CMake et al.), and hence can't start working out how to change the behaviour.
EDIT1: Adding color to the story...
I grep'd my CMake install for /nologo and found several .cmake files with lines like:
35: set(CMAKE_CL_NOLOGO "/nologo")
I changed them all to variations of
35: set(CMAKE_CL_NOLOGO "/nologo_EDD")
But none of them turn up in the confusing set of options. So either I've missed something or they're being introduced from something outside CMake?
EDIT2:
#squareskittles said:
you can remove the default flag from the CMAKE_CXX_FLAGS_INIT variable
Fair enough, I can see the logic to that: it's simple and it directly addresses my immediate issue. But it seems a bit blunt and doesn't address where the flags are coming from in the first place.
Anyway, objections aside, I added this ugly morsel to my script:
message( "--------------------------- ${CMAKE_CXX_FLAGS_INIT}")
string (REGEX REPLACE "/W[0-4]" "" CMAKE_CXX_FLAGS_INIT "${CMAKE_CXX_FLAGS_INIT}")
message( "--------------------------- ${CMAKE_CXX_FLAGS_INIT}")
With CMake 3.9, I see:
--------------------------- /DWIN32 /D_WINDOWS /W3 /GR /EHsc
--------------------------- /DWIN32 /D_WINDOWS /GR /EHsc
as hoped. But with 3.15, I see:
--------------------------- /DWIN32 /D_WINDOWS /GR /EHsc
--------------------------- /DWIN32 /D_WINDOWS /GR /EHsc
ie, the /W3 flag isn't there in the first place (let alone a /W1 flag) due to the policy change. But even worse, the primary issue still persists in both cases:
nvcc.exe ... --compiler-options=/W4 -Xcompiler="/EHsc -Zi -Ob0" ... -Xcompiler "/EHsc /W3 /nologo /Od /Fdfish.dir\Debug\fish.pdb /FS /Zi /RTC1 /MDd /GR" ...
EDIT3: Some progress!
With this simple change:
#add_compile_options("$<$<COMPILE_LANGUAGE:CUDA>:--compiler-options=/W4>")
add_compile_options("$<$<COMPILE_LANGUAGE:CUDA>:-Xcompiler=/W4>")
there are no more complaints! The confusing flags persist, but now read:
-Xcompiler "/EHsc /W4 /nologo /Od /Ffish.dir\Debug\fish.pdb
/FS /Zi /RTC1 /MDd /GR"
ie. CMake has spotted the /W4 and overridden its default.
But despite scouring the CMake source, I've still yet to determine where all this is done.
EDIT4: MSVC has a hand in all this.
The 'mystery' options are generated somewhat automatically by MSVC from the .vcxproj file. If I use the -Xcompile=/W4 approach, the project XML contains:
<CudaCompile>
...
<AdditionalOptions>%(AdditionalOptions) -Xcompiler="/EHsc -Zi -Ob1"</AdditionalOptions>
...
<Warning>W4</Warning>
</CudaCompile>
But with the original, --compiler-options=/W4 I get:
<CudaCompile>
<AdditionalOptions>%(AdditionalOptions) --compiler-options=/W4 -Xcompiler="/EHsc -Zi -Ob1"</AdditionalOptions>
...
</CudaCompile>
And this all sort of makes sense; with no Warning level specified, MSVC must default to W1, hence the complaint.
Also, there was a clue-with-hindsight: the build step had:
-Xcompiler="/EHsc -Zi -Ob0"
-Xcompiler "/EHsc /W1 /nologo /Od ..."
Note one has an equals and one does not implying they come from different places.
I think my investigation has gone far enough for now.

Answer 1

A TL;DR summary of my own shaggy-dog-tale of a question:
I had:
add_compile_options("$<$<COMPILE_LANGUAGE:CUDA>:--compiler-options=/W4>")
but found:
add_compile_options("$<$<COMPILE_LANGUAGE:CUDA>:-Xcompiler=/W4>")
to be more appropriate.
Basically, the CMake/NVCC/MSVC pipeline understood that the -Xcompiler option overrode the default, but didn't realise that --compiler-options was an equivalent statement, resulting in a command line with ambiguous directives.

Answer 2

I had a similar issue (but with another flag) and used CMake's regular expressions to fix that. The code is like this.
foreach(flag_var
CMAKE_CXX_FLAGS CMAKE_CXX_FLAGS_DEBUG CMAKE_CXX_FLAGS_RELEASE
CMAKE_CXX_FLAGS_MINSIZEREL CMAKE_CXX_FLAGS_RELWITHDEBINFO)
STRING (REGEX REPLACE "/W3" "/W4" ${flag_var} "${${flag_var}}")
endforeach(flag_var)