I've encountered a problem trying to set up a C++ project on my Mac (Yosemite with Xcode 6) using CMake with Unix style Makefiles and vim with the youcompleteme plugin (I'm a Linux veteran and Mac newbie so I prefer this setup to Xcode). The code builds and runs but youcompleteme throws some bogus errors which I think boil down to it not being able to find the <cstdint> header.
I've just tried it on Linux too, and had the same problem.
I've configured .ycm_extra_conf.py to use a compile_commands.json generated by cake. The "command" lines in compile_commands.json use these flags:
"command": "/usr/bin/c++ -std=c++11 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk -F/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks -I/usr/local/include -I/Users/tony/Dev/cow/jni -I/Users/tony/Library/Frameworks/SDL2.framework/Headers -Wall -Wextra -Wunused -F/Users/tony/Library/Frameworks -o ...
There doesn't seem to be an explicit reference there to any directory containing stdint as a direct parent.
Is there a way I can get youcompleteme to do its stuff with libclang in such a way that it can find the directory implicitly, which seems to work when running c++ on the command line? Or what's the best way to get cmake to add an appropriate system header path without hardwiring it? I want my CMakeLists.txt to be portable and to be able to cope with toolchain upgrades.
My .ycm_extra_conf.py was pretty much a copy of the supplied example modified slightly to find compile_commands.json where I put it.
As #ladislas said, YCM needs to be explicitly pointed to all relevant include directories as libclang won't use the same implicit location a normal compiler driver invocation (i.e. clang++ from the command line) would use.
What I usually do, on OSX, is to let YCM know about Xcode's libc++ headers with something like (in .ycm_extra_conf.py):
import os
import ycm_core
import subprocess
xcode_cpp11headers = subprocess.Popen("xcode-select -p", stdout = subprocess.PIPE, shell=True).communicate()[0].rstrip('\n') + '/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1'
.
.
flags = [
.
.
'-isystem',
xcode_cpp11headers,
.
.
]
The "xcode_cpp11headers" variable is populated with the proper path depending on the location of your current Xcode's installation, and you can change it accordingly if you want to use say the commandline-tools version of libc++ (i.e. the includes are in /Library/Developer/CommandLineTools/usr/include/c++/v1) or a source distribution of libc++ if you have compiled your own.
Of course this is platform dependant and either you provide a platform specific .ycm_extra_conf.py alongside your project, or you can populate that variable differently with some extra python code accordingly to the current platform.
You must add all the paths where YCM needs to look for your sources, libraries, and so on.
It does not work recursively, so it's a little bit cumbersome at first but is should not change once set for your project.
As an example, here is mine for an Arduino project :
https://github.com/ladislas/Bare-Arduino-Project/blob/master/.ycm_extra_conf.py#L21
Hope this helps!
EDIT - 2015/01/08
The solution of #abigagli is very elegant! I too use something like this to parse my lib directory and look for .h files to add their path to flags.
Here it is if it can be useful :) http://git.io/IiR1JA
As I found out from the above answers, YCM needs to be told the compiler's system include paths which are usually implicit in other ways of using the compiler. I added a function GetSystemIncludePaths() to .ycm_extra_conf.py to discover and cache these paths portably. Here's the full file with comments and irrelevant content of flags list snipped. The original is Copyright (C) 2014 Google Inc with a GPL2+ licence:
import subprocess, os
import ycm_core
flags = []
def DirectoryOfThisScript():
return os.path.dirname( os.path.abspath( __file__ ) )
compilation_database_folder = os.path.abspath(
os.path.join(DirectoryOfThisScript(), 'build-make'))
if os.path.exists( compilation_database_folder ):
database = ycm_core.CompilationDatabase( compilation_database_folder )
else:
database = None
SOURCE_EXTENSIONS = [ '.cpp', '.cxx', '.cc', '.c', '.m', '.mm' ]
def MakeRelativePathsInFlagsAbsolute( flags, working_directory ):
if not working_directory:
return list( flags )
new_flags = []
make_next_absolute = False
path_flags = [ '-isystem', '-I', '-iquote', '--sysroot=' ]
for flag in flags:
new_flag = flag
if make_next_absolute:
make_next_absolute = False
if not flag.startswith( '/' ):
new_flag = os.path.join( working_directory, flag )
for path_flag in path_flags:
if flag == path_flag:
make_next_absolute = True
break
if flag.startswith( path_flag ):
path = flag[ len( path_flag ): ]
new_flag = path_flag + os.path.join( working_directory, path )
break
if new_flag:
new_flags.append( new_flag )
return new_flags
def IsHeaderFile( filename ):
extension = os.path.splitext( filename )[ 1 ]
return extension in [ '.h', '.hxx', '.hpp', '.hh' ]
def GetCompilationInfoForFile( filename ):
if IsHeaderFile( filename ):
basename = os.path.splitext( filename )[ 0 ]
for extension in SOURCE_EXTENSIONS:
replacement_file = basename + extension
if os.path.exists( replacement_file ):
compilation_info = database.GetCompilationInfoForFile(
replacement_file )
if compilation_info.compiler_flags_:
return compilation_info
return None
return database.GetCompilationInfoForFile( filename )
def GetSystemIncludePaths():
cache = os.path.join(DirectoryOfThisScript(), ".ycm_sys_incs")
if os.path.exists(cache):
fp = open(cache, 'r')
flags = fp.readlines()
fp.close()
flags = [s.strip() for s in flags]
else:
devnull = open(os.devnull, 'r')
child = subprocess.Popen(["/usr/bin/cpp", "-xc++", "-v"],
stdin = devnull, stderr = subprocess.PIPE)
output = child.communicate()[1].split('\n')
devnull.close()
flags = []
status = 0
for l in output:
l = l.strip()
if l == '#include "..." search starts here:':
status = 1
elif l == '#include <...> search starts here:':
status = 2
elif status:
if l == 'End of search list.':
break
elif l.endswith('(framework directory)'):
continue
elif status == 1:
flags.append('-I')
elif status == 2:
flags.append('-isystem')
flags.append(os.path.normpath(l))
fp = open(cache, 'w')
fp.write('\n'.join(flags))
fp.close()
return flags
def FlagsForFile( filename, **kwargs ):
if database:
compilation_info = GetCompilationInfoForFile( filename )
if not compilation_info:
return None
final_flags = MakeRelativePathsInFlagsAbsolute(
compilation_info.compiler_flags_,
compilation_info.compiler_working_dir_ )
sys_incs = GetSystemIncludePaths()
if sys_incs:
final_flags += sys_incs
else:
relative_to = DirectoryOfThisScript()
final_flags = MakeRelativePathsInFlagsAbsolute( flags, relative_to )
return {
'flags': final_flags,
'do_cache': True
}
Related
I'm trying to build a flex-bison parser with SCons, using a custom decider. (You can learn more about the decider in this question, although, the details probably aren't too relevant to the current problem.)
Files:
My SConstruct:
env = DefaultEnvironment()
deciderEnv = env.Clone()
def source_and_target_decider(dependency, target, prev_ni, repo_node=None):
src_old_csig = prev_ni.csig if hasattr(prev_ni, 'csig') else None
src_new_csig = dependency.get_csig()
tgt_stored_info = target.get_stored_info()
tgt_old_csig = tgt_stored_info.ninfo.csig if hasattr(tgt_stored_info.ninfo, 'csig') else None
tgt_new_csig = target.get_csig()
return src_new_csig != src_old_csig or tgt_new_csig != tgt_old_csig
deciderEnv.Decider(source_and_target_decider)
deciderEnv['YACCFLAGS'] = ['-d']
deciderEnv['YACCHXXFILESUFFIX'] = '.hh'
bison_source = deciderEnv.CXXFile('src/test.yy.cc', 'src/test.yy')[0]
deciderEnv['LEXFLAGS'] = [f'--header-file=${{SOURCE}}.hh']
flex_source = deciderEnv.CXXFile('src/test.ll.cc', 'src/test.ll')[0]
Program('test', [bison_source, flex_source])
src/test.yy:
%language "c++"
%code {
int yylex(int*);
}
%token X
%%
everything: X {}
and src/test.ll:
%{
#include "./test.yy.hh"
%}
%option c++
%%
. { return yy::parser::token::X; }
Result:
I run a command scons ./src/test.ll.o.
The first time you run it, it works correctly.
flex --header-file=src/test.ll.hh -t src/test.ll > src/test.ll.cc
bison -d -o src/test.yy.cc src/test.yy
g++ -o src/test.ll.o -c src/test.ll.cc
SCons seems to understand that it needs to generate src/test.yy.hh using bison before it compiles the src/test.ll.o because src/test.ll.cc #includes that header.
However, if I then remove .sconsign.dblite (leaving the rest of the files intact), SCons seems to lose that understanding for the duration of the next build.
Running scons ./src/test.ll.o yields:
flex --header-file=src/test.ll.hh -t src/test.ll > src/test.ll.cc
g++ -o src/test.ll.o -c src/test.ll.cc
and only after calling it a second time it decides to build the src/test.yy.hh:
bison -d -o src/test.yy.cc src/test.yy
g++ -o src/test.ll.o -c src/test.ll.cc
Dependency tree:
Normally the dependency tree (scons --tree=all src/test.ll.o) looks like this:
+-src/test.ll.o
+-src/test.ll.cc
| +-src/test.ll
| +-/bin/flex
+-src/test.yy.hh
| +-src/test.yy
| +-/bin/bison
+-/bin/g++
However, right after deleting the .sconsign.dblite it is incomplete:
+-src/test.ll.o
+-src/test.ll.cc
| +-src/test.ll
| +-/bin/flex
+-/bin/g++
Further observations:
The last line of the custom decider seems to be the culprit.
Specifically, the part or tgt_new_csig != tgt_old_csig. If you remove this part, the problem no longer occurs.
However, I have no clue why this particular piece of code breaks the build.
Any idea how to fix this?
The sconsign is where SCons stores info about previous builds.
If you remove it, then SCons will not have that info and will rebuild everything.
Because in your decider you are setting:
tgt_old_csig = tgt_stored_info.ninfo.csig if hasattr(tgt_stored_info.ninfo, 'csig') else None
and then comparing None to the current csig, which will not be equal and require a rebuild.
You'd likely need to change as follows to work when you're explicitly removing the source of the tgt_stored_info (the sconsign file).
def source_and_target_decider(dependency, target, prev_ni, repo_node=None):
src_old_csig = prev_ni.csig if hasattr(prev_ni, 'csig') else None
src_new_csig = dependency.get_csig()
tgt_stored_info = target.get_stored_info()
tgt_old_csig = tgt_stored_info.ninfo.csig if hasattr(tgt_stored_info.ninfo, 'csig') else None
tgt_new_csig = target.get_csig()
if tgt_old_csig:
return src_new_csig != src_old_csig or tgt_new_csig != tgt_old_csig
else:
return src_new_csig != src_old_csig
Why are you removing .sconsign.dblite?
I build a simple software in my Debian VM. It tries the download a file from a FTP server. For that I use the libcurl library(libcurl-openssl-dev).
Now I have a SAMA5D3x-Evalboard available. For self education purpose I like to build a Linux + Toolchain for it. I found the Yocto project. The meta layer concept seems to be good approach for managing different platforms - so let's try it!
I found some examples at the net. But no which covers the combination of "hello world" + bigger lib.
I want:
get the X-compile toolchain + Linux image(includes my hello world + the libcurl shared objects)
My approach:
meta-atmel: for the SAMA5D3x eval board
meta-debian: for the lib "libcurl-openssl-dev"
meta-mySoftware: for the hello world program
The constrains are:
main.c - name of the hello world program
libcurl - c lib name (I have chosen this lib, because it has a lot of dependencies)
How to setup it right? If there are better ideas they are welcome!
Installation & Configuration of Yocto
I.) install toolchain
I.1.) apt-get:
sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath socat libsdl1.2-dev xterm
I.2.) opt. configure git after proxy
git config --global core.gitproxy <gitproxy>
I.3.) download Toolchain + meta packages and checkout the "right" commit
git clone git://git.yoctoproject.org/poky
cd poky
git checkout jethro
git clone git://github.com/linux4sam/meta-atmel.git
cd meta-atmel
git checkout jethro
cd ..
git clone git://git.openembedded.org/meta-openembedded
cd meta-openembedded
git checkout jethro
cd ..
II.) build kernel+rootfs for SAMA5D3x
II.1) create new project
source oe-init-build-env
the new project is created at the "build" dir
modify layers "conf/bblayers.conf" via commandos
bitbake-layers add-layer "../meta-atmel"
bitbake-layers add-layer "../meta-openembedded/meta-oe"
bitbake-layers add-layer "../meta-openembedded/meta-networking"
bitbake-layers add-layer "../meta-openembedded/meta-python"
edit of "conf/local.conf"
MACHINE ??= "sama5d3xek"
DL_DIR ?= "${TOPDIR}/../downloads"
DISTRO ?= "poky-atmel"
the download directory is outside of the build environment. The downloaded sources can be used for different builds.
II.2) create image
bitbake -c clean core-image-minimal
bitbake core-image-minimal
the result files are created at "tmp/deploy/images/sama5d3xek"
II.3) flash image
The sam-ba tool from Atmel is needed. The Flash-Process can be automated by a script(we create a tcl file). The self written tool "buildAtmelImages.py" copies the needed image files for the SAMA5D35, creates the u-boot configuration and a Flash-Script. An Example:
python buildAtmelImages.py --help
python buildAtmelImages.py -s "/home/user/poky/build/tmp/deploy/images/sama5d3xek" -d "/home/user/images" -t "\\\\1.2.3.4\\admin\\images"
here comes the code for "buildAtmelImages.py":
#!/usr/bin/python
# encoding: utf-8
'''
buildAtmelImages -- make yocto build images ready of the sam-ba flash tool
buildAtmelImages is a description
- reverse engineered /meta-atmel/scripts/nandflash.tcl script
- works only for the SAMA5D35
- params:
-s "<dir>" = source dir of the images generated by yocto. like "/home/user/poky/build/tmp/deploy/images/sama5d3xek"
-d "<dir>" = destination dir
-t "<dir/url>" = tcl refence dir. The path to the images, used to flash, is hard coded in the tcl script. typical example "\\\\192.168.1.2\\admin\\images" for an samba based network
- way of operation:
# hints:
I.) the SAMA5D3x implements some ECC features. This means that in every NAND sector are bits for error correction reserved.
This acts into 2 directions:
1.) the binary files have to be flashed by takeing care of the extra ECC bits. This is done by the sam-ba flash tool
2.) the kernel needs to take care, that the data stored on the NAND drive has some extra handling. This is managed by the meta-atmel package used by the yocto toolchain.
II.) u boot configuration
1.) the u boot configuration is placed into the NAND. But the configuration is not generated by the yocto toolchain. Atmel delivers a flashing script for its demo boards. This was reversed engineered into this python script.
The layout of the boot config is a defined memory space which is guared by an crc32(litle endian). The boot configuration is placed as ANSII string parameters. These parameters are seperated by a zero.
2.) because of the ecc the binary device tree(dtb) and kernel needs to be copied to memory before been executed. For that it is needed that u-boot loads them into ram before starting the kernel.
# operation:
First the generated images are copied. After the configuartion is generated and placed into the output dir. At least the tcl script is generated and placed into the output dir, as well.
#author: Stefan Jaritz
#copyright:
#license: license
#contact:
#deffield updated: Updated
'''
import sys
import os
import shutil
import binascii
import struct
from argparse import ArgumentParser
from argparse import RawDescriptionHelpFormatter
__all__ = []
__version__ = 0.1
__date__ = '2016-04-19'
__updated__ = '2016-04-19'
class _memorySeg():
def __init__(self, add, sf, df):
self.add = add
self.sf = sf
self.df = df
gArch = 'sama5d3xek'
gTCLfilename = 'SEK4.tcl'
gMemSetup = {
'bootloader' : _memorySeg(0x00000000, 'at91bootstrap-sama5d3xek.bin', 'bootloader.bin'),
'uboot' : _memorySeg(0x00040000, 'u-boot-sama5d3xek.bin', 'uboot.bin'),
'ubootEnv' : _memorySeg(0x000c0000, None, 'ubootEnv.bin'),
'ubootEnvRed' : _memorySeg(0x00100000, None, 'ubootEnv.bin'),
'dtb' : _memorySeg(0x00180000,'zImage-sama5d35ek.dtb', 'kernel.dtb'),
'kernel' : _memorySeg(0x00200000,'zImage', 'kernel'),
'rootfs' : _memorySeg(0x00800000,'core-image-minimal-sama5d3xek.ubi', 'rootfs.ubi')
}
gSourceDir = ''
gDestDir = ''
gTCLfilepath = ''
gECCcnfg = {
'at91sam9x5ek' : 0xc0c00405,
'at91sam9n12ek' : 0xc0c00405,
'sama5d3xek' :0xc0902405,
'sama5d3_xplained' : 0xc0902405,
'sama5d4ek' : 0xc1e04e07,
'sama5d4_xplained' : 0xc1e04e07
}
def _copyFiles ():
''' copies image files from source to the destination'''
global gMemSetup
for (k, ms) in gMemSetup.items():
if ms.sf is not None:
print("copy {k}".format(k=k))
shutil.copy(os.path.join(gSourceDir, ms.sf), os.path.join(gDestDir, ms.df))
def _genUBootCfg ():
global gMemSetup
global gSourceDir
global gDestDir
print("generate u-boot config")
bootCfgStr = "bootcmd=nand read 0x{dtbLoadAddr:X} 0x{dtbAddr:X} {dtbSize}; nand read 0x{kernelLoadAddr:X} 0x{kernelAddr:X} {kernelSize}; bootz 0x{kernelLoadAddr:X} - 0x{dtbLoadAddr:X}".format(
dtbLoadAddr=0x21000000,
dtbAddr=gMemSetup['dtb'].add,
dtbSize=os.path.getsize(os.path.join(gSourceDir, gMemSetup['dtb'].sf)),
kernelLoadAddr=0x22000000,
kernelAddr=gMemSetup['kernel'].add,
kernelSize=os.path.getsize(os.path.join(gSourceDir, gMemSetup['kernel'].sf))
)
bootVars = (
'bootdelay=0',
'baudrate=115200',
'stdin=serial',
'stdout=serial',
'stderr=serial',
'bootargs=console=ttyS0,115200 mtdparts=atmel_nand:256k(bootstrap)ro,512k(uboot)ro,256k(env),256k(env_redundant),256k(spare),512k(dtb),6M(kernel)ro,-(rootfs) rootfstype=ubifs ubi.mtd=7 root=ubi0:rootfs rw',
bootCfgStr
)
cfgMemCfg = bytearray(0x20000-5)
l = len(cfgMemCfg)
indx = 0
for v in bootVars:
l = len(v)
cfgMemCfg[indx:indx+l] = v.encode('utf-8')
indx += l + 1
crc = binascii.crc32(cfgMemCfg)
with open(os.path.join(gDestDir, gMemSetup['ubootEnv'].df), mode='wb') as f:
# crc32 as little endian
f.write(struct.pack('<I',crc))
f.write(bytearray(1))
f.write(cfgMemCfg)
f.close()
def _helper_genTCLFilename (idName):
pstr = os.path.join(gTCLfilepath ,gMemSetup[idName].df)
pstr = pstr.replace('/', '\\')
return pstr.replace('\\', '\\\\')
def _genFlashScript():
global gMemSetup
global gECCcnfg
global gTCLfilename
global gDestDir
global gArch
print("generate tcl script for sam-ba")
tclStr = """puts "-I- start flashing"
NANDFLASH::Init
NANDFLASH::NandHeaderValue HEADER 0x{pmeccConfig:X}
NANDFLASH::EraseAllNandFlash
NANDFLASH::SendBootFilePmeccCmd "{bootstrapFile}"
send_file {{NandFlash}} "{ubootFile}" 0x{ubootAddr:X} 0
send_file {{NandFlash}} "{ubootEnvFile}" 0x{ubootEnvAddr:X} 0
send_file {{NandFlash}} "{ubootEnvFileRed}" 0x{ubootEnvAddrRed:X} 0
send_file {{NandFlash}} "{dtbFile}" 0x{dtbAddr:X} 0
send_file {{NandFlash}} "{kernelFile}" 0x{kernelAddr:X} 0
NANDFLASH::NandSetTrimffs 1
send_file {{NandFlash}} "{rootfsFile}" 0x{rootfsAddr:X} 0
puts "-I- finished flashing"
""".format(
pmeccConfig=gECCcnfg[gArch],
bootstrapFile=_helper_genTCLFilename('bootloader'),
ubootFile=_helper_genTCLFilename('uboot'),
ubootAddr=gMemSetup['uboot'].add,
ubootEnvFile=_helper_genTCLFilename('ubootEnv'),
ubootEnvAddr=gMemSetup['ubootEnv'].add,
ubootEnvFileRed=_helper_genTCLFilename('ubootEnvRed'),
ubootEnvAddrRed=gMemSetup['ubootEnvRed'].add,
dtbFile=_helper_genTCLFilename('dtb'),
dtbAddr=gMemSetup['dtb'].add,
kernelFile=_helper_genTCLFilename('kernel'),
kernelAddr=gMemSetup['kernel'].add,
rootfsFile=_helper_genTCLFilename('rootfs'),
rootfsAddr=gMemSetup['rootfs'].add
)
with open(os.path.join(gDestDir, gTCLfilename), mode='w') as f:
f.write(tclStr)
f.close()
class _CLIError(Exception):
'''Generic exception to raise and log different fatal errors.'''
def __init__(self, msg):
super(_CLIError).__init__(type(self))
self.msg = "E: %s" % msg
def __str__(self):
return self.msg
def __unicode__(self):
return self.msg
def main(argv=None): # IGNORE:C0111
'''Command line options.'''
global gSourceDir
global gDestDir
global gTCLfilepath
if argv is None:
argv = sys.argv
else:
sys.argv.extend(argv)
program_name = os.path.basename(sys.argv[0])
program_version = "v%s" % __version__
program_build_date = str(__updated__)
program_version_message = '%%(prog)s %s (%s)' % (program_version, program_build_date)
program_shortdesc = __import__('__main__').__doc__.split("\n")[1]
program_license = '''%s
Created by Stefan Jaritz on %s.
Copyright 2016 organization_name. All rights reserved.
Licensed under the Apache License 2.0
http://www.apache.org/licenses/LICENSE-2.0
Distributed on an "AS IS" basis without warranties
or conditions of any kind, either express or implied.
USAGE
''' % (program_shortdesc, str(__date__))
try:
# Setup argument parser
parser = ArgumentParser(description=program_license, formatter_class=RawDescriptionHelpFormatter)
parser.add_argument("-s", "--source", dest="srcD", help="source path of the images [default: %(default)s]", default=os.getcwd())
parser.add_argument("-d", "--destination", dest="dstD", help="destination where the images and the flash script is copied [default: %(default)s]", default=os.getcwd() )
parser.add_argument("-t", "--tclFilePath", dest="tclD", help="TCL script path. The path where the TCL file gona be located [default: %(default)s]", default='' )
parser.add_argument('-V', '--version', action='version', version=program_version_message)
# Process arguments
args = parser.parse_args()
gSourceDir = args.srcD
gDestDir = args.dstD
gTCLfilepath = args.tclD
shutil.rmtree(gDestDir, ignore_errors=True)
os.makedirs(gDestDir)
print("start")
_copyFiles()
_genUBootCfg()
_genFlashScript()
print("finished")
return 0
except KeyboardInterrupt:
### handle keyboard interrupt ###
return -1
except Exception as e:
indent = len(program_name) * " "
sys.stderr.write(program_name + ": " + repr(e) + "\n")
sys.stderr.write(indent + " for help use --help")
return -2
if __name__ == "__main__":
sys.exit(main())
III.) Create own package
yocto-layer create own
bitbake-layers add-layer "../meta-own"
modify the example.bb
SRC_URI = "file://testFTP.c"
S = "${WORKDIR}"
DEPENDS ="curl"
do_compile() {
${CC} testFTP.c -o testFTP ${CFLAGS} -lcurl
}
do_install() {
install -d ${D}${bindir}
install -m 0755 testFTP ${D}${bindir}
}
now add in "layer.conf"
IMAGE_INSTALL_append = " vim testftp"
Now the vim editor and the testftp recipe, which includes the testFTP binary is installed at the "user/bin" at the rootfs.
After the flashing, start the the SAMA5D35 Evalboard. Login as "root". Type: "testFTP" or "vim" and enjoy the output.
Cheers!
I am trying to configure the vim Plugin "YouCompleteMe". My C++ project consists of many header files, which are spread all over the directory tree. In order to add header directories I have to add them in the ".ycm_extra_conf.py".
Excerpt:
'-I',
'./src/base/utils',
'-I',
'./src/base/modules',
But something like this does not work:
'-I',
'./src/base/*',
Is there a way to tell YCM to recursively search for header files?
Thank you.
I have added a new syntax -ISUB to include all sub-directories.
e.g.
"-ISUB./Pods/Headers/Public"
full .ycm_extra_conf.py here
import os
import ycm_core
flags = [
#custom definition, include subfolders
'-ISUB./Pods/Headers/Public',
'-I./Pod/Classes',
]
def Subdirectories(directory):
res = []
for path, subdirs, files in os.walk(directory):
for name in subdirs:
item = os.path.join(path, name)
res.append(item)
return res
def IncludeFlagsOfSubdirectory( flags, working_directory ):
if not working_directory:
return list( flags )
new_flags = []
make_next_include_subdir = False
path_flags = [ '-ISUB']
for flag in flags:
# include the directory of flag as well
new_flag = [flag.replace('-ISUB', '-I')]
if make_next_include_subdir:
make_next_include_subdir = False
for subdir in Subdirectories(os.path.join(working_directory, flag)):
new_flag.append('-I')
new_flag.append(subdir)
for path_flag in path_flags:
if flag == path_flag:
make_next_include_subdir = True
break
if flag.startswith( path_flag ):
path = flag[ len( path_flag ): ]
for subdir in Subdirectories(os.path.join(working_directory, path)):
new_flag.append('-I' + subdir)
break
new_flags =new_flags + new_flag
return new_flags
I had the same problem so I created a function that do it.
add the following to your ".ycm_extra_conf.py" right after the flags list:
import glob
flagsRec=['/opt/e17/include/*']
def AddDirsRecursively( flagsRec ):
global flags
new_flags = []
for flag in flagsRec:
for d in glob.glob(flag) :
if os.path.isdir(d):
new_flags.append('-I')
new_flags.append(d)
flags += new_flags
AddDirsRecursively( flagsRec )
where "flagsRec" is the list of dirs(regular expressions) you want to traverse and add to "flags"
Let's say I want to strip all the debug symbols in the shared libraries that I build whiling keeping the original file name.
I tried to add an command in the method:
def mySharedLibrary(self, *args, **kwargs):
# do some common work for every shared library like add a soname or append some lib files to LIBS parameter
target = SharedLibary(*args, **kwargs)
target = env.Command(target,target, "objcopy --strip-debug ${SOURCE}")
return target
I get this error: two different method was given to the same target,
I guess it's because the two targets returned by env.Command and SharedLibrary are exactly the same name.
Any ideas to do this?
Thanks in advance!
I had the same problem and got the same error. What I had to do was to create an intermediate target/library. The intermediate and final targets each had their own library name, so SCons doesnt get confused.
You could probably do something like the following:
env.SharedLibrary(target = 'namePreStrip', source = 'yourSource')
env.Command(target = 'name', source = 'namePreStrip', 'objcopy...')
I used objcopy to build a library out of several libraries. Here's the actual source code I implemented:
#
# Build an object file out of several other source files, objects, and libraries
# Optionally execute objcopy on the resulting library, depending if objcopyFlags
# has been populated
#
# env - SCons Environment used to build, Mandatory arg
# target - resulting library name, without LIBPREFIX and LIBSUFFIX, ej 'nsp2p',
# Mandatory arg
# sourceFiles - list of '.cc' files that will be compiled and included in the
# resulting lib, Optional arg
# objects - list of already compiled object files to be included in resulting lib,
# Optional arg
# libraries - list of libraries to be included in resulting lib, Optional arg
# objcopyFlags - list of flags to pass to objcopy command. objcopy will only
# be executed if this list is populated, Optional arg
#
# One of [sourceFiles, objects, or libraries] must be specified, else nothing
# will be performed
#
# Not using a custom builder because I dont like the way SCons prints the
# entire command each time its called, even if its not going to actually
# build anything AND I need more method args than provided by custom builders
#
def buildWholeArchive(self, env, target, sourceFiles, objects, libraries, objcopyFlags):
if len(sourceFiles) == 0 and len(objects) == 0 and len(libraries) == 0:
print "Incorrect use of buildWholeArchive, at least one of [sourceFiles | objects | librarires] must be specified, no build action will be performed"
return None
# Compile each source file
objNodes = []
if len(sourceFiles) > 0:
objNodes = env.Object(source = sourceFiles)
cmdList = []
cmdList.append(env['CXX'])
cmdList.append('-nostdlib -r -o $TARGET -Wl,--whole-archive')
for obj in objNodes:
cmdList.append(env.File(obj).abspath)
for obj in objects:
cmdList.append(env.File(obj).abspath)
for lib in libraries:
cmdList.append(lib)
cmdList.append('-Wl,--no-whole-archive')
cmd = ' '.join(cmdList)
libTarget = '%s%s%s' % (env['LIBPREFIX'], target, env['LIBSUFFIX'])
if len(objcopyFlags) > 0:
# First create the library, then run objcopy on it
objTarget = '%s%s_preObjcopy%s' % (env['LIBPREFIX'], target, env['LIBSUFFIX'])
preObjcopyTarget = env.Command(target = objTarget, source = [], action = cmd)
env.Depends(preObjcopyTarget, [objNodes, sourceFiles, objects, libraries])
objCmdList = [env['OBJCOPY']]
objCmdList.extend(objcopyFlags)
objCmdList.append('$SOURCE $TARGET')
objcopyCmd = ' '.join(objCmdList)
archiveTarget = env.Command(target = libTarget, source = preObjcopyTarget, action = objcopyCmd)
else:
# Just create the library
archiveTarget = env.Command(target = libTarget, source = [], action = cmd)
env.Depends(archiveTarget, [objNodes, sourceFiles, objects, libraries])
return archiveTarget
And here is how I called it:
sourceFiles = ['file1.cc', 'file2.cc']
libSource = []
if 'OcteonArchitecture' in env:
libSource.append(lib1)
libSource.append(lib2)
libSource.append(lib3)
objcopy = []
if 'OcteonArchitecture' in env:
objcopy.extend([
'--redefine-sym calloc=ns_calloc',
'--redefine-sym free=ns_free',
'--redefine-sym malloc=ns_malloc',
'--redefine-sym realloc=ns_realloc'])
archiveTarget = clonedEnv.buildWholeArchive(target = libName,
sourceFiles = sourceFiles,
objects = [],
libraries = libSource,
objcopyFlags = objcopy)
env.Alias('libMyLib', archiveTarget)
I'm using SCons for building a project and need to add a symbolic link to a file it is installing via env.Install. What command(s) will make a link that's the equivalent of running ln -s on the command line?
SCons doesn't have a dedicated symbolic link command, but you can use os.symlink(src, dst) from Python's os module:
import os
env = Environment()
def SymLink(target, source, env):
os.symlink(os.path.abspath(str(source[0])), os.path.abspath(str(target[0])))
env.Command("file.out", "file.in", SymLink)
This may not work correctly on Windows, I've only tried it on Linux.
There seems to be little advancement in the SCons core code for symbolic link support and I wasn't satisfied any one solution I found on the web. Here is a potential builder which incorporates aspects of both Nick's and richq's answers. Additionally, it will catch name changes (due to the emitter method) and is as platform-agnostic as I could get it.
I prefer this builder because it will make links relative to the directory in which they are installed. One could add an option to force the link to be absolute I suppose, but I have not needed or wanted that yet.
Currently, if the OS doesn't support symlinks, I just pass and do nothing, but one could use os.copytree() for example however the dependency becomes messy if the source is a directory so the emitter would need to do something fancy. I'm up for any suggestions here.
One can put the following code into the file site_scons/site_tools/symlink.py (with blank _init_.py files in the appropriate places). Then do this in the SConstruct file:
SConstruct:
env = Environment()
env.Tool('symlink')
env.SymLink('link_name.txt', 'real_file.txt')
symlink.py:
import os
from os import path
from SCons.Node import FS
from SCons.Script import Action, Builder
def generate(env):
'''
SymLink(link_name,source)
env.SymLink(link_name,source)
Makes a symbolic link named "link_name" that points to the
real file or directory "source". The link produced is always
relative.
'''
bldr = Builder(action = Action(symlink_builder,symlink_print),
target_factory = FS.File,
source_factory = FS.Entry,
single_target = True,
single_source = True,
emitter = symlink_emitter)
env.Append(BUILDERS = {'SymLink' : bldr})
def exists(env):
'''
we could test if the OS supports symlinks here, or we could
use copytree as an alternative in the builder.
'''
return True
def symlink_print(target, source, env):
lnk = path.basename(target[0].abspath)
src = path.basename(source[0].abspath)
return 'Link: '+lnk+' points to '+src
def symlink_emitter(target, source, env):
'''
This emitter removes the link if the source file name has changed
since scons does not seem to catch this case.
'''
lnk = target[0].abspath
src = source[0].abspath
lnkdir,lnkname = path.split(lnk)
srcrel = path.relpath(src,lnkdir)
if int(env.get('verbose',0)) > 3:
ldir = path.relpath(lnkdir,env.Dir('#').abspath)
if rellnkdir[:2] == '..':
ldir = path.abspath(ldir)
print ' symbolic link in directory: %s' % ldir
print ' %s -> %s' % (lnkname,srcrel)
try:
if path.exists(lnk):
if os.readlink(lnk) != srcrel:
os.remove(lnk)
except AttributeError:
# no symlink available, so we remove the whole tree? (or pass)
#os.rmtree(lnk)
print 'no os.symlink capability on this system?'
return (target, source)
def symlink_builder(target, source, env):
lnk = target[0].abspath
src = source[0].abspath
lnkdir,lnkname = path.split(lnk)
srcrel = path.relpath(src,lnkdir)
if int(env.get('verbose',0)) > 4:
print 'target:', target
print 'source:', source
print 'lnk:', lnk
print 'src:', src
print 'lnkdir,lnkname:', lnkdir, lnkname
print 'srcrel:', srcrel
if int(env.get('verbose',0)) > 4:
print 'in directory: %s' % path.relpath(lnkdir,env.Dir('#').abspath)
print ' symlink: %s -> %s' % (lnkname,srcrel)
try:
os.symlink(srcrel,lnk)
except AttributeError:
# no symlink available, so we make a (deep) copy? (or pass)
#os.copytree(srcrel,lnk)
print 'no os.symlink capability on this system?'
return None
This creates a builder to perform the job:
mylib = env.SharedLibrary("foobar", SRCS)
builder = Builder(action = "ln -s ${SOURCE.file} ${TARGET.file}", chdir = True)
env.Append(BUILDERS = {"Symlink" : builder})
mylib_link = env.Symlink("_foobar.so", mylib)
env.Default(mylib)
env.Default(mylib_link)
Again, this solution is for Linux.
If you wanted to issue the command directly to the shell and know the OS, subprocess can be used as well.
E.g.: subprocess.call(['ln', '-s', '</src/path>', '</dest/path>'])
In addition to Nicks solution, you can add a directory symlink by using a file as a directory name carrier. It's not the cleanest solution and debugging path names is a pain, but this works well:
def symlink_last(target_source_env):
src = os.path.basename(os.path.dirname(str(source[0])))
link = "deliverables/last"
print "Symlinking "+ src + "as" + link
os.symlink(src, link)
BUILD_TARGETS.append('link')
install_dir = "deliverables/subdir"
carrier_file = "filename"
builder = Builder(action = symlink_last, chdir=False)
env.Append(BUILDERS={ "Symlink" : builder })
env.Alias(target="link", source=env.Symlink(dir="deliverables", source = install_dir + carrier_file)
This will make a link to deliverables/subdir named deliverables/last, provided that a file deliverables/subdir/filename exists.