Adding non-python files to colcon build - python-3.x

Building my workspace with colcon, some OSM files which are found in a directory "OSM" in a sub_package in the workspace are not found in built space. So when I go to the install space, the files are not there. I am not sure how to do this and if I should put it in the setup.py.
I tried putting this in the setup.py file in arguments of setup():
setup(
name=package_name,
version='0.0.0',
packages=[package_name, submodules, osm],
data_files=[
('share/ament_index/resource_index/packages',
['resource/' + package_name]),
('share/' + package_name, ['package.xml']),
(os.path.join('share', package_name), glob('launch/*.launch.py')),
('.package_name/sub_package', glob('OSM_folder/*.osm')),
],
.
.
.
) # close setup()
but it did not work.
I am using ROS2 Galactic.
Directory structure:
package_name
┃
┣━━━━setup.py
┣━━━━package.xml
┣━━━━resource/
┣━━━━launch/
┗━━━━package_name
┗━sub_package_name
┗━OSM
┣━__init__.py
┗━some_osm_files.osm
I have the OSM directory in the built workspace but it has only the init.py file

I solved it. I am not sure if this is the right way of doing this or is there like another better/proper way or not, but here we go.
In the setup.py file, I added the line
(os.path.join('lib/python3.8/site-packages/package_name/sub_package/OSM'),glob(package_name+'/sub_package_name/OSM/*.osm')), in the data_files variable.
The first part of the new line which is os.path.join('lib/python3.8/site-packages/package_name/sub_package_name/OSM') determines the new location of the files in the install folder after building the workspace.
The second part which is glob(package_name+'/sub_package_name/OSM/*.osm') determines the files original location in the project workspace.
So the result is that it takes the files from the location mentioned in the second part and puts them in the location mentioned in the first part.
The resulting block is:
setup(
name=package_name,
version='0.0.0',
packages=[package_name, submodules, osm],
data_files=[
('share/ament_index/resource_index/packages',
['resource/' + package_name]),
('share/' + package_name, ['package.xml']),
(os.path.join('share', package_name), glob('launch/*.launch.py')),
(os.path.join('lib/python3.8/site-packages/package_name/sub_package_name/OSM'), glob(package_name+'/sub_package_name/OSM/*.osm')),
],
.
.
.
)

Related

Node creating file 1 level up from the current path

I was using path.resolve but this command created a monster folder that can't be deleted called lib..
path.resolve(__dirname + "../assets/pic/" + `${fileName}.png`)
Question 1
What is the propper usage to create a folder 1 level up from the current path?
Question 2
How to remove the lib../assets/pic folder? Deleting the entire project or using git reset --hard, git stash doesn't work because Windows 10 says the folder doesn't exist.
Answer to Question 1:
tl;dr
const fs = require('fs')
const folderName1DirUp = '../SomeFolder'
try {
if (!fs.existsSync(folderName1DirUp)){
fs.mkdirSync(folderName1DirUp)
}
} catch (err) {
console.error(err)
}
The back story:
Two ways to reference the current folder in a Node.js script. You can use ./ or __dirname. Note in addition to ./ you can also use ../ which points to the parent folder.
The difference between the two.
__dirname in a Node script will return the path of the folder where the current JavaScript file resides.
./ will give you the current working directory (and ../ the parent). For ./ a call to process.cwd(). returns the same thing.
Initially the current working directory is the path of the folder where you ran the node command, but during the execution of your script it can change by calling process.chdir(...).
There is one place where ./ refers to the current file path which is in a require(...) call. There the ./, for convenience, refers to the JavaScript file path which lets you import other modules based on the folder structure.
Thus the call to fs.mkdirSync('../SomeFolder') with the given folder name makes a folder one level up from the current working directory.
Answer to Question 2:
From a PowerShell prompt Remove-Item './../Folder1UpToDelete' -Force The ./ is for the current folder. The ../ is for one folder up from the current. Finally the Folder1UpToDelete is the one folder up from the current that you want to delete. The -Force makes sure to delete all sub-folders/files under the folder you want deleted including hidden and/or read-only files.
To answer the first question, to create a path 1 level up, you can use path.join():
path.join(__dirname, "../assets/pics", `${fileName}.png`);
For the second question, if deleting it through the explorer doesn't work, you can try:
fs.rmdirSync("E:/path/to/broken/folder..");
Using Git Bash and running
cd /c/path/to/broken/
rmdir folder..

how to force SCons to build with relative paths?

I have the following project structure:
/project
/src_dirs
/src_dir_1
/...
/include_dirs
/inc_dir_1
/...
/output
SConstruct
/Sconscripts
lib1_sconscript
lib2_sconscript
/...
objects/
/...
libs/
/...
The build process invoked from the output directory. so all the paths in the sconscripts, are relative to output directory. my sconscript files are auto generated. as you can see below, the paths to source files and to include files are relative paths. this is a demo sconscript file:
Import('libaEnv')
includes = ['../../project/include_dirs/inc_dir_1/be/cmn/inc', '../../project/include_dirs/inc_dir_1/be/be/cmn/api_inc']
directives = ['-CC','-c','-g']
code_switches = ['FAST_RUNG_MLK', 'EN_PRIORITY']
listDefines = code_switches
linkLibs = []
crt = libaEnv.Object(target='objects/crt.o', source='../../project/src_dirs/src_dir_1/crt.c', CPPDEFINES=listDefines, CPPFLAGS=directives, CPPPATH=includes)
ert = libaEnv.Object(target='objects/ert.o', source='../../project/src_dirs/src_dir_1/ert.c', CPPDEFINES=listDefines, CPPFLAGS=directives, CPPPATH=includes)
urt = libaEnv.Object(target='objects/urt.o', source='../../project/src_dirs/src_dir_1/urt.c', CPPDEFINES=listDefines, CPPFLAGS=directives, CPPPATH=includes)
liba = libaEnv.Library (target='libs/liba.a', source=[crt,ert,urt])
Return('liba')
I have seen that scons invokes the compiler with absolute paths to the source and the headers files. i have seen this by running scons with -verbose (i have also validate this by printing the command line in Action.py in the spawn method).
My scons version is: SCons 2.5.1 and i am running with python 2.7.
How can i force scons to invoke the compiler with relative paths only ?
One approach here is you can use top-relative addressing, as in:
includes = ['#project/include_dirs/inc_dir_1/be/cmn/inc', '#project/include_dirs/inc_dir_1/be/be/cmn/api_inc']
This still gets you absolute paths, but they'll be computed based on a valid starting point (# = "the directory containing the top-level SConsctruct", thus top-relative)

including folder and files inside a Python package

I'm trying to make a python package and I have most of the things already setup by when I try to install the library from Github here, it installs everything except for the folder called champs and it's files
This is my File directory structure
LeagueYue
champs
-Lname_num.json
-Lname_Uname.json
-num_Uname.json
-__init__.py
-champion_files.py
-external.py
-match.py
-rank.py
-status.py
-summoner.py
-requirements.txt
-setup.py
All the files are installed except for the folder and the files inside champs
As this question answers:
There are 2 ways to add the static files:
1) Include_package_data=True + MANIFEST.in
A MANIFEST.in file in the same directory of setup.py, that looks like this:
include src/static/*
include src/Potato/*.txt
2) Package_data in setup.py
package_data = {
'static': ['*'],
'Potato': ['*.txt']
}
Specify the files inside the setup.py.
Two of the files could probably be derived at runtime from num_Uname.json, but that's fine.
I do not yet see a data_files directive in https://github.com/CharmingMother/LeagueLib/blob/async/setup.py
Thomas Cokelaer suggests using an expression like
datafiles = [(datadir, list(glob.glob(os.path.join(datadir, '*'))))]
and then
setup(
...
    data_files = datafiles,
)
in http://thomas-cokelaer.info/blog/2012/03/how-to-embedded-data-files-in-python-using-setuptools/
In your case this could be as simple as:
data_files = [('', ['champs/num_Uname.json'])],
Martin Thoma explains you should access them using filepath = pkg_resources.resource_filename(__name__, path)
in How to read a (static) file from inside a Python package?
When I Read The Fine Manual, this setup.cfg alternative surfaces:
[options.data_files]
...
data = data/img/logo.png, data/svg/icon.svg
suggesting a line like . = champs/num_Uname.json or champs = num_Uname.json

conan.io: call exe with virtual run environment

I have a hello tool which contains only exe files (without source).
Hello tool structure:
bin
helloBin.exe
helloRoot.exe
conanfile.py
conanfile.py content:
class ToolHelloConan(ConanFile):
name = "ToolHello"
version = "0.1"
settings = "os", "compiler", "build_type", "arch"
def package(self):
self.copy("*")
def package_info(self):
self.cpp_info.libs = self.collect_libs()
I've exported the hello tool to local cache: conan export-pkg . ToolHello/0.1#user/testing. This copied all exe in local_cache/ToolHello/0.1/user/testing/package/hash/bin. The bin in local cache looks like this:
bin
helloBin.exe
helloRoot.exe
I've defined a tool integration project which contains only the conanfile.txt
[requires]
ToolHello/0.1#user/testing
[generators]
virtualrunenv
After running conan install . in tool integration project and activating the virtual run environment, I am able to call only the helloRoot.exe because it's located right in the bin directory, but I'm not able to execute the bin/bin/helloBin.exe
Question: How do I run exe files which are not located directly in the local_cache/ToolHello/0.1/user/testing/package/hash/bin, but in local_cache/ToolHello/0.1/user/testing/package/hash/bin/directory?
You need to define the bindirs that are not the default one (bin). Add this to your conanfile.py:
def package_info(self):
self.cpp_info.bindirs = ["bin", "bin/directory"]
If you need to also include the root of the package folder, you might need to use:
def package_info(self):
self.cpp_info.bindirs = ["", "bin", "bin/directory"]

scons fails to resolve relative source path

I have the following project structure:
/prj
SConstruct
/src
/app
/lib1
/lib2
/...
'/prj/src/lib1' structure:
/lib1
/src
/test
SConscript
'lib1/SConscript':
SConscript('test/test1/SConscript',
exports = 'env',
variant_dir = '#build/release/lib1/test',
duplicate = 0)
and, finally, 'test' directory:
/test
/common
helpers.cpp
/test1
SConscript
main.cpp
In 'test/test1/SConscript' sources specified as:
Sources = ['../common/helpers.cpp', 'main.cpp']
Result:
scons: *** [build/release/lib1/common/helpers.o]
Source `build/release/lib1/common/helpers.cpp' not found,
needed by target `build/release/lib1/common/helpers.o'
As can be seen the problem is that scons tries to find out the source file 'helpers.cpp' in build directory, not in source one.
Some research shows that the problem raised when source file path begins with '../'. When all sources defined underneath 'SConscript' file all is Ok.
Scons v2.5.1 and v3.0.1 demonstrate the same behavior.
What I did wrong? I've found this answer where the author advised:
You could use ../test.cpp as filename
but I doing exactly the same. Is such scons behavior intended or this is a bug?
Your "code" in "lib1/SConscript":
SConscript('test/test1/SConscript',
exports = 'env',
variant_dir = '#build/release/lib1/test',
duplicate = 0)
uses the name of the given "test/test1/SConscript" and implicitly links the folder "test/test1" as "variant folder" (=variant_dir) to the target directory "#build/release/lib1/test". So if a required file can't be found in "#build/release/lib1/test", SCons tries to do an alternative "lookup" in "test/test1/". But this link is not automatically set for the "common" folder as well, and that's why the lookup of the "helpers.cpp" fails. This is the intended behaviour and correct in SCons.
A solution for your current problem would be to move the "test/test1/SConscript" one folder level higher, include the sources there as "common/helpers.cpp" and "test/main.cpp" and call this new SConscript from "lib1/SConscript" as:
SConscript('test/SConscript',
exports = 'env',
variant_dir = '#build/release/lib1/test',
duplicate = 0)

Resources