Platform independent resource management [duplicate] - linux

This question already has answers here:
Is there a Linux equivalent of Windows' "resource files"?
(2 answers)
Closed 4 years ago.
I'm looking for a way to embed text files in my binaries (like windows resource system). I need something thats also platform independent (works in windows and linux). I found Qt resource management to be what I need but I'm not keen on my app depending on Qt for this alone. I also found this tool at http://www.taniwha.com/~paul/res/ .. but it is too platform specific.

The xxd utility can be used to create a C source file, containing your binary blobs as an array (with the -i command line option). You can compile that to an object which is linked into your executable.
xxd should be portable to most platforms.

If you're using QT 4.5, you can make sure that program is only dependent on one small piece of QT, such as libqtcore. QResource is a part of libqtcore.

You can simlpy append all kinds of data to your normal binary. Works in both Windows and Linux. You'll have to open your own binary at runtime and read the data from there.
However, I have to agree that embedding data in binaries is a strange idea. It's common practice to include such data as separate files packaged with the application.

That is not such a great idea. On Linux, for example, data is expected to be installed in a subdirectory of "$datadir" which is, by default, defined to be "$prefix/share", where "$prefix" is the installation prefix. On Mac OS X, resources are expected to be installed in $appbundle/Contents/Resources, where $appbundle is the name of the folder ending in ".app". On Windows, installing data in a folder that is a sibling of the executable is not an uncommon practice. You may be better off using the CMake build system, and using its CPack packaging features for installing/bundling in the default, preferred platform-specific manner.
Although bundling your resources into the executable, itself, may seem cool, it is actually a dangerous idea... for example, will the embedded data be allocated in an executable page? What will happen if you attempt to overwrite or modify the data? What if you want to tweak or modify the data at runtime? Things to think about.

This looks very promising: https://github.com/cyrilcode/embed-resource
CMake based and platform-independent.

As I also do not like the idea of converting files into C arrays only to have them converted back to binaries, I created my own resource compiler using LLVM and Clang:
https://github.com/nohajc/resman
I tested it on Windows, Linux and macOS but it can potentially be run on any platform supported by LLVM.
It is used like this:
Create header file, e.g. res_list.h
#pragma once
#include "resman.h"
// Define a global variable for each file
// It will be used to refer to the resource
constexpr resman::Resource<1> gRes1("resource_file1.jpg"); // resource with ID 1
constexpr resman::Resource<2> gRes2("resource_file2.txt"); // resource with ID 2
constexpr resman::Resource<3> gRes3("resource_file3.mp3"); // resource with ID 3
...
Run resource compiler
$ rescomp res_list.h -o res_bundle.o
Link res_bundle.o to your project
Use the resource files
#include "res_list.h"
...
resman::ResourceHandle handle{gRes1};
// ResourceHandle provides convenient interface to do things like:
// iterate over bytes
for (char c : handle) { ... }
// convert bytes to string
std::string str{handle.begin(), handle.end()};
// query size and id
unsigned size = handle.size();
unsigned id = handle.id();
The resource compiler parses res_list.h (using Clang) but instead of generating cpp files, it goes straight to the native object file (or static library) format (using LLVM).

Related

Rcpp with structured library in /src

I'm trying to write a wrapper for a C++ function I've written, making use of the Point Clouds Library (PCL). This is my first try interfacing R and C++, so I apologise if any solution is too trivial. My goal is to make a few functions available for myself and my colleagues directly in R, on mac and windows. My example function cloudSize is included at the bottom of the text. I will try to be as clear as possible.
I've installed PCL with the vcpkg package manager for winx64 at C:\src\vcpkg\vcpkg.
This is added to my Environmental Variable Path for my user.
I created an empty R-package with Rcpp.package.skeleton():
C:/User/csvi0001/Desktop/GitHub/RPCLpackage/PCLR
PCL is a massive library, but thankfully modular,and so I only #include the headers that are needed to compile the executable: pcl/io/pcd_io.h, pcl/point_types.h, pcl/registration/icp.h.
Now, since I'd like this to work on more than one OS - and therefore compile on install (?) - I should use a dynamic library? I'll presume that the person installing my package already has a compiled copy of pcl. However, I do not know how to find a flag showing that pcl is installed - how do I find these for inclusion in Makevars(?). CMake must find them when testing the C++ function in VSCode after adding an include path. In lieu of this:
I copy the pcl folder installed by vcpkg to ./src . When I tried copying all the .h files, they seemed to lose track of one another as they refer to eachother through which module they are placed in, e.g. <pcl/memory.h> cannot be found if memory.h is placed directly in ./src. However, flattening the structure of the modules means that every single dependency and #include must be manually changed, in some cases there are also files with the same name in different folders. e.g. pcl/kdtree.h and pcl/search/kdtree.h. After this, it must be done again when replacing < > with " " for each header.
Is there any way of telling Rcpp that the library included in /src is structured?
I'm working on Win 10 winx64.
Since I'm making use of the depends RcppEigen and BH; and I must have C++14 or higher (choice: C++17) I add to my DESCRIPTION file:
LinkingTo: Rcpp, RcppEigen, BH
SystemRequirements: C++17
My actual C++ function:
//PCL requires at least C++14
//[[Rcpp::plugins(cpp17)]]
//[[Rcpp::depends(RcppEigen)]]
//[[Rcpp::depends(BH)]]
#include <Rcpp.h>
#include <iostream>
#include "pcl/io/pcd_io.h"
#include "pcl/point_types.h"
#include "pcl/registration/icp.h"
//[[Rcpp::export]]
int cloudSize(Rcpp::DataFrame x)
{
pcl::PointCloud<pcl::PointXYZ> sourceCloud;
for(int i=0;i<x.nrows();i++)
{
sourceCloud.push_back(pcl::PointXYZ(x[0][i],x[1][i],x[2][i])); //This way of referring to elements in a Rcpp::DataFrame may be erroneous.
}
int cloudSize = sourceCloud->size();
return (cloudSize);
}
That is a non-trivial question. In the simplest case, use a 'hook' offered by configure and configure.win to pre-build a (static) library you ship in your sources and then link your package to that.
That said, the Writing R Extensions manual and/or the CRAN Repository Policy (both of which are the references here) expressed more of a preference for an external library -- which may not be an option here if PCL is too exotic.
As the topic comes up with Rcpp, I wrote a short paper about it (at arXiv here) which is also included as a vignette in the package. It requires a few pages to cover the common cases but even then it cannot cover all.
Your main source of reference may be CRAN. The are lots of packages in this space. A few of mine use external libraries, I contributed to package nloptr which uses a hybrid approach ("use system library if found, else build") and some like httpuv always build (a small-ish library).

Delphi Linux64: how to retrieve the version information set by Project Options Version?

I gather there is no universal standard for putting version numbers into executables on Linux, in contrast to Windows which requires a certain structure for those details.
FreePascal has made its own standard and Delphi lets you define Version information for a Linux executable.
If we can put Version information in, we must be able to get it back out? How? Specifically on Linux64?
I have searched *.pas and *.inc in Studio\19.0\source\rtl\posix and I have not found anything on 'version' nor 'fileinfo' that could help.
Back with Kylix, I used to use argp_program_version from libc.
On a clean new DUnitX project, after adding {$ *.res} to the DPR so that the version information will stick, I can look at Project Options for Linux64 and I can see that they have a CFBundleVersion number defaulting to 1.0.0 for the project. How can I get the CFBundleVersion at runtime in my Linux64 executable?
Or, if that is not possible, could/should Delphi match the FreePascal standard and put the fileinfo into something which both Lazarus and Delphi could view at runtime??
This excerpt (metioned at both Windows and MacOS/iOS sections) from https://docwiki.embarcadero.com/RADStudio/Alexandria/en/Version_Info may help:
Go to Tools > Options > IDE > Environment Variables and add a new User variable called SAVEVRC with Value=TRUE. This variable enables the generation of a .vrc file with an auto generated build number and other information.
another excerpt from there (not sure if related to the aforementioned SAVEVRC or not) is:
Note: If you define the VersionInfo resource and add it to the project via a $R directive in the project file, the compiler will ignore settings of Version Info at Project Options dialog box and use the custom resource data instead, given an application can have only one VersionInfo resource on Windows.

How to generate a .so file

I am writing an extension for the Scratch text editor application on Elementary OS Luna. But the documentation is practically non-existent for extension creation and I have no idea how to proceed after writing my main code for the extension.
I've already written the extension. I cannot use or test it yet as it needs to be "installed". I spent hours looking for docs but they do not exist. I did however, find a comment on the scratch launchpad page that says
Generally you have to generate a pluginname.so file and put it in
lib/scratch/plugins/pluginname with a pluginname.plugin file
Great. This seems like the last part of creating an extension for Scratch. What is a .so file, and how do I generate one? I've already created the other necessary files like the .plugin file and the .vala file.
Yes, I have searched for .so files but all I found were random things about it like using it with C, or C++ headers but obviously that won't work for me, since I'm using Vala?
How do I generate a .so file? What do I generate it from?
I think the The Vala tutorial could be helpful on creating the actual Shared library or Shared Object.
You can use the autotools, cmake or you can compile and link a shared library directly from the command line, quoted from the Vala tutorial:
Compilation and linking using Command Line
Vala is not yet capable of directly creating dynamic or static
libraries. To create a library, proceed with the -c (compile only)
switch and link the object files with your favourite linker, i.e.
libtool or ar.
$ valac -c ...(source files)
$ ar cx ...(object files)
or by compiling the intermediate C code with gcc
$ valac -C ...(source files)
$ gcc -o my-best-library.so --shared -fPIC ...(compiled C code files)...
From the Scratch Wiki
Due to browser security restrictions, Scratch 2.0 cannot interact with
hardware devices directly. Instead, hardware extensions come with a
helper app, a separate application that the user must install and run
on their computer. Scratch communicates with the helper app via HTTP
requests, and the helper app talks to the hardware. In the future,
some extensions may package their helper apps as browser plugins.
Here are the steps for creating and testing a Scratch extension:
Create an extension description file
Create your helper app and start it
Open the Scratch 2 Offline Editor
Import the extension description (shift-click on "File" and select "Import Experimental Extension" from the menu)
The new extension blocks will appear in the More Blocks palette
Test your extension and iterate!
Helper apps can be written in any language that supports server sockets, such as Python, Node.js, Java, C, etc.
Like you I've chased around the wiki but I cannot find an example with source. So all I can do is to address your question generally.
If you look at Build .so file from .c file using gcc command line you'll see how a simple .so can be created. However what code goes into a Scratch extension I don't know. Sorry.
Edit: More googling and I've found a sample from Nathan Dinsmore on GitHub that doesn't use C rather it uses JavaScript, and though it doesn't create a .so extension, it does have what appears to be a full description of creating an extension. He also provides a couple of tutorials.
Edit++ : And here is another sample written in Java.

How to embed version information into shared library and binary?

On Linux, is there a way to embed version information into an ELF binary? I would like to embed this info at compile time so it can then be extract it using a script later. A hackish way would be to plant something that can be extracted using the strings command. Is there a more conventional method, similar to how Visual Studio plant version info for Windows DLLs (note version tab in DLL properties)?
One way to do it if using cvs or subversion is to have a special id string formatted specially in your source file. Then add a pre-commit hook to cvs or svn that updates that special variable with the new version of the file when a change is committed. Then, when the binary is built, you can use ident to extract that indformation. For example:
Add something like this to your cpp file:
static char fileid[] = "$Id: fname.cc,v 1.124 2010/07/21 06:38:45 author Exp $";
And running ident (which you can find by installing rcs) on the program should show the info about the files that have an id string in them.
ident program
program:
$Id: fname.cc,v 1.124 2010/07/21 06:38:45 author Exp $
Note As people have mentioned in the comments this technique is archaic. Having the source control system automatically change your source code is ugly and the fact that source control has improved since the days when cvs was the only option means that you can find a better way to achieve the same goals.
To extend the #sashang answer, while avoiding the "$Id:$" issues mentioned by #cdunn2001, ...
You can add a file "version_info.h" to your project that has only:
#define VERSION_MAJOR "1"
#define VERSION_MINOR "0"
#define VERSION_PATCH "0"
#define VERSION_BUILD "0"
And in your main.c file have the line:
static char version[] = VERSION_MAJOR "." VERSION_MINOR "." VERSION_PATCH "." VERSION_BUILD;
static char timestamp[] = __DATE__ " " __TIME__;
(or however you want to use these values in your program)
Then set up a pre-build step which reads the version_info.h file, bumps the numbers appropriately, and writes it back out again. A daily build would just bump the VERSION_BUILD number, while a more serious release would bump other numbers.
If your makefile lists this on your object's prerequisite list, then the build will recompile what it needs to.
The Intel Fortran and C++ compilers can certainly do this, use the -sox option. So, yes there is a way. I don't know of any widespread convention for embedding such information in a binary and I generally use Emacs in hexl-mode for reading the embedded information, which is quite hackish.
'-sox' also embeds the compiler options used to build an executable, which is very useful.
If you declare a variable called program_version or similar you can find out at which address the variable is stored at and then proceed to extract its value. E.g.
objdump -t --demangle /tmp/libtest.so | grep program_version
0000000000600a24 g O .data 0000000000000004 program_version
tells me that program_version resides at address 0000000000600a24 and is of size 4. Then just read the value at that address in the file.
Or you could just write a simple program that links the library in questions and prints the version, defined either as an exported variable or a function.

VC++ resources in a static library

Is it possible to build resources into a static library and reuse them by simply linking with the library?
I'm primarily thinking about the case where you call a function in the library which in turn accesses resources.
The only thing you need to do to use resources (images, dialogs, etc...) in a static library in Visual C++ (2008), is include the static library's associated .res file in your project. This can be done at "Project settings/Linker/Input/Additional dependencies".
With this solution, the resources of the static library are packed into the .exe, so you don't need an extra DLL. Regrettably, Visual Studio does not include the .res file automatically as it does for the .lib file (when using the "project dependencies"-feature), but I think this small extra step is acceptable.
I have looked for a very long time for this solution, and now it surprises me it is that simple. The only problem is that it is totally undocumented.
It can be done, but it's quite painful: You can't do it by simply linking with the static library.
Consider this: resources are embedded in an EXE or DLL. When some code in the static library calls (e.g.) LoadIcon, it'll get the resources from the EXE or DLL that it's linked with.
So, if your static library requires resources to be available, you've got a couple of options:
You can have the library build them on the fly, and then use (e.g.) CreateDialogIndirect. See Raymond Chen's "Building a dialog template at run-time".
You can have them embedded in the library as simple arrays (i.e.) char my_dialog_resource[] = { .... };, and then use (e.g.) CreateDialogIndirect. You'll probably need to find (or write) a utility that converts from .RES files to .CPP files.
You can ship the LIB file with a resource script (.RC file) and corresponding header file. You then #include them as relevant. You'll need to reserve a range of resource IDs for the LIB to use, so that they don't collide with those of the main EXE or DLL. This is what MFC does when used as a static library. Or you can use string resource IDs (this doesn't work for STRINGTABLE resources).
Your static library can ship with a separate resource DLL.
I just went through this with the MS Visual Studio compiler. We were converting some legacy projects from DLLs into static libraries. Several of these DLLs had dialog or string resources embedded in them. I was able to compile the .RC scripts for these DLLs into our main application by including them in the main application's RC script file via the "TEXTINCLUDE" mechanism. I found it easiest to do this by editing the RC file directly, but Visual Studio provides a slightly more "wizardy" mechanism as well. The implementation is most likely different in other compilers.
To manipulate the main RC script directly:
.1. In the "2 TEXTINCLUDE" section, include the header file that defines the resource IDs for your library. The syntax is
2 TEXTINCLUDE
BEGIN
"#include ""my_first_lib_header.h""\r\n"
"#include ""my_second_lib_header.h""\0"
END
.2. In the "3 TEXTINCLUDE" section, include the RC script from your library.
3 TEXTINCLUDE
BEGIN
"#include ""my_first_library.rc""\r\n"
"#include ""my_second_library.rc""\0"
END
Steps 3 and 4 should happen automatically, but I found it was more reliable to just enter them myself, rather than depending on Microsoft's resource script compiler to take care of things.
.3. Add the header file with your libraries resource defines to the read only symbols list. This list is usually near the top of the file.
#define APSTUDIO_READONLY_SYMBOLS
#include "my_first_lib_header.h"
#include "my_second_lib_header.h"
#undef APSTUDIO_READONLY_SYMBOLS
.4. Include your library's RC script in the APSTUDIO_INVOKED section. This is usually at the bottom of the file.
#ifndef APSTUDIO_INVOKED
#include "my_first_library.rc"
#include "my_second_library.rc"
#endif
You can also do all of this automatically through the visual studio IDE, but I found it didn't always apply when I expected it to.
Open the "Resource View" window in Visual Studio.
Right-click on your main application's resource file and choose "Resource Includes..." from the context menu.
In the box labeled "Read-only symbol directives," add the include statements for the .h files that define the resource ID's for your libraries.
In the box labeled "Compile-time directives," add the include statements for your library's .rc script.
Click okay. You may also want to manually trigger the RC script compilation, to make sure it happens.
If your library's resource script references any files on disk (text files, icons files, etc.), you'll need to make sure that the main application project knows where to find them. You can either copy these files to somewhere your application can find them or you can add an additional include path in the compiler settings.
To add an additional include path:
Open up the properties dialog for your main application.
Select "Configuration Properties/Resources/General" from the left-hand navigation pane.
In the properties list, Enter any pertinent paths next to "Additional Include Directories."
As per Visual Studio 2010, the development tools from Microsoft apparently cannot properly handle compiled resource data inside static libraries at all.
To distribute a compiled resource file (a .res file), you have two choices:
Distribute the .res files separately, and instruct the client code to link against them;
Use cvtres to merge several .res files into a single object (.obj) file, and provide it separately.
Note that you can't lib in object files created with cvtres. If multiple object files are provided, lib complains as though as multiple .res files were given; if a single object file is provided, lib does not complain, but the linker simply ignores the embedded resource data in the lib file.
It might be the case that there is a way to force the linker to read and link the libbed in resource data (with some command-line option, section manipulation and so on), since the resource data is indeed available in the library (as dumpbin reveals). So far, I haven't found a solution, and, unless one is willing to hack the development tools, anything better than this simple solution is probably not worth the effort.
The only way to ship resource data in a static library (in this case, with a static library) is to distribute the resources separately and explicitly link them in the client code. Using cvtres can reduce the number of distributed resource files to one, if you have many of them.
I don't think so. Static library doesn't have it's own HINSTANCE. It's code is executed in the context of DLL or EXE which links it. That's why all the resources you'll try to load from the static library's code will be of that enclosing DLL/EXE.
I did that kind of resources reuse with a DLL though, as far as it has it's own address space, and you can call LoadResource with DLL's HINSTANCE.
The recommended way is to provide a dll with the resources together with your library.
When the following method is used, any resource (in this example, an icon) can be used as an integral part of a static library and such library can be used by any type of application, including a console one (which doesn't have any resource segment whatsoever).
Icon is converted to a static array of BYTE. bin2c can be used for that.
Data is converted into a HICON handle. Here is how I have done that:
HICON GetIcon()
{
DWORD dwTmp;
int offset;
HANDLE hFile;
HICON hIcon = NULL;
offset = LookupIconIdFromDirectoryEx(s_byIconData, TRUE, 0, 0, LR_DEFAULTCOLOR);
if (offset != 0)
{
hIcon = CreateIconFromResourceEx(s_byIconData + offset, 0, TRUE, 0x00030000, 0, 0, LR_DEFAULTCOLOR | LR_DEFAULTSIZE);
}
return hIcon;
}
GetIcon is used instead of LoadIcon.
Instead of calling:
m_hIcon = ::LoadIcon(hInstanceIcon, MAKEINTRESOURCE(pXMB->nIcon));
Then call
m_hIcon = GetIcon()

Resources