Why doesn't libicudata.so get smaller when I remove items from the data library? - linux

I'm trying to build a custom ICU with a minimal data set. I've tried to follow the instructions at Reducing the Size of ICU's Data: Conversion Tables, but many of the files referenced don't exist in the ICU 4.8.1 source distribution. Specifically, I cannot find any files that match ucm*.mk.
I've also tried creating reslocal.mk files as indicated in e.g. ICU's source/data/lang/resfiles.mk. That did not help either. My build is the typical:
$ ./configure --prefix=/some/dir
$ make
$ make install
Regardless of what I do, libicudata.so.48.1 is about 17M. It shouldn't matter, but I'm building on Ubuntu 11.04.

See the note at the top of that page: (see
Note that ICU for C by default comes with pre-built data. The source
data files are not included unless ICU is downloaded from the
source repository. Alternatively, the Data Customizer may be used to
customize the pre-built data.
Your ICU is reading the prebuilt package from icu/source/data/in/*.dat and ignoring the .mk files. We have had requests for the source data to be included as a downloadable .zip and so we plan to do this in the future.
If you have any suggestions for how our instructions can be made more clear, please file a bug. I've added a copy of that notice to the section you referenced.

Related

wxWidgets fails to build due to missing wxxml.lib

Apparently anything GUI-related in terms of components involves XML. I cannot go around actually configuring and building wxWidgets from source because of that. I'm new to wxWidgets.
My current setup is on Win10 with MSVC v141 (Visual Studio 2017) with the latest CMake version (currently 3.21).
Inside the config.cmake of the wxWidgets projects (using latest master branch) I see
wx_get_dependencies(EXTRALIBS_XML xml)
I am also calling CMake with -DwxUSE_XML=ON (among other parameters) but this still leads to:
the XML dependency is nowhere to be found
respectively it's not built
Linking then fails with the following error:
LINK : fatal error LNK1104: cannot open file 'wxxml.lib' [C:\Users\...\CMakeBuilds\ef5b5ada-ee42-7735-988a-ae37c735ccff\build\deps\build\wxwidgets\libs\qa\wxqa.vcxproj]
What library is actually wxWidgets using and how do I trigger it's retrieval and accordingly configuration and building? Since I am adding wxWidgets to my CMake project as an ExternalProject component, I would appreciated something in that line of thought. However any kind of information regarding this issue is more than welcome especially since it will shine light on how to configure other features (if I want them in the future) such as WebView.
The wxxml.lib issue is fixed now. While fixing it I also discovered a bug (of sort) in the build system of wxWidgets.
The reason why it failed to build this library in particular was actually quite simple but due to the lack of knowledge in the dependencies of wxWidgets. I thought that wxWidgets, given it depends on XML so much, has its own XML parser. Well, not really. The wxXML component actually uses and underlying 3rd party dependency called EXPAT, which - as you can see in my question - I have deactivated since it was giving me issues during the build (due to the still present problem of not being able to automatically retrieve dependencies).
What I did was to clone the libexpat repository, add it as an ExternalProject, set the variables for the libraries and include directory and pass them onto my wxWidgets project. But there is a catch...
The expat.cmake file looks as follows:
#############################################################################
# Name: build/cmake/lib/expat.cmake
# Purpose: Use external or internal expat lib
# Author: Tobias Taschner
# Created: 2016-09-21
# Copyright: (c) 2016 wxWidgets development team
# Licence: wxWindows licence
#############################################################################
if(wxUSE_EXPAT STREQUAL "builtin")
# TODO: implement building expat via its CMake file, using
# add_subdirectory or ExternalProject_Add
wx_add_builtin_library(wxexpat
src/expat/expat/lib/xmlparse.c
src/expat/expat/lib/xmlrole.c
src/expat/expat/lib/xmltok.c
)
set(EXPAT_LIBRARIES wxexpat)
set(EXPAT_INCLUDE_DIRS ${wxSOURCE_DIR}/src/expat/expat/lib)
elseif(wxUSE_EXPAT)
find_package(EXPAT REQUIRED)
endif()
I would use the *.cmake files of the 3rd party dependencies stored inside <ROOT_OF_WXWIDGETS_PROJECT>/build/cmake/lib to determine which variables I need to set if builtin is selected as the value for the respective library. Since I want to use my own I need sys (e.g. -DwxUSE_EXPAT=sys as a CMAKE_ARGS inside my wxWidgets ExternalProject) and also to pass the headers and libraries accordingly.
Given the file above one would assume that EXPAT_LIBRARIES is required. However after failing to build (yet again) and seeing that the reason was the activated expat build and that it was set as builtin I checked the log in detail and found the following error:
Could NOT find EXPAT (missing: EXPAT_LIBRARY) (found version "2.2.6")
Notice the EXPAT_LIBRARY. After passing it (-DEXPAT_LIBRARY=...) my build was complete. For me this is a bug or simply inconsistency between the dependency cmake file and the rest of the wxWidgets project.
It is important to note that I do not retrieve the external dependency through wxWidgets itself (see config.cmake and more precisely the macro wx_get_dependencies(...)). This solves the problem with a basic configuration and build of wxWidgets but if you don't want to tackle every dependency of wxWidgets on your own (why should you?), I recommend looking for a solution where the dependencies (at least the ones you don't want to deal with) are automatically retrieved, configured and build as builtin.

Where do I find the source package for `mingw64/mingw-w64-x86_64-gcc-libs` 8.2.0-3?

I need the source package for the mingw64/mingw-w64-x86_64-gcc-libs-8.2.0-3 package. I believe this is generated from the mingw-w64-x86_64-gcc source package. Looking in the repository I can find mingw-w64-gcc-7.3.0-2.src.tar.gz but nothing for gcc-8.*.
Most other packages have a simple relationship between binary and source package names. In a few cases (GCC is one) a single source package is used to generate multiple binary packages. However the naming is usually fairly obvious, and version numbers stay the same. I can't find any GCC-related source packages with the "8.2.0" version number.
Does anyone know where to get the source for the gcc-libs mingw package from?
You might want to look at here. You can find GCC 8.1.0.
Open an issue on https://github.com/Alexpux/MINGW-packages/issues
Apparently the maintainer forgot to upload the source package, or some script is failing.

CMakeLists unable to find newly installed HDF5?

I'm working on a program that worked on another computer (1st one) and trying to get it to work on my new computer (3rd one). When I go to build and type ccmake .., I get this error:
CMake Error at build/share/cmake/hdf5/hdf5-targets.cmake:67 (message):
The imported target "hdf5" references the file
"/home/myname/Desktop/MyProject/build/lib/libhdf5.a"
but this file does not exist. Possible reasons include:
* The file was deleted, renamed, or moved to another location.
* An install or uninstall procedure did not complete successfully.
* The installation package was faulty and contained
"/home/myname/Desktop/MyProject/build/share/cmake/hdf5/h
df5-targets.cmake"
but not all the files it references.
Call Stack (most recent call first):
build/share/cmake/hdf5/hdf5-config.cmake:70 (INCLUDE)
build/share/cmake/hdf5/FindHDF5.cmake:85 (INCLUDE)
src/Thing/CMakeLists.txt:66 (find_package)
Even when I then installed hdf5 using sudo apt-get, I got the same error
I had to delete that lib directory because it contained boost files that are outdated and caused a huge list of errors, possibly because of a conflict since I newly installed boost files in usr/lib and usr/include on this new computer. I got this program to work on a 2nd computer by simply deleting that lib directory
Is the way to resolve this to change the CMakeLists.txt so that it refers to the directory of the newly installed hdf5? The CMakeLists.txt looks like:
CMAKE_MINIMUM_REQUIRED (VERSION 2.6)
PROJECT (Projectname)
# CMake Modules
SET(CMAKE_MODULE_PATH
${PROJECT_BINARY_DIR}/share/cmake/hdf5
${CMAKE_SOURCE_DIR}/CMake
${CMAKE_MODULE_PATH})
SET(ENV{HDF5_ROOT_DIR_HINT} ${PROJECT_BINARY_DIR})
# Build output
set(LIBRARY_OUTPUT_PATH "${CMAKE_BINARY_DIR}/lib")
set(EXECUTABLE_OUTPUT_PATH "${CMAKE_BINARY_DIR}/bin")
mark_as_advanced(LIBRARY_OUTPUT_PATH EXECUTABLE_OUTPUT_PATH)
#other stuff I excluded to keep this brief
Also, even if I delete all the files in the lib directory at home/myname/Desktop/MyProject/build/lib except for libhdf5.a, I get a bunch of boost errors. Its a very long output and I think its because make creates new boost files in lib that were originally from home/myname/Desktop/MyProject/build/lib, instead of referring to the newly installed boost files in usr/lib and usr/include
In case it helps, line 66 of src/Thing/CMakeLists.txt looks likefind_package(HDF5 COMPONENTS C CXX)
I could also post the contents of hdf5-config.cmake and FindHDF5.cmake, if necessary
EDIT: dpkg -l | greap hdf5 gives
ii hdf5-helpers 1.8.15-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - Helper tools
ii hdf5-tools 1.8.15-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - Runtime tools
ii libhdf5-10:amd64 1.8.15-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - runtime files - serial version
ii libhdf5-cpp-10:amd64 1.8.15-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - C++ libraries
ii libhdf5-dev 1.8.15-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - development files - serial version
ii libhdf5-openmpi-10:amd64 1.8.15-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - runtime files - OpenMPI version
ii libhdf5-openmpi-dev 1.8.15-patch1+docs-4 amd64 Hierarchical Data Format 5 (HDF5) - development files - OpenMPI version
ii libhdf5-serial-dev 1.8.15-patch1+docs-4 all transitional dummy package
A not so short introduction to linking of third-party libraries on Linux:
Generally you should decide 2 aspects.
The first one is about "system vs custom library"
you may link your application against a library installed from a system package (in Debian using apt-get). This is the preferred way to compile programs on Linux unless you have some reasons against (see below). Also usually this is the easiest way, 'cause all build tools are configured to find library components in certain "well-known" places.
Link against a custom build of a library. Use this method if you need some custom build options for a library, or need a specific version of library not available in repostitories and so on.
The second step is to choose between static and dynamic library.
Usually in Linux applications are linked against dynamically loaded libraries (.so). This reduces applications size, may decrease memory consumption and application loading time (because libraries for a given application are in memory already), and sometimes simplifies small updates like security fixes - one needs to update just a single library instead of all applications that use this vulnerable library
But there're drawbacks as well :) If you plan to distribute your application linked with dynamic libraries to other computers you need to make sure that the libraries of required versions are on a given target computer. Each library may in turn be linked with other libraries and so on. In the end you have to distribute a whole copy of an operating system, like e.g. Steam for Linux does: internally it carries a large part of Ubuntu. So sometimes it may be wiser to link against a static library and include everything needed rignt into your app.
Back to the practical questions. Since you've already installed system packages with libhdf5 development files (libhdf5-dev) I suggest you to go with the "system library" (p1.1).
In this case you should remove all other copies of the library from your build tree completely and use the following code chunk in your CMakeLists.txt:
find_package(HDF5 REQUIRED)
...
include_directories(${HDF5_INCLUDE_DIRS})
...
target_link_libraries(yourapp ${HDF5_LIBRARIES} <other required libs>)
if your application is a C++ app you may use ${HDF5_CXX_LIBRARIES} instead of ${HDF5_LIBRARIES}.
If you decided to link against a static library, then you need to define a special variable HDF5_USE_STATIC_LIBRARIES before calling find_package(HDF5...):
set(HDF5_USE_STATIC_LIBRARIES ON)
find_package(HDF5 REQUIRED)
...
If you still wish to use a custom build of the library then you should
"explain" to find_package where to look for library components. You may run make DESTDIR=/some/path install in the HDF5 build tree and then use that /some/path as the value of HDF5_ROOT environmental variable when running cmake in your application build tree:
HDF5_ROOT=/some/path cmake .
In this case CMake will try to locate HDF5 components in that directory. I would remove sytem package libhdf5-dev before this operation (just in case) using apt-get remove libhdf5-dev.
But again I believe that you should have good reasons not to use the system libhdf5 package.

Using library with different names within autoconf

I am trying to build an application with OpenSync 0.4 (0.3.9 indeed) dependency.
In the project's configure.ac the opensync library is written as libopensync1. However, this doesn't build on my Gentoo system. Changing libopensync1 to libopensync does fix the issue for me.
I searched with Google and found that libopensync1 is used in some distributions, while libopensync in others. So how to resolve this issue in configure.ac?
Thanks.
The macro AC_SEARCH_LIBS does what you need. (There is much heated debate about whether or not pkg-config should ever be used. If you choose to rely on it, ptomato gives a reasonable approach.) Simply add this to your configure.ac:
AC_SEARCH_LIBS([osync_mapping_new],[opensync1 opensync],[],
[AC_MSG_ERROR([can't find opensync])])
This will first look for a library named opensync1; if
it doesn't find that, it will look for opensync.
The primary drawback of using pkg-config is that most projects that
rely on it do not actually check if the data provided by the .pc
file is reliable, so configure may succeed but a subsequent build
will fail. It is always possible for a user to set PKG_CONFIG=true
when running configure and completely eliminate all of the data
provided by any associated .pc files, setting LIBS, CFLAGS, etc by
hand the 'old-fashioned' way.
The primary drawback of not using pkg-config is that the user
has to set LIBS, CFLAGS, etc. the old-fashioned way. In practice,
this is pretty trivial, and all pkg-config has done is move the
data from a single CONFIG_SITE file to separately maintained
.pc files for each package.
If you do use PKG_MODULE_CHECK, follow it up with a call to
AC_CHECK_LIB or AC_SEARCH_LIBS to validate the data in whatever
.pc file was located by PKG_CHECK_MODULES
I'm assuming that the place at which this occurs inside your configure.ac is inside a PKG_CHECK_MODULES call.
Looking at the libopensync sources, it seems that libopensync1 is the newer name, and libopensync is the old name. So, we'll use pkg-config macros to look for the newer name unless it doesn't exist.
Put this in your configure.ac:
# Check if libopensync1 is known to pkg-config, and if not, look for libopensync instead
PKG_CHECK_EXISTS([libopensync1], [OPENSYNC=libopensync1], [OPENSYNC=libopensync])
Then later in your PKG_CHECK_MODULES call, replace libopensync1 with $OPENSYNC.

Linking with a different .so file in linux

I'm trying to compile a piece of software which has the standard build process e.g.
configure
make
make install
The software requires a library e.g. libreq.so which is installed in /usr/local/lib. However, my problem is I'd like to build the software and link it with a different version of the same library (i have the source for the library as well) that I've installed in /home/user/mylibs.
My question is, how do I compile and link the software against the library in /home/user/mylibs rather than the one in /usr/local/lib
I tried setting "LD_LIBRARY_PATH" to include /home/user/mylibs but that didn't work.
Thanks!
When you have an autoconf configure script, use:
CPPFLAGS=-I/home/user/include LDFLAGS=-L/home/user/mylibs ./configure ...
This adds the nominated directory to the list of directories searched for headers (usually necessary when you're using a library), and adds the other nominated directory to the list searched for actual libraries.
I use this all the time - on my work machine, /usr/local is 'maintained' by MIS and contains obsolete code 99.9% of the time (and is NFS-mounted, read-only), so I struggle to avoid using it at all and maintain my own, more nearly current, versions of the software under /usr/gnu. It works for me.
Try using LD_PRELOAD set to your actual file.
LD_LIBRARY_PATH is for finding the dynamic link libraries at runtime. At compiling you should add -L parameters to gcc/g++ to specify in which directory the *.so files are. You also need to add the library name with -l<NAME> (where the library is libNAME.so).
Important! For linking you not only need the libNAME.so file but a libNAME.a is needed too.
When you run the application, don't forget to add the dir to the LD_LIBRARY_PATH.
When you added the /home/user/mylibs to the LD_LIBRARY_PATH did you add it to the front or end of the existing paths? The tokens are searched in-order so you will want yours to appear first in the list.
Also, many standard build environments that use configure will allow you to specify an exact library for each required piece. You will have to run ./configure --help but you should see something like --using-BLAH-lib=/path/to/your/library or similar.

Resources