I am trying to create a JNI wrapper for a C library that depends on the ICU libraries (libicuuc.so and libicui18n.so).
I tried building ICU4C in my NDK (both standard and CrystaX versions, on a Mac OS X machine) and kept running into linking issues like this:
/Users/kyip/KyVmShared/KyAndroid/myproject/obj/local/armeabi/objs/icuuc/udata.o: In function `openCommonData':
/Users/kyip/KyVmShared/KyAndroid/myproject/jni/icu4c/common/udata.c:836: undefined reference to `icudt42_dat'
/Users/kyip/KyVmShared/KyAndroid/myproject/obj/local/armeabi/objs/icuuc/ustr_wcs.o: In function `_strFromWCS':
/Users/kyip/KyVmShared/KyAndroid/myproject/jni/icu4c/common/ustr_wcs.c:365: undefined reference to `wcstombs'
/Users/kyip/KyVmShared/KyAndroid/myproject/jni/icu4c/common/ustr_wcs.c:415: undefined reference to `wcstombs'
/Users/kyip/KyVmShared/KyAndroid/myproject/jni/icu4c/common/ustr_wcs.c:314: undefined reference to `wcstombs'
/Users/kyip/KyVmShared/KyAndroid/myproject/obj/local/armeabi/objs/icuuc/ustr_wcs.o: In function `_strToWCS':
/Users/kyip/KyVmShared/KyAndroid/myproject/jni/icu4c/common/ustr_wcs.c:164: undefined reference to `mbstowcs'
collect2: ld returned 1 exit status
I also tried the suggestion given at unicode support in android ndk but no luck. I got stuck at:
arm-eabi-g++ -I/ky/crystax/android-ndk-r4-crystax/build/platforms/android-8/arch-arm/usr/include/ -O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=0 -DU_GNUC_UTF16_STRING=0 -fno-short-enums -nostdlib -fPIC -DU_COMMON_IMPLEMENTATION -D_REENTRANT -I../common -I../../icu/source/common -I../../icu/source/i18n "-DDEFAULT_ICU_PLUGINS=\"/usr/local/lib/icu\" " -DU_COMMON_IMPLEMENTATION -DHAVE_CONFIG_H -I/ky/crystax/android-ndk-r4-crystax/build/platforms/android-8/arch-arm/usr/include/ -O3 -fno-short-wchar -DU_USING_ICU_NAMESPACE=0 -DU_GNUC_UTF16_STRING=0 -fno-short-enums -nostdlib -fPIC -DU_COMMON_IMPLEMENTATION -std=c++0x -fvisibility=hidden -c -o errorcode.ao ../../icu/source/common/errorcode.cpp
In file included from ../../icu/source/common/unicode/ptypes.h:23,
from ../../icu/source/common/unicode/umachine.h:52,
from ../../icu/source/common/unicode/utypes.h:36,
from ../../icu/source/common/errorcode.cpp:17:
/ky/crystax/android-ndk-r4-crystax/build/platforms/android-8/arch-arm/usr/include/sys/types.h:122: error: 'uint64_t' does not name a type
make[1]: *** [errorcode.ao] Error 1
make: *** [all-recursive] Error 2
Any help would be appreciated.
It seems that two files are involved in this issue. icu/source/common/unicode/ptypes.h which calls sys/types.h includes
#if ! U_HAVE_UINT64_T
typedef unsigned long long uint64_t;
/* else we may not have a 64-bit type */
#endif
By including sys/types.h from Android, we involve (near line 122/124)
#ifdef __BSD_VISIBLE
typedef unsigned char u_char;
typedef unsigned short u_short;
typedef unsigned int u_int;
typedef unsigned long u_long;
typedef uint32_t u_int32_t;
typedef uint16_t u_int16_t;
typedef uint8_t u_int8_t;
typedef uint64_t u_int64_t;
#endif
It seems that uint64_t has not been declared when it is assigned to u_int64_t. Indeed, sys/types.h includes stdint.h which has the following:
#if !defined __STRICT_ANSI__ || __STDC_VERSION__ >= 199901L
# define __STDC_INT64__
#endif
typedef __int8_t int8_t;
typedef __uint8_t uint8_t;
typedef __int16_t int16_t;
typedef __uint16_t uint16_t;
typedef __int32_t int32_t;
typedef __uint32_t uint32_t;
#if defined(__STDC_INT64__)
typedef __int64_t int64_t;
typedef __uint64_t uint64_t;
#endif
Likely STRICT_ANSI is not defined. Seems like this is a bug in the Android code in sys/types.h. If STDC_INT64 is not defined, it will not define uint64_t so it can't define u_int64_t. Perhaps the real solution is to have sys/types.h modified so that it has
#ifdef __BSD_VISIBLE
typedef unsigned char u_char;
typedef unsigned short u_short;
typedef unsigned int u_int;
typedef unsigned long u_long;
typedef uint32_t u_int32_t;
typedef uint16_t u_int16_t;
typedef uint8_t u_int8_t;
$if defined(__STDC_INT64__)
typedef uint64_t u_int64_t;
#endif
#endif
If you fix this, the next error will be in cstring.h:109
icu/source/common/cstring.h:109: error: 'int64_t' has not been declared
If you instead #define STDC_INT64 in common/unicode/ptypes.h it will go substantially farther, but will end at
icu/source/common/ustrenum.cpp:118: error: must #include <typeinfo> before using typeid
with more info here: http://groups.google.com/group/android-ndk/browse_thread/thread/2ec9dc289d815ba3?pli=1 but no real solutions
I also had this issue:
undefined reference to `mbstowcs'
You should build and link with higher version of android api.
Note:
I tried to link it with libraries from android-ndk/platforms/android-4... I had thought that 4 is version of Android, but 4 is version of Android API. And Android API 4 corresponds to Android 1.6 witch is very very old there is really no mbstowcs function in libc
Here is how i solved the problem. It is dirty but it works. The lib got compiled:
1. file: /icu4c/common/cwchar.h
comment out the #if U_HAVE_WCHAR_H and the respective #endif so the <wchar.h> is always included.
replace the previous uprv_wcstombs definition with:
#define uprv_wcstombs(mbstr, wcstr, count) U_STANDARD_CPP_NAMESPACE wcs2mbs(mbstr, wcstr, count)
replace the previous uprv_mbstowcs definition with:
#define uprv_mbstowcs(wcstr, mbstr, count) U_STANDARD_CPP_NAMESPACE mbs2wcs(wcstr, mbstr, count)
2. file: /icu4c/common/ustr_wcs.cpp
somewhere at the top, under the already existing includes add the line:
#include "../wcsmbs.h"
3. create new file "icu4c/wcsmbs.h"
size_t mbs2wcs(wchar_t * __ restrict pwcs, const char * __ restrict s, size_t n)
{
mbstate_t mbs;
const char *sp;
memset(&mbs, 0, sizeof(mbs));
sp=s;
return (mbsrtowcs(pwcs,&sp,n,&mbs));
}
size_t wcs2mbs(char * __restrict s, const wchar_t * __restrict pwcs, size_t n)
{
mbstate_t mbs;
const wchar_t *pwcsp;
memset(&mbs,0,sizeof(mbs));
pwcsp = pwcs;
return (wcsrtombs(s,&pwcsp,n,&mbs));
}
Hope it helps.
There has been an effort to provide NDK wrappers for the ICU libraries that are part of the system: https://android-review.googlesource.com/c/153001/.
Related
This question already has answers here:
Format specifiers for uint8_t, uint16_t, ...?
(7 answers)
Closed 3 years ago.
My test code:
#include <cstdint>
#include <cstdio>
int main() {
const constexpr uint8_t x = 64;
printf("%u", x);
}
Here is how I compiled with GCC 8.2:
g++ -Wall test_format.cpp -o test_format -O3 -std=c++17 -Wformat-signedness
And here is GCC's output:
test_format.cpp: In function ‘int main()’:
test_format.cpp:6:9: warning: format ‘%u’ expects argument of type ‘unsigned int’, but argument 2 has type ‘int’ [-Wformat=]
printf("%u", x);
^~~~
Tho, if I try to print an uint32_t, it has no error/warning.
I wonder why GCC expects uint8_t to be signed int.
Thanks.
Default argument promotions are applied to operands of a variadic function. Under these, an expression of type unsigned char is promoted to int.
In C and C++ types narrower than int are always promoted to int. See Why must a short be converted to an int before arithmetic operations in C and C++?
And inside variadic functions default promotion also applies, which means you can't pass types narrower than int to vararg functions. So uint8_t must be printed with %d, not %u. But anyway you're printing it the wrong way. The correct way is to use PRIu8
printf("%" PRIu8 "\n", x);
Format specifiers for uint8_t, uint16_t, ...?
printing the uint8_t
Why is the format specifier for uint8_t and uint16_t the same (%u)?
How do I print uint32_t and uint16_t variables value?
To print a uint8_t variable with printf(), you should do something like the following:
#include <cinttypes>
#include <cstdio>
int print_u8(std::uint8_t x) {
return std::printf("%" PRIu8 "\n", x);
}
The <cinttypes> header includes printf and scanf format specifiers for all the <cstdint> types (and explicitly includes that header) that should be used for maximum portability.
I have a file p2.cpp and 2d.cpp which I'm trying to link with 2d.h.
I have included 2d.h in both .cpp files and I'm getting an error:
2d.obj : error LNK2005: "float (* v)[3]" (?v##3PAY02MA) already defined in p2.obj
1: fatal error LNK1169: one or more multiply defined symbols found.
What should I do?
I have a file p2.cpp and 2d.cpp which I'm trying to link with 2d.h. I
have included 2d.h in both .cpp files and I'm getting an error:
Each symbol may only be defined in a program once (refer One definition rule). I'm not sure what you're header file looks like, but typically this means something to the effect of defining something in you're header file that is included in more than one compilation unit. You could "extern" it in you're header, and ensure that it is defined in a separate compilation unite.
From the compiler error it looks like you've define an array of pointers to functions in your header file. Extern this and provide a single definition in a source file.
This code effectively causes the problem:
//--- Def.h
#ifndef DEF_H
#define DEF_H
float foo();
/*extern */float (*floatFunctionArray[3])();
#endif /* DEF_H */
//--- Def.cpp
#include "Def.h"
float foo()
{
return 0;
}
float (*floatFunctionArray[3])() =
{
foo, foo, foo
};
//--- main.cpp
#include "Def.h"
int
main(int argc, char** argv)
{
return 0;
}
Adding the commented out "extern" solves the issue.
I have a GNU/Linux application with uses a number of shared memory objects. It could, potentially, be run a number of times on the same system. To keep things tidy, I first create a directory in /dev/shm for each of the set of shared memory objects.
The problem is that on newer GNU/Linux distributions, I no longer seem to be able create these in a sub-directory of /dev/shm.
The following is a minimal C program with illustrates what I'm talking about:
/*****************************************************************************
* shm_minimal.c
*
* Test shm_open()
*
* Expect to create shared memory file in:
* /dev/shm/
* └── my_dir
* └── shm_name
*
* NOTE: Only visible on filesystem during execution. I try to be nice, and
* clean up after myself.
*
* Compile with:
* $ gcc -lrt shm_minimal.c -o shm_minimal
*
******************************************************************************/
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <sys/types.h>
int main(int argc, const char* argv[]) {
int shm_fd = -1;
char* shm_dir = "/dev/shm/my_dir";
char* shm_file = "/my_dir/shm_name"; /* does NOT work */
//char* shm_file = "/my_dir_shm_name"; /* works */
// Create directory in /dev/shm
mkdir(shm_dir, 0777);
// make shared memory segment
shm_fd = shm_open(shm_file, O_RDWR | O_CREAT, 0600);
if (-1 == shm_fd) {
switch (errno) {
case EINVAL:
/* Confirmed on:
* kernel v3.14, GNU libc v2.19 (ArchLinux)
* kernel v3.13, GNU libc v2.19 (Ubuntu 14.04 Beta 2)
*/
perror("FAIL - EINVAL");
return 1;
default:
printf("Some other problem not being tested\n");
return 2;
}
} else {
/* Confirmed on:
* kernel v3.8, GNU libc v2.17 (Mint 15)
* kernel v3.2, GNU libc v2.15 (Xubuntu 12.04 LTS)
* kernel v3.1, GNU libc v2.13 (Debian 6.0)
* kernel v2.6.32, GNU libc v2.12 (RHEL 6.4)
*/
printf("Success !!!\n");
}
// clean up
close(shm_fd);
shm_unlink(shm_file);
rmdir(shm_dir);
return 0;
}
/* vi: set ts=2 sw=2 ai expandtab:
*/
When I run this program on a fairly new distribution, the call to shm_open() returns -1, and errno is set to EINVAL. However, when I run on something a little older, it creates the shared memory object in /dev/shm/my_dir as expected.
For the larger application, the solution is simple. I can use a common prefix instead of a directory.
If you could help enlighten me to this apparent change in behavior it would be very helpful. I suspect someone else out there might be trying to do something similar.
So it turns out the issue stems from how GNU libc validates the shared memory name. Specifically, the shared memory object MUST now be at the root of the shmfs mount point.
This was changed in glibc git commit b20de2c3d9 as the result of bug BZ #16274.
Specifically, the change is the line:
if (name[0] == '\0' || namelen > NAME_MAX || strchr (name, '/') != NULL)
Which now disallows '/' from anywhere in the filename (not counting leading '/')
If you have a third party tool that was broken by this shm_open change, a brilliant coworker found a workaround : preload a library that overrides the shm_open call and swaps slashes for underscores. It does the same for shm_unlink as well, so the application can properly free shared memory when needed.
deslash_shm.cc :
#include <dlfcn.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <algorithm>
#include <string>
// function used in place of the standard shm_open() function
extern "C" int shm_open(const char *name, int oflag, mode_t mode)
{
// keep a function pointer to the real shm_open() function
static int (*real_open)(const char *, int, mode_t) = NULL;
// the first time in, ask the dynamic linker to find the real shm_open() function
if (!real_open) real_open = (int (*)(const char *, int, mode_t)) dlsym(RTLD_NEXT,"shm_open");
// take the name we were given and replace all slashes with underscores instead
std::string n = name;
std::replace(n.begin(), n.end(), '/', '_');
// call the real open function with the patched path name
return real_open(n.c_str(), oflag, mode);
}
// function used in place of the standard shm_unlink() function
extern "C" int shm_unlink(const char *name)
{
// keep a function pointer to the real shm_unlink() function
static int (*real_unlink)(const char *) = NULL;
// the first time in, ask the dynamic linker to find the real shm_unlink() function
if (!real_unlink) real_unlink = (int (*)(const char *)) dlsym(RTLD_NEXT, "shm_unlink");
// take the name we were given and replace all slashes with underscores instead
std::string n = name;
std::replace(n.begin(), n.end(), '/', '_');
// call the real unlink function with the patched path name
return real_unlink(n.c_str());
}
To compile this file:
c++ -fPIC -shared -o deslash_shm.so deslash_shm.cc -ldl
And preload it before starting a process that tries to use non-standard slash characters in shm_open:
in bash:
export LD_PRELOAD=/path/to/deslash_shm.so
in tcsh:
setenv LD_PRELOAD /path/to/deslash_shm.so
This might sound stupid but is there a way to activate support of the inner members of an SSE vector type ?
I know this works fine on MSVC, And I ve found some comments on forums and SO like this.
The question, is can I activate this on CLang at least without creating my own unions ?
Thank you
[edit, workaround]
Currently I decided to create a vec4 type to help me.
here is the code
#include <emmintrin.h>
#include <cstdint>
#ifdef _WIN32
typedef __m128 vec4;
typedef __m128i vec4i;
typedef __m128d vec4d;
#else
typedef union __declspec(align(16)) vec4{
float m128_f32[4];
uint64_t m128_u64[2];
int8_t m128_i8[16];
int16_t m128_i16[8];
int32_t m128_i32[4];
int64_t m128_i64[2];
uint8_t m128_u8[16];
uint16_t m128_u16[8];
uint32_t m128_u32[4];
} vec4;
typedef union __declspec(align(16)) vec4i{
uint64_t m128i_u64[2];
int8_t m128i_i8[16];
int16_t m128i_i16[8];
int32_t m128i_i32[4];
int64_t m128i_i64[2];
uint8_t m128i_u8[16];
uint16_t m128i_u16[8];
uint32_t m128i_u32[4];
} vec4i;
typedef union __declspec(align(16)) vec4d{
double m128d_f64[2];
} vec4d;
#endif
On recent clangs, this Just Works without you needing to do anything at all:
#include <immintrin.h>
float foo(__m128 x) {
return x[1];
}
AFAIK it Just Works in recent GCC builds as well.
However, I should note the following:
Consider carefully whether or not you really need to do element-wise access in your vector code. If you can keep your operations in-lane, they will almost certainly be significantly more efficient.
If you really do need to do a significant number of lanewise or horizontal operations, and you don’t need portability, consider using Clang extended vectors (or “OpenCL vectors") instead of the basic SSE intrinsic types. You can pass them to intrinsics just like __m128 and friends, but they also have much nicer syntax for vector-scalar operations, lane wise operations, vector literals, etc.
I am working through a sample program that uses both C++ source code as well as CUDA. This is the essential content from my four source files.
matrixmul.cu (main CUDA source code):
#include <stdlib.h>
#include <cutil.h>
#include "assist.h"
#include "matrixmul.h"
int main (int argc, char ** argv)
{
...
computeGold(reference, hostM, hostN, Mh, Mw, Nw); //reference to .cpp file
...
}
matrixmul_gold.cpp (C++ source code, single function, no main method):
void computeGold(float * P, const float * M, const float * N, int Mh, int Mw, int Nw)
{
...
}
matrixmul.h (header for matrixmul_gold.cpp file)
#ifndef matrixmul_h
#define matrixmul_h
extern "C"
void computeGold(float * P, const float * M, const float * N, int Mh, int Mw, int Nw);
#endif
assist.h (helper functions)
I am trying to compile and link these files so that they, well, work. So far I can get matrixmul_gold.cpp compiled using:
g++ -c matrixmul_gold.cpp
And I can compile the CUDA source code with out errors using:
nvcc -I/home/sbu/NVIDIA_GPU_Computing_SDK/C/common/inc -L/home/sbu/NVIDIA_GPU_Computing_SDK/C/lib matrixmul.cu -c -lcutil_x86_64
But I just end up with two .O files. I've tried a lot of different ways to link the two .O files but so far it's a no-go. What's the proper approach?
UPDATE: As requested, here is the output of:
nm matrixmul_gold.o matrixmul.o | grep computeGold
nm: 'matrixmul.o': No such file
0000000000000000 T _Z11computeGoldPfPKfS1_iii
I think the 'matrixmul.o' missing error is because I am not actually getting a successful compile when running the suggested compile command:
nvcc -I/home/sbu/NVIDIA_GPU_Computing_SDK/C/common/inc -L/home/sbu/NVIDIA_GPU_Computing_SDK/C/lib -o matrixmul matrixmul.cu matrixmul_gold.o -lcutil_x86_64
UPDATE 2: I was missing an extern "C" from the beginning of matrixmul_gold.cpp. I added that and the suggested compilation command works great. Thank you!
Conventionally you would use whichever compiler you are using to compile the code containing the main subroutine to link the application. In this case you have the main in the .cu, so use nvcc to do the linking. Something like this:
$ g++ -c matrixmul_gold.cpp
$ nvcc -I/home/sbu/NVIDIA_GPU_Computing_SDK/C/common/inc \
-L/home/sbu/NVIDIA_GPU_Computing_SDK/C/lib \
-o matrixmul matrixmul.cu matrixmul_gold.o -lcutil_x86_64
This will link an executable binary called matrimul from matrixmul.cu, matrixmul_gold.o and the cutil library (implicitly nvcc will link the CUDA runtime library and CUDA driver library as well).