Is it possible to identify a hash type? - security

I know you can compare the length but many hash types have the same lengths.
Is there a way to identify a hash's type and whether it has been salted?
For example:
hash=2bf231b0e98be99a969bd6724f42a691
hash=4ac5a4ff764807d6ef464e27e4d1bee3
hash=4d177cec31d658ed22cc229e45d7265e

Yes, it is possible to a degree of some certainty to identify the type of hash algorithm that was used.
One tool that I use a lot to do this is hash-identifier.
For example, I create a hash of the Hash_ID.py file:
$ openssl sha -sha256 Hash_ID.py
SHA256(Hash_ID.py)= 5382a8826c972f8fa8687efe1f68e475c02af4bf542b0d7e68b9deffd388db96
When running Hash_ID.py it will ask for the Hash to be entered:
$ python Hash_ID.py
#########################################################################
# __ __ __ ______ _____ #
# /\ \/\ \ /\ \ /\__ _\ /\ _ `\ #
# \ \ \_\ \ __ ____ \ \ \___ \/_/\ \/ \ \ \/\ \ #
# \ \ _ \ /'__`\ / ,__\ \ \ _ `\ \ \ \ \ \ \ \ \ #
# \ \ \ \ \/\ \_\ \_/\__, `\ \ \ \ \ \ \_\ \__ \ \ \_\ \ #
# \ \_\ \_\ \___ \_\/\____/ \ \_\ \_\ /\_____\ \ \____/ #
# \/_/\/_/\/__/\/_/\/___/ \/_/\/_/ \/_____/ \/___/ v1.1 #
# By Zion3R #
# www.Blackploit.com #
# Root#Blackploit.com #
#########################################################################
-------------------------------------------------------------------------
HASH: 5382a8826c972f8fa8687efe1f68e475c02af4bf542b0d7e68b9deffd388db96
Possible Hashs:
[+] SHA-256
[+] Haval-256
Least Possible Hashs:
[+] GOST R 34.11-94
[+] RipeMD-256
[+] SNEFRU-256
[+] SHA-256(HMAC)
[+] Haval-256(HMAC)
[+] RipeMD-256(HMAC)
[+] SNEFRU-256(HMAC)
[+] SHA-256(md5($pass))
[+] SHA-256(sha1($pass))
The way Hash ID works is by checking the hash given against criteria for all the hash types it supports and will give a list of possible hash types.

That particular example is a 32 character alphanumeric representation, which is almost certainly MD5.
SHA-1 usually comes as a 40 character alphanumeric string (as does SHA-0)
MD5 and SHA-1 account for the vast majority of hashes you'll find in the wild.

No; you pretty much can only identify it by the length.
-- Edit:
Obviously, however, if you have access to the program generating the hashes, and you can provide input, then you can compare with some result you also calculate (assuming you know the salt.
If you're really stuck, you can also infer it from the language that's being used (i.e. if it's PHP, it's most likely MD5), and so on.
But from a technical point of view, there is no way to identify a hash; as it would be counter-productive to the goal of security :) (it would take up useless bits in the hash itself to do this identification).

Post from the future:
2bf231b0e98be99a969bd6724f42a691 MD5 : gombaliste0
4ac5a4ff764807d6ef464e27e4d1bee3 MD5 : gombaliste2
4d177cec31d658ed22cc229e45d7265e MD5 : gombaliste129

Related

How to cross-compile a rust application for ARM which uses Rocket web-server and requires nightly toolchain using Yocto?

I want to compile myRustApp which uses Rocket webserver in my meta layer. The issu I'm having is that openembedded does not support nightly release of Rust, which is required to run Rocket web-server.
The alternative was to use meta-rust, but it got me no where, because if I understand correctly the meta layer only supports native builds.
So I ended up using meta-rust-bin layer with pre-built nightly toolchain.
I was able to build nightly by executing ./build-new-version.sh nightly under meta-rust-bin.
After all this my recipe build by bitbake returns an Error:
WARNING: /home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-cross_1.59.0.bb: Exception during build_dependencies for TARGET_LLVM_FEATURES
WARNING: /home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-cross_1.59.0.bb: Error during finalise of /home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-cross_1.59.0.bb
ERROR: ExpansionError during parsing /home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-cross_1.59.0.bb
Traceback (most recent call last):
File "Var <TARGET_LLVM_FEATURES>", line 1, in <module>
File "/home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-common.inc", line 117, in llvm_features(d=<bb.data_smart.DataSmart object at 0x7fe03b341f60>):
return ','.join(llvm_features_from_tune(d) +
> llvm_features_from_cc_arch(d) +
llvm_features_from_target_fpu(d))
File "/home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-common.inc", line 36, in llvm_features_from_tune(d=<bb.data_smart.DataSmart object at 0x7fe03b341f60>):
> if target_is_armv7(d):
f.append('+v7')
bb.data_smart.ExpansionError: Failure expanding variable TARGET_LLVM_FEATURES, expression was ${#llvm_features(d)} which triggered exception NameError: name 'target_is_armv7' is not defined
The variable dependency chain for the failure is: TARGET_LLVM_FEATURES
ERROR: Parsing halted due to errors, see error messages above
WARNING: /home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-cross-canadian_1.59.0.bb: Exception during build_dependencies for TARGET_LLVM_FEATURES
WARNING: /home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-cross-canadian_1.59.0.bb: Error during finalise of /home/jonas/project/sources/openembedded-core/meta/recipes-devtools/rust/rust-cross-canadian_1.59.0.bb
My question is:
What causes this error I'm getting? How can I fix it?
Did someone already tried to cross-compile Rust nightly apps? Are there any examples anywhere?
My Cargo.toml:
[package]
name = "MyRustApp"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
rocket = "0.4.10"
rocket_cors = "0.5.1"
serde = "1.0"
serde_json = "1.0"
serde_derive = "1.0"
[dependencies.rocket_contrib]
version = "0.4.10"
default-features = false
features = ["json"]
[profile.release]
strip = true
opt-level = "z"
lto = true
My recipe generated by cargo bitbake:
# Auto-Generated by cargo-bitbake 0.3.16
#
inherit cargo
# If this is git based prefer versioned ones if they exist
# DEFAULT_PREFERENCE = "-1"
# how to get strain-webserver could be as easy as but default to a git checkout:
# SRC_URI += "crate://crates.io/MyRustApp/0.1.0"
SRC_URI += "git://git#bitbucket.org/work/MyRustApp.git;protocol=ssh;nobranch=1;branch=main"
SRCREV = "xxx"
S = "${WORKDIR}/git"
CARGO_SRC_DIR = ""
PV:append = ".AUTOINC+d2562d3c92"
# please note if you have entries that do not begin with crate://
# you must change them to how that package can be fetched
SRC_URI += " \
crate://crates.io/aead/0.3.2 \
crate://crates.io/aes-gcm/0.8.0 \
crate://crates.io/aes-soft/0.6.4 \
crate://crates.io/aes/0.6.0 \
crate://crates.io/aesni/0.10.0 \
crate://crates.io/aho-corasick/0.7.20 \
crate://crates.io/atty/0.2.14 \
crate://crates.io/autocfg/1.1.0 \
crate://crates.io/base64/0.13.1 \
crate://crates.io/base64/0.9.3 \
crate://crates.io/bitflags/1.3.2 \
crate://crates.io/block-buffer/0.9.0 \
crate://crates.io/byteorder/1.4.3 \
crate://crates.io/cfg-if/0.1.10 \
crate://crates.io/cfg-if/1.0.0 \
crate://crates.io/cipher/0.2.5 \
crate://crates.io/cookie/0.11.5 \
crate://crates.io/cpufeatures/0.2.5 \
crate://crates.io/cpuid-bool/0.2.0 \
crate://crates.io/crypto-mac/0.10.1 \
crate://crates.io/ctr/0.6.0 \
crate://crates.io/devise/0.2.1 \
crate://crates.io/devise_codegen/0.2.1 \
crate://crates.io/devise_core/0.2.1 \
crate://crates.io/digest/0.9.0 \
crate://crates.io/filetime/0.2.18 \
crate://crates.io/form_urlencoded/1.1.0 \
crate://crates.io/fsevent-sys/2.0.1 \
crate://crates.io/fsevent/0.4.0 \
crate://crates.io/fuchsia-zircon-sys/0.3.3 \
crate://crates.io/fuchsia-zircon/0.3.3 \
crate://crates.io/generic-array/0.14.6 \
crate://crates.io/getrandom/0.2.8 \
crate://crates.io/ghash/0.3.1 \
crate://crates.io/glob/0.3.0 \
crate://crates.io/hashbrown/0.12.3 \
crate://crates.io/hermit-abi/0.1.19 \
crate://crates.io/hkdf/0.10.0 \
crate://crates.io/hmac/0.10.1 \
crate://crates.io/httparse/1.8.0 \
crate://crates.io/hyper/0.10.16 \
crate://crates.io/idna/0.1.5 \
crate://crates.io/idna/0.3.0 \
crate://crates.io/indexmap/1.9.2 \
crate://crates.io/inotify-sys/0.1.5 \
crate://crates.io/inotify/0.7.1 \
crate://crates.io/iovec/0.1.4 \
crate://crates.io/itoa/1.0.4 \
crate://crates.io/kernel32-sys/0.2.2 \
crate://crates.io/language-tags/0.2.2 \
crate://crates.io/lazycell/1.3.0 \
crate://crates.io/libc/0.2.137 \
crate://crates.io/log/0.3.9 \
crate://crates.io/log/0.4.17 \
crate://crates.io/matches/0.1.9 \
crate://crates.io/memchr/2.5.0 \
crate://crates.io/mime/0.2.6 \
crate://crates.io/mio-extras/2.0.6 \
crate://crates.io/mio/0.6.23 \
crate://crates.io/miow/0.2.2 \
crate://crates.io/net2/0.2.38 \
crate://crates.io/notify/4.0.17 \
crate://crates.io/num_cpus/1.14.0 \
crate://crates.io/opaque-debug/0.3.0 \
crate://crates.io/pear/0.1.5 \
crate://crates.io/pear_codegen/0.1.5 \
crate://crates.io/percent-encoding/1.0.1 \
crate://crates.io/percent-encoding/2.2.0 \
crate://crates.io/polyval/0.4.5 \
crate://crates.io/ppv-lite86/0.2.17 \
crate://crates.io/proc-macro2/0.4.30 \
crate://crates.io/proc-macro2/1.0.47 \
crate://crates.io/quote/0.6.13 \
crate://crates.io/quote/1.0.21 \
crate://crates.io/rand/0.8.5 \
crate://crates.io/rand_chacha/0.3.1 \
crate://crates.io/rand_core/0.6.4 \
crate://crates.io/redox_syscall/0.2.16 \
crate://crates.io/regex-syntax/0.6.28 \
crate://crates.io/regex/1.7.0 \
crate://crates.io/rocket/0.4.11 \
crate://crates.io/rocket_codegen/0.4.11 \
crate://crates.io/rocket_contrib/0.4.11 \
crate://crates.io/rocket_cors/0.5.2 \
crate://crates.io/rocket_http/0.4.11 \
crate://crates.io/ryu/1.0.11 \
crate://crates.io/safemem/0.3.3 \
crate://crates.io/same-file/1.0.6 \
crate://crates.io/serde/1.0.147 \
crate://crates.io/serde_derive/1.0.147 \
crate://crates.io/serde_json/1.0.89 \
crate://crates.io/sha2/0.9.9 \
crate://crates.io/slab/0.4.7 \
crate://crates.io/smallvec/1.10.0 \
crate://crates.io/state/0.4.2 \
crate://crates.io/subtle/2.4.1 \
crate://crates.io/syn/0.15.44 \
crate://crates.io/syn/1.0.103 \
crate://crates.io/time/0.1.44 \
crate://crates.io/tinyvec/1.6.0 \
crate://crates.io/tinyvec_macros/0.1.0 \
crate://crates.io/toml/0.4.10 \
crate://crates.io/traitobject/0.1.0 \
crate://crates.io/typeable/0.1.2 \
crate://crates.io/typenum/1.15.0 \
crate://crates.io/unicase/1.4.2 \
crate://crates.io/unicase/2.6.0 \
crate://crates.io/unicase_serde/0.1.0 \
crate://crates.io/unicode-bidi/0.3.8 \
crate://crates.io/unicode-ident/1.0.5 \
crate://crates.io/unicode-normalization/0.1.22 \
crate://crates.io/unicode-xid/0.1.0 \
crate://crates.io/universal-hash/0.4.1 \
crate://crates.io/url/1.7.2 \
crate://crates.io/url/2.3.1 \
crate://crates.io/version_check/0.1.5 \
crate://crates.io/version_check/0.9.4 \
crate://crates.io/walkdir/2.3.2 \
crate://crates.io/wasi/0.10.0+wasi-snapshot-preview1 \
crate://crates.io/wasi/0.11.0+wasi-snapshot-preview1 \
crate://crates.io/winapi-build/0.1.1 \
crate://crates.io/winapi-i686-pc-windows-gnu/0.4.0 \
crate://crates.io/winapi-util/0.1.5 \
crate://crates.io/winapi-x86_64-pc-windows-gnu/0.4.0 \
crate://crates.io/winapi/0.2.8 \
crate://crates.io/winapi/0.3.9 \
crate://crates.io/windows-sys/0.42.0 \
crate://crates.io/windows_aarch64_gnullvm/0.42.0 \
crate://crates.io/windows_aarch64_msvc/0.42.0 \
crate://crates.io/windows_i686_gnu/0.42.0 \
crate://crates.io/windows_i686_msvc/0.42.0 \
crate://crates.io/windows_x86_64_gnu/0.42.0 \
crate://crates.io/windows_x86_64_gnullvm/0.42.0 \
crate://crates.io/windows_x86_64_msvc/0.42.0 \
crate://crates.io/ws2_32-sys/0.2.1 \
crate://crates.io/yansi/0.5.1 \
"
# FIXME: update generateme with the real MD5 of the license file
LIC_FILES_CHKSUM = " \
"
SUMMARY = "Webserver with Rust"t"
LICENSE = "CLOSED"
# includes this file if it exists but does not fail
# this is useful for anything you may want to override from
# what cargo-bitbake generates.
include MyRustApp-${PV}.inc
include MyRustApp.inc

why run "python run_squad.py" doesn't work?

I want fine tune on squad with huggingface run_squad.py, but meet the following question:
1, when I use "--do_train" without "True" as following code, after 20 minutes runing,there is no models in output_dir:
!python run_squad.py \
--model_type bert \
--model_name_or_path bert-base-uncased \
--output_dir models/bert/ \
--data_dir data/squad \
--overwrite_output_dir \
--overwrite_cache \
--do_train \
--train_file train-v2.0.json \
--version_2_with_negative \
--do_lower_case \
--do_eval \
--predict_file dev-v2.0.json \
--per_gpu_train_batch_size 2 \
--learning_rate 3e-5 \
--num_train_epochs 2.0 \
--max_seq_length 384 \
--doc_stride 128 \
--threads 10 \
--save_steps 5000
2, when I use "--do_train=True" as following code, the error message is "run_squad.py: error: argument --do_train: ignored explicit argument 'True'":
!python run_squad.py \
--model_type bert \
--model_name_or_path bert-base-uncased \
--output_dir models/bert/ \
--data_dir data/squad \
--overwrite_output_dir \
--overwrite_cache \
--do_train=True \
--train_file train-v2.0.json \
--version_2_with_negative \
--do_lower_case \
--do_eval \
--predict_file dev-v2.0.json \
--per_gpu_train_batch_size 2 \
--learning_rate 3e-5 \
--num_train_epochs 2.0 \
--max_seq_length 384 \
--doc_stride 128 \
--threads 10 \
--save_steps 5000
3, when I use "--do_train True" as following code, the error message is "run_squad.py: error: unrecognized arguments: True":
!python run_squad.py \
--model_type bert \
--model_name_or_path bert-base-uncased \
--output_dir models/bert/ \
--data_dir data/squad \
--overwrite_output_dir \
--overwrite_cache \
--do_train True \
--train_file train-v2.0.json \
--version_2_with_negative \
--do_lower_case \
--do_eval \
--predict_file dev-v2.0.json \
--per_gpu_train_batch_size 2 \
--learning_rate 3e-5 \
--num_train_epochs 2.0 \
--max_seq_length 384 \
--doc_stride 128 \
--threads 10 \
--save_steps 5000
I run code in colab with GPU: Tesla P100-PCIE-16GB
Judging by the running time, I think the code didn't through training process, but I don't know how to set parameters in order to let training go.what should I do?

What language is this old program written in?

"Hi, could you rewrite something for me", my boss said, "some legacy code". Yeah, legacy code written somewhere in early Mesozoic.
I have 30 hours left, and I still don't know, what kind of syntax is this! VHLD? VBA? Program is dedicated do to something with audio files, and dedicated to run under DOS.
Could you give me a hint what is this? How to compile it?
Code snippet:
enter \ Load - main screen
empty forth definitions decimal
application warning on
: TITLE ." KCS version 0.8 28-Jan-06" cr ;
cr .( Compiling: ) title 2 load
cr .( Save to disk? ) y/n
[if]
\ pad reserve # + 256 + limit s0 # - + set-limit
turnkey program KCS
[then]
\ Load - defaults
variable RESERVE 0 reserve ! \ reserved memory tally
defer ?BREAK ' noop is ?break \ break check off
defer SET-IO ' bios-io is set-io \ default console mode
defer ERRFIX ' noop is errfix \ reset on-error handler
blk # 1+ #screens 1- thru \ load electives & application
' (?break) is ?break \ enable user break
\ ' dos-io is set-io \ enable console redirection
\ ' deloutfile +is errfix \ delete outfile on error
\ wrtchk off \ disable overwrite check
\ Load - electives
1 fload DOSLIB \ load DOSLIB library
_Errors \ error handler
_Inout1 \ number output
_Inout2 \ string & number input
_String1 \ basic strings
\ _String2 \ extra strings
_Parsing \ command-line parsing
_Fileprims \ file primitives
_Files \ default files
_Bufinfile \ buffered input file
_Bufoutfile \ buffered output file
\ DECODE
\ Convert wave file to program
: DECODE ( -- )
0. decodecount 2! 0. paritycount 2! 0 errors !
skipheader
begin
['] decodebyte 1 ?catch 0=
while ( not EOF )
conout # if emit else writechar then
1 decodecount m+!
repeat
.decoded ;
\ SETMODE
\ Select Kansas City Standard or Processor Tech. CUTS mode
: SETMODE ( -- )
mode # if ( CUTS )
8 to databits 2 sbits ! 4 speed ! parity off pace off
nullcnt off
['] 0bit-sqr-cuts is 0bit-sqr ['] 1bit-sqr-cuts is 1bit-sqr
['] 0bit-sin-cuts is 0bit-sin ['] 1bit-sin-cuts is 1bit-sin
['] seekstart-cuts is seekstart ['] getbit-cuts is getbit
else ( KCS )
['] 0bit-sqr-kcs is 0bit-sqr ['] 1bit-sqr-kcs is 1bit-sqr
['] 0bit-sin-kcs is 0bit-sin ['] 1bit-sin-kcs is 1bit-sin
['] seekstart-kcs is seekstart ['] getbit-kcs is getbit
then ;
\ (RUN)
\ Run application
: (RUN) ( -- )
setmode
r/o openinfile
decoding # if
conout # 0= if r/w makeoutfile then
cr decode
else
r/w makeoutfile
cr encode
then
closefiles
;
\ DEFAULTS
\ Set application defaults
: DEFAULTS ( -- )
mode off decoding on strict off ignore off conout off
1 speed ! 2 sbits ! parity off 5 leadtime ! 10 nullchar !
pace off nullcnt off wave off tone off inverted off ;
defaults
\ RUN PROGRAM
\ Run application with error handling
: RUN ( -- )
['] (run) catch ?dup if >r errfix r> throw then ;
\ Main
: PROGRAM ( -- )
set-io \ set console mode
defaults \ set defaults
cr title \ show application name
parsecmd \ get options/filenames
run \ run application
cr ." done" \ show success
;
It's written in Forth, probably the DX-Forth dialect. The program decodes and encodes WAVE files that contain data in the Kansas City standard format. This format was used to record data on cassette tapes on early S-100 CP/M machines. Searching the web reveals that there was a program written in DX-Forth that could decode and encode WAVE files in this format, so I'm guessing it's the program you've be tasked with rewriting.
Rather than rewriting this code however, a simpler thing to do would be to use existing free software that already does the job. For example there's a program called py-kcs written in Python that should be a functional replacement and one called hx-kcs written in Haxe that can do decoding.

Linux try_cmpxchg mysterious inline assembly

I need some help understanding Linux's `try_cmpxchg semantics and implementation. In the kernel source, it is implemented as:
#define __raw_try_cmpxchg(_ptr, _pold, _new, size, lock) \
({ \
bool success; \
__typeof__(_ptr) _old = (_pold); \
__typeof__(*(_ptr)) __old = *_old; \
__typeof__(*(_ptr)) __new = (_new); \
switch (size) { \
case __X86_CASE_B: \
{ \
volatile u8 *__ptr = (volatile u8 *)(_ptr); \
asm volatile(lock "cmpxchgb %[new], %[ptr]" \
CC_SET(z) \
: CC_OUT(z) (success), \
[ptr] "+m" (*__ptr), \
[old] "+a" (__old) \
: [new] "q" (__new) \
: "memory"); \
break; \
} \
case __X86_CASE_W: \
{ \
volatile u16 *__ptr = (volatile u16 *)(_ptr); \
asm volatile(lock "cmpxchgw %[new], %[ptr]" \
CC_SET(z) \
: CC_OUT(z) (success), \
[ptr] "+m" (*__ptr), \
[old] "+a" (__old) \
: [new] "r" (__new) \
: "memory"); \
break; \
} \
case __X86_CASE_L: \
{ \
volatile u32 *__ptr = (volatile u32 *)(_ptr); \
asm volatile(lock "cmpxchgl %[new], %[ptr]" \
CC_SET(z) \
: CC_OUT(z) (success), \
[ptr] "+m" (*__ptr), \
[old] "+a" (__old) \
: [new] "r" (__new) \
: "memory"); \
break; \
} \
case __X86_CASE_Q: \
{ \
volatile u64 *__ptr = (volatile u64 *)(_ptr); \
asm volatile(lock "cmpxchgq %[new], %[ptr]" \
CC_SET(z) \
: CC_OUT(z) (success), \
[ptr] "+m" (*__ptr), \
[old] "+a" (__old) \
: [new] "r" (__new) \
: "memory"); \
break; \
} \
default: \
__cmpxchg_wrong_size(); \
} \
if (unlikely(!success)) \
*_old = __old; \
likely(success); \
})
#define __try_cmpxchg(ptr, pold, new, size) \
__raw_try_cmpxchg((ptr), (pold), (new), (size), LOCK_PREFIX)
#define try_cmpxchg(ptr, pold, new) \
__try_cmpxchg((ptr), (pold), (new), sizeof(*(ptr)))
I am curious what those CC_SET and CC_OUT means. They are defined as:
/*
* Macros to generate condition code outputs from inline assembly,
* The output operand must be type "bool".
*/
#ifdef __GCC_ASM_FLAG_OUTPUTS__
# define CC_SET(c) "\n\t/* output condition code " #c "*/\n"
# define CC_OUT(c) "=#cc" #c
#else
# define CC_SET(c) "\n\tset" #c " %[_cc_" #c "]\n"
# define CC_OUT(c) [_cc_ ## c] "=qm"
#endif
Also, it would be great if you can explain the exact semantics of try_cmpxchg (not quite understand how can a atomic cmpxchg fail...)
Newer versions of gcc (I believe from version 6) support specific flag outputs. The macros are there to use this support if available, else fall back to the old way by doing a setCC instruction and a temporary output.
As to how cmpxchg can "fail": it does a compare so it fails if that compare fails, in which case the destination is unchanged and the current value is fetched from memory. Consult an instruction set reference for the details.

How to build protocol buffer by Android NDK [closed]

It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I want to built a native version of Google's protocol buffers library.
How would I do that?
I Use this Android.mk and build SUCCESSFUL
# Copyright (C) 2009 The Android Open Source Project
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
CC_LITE_SRC_FILES := \
src/google/protobuf/stubs/common.cc \
src/google/protobuf/stubs/once.cc \
src/google/protobuf/stubs/hash.cc \
src/google/protobuf/extension_set.cc \
src/google/protobuf/generated_message_util.cc \
src/google/protobuf/message_lite.cc \
src/google/protobuf/repeated_field.cc \
src/google/protobuf/wire_format_lite.cc \
src/google/protobuf/io/coded_stream.cc \
src/google/protobuf/io/zero_copy_stream.cc \
src/google/protobuf/io/zero_copy_stream_impl_lite.cc
COMPILER_SRC_FILES := \
src/google/protobuf/descriptor.cc \
src/google/protobuf/descriptor.pb.cc \
src/google/protobuf/descriptor_database.cc \
src/google/protobuf/dynamic_message.cc \
src/google/protobuf/extension_set.cc \
src/google/protobuf/extension_set_heavy.cc \
src/google/protobuf/generated_message_reflection.cc \
src/google/protobuf/generated_message_util.cc \
src/google/protobuf/message.cc \
src/google/protobuf/message_lite.cc \
src/google/protobuf/reflection_ops.cc \
src/google/protobuf/repeated_field.cc \
src/google/protobuf/service.cc \
src/google/protobuf/text_format.cc \
src/google/protobuf/unknown_field_set.cc \
src/google/protobuf/wire_format.cc \
src/google/protobuf/wire_format_lite.cc \
src/google/protobuf/compiler/code_generator.cc \
src/google/protobuf/compiler/command_line_interface.cc \
src/google/protobuf/compiler/importer.cc \
src/google/protobuf/compiler/main.cc \
src/google/protobuf/compiler/parser.cc \
src/google/protobuf/compiler/plugin.cc \
src/google/protobuf/compiler/plugin.pb.cc \
src/google/protobuf/compiler/subprocess.cc \
src/google/protobuf/compiler/zip_writer.cc \
src/google/protobuf/compiler/cpp/cpp_enum.cc \
src/google/protobuf/compiler/cpp/cpp_enum_field.cc \
src/google/protobuf/compiler/cpp/cpp_extension.cc \
src/google/protobuf/compiler/cpp/cpp_field.cc \
src/google/protobuf/compiler/cpp/cpp_file.cc \
src/google/protobuf/compiler/cpp/cpp_generator.cc \
src/google/protobuf/compiler/cpp/cpp_helpers.cc \
src/google/protobuf/compiler/cpp/cpp_message.cc \
src/google/protobuf/compiler/cpp/cpp_message_field.cc \
src/google/protobuf/compiler/cpp/cpp_primitive_field.cc \
src/google/protobuf/compiler/cpp/cpp_service.cc \
src/google/protobuf/compiler/cpp/cpp_string_field.cc \
src/google/protobuf/compiler/java/java_enum.cc \
src/google/protobuf/compiler/java/java_enum_field.cc \
src/google/protobuf/compiler/java/java_extension.cc \
src/google/protobuf/compiler/java/java_field.cc \
src/google/protobuf/compiler/java/java_file.cc \
src/google/protobuf/compiler/java/java_generator.cc \
src/google/protobuf/compiler/java/java_helpers.cc \
src/google/protobuf/compiler/java/java_message.cc \
src/google/protobuf/compiler/java/java_message_field.cc \
src/google/protobuf/compiler/java/java_primitive_field.cc \
src/google/protobuf/compiler/java/java_service.cc \
src/google/protobuf/compiler/javamicro/javamicro_enum.cc \
src/google/protobuf/compiler/javamicro/javamicro_enum_field.cc \
src/google/protobuf/compiler/javamicro/javamicro_field.cc \
src/google/protobuf/compiler/javamicro/javamicro_file.cc \
src/google/protobuf/compiler/javamicro/javamicro_generator.cc \
src/google/protobuf/compiler/javamicro/javamicro_helpers.cc \
src/google/protobuf/compiler/javamicro/javamicro_message.cc \
src/google/protobuf/compiler/javamicro/javamicro_message_field.cc \
src/google/protobuf/compiler/javamicro/javamicro_primitive_field.cc \
src/google/protobuf/compiler/python/python_generator.cc \
src/google/protobuf/io/coded_stream.cc \
src/google/protobuf/io/gzip_stream.cc \
src/google/protobuf/io/printer.cc \
src/google/protobuf/io/tokenizer.cc \
src/google/protobuf/io/zero_copy_stream.cc \
src/google/protobuf/io/zero_copy_stream_impl.cc \
src/google/protobuf/io/zero_copy_stream_impl_lite.cc \
src/google/protobuf/stubs/common.cc \
src/google/protobuf/stubs/hash.cc \
src/google/protobuf/stubs/once.cc \
src/google/protobuf/stubs/structurally_valid.cc \
src/google/protobuf/stubs/strutil.cc \
src/google/protobuf/stubs/substitute.cc
# C++ full library
# =======================================================
#include $(CLEAR_VARS)
LOCAL_MODULE := libprotobuf
LOCAL_MODULE_TAGS := optional
LOCAL_CPP_EXTENSION := .cc
LOCAL_SRC_FILES := \
$(CC_LITE_SRC_FILES) \
src/google/protobuf/stubs/strutil.cc \
src/google/protobuf/stubs/substitute.cc \
src/google/protobuf/stubs/structurally_valid.cc \
src/google/protobuf/descriptor.cc \
src/google/protobuf/descriptor.pb.cc \
src/google/protobuf/descriptor_database.cc \
src/google/protobuf/dynamic_message.cc \
src/google/protobuf/extension_set_heavy.cc \
src/google/protobuf/generated_message_reflection.cc \
src/google/protobuf/message.cc \
src/google/protobuf/reflection_ops.cc \
src/google/protobuf/service.cc \
src/google/protobuf/text_format.cc \
src/google/protobuf/unknown_field_set.cc \
src/google/protobuf/wire_format.cc \
src/google/protobuf/io/gzip_stream.cc \
src/google/protobuf/io/printer.cc \
src/google/protobuf/io/tokenizer.cc \
src/google/protobuf/io/zero_copy_stream_impl.cc \
src/google/protobuf/compiler/importer.cc \
src/google/protobuf/compiler/parser.cc
LOCAL_C_INCLUDES := \
$(LOCAL_PATH)/src
LOCAL_C_INCLUDES := \
$(LOCAL_PATH)/android \
bionic \
$(LOCAL_PATH)/src \
$(JNI_H_INCLUDE)
LOCAL_SHARED_LIBRARIES := \
libz libcutils libutils
LOCAL_LDLIBS := -lz
# stlport conflicts with the host stl library
ifneq ($(TARGET_SIMULATOR),true)
LOCAL_C_INCLUDES += external/stlport/stlport
LOCAL_SHARED_LIBRARIES += libstlport
endif
# Define the header files to be copied
#LOCAL_COPY_HEADERS := \
# src/google/protobuf/stubs/once.h \
# src/google/protobuf/stubs/common.h \
# src/google/protobuf/io/coded_stream.h \
# src/google/protobuf/generated_message_util.h \
# src/google/protobuf/repeated_field.h \
# src/google/protobuf/extension_set.h \
# src/google/protobuf/wire_format_lite_inl.h
#
#LOCAL_COPY_HEADERS_TO := $(LOCAL_MODULE)
LOCAL_CFLAGS := -DGOOGLE_PROTOBUF_NO_RTTI
include $(BUILD_SHARED_LIBRARY)
Android.mk mentioned above works fine, thanks a lot.
I would like to describe all the steps for building protobuf for android.
Configure sources for android.
Use the following script (got it here: http://habrahabr.ru/post/119693/):
PREBUILT=/Users/user/android-ndk-r7b/toolchains/arm-linux-androideabi-4.4.3
PLATFORM=/Users/user/android-ndk-r7b/platforms/android-3/arch-arm/
export CC="/Users/user/android-ndk-r7b/toolchains/arm-linux-androideabi- 4.4.3/prebuilt/darwin-x86/bin/arm-linux-androideabi-gcc"
export CFLAGS="-fPIC -DANDROID -nostdlib"
export ANDROID_ROOT="/Users/user/android-ndk-r7b"
export LDFLAGS="-Wl,-rpath-link=$ANDROID_ROOT/platforms/android-3/arch-arm/usr/lib/ -L$ANDROID_ROOT/platforms/android-3/arch-arm/usr/lib/"
export CPPFLAGS="-I$ANDROID_ROOT/platforms/android-3/arch-arm/usr/include/"
export LIBS="-lc "
./configure --host=arm-eabi
Delete the following lines from mentioned above Android.mk file:
src/google/protobuf/stubs/strutil.cc \
src/google/protobuf/stubs/substitute.cc \
src/google/protobuf/stubs/structurally_valid.cc \
src/google/protobuf/descriptor.cc \
src/google/protobuf/descriptor.pb.cc \
src/google/protobuf/descriptor_database.cc \
src/google/protobuf/dynamic_message.cc \
src/google/protobuf/extension_set_heavy.cc \
src/google/protobuf/generated_message_reflection.cc \
src/google/protobuf/message.cc \
src/google/protobuf/reflection_ops.cc \
src/google/protobuf/service.cc \
src/google/protobuf/text_format.cc \
src/google/protobuf/unknown_field_set.cc \
src/google/protobuf/wire_format.cc \
src/google/protobuf/io/gzip_stream.cc \
src/google/protobuf/io/printer.cc \
src/google/protobuf/io/tokenizer.cc \
src/google/protobuf/io/zero_copy_stream_impl.cc \
src/google/protobuf/compiler/importer.cc \
src/google/protobuf/compiler/parser.cc
It is needed for building lite version of library.
Unfortunately full version of the lib works just from Android 2.3.
I tested lite version for Android 1.6, 2.1, 2.3.
build the library using ndk-build.
for generating c++ files (using protoc compiler) for the lite library it is needed to include: option optimize_for = LITE_RUNTIME; in .proto file.

Resources