I've started taking a look at Nim for hobby game modding purposes.
Intro
Yet, I found it difficult to work with Nim compared to C when it comes to machine-specific low-level memory layout and would like to know if Nim actually has better support here.
I need to control byte order and be able to de/serialize arbitrary Plain-Old-Datatype objects to binary custom file formats. I didn't directly find a Nim library which allows flexible storage options like representing enum and pointers with Big-Endian 32-bit. Or maybe I just don't know how to use the feature.
std/marshal : just JSON, i.e. no efficient, flexible nor binary format but cross-compatible
nim-serialization : seems like being made for human readable formats
nesm : flexible cross-compatibility? (It has some options and has a good interface)
flatty : no flexible cross-compatibility, no byte order?
msgpack4nim : no flexible cross-compatibility, byte order?
bingo : ?
Flexible cross-compatibility means, it must be able to de/serialize fields independently of Nim's ABI but with customization options.
Maybe "Kaitai Struct" is more what I look for, a file parser with experimental Nim support.
TL;DR
As a workaround for a serialization library I tried myself at a recursive "member fields reverser" that makes use of std/endians which is almost sufficient.
But I didn't succeed with implementing byte reversal of arbitrarily long objects in Nim. Not practically relevant but I still wonder if Nim has a solution.
I found reverse() and reversed() from std/algorithm but I need a byte array to reverse it and turn it back into the original object type. In C++ there would be reinterprete_cast, in C there is void*-cast, in D there is a void[] cast (D allows defining array slices from pointers) but I couldn't get it working with Nim.
I tried cast[ptr array[value.sizeof, byte]](unsafeAddr value)[] but I can't assign it to a new variable. Maybe there was a different problem.
How to "byte reverse" arbitrary long Plain-Old-Datatype objects?
How to serialize to binary files with byte order, member field size, pointer as file "offset - start offset"? Are there bitfield options in Nim?
It is indeed possible to use algorithm.reverse and the appropriate cast invocation to reverse bytes in-place:
import std/[algorithm,strutils,strformat]
type
LittleEnd{.packed.} = object
a: int8
b: int16
c: int32
BigEnd{.packed.} = object
c: int32
b: int16
a: int8
## just so we can see what's going on:
proc `$`(b: LittleEnd):string = &"(a:0x{b.a.toHex}, b:0x{b.b.toHex}, c:0x{b.c.toHex})"
proc `$`(l:BigEnd):string = &"(c:0x{l.c.toHex}, b:0x{l.b.toHex}, a:0x{l.a.toHex})"
var lit = LittleEnd(a: 0x12, b:0x3456, c: 0x789a_bcde)
echo lit # (a:0x12, b:0x3456, c:0x789ABCDE)
var big:BigEnd
copyMem(big.addr,lit.addr,sizeof(lit))
# here's the reinterpret_cast you were looking for:
cast[var array[sizeof(big),byte]](big.addr).reverse
echo big # (c:0xDEBC9A78, b:0x5634, a:0x12)
for C-style bitfields there is also the {.bitsize.} pragma
but using it causes Nim to lose sizeof information, and of course bitfields wont be reversed within bytes
import std/[algorithm,strutils,strformat]
type
LittleNib{.packed.} = object
a{.bitsize: 4}: int8
b{.bitsize: 12}: int16
c{.bitsize: 20}: int32
d{.bitsize: 28}: int32
BigNib{.packed.} = object
d{.bitsize: 28}: int32
c{.bitsize: 20}: int32
b{.bitsize: 12}: int16
a{.bitsize: 4}: int8
const nibsize = 8
proc `$`(b: LittleNib):string = &"(a:0x{b.a.toHex(1)}, b:0x{b.b.toHex(3)}, c:0x{b.c.toHex(5)}, d:0x{b.d.toHex(7)})"
proc `$`(l:BigNib):string = &"(d:0x{l.d.toHex(7)}, c:0x{l.c.toHex(5)}, b:0x{l.b.toHex(3)}, a:0x{l.a.toHex(1)})"
var lit = LitNib(a: 0x1,b:0x234, c:0x56789, d: 0x0abcdef)
echo lit # (a:0x1, b:0x234, c:0x56789, d:0x0ABCDEF)
var big:BigNib
copyMem(big.addr,lit.addr,nibsize)
cast[var array[nibsize,byte]](big.addr).reverse
echo big # (d:0x5DEBC0A, c:0x8967F, b:0x123, a:0x4)
It's less than optimal to copy the bytes over, then rearrange them with reverse, anyway, so you might just want to copy the bytes over in a loop. Here's a proc that can swap the endianness of any object, (including ones for which sizeof is not known at compiletime):
template asBytes[T](x:var T):ptr UncheckedArray[byte] =
cast[ptr UncheckedArray[byte]](x.addr)
proc swapEndian[T,U](src:var T,dst:var U) =
assert sizeof(src) == sizeof(dst)
let len = sizeof(src)
for i in 0..<len:
dst.asBytes[len - i - 1] = src.asBytes[i]
Bit fields are supported in Nim as a set of enums:
type
MyFlag* {.size: sizeof(cint).} = enum
A
B
C
D
MyFlags = set[MyFlag]
proc toNum(f: MyFlags): int = cast[cint](f)
proc toFlags(v: int): MyFlags = cast[MyFlags](v)
assert toNum({}) == 0
assert toNum({A}) == 1
assert toNum({D}) == 8
assert toNum({A, C}) == 5
assert toFlags(0) == {}
assert toFlags(7) == {A, B, C}
For arbitrary bit operations you have the bitops module, and for endianness conversions you have the endians module. But you already know about the endians module, so it's not clear what problem you are trying to solve with the so called byte reversal. Usually you have an integer, so you first convert the integer to byte endian format, for instance, then save that. And when you read back, convert from byte endian format and you have the int. The endianness procs should be dealing with reversal or not of bytes, so why do you need to do one yourself? In any case, you can follow the source hyperlink of the documentation and see how the endian procs are implemented. This can give you an idea of how to cast values in case you need to do some yourself.
Since you know C maybe the last resort would be to write a few serialization functions and call them from Nim, or directly embed them using the emit pragma. However this looks like the least cross platform and pain free option.
Can't answer anything about generic data structure serialization libraries. I stray away from them because they tend to require hand holding imposing certain limitations on your code and depending on the feature set, a simple refactoring (changing field order in your POD) may destroy the binary compatibility of the generated output without you noticing it until runtime. So you end up spending additional time writing unit tests to verify that the black box you brought in to save you some time behaves as you want (and keeps doing so across refactorings and version upgrades!).
In RcppArmadillo, I need to know how I can convert arma::mat to c-style array double * for use in other functions.
When I run the following functions, the computer crashes:
R part:
nn3 <- function(x){
results=.Call("KNNCV", PACKAGE = "KODAMA", x)
results
}
C++ part:
double KNNCV(arma::mat x) {
double *cvpred = x.memptr();
return cvpred[1];
}
and at the end, I try:
nn3(as.matrix(iris[,-5]))
Can you help me to find the errors, please?
First, there is no such such thing as vector stored in a double*. You can cast to a C-style pointer to double; but without length information that does not buy you much.
By convention, most similar C++ classes give you a .begin() iterator to the beginning of the memory block (which Armadillo happens to guarantee to be contiguous, just like std::vector) so you can try that.
Other than that the (very fine indeed) Armadillo documentation tells you about memptr() which is probably what you want here. Straight copy from the example there:
mat A = randu<mat>(5,5);
const mat B = randu<mat>(5,5);
double* A_mem = A.memptr();
const double* B_mem = B.memptr();
This is the Java code that results 897986030:
import java.util.Arrays;
import java.util.Scanner;
class Algorithm {
public static void main(String args[]) throws Exception {
int mod = 1000000007;
long factor = 900414279;
long p1 = 883069911;
long p2 = 32;
long val = 560076994;
val = (val%mod+factor*p1*p2%mod)%mod;
System.out.println(val);
}
}
This is the equivalent Python code that outputs 480330031:
factor = 900414279
p1 = 883069911
p2 = 32;
val = 560076994;
mod = 1000000007;
val = (val%mod+factor*p1*p2%mod)%mod;
print val
Please help. Thanks!
The answer lies in the fact that you are using primitive types in java that are prone to overflows.
Let me explain if you are not aware of this concept already. In java, C, C++ and the likes primitive types have a certain amount of space allocated for them and the variables cannot use any space more than that. This means that there is a maximum number that the long data type can store. This is done for performance reasons.
What might be happening in the code above is that when you multiply two long values the result might become larger than the maximum the long data type can store. And this results in an overflow, causing the data to be narrowed. So the results of the math expression are messed up.
With Python this is not that much of an issue because Python can store numbers of a much larger range. Overflows are rare in Python. And this is why things like cryptographic applications where big numbers are used are easy to write in Python.
I want the app when it is executed to start a different layout, I have this code that allows me to choose the layout where to start but want it to be random thanks.
myPager.setCurrentItem(0);
PD: excuse my English, not speaking, he used the translator
You can use Java's built-in pseudo random number generator to get a random value in a defined range. Use this to choose a random value for setCurrentItem. For example:
Random r = new Random();
int randomValue = r.nextInt( 5 );
myPager.setCurrentItem( randomValue );
The value you pass to the nextInt method is one more than the maximum integer you want to receive; the example above will return a random value between 0 and 4 (excluding 5).
I want to be able to pass in a string - list - into a C node for Erlang and return back a string to the Erlang process after some computation.
result = function(input_string),
where input_string is of type ETERM *
My computation on the string uses char * variables for strings.
Please let me know if this is sufficient information to go on.
[The C node example in the erl_interface tutorial uses integer input to and output from the C node]
TIA,
BR,
Shailen
Use erl_iolist_to_string for input (every Erlang "string" is also an iolist).
Use erl_mk_string instead of erl_mk_int for output.
http://erlang.org/doc/man/erl_eterm.html#erl_iolist_to_string
http://erlang.org/doc/man/erl_eterm.html#erl_mk_string