How can I convert a Perl string to bytes32 like the java function below:
public static Bytes32 stringToBytes32(String string) {
byte[] byteValue = string.getBytes();
byte[] byteValueLen32 = new byte[32];
System.arraycopy(byteValue, 0, byteValueLen32, 0, byteValue.length);
return new Bytes32(byteValueLen32);
}
Is there any module available in CPAN to do this?
It looks like all this function does it to encode a string into bytes, then truncate/pad it so the result is exactly 32 bytes long.
The first part may be tricky because according to the documentation:
public byte[] getBytes()
Encodes this String into a sequence of bytes using the platform's default charset, storing the result into a new byte array.
The behavior of this method when this string cannot be encoded in the default charset is unspecified.
Perl doesn't really have a concept of a "default charset", but if you're willing to settle for UTF-8, it's not hard:
sub stringToBytes32 {
my ($str) = #_;
utf8::encode $str;
return pack 'a32', $str;
}
(See Encode::encode if you need a different encoding.)
pack is handy for producing data in binary formats. Here we use it to truncate/pad to 32 bytes.
Related
I have created two helper function(One for ByteBuffer to String & Vice-versa)
public static Charset charset = Charset.forName("UTF-8");
public static String bb_to_str(ByteBuffer buffer, Charset charset){
System.out.println("Printing start");
byte[] bytes;
if(buffer.hasArray()) {
bytes = buffer.array();
} else {
bytes = new byte[buffer.remaining()];
buffer.get(bytes);
}
return new String(bytes, charset);
}
public static ByteBuffer str_to_bb(String msg, Charset charset){
return ByteBuffer.wrap(msg.getBytes(charset));
}
I have a data key that I am encrypting using AWS KMS which is giving my ByteBuffer.
// Encrypt the data key using AWS KMS
ByteBuffer plaintext = ByteBuffer.wrap("ankit".getBytes(charset));
EncryptRequest req = new EncryptRequest().withKeyId(keyId);
req.setPlaintext(plaintext);
ByteBuffer ciphertext = kmsClient.encrypt(req).getCiphertextBlob();
// Convert the byte buffer to String
String cip = bb_to_str(ciphertext, charset);
Now the issue is that this is not working :
DecryptRequest req1 = new DecryptRequest().withCiphertextBlob(str_to_bb(cip, charset)).withKeyId(keyId);
but this is working.
DecryptRequest req1 = new DecryptRequest().withCiphertextBlob(ciphertext).withKeyId(keyId);
What is wrong with my code?
The error is trying to convert an arbitrary byte array into a String in bb_to_str(ciphertext, charset);.
ciphertext does not in any reasonable way represent a readable string, and definitely doesn't use the charset that you specify (whichever one it is).
String is meant to represent Unicode text. Trying to use it to represent anything else will run into any number of problems (mostly related to encodings).
In some programming languages the string type is a binary string (i.e. doesn't strictly represent Unicode text), but those are usually the same languages that cause massive encoding confusions.
If you want to represent an arbitrary byte[] as a String for some reason, then you need to pick some encoding to represent it. Common one is Base64 or hex strings. Base64 is more compact and hex string conceptually simpler, but takes up more space for the same amount of input data.
For the application that I am currently working on, I am required to read UTF-8 encoded strings from a binary file. These strings are not null-terminated, but rather are prefaced with a byte specifying their length.
When I attempt to read in such a string, all multibyte UTF-8 characters become ?. Find below a sample:
public void main(string[] args) {
File file = File.new_for_path("test.bin");
DataInputStream instream = new DataInputStream(file.read());
uint8[] chars = new uint8[instream.read_byte()];
instream.read(chars);
print(#"$((string) chars)\n");
}
This is, of course, a stripped sample. The actual binary files in question are encrypted, which is not reflected here. If I use this with a sample file test.bin that contains the byte sequence 09 52 C3 AD 61 73 74 72 61 64, or Ríastrad prefaced with its byte length in UTF-8. The expected output is thus Ríastrad, but the actual output is R?astrad.
Might anyone be able to shed some light on the problem and, perhaps, a solution?
You need to add Intl.setlocale (); to your code:
public void main(string[] args) {
Intl.setlocale ();
File file = File.new_for_path("test.bin");
DataInputStream instream = new DataInputStream(file.read());
uint8[] chars = new uint8[instream.read_byte()];
instream.read(chars);
print(#"$((string) chars)\n");
}
The default locale for print () is the C locale, which is US ASCII. Any character outside the US ASCII character range is presented as a ?. Using Intl.setlocale (); sets the locale to be the same as the machine running the program.
So I already know how to convert wstring to string (How to convert wstring into string?).
However, I would like to to know whether it is safe to make the conversion, meaning, the wstring variable does not contain any characters that are not supported in string type.
strings can hold any data, if you use the right encoding. They are just sequences of bytes. But you need to check with your particular encoding / conversion routine.
Should be simply a matter of round-tripping. An elegant solution to many things.
Warning, Pseudo-code, there is no literal convert_to_wstring() unless you make it so:
if(convert_to_wstring(convert_to_string(ws)) == ws)
happy_days();
If what goes in comes out, it is non-lossy (at least for your code points).
Not that its the most efficient solution, but should allow you to build from your favorite conversion routines.
// Round-trip and see if we lose anything
bool check_ws2s(const std::wstring& wstr)
{
return (s2ws(ws2s(str)) == wstr);
}
Using #dk123's conversions for C++11 at How to convert wstring into string? (Upvote his answer here https://stackoverflow.com/a/18374698/257090)
wstring s2ws(const std::string& str)
{
typedef std::codecvt_utf8<wchar_t> convert_typeX;
std::wstring_convert<convert_typeX, wchar_t> converterX;
return converterX.from_bytes(str);
}
string ws2s(const std::wstring& wstr)
{
typedef std::codecvt_utf8<wchar_t> convert_typeX;
std::wstring_convert<convert_typeX, wchar_t> converterX;
return converterX.to_bytes(wstr);
}
Note, if your idea of conversion is truncating the wide chars to chars, then it is simply a matter of iterating and checking that each wide char value fits in a char. This will probably do it.
WARNING: Not appropriate for multibyte encoding.
for(wchar_t& wc: ws) {
if(wc > static_cast<char>::(wc))
return false;
}
return true;
Or:
// Could use a narrowing cast comparison, but this avoids any warnings
for(wchar_t& wc: ws) {
if(wc > std::numeric_limits<char>::max())
return false;
}
return true;
FWIW, in Win32, there are conversion routines that accept a parameter of WC_ERR_INVALID_CHARS that tells the routine to fail instead of silently dropping code points. Non-standard solutions, of course.
Example: WideCharToMultiByte()
http://msdn.microsoft.com/en-us/library/windows/desktop/dd374130(v=vs.85).aspx
I am going through RPC tutorial and learn few techniques in rpcgen. I have the idea of adding, multiplying different data types using rpcgen.
But I have not found any clue that how could I declare a function in .x file which will return a string. Actually I am trying to build a procedure which will return a random string(rand string array is in server).
Can any one advise me how to proceed in this issue? It will be helpful if you advise me any tutorial regarding this returning string/pointer issue.
Thank you in advance.
Ok, answering to the original question (more than 2 years old), the first answer is correct but a little tricky.
In your .x file, you define your structure with the string inside, having defined previously the size of the string:
typedef string str_t<255>;
struct my_result {
str_t data;
};
...
Then you invoke rpcgen on your .x file to generate client and server stubs and .xdr file:
$rpcgen -N *file.x*
Now you can compile client and server in addition to any program where you pretend to use the remote functions. To do so, I followed the "repcgen Tutorial" in ORACLE's web page:
https://docs.oracle.com/cd/E19683-01/816-1435/rpcgenpguide-21470/index.html
The tricky part is, although you defined a string of size m (array of m characters) what rpcgen and .xdr file create is a pointer to allocated memmory. Something like this:
.h file
typedef char *str_t;
struct my_result {
int res;
str_t data;
};
typedef struct my_result my_result;
.xdr file
bool_t xdr_str_t (XDR *xdrs, str_t *objp)
{
register int32_t *buf;
if (!xdr_string (xdrs, objp, 255))
return FALSE;
return TRUE;
}
So just take into account when using this structure in your server side that it is not a string of size m, but a char pointer for which you'll have to reserve memory before using it or you'll be prompted the same error than me on execution:
Segmentation fault!
To use it on the server you can write:
static my_result response;
static char text[255];
memset(&response, '\0', sizeof(my_result));
memset(text, '\0', sizeof(text));
response.data = text;
And from there you are ready to use it wisely! :)
According to the XDR protocol specification you can define a string type where m is the length of the string in bytes:
The standard defines a string of n (numbered 0 to n -1) bytes to be the number n encoded as an unsigned integer (as described above), and followed by the n bytes of the string. Each byte must be regarded by the implementation as being 8-bit transparent data. This allows use of arbitrary character set encodings. Byte m of the string always precedes byte m +1 of the string, and byte 0 of the string always follows the string's length. If n is not a multiple of four, then the n bytes are followed by enough (0 to 3) residual zero bytes, r, to make the total byte count a multiple of four.
string object<m>;
You can then define a struct with the string type str_t as one of the variables:
typedef string str_t<255>;
struct my_result {
str_t data;
};
Then in your .x file you can define an RPC in your program which returns a struct of type my_result. Since rpcgen will give you a pointer to this struct (which I have called res) you can print the message with prinf("%s\n", res->data);.
program HELLO_PROG {
version HELLO_VERSION {
my_result abc() = 1;
} = 1;
} = 1000;
Iam have a DBC file, which is a database file for a game, containing ingame usable spell data, like ID, SpellName, Category etc...
Struct is something like this:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1)]
public struct SpellEntry
{
public uint ID;
public uint Category;
public float speed;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 8, ArraySubType = UnmanagedType.I4)]
public int[] Reagent;
public int EquippedItemClass;
[MarshalAs(UnmanagedType.LPStr)] // Crash here
public string SpellName;
}
Iam reading the file with a binary reader, and marshaling it to the struct. Snippet:
binReader.BaseStream.Seek(DBCFile.HEADER_SIZE + (index * 4 * 234), SeekOrigin.Begin);
buff = binReader.ReadBytes(buff.Length);
GCHandle handdle = GCHandle.Alloc(buff, GCHandleType.Pinned);
Spell.SpellEntry testspell = (Spell.SpellEntry)Marshal.PtrToStructure(handdle.AddrOfPinnedObject(), typeof(Spell.SpellEntry));
handdle.Free();
Now to be more complex, lets see how does the DBC file storing the strings, for example the SpellName. Its not in the records, strings are contained in the end of the file, in a "string table" block. The string data in the records contains a number (offset) to the string in the string table. (so its not really a string).
I managed to read all the strings from the string block (at the end of the file), to a string[]. (this is dont before start reading the records)
Then I would start reading the records, but first problem Is :
1.) I cant read it, because it "crashes" on the last line of my struct (because its not a string really)
2.) I cant assign a string to the number.
When I read it, it will be a number, but at the end, as a result, I have to assign that string to the SpellName, thats got pointed by the number, in the string table. Jeez .
public struct SpellEntry
{
//...
private int SpellNameOffset;
public string SpellName {
get { return Mumble.GetString(SpellNameOffset); }
}
}
This is hard to get right, Mumble must be a static class since you cannot add any members to SpellEntry. That screws up Marshal.SizeOf(), making it too large. You'll need to initialize Mumble so that its static GetString() method can access the string table. Moving the SpellName property into another class solves the problem but makes the code ugly too.
This is liable to confuse you badly. If you got a version going that uses BitConverter then you're definitely better off by using it instead. Separating the file format from the runtime format is in fact an asset here.