I'm working on my Web Server using Node JS and Express.js. I need to insert hex value in a Buffer, starting from a binary string. My code is something like this:
var buffer = fs.readFileSync("myBINfile.dat");
setValue(buffer, 4);
function setValue(buffer, i) {
var value1 = parseInt(buffer[i].toString(16), 16).toString(2).toString()
value1 = "0" + value1.substring(1, value1.length);
var hex = parseInt(value1, 2).toString(16);
console.log(hex); // print a8 (correct)
buffer[i] = hex;
console.log(buffer[i]); // print 0 (why?)
}
The buffer contains a hex file. value1 is read correctly. How can fix this this problem?
Thanks
You're writing a string value into your Buffer object rather than a numerical value that it expects. Replace the line:
var hex = parseInt(value1, 2).toString(16);
with:
var hex = parseInt(value1, 2);
"a8" is really just the integer value 168 (you'll find if you console.log(value1) before your var hex = parseInt(value1, 2).toString(16); line, you'll get 10101000 (168 in binary)). When you write this value to the buffer, you really just want to write the integer value, not the string "a8".
The "hex value" as you put it, is actually just a number, hex is simply a presentation format. You don't store "hex numbers" or "binary numbers", you just store the numbers themselves.
As a result doing this you'll find console.log(hex); outputs 168 instead and think "That's wrong though!", but it's not, because 168 is a8.
The reason why you'll find it works with some values but not others is because any values which result in a purely numerical hex value (e.g. "22" or "67") will be automatically converted the their numerical equivalent (22 or 67). In your case however "a8" the value cannot be converted to the number type required by the buffer and so is discarded and 0 is written.
Related
I have the following code in node and I am trying to convert to ColdFusion:
// a correct implementation of PKCS7. The rijndael js has a PKCS7 padding already implemented
// however, it incorrectly pads expecting the phrase to be multiples of 32 bytes when it should pad based on multiples
// 16 bytes. Also, every javascript implementation of PKCS7 assumes utf-8 char encoding. C# however is unicode or utf-16.
// This means that chars need to be treated in our code as 2 byte chars and not 1 byte chars.
function padBytes(string){
const strArray = [...new Buffer(string, 'ucs2')];
const need = 16 - ((strArray.length) % 16);
for(let i = 0; i < need; i++) {
strArray.push(need);
}
return Buffer.from(strArray);
}
I'm trying to understand exactly what this function is doing to convert it. As I think I understand it, it's converting the string to UTF-16 (UCS2) and then adding padding to each character. However, I don't understand why the need variable is the value it is, nor how exactly to achieve that in CF.
I also don't understand why it's only pushing the same value into the array over and over again. For starters, in my example script the string is 2018-06-14T15:44:10Z testaccount. The string array length is 64. I'm not sure how to achieve even that in CF.
I've tried character encoding, converting to binary and stuff to UTF-16 and just don't understand well enough the js function to replicate it in ColdFusion. I feel I'm missing something with the encoding.
EDIT:
The selected answer solves this problem, but because I was eventually trying to use the input data for encryption, the easier method was to not use this function at all but do the following:
<cfset stringToEncrypt = charsetDecode(input,"utf-16le") />
<cfset variables.result = EncryptBinary(stringToEncrypt, theKey, theAlgorithm, theIV) />
Update:
We followed up in chat and turns out the value is ultimately used with encrypt(). Since encrypt() already handles padding (automatically), no need for the custom padBytes() function. However, it did require switching to the less commonly used encryptBinary() function to maintain the UTF-16 encoding. The regular encrypt() function only handles UTF-8, which produces totally different results.
Trycf.com Example:
// Result with sample key/iv: P22lWwtD8pDrNdQGRb2T/w==
result = encrypt("abc", theKey, theAlgorithm, theEncoding, theIV);
// Result Result with sample key/iv: LJCROj8trkXVq1Q8SQNrbA==
input = charsetDecode("abc", "utf-16le");
result= binaryEncode(encryptBinary(input, theKey, theAlgorithm, theIV), "base64);
it's converting the string to utf-16
(ucs2) and then adding padding to each character.
... I feel I'm missing something with the encoding.
Yes, the first part seems to be decoding the string as UTF-16 (or UCS2 which are slightly different). As to what you're missing, you're not the only one. I couldn't get it to work either until I found this comment which explained "UTF-16" prepends a BOM. To omit the BOM, use either "UTF-16BE" or "UTF-16LE" depending on the endianess needed.
why it's only pushing the same value into the array over and over again.
Because that's the definition of PCKS7 padding. Instead of padding with something like nulls or zeroes, it calculates how many bytes padding are needed. Then uses that number as the padding value. For example, say a string needs an extra three bytes padding. PCKS7 appends the value 3 - three times: "string" + "3" + "3" + "3".
The rest of the code is similar in CF. Unfortunately, the results of charsetDecode() aren't mutable. You must build a separate array to hold the padding, then combine the two.
Note, this example combines the arrays using CF2016 specific syntax, but it could also be done with a simple loop instead
Function:
function padBytes(string text){
var combined = [];
var padding = [];
// decode as utf-16
var decoded = charsetDecode(arguments.text,"utf-16le");
// how many padding bytes are needed?
var need = 16 - (arrayLen(decoded) % 16);
// fill array with any padding bytes
for(var i = 0; i < need; i++) {
padding.append(need);
}
// concatenate the two arrays
// CF2016+ specific syntax. For earlier versions, use a loop
combined = combined.append(decoded, true);
combined = combined.append(padding, true);
return combined;
}
Usage:
result = padBytes("2018-06-14T15:44:10Z testaccount");
writeDump(binaryEncode( javacast("byte[]", result), "base64"));
I need to split a uint to a list of bits (list of chars, where every char is "0" or "1", is also Ok). The way I try to do it is to concatenate the uint into string first, using binary representation for numeric types - bin(), and then to split it using str_split_all():
var num : uint(bits:4) = 0xF; // Can be any number
print str_split_all(bin(num), "/w");
("/w" is string match pattern that means any char).
The output I expect:
"0"
"b"
"1"
"1"
"1"
"1"
But the actual output is:
0. "0b1111"
Why doesn't it work? Thank you for your help.
If you want to split an integer into a list of bits, you can use the %{...} operator:
var num_bits : list of bit = %{num};
You can find a working example on EDAPlayground.
As an extra clarification to your question, "/w" doesn't mean match any character. The string "/\w/" means match any single character in AWK Syntax. If you put that into your match expression, you'll get (almost) the output you want, but with some extra blanks interleaved (the separators).
Regardless, if you want to split a string into its constituting characters, str_split_all(...) isn't the way to go. It's easier to convert the string into ASCII characters and then convert those back to string again:
extend sys {
run() is also {
var num : uint(bits:4) = 0xF; // Can be any number
var num_bin : string = bin(num);
var num_bin_chars := num_bin.as_a(list of byte);
for each (char) in num_bin_chars {
var char_as_string : string;
unpack(packing.low, %{8'b0, char}, char_as_string);
print char_as_string;
};
};
};
The unpack(...) syntax is directly from the e Reference Manual, Section 2.8.3 Type Conversion Between Strings and Scalars or Lists of Scalars
Can I convert directly between a Swift Character and its Unicode numeric value? That is:
var i:Int = ... // A plain integer index.
var myCodeUnit:UInt16 = myString.utf16[i]
// Would like to say myChar = myCodeUnit as Character, or equivalent.
or...
var j:String.Index = ... // NOT an integer!
var myChar:Character = myString[j]
// Would like to say myCodeUnit = myChar as UInt16
I can say:
myCodeUnit = String(myChar).utf16[0]
but this means creating a new String for each character. And I am doing this thousands of times (parsing text) so that is a lot of new Strings that are immediately being discarded.
The type Character represents a "Unicode grapheme cluster", which can be multiple Unicode codepoints. If you want one Unicode codepoint, you should use the type UnicodeScalar instead.
As per the swift book:
String to Code Unit
To get codeunit/ordinals for each character of the String, you can do the following:
var yourSwiftString = "甲乙丙丁"
for scalar in yourSwiftString.unicodeScalars {
print("\(scalar.value) ")
}
Code Unit to String
Because swift current does not have a way to convert ordinals/code units back to UTF, the best way I found is to still NSString. i.e. if you have int ordinals (32bit but representing the 21bit codepoints) you can use the following to convert to Unicode:
var i = 22247
var unicode_str = NSString(bytes: &i, length: 4, encoding: NSUTF32LittleEndianStringEncoding)
Obviously if you want to convert a array of ints, you'll need to pack them into a array first.
I spoke to an Apple engineer who is working on Unicode and he says they have not completed the implementation of unicode characters in strings. Are you looking at getting a code unit or a full character? Because the only and proper way to get at a full unicode character is by using a for each loop on a string. ie
for c in "hello" {
// c is a unicode character of type Character
}
But, this is not implemented as of yet.
I have a binary vector which I converted to hex. I want this hex data to go to a .bin(or any other fornat) file. Since this is a vector,I tried to first convert it to a string so that the hex data can be formatted and then output to a file. Please see below the code I was trying to use. Also shown is the snapshot of the problem. As you can see, all the converted hex data is present in 1 cell. I want it to be byte wise for each cell.
my_new_vector = binaryVectorToHex(M); %M is my input binary matrix
%cellfun(FormatHexStr, mat2cell(my_new_vector), 'UniformOutput', false)
%new_vector = mat2cell(my_new_vector);
vect2str(my_new_vector); %[matlab file exchange function][2] for converting vector to string
FormatHexStr(my_new_vector,2); %FormatHexStr is a function for formatting hex values which requires a hex string as an input [function is here][3]
[n_rows,n_cols] = size(my_new_vector);
fileID = fopen('my_flipped_data.bin','wt');
for row = 1:n_cols
fprintf(fileID,'%d\n',my_new_vector(:,row));
end
fclose(fileID);
Convert the binary values into blocks of 8 in a uint8 (byte) variable, then write it to file.
nbytes = floor(length(M)/8);
bytevec = zeros(1,nbytes, 'uint8');
for i = 1:8
bytevec = bytevec + uint8((2^(8-i))*M(i:8:end));
end
fileID = fopen('my_flipped_data.bin','wb');
fwrite(fileID, bytevec);
fclose(fileID);
This writes the first bit as the MSB of the first byte. Use (2^(i-1)) for first bit as LSB.
Suppose there is a string:
String str="Hello";
HOw can i get the ASCII value of that above mentioned string?
Given your comment, it sounds like all you need is:
char[] chars = str.ToCharArray();
Array.Sort(chars);
A char value in .NET is actually a UTF-16 code unit, but for all ASCII characters, the UTF-16 code unit value is the same as the ASCII value anyway.
You can create a new string from the array like this:
string sortedText = new string(chars);
Console.WriteLine(chars);
As it happens, "Hello" is already in ascending ASCII order...
byte[] asciiBytes =Encoding.ASCII.GetBytes(str);
You now have an array of the ASCII value of the bytes