I am dealing with some legacy data where I routinely need to convert a uint16 to a 2 byte string.
Here is what I am using (where i is a uint16):
string([]byte {byte(i >> 8), byte(i & 0xFF)})
https://play.golang.org/p/423CAL-SJv
This seems rather clunky. Is there an existing library function to do this? I have looked at both the strings and binary packages, but nothing seemed immediately obvious.
While that is perfectly fine for what you're trying to do, the encoding/binary package has much more functionality for reading and writing binary values.
You can use
i := uint16(0x474F)
b := make([]byte, 2)
binary.BigEndian.PutUint16(b, i)
fmt.Println(string(b))
// GO
https://play.golang.org/p/IdDnnOtS2V
Try the following
t := strconv.Itoa(123)
Related
I'm pulling in data that is in long hexadecimal string form which I need to convert into decimal notation, truncate 18 decimal places, and then serve up in JSON.
For example I may have the hex string:
"0x00000000000000000000000000000000000000000000d3c21bcecceda1000000"
At first I was attempting to use ParseUint(), however since the highest it supports is int64, my number ends up being way too big.
This example after conversion and truncation results in 10^6.
However there are instances where this number can be up to 10^12 (meaning pre truncation 10^30!).
What is the best strategy to attack this?
Use math/big for working with numbers larger than 64 bits.
From the Int.SetString example:
s := "d3c21bcecceda1000000"
i := new(big.Int)
i.SetString(s, 16)
fmt.Println(i)
https://play.golang.org/p/vf31ce93vA
The math/big types also support the encoding.TextMarshaler and fmt.Scanner interfaces.
For example
i := new(big.Int)
fmt.Sscan("0x000000d3c21bcecceda1000000", i)
Or
i := new(big.Int)
fmt.Sscanf("0x000000d3c21bcecceda1000000", "0x%x", i)
I'm converting a node service to go. For this I need a compatible md5 hash (not for storing passwords!!) generator. However, in this example, I keep getting different results:
Node's crypto takes an encoding parameter when creating md5s.
> crypto.createHash("md5").update("1Editor’s notebook: Escaping temptation for turf145468066").digest("hex")
'c7c3210bd977b049f42c487b8c6d0463'
In golang: (test_encode.go)
package main
import (
"crypto/md5"
"encoding/hex"
"testing"
)
func TestFoo(t *testing.T) {
const result = "c7c3210bd977b049f42c487b8c6d0463"
stringToEncode := "1Editor’s notebook: Escaping temptation for turf145468066"
hash := md5.Sum([]byte(stringToEncode))
hashStr := hex.EncodeToString(hash[:])
if hashStr != result {
t.Error("Got", hashStr, "expected", result)
}
}
Then go test test_encode.go results in:
--- FAIL: TestFoo (0.00s)
encode_test.go:17: Got c3804ddcc59fabc09f0ce2418b3a8335 expected c7c3210bd977b049f42c487b8c6d0463
FAIL
FAIL command-line-arguments 0.006s
I've tracked it down to the encoding parameter of crypto.update in the node code. And the fact that the string as a ’ quote character in it. If I specify "utf8" it works.
crypto.createHash("md5").update("1Editor’s notebook: Escaping temptation for turf145468066", "utf8").digest("hex")
BUT: I can't change the node code, so the go code has to be compatible. Any ideas on what to do?
As you've already noted: you must convert the UTF8 string to whatever encoding is used in your node application. This can be done with encoding packages such as:
golang.org/x/text/encoding/charmap
isoString, err := charmap.ISO8859_1.NewEncoder().Bytes([]byte(stringToEncode))
Considering that the character ’ is not allowed in iso-8859-1, we can assume you have a different encoding. Now you just need to figure out which one!
And in worse case, you might have to use another package than charmap.
After a lot of digging in node and V8 I was able to conclude the following:
require("crypto").createHash("md5").update(inputString).digest("hex");
Is pretty dangerous, as not specifying a encodes the input string as "ASCII". Which, after a lot of digging, is the equivalent (verified on a large input set from my end):
// toNodeASCIIString converts a string to a byte of node compatible ASCII string
func toNodeASCIIString(inputString string) []byte {
lengthOfString := utf8.RuneCountInString(string(inputString))
stringAsRunes := []rune(inputString)
bytes := make([]byte, lengthOfString)
for i, r := range stringAsRunes {
bytes[i] = byte(r % 256)
}
return bytes
}
What is basically does is mods by 256 and forgets a large part of the input string.
The node example above is pretty much the standard and copy-pasted-everywhere way to create MD5 hashes in node. I have not checked but I'm assuming this works the same for all other hashes (SHA1, SHA256, etc).
I would love to hear someones thoughts on why this is not huge security hole.
I'm using Matlab Coder to convert some Matlab code to C++, however I'm having trouble converting intergers to strings.
int2str() is not supported for code generation, so I must find some other way to convert ints to strings. I've tried googling it, without success. Is this even possible?
This can be done manually (very painful, though)
function s = thePainfulInt2Str( n )
s = '';
is_pos = n > 0; % //save sign
n = abs(n); %// work with positive
while n > 0
c = mod( n, 10 ); % get current character
s = [uint8(c+'0'),s]; %// add the character
n = ( n - c ) / 10; %// "chop" it off and continue
end
if ~is_pos
s = ['-',s]; %// add the sign
end
sprintf is another very basic function, so it possibly works in C++ as well:
x = int64(1948)
str = sprintf('%i',x)
It is also the underlying function used by int2str.
According to this comprehensive list of supported functions, as pointed out by Matt in the comments, sprintf is not supported, which is surprising. However there is the undocumented helper function (therefore not in the list) sprintfc which seems to work and can be used equivalently:
str = sprintfc('%i',x)
I use the following workaround to enable sprintf for general use with Matlab Coder:
1) Create the following m-file named "sprintf.m", preferably in a folder NOT on your Matlab path:
function s = sprintf(f, varargin)
if (coder.target('MATLAB'))
s = builtin('sprintf',f,varargin{:});
elseif (coder.target('MEX'))
s = builtin('sprintf',f,varargin{:});
else
coder.cinclude('stdio.h');
s = char(zeros(1024,1));
cf = [f,0]; % NULL-terminated string for use in C
coder.ceval('sprintf_s', coder.ref(s(1)), int32(1024), coder.rref(cf(1)), varargin{:});
end
2) Ensure that sprintf is not specified as extrinsic via coder.extrinsic
3) Specify the folder containing the newly created "sprintf.m" as additional include directory when generating code. If you use the codegen function, this can be done via the -I switch. If you use the Coder App, this can be done under "More Settings -> Custom Code -> Additional include directories" from the "Generate" tab.
4) Convert from int to string as follows: s=sprintf('%d',int32(n));
Notes:
The specified "sprintf.m" shadows the built-in sprintf function and executes instead of the built-in function every time you call sprintf from generated code. By placing this file in a folder that is not on the Matlab path, you avoid calling it from other code made to run in Matlab. The coder.target call also helps to navigate back to the built-in function in case this gets called in a normal Matlab session or from a MEX file.
The code above limits the result to 1023 characters (a terminating zero is required at the end). The call to sprintf_s instructs the C++ compiler to throw a runtime exception if the result exceeds this value. This prevents memory corruption that is often only caught much later and is harder to trace back to the offending call. This limit can be modified to your own requirements.
Numeric types must be cast to the correct class before passing them to sprintf, e.g. cast to int32 to match a %d in the format string. This requirement is the same when using fprintf with Matlab Coder. However, in the fprintf case, Matlab Coder catches type errors for you. For sprintf, the C++ compiler may fail or the resulting string may contain errors.
String arguments must be NULL-terminated manually to be used in C calls, as Matlab Coder does not do this automatically (credit to Ryan Livingston for pointing this out). The code above ensures that the format string f is NULL-terminated, but NULL-termination of other string arguments remains the responsibility of the calling function.
This code was tested on a Windows platform with a Visual Studio C++ compiler and Matlab R2016a (Matlab Coder 3.1), but is expected to work in most other environments as well.
Edit: As of MATLAB R2018a, sprintf is supported for code generation by MATLAB Coder.
Pre R2018a Answer
You could also call the C runtime sprintf or snprintf using coder.ceval. This has the benefit of making supporting floating point inputs easy as well. You can also change the formatting as desired by tweaking the format string.
Supposing that your compiler provides snprintf one could use:
function s = cint2str(x)
%#codegen
if coder.target('MATLAB')
s = int2str(x);
else
coder.cinclude('<stdio.h>');
assert(isfloat(x) || isinteger(x), 'x must be a float or an integer');
assert(x == floor(x) && isfinite(x), 'x must be a finite integer value');
if isinteger(x)
switch class(x)
% Set up for Win64, change to match your target
case {'int8','int16','int32'}
fmt = '%d';
case 'int64'
fmt = '%lld';
case {'uint8','uint16','uint32'}
fmt = '%u';
otherwise
fmt = '%llu';
end
else
fmt = '%.0f';
end
% NULL-terminate for C
cfmt = [fmt, 0];
% Set up external C types
nt = coder.opaque('int','0');
szt = coder.opaque('size_t','0');
NULL = coder.opaque('char*','NULL');
% Query length
nt = coder.ceval('snprintf',NULL,szt,coder.rref(cfmt),x);
n = cast(nt,'int32');
ns = n+1; % +1 for trailing null
% Allocate and format
s = coder.nullcopy(blanks(ns));
nt = coder.ceval('snprintf',coder.ref(s),cast(ns,'like',szt),coder.rref(cfmt),x);
assert(cast(nt,'int32') == n, 'Failed to format string');
end
Note that you'll possibly need to tweak the format string to match the hardware on which you're running since this assumes that long long is available and maps 64-bit integers to it.
im working on delphi 7 and i was working on a strings, i came across this
For a string of default length, that is, declared simply as string, max size is always 255. A ShortString is never allowed to grow to more than 255 characters.
on delphi strings
once i had to do something like this in my delphi code (that was for a really big query)
var
sMyStringOF256characters : string;
ilength : integer;
begin
sMyStringOF256characters:='ThisStringisofLength256,ThisStringisofLength256,.....'
//length of sMyStringOF256characters is 256
end;
i get this error
[Error] u_home.pas(38): String literals may have at most 255 elements.
but when i try this
var
iCounter : integer;
myExtremlyLongString : string;
begin
myExtremlyLongString:='';
Label1.Caption:='';
for iCounter:=0 to 2500 do
begin
myExtremlyLongString:=myExtremlyLongString+inttostr(iCounter);
Label1.Caption:=myExtremlyLongString;
end;
Label2.Caption:=inttostr(length(myExtremlyLongString));
end;
and the result is
As you can see the length of myExtremlyLongString is 8894 characters.
why did not delphi give any error saying the length is beyond 255 for myExtremlyLongString?
EDIT
i used
SetLength(sMyStringOF256characters,300);
but it doesnt work.
why did not delphi give any error saying the length is beyond 255 for
myExtremlyLongString?
You have your answer a bit down in the text in section Long String (AnsiString).
In current versions of Delphi, the string type is simply an alias for
AnsiString,
So string is not limited to 255 characters but a string literal is. That means that you can build a string that is longer than 255 characters but you can not have a string value in code that is longer than 255 characters. You need to split them if you want that.
sMyString:='ThisStringisofLength255'+'ThisStringisofLength255';
Split it up into:
sMyStringOF256characters :=
'ThisStringis' +
'ofLength256' +
'And ManyManyManyManyManyManyManyManyManyManyManyManyMany' +
'ManyManyManyManyManyManyManyManyManyManyManyManyMany' +
'ManyManyManyManyManyManyManyManyManyManyManyManyMany' +
'ManyManyManyManyManyManyManyManyManyManyManyManyMany' +
'ManyManyManyManyManyManyManyManyManyManyManyManyMany' +
'ManyManyManyManyManyManyManyManyManyManyManyManyMany' +
'ManyManyManyManyManyManyManyManyManyManyManyManyMany' +
'ManyManyManyManyManyManyManyManyManyManyManyManyMany' +
'CharactersCharactersCharactersCharactersCharactersCharactersCharactersCharacters';
Back in old DOS/Turbo Pascal days, "strings" were indeed limited to 255 characters. In large part because the 1st byte contained the string length, and a byte can only have a value between 0 and 255.
That is no longer an issue in contemporary versions of Delphi.
"ShortString" is the type for the old DOS/Pascal string type.
"LongString" has been the default string type for a long time (including the Borland Delphi 2006 I currently use for most production work). LongStrings (aka "AnsiStrings") hold 8-bit characters, and are limited only by available memory.
Recent versions of Delphi (Delphi 2009 and higher, including the new Delphi XE2) all now default to multi-byte Unicode "WideString" strings. WideStrings, like AnsiStrings, are also effectively "unlimited" in maximum length.
This article explains in more detail:
http://delphi.about.com/od/beginners/l/aa071800a.htm
The difference is that in your first code example you are putting the string as part of your code - literal string. That has a limitation on how many characters it will allow.
In your second code example you are generating it dynamically and not putting it as one big literal string.
String type in Delphi (unlike shortstring that can only be up to 255) can be as big as your memory.
You could try using the StringBuilder class:
procedure TestStringBuilder;
var
I: Integer;
StringBuilder: TStringBuilder;
begin
StringBuilder := TStringBuilder.Create;
try
for I := 1 to 10 do
begin
StringBuilder.Append('a string ');
StringBuilder.Append(66); //add an integer
StringBuilder.Append(sLineBreak); //add new line
end;
OutputWriteLine('Final string builder length: ' + IntToStr(StringBuilder.Length));
finally
StringBuilder.Free;
end;
end;
If you need realy long string in Delphi, you can load it from other resources like a txt files or just plain text with any extension. Im using it and it works. You can create "like a" array tables using plain text lines numbers. In delphi code, you can do as #arjen van der Spek and others says only.
For me, files with text as var's formated -
sometext:string=
'txt...'+
'txt...'+
'txt...';
are bad for future editing.
pros: you can use any long text.
cons: text code is open, anybody can read it opening file in notepad etc.
I've just encountered a weird problem. I am trying to load a model to OpenGL and in the part where I load textures I use auxDIBImageLoadA(dibfile:PWideChar) function. Here is my code calling it
procedure CreateTexture(var textureArray: array of UINT; strFileName: string; textureID: integer); // Vytvožení textury
var
pBitmap: PTAUX_RGBImageRec;
begin
if strFileName = '' then exit;
MessageBox(0,PWideChar(strFileName),nil,SW_SHOWNORMAL);
pBitmap := auxDIBImageLoadA(PWideChar(strFileName));
if pBitmap = nil then exit;
...
The MessageBox is just for control. This is what happens: I run the application, a box with "FACE.BMP" appears. Okay. But then I get an error saying "Failed to open DIB file F". When i set the stFileName to xFACE.BMP, I get an "Failed to open DIB file x". So for some reason it appears that the function is taking only the first char.
Am I missing something? I'm using glaux.dll which I downloaded like 5 times from different sources, so it should be bug-free (I hope, every OpenGL site referred to it).
That's odd, functions ending in "A" generally take PAnsiChar pointers, and those ending in "W" take PWideChar pointers. Is there a auxDIBImageLoadW call also? If there is use that one, or try with PAnsiChar, since the PWideChar you pass (two bytes per position) would look like a string one character long if it is evaluated as a 1-byte string.
You need to convert your Unicode string to ANSI. Do it like this
pBitmap := auxDIBImageLoadA (PAnsiChar(AnsiString(strFileName)))
You would be better off calling the Unicode version though
pBitmap := auxDIBImageLoadW (PWideChar(strFileName))