I am familiar with C++ and I try to start to work with python.
From a serial line I manage to receive a string of binary characters (not ASCII) with python, let's say the rx 'buffer'.
I have to split this string into different fields, the method I am using is:
stx = (rx[0])
ctl = (rx[1])
node = (rx[2])
cTime = (rx[3:6])
nTime = (rx[7:10])
etx = (rx[11])
(currently I have not find a way to define a structure as for C++).
Now my problem is to print these fields as ASCII typically using:
print "%d-%d-%d-%ld-%ld-%d" % (stx,ctl,node,cTime,nTime,etx)
The error message is:
TypeError: %d format: a number is required, not str
I have already tried to convert the fields in different formats, but nothing works.
Can somebody help me?
You probably need to convert your variables to numbers by float(stx) (or int) but if you only want to print them just use %s instead of %d which expects the variable to be a double (meaning float).
For example:
rx = '1' * 12
stx = float(rx[0])
ctl = float(rx[1])
node = float(rx[2])
cTime = float(rx[3:6])
nTime = float(rx[7:10])
etx = float(rx[11])
print "%d-%d-%d-%ld-%ld-%d" % (stx,ctl,node,cTime,nTime,etx)
prints
1-1-1-111-111-1
or with strings you change the last line:
print "%s-%s-%s-%s-%s-%s" % (stx,ctl,node,cTime,nTime,etx)
In case you are dealing with raw binary data you might need the builtin struct module.
As far as I understand it (given you haven't posted an actual content and desired output I cannot verify it) you might want something like:
import struct
stx, ctl, node, cTime, nTime, etx = struct.unpack('fffddf', rx)
That would expect 3 floats, 2 doubles and then again 1 float. If you have integer or other datatypes you must edit the fffddf string. See the format types of struct
Brrr awfull
I have finally found one way for such a simple function:
> print int(stx.encode('hex'),16),int(ctl.encode('hex'),16),
> int(node.encode('hex'), 16), int(cTime.encode('hex'), 16),
> int(nTime.encode('hex'), 16), int(etx.encode('hex'), 16)
Not sure I will like python
Related
I have an abstract class Writer which allows clients to write into something. Could be the screen, could be a file. Now, I try to create a derived class to write into a string.
I have two problems with the denoted line in method write(...):
It's probably very inefficient. Is there something like a string buffer in Matlab?
It writes escape sequences like \n plain into the string, instead of taking their actual meaning.
How can I get the denoted line properly?
Code:
classdef StringTextWriter < Writer
properties
str;
end
methods
function this = StringTextWriter()
% Init the write-target which is a string in our case.
% (Other Writer classes would maybe open a file.)
this.str = '';
end
function write(this, val)
% Write to writer target.
% (Other Writer classes would maybe use fprinf here for file write.)
% ?????????????????????????????
this.str = [this.str val]; % How to do this properly?
% ?????????????????????????????
end
end
end
To answer your questions point by point:
The closest notion to a string buffer would be a string cell. Instead of:
str = '';
str = [strbuf, 'abc\n'];
str = [strbuf, 'def\n'];
str = [strbuf, 'ghi\n'];
%// and so on...
one may use
strbuf = {};
strbuf{end+1} = 'abc\n';
strbuf{end+1} = 'def\n';
strbuf{end+1} = 'ghi\n';
%// and so on...
str = sprintf([strbuf{:}]); %// careful about percent signs and single quotes in string
the drawback being that you have to reconstruct the string every time you ask for it. This can be alleviated by setting a modified flag every time you add strings to the end of strbuf, resetting it every time you concatenate the strings, and memoizing the result of concatenation in the last line (rebuild if modified, or last result if not).
Further improvement can be achieved by choosing a better strategy for growing the strbuf cell array; probably this would be effective if you have a lot of write method calls.
The escape sequences are really linked to the <?>printf family and not to the string literals, so MATLAB in general doesn't care about them, but sprintf in particular might.
I'm using Matlab Coder to convert some Matlab code to C++, however I'm having trouble converting intergers to strings.
int2str() is not supported for code generation, so I must find some other way to convert ints to strings. I've tried googling it, without success. Is this even possible?
This can be done manually (very painful, though)
function s = thePainfulInt2Str( n )
s = '';
is_pos = n > 0; % //save sign
n = abs(n); %// work with positive
while n > 0
c = mod( n, 10 ); % get current character
s = [uint8(c+'0'),s]; %// add the character
n = ( n - c ) / 10; %// "chop" it off and continue
end
if ~is_pos
s = ['-',s]; %// add the sign
end
sprintf is another very basic function, so it possibly works in C++ as well:
x = int64(1948)
str = sprintf('%i',x)
It is also the underlying function used by int2str.
According to this comprehensive list of supported functions, as pointed out by Matt in the comments, sprintf is not supported, which is surprising. However there is the undocumented helper function (therefore not in the list) sprintfc which seems to work and can be used equivalently:
str = sprintfc('%i',x)
I use the following workaround to enable sprintf for general use with Matlab Coder:
1) Create the following m-file named "sprintf.m", preferably in a folder NOT on your Matlab path:
function s = sprintf(f, varargin)
if (coder.target('MATLAB'))
s = builtin('sprintf',f,varargin{:});
elseif (coder.target('MEX'))
s = builtin('sprintf',f,varargin{:});
else
coder.cinclude('stdio.h');
s = char(zeros(1024,1));
cf = [f,0]; % NULL-terminated string for use in C
coder.ceval('sprintf_s', coder.ref(s(1)), int32(1024), coder.rref(cf(1)), varargin{:});
end
2) Ensure that sprintf is not specified as extrinsic via coder.extrinsic
3) Specify the folder containing the newly created "sprintf.m" as additional include directory when generating code. If you use the codegen function, this can be done via the -I switch. If you use the Coder App, this can be done under "More Settings -> Custom Code -> Additional include directories" from the "Generate" tab.
4) Convert from int to string as follows: s=sprintf('%d',int32(n));
Notes:
The specified "sprintf.m" shadows the built-in sprintf function and executes instead of the built-in function every time you call sprintf from generated code. By placing this file in a folder that is not on the Matlab path, you avoid calling it from other code made to run in Matlab. The coder.target call also helps to navigate back to the built-in function in case this gets called in a normal Matlab session or from a MEX file.
The code above limits the result to 1023 characters (a terminating zero is required at the end). The call to sprintf_s instructs the C++ compiler to throw a runtime exception if the result exceeds this value. This prevents memory corruption that is often only caught much later and is harder to trace back to the offending call. This limit can be modified to your own requirements.
Numeric types must be cast to the correct class before passing them to sprintf, e.g. cast to int32 to match a %d in the format string. This requirement is the same when using fprintf with Matlab Coder. However, in the fprintf case, Matlab Coder catches type errors for you. For sprintf, the C++ compiler may fail or the resulting string may contain errors.
String arguments must be NULL-terminated manually to be used in C calls, as Matlab Coder does not do this automatically (credit to Ryan Livingston for pointing this out). The code above ensures that the format string f is NULL-terminated, but NULL-termination of other string arguments remains the responsibility of the calling function.
This code was tested on a Windows platform with a Visual Studio C++ compiler and Matlab R2016a (Matlab Coder 3.1), but is expected to work in most other environments as well.
Edit: As of MATLAB R2018a, sprintf is supported for code generation by MATLAB Coder.
Pre R2018a Answer
You could also call the C runtime sprintf or snprintf using coder.ceval. This has the benefit of making supporting floating point inputs easy as well. You can also change the formatting as desired by tweaking the format string.
Supposing that your compiler provides snprintf one could use:
function s = cint2str(x)
%#codegen
if coder.target('MATLAB')
s = int2str(x);
else
coder.cinclude('<stdio.h>');
assert(isfloat(x) || isinteger(x), 'x must be a float or an integer');
assert(x == floor(x) && isfinite(x), 'x must be a finite integer value');
if isinteger(x)
switch class(x)
% Set up for Win64, change to match your target
case {'int8','int16','int32'}
fmt = '%d';
case 'int64'
fmt = '%lld';
case {'uint8','uint16','uint32'}
fmt = '%u';
otherwise
fmt = '%llu';
end
else
fmt = '%.0f';
end
% NULL-terminate for C
cfmt = [fmt, 0];
% Set up external C types
nt = coder.opaque('int','0');
szt = coder.opaque('size_t','0');
NULL = coder.opaque('char*','NULL');
% Query length
nt = coder.ceval('snprintf',NULL,szt,coder.rref(cfmt),x);
n = cast(nt,'int32');
ns = n+1; % +1 for trailing null
% Allocate and format
s = coder.nullcopy(blanks(ns));
nt = coder.ceval('snprintf',coder.ref(s),cast(ns,'like',szt),coder.rref(cfmt),x);
assert(cast(nt,'int32') == n, 'Failed to format string');
end
Note that you'll possibly need to tweak the format string to match the hardware on which you're running since this assumes that long long is available and maps 64-bit integers to it.
I wish to read some data from a .dat file without saving the file first. In order to do so, my code looks as follows:
urlsearch= 'http://minorplanetcenter.net/db_search/show_object?utf8=&object_id=2005+PM';
url= 'http://minorplanetcenter.net/tmp/2005_PM.dat';
urlmidstep=urlread(urlsearch);
urldata=urlread(url);
received= textscan(urldata , '%5s %7s %1s %1s %1s %17s %12s %12s %9s %6s %6s %3s ' ,'delimiter', '', 'whitespace', '');
data_received = received{:}
urlmidstep's function is just to do a "search", in order to be able to create the temporary .dat file. This data is then stored in urldata, which is a long char array. When I then use textscan in MATLAB, I get 12 columns as desired, which are stored in a cell array data_received.
However, in Octave I get various warning messages: warning: strread: field width '%5s' (fmt spec # 1) extends beyond actual word limit (for various field widths). My question is, why is my result different in Octave and how could I fix this? Shouldn't Octave behave the same as MATLAB, as in theory any differences should be dealt with as bugs?
Surely specifying the width of the strings and leaving both the delimiter and whitespace input arguments empty should tell the function to only deal with width of string, allowing spaces to be a valid characters.
Any help would be much appreciated.
I thinhk textscan works differently in MATLAB and Octave. To illustrate let's simplify the example. The code:
test_line = 'K05P00M C2003 01 28.38344309 37 57.87 +11 05 14.9 n~1HzV645';
test = textscan(test_line,'%5s','delimiter','');
test{:}
will would yield the following in MATLAB:
>> test{:}
ans =
'K05P0'
'0M C'
'2003 '
'01 28'
'.3834'
'4309 '
'37 57'
'.87 +'
'11 05'
'14.9 '
'n~1Hz'
'V645'
whereas in Octave, you get:
>> test{:}
ans =
{
[1,1] = K05P0
[2,1] = C2003
[3,1] = 01
[4,1] = 28.38
[5,1] = 37
[6,1] = 57.87
[7,1] = +11
[8,1] = 05
[9,1] = 14.9
[10,1] = n~1Hz
}
So it looks like Octave jumps to the next word and discards any remaining character in the current word, whereas MATLAB treats the whole string as one continuous word.
Why that is and which is one is correct, I do not know, but hopefully it'll point you in the right direction for understanding what is going on. You can try adding the delimiter to see how it affects the results.
I'm reading fixed-width (9 characters) data from a text file using textscan. Textscan fails at a certain line containing the string:
' 9574865.0E+10 '
I would like to read two numbers from this:
957486 5.0E+10
The problem can be replicated like this:
dat = textscan(' 9574865.0E+10 ','%9f %9f','Delimiter','','CollectOutput',true,'ReturnOnError',false);
The following error is returned:
Error using textscan
Mismatch between file and format string.
Trouble reading floating point number from file (row 1u, field 2u) ==> E+10
Surprisingly, if we add a minus, we don't get an error, but a wrong result:
dat = textscan(' -9574865.0E+10 ','%9f %9f','Delimiter','','CollectOutput',true,'ReturnOnError',false);
Now dat{1} is:
-9574865 0
Obviously, I need both cases to work. My current workaround is to add commas between the fields and use commas as a delimiter in textscan, but that's slow and not a nice solution. Is there any way I can read this string correctly using textscan or another built-in (for performance reasons) MATLAB function?
I suspect textscan first trims leading white space, and then parses the format string. I think this, because if you change yuor format string from
'%9f%9f'
to
'%6f%9f'
your one-liner suddenly works. Also, if you try
'%9s%9s'
you'll see that the first string has its leading whitespace removed (and therefore has 3 characters "too many"), but for some reason, the last string keeps its trailing whitespace.
Obviously, this means you'd have to know exactly how many digits there are in both numbers. I'm guessing this is not desirable.
A workaround could be something like the following:
% Split string on the "dot"
dat = textscan(<your data>,'%9s%9s',...
'Delimiter' , '.',...
'CollectOutput' , true,...
'ReturnOnError' , false);
% Correct the strings; move the last digit of the first string to the
% front of the second string, and put the dot back
dat = cellfun(#(x,y) str2double({y(1:end-1), [y(end) '.' x]}), dat{1}(:,2), dat{1}(:,1), 'UniformOutput', false);
% Cast to regular array
dat = cat(1, dat{:})
I had a similar problem and solved it by calling textscan twice, which proved to be way faster than cellfun or str2double and will work with any input that can be interpreted by Matlab's '%f'
In your case I would first call textscan with only string arguments and Whitespace = '' to correctly define the width of the fields.
data = ' 9574865.0E+10 ';
tmp = textscan(data, '%9s %9s', 'Whitespace', '');
Now you need to interweave and append a delimiter that won't interfere with your data, for example ;
tmp = [char(join([tmp{:}],';',2)) ';'];
And now you can apply the right format to your data by calling textscan again with a delimiter like:
result = textscan(tmp, '%f %f', 'Delimiter', ';', 'CollectOutput', true);
format shortE
result{:}
ans =
9.5749e+05 5.0000e+10
Comparing the speed of this approach with str2double:
n = 50000;
data = repmat(' 9574865.0E+10 ', n, 1);
% Approach 1 with str2double
tic
tmp = textscan(data', '%9s %9s', 'Whitespace', '');
result1 = str2double([tmp{:}]);
toc
Elapsed time is 2.435376 seconds.
% Approach 2 with double textscan
tic
tmp = textscan(data', '%9s %9s', 'Whitespace', '');
tmp = [char(join([tmp{:}],';',2)) char(59)*ones(n,1)]; % char(59) is just ';'
result2 = cell2mat(textscan(tmp', '%f %f', 'Delimiter', ';', 'CollectOutput', true));
toc
Elapsed time is 0.098833 seconds.
I'm trying to read hex values from a binary file. I don't have a problem with extracting a string and converting letters to hex values, but how can I do that with control characters and other non-printable characters? Is there a way of reading the string directly in hex values without the need of converting it?
Have a look over here:
As a last example, the following program makes a dump of a binary file. Again, the first program argument is the input file name; the output goes to the standard output. The program reads the file in chunks of 10 bytes. For each chunk, it writes the hexadecimal representation of each byte, and then it writes the chunk as text, changing control characters to dots.
local f = assert(io.open(arg[1], "rb"))
local block = 10
while true do
local bytes = f:read(block)
if not bytes then break end
for b in string.gfind(bytes, ".") do
io.write(string.format("%02X ", string.byte(b)))
end
io.write(string.rep(" ", block - string.len(bytes) + 1))
io.write(string.gsub(bytes, "%c", "."), "\n")
end
From your question it's not clear what exactly you aim to do, so I'll give 2 approaches.
Either you have a file full with hex values, and read it like this:
s='ABCDEF1234567890'
t={}
for val in s:lower():gmatch'(%x%x)' do
-- do whatever you want with the data
t[#t+1]=s:char(val)
end
Or you have a binary file, and you convert it to hex values:
s='kl978331asdfjhvkasdf'
t={s:byte(1,-1)}