I just have this issue with reading from a network stream in C#. Since am more of a Java developer I came across this issue.
In java I have this option of knowing the length of the received packet using the following code
int length = dataiInputStream.read(rcvPacket);
eventhough the size of the byte array rcvPacket assigned is larger than the amount of elements contained in it. which will allow me to read only the required length of elements so that i do not have elements in the byte array containing zeros.
While i was trying to use a similar thing on C# which was
long len = networkStream.length;
but the docucmentation says that this property is not supported. is there a workaround for this?
Thank you
The Java code doesn't show you the length of a packet. It shows you the amount of data read within that read call, which could have come from multiple packets.
NetworkStream.Read returns the same information, except using 0 to indicate the end of the stream instead of -1.
Related
we are trying to process a 152 bytes UDP data frame from a remote service. By following the PeerToPeer Beckhoff infosys example (https://infosys.beckhoff.com/content/1033/tf6310_tc3_tcpip/18014398593720075.html?id=9052404215823027436) we are not able to see the entire 152 bytes message, just a couple of bytes.
Is it possible that the String variable would be only showing the characters until the first 00 bytes or similar (null delimiter)?
In the image below you can see the full UDP frame and what we get as message.
Thanks in advance.
You are correct, the Beckhoff PeerToPeer example will not work with binary data because it uses strings that will cut off at zero value. So it doesn't like the UDP data you have for it.
Instead you should use function blocks like e.g. ReceiveData which will work with a data array and pointers, thus allowing any byte value received. You can do a google search for 'Beckhoff ReceiveData' to get the precise information.
I've been working on a server that expects data to be received through a buffer. I have an object which is defined like this and some procedures that modify the buffer in it:
Packet* = ref object
buf*: seq[int8]
#buf*: array[0..4096, int8]
pos*: int
proc newPacket*(size: int): Packet =
result = Packet(buf: newSeq[int8](size))
#result = Packet()
proc sendPacket*(s: AsyncSocket, p: Packet) =
aSyncCheck s.send(addr(p.buf), p.pos)
Now the reason I have two lines commented is because that was the code I originally used, but creating an object that initialises an array with 4096 elements every time probably wasn't very good for performance. However, it works and the seq[int8] version does not.
The strange thing is though, my current code will work perfectly fine if I use the old static buffer buf*: array[0..4096, int8]. In sendPacket, I have made sure to check the data contained in the buffer to make sure both the array and seq[int8] versions are equal, and they are. (Or at least appear to be). In other words, if I were to do var p = createPacket(17) and write to p.buf with exactly 17 bytes, the values of the elements appear to be the same in both versions.
So despite the data appearing to be the same in both versions, I get a different result when calling send when passing the address of the buffer.
In case it matters, the data would be read like this:
result = p.buf[p.pos]
inc(p.pos)
And written to like this:
p.buf[p.pos] = cast[int8](value)
inc(p.pos)
Just a few things I've looked into, which were probably unrelated to my problem anyway: I looked at GC_ref and GC_unref which had no effect on my problem and also looked at maybe trying to use alloc0 where buf is defined as pointer but I couldn't seem to access the data of that pointer and that probably isn't what I should be doing in the first place. Also if I do var data = p.buf and pass the addr of data instead, I get a different result, but still not the intended one.
So I guess what I want to get to the bottom of is:
Why does send work perfectly fine when I use array[0..4096, int8] but not seq[int8] which is initialised with newSeq, even when they appear to contain the same data?
Does my current layout for receiving and writing data even make sense in a language like Nim (or any language for that matter)? Is there a better way?
In order not to initialize the array you can use the noinit pragma like this:
buf* {.noinit.}: array[0..4096, int8]
You are probably taking the pointer to the seq, not the pointer to the data inside the seq, so try using addr(p.buf[0]).
A pos field is useless if you are using the seq version since you have p.buf.len already, but you probably know that already and just left it in for the array. If you want to use the seq and expect large packets, make sure to use newSeqOfCap to only allocate the memory once.
Also, your array is 1 byte too big, it goes from 0 to 4096 inclusively! Instead you can use [0..4095, int8] or just [4096, int8].
Personally I would prefer to use a uint8 type inside of buf, so that you can just put in values from 0 to 255 instead of -128 to 127
Using a seq inside of a ref object means you have two layers of indirection when accessing buf, as well as two objects that the GC will have to clean up. You could just make Packet an alias for seq[uint8] (without ref): type Packet* = seq[uint8]. Or you can use the array version if you want to store some more data inside the Packet later on.
I've never worked with bluetooth before. I have to sends data via BLE and I've found the limit of 20 bytes per chunk.
The sender is an Arduino and the receiver could be both an Android or a Node.js app on a pc.
I have to send 9 values, stored in float values, so 4 bytes * 9 = 36 bytes. I need 2 chunks for all my data via BLE. The receiving part needs both chunks to process them. If some data are lost, I don't care.
I'm not expert in network protocols and I think I have to give each message an incremental timestamp so that the receiver can glue the two chunks with the same timestamp or discard the last one if the new timestamp is higher. But I'm not sure how to do a checksum, if I really need it or not, if I really have to care about it, or if - for a simple beta version of my system - I can ignore all those problems..
Does anyone can give me some advice? Like examples of similar situations handled with BLE communication?
You can get around the size limitation using the "Read Blob Request" of ATT. It allows you to read an attribute and also give an offset. So, you can use it to read the attribute with an offset of 0, if there's more than ATT_MTU bytes than you can request again with the offset at ATT_MTU*1, if there's still more ATT_MTU*2, etc... (You can read it in 3.4.4.5 of the Bluetooth v4.1 specifications; it's in the 4.0 spec too but I don't have that in front of me right now)
If the value changes between request, I'm not sure how you could go about detecting such a change. You could have the attribute send notifications when there's a change to interrupt the process in case the value changes in the middle of reading it.
I have been trying to use OMXCodec through Stagefright. I have implemented the code for ICS version of Android.I have two classes CustomDataSource which derives MediaSource and another is CustomOmxCodec which calls OMXCodec::Create method and execute read operation to decode h264 frames. I have tested this implementation on a device with omx.google.video.avc software decoder and it works fine. Now, when I try to run the same implementation on an android phone with hardware h264 decode, it returns error on read call. The error is as below:
[OMX.MTK.VIDEO.DECODER.AVC] ERROR (0x80001005, 0)
0x80001005 is for OMX_ErrorBadParameter.
and I get the error code -1103 on read operation.
I tried various parameters but no success.
The complete log is as below:
[OMX.MTK.VIDEO.DECODER.AVC] mVideoInputErrorRate (0.000000)
!##!>>create tid (21087) O<XCodec mOMXLivesLocally=0, mIsVideoDecoder (1), mIsVideoEncoder (0), mime(video/avc)
[OMX.MTK.VIDEO.DECODER.AVC] video dimensions are 640X480
mSupportesPartialFrames 1 err 0
[OMX.MTK.VIDEO.DECODER.AVC] allocating 10 buffers of size 65536 on input port.
[OMX.MTK.VIDEO.DECODER.AVC] mMemHeapBase = 0x00E8C288, mOutputBufferPoolMemBase=0x51F8E000, size = 9578848
[OMX.MTK.VIDEO.DECODER.AVC] ERROR (0x80001005, 0)
OMXCodec::onEvent--OMX Bad Parameter!!
Read Error : -1103
I'd grateful for any direction on this.
From the question, the hardware codec i.e. OMX.MTK.VIDEO.DECODER.AVC is not supporting one of the parameters being passed as part of the configuration steps.
From OMXCodec::create, configureCodec will be invoked which internally invokes a lot of other functions. Since the error is coming as part of OMXCodec::onEvent, one of the possible scenarios could be that the component encountered an error while decoding the first few bytes of the first frame.
Specifically, when the component encounters SPS and PPS (part of codec specific data), the component would typically trigger a portSettingsChanged. From your response, I feel that during this process, there is some error and hence, onEvent has been triggered.
Please share more logs to analyze further.
The MTK H264 decoder need the parameter csd-0 and csd-1 to init the decoder(You can get some information at http://developer.android.com/reference/android/media/MediaCodec.html). csd-0 and csd-1 stands for SPS and PPS of H264.I have asked a MTK engineer and he said that we can use the code below to set these two parameters.
byte[] sps = {0,0,0,1,103,100,0,40,-84,52,-59,1,-32,17,31,120,11,80,16,16,31
,0,0,3,3,-23,0,0,-22,96,-108};
byte[] pps = {0,0,0,1,104,-18,60,-128};
MediaFormat mFormat = MediaFormat.createVideoFormat("video/avc", width, height);
mFormat.setByteBuffer("csd-0", ByteBuffer.wrap(sps));
mFormat.setByteBuffer("csd-1", ByteBuffer.wrap(pps));
Maybe that's whay we got the OMX Bad Parameter error message.
From the logs and mapping the same to the implemented code, I feel that the following is happening
[OMX.MTK.VIDEO.DECODER.AVC] allocating 10 buffers of size 65536 on input port.
This step allocates the buffers on the input port of the decoder
From the flow of code, after input port buffers are allocated, the buffers on output port are allocated from nativeWindow through allocateOutputBuffersFromNativeWindow.
One of the steps as part of this method implementation is to increase the number of buffers on the output port by 2 and set the same to the OMX component as shown here.
I feel your error might be stemming from this specific point as nBufferSize is a read-only parameter of OMX_IndexParamPortDefinition index. Please refer to the OMX Standard, section 3.1.3.12.1, page 83, which clearly shows that nBufferSize is a read-only parameter.
It appears that your OMX component may be strictly OMX compliant component, whereas in Android from ICS onwards, certain deviations are expected. This could be one of the potential causes of your error.
P.S: If you could share more information, we could help further.
I am creating a protocol to have two applications talk over a TCP/IP stream and am figuring out how to design a header for my messages. Using the TCP header as an initial guide, I am wondering if I will need padding. I understand that when we're dealing with a cache, we want to make sure that data being stored fits in a row of cache so that when it is retrieved it is done so efficiently. However, I do not understand how it makes sense to pad a header considering that an application will parse a stream of bytes and store it how it sees fit.
For example: I want to send over a message header consisting of a 3 byte field followed by a 1 byte padding field for 32 bit alignment. Then I will send over the message data.
In this case, the receiver will just take 3 bytes from the stream and throw away the padding byte. And then start reading message data. As I see it, he will not be storing the 3 bytes and the message data the way he wants. The whole point of byte alignment is so that it will be retrieved in an efficient manner. But if the retriever doesn't care about the padding how will it be retrieved efficiently?
Without the padding, the retriever just takes the 3 header bytes from the stream and then takes the data bytes. Since the retriever stores these bytes however he wants, how does it matter whether or not the padding is done?
Maybe I'm missing the point of padding.
It's slightly hard to extract a question from this post, but with what I've said you guys can probably point out my misconceptions.
Please let me know what you guys think.
Thanks,
jbu
If word alignment of the message body is of some use, then by all means, pad the message to avoid other contortions. The padding will be of benefit if most of the message is processed as machine words with decent intensity.
If the message is a stream of bytes, for instance xml, then padding won't do you a whole heck of a lot of good.
As far as actually designing a wire protocol, you should probably consider using a plain text protocol with compression (including the header), which will probably use less bandwidth than any hand-designed binary protocol you could possibly invent.
I do not understand how it makes sense to pad a header considering that an application will parse a stream of bytes and store it how it sees fit.
If I'm a receiver, I might pass a buffer (i.e. an array of bytes) to the protocol driver (i.e. the TCP stack) and say, "give this back to me when there's data in it".
What I (the application) get back, then, is an array of bytes which contains the data. Using C-style tricks like "casting" and so on I can treat portions of this array as if it were words and double-words (not just bytes) ... provided that they're suitably aligned (which is where padding may be required).
Here's an example of a statement which reads a DWORD from an offset in a byte buffer:
DWORD getDword(const byte* buffer)
{
//we want the DWORD which starts at byte-offset 8
buffer += 8;
//dereference as if it were pointing to a DWORD
//(this would fail on some machines if the pointer
//weren't pointing to a DWORD-aligned boundary)
return *((DWORD*)buffer);
}
Here's the corresponding function in Intel assembly; note that it's a single opcode i.e. quite an efficient way to access the data, more efficient that reading and accumulating separate bytes:
mov eax,DWORD PTR [esi+8]
Oner reason to consider padding is if you plan to extend your protocol over time. Some of the padding can be intentionally set aside for future assignment.
Another reason to consider padding is to save a couple of bits on length fields. I.e. always a multiple of 4, or 8 saves 2 or 3 bits off the length field.
One other good reason that TCP has padding (which probably does not apply to you) is it allows dedicated network processing hardware to easily separate the data from the header. As the data always starts on a 32 bit boundary, it's easier to separate the header from the data when the packet gets routed.
If you have a 3 byte header and align it to 4 bytes, then designate the unused byte as 'reserved for future use' and require the bits to be zero (rejecting messages where they are not as malformed). That leaves you some extensibility. Or you might decide to use the byte as a version number - initially zero, and then incrementing it if (when) you make incompatible changes to the protocol. Don't let the value be 'undefined' and "don't care"; you'll never be able to use it if you start out that way.