What is the procedure for understanding how the FEC mechanism in the Unetstack works, and what algorithm it uses to detect and correct the errors?
To understand, how FEC(Forward Error Correction) is working in the Unetstack, Firstly I tried to introduce errors while transmitting data from sender to receiver. The method being, altering the parameters of modem/channel such as noise levels, depth, powerlevels etc; thinking that these changes would introduce some errors and send erroneous data to the receiver, but it was noted that if we change the parameters after a certain limit, there is no data received on the receiver side instead just a BadFrameNtf was thrown. So, I wasn't able to analyze the FEC mechanism here as no data was received.
It would be really helpful if you could provide recommendations for improving the current technique followed or in offering new ways to introduce errors and analyse the FEC mechanism during data transmission in the Unetstack.
The community version of UnetStack is primarily a network simulator, and abstracts out the physical layer with a modem model that computes the probability of packet loss. In this model, there is no need for any FEC to be implemented per se, although the packet loss probability may include a notional FEC.
The community version of UnetStack audio implements a ½-rate interleaved convolution code as the only available FEC. You can see that by typing phy[1].fecList or phy[2].fecList on the modem shell:
> phy[1].fecList
[iconv-1/2]
The commercial versions of UnetStack use more performant FECs including LDPC, BCH, etc, for various code rates (differing correction capability). If you own a modem with UnetStack, you will have access to those. Example:
> phy[1].fecList
[ldpcv3-1/1, ldpc-1/1, ldpc-2/3, iconv-1/2, janus-spec, janus-ref, ldpc-1/2, bch-1/3, ldpcv3-1/3, ldpc-1/3, bch-1/4, ldpcv3-1/4, ldpc-1/4, bch-1/5, ldpcv3-1/5, ldpc-1/5, ldpcv3-1/6, ldpc-1/6]
Related
I'm building sound recognition model to detect "ding-dong" sound.
There are two procedures, training and testing.
Training data are "ding-dong" sounds generated by a device.
The model can detect "ding-dong" sounds generated by the same device, it works well.
But, when there is a new "ding-dong" sound generated by the second device, the performance will be bad.
I know the possible solution of this issue: record "ding-dong" sound generated by the second device and add it to training data.
But, there is always a new device, new "ding-dong" sound.
What should I do ?
You are facing overfitting problem. Overfitting means that your model has trained to work optimally on specific cases which are the training data set. In order to overcome this problem you should train your model on many devices and then make interpolation between them. Interpolation may be guaranteed by the model you are using.
However, the previous information is so general. In your case, you may find much much easier way to do it. All is depend about how you define "ding-dong". If you could find a siguntur for the "ding-dong" it would be great. This signature should be invariant to all undesirable features.
For example, should "Diiiiing-doooooong" be accepted? if yes, you should find a signature which invariant to length audio clip. Is "ding-dong" with higher frequency acceptable? If yes, you should find a signature which take frequencies as fraction of each other not as absolute values and so on...
BTW, I am sure you may google this and find many many papers about your problem but it may be about "dang-dong" not "ding-dong" but you will still able to benefit from it ;)
So you want to recognize "ding dong sounds" from "other sounds".
One approach could be to train also the data to recognize "other sounds" as an other class. Therefore, a new ding dong could be more easily associated with the "ding-dong sounds" than the "others-sounds".
One drawback of this method could be a growth in the number of "false alarms", but this task always leads to a compromise between precision and recall.
I know that in networking, error detection (or sometimes correction) mechanisms are enforced in the data link layers, networking layers, in tcp or even higher layers. But for example, for each 4KB of data, considering error detection of all layers, a total of as much as 200 bytes of error checking bytes are used. So even with good checksum functions, theoretically, collisions are possible. Why do people use these error detection mechanisms then? Are anomalis that much unlikely to occur?
If you want short answer than no, they cannot always be relied on and if you have really critical data than you should encapsulate the data yourself or transfer with seperate channel some good hash like f.e. SHA-256 to confirm that data was transfered without mistakes.
Ethernet CRC will catch most of errors like single bit error or any odd number of single bit errors. Some errors can go undetected but its extremely rare and its discussive what exact probability of error is but it's less than 1 in 2^32. Moreover every Ethernet device between source and destination is recalculating so it is more robust to errors assuming that every device is working properly.
These remaining errors should be caught by IP and TCP checksums. But these checksum calculation could not detect all errors f.e. : reordering two byte words or multiple errors that sum to zero.
In "Performance of Checksums and CRCs over Real Data" by Jonathan Stone, Michael Greenwald, Craig Partridge and Jim Hughes you could find some real data that suggest that about one in billion TCP segments have correct checksum while containing corrupted data.
So I will say that error detection mechanism in ISO/OSI model give us enough protection in most applications, get rid of most errors while being effective and fast. But if you use some additional hash than you are allmost robust to errors. Just check these table from article on hash collisions
I have an application that using libpcap as the mechanism for acquiring packets, and I need to filter out different protocols for different parts of the application. I need to consider optimization as the streams will have a high rate of traffic (100-400 Mbps).
What I would really like to be able to do is set up a live capture (no filter), and then selectively filter packets after the capture is made. It doesn't seem like this is possible (the bpf is built into the capture mechanism from what I can tell).
If this indeed is not possible, there are two other ways of doing it (that I have thought of), and I am not sure what would be considered more efficient or 'better':
Make multiple captures each with their own filter
Make one capture (no filter) that dumps to fifos, and have other captures read from those fifo (with their own filters)
The fifo-approach is probably not very efficient as it involves copying lots and lots of memory from A to B (e.g. 400mbps buffered - they must not block each other - to four fifos, each having a different filter, deciding to throw away 99.99% of accumulated 1600mbps). Multiple captures on the other hand only trigger action in userland if there is actually stuff to do. The filtering is (usually) done in the kernel.
A third approach is to use libwireshark, the lower portion of Wireshark, to do filtering (and wtap for capturing). This involves quite some code overhead as libwireshark is not exactly in perfect shape for third party use outside of Wireshark.
However this does come with the ability to use Wireshark's "Display Filters", which are compiled to bytecode and reasonably efficient. Many filters may be compiled once and may look at the same frame one after another. You may be able to "stack" filters as e.g. "ip.tcp" implies "ip".
This becomes quite efficient if you are able to generate the most common element of all filters and place it as a BPF-filter on your capturing device. The display-filters then only look at data that might interest at least one of them.
I'm currently working with a system that makes scheduling decisions based on a series of requests and the state of the system.
I would like to take the stream of real inputs, mock out some of the components, and run simulations against the rest. The idea is to use it for planning with respect to system capacity (i.e. when to scale certain components), tracking down certain failure modes, and analyzing the effects of changes to the codebase (i.e. simulations with version A compared to simulations with version B).
I can do everything related to this, except generate a suitable input stream. Replaying the exact input from production hasn't been very helpful because it's hard to get a long enough data stream to tease out some of the behavior that I'm trying to find. In other words, if production falls over at 300 days of input, I don't have enough data to find out until after it fell over. Repeating the same input set has been considered; but after a few initial tries, the developers all agree that the simulation seems to "need more random".
About this particular system:
The input is a series of irregularly spaced events (i.e. a stochastic process with discrete time and continuous state space).
Properties are not independent of each other.
Even the more independent of the properties are composites of other properties that will always be, by nature, invisible to me (leading to a multi-modal distribution).
Request interval is not independent of other properties (i.e. lots of requests for small amounts of resources come through in a batch, large requests don't).
There are feedback loops in it.
It's provably chaotic.
So:
Given a stream of input events with a certain distribution of various properties (including interval), how do I generate an infinite stream of events with the same distribution across a number of non-independent properties?
Having looked around, I think I need to do a Markov-Chain Monte-Carlo Simulation. My problem is figuring out how to build the Markov-Chain from the existing input data.
Maybe it is possible to model the input with a Copula. There are tools that help you doing so, e.g. see this paper. Apart from this, I would suggest to move the question to http://stats.stackexchange.com, as this is a statistical problem and will likely draw more attention over there.
I'm new to networking in general and I read about this protocol called Aloha, and I would like to make a simple simulator for the Pure version of it.
Although I understand the concepts, I find it difficult to start.
Basically we have a N senders in the network. Each sender wants to send a packet. Now each sender doesn't care if the network is busy or under occupation by some other sender. If it wants to send data, it just sends it.
The problem is that if 2 senders send some data at the same time, then they will both collide and thus both packets will be destroyed.
Since they are destroyed the two senders will need to send again the same packets.
I understand this simple concept, the difficulty is, how to modulate this using probabilities.
Now, I need to find out the throughput which is the rate of (successful) transmission of frames.
Before going any further, we have to make some assumptions:
All frames have the same length.
Stations cannot generate a frame while transmitting or trying to transmit. (That is, if a station keeps trying to send a frame, it cannot be allowed to generate more frames to send.)
The population of stations attempts to transmit (both new frames and old frames that collided) according to a Poisson distribution.
I can't really understand the third assumption, how will I apply this probability in aloha?
I can't find a single code online in order to get an idea how this would be done...
here is some further information on this protocol:
http://en.wikipedia.org/wiki/ALOHAnet#Pure_ALOHA
I think you have to split the time into intervals; in each interval a certain number of stations attempts to transmit. This number is the number of events occurring in a fixed interval or time according to http://en.wikipedia.org/wiki/Poisson_distribution.
You have to model it according to the Poisson distribution.
My 2 cents, hope this helps