I use knowles sph0645lm4h-b microphone to acquire data, which is a 24-bits PCM format with 18 data presicion. Then the 24-bits PCM data is truncated to 18-bits data, because the last 6 bits is alway 0 according to the specification. After that, the 18-bits data is stored as a 32-bits unsigned integer. When the MSB bit is 0, which means it's a positive integer, and the MSB is 0, which means it's a negative integer.
After that, i find all data is positive, no matter which sound i used to test. I tested it with a dual frequency, and do a FFT, then I found the result is almost right except the lower frequency about 0-100Hz is larger. But i reconstructed the sound with the data, which i used for FFT algorithm. The reconstructed sound is almost right but with noise.
I use a buffer to store the microphone data, which is transmitted using DMA. The buffer is
uint16_t fft_buffer[FFT_LENGTH*4]
The DMA configuration is doing as following:
DMA_InitStructure.DMA_Channel = DMA_Channel_0;
DMA_InitStructure.DMA_PeripheralBaseAddr = (uint32_t)&(SPI2->DR);
DMA_InitStructure.DMA_Memory0BaseAddr = (uint32_t)fft_buffer;
DMA_InitStructure.DMA_DIR = DMA_DIR_PeripheralToMemory;
DMA_InitStructure.DMA_PeripheralInc = DMA_PeripheralInc_Disable;
DMA_InitStructure.DMA_MemoryInc = DMA_MemoryInc_Enable;
DMA_InitStructure.DMA_PeripheralDataSize =DMA_PeripheralDataSize_HalfWord;
DMA_InitStructure.DMA_MemoryDataSize = DMA_MemoryDataSize_HalfWord;
DMA_InitStructure.DMA_BufferSize = FFT_LENGTH*4;
DMA_InitStructure.DMA_Mode = DMA_Mode_Normal;
DMA_InitStructure.DMA_Priority = DMA_Priority_VeryHigh;
DMA_InitStructure.DMA_FIFOMode = DMA_FIFOMode_Disable;
DMA_InitStructure.DMA_FIFOThreshold = DMA_FIFOThreshold_Full;
DMA_InitStructure.DMA_MemoryBurst = DMA_MemoryBurst_Single;
DMA_InitStructure.DMA_PeripheralBurst = DMA_PeripheralBurst_Single;
extract data from buffer, truncate to 18 bits and extends it to 32 bits and the store at fft_integer:
int32_t fft_integer[FFT_LENGTH];
fft_buffer stores the original data from one channel and redundant data from other channel. Original data is store at two element of array, like fft_buffer[4] and fft_buffer[5], which are both 16 bits. And fft_integer store just data from one channel and each data take a 32bits.This is why the size of fft_buffer Array is [FFT_LENGTH*4]. 2 elements are used for data from one channel and 2 element is used for the other channel. But for fft_integer, the size of fft_integer array is FFT_LENGTH. Because data from one channel is stored and 18bits can be stored in one element of type int32_t.
for (t=0;t<FFT_LENGTH*4;t=t+4){
uint8_t first_8_bits, second_8_bits, last_2_bits;
uint32_t store_int;
/* get the first 8 bits, middle 8 bits and last 2 bits, combine it to a new value */
first_8_bits = fft_buffer[t]>>8;
second_8_bits = fft_buffer[t]&0xFF;
last_2_bits = (fft_buffer[t+1]>>8)>>6;
store_int = ((first_8_bits <<10)+(second_8_bits <<2)+last_2_bits);
/* convert it to signed integer number according to the MSB of value
* if MSB is 1, then set all the bits before MSB to 1
*/
const uint8_t negative = ((store_int & (1 << 17)) != 0);
int32_t nativeInt;
if (negative)
nativeInt = store_int | ~((1 << 18) - 1);
else
nativeInt = store_int;
fft_integer[cnt] = nativeInt;
cnt++;
}
The microphone is using I2S Interface and it's a single mono microphone, which means that there is just half of the data is effective at half of the transmission time. It works for about 128ms, and then will stop working.
This picture shows the data, which i convert to a integer.
My question is why there is are large components of lower frequency although it can reconstruct the similar sound. I'm sure there is no problem in Hardware configuration.
I have done a experiment to see which original data is stored in buffer. I have done the following test:
uint8_t a, b, c, d
for (t=0;t<FFT_LENGTH*4;t=t+4){
a = (fft_buffer[t]&0xFF00)>>8;
b = fft_buffer[t]&0x00FF;
c = (fft_buffer[t+1]&0xFF00)>>8;
/* set the tri-state to 0 */
d = fft_buffer[t+1]&0x0000;
printf("%.2x",a);
printf("%.2x",b);
printf("%.2x",c);
printf("%.2x\n",d);
}
The PCM data is shown like following:
0ec40000
0ec48000
0ec50000
0ec60000
0ec60000
0ec5c000
...
0cf28000
0cf20000
0cf10000
0cf04000
0cef8000
0cef0000
0cedc000
0ced4000
0cee4000
0ced8000
0cec4000
0cebc000
0ceb4000
....
0b554000
0b548000
0b538000
0b53c000
0b524000
0b50c000
0b50c000
...
Raw data in Memory:
c4 0e ff 00
c5 0e ff 40
...
52 0b ff c0
50 0b ff c0
I use it as little endian.
The large low-frequency component starting from DC in the original data is due to the large DC offset caused by incorrectly translating the 24 bit two's complement samples to int32_t. DC offset is inaudible unless it caused clipping or arithmetic overflow to occur. There are not really any low frequencies up to 100Hz, that is merely an artefact of the FFT's response to the strong DC (0Hz) element. That is why you cannot hear any low frequencies.
Below I have stated a number of assumptions as clearly as possible so that the answer may perhaps be adapted to match the actualité.
Given:
Raw data in Memory:
c4 0e ff 00
c5 0e ff 40
...
52 0b ff c0
50 0b ff c0
I use it as little endian.
and
2 elements are used for data from one channel and 2 element is used for the other channel
and given the subsequent comment:
fft_buffer[0] stores the higher 16 bits, fft_buffer[1] stores the lower 16 bits
Then the data is in fact cross-endian such that for example, for:
c4 0e ff 00
then
fft_buffer[n] = 0x0ec4 ;
fft_buffer[n+1] = 0x00ff ;
and the reconstructed sample should be:
0x00ff0ec4
then the translation is a matter of reinterpreting fft_buffer as a 32 bit array, swapping the 16 bit word order, then a shift to move the sign-bit to the int32_t sign-bit position and (optionally) a re-scale, e.g.:
c4 0e ff 00 => 0x00ff0ec4
0x00ff0ec4<< 8 = 0xff0ec400
0xff0ec400/ 16384 = 0xffff0ec4(-61756)
thus:
// Reinterpret DMA buffer as 32bit samples
int32_t* fft_buffer32 = (int32_t*)fft_buffer ;
// For each even numbered DMA buffer sample...
for( t = 0; t < FFT_LENGTH * 2; t += 2 )
{
// ... swap 16 bit word order
int32_t sample = fft_buffer32 [t] << 16 |
fft_buffer32 [t] >> 16 ;
// ... from 24 to 32 bit 2's complement and rescale to
// maintain original magnitude. Copy to single channel
// fft_integer array.
fft_integer[t / 2] = (sample << 8) / 16384 ;
}
Related
I'm trying to convert a 24 bit usb audio stream into a 32 bit stream so my microcontroller's peripherals can play happily with the stream (it can only handle 16 or 32 bit data like most mcus...).
The following code is what I got from the mcu's company... didn't work as expected and I ended up getting really distorted audio.
// Function takes usb stream and processes the data for our peripherals
// #data - usb stream data
// #byte_count - size of stream
void process_usb_stream(uint8_t *data, uint16_t byte_count) {
// Etc code that gets buffers ready to read the stream...
// Conversion here!
int32_t *buffer;
int sample_count = 0;
for (int i = 0; i < byte_count; i += 3) {
buffer[sample_count++] = data[i] | data[i+1] << 8 | data[i+2] << 16;
}
// Send buffer to peripherals for them to use...
}
Any help with converting the data from a 24 bit stream to 32 bit stream would be super awesome! This area of work is very hard for me :(
data[...] is a uint8_t. You need to cast that before shifting, because data[...]<<8 and data[...]<<16 are undefined. They'll either be 0 or unchanged, neither of which is what you want.
Also, you need to shift by another 8 bits to get the full range and put the sign bit in the right place.
Also, you're treating the data as if it were in little-endian format. Make sure it is. I'll assume that's correct, so something like this works:
int32_t *buffer;
int sample_count = 0;
for (int i = 0; i+3 <= byte_count; ) {
int32_t v = ((int32_t)data[i++])<<8;
v |= ((int32_t)data[i++])<<16;
v |= ((int32_t)data[i++])<<24;
buffer[sample_count++] = v;
}
Finally, note that this assumes that byte_count is divisible by 3 -- make sure that's true!
this is DSP stuff if, also post this question on http://dsp.stackexchange.com
In DSP the process of changing the bit depth is called scaling
16 bit resolution has 65536 values
24 bit resolution has 16777216
possible values
32 bit has 4294967296 values so the factor is 256
According to https://electronics.stackexchange.com/questions/229268/what-is-name-of-process-used-to-change-sample-bit-depth/229271
reduction from 24 bit to 16 bit is called scaling down and is done by dividing each value by 256.
This can be done by bitwise shifting every bit by 8
y = x >> 8. When scaling down this way the LSB is lost
Scaling up to 32 bit is more complicated and there are several approaches how to do this. It may work by multiplying each bit of the value with a value between 2⁰ and 2⁸.
Push the 24 bit value in a 32 bit register and then left-shifting each bit by a value between 2⁰ and 2⁸:
data32[31] = data32[23] << 8;
data32[22] = data32[14] << 8;
...
data32[0] = data32[0];
and interpolate the bits you do not get with this (linear interpolation)
Maybe there are much better scaling up algortihms ask on http://dsp.stackexchange.com
See also http://blog.bjornroche.com/2013/05/the-abcs-of-pcm-uncompressed-digital.html for the scaling up problem...
I'm receiving a high volume of "hex array" strings in the form:
'16 03 03 00 50 40 f2 12 71 0b c0 4f 99 dc 87 6f'
What's the most efficient way of copying them into an existing, larger buffer?
I'm guessing the naive way would be:
var lineBuffer = new Buffer(line.replace(/\s+/g, ''), 'hex');
lineBuffer.copyTo(mainBuffer, offset);
offset += 16;
I'm wary of using line[index] and doing the simple bit shift and sum, because string[index] just resolves to another string.
To move this out of the comments,
The solution you're looking for is probably something like this:
const line = '16 03 03 00 50 40 f2 12 71 0b c0 4f 99 dc 87 6f';
const regex = / /g;
const encoding = 'hex';
const replacement = '';
function getHexString(input) {
return input.replace(regex, replacement);
}
const existingBuffer = Buffer.alloc(1024); // just as an example
const offset = existingBuffer.length; // or wherever you need them to go.
function write(buffer, str, pos) {
const hexString = getHexString(str);
buffer.write(hexString, pos, encoding);
}
write(existingBuffer, line, 0);
I can get about 1,600,000 ops/sec through benchmark.js with () => write(existingBuffer, line, 0). Cheating a bit, since I'm constantly writing to position 0 instead of appending, but it should be close enough to get you an idea. The closest I could come with other combinations I was trying was about 1,200,000 ops/sec.
Also as a side note, I would strongly suggest using Buffer.alloc() wherever you are creating your original buffer, if you aren't. You can also use Buffer.allocUnsafe() which is faster, but that may leave non-zero data in the buffer. Which may be okay, if you know you're going to fill the entire buffer with new data before using it (or are only using known-filled slices). More reading here.
Background: I'm using node.js to get the volume setting from a device via serial connection. I need to obtain this data as an integer value.
I have the data in a buffer ('buf'), and am using readInt16BE() to convert to an int, as follows:
console.log( buf )
console.log( buf.readInt16BE(0) )
Which gives me the following output as I adjust the external device:
<Buffer 00 7e>
126
<Buffer 00 7f>
127
<Buffer 01 00>
256
<Buffer 01 01>
257
<Buffer 01 02>
258
Problem: All looks well until we reach 127, then we take a jump to 256. Maybe it's something to do with signed and unsigned integers - I don't know!
Unfortunately I have very limited documentation about the external device, I'm having to reverse engineer it! Is it possible it only sends a 7-bit value? Hopefully there is a way around this?
Regarding a solution - I must also be able to convert back from int to this format!
Question: How can I create a sequential range of integers when 7F seems to be the largest value my device sends, which causes a big jump in my integer scale?
Thanks :)
127 is the maximum value of a signed 8-bit integer. If the integer is overflowing into the next byte at 128 it would be safe to assume you are not being sent a 16 bit value, but rather 2 signed 8-bit values, and reading the value as a 16-bit integer would be incorrect.
I would start by using the first byte as a multiplier of 128 and add the second byte, this will give the series you are seeking.
buf = Buffer([0,127]) //<Buffer 00 7f>
buf.readInt8(0) * 128 + buf.readInt8(1)
>127
buf = Buffer([1,0]) //<Buffer 01 00>
buf.readInt8(0) * 128 + buf.readInt8(1)
>128
buf = Buffer([1,1]) //<Buffer 01 01>
buf.readInt8(0) * 128 + buf.readInt8(1)
>129
The way to get back is to divide by 128, round it down to the nearest integer for the first byte, and the second byte contains the remainder.
i = 129
buf = Buffer([Math.floor(i / 128), i % 128])
<Buffer 01 01>
Needed to treat the data as two signed 8-bit values. As per #forrestj the solution is to do:
valueInt = buf.readInt8(0) * 128 + buf.readInt8(1)
We can also convert the int value into the original format by doing the following:
byte1 = Math.floor(valueInt / 128)
byte2 = valueInt % 128
I have a buffer in node <Buffer 42 d9 00 00> that is supposed to represent the decimal 108.5. I am using this module to try and decode the buffer: https://github.com/feross/ieee754.
ieee754.read = function (buffer, offset, isLE, mLen, nBytes)
The arguments mean the following:
buffer = the buffer
offset = offset into the buffer
value = value to set (only for write)
isLe = is little endian?
mLen = mantissa length
nBytes = number of bytes
I try to read the value: ieee754.read(buffer, 0, false, 5832704, 4) but am not getting the expected result. I think I am calling the function correctly, although I am unsure about the mLen argument.
[I discovered that] the node Buffer class has that ability built in: buffer.readFloatBE(0).
I am studying ASLR randomization of mmap(), on x86 system.
I have read in a lot of places that there are 16bits of randomization on the address loaded with mmap().
But in the source code i have found:
static unsigned long mmap_rnd(void)
02 {
03 unsigned long rnd = 0;
04
05 /*
06 * 8 bits of randomness in 32bit mmaps, 20 address space bits
07 * 28 bits of randomness in 64bit mmaps, 40 address space bits
08 */
09 if (current->flags & PF_RANDOMIZE) {
10 if (mmap_is_ia32())
11 rnd = (long)get_random_int() % (1<<8);
12 else
13 rnd = (long)(get_random_int() % (1<<28));
14 }
15 return rnd << PAGE_SHIFT;
16 }
So, that would be only 8bits of randomness.
But in fact, running some test, i get the following address (stack-heap-mmap)
bf937000,09a60000,b774b000
bfa86000,090ef000,b76e2000
Its more than 16 bits if it can be b77XX000 and b76XX000!!!!
Any help on this?
PAGE_SHIFT is shifting that randomness to a different bit position. The difference between your mmap addresses is indeed:
b774b000
-b76e2000
---------
69000
I don't know what the value of PAGE_SHIFT is, but if it's 12 for example, then you have 0x69 difference which perfectly fits in 8-bits.