I have managed to send audio from a microphone using the code found here.
However I have not been able to do this using NAudio.
The code from CodeProject has explicit code to encode and decode such as:
G711.Encode_aLaw
G711.Decode_uLaw
to translate and return bytes to send across the network.
Is it possible to get some sample code for NAudio for the CodeProject application above?
Here's a quick C# Console App that I wrote using NAudio, microphone input, speaker output, with u-Law or A-Law encoding. The NAudio.Codecs namespace contains A-Law and u-Law encoders and decoders.
This program does not send data across the network (it's not hard to do, I just didn't feel like doing it here). I'll leave that to you. Instead, it contains a "Sender" thread and a "Receiver" thread.
The microphone DataAvailable event handler just drops the byte buffer into a queue (it makes a copy of the buffer - you don't want to hold on to the actual buffer from the event). The "Sender" thread grabs the queued buffers, converts the PCM data to g.711 and drops it into a second queue. This "drops into a second queue" part is where you'd send to a remote UDP destination for your particular app.
The "Receiver" thread reads the data from the second queue, converts it back to PCM, and feeds it to a BufferedWaveProvider that's being used by the WaveOut (speaker) device. You would replace this input with a UDP socket receive for your networked application.
Note that the program guarantees that the PCM input and output (microphone and speaker) are using the same WaveFormat. That's something that you'd also have to do for networked endpoints.
Anyway, it works. So here's the code. I won't go into too much detail. There are lots of comments to try to help understand what's going on:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using NAudio.Wave;
using NAudio.Codecs;
namespace G711MicStream
{
class Program
{
delegate byte EncoderMethod( short _raw );
delegate short DecoderMethod( byte _encoded );
// Change these to their ALaw equivalent if you want.
static EncoderMethod Encoder = MuLawEncoder.LinearToMuLawSample;
static DecoderMethod Decoder = MuLawDecoder.MuLawToLinearSample;
static void Main(string[] args)
{
// Fire off our Sender thread.
Thread sender = new Thread(new ThreadStart(Sender));
sender.Start();
// And receiver...
Thread receiver = new Thread(new ThreadStart(Receiver));
receiver.Start();
// We're going to try for 16-bit PCM, 8KHz sampling, 1 channel.
// This should align nicely with u-law
CommonFormat = new WaveFormat(16000, 16, 1);
// Prep the input.
IWaveIn wavein = new WaveInEvent();
wavein.WaveFormat = CommonFormat;
wavein.DataAvailable += new EventHandler<WaveInEventArgs>(wavein_DataAvailable);
wavein.StartRecording();
// Prep the output. The Provider gets the same formatting.
WaveOut waveout = new WaveOut();
OutProvider = new BufferedWaveProvider(CommonFormat);
waveout.Init(OutProvider);
waveout.Play();
// Now we can just run until the user hits the <X> button.
Console.WriteLine("Running g.711 audio test. Hit <X> to quit.");
for( ; ; )
{
Thread.Sleep(100);
if( !Console.KeyAvailable ) continue;
ConsoleKeyInfo info = Console.ReadKey(false);
if( (info.Modifiers & ConsoleModifiers.Alt) != 0 ) continue;
if( (info.Modifiers & ConsoleModifiers.Control) != 0 ) continue;
// Quit looping on non-Alt, non-Ctrl X
if( info.Key == ConsoleKey.X ) break;
}
Console.WriteLine("Stopping...");
// Shut down the mic and kick the thread semaphore (without putting
// anything in the queue). This will (eventually) stop the thread
// (which also signals the receiver thread to stop).
wavein.StopRecording();
try{ wavein.Dispose(); } catch(Exception){}
SenderKick.Release();
// Wait for both threads to exit.
sender.Join();
receiver.Join();
// And close down the output.
waveout.Stop();
try{ waveout.Dispose(); } catch(Exception) {}
// Sleep a little. This seems to be accepted practice when shutting
// down these audio components.
Thread.Sleep(500);
}
/// <summary>
/// Grabs the mic data and just queues it up for the Sender.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
static void wavein_DataAvailable(object sender, WaveInEventArgs e)
{
// Create a local copy buffer.
byte [] buffer = new byte [e.BytesRecorded];
System.Buffer.BlockCopy(e.Buffer, 0, buffer, 0, e.BytesRecorded);
// Drop it into the queue. We'll need to lock for this.
Lock.WaitOne();
SenderQueue.AddLast(buffer);
Lock.ReleaseMutex();
// and kick the thread.
SenderKick.Release();
}
static
void
Sender()
{
// Holds the data from the DataAvailable event.
byte [] qbuffer = null;
for( ; ; )
{
// Wait for a 'kick'...
SenderKick.WaitOne();
// Lock...
Lock.WaitOne();
bool dataavailable = ( SenderQueue.Count != 0 );
if( dataavailable )
{
qbuffer = SenderQueue.First.Value;
SenderQueue.RemoveFirst();
}
Lock.ReleaseMutex();
// If the queue was empty on a kick, then that's our signal to
// exit.
if( !dataavailable ) break;
// Convert each 16-bit PCM sample to its 1-byte u-law equivalent.
int numsamples = qbuffer.Length / sizeof(short);
byte [] g711buff = new byte [numsamples];
// I like unsafe for this kind of stuff!
unsafe
{
fixed( byte * inbytes = &qbuffer[0] )
fixed( byte * outbytes = &g711buff[0] )
{
// Recast input buffer to short[]
short * buff = (short *)inbytes;
// And loop over the samples. Since both input and
// output are 16-bit, we can use the same index.
for( int index = 0; index < numsamples; ++index )
{
outbytes[index] = Encoder(buff[index]);
}
}
}
// This gets passed off to the reciver. We'll queue it for now.
Lock.WaitOne();
ReceiverQueue.AddLast(g711buff);
Lock.ReleaseMutex();
ReceiverKick.Release();
}
// Log it. We'll also kick the receiver (with no queue addition)
// to force it to exit.
Console.WriteLine("Sender: Exiting.");
ReceiverKick.Release();
}
static
void
Receiver()
{
byte [] qbuffer = null;
for( ; ; )
{
// Wait for a 'kick'...
ReceiverKick.WaitOne();
// Lock...
Lock.WaitOne();
bool dataavailable = ( ReceiverQueue.Count != 0 );
if( dataavailable )
{
qbuffer = ReceiverQueue.First.Value;
ReceiverQueue.RemoveFirst();
}
Lock.ReleaseMutex();
// Exit on kick with no data.
if( !dataavailable ) break;
// As above, but we convert in reverse, from 1-byte u-law
// samples to 2-byte PCM samples.
int numsamples = qbuffer.Length;
byte [] outbuff = new byte [qbuffer.Length * 2];
unsafe
{
fixed( byte * inbytes = &qbuffer[0] )
fixed( byte * outbytes = &outbuff[0] )
{
// Recast the output to short[]
short * outpcm = (short *)outbytes;
// And loop over the u-las samples.
for( int index = 0; index < numsamples; ++index )
{
outpcm[index] = Decoder(inbytes[index]);
}
}
}
// And write the output buffer to the Provider buffer for the
// WaveOut devices.
OutProvider.AddSamples(outbuff, 0, outbuff.Length);
}
Console.Write("Receiver: Exiting.");
}
/// <summary>Lock for the sender queue.</summary>
static Mutex Lock = new Mutex();
static WaveFormat CommonFormat;
/// <summary>"Kick" semaphore for the sender queue.</summary>
static Semaphore SenderKick = new Semaphore(0, int.MaxValue);
/// <summary>Queue of byte buffers from the DataAvailable event.</summary>
static LinkedList<byte []> SenderQueue = new LinkedList<byte[]>();
static Semaphore ReceiverKick = new Semaphore(0, int.MaxValue);
static LinkedList<byte []> ReceiverQueue = new LinkedList<byte[]>();
/// <summary>WaveProvider for the output.</summary>
static BufferedWaveProvider OutProvider;
}
}
Related
I am making a simple game whose audio speed should increase as the player is approaching the end of the level it is playing. So now I was wondering if there was a way to do this using SDL_Mixer. If SDL_Mixer is not the way to go could you please tell me how could I make this change in the audio file itself to make it faster. I am working with a 8-bit .wav file with 2 channels at the samplerate of 22050.
According to this forum here: https://forums.libsdl.org/viewtopic.php?p=44663, you can use a different library called "SoLoud" to change the playback speed of your sounds on the fly. You can get/see more details on SoLoud here: http://sol.gfxile.net/soloud/. From what I can tell, you cannot do this using SDL2, and SoLoud seems easy enough to use, so that would be my suggestion.
A few years back I was trying to achieve something very similar and, after a lot of web search, I came up with this solution, involving using Mix_RegisterEffect function, which got close:
#include <SDL2/SDL.h>
#include <SDL2/SDL_mixer.h>
#include <iostream>
#include <cstdlib>
#include <cmath>
/* global vars */
Uint16 audioFormat; // current audio format constant
int audioFrequency, // frequency rate of the current audio format
audioChannelCount, // number of channels of the current audio format
audioAllocatedMixChannelsCount; // number of mix channels allocated
static inline Uint16 formatSampleSize(Uint16 format)
{
return (format & 0xFF) / 8;
}
// Get chunk time length (in ms) given its size and current audio format
static int computeChunkLengthMillisec(int chunkSize)
{
/* bytes / samplesize == sample points */
const Uint32 points = chunkSize / formatSampleSize(audioFormat);
/* sample points / channels == sample frames */
const Uint32 frames = (points / audioChannelCount);
/* (sample frames * 1000) / frequency == play length, in ms */
return ((frames * 1000) / audioFrequency);
}
// Custom handler object to control which part of the Mix_Chunk's audio data will be played, with which pitch-related modifications.
// This needed to be a template because the actual Mix_Chunk's data format may vary (AUDIO_U8, AUDIO_S16, etc) and the data type varies with it (Uint8, Sint16, etc)
// The AudioFormatType should be the data type that is compatible with the current SDL_mixer-initialized audio format.
template<typename AudioFormatType>
struct PlaybackSpeedEffectHandler
{
const AudioFormatType* const chunkData; // pointer to the chunk sample data (as array)
const float& speedFactor; // the playback speed factor
int position; // current position of the sound, in ms
const int duration; // the duration of the sound, in ms
const int chunkSize; // the size of the sound, as a number of indexes (or sample points). thinks of this as a array size when using the proper array type (instead of just Uint8*).
const bool loop; // flags whether playback should stay looping
const bool attemptSelfHalting; // flags whether playback should be halted by this callback when playback is finished
bool altered; // true if this playback has been pitched by this handler
PlaybackSpeedEffectHandler(const Mix_Chunk& chunk, const float& speed, bool loop, bool trySelfHalt)
: chunkData(reinterpret_cast<AudioFormatType*>(chunk.abuf)), speedFactor(speed),
position(0), duration(computeChunkLengthMillisec(chunk.alen)),
chunkSize(chunk.alen / formatSampleSize(audioFormat)),
loop(loop), attemptSelfHalting(trySelfHalt), altered(false)
{}
// processing function to be able to change chunk speed/pitch.
void modifyStreamPlaybackSpeed(int mixChannel, void* stream, int length)
{
AudioFormatType* buffer = static_cast<AudioFormatType*>(stream);
const int bufferSize = length / sizeof(AudioFormatType); // buffer size (as array)
const int bufferDuration = computeChunkLengthMillisec(length); // buffer time duration
const float speedFactor = this->speedFactor; // take a "snapshot" of speed factor
// if there is still sound to be played
if(position < duration || loop)
{
// if playback is unaltered and pitch is required (for the first time)
if(!altered && speedFactor != 1.0f)
altered = true; // flags playback modification and proceed to the pitch routine.
if(altered) // if unaltered, this pitch routine is skipped
{
const float delta = 1000.0/audioFrequency, // normal duration of each sample
vdelta = delta*speedFactor; // virtual stretched duration, scaled by 'speedFactor'
for(int i = 0; i < bufferSize; i += audioChannelCount)
{
const int j = i/audioChannelCount; // j goes from 0 to size/channelCount, incremented 1 by 1
const float x = position + j*vdelta; // get "virtual" index. its corresponding value will be interpolated.
const int k = floor(x / delta); // get left index to interpolate from original chunk data (right index will be this plus 1)
const float proportion = (x / delta) - k; // get the proportion of the right value (left will be 1.0 minus this)
// usually just 2 channels: 0 (left) and 1 (right), but who knows...
for(int c = 0; c < audioChannelCount; c++)
{
// check if k will be within bounds
if(k*audioChannelCount + audioChannelCount - 1 < chunkSize || loop)
{
AudioFormatType leftValue = chunkData[( k * audioChannelCount + c) % chunkSize],
rightValue = chunkData[((k+1) * audioChannelCount + c) % chunkSize];
// put interpolated value on 'data' (linear interpolation)
buffer[i + c] = (1-proportion)*leftValue + proportion*rightValue;
}
else // if k will be out of bounds (chunk bounds), it means we already finished; thus, we'll pass silence
{
buffer[i + c] = 0;
}
}
}
}
// update position
position += bufferDuration * speedFactor; // this is not exact since a frame may play less than its duration when finished playing, but its simpler
// reset position if looping
if(loop) while(position > duration)
position -= duration;
}
else // if we already played the whole sound but finished earlier than expected by SDL_mixer (due to faster playback speed)
{
// set silence on the buffer since Mix_HaltChannel() poops out some of it for a few ms.
for(int i = 0; i < bufferSize; i++)
buffer[i] = 0;
if(attemptSelfHalting)
Mix_HaltChannel(mixChannel); // XXX unsafe call, since it locks audio; but no safer solution was found yet...
}
}
// Mix_EffectFunc_t callback that redirects to handler method (handler passed via userData)
static void mixEffectFuncCallback(int channel, void* stream, int length, void* userData)
{
static_cast<PlaybackSpeedEffectHandler*>(userData)->modifyStreamPlaybackSpeed(channel, stream, length);
}
// Mix_EffectDone_t callback that deletes the handler at the end of the effect usage (handler passed via userData)
static void mixEffectDoneCallback(int, void *userData)
{
delete static_cast<PlaybackSpeedEffectHandler*>(userData);
}
// function to register a handler to this channel for the next playback.
static void registerEffect(int channel, const Mix_Chunk& chunk, const float& speed, bool loop, bool trySelfHalt)
{
Mix_RegisterEffect(channel, mixEffectFuncCallback, mixEffectDoneCallback, new PlaybackSpeedEffectHandler(chunk, speed, loop, trySelfHalt));
}
};
// Register playback speed effect handler according to the current audio format; effect valid for a single playback; if playback is looped, lasts until it's halted
void setupPlaybackSpeedEffect(const Mix_Chunk* const chunk, const float& speed, int channel, bool loop=false, bool trySelfHalt=false)
{
// select the register function for the current audio format and register the effect using the compatible handlers
// XXX is it correct to behave the same way to all S16 and U16 formats? Should we create case statements for AUDIO_S16SYS, AUDIO_S16LSB, AUDIO_S16MSB, etc, individually?
switch(audioFormat)
{
case AUDIO_U8: PlaybackSpeedEffectHandler<Uint8 >::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
case AUDIO_S8: PlaybackSpeedEffectHandler<Sint8 >::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
case AUDIO_U16: PlaybackSpeedEffectHandler<Uint16>::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
default:
case AUDIO_S16: PlaybackSpeedEffectHandler<Sint16>::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
case AUDIO_S32: PlaybackSpeedEffectHandler<Sint32>::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
case AUDIO_F32: PlaybackSpeedEffectHandler<float >::registerEffect(channel, *chunk, speed, loop, trySelfHalt); break;
}
}
// example
// run the executable passing an filename of a sound file that SDL_mixer is able to open (ogg, wav, ...)
int main(int argc, char** argv)
{
if(argc < 2) { std::cout << "missing argument" << std::endl; return 0; }
SDL_Init(SDL_INIT_AUDIO);
Mix_OpenAudio(MIX_DEFAULT_FREQUENCY, MIX_DEFAULT_FORMAT, MIX_DEFAULT_CHANNELS, 4096);
Mix_QuerySpec(&audioFrequency, &audioFormat, &audioChannelCount); // query specs
audioAllocatedMixChannelsCount = Mix_AllocateChannels(MIX_CHANNELS);
float speed = 1.0;
Mix_Chunk* chunk = Mix_LoadWAV(argv[1]);
if(chunk != NULL)
{
const int channel = Mix_PlayChannelTimed(-1, chunk, -1, 8000);
setupPlaybackSpeedEffect(chunk, speed, channel, true);
// loop for 8 seconds, changing the pitch dynamically
while(SDL_GetTicks() < 8000)
speed = 1 + 0.25*sin(0.001*SDL_GetTicks());
}
else
std::cout << "no data" << std::endl;
Mix_FreeChunk(chunk);
Mix_CloseAudio();
Mix_Quit();
SDL_Quit();
return EXIT_SUCCESS;
}
While this works, it's not a perfect solution, since the result has some artifacts (crackling) in most cases, which I wasn't able to figure out why.
Github gist I created for this a while ago.
I am using ESP8266 and ModbusMaster.h library to communicate with RS485 enabled power meter. Communication works fine but responses are the ones are confusing me and I can not get correct values. My power meter shows 1.49 kWh but response from Modbus is 16318. Here is my code:
#include <ArduinoOTA.h>
#include <BlynkSimpleEsp8266.h>
#include <SimpleTimer.h>
#include <ModbusMaster.h>
#include <ESP8266WiFi.h>
/*
Debug. Change to 0 when you are finished debugging.
*/
const int debug = 1;
#define ARRAY_SIZE(A) (sizeof(A) / sizeof((A)[0]))
int timerTask1, timerTask2, timerTask3;
float battBhargeCurrent, bvoltage, ctemp, btemp, bremaining, lpower, lcurrent, pvvoltage, pvcurrent, pvpower;
float stats_today_pv_volt_min, stats_today_pv_volt_max;
uint8_t result;
// this is to check if we can write since rs485 is half duplex
bool rs485DataReceived = true;
float data[100];
ModbusMaster node;
SimpleTimer timer;
// tracer requires no handshaking
void preTransmission() {}
void postTransmission() {}
// a list of the regisities to query in order
typedef void (*RegistryList[])();
RegistryList Registries = {
AddressRegistry_0001 // samo potrosnju
};
// keep log of where we are
uint8_t currentRegistryNumber = 0;
// function to switch to next registry
void nextRegistryNumber() {
currentRegistryNumber = (currentRegistryNumber + 1) % ARRAY_SIZE( Registries);
}
void setup()
{
// Serial.begin(115200);
Serial.begin(9600, SERIAL_8E1); //, SERIAL_8E1
// Modbus slave ID 1
node.begin(1, Serial);
node.preTransmission(preTransmission);
node.postTransmission(postTransmission);
// WiFi.mode(WIFI_STA);
while (Blynk.connect() == false) {}
ArduinoOTA.setHostname(OTA_HOSTNAME);
ArduinoOTA.begin();
timerTask1 = timer.setInterval(9000, updateBlynk);
timerTask2 = timer.setInterval(9000, doRegistryNumber);
timerTask3 = timer.setInterval(9000, nextRegistryNumber);
}
// --------------------------------------------------------------------------------
void doRegistryNumber() {
Registries[currentRegistryNumber]();
}
void AddressRegistry_0001() {
uint8_t j;
uint16_t dataval[2];
result = node.readHoldingRegisters(0x00, 2);
if (result == node.ku8MBSuccess)
{
for (j = 0; j < 2; j++) // set to 0,1 for two
datablocks
{
dataval[j] = node.getResponseBuffer(j);
}
terminal.println("---------- Show power---------");
terminal.println("kWh: ");
terminal.println(dataval[0]);
terminal.println("crc: ");
terminal.println(dataval[1]);
terminal.println("-----------------------");
terminal.flush();
node.clearResponseBuffer();
node.clearTransmitBuffer();
} else {
rs485DataReceived = false;
}
}
void loop()
{
Blynk.run();
// ArduinoOTA.handle();
timer.run();
}
I have tried similar thing but with Raspberry Pi and USB-RS485 and it works.
Sample of NodeJS code is below. It looks similar to Arduino code.
// create an empty modbus client
var ModbusRTU = require("modbus-serial");
var client = new ModbusRTU();
// open connection to a serial port
client.connectRTUBuffered("/dev/ttyUSB0", { baudRate: 9600, parity: 'even' }, read);
function write() {
client.setID(1);
// write the values 0, 0xffff to registers starting at address 5
// on device number 1.
client.writeRegisters(5, [0 , 0xffff])
.then(read);
}
function read() {
// read the 2 registers starting at address 5
// on device number 1.
console.log("Ocitavanje registra 0000: ");
client.readHoldingRegisters(0000, 12)
.then(function(d) {
var floatA = d.buffer.readFloatBE(0);
// var floatB = d.buffer.readFloatBE(4);
// var floatC = d.buffer.readFloatBE(8);
// console.log("Receive:", floatA, floatB, floatC); })
console.log("Potrosnja u kWh: ", floatA); })
.catch(function(e) {
console.log(e.message); })
.then(close);
}
function close() {
client.close();
}
This code displays 1.493748298302 in console.
How can I implement this var floatA = d.buffer.readFloatBE(0); in Arduino? Looks like that readFloatBE(0) does the trick, but available only in NodeJS / javascript.
Here i part of datasheet for my device
Here is what I am getting as result from original software that came with device:
If someone could point me in better direction I would be thenkfull.
UPDATE:
I found ShortBus Modbus Scanner software and tested readings.
Library read result as Unsigned integer, but need Floating Point and Word Order swapped. It is shown on image below.
Can someone tell how to set proper conversion please.
Right, so indeed the issue is with the part done by var floatA = d.buffer.readFloatBE(0);Modbus returns an array of bytes, and the client has to interpret those bytes, ideally done by the library you're using, but if not available on Arduino, you may try manually with byte decoding functions, with the following considerattions:
Modbus registers are 16 bit in length, so length 1 = 16 bits, length
2 = 32 bits, hence the data type noted on the docs as float32 means
"2 registers used for this value, interpret as float".
Therefore, on client.readHoldingRegisters(0000, 12)you're asking to read the register with address 00, and size 12... so this makes no sense, you only need 2 registers.
On your sample Node code, first you're writing
2 registers to address 5 in client.writeRegisters(5, [0 , 0xffff])
register 5 = 0, and register 6 = 0xFFFF, why? Then you go and read
from address 0, in read(), which is the address for Total KwH per
your docs.
So, you should get an array of bytes, and you need to
decode them as a float. Modbus is Big Endian for words and bytes, so
you need to use those in the decoding functions. I don't know exactly
what is available in Arduino, but hopefully you can figure it out
with this extra info.
I suppose that if you just send the buffer to print, you'll get an integer interpretation of the value, hence the problem
FIRST see below EDIT part.
I use beaglebone black, ker 3.8, and GCC compiler for a signal processing project.
I receive the raw data from three GPS modules through uart communication asynchronously. So, I used 3 threads which check 3 UARTs (BB-UART1, BB-UART2, BB-UART4) continuously to receive raw data, here called the "reading threads". after each data packet received from each module, I decode the received data packet and extract important data packet.
when, decoding is finished in the threads, I perform the signal processing in a separate thread, called "signal processing thread", using above 3-important decoded data packet.
as it's obvious, I should synchronize the reading threads with signal processing thread. I use pthread_cond_wait and pthread_cond_signal for that.
the code operates fine and synchronization and signal processing are performed efficiently. each data packet received in 0.1 second (10 times in a second).
In the signal processing thread, after signal processing, I send the signal processing result to the user through separate UART, BB-UART5.
when I add this part of the code, "THIS" line, after some time, which all parts or OK and the signal processing results are sent to user, the signal processing thread is frozen and locked in a mutex. In fact the mutex unlocking is not performed in previous step.
I spent many time, some weeks, to find the reason. when I remove the mutex and other tools for threads synchronization (to make the whole code simple to debug) an array of data in somewhere is overflowed and its data are changed to overflowed values. However when I don't add "THIS" line, overflow not occurred any time.
when I remove the "write" function (of BB-UART5) in signal processing thread all operations are OK.
the signal processing thread:
void *signal_processing_thread (void *arg){
int i, j;
char str[512];
printf("000000000000000000000000000000000000000000\r\n");
printf("0-signal_processing_thread is running!\r\n");
printf("000000000000000000000000000000000000000000\r\n");
while(1){
pthread_mutex_lock(&th1); // the code lock here after add "THIS" line
pthread_mutex_lock(&th2);
pthread_mutex_lock(&th3);
if ((!decode_completed[0])|(!decode_completed[1])|(!decode_completed[2])){
pthread_mutex_unlock(&th1);
pthread_mutex_unlock(&th2);
pthread_mutex_unlock(&th3);
continue;
}
// data packets are ready in reading threads
// signal processing start
// signal processing done
// send the results
sprintf (str,"some string\r\n\0",some variables);
printf (str);
for (i=0;i<256;i++)
if (str[i]==0)
break;
write (uart5_id, str, i); // "THIS" line
decode_completed [0] = 0;
decode_completed [1] = 0;
decode_completed [2] = 0;
pthread_cond_signal(&cv1);
pthread_mutex_unlock(&th1);
pthread_cond_signal(&cv2);
pthread_mutex_unlock(&th2);
pthread_cond_signal(&cv3);
pthread_mutex_unlock(&th3);
}
printf("signal processing thread is closed!\r\n");
}
the reading threads:
void *getdecodedata1_thread (void *arg){
int ret, count, count_decode=0;
char buffer[2024], buffer_decode[2024];
int i;
printf("1-getdecodedata_thread is running!\r\n");
count = 0;
pthread_mutex_lock(&th1);
while(1){
for (i=0;i<500;i++){
ret = read(uart1_id, buffer+count ,255);
if (ret<1)
continue;
// data received
count += ret;
if (count>1000) break;
}
if (count>0){ // packet received
for (i=0;i<count;i++)
buffer_decode1[count_decode3+i]=buffer[i];
count_decode1 += count;
if (count_decode1>15){
// decode
// ....
// decode done
}
count = 0;
}
if (decode is completed) {
decode_completed [0] = 1; // newdata
printf("WAIT_1\n");
pthread_cond_wait(&cv1,&th1);
pthread_mutex_unlock(&th1);
pthread_mutex_lock(&th1);
printf("RELEASE_1\n");
}
}
}
void *getdecodedata2_thread (void *arg){
int ret, count, count_decode=0;
char buffer[2024], buffer_decode[2024];
int i;
printf("2-getdecodedata_thread is running!\r\n");
count = 0;
pthread_mutex_lock(&th2);
while(1){
for (i=0;i<500;i++){
ret = read(uart2_id, buffer+count ,255);
if (ret<1)
continue;
// data received
count += ret;
if (count>1000) break;
}
if (count>0){ // packet received
for (i=0;i<count;i++)
buffer_decode3[count_decode2+i]=buffer[i];
count_decode2 += count;
if (count_decode2>15){
// decode
// ....
// decode done
}
count = 0;
}
if (decode is completed) {
decode_completed [1] = 1; // newdata
printf("WAIT_2\n");
pthread_cond_wait(&cv2,&th2);
pthread_mutex_unlock(&th2);
pthread_mutex_lock(&th2);
printf("RELEASE_2\n");
}
}
}
void *getdecodedata3_thread (void *arg){
int ret, count, count_decode=0;
char buffer[2024], buffer_decode[2024];
int i;
printf("3-getdecodedata_thread is running!\r\n");
count = 0;
pthread_mutex_lock(&th3);
while(1){
for (i=0;i<500;i++){
ret = read(uart4_id, buffer+count ,255);
if (ret<1)
continue;
// data received
count += ret;
if (count>1000) break;
}
if (count>0){ // packet received
for (i=0;i<count;i++)
buffer_decode3[count_decode3+i]=buffer[i];
count_decode3 += count;
if (count_decode3>15){
// decode
// ....
// decode done
}
count = 0;
}
if (decode is completed) {
decode_completed [2] = 1; // newdata
printf("WAIT_3\n");
pthread_cond_wait(&cv3,&th3);
pthread_mutex_unlock(&th3);
pthread_mutex_lock(&th3);
printf("RELEASE_3\n");
}
}
}
EDIT: I've found that the problem is in another place of the code. in some where I use "write" function to send some bytes to UART5 and in a separate thread I read simultaneously (non-blocking) UART5 to receive commands. I think a problem like "SegFault" is occurred and the above problem is seen. when I comment the "read" function of UART5, all things is correct and mutexes work finely. How can I use the UART5 to read and write simultaneously?
after about 2 months, I find that the problem is originated from an invalid event in an IC. I use a ttl to RS485 converter and its supply bus has sum noise and distortion which leads to send some invalid characters to serial input. In-fact when I send some characters using serial output, I think probably, serial input receive some invalid data. so it freezes the code in one of critical condition mechanisms. when I make the IC separate from the BBB, the problem is fixed.
I don't Know why? and How? How can a invalid data on serial input can make critical conditions lock? I use ttyO[] files to read and write the serial port.
At a highlevel, my goal is take the microphone input from one stream, do some processing on it, and copy that to the microphone input to another stream. With the latter being my default device, my other applications (which for reasons that are out of my hands, can't specify any other device) can record from the default device and still get processed input.
Here's a snippet of my code:
int stream1_callback(const void *card_capture, void *card_playback, unsigned long frameCount, const PaStreamCallbackTimeInfo *timeInfo, PaStreamCallbackFlags statusFlags, void *userData) {
main_t *data = (main_t *) userData;
// UNUSED(card_capture);
UNUSED(card_playback);
UNUSED(frameCount);
UNUSED(timeInfo);
UNUSED(statusFlags);
// UNUSED(userData);
deinterleave_i16_i16(card_capture, data->mic_unprocessed, NUM_MICS, BLOCKSIZE_48KHZ);
printf("De-interleaved!\n");
process_microphones(data->mic_unprocessed, data->mic_processed);
printf("Processed!\n");
return 0;
}
int stream2_callback(const void *inputBuffer, void *outputBuffer, unsigned long frameCount, const PaStreamCallbackTimeInfo *timeInfo, PaStreamCallbackFlags statusFlags, void *userData) {
main_t *data = (main_t *) userData;
// UNUSED(inputBuffer);
UNUSED(outputBuffer);
UNUSED(frameCount);
UNUSED(timeInfo);
UNUSED(statusFlags);
// UNUSED(userData);
interleave_i16_i16(data->mic_processed, (int16 *)inputBuffer, 1, BLOCKSIZE_16KHZ);
printf("Interleaved!\n");
return 0;
}
int main() {
// ...
/* -- setup stream1 -- */
err = Pa_OpenStream(&stream1, &stream1InputParams, NULL, 48000, BLOCKSIZE_48KHZ, paNoFlag, stream1_callback, &main_data);
if (err != paNoError) {
goto error;
}
/* -- setup stream2 -- */
err = Pa_OpenDefaultStream(&stream2, 1, 0, paInt16, 16000, BLOCKSIZE_16KHZ, stream2_callback, &main_data);
if (err != paNoError) {
goto error;
}
//...
}
So I'm wondering if the callback's input buffer is actually writable. Or if there is some other (better) way that I can write to the input of another device.
In general, capture devices cannot be written to; they go directly to the hardware.
To be able to inject your own samples data into a capture device, its driver must be written to allow that. This is the case for the loopback driver, snd-aloop.
hope you can help me :)
I'm trying to get audio data from a multichannel ASIO device with PortAudio library. Everything is ok: I managed to set the default host API as ASIO and I also managed to select 4 specific channels as inputs. Then,I get an interleaved audio stream which sounds correctly but i would like to get each channel data separately.
PortAudio allows to do a non-interleaved recording, but I don't know how to write or modify my RecordCallBack and the multibuffer pointer (one buffer per channel). Sure I've tried... :(
It would be of massive help to me if someone knows how to deal with this issue.
The original RecordCallBack function is taken from a well known stereo example (slightly modified to manage 4 channles instead of 2) but it manages a single interleaved buffer:
static int recordCallback( const void *inputBuffer, void *outputBuffer,
unsigned long framesPerBuffer,
const PaStreamCallbackTimeInfo* timeInfo,
PaStreamCallbackFlags statusFlags,
void *userData )
{
paTestData *data = (paTestData*)userData;
const short *rptr = (const short*)inputBuffer;
short *wptr = &data->recordedSamples[data->frameIndex * NUM_CHANNELS_I];
long framesToCalc;
long i;
int finished;
unsigned long framesLeft = data->maxFrameIndex - data->frameIndex;
(void) outputBuffer; /* Prevent unused variable warnings. */
(void) timeInfo;
(void) statusFlags;
(void) userData;
if( framesLeft < framesPerBuffer )
{
framesToCalc = framesLeft;
finished = paComplete;
}
else
{
framesToCalc = framesPerBuffer;
finished = paContinue;
}
if( inputBuffer == NULL )
{
for( i=0; i<framesToCalc; i++ )
{
*wptr++ = SAMPLE_SILENCE; /* ch1*/
if( NUM_CHANNELS_I == 4 ){
*wptr++ = SAMPLE_SILENCE;/* ch2*/
*wptr++ = SAMPLE_SILENCE;/* ch3*/
*wptr++ = SAMPLE_SILENCE;} /* ch4*/
}
}
else
{
for( i=0; i<framesToCalc; i++ )
{
*wptr++ = *rptr++; /* ch1*/
if( NUM_CHANNELS_I == 4 ){
*wptr++ = *rptr++;/* ch2*/
*wptr++ = *rptr++;/* ch3*/
*wptr++ = *rptr++;} /* ch4*/
}
}
data->frameIndex += framesToCalc;
return finished;
}
The *inputbuffer pointer is declared as:
PaStream* stream;
And the Open_Stream function is called:
err = Pa_OpenStream(
&stream,
NULL, /* no input */
&outputParameters,
SAMPLE_RATE,
FRAMES_PER_BUFFER,
paClipOff, /* we won't output out of range samples so don't bother clipping them */
playCallback,
&data );
Interleaved just means the bytes for each channel follow after each other as in :
aabbccddeeaabbccddeeaabbccddee (each character represents one byte)
where this input buffer contains two bytes (16 bits) per each of the 5 channels : a, b, c, d & e as it makes 3 repeats across the set of channels which equates to 3 samples per channel ... so knowing input is interleaved, it could be extracted into separate output channel buffers one per channel, but in your code you have just a single output buffer which as you say is due to the necessary callback signature ... one approach would be to write each output channel into the single output buffer separated by distinct offsets per channel so output would be
aaaaaabbbbbbccccccddddddeeeeee
then outside the callback extract out each channel also using same offset per channel
First you need to obtain size of the given output buffer, say X, number of channels, Y, and number of bytes per channel per sample, Z. So global channel offset would be
size_offset = X / (Y * Z) # assure this is an integer
# if its a fraction then error in assumptions
so when addressing output buffer both inside callback and outside we use this offset and knowledge of which channel we are on, W (values 0, 1, 2, 3, ...), and which sample K :
index_output_buffer = K + (W * size_offset) # 1st byte of sample pair
now use index_output_buffer ... then calculate follow-on index :
index_output_buffer = K + (W * size_offset) + 1 # 2nd byte of sample pair
and use it ... you could put above two commands for a given sample into a loop using Z to control number of iterations if Z were to vary but above assumes samples are two bytes
Thank's Scott for your help. The solution was right in front of my eyes and I finally didn't have to work with samples offset. I didn't give you enough information about the code, so your approach was excellent, but the code itself provides an easier way to do that:
The data is storaged in an structrue:
typedef struct
{
int frameIndex; /* Index into sample array. */
int maxFrameIndex;
short *recordedSamples;
}
paTestData;
I modified it to:
typedef struct
{
int frameIndex; /* Index into sample array. */
int maxFrameIndex;
short *recordedSamples;
short * recordedSamples2; //ch2
short * recordedSamples3; //ch3
short *recordedSamples4; //ch4
}
paTestData;
Then I just had to allocate this variables in memory and modified the recordCallback function as follows:
static int recordCallback( const void *inputBuffer, void *outputBuffer,
unsigned long framesPerBuffer,
const PaStreamCallbackTimeInfo* timeInfo,
PaStreamCallbackFlags statusFlags,
void *userData )
{
paTestData *data = (paTestData*)userData;
const short *rptr = (const short*)inputBuffer;
short *wptr = &data->recordedSamples[data->frameIndex];
short *wptr2=&data->recordedSamples2[data->frameIndex];
short *wptr3=&data->recordedSamples3[data->frameIndex];
short *wptr4=&data->recordedSamples4[data->frameIndex];
long framesToCalc;
long i;
int finished;
unsigned long framesLeft = data->maxFrameIndex - data->frameIndex;
(void) outputBuffer; /* Prevent unused variable warnings. */
(void) timeInfo;
(void) statusFlags;
(void) userData;
if( framesLeft < framesPerBuffer )
{
framesToCalc = framesLeft;
finished = paComplete;
}
else
{
framesToCalc = framesPerBuffer;
finished = paContinue;
}
if( inputBuffer == NULL )
{
for( i=0; i<framesToCalc; i++ )
{
*wptr++ = SAMPLE_SILENCE; //ch1
if( NUM_CHANNELS_I == 4 ){
*wptr2++ = SAMPLE_SILENCE;//ch2
*wptr3 ++= SAMPLE_SILENCE;//ch3
*wptr4++ = SAMPLE_SILENCE;} //ch4
}
}
else
{
for( i=0; i<framesToCalc; i++ )
{
*wptr++ = *rptr++; //ch1
if( NUM_CHANNELS_I == 4 ){
*wptr2++ = *rptr++;//ch2
*wptr3++ = *rptr++;//ch3
*wptr4 ++= *rptr++;} //ch4
}
}
data->frameIndex += framesToCalc;
return finished;
}
Hope this can help other people. And thank's again, Scott