I am playing around with some basics of the Audiostream package for Kivy.
I would like to make a simple online input-filter-output system, for example, take in microphone data, impose a band-pass filter, send to speakers.
However, I can't seem to figure out what data format the microphone input is in or how to manipulate it. In code below, buf is type string, but how can I get the data out of it to manipulate it in such a way [i.e. function(buf)] to do something like a band-pass filter?
The code currently functions to just send the microphone input directly to the speakers.
Thanks.
from time import sleep
from audiostream import get_input
from audiostream import get_output, AudioSample
#get speakers, create sample and bind to speakers
stream = get_output(channels=2, rate=22050, buffersize=1024)
sample = AudioSample()
stream.add_sample(sample)
#define what happens on mic input with arg as buffer
def mic_callback(buf):
print 'got', len(buf)
#HERE: How do I manipulate buf?
#modified_buf = function(buf)
#sample.write(modified_buf)
sample.write(buf)
# get the default audio input (mic on most cases)
mic = get_input(callback=mic_callback)
mic.start()
sample.play()
sleep(3) #record for 3 seconds
mic.stop()
sample.stop()
The buffer is composed of bytes that need to be interpreted as signed short. You can use struct or array module to get value. In your example, you have 2 channels (L/R). Let's say you wanna to have the right channel volume down by 20% (aka 80% of the original sound only for right channel)
from array import array
def mic_callback(buf):
# convert our byte buffer into signed short array
values = array("h", buf)
# get right values only
r_values = values[1::2]
# reduce by 20%
r_values = map(lambda x: x * 0.8, r_values)
# you can assign only array for slice, not list
# so we need to convert back list to array
values[1::2] = array("h", r_values)
# convert back the array to a byte buffer for speaker
sample.write(values.tostring())
Related
I'm currently using a keysight VNA product and I control it using PyVisa. Since I have a rapid changing system, I wish to query binary data instead of ascii data from the machine since it is about 10 times faster. The issue I am having is to convert the data to ascii again.
Minimal exampel code:
import pyvisa as visa
import numpy as np
device_adress = ''TCPIP0::localhost::hislip1,4880::INSTR''
rm = visa.ResourceManager('C:\\Windows\\System32\\visa32.dll')
device = rm.open_resource(device_adres)
# presetting device for SNP data measurment
# ...
device.query_ascii_values('CALC:DATA:SNP? 2', container = np.ndarray) # works super but is slow
device.write('FORM:DATA REAL,64')
device.query_binary_values('CALC:DATA:SNP? 2', container = np.ndarray) # 10 times faster but how to read data
Official docs to query binary doesn't give me anything. I found the functions for the code on git here and some helper functions for converting data here, but I am still unable to convert the data such that the converted data is the same as the one I got from the ascii query command. If possible I would like the 'container=np.ndarray' to kept.
Functions from the last link that I have tested:
bin_data = device.query_binary_values('CALC:DATA:SNP? 2', container = np.ndarray)
num = from_binary_block(bin_data) # "Convert a binary block into an iterable of numbers."
ascii_data = to_ascii_block(num) # "Turn an iterable of numbers in an ascii block of data."
but the data from query_ascii_values and the values of ascii_data don't match. Any help is higly appreciated.
Edit:
With the following code
device.write(f"SENS:SWE:POIN 5;")
data_bin = device.query_binary_values('CALC:DATA? SDATA', container=np.ndarray)
I got
data_bin = array([-5.0535379e-34, 1.3452465e-43, -1.7349754e+09, 1.3452465e-43,
-8.6640313e+22, 8.9683102e-44, 5.0314407e-06, 3.1389086e-43,
4.8143607e-36, 3.1389086e-43, -4.1738553e-12, 1.3452465e-43,
-1.5767541e+11, 8.9683102e-44, -2.8241991e+32, 1.7936620e-43,
4.3024710e+16, 1.3452465e-43, 2.1990014e+07, 8.9683102e-44],
dtype=float32)
I use the following code to extract a specific frame from a video file. In this example, I'm simply getting the middle frame:
import cv2
video_path = '/tmp/wonderwall.mp4'
vidcap = cv2.VideoCapture(video_path)
middle_frame = int(vidcap.get(cv2.CAP_PROP_FRAME_COUNT) / 2)
success, image = vidcap.read()
count = 0
success = True
while success:
success, image = vidcap.read()
if count == middle_frame:
temp_file = tempfile.NamedTemporaryFile(suffix='.jpg', delete=False)
cv2.imwrite(temp_file.name, image)
count += 1
However, with this method, capturing the middle frame in a very large file can take a while.
Apparently, in the older cv module, one could do:
import cv
img = cv.QueryFrame(capture)
Is there a similar way in cv2 to grab a specific frame in a video file, without having to iterate through all frames?
You can do it in the same way, in C++ (python conversion should be more than easy).
cv::VideoCapture cap("file.avi");
double number_of_frame = cap.get(CV_CAP_PROP_FRAME_COUNT);
cap.set(CV_CAP_PROP_POS_FRAMES, IndexOfTheFrameYouWant);
cv::Mat frameIwant = cap.read();
For reference :
VideoCapture::get(int propId)
Can take various flag returning nearly all you can wish for (http://docs.opencv.org/2.4/modules/highgui/doc/reading_and_writing_images_and_video.html and look for get() ).
VideoCapture::set(int propId, double value)
Set will do what you want (same doc look for set) if you use the propID 1, and the index of the frame you desire.
You should note that if the index you use as parameter is superior to the max frame that the code will grab the last frame of the video if you are lucky, or crash at run time.
Im trying to load an array at a specific time frame (for example if it has 50 frames or time units then get an array corresponding to the 2nd time frame) from netCDF files (.nc). Im currently using vtkNetCDFCFReader and getting the data array "vwnd" from the 1st time frame like this:
vtkSmartPointer<vtkNetCDFCFReader> reader = vtkSmartPointer<vtkNetCDFCFReader>::New();
reader->SetFileName(path.c_str());
reader->UpdateMetaData();
vtkSmartPointer<vtkStructuredGridGeometryFilter> geometryFilter = vtkSmartPointer<vtkStructuredGridGeometryFilter>::New();
geometryFilter->SetInputConnection(reader->GetOutputPort());
geometryFilter->Update();
vtkSmartPointer<vtkPolyData> ncPolydata = vtkSmartPointer<vtkPolyData>::New();
ncPolydata = geometryFilter->GetOutput();
vtkSmartPointer<vtkDataArray> dataArray = ncPolydata->GetCellData()->GetArray("vwnd");
Variable Arrays are : lat, lon, time, vwnd (vwnd has dimensions (lat,lon)). Im also interested in getting arrays for lat and lon. Any help would be appreciated.
Thanks in advance
As the dimension of lat/lon is different from vwnd, you will need 2 vtknetCDFreaders to read in data with different dimensions. Just remember to set the dimension after creating the reader.
For example in C++:
vtknetCDFReader* reader = vtknetCDFReader::New();
reader->SetFileName(fileName.c_str());
reader->UpdateMetaData();
//here you specify the dimension of the reader
reader->SetDimension(dim);
reader->SetVariableArrayStatus("lat",1)
reader->SetVariableArrayStatus("lon",1)
reader->Update();
If you are doing it correctly, you could read in any arrays and store it into vtkDataArray.
If you want to read in the vwnd data in the second time step, just skip the first lat*lon values.
For BLE, I use BLUEZ5 stack on my Raspberry pi 3 Device,
For running gatt server i used example-gatt-server.py in my board(server) and used mobile for central device(client side)
when i write value from characteristics from my mobile,its received in my device(callback) is in byte array format like "dbus.Array([dbus.Byte(1), dbus.Byte(35)], signature=dbus.Signature('y'))"
i can't decoded this,
How I can extract any useful information from the dbus bluez api that returns a byte array
I use following link for example-gatt-server.py :
https://github.com/RadiusNetworks/bluez/blob/master/test/example-gatt-server
The following python code is callback function that prints the parameters. In my case, each byte of the array represents an ASCII character).
def notification_callback(*args, **kwargs):
"""
Function that will receive DBus notifications signals (properties_changed signals)
when the temperature is updated
"""
#Get the byte array
byte_array = args[1]['Value']
#Convert the byte array into a string
received_value = ''.join(chr(byte) for byte in byte_array)
print ("Received value " + received_value)
This is my second question ever posted on here, so please let me know if I have done something wrong.
I am faced with an interesting issue today. I work for a call center and one of the clients my company verifies information for wants to gather bank account numbers from customers and wants our customer service agents to enter said bank account numbers into the clients external website.
These bank account numbers are not going to be saved anywhere on our local databases, but the audio in which our CSRs are gathering the bank account numbers will be saved in our system. The plain text will not be available but the sound files will. My question is if there is a way to automatically scramble a certain part of a recording automatically using a program? I know this is a serious shot in the dark. Thank you.
While your question does not ask for specific programming related issue, I will try to answer it since i am working on something similar.
Can we automatically scramble a certain part of a recording automatically using a program?
We certainly can. It will depend upon how much complex you would like to make it.
While there are complex method, from a very basic concept perspective, we will need to take the recorded audio file and process in following stages.
Split words in audio file : This will need silence recognition
between the words.
Pass each word through an speech recognition system
Come up with a method to scramble. Do you want to silence ,
jumble , fill it with white noise or encode.
Compare recognized word against the word you want to scramble ,
if there is a match scramble based upon chosen method
Combine (concatenate) all the words in proper sequence and store
it.
I've put together a basic prototype that will do above except (4).
The program heavily utilizes pydub , which provides easier ways to manipulate audio. A tutorial on that can be found here.
The program basically,
1) I downloaded open-source pre-recorded wav file from this site for numbers 0 to 10 and concatenated them using pydub.
The program slices given audio file in chunks of one second. I've used audacity to separate each word so that they fit within one second window. In real life, that will not be the case.
2) It then passes the word through google speech recognition engine and shows the recognized word. As you will see the word six is not recognized properly. You will need a robust speech recognition engine for this purpose.
3) The program offers three different scramble method.
a) Reverse the word
b) Replace word with equivalent white noise
c) Replace word with silence
4) It then choose three words 9 , 4 and 2 and apply above scramble method and replace the corresponding word file
5)It then concatenates all the words along-with scrambled words in proper order and create output file.
Note: I didn't have enough time to add comparison between word to scramble and recognized word.
Let me know if there are any questions.
****Demo Code:****
""" Declarations """
import speech_recognition as sr
from pydub import AudioSegment
from pydub.silence import split_on_silence
from pydub.generators import WhiteNoise
from pydub.playback import play
""" Function for Speech Recognition """
def processAudio(WAV_FILE):
r = sr.Recognizer()
with sr.WavFile(WAV_FILE) as source:
audio = r.record(source) # read the entire WAV file
# recognize speech using Google Speech Recognition
try:
print("recognizedWord=" + r.recognize_google(audio))
except sr.UnknownValueError:
print("Could not understand audio")
except sr.RequestError as e:
print("Could not request results from GSR; {0}".format(e))
""" Function to scramble word based upon choice """
def scramble_audio(aWord, option):
scramble_file = export_path + "slice" + str(aWord) +".wav"
scramble_audioseg = AudioSegment.from_wav(scramble_file)
aWord_length = scramble_audioseg.__len__() #Get length of word segment to scramble
if option == "reverse": #Reverse word to scramble
scrambled_word = scramble_audioseg.reverse()
elif option == "whiteNoise": #Replace word to scramble with white noise
wn = WhiteNoise() #Instantiate White Noise Object
aWord_length = scramble_audioseg.__len__() #Get length of word segment
scrambled_word = wn.to_audio_segment(duration=aWord_length) #Create audio_segment
elif option == "silence": #Replace word to scramble with silence
scrambled_word = AudioSegment.silent(duration=aWord_length)
print ("Scrambling and Exporting %s" % scramble_file)
scrambled_word.export(scramble_file, format="wav") #Export merged audio file
if __name__ == "__main__":
export_path = ".//splitAudio//"
in_audio_file = "0-10.wav"
out_audio_file = export_path + "scrambledAudio.wav"
#Read main audio file to be processed. Assuming in the same folder as this script
sound = AudioSegment.from_wav(in_audio_file)
sec2_splice = 1 #Splice threshold in sec
audio_length = len(sound) # Total Audio Length In millisec
q, r = divmod(audio_length, sec2_splice) #Get quotient and remainder
#Get total segments and rounds to next greater integer
total_segments= (q + int(bool(r)) ) / 1000 #Converting to sec
#Iterate through slices every one second and export
print ("")
n=0
while n <= total_segments:
print ("Making slice from %d to %d (sec)" % (n , sec2_splice ))
temp_object = sound[ (n * 1000) : (sec2_splice * 1000)] #Slicing is done in millisec
myaudio_file = export_path + "slice" + str(n) +".wav"
temp_object.export(myaudio_file , format="wav")
print ("Trying to recognize %d " %n)
processAudio(myaudio_file)
n = sec2_splice
sec2_splice += 1
#Scramble desired audio slice
print ("")
scramble_word = 9
scramble_audio(scramble_word, "reverse" )
scramble_word = 4
scramble_audio(scramble_word, "whiteNoise" )
scramble_word = 2
scramble_audio(scramble_word, "silence" )
#Combine modified audio
final_audio = AudioSegment.empty() #Create empty AudioSegment
print ("")
i = 0
while i <= total_segments:
temp_audio_file = export_path + "slice" + str(i) +".wav"
temp_audio_seg = AudioSegment.from_wav(temp_audio_file)
print ("Combining %s" % temp_audio_file )
final_audio = final_audio.append(temp_audio_seg, crossfade=0)
i += 1
print ("Exporting final audio %s" % out_audio_file )
final_audio.export(out_audio_file , format="wav")
Output:
Python 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>>
Making slice from 0 to 1 (sec)
Trying to recognize 0
recognizedWord=0
Making slice from 1 to 2 (sec)
Trying to recognize 1
recognizedWord=1
Making slice from 2 to 3 (sec)
Trying to recognize 2
Could not understand audio
Making slice from 3 to 4 (sec)
Trying to recognize 3
recognizedWord=3
Making slice from 4 to 5 (sec)
Trying to recognize 4
recognizedWord=4
Making slice from 5 to 6 (sec)
Trying to recognize 5
recognizedWord=5
Making slice from 6 to 7 (sec)
Trying to recognize 6
recognizedWord=sex
Making slice from 7 to 8 (sec)
Trying to recognize 7
recognizedWord=7
Making slice from 8 to 9 (sec)
Trying to recognize 8
recognizedWord=8
Making slice from 9 to 10 (sec)
Trying to recognize 9
recognizedWord=9
Making slice from 10 to 11 (sec)
Trying to recognize 10
recognizedWord=10
Scrambling and Exporting .//splitAudio//slice9.wav
Scrambling and Exporting .//splitAudio//slice4.wav
Scrambling and Exporting .//splitAudio//slice2.wav
Combining .//splitAudio//slice0.wav
Combining .//splitAudio//slice1.wav
Combining .//splitAudio//slice2.wav
Combining .//splitAudio//slice3.wav
Combining .//splitAudio//slice4.wav
Combining .//splitAudio//slice5.wav
Combining .//splitAudio//slice6.wav
Combining .//splitAudio//slice7.wav
Combining .//splitAudio//slice8.wav
Combining .//splitAudio//slice9.wav
Combining .//splitAudio//slice10.wav
Exporting final audio .//splitAudio//scrambledAudio.wav
>>>