Say "hello" when Twilio call connected - python-3.x

Question

I have a websocket in python Flask that listens to a twilio call. When the call is started I want to say "hello" here is the code.
if data['event'] == "start":
speakBytes = speaker.speak("Hello") // using micrsoft cognitive service to convert the text to bytes
convertedBytes = ap.lin2ulaw(speakBytes.audio_data,1)
ws.send(responseString.format(base64.b64encode(convertedBytes), str(data['streamSid'])))
But the above is not working. I checked microsoft cognitive services speech sunthesizer returns the bytes in WAV format so I have used lin2ulaw form python audioop module.
Need help. Thanks in advance.

Answer 1

Twilio developer evangelist here.
It looks like you are correctly creating the audio to send to the Twilio Media Stream, however I don't think you are sending the correct format.
Twilio Media Streams expect a media message to be a JSON object with the following properties:
event: the value "media"
streamSid: the SID of the stream
media: an object with a "payload" property that then contains the base64 encoded mulaw/8000 audio
Something like this might work:
message = {
"streamSid": data['streamSid'],
"event": "media",
"media": {
"payload": base64.b64encode(convertedBytes)
}
}
# Serializing json
json_object = json.dumps(message)
ws.send(json_object)

Answer 2

If you're using Twilio to connect the number then you'll need to reply with TwiML to the call:
from twilio.twiml.voice_response import VoiceResponse
response = VoiceResponse()
response.say('Hello')
return str(response)
See the doc of <Say></Say.
If you want to use the .wav you created then you would need to save it somewhere accessible (e.g. an Amazon S3 bucket) and then you can use TwiML <Play></Play>.

Answer 3

Thanks for the answers everyone. The solution turned out to be a small change.
I had to change ap.lin2ulaw(speakBytes.audio_data,1) to ap.lin2ulaw(speakBytes.audio_data,4) and it worked fine. It seems to be the compatibility of microsoft text to speech and twilio formats.