Streaming audio from S3 via Express server takes too long - node.js

Question

I am trying to stream audio files from S3 to my React client by making a request to my Node/Express server. I managed to implement something that works, but I am not sure if I am actually streaming the file here, or simply downloading it. I suspect I might be downloading the file, because my requests to the server take a long time to come back:
Established database connection.
Server listening on port 9000!
::1 - - [18/Apr/2017:21:13:43 +0000] "GET / HTTP/1.1" 200 424 6.933 ms
::1 - - [18/Apr/2017:21:13:43 +0000] "GET /static/js/main.700ba5c4.js HTTP/1.1" 200 217574 1.730 ms
::1 - - [18/Apr/2017:21:13:43 +0000] "GET /index.css HTTP/1.1" 200 - 8.722 ms
Server received a request: GET /tracks
::1 - - [18/Apr/2017:21:13:43 +0000] "GET /tracks HTTP/1.1" 304 - 41.468 ms
Server received a request: GET /tracks/1/stream
::1 - - [18/Apr/2017:21:14:13 +0000] "GET /tracks/1/stream HTTP/1.1" 200 - 636.249 ms
Server received a request: GET /tracks/2/stream
Database query threw an error: ETIMEDOUT
Note the 636.249 ms!
Can you guys tell if I am doing anything wrong here? I pasted the code from my current approach below; It does the following:
Client makes a fetch call to /tracks/id/stream
Server queries database to get the track
Server uses downloadStream (from the s3 package) to get the file
Server pipes the data to the client
Client receives the data as an ArrayBuffer
Client decodes the buffer and passes it to an AudioBufferSourceNode
AudioBufferSourceNode plays the audio
The server-side:
app.get('/tracks/:id/stream', (req, res) => {
const id = req.params.id
// Query the database for all tracks
database.query(`SELECT * FROM Tracks WHERE id = ${id}`, (error, results, fields) => {
// Upon failure...
if (error) {
res.sendStatus(500)
}
// Upon success...
const params = {
Bucket: bucketName, // use bucketName defined elsewhere
Key: results[0].key // use the key from the track object
};
// Download stream and pipe to client
const stream = client.downloadStream(params)
stream.pipe(res)
})
});
The client-side fetch call:
const URL = `/tracks/${id}/stream`
const options = { method: 'GET' }
fetch(URL, options)
.then(response => response.arrayBuffer())
.then(data => {
// do some stuff
AudioPlayer.play(data)
})
The client-side AudioPlayer, responsible for handling the actual AudioBufferSourceNode:
const AudioPlayer = {
play: function(data) {
// Decode the audio data
context.decodeAudioData(data, buffer => {
// Create a new buffer source
source = context.createBufferSource()
source.buffer = buffer
source.connect(context.destination)
// Start playing immediately
source.start(context.currentTime)
})
},
...

Answer 1

There's a lot wrong here, so let's just go through it piece by piece, whether it was related to your original question or not.
database.query(`SELECT * FROM Tracks WHERE id = ${id}`, (error, results, fields) => {
In this line, you open yourself up to SQL injection attacks. Never concatenate arbitrary data into the context of a query (or any other context for that matter) without proper escaping. Whatever database library you're using will have a parameterized method that you should be utilizing.
I suspect I might be downloading the file, because my requests to the server take a long time to come back
Who knows... you didn't show us a single location where you're doing the logging so it's hard to say whether or not the logged line is before or after the request is complete. One thing that is clear however is that the response has to at least begin, otherwise the response status code wouldn't be known. A 600 ms response time for the first resource byte from S3 isn't unheard of anyway.
Server uses downloadStream (from the s3 package) to get the file
Server pipes the data to the client
You're wasting a lot of bandwidth with this. Rather than fetching the file and relaying it to the client, what you should do is sign a temporary URL with a 15-minute expiration or so, and redirect the client to it. The client will follow the redirect and now S3 is responsible for handling your clients. It will cost you half as much bandwidth, less CPU resource, and will be delivered from a location likely closer to your users. You can create this signed URL with the AWS JS SDK.
Client receives the data as an ArrayBuffer
There's no streaming happening here. Your client is downloading the entire resource before it's playing anything.
What you should do is be creating a normal Audio instance. It will automagically follow your redirect from your Node.js app to your signed S3 URL and handle all the buffering and streaming for you.
let a = new Audio('/tracks/' + encodeURIComponent(id) + '/stream');
a.play();