I see how I can create a readable stream for a file in GridFS in Node.js using the native mongo driver. However, I'm writing a server that responds to byte range requests, so I'd like to only stream back part of the file. What's the best way to do this? Should I just implement my own read stream that pulls data from the database in chunks? Thanks!
Unfortunately the stream does not support arbitrary starting points so you'll have to implement your own that seeks into the correct chunk and then streams from there.
Related
We have a Redis server that all clients attach to for a variety of data transfer and coordination tasks. We have a new requirement that we support video streaming. I would like to avoid running a dedicated service (with all the accompanying network and security requirements that entails) and just stream over Redis.
Redis seems like a good fit for real time streaming, in particular using Redis streams. I realize that "Redis streams" have no relation to "video streaming", however, our use case follows Redis stream structure well. We want to buffer X seconds of video continuously allowing clients to attach to that real-time stream at any time. We have no need to store history or serve static video content.
Redis seems like a good solution, my problem is I don't know how to
stream an appropriate video codec (Motion JPEG maybe?) over Redis.
I wouldn't know how to join a stream mid-broadcast (join at a keyframe
perhaps?).
I wouldn't know how to serialize the stream to bytes at
the server (Python based) and de-serialize the stream to a video codec and player on
the client (a browser). Perhaps it's as simple as seralization/deseralization in opencv or equivalent and I'm just over thinking it?
These are all features I would typically look to an API to perform, but is there an API is capable of this? I'm inexperienced in the field of video streaming.
At a high level, I prefer viewing streaming as a pub-sub problem. Where producers produce chunks of information and consumers read that information on need basis.
Some solution may not be readily available, we may need to perform the following steps:
Publish:
1. chunk-id : content
2. chunk-id-fwd : (nextChunkId)
3. videoId : latestChunkId (Assuming your realtime usecase is for live streams, this can help users access 'go-live' button)
Consume:
Start:
1. Get latest chunk
2. Get content from latest chunkId
3. Get nextChunkId from chunk-id-fwd
I can save the file to disk with formidable and then send the file bits to mongo with node, but how can I just handle streaming the file bits directly to mongo?
I don't need gridfs, these are small files. Just want to write them to the normal store.
Use options.fileWriteStreamHandler to setup your own stream. Then write to mongodb if the API accepts a stream
I'm using socket.io-stream to share file over socket from server t browser. I'd like to use the same to share audio stream from browser to server. Is it possible? I know that browser audio stream is different from node.js stream, so i need to convert it, how?
Not 100% sure what you're expecting to do with the data, but this answer may be of use to you.
Specifically, I'd suggest you use getUserMedia to get your audio, hook it up to a Script Processor, convert the data, and emit those data chunks to socket.io. Then on your server, you can capture those chunks and write them to your node.js stream. Code samples are at the link; they're fairly lengthy and I don't want to spam, so I won't reproduce them here.
I have a readable stream (s3) that I would like to pipe to dropbox's put endpoint, however this does not support Transfer-Encoding: chunked required for streaming data.
I see 2 possible solutions
read the stream into a variable of some sort then send up, memory can then be a problem
write the stream to disk then read it back and upload, which feels dirty and will be slow
What is the best solution to this problem?
I have just started using node.js, I'm running a node server with sockets.io and i need to send a buffer of bytes to the client.
I understand that this can be done by first translating the byte buffer to base64 and sending that, then translating it back on the client side. but i was wondering if there is a more elegant way of getting the byte stream to the client.
Socket.IO 1.0 Now supports Binary data transfer. Please have a look here . You can use Blob, ArrayBuffer and File.
https://github.com/binaryjs/binaryjs can be a solution. base 64 have ~30% of overhead size, so if you need to transfer large amount of data it will become inefficient.
There is also socket.io-stream https://github.com/nkzawa/socket.io-stream
It is little difficult to use binaryjs with socket.io.
Try deliveryjs
https://github.com/liamks/Delivery.js
which provides the means of communication between clients and server via socket.io.
However this module also uses the base64 conversion, which is a drawback.