How can I cancel a stream created with const stream = file.createReadStream()?
I'm using node.js and sending files via fastify to the client. If the client closes the connection, I'd like also to cancel the reading from the stream, so to close the connection to cloud storage and stop transfering data.
Currently my server continues reading from the stream but the content is never used since the outgoing stream to the client is already destroyed. fastify calls stream.destroy() when the receiving end is prematurely closed. But the read stream happily receives data from the cloud storage.
From reading the source code, this is a PassThrough stream, that gets its content only via events and has no connection to the underlying receiving stream.
Is there a method to stop this cloud stream?
Christian
So far the only way I found to avoid this was to call reply.hijack()
https://www.fastify.io/docs/latest/Reference/Reply/#hijack
reply.hijack()
reply.raw.setHeader('Content-Type', object.contentType)
storage.createReadStream().pipe(reply.raw)
Related
Is there an API in the browser (outside of websockets) which allows us to stream data from a file to the browser? something like this:
const reader = new FileReader();
reader.readAsText(file, "UTF-8");
reader.on('data', d => { // imaginary api
// new line of data d
});
what could happen is the user selects the local file, and some process on the local OS writes to it. If this doesn't work, then websockets is an option.
Browsers can consume streaming data using the Streams API, here how to use it, from those links:
The basic usage of Streams hinges around making responses available as streams. For example, the response body returned by a successful fetch request can be exposed as a ReadableStream, and you can then read it using a reader created with ReadableStream.getReader(), cancel it with ReadableStream.cancel()
// Fetch the original image
fetch('./tortoise.png')
// Retrieve its body as ReadableStream
.then((response) => {
const reader = response.body.getReader();
// …
});
A good post about the Streams API
Another option could be using server sent events implementing the "streaming" as a sequence of reactions to events (new lines from the file?), still from mdn links EventSource Interface:
Unlike WebSockets, server-sent events are unidirectional; that is, data messages are delivered in one direction, from the server to the client (such as a user's web browser). That makes them an excellent choice when there's no need to send data from the client to the server in message form.
Here a link to another question with a lot of cool info and links
These solutions involve some Server side work of course
Background: Stream Consumption
This is how you can consume and read a stream of data-bytes that is received in the client:
Get a Response object (example a fetch response.)
Retrieve the ReadableStream from the body field of it i.e response.body
As body is an instance of ReadableStream, we can do body.getReader()
And use the reader as reader.read()
That is simple enough, we can consume the stream of bytes coming from the server and we can consume it the same way in NodeJS and Browser through the Node Web Stream API.
Stream from path in NodeJS
To create a stream from URLPath in Node JS it is quite easy (you just pass the path to a stream.) You can also switch from NodeStream to WebStream with ease (Stream.toWebStream())
Problem: Stream from user upload action
Can we get a stream right from user file uploads in the browser ? Can you give a simple example ?
The point of it would be to process the files as the user is uploading them, not to use the Blob stream.
Is it possible analyze user data as it is being uploaded i.e is it possible for it to be a stream ?
I'm facing a problem with a tcp server for IoT devices, I'm using a stream approach implementing a transform class to process incoming data (the device class).
this.server = net.createServer(async function onConnection(socket) {
const device = new gateway.DeviceClass(gateway);
socket.setTimeout(0);
socket
.pipe(device)
then on close event I disconnect the stream processor and destroy the socket
socket.once('close', ()=>{
this.socket.unpipe();
this.socket.removeAllListeners();
this.socket.destroy();
this.socket.unref();
})
The problem is that sometime the client is still sending data? and I got a closed stream write error. Also sometime it seems I'm missing data that is transmitted but never received?
from the documentation close event is emitted "Emitted once the socket is fully closed. The argument hadError is a boolean which says if the socket was closed due to a transmission error." so I expect no more data is sent over it.
I'm using puppeteer-stream to get a stream of a browser controlled by Node, running on a server. I am able to write this stream out to a file with no issues.
I wanted to stream this stream via WebRTC to a browser (basically to see what the browser is running in realtime). For webrtc, I'm trying to use simple-peer since it has ready bindings for Node as well as the browser-side.
However, when I try to pass this stream to simple-peer, I get the following error:
/Users/my_user/my_project/node_modules/simple-peer/index.js:286
stream.getTracks().forEach(track => {
^
TypeError: stream.getTracks is not a function
at Peer.addStream (/Users/my_user/my_project/node_modules/simple-peer/index.js:286:12)
This is because the Stream I have is a ReadableStream, but simple-peer (or most webrtc libs) expects a MediaStream.
How can I convert a realtime ReadableStream into a MediaStream that can be used with WebRTC? I found examples to convert MediaStreams into ReadableStreams such as here but not vice versa.
Am I missing something here?
I'm writing a CLI script to a resumable upload file stream.
When I click CTRL-C save current state before exit.
Problem is I don't know how to cancel upload stream proper way with request.js.
var stream = fs.createReadStream(filepath);
stream.pipe(request.put(options, callback));