Does NodeJs stream pipe is symmetric? - node.js

Question

I'm building a server which transfers files from endpoint A to endpoint B.
I'm wondering if the NodeJs stream pipe is symmetric?
If I do the following: request.get(A).pipe(request.put(B));, does it upload as fast as it downloads?
I'm asking this question, because my server has an asymmetric connexion (it downloads faster than upload), and I try to avoid memory consumption.

Answer 1

According to node's documentation on stream#pipe
pipe will switch the read stream to flowing mode - it will read only when the write stream has finished consuming previous packets.
readable.pipe() method attaches a Writable stream to the readable, causing it to switch automatically into flowing mode and push all of its data to the attached Writable. The flow of data will be automatically managed so that the destination Writable stream is not overwhelmed by a faster Readable stream.
So your transfer may be asymmetrical, due to different send/download speed - the difference may be buffered in Node's memory - Buffering of streams
Buffering#
Both Writable and Readable streams will store data in an internal
buffer that can be retrieved using writable._writableState.getBuffer()
or readable._readableState.buffer, respectively.
The amount of data potentially buffered depends on the highWaterMark
option passed into the streams constructor. For normal streams, the
highWaterMark option specifies a total number of bytes. For streams
operating in object mode, the highWaterMark specifies a total number
of objects.
Data is buffered in Readable streams when the implementation calls
stream.push(chunk). If the consumer of the Stream does not call
stream.read(), the data will sit in the internal queue until it is
consumed.
Once the total size of the internal read buffer reaches the threshold
specified by highWaterMark, the stream will temporarily stop reading
data from the underlying resource until the data currently buffered
can be consumed (that is, the stream will stop calling the internal
readable._read() method that is used to fill the read buffer).
Data is buffered in Writable streams when the writable.write(chunk)
method is called repeatedly. While the total size of the internal
write buffer is below the threshold set by highWaterMark, calls to
writable.write() will return true. Once the the size of the internal
buffer reaches or exceeds the highWaterMark, false will be returned.
A key goal of the stream API, and in particular the stream.pipe()
method, is to limit the buffering of data to acceptable levels such
that sources and destinations of differing speeds will not overwhelm
the available memory.
Because Duplex and Transform streams are both Readable and Writable,
each maintain two separate internal buffers used for reading and
writing, allowing each side to operate independently of the other
while maintaining an appropriate and efficient flow of data. For
example, net.Socket instances are Duplex streams whose Readable side
allows consumption of data received from the socket and whose Writable
side allows writing data to the socket. Because data may be written to
the socket at a faster or slower rate than data is received, it is
important each side operate (and buffer) independently of the other.
I recommend that you look at this question here the topic is elaborated a further.
If you run the following sample
const http = require('http');
http.request({method:'GET', host:'somehost.com', path: '/cat-picture.jpg'}, (response)=>{
console.log(response);
}).end()
you can explore the underlying sockets - on my system they all have the highWaterMark : 16384 property. So if I understand the documentation, and the above-mentioned questions, in your case about 16KB may be buffered in the faster GET socket on Node.js level - what happens below is probably highly dependent on your system/network configuration.