I try to create a http server that will download the HTML at https://google.com
And serve it (at localhost:3000). Kind of a proxy.
With this code:
const express = require('express')
const https = require('https')
const app = express()
app.get('/', function (req, mainRes) {
https.get('https://www.google.fr', (res) => {
res.on('data', (d) => {
mainRes.send(d)
})
})
})
app.listen(3000)
The html from google.com seems to be downloaded, but the server crashes with this error :
Error: Can't set headers after they are sent.
I Understand it's related to the 2 wrapped requests but I don't know how to fix it.
In your code the argument res is a stream (see the docs for https.get).
Currently you are attempting to send the complete response each time a chunk is received through that stream. Hence you are getting the error Can't set headers after they are sent., because the second time mainRes.send() is called you are trying to send the whole response again. (See the docs for express' res.send.)
You want to pipe res through the express response object, as this is also a stream:
const express = require('express')
const https = require('https')
const app = express()
app.get('/', function (req, mainRes) {
https.get('https://www.google.fr', (res) => {
mainRes.pipe(res)
})
})
app.listen(3000)
I am not 100% sure in your case but I would suggest you to add a debugger statement and step through it.
#sdgluck && #kwesi1337 helped me find a solution, I need to concat all chunks from the data event :
const express = require('express')
const https = require('https')
const app = express()
let data = ''
app.get('/', function (req, mainRes) {
https.get('https://www.google.fr', (res) => {
res.on('data', (chunk) => {
data += chunk;
})
res.on('end', () => {
mainRes.end(data);
});
})
})
app.listen(3000)
Related
I have the following code to fetch a video from a remote url and then play it in the browser:
const express = require('express')
const http = require('https')
const url = 'https://www.exmaple.com/video.mp4'
const app = express()
const port = 3000
const url = require('url')
app.get('/', (req, res) => {
res.sendFile(__dirname + '/player.html') //simple html document that contains a <video>
// tag and it's 'src' is '/bro'
})
app.get('/bro', (req, res) => { //this request is made by the <video> tag in the player.html file
http.get(url, (response) => {
response.pipe(res)
})
})
but, the player hangs for like 45 seconds before it starts playing the video, and if the video duration was like 40 minutes or more, the player only plays the first 3 minutes of it.
MY QUESTION IS: How can I use the 'data' event on the http response to play or pipe that incoming chunks of data to the browser?
Is there a way to process those chunks to make them playable and pass the to the browser?
I used it like the following and It didn't work.
app.get('/bro', (req, res) => {
http.get(url, (response) => {
response.on('data', (data) => {
data.pipe(res)
})
})
})
I'm writing a server and a client with Node.js, the server uses express and the client uses axios.
I'm trying to send an image file from the client to the server. I found somewhere here this bit of code for the client:
let file = fs.createReadStream(file_path);
let form_data = new FormData();
form_data.append("picture", file);
let post_config = {
method: "post",
url: SERVER_PICTURE_URL,
headers: {"Content-Type": "multipart/form-data"},
data: form_data
}
axios(post_config).then(_ => {console.log("sent");} );
But I can't figure out what's supposed to be on the server side. I've tried the most obvious solution, writing response.data or response.form to a file, but both are undefined.
Is there some parser I'm supposed to use? And if so, how?
I'm not sure about the client that you wrote, but in express, you need to use express-fileupload package for getting the picture from req.files
const express = require('express');
const fileupload = require("express-fileupload");
const app = express();
const port = 3000;
app.use(fileupload());
app.post('/picture', (req, res) => {
const files=req.files;
res.send(files)
})
app.listen(port, () => {
console.log(`app listening at http://localhost:${port}`)
})
Im trying to send 10 GB data from a node client to a node http server. It works fine until i reach about 5 GB. Protocol is http.
It works fine until i reach about 5 GB. Error I get on server side is [ERR_STREAM_PREMATURE_CLOSE] and on client side RequestError: read ECONNRESET.
example server:
const http = require('http')
const fs = require('fs')
const util = require('util')
const stream = require('stream')
const pipeline = util.promisify(stream.pipeline)
const host = 'localhost'
const port = 4000
const server = http.createServer(async (req, res) => {
try {
const writeStream = fs.createWriteStream('file', { encoding: 'binary' })
await pipeline(req, writeStream)
} catch (e) {
console.log(e)
}
})
server.listen(port, host, () => {
console.log(`Server is running on http://${host}:${port}`)
})
example client
const got = require('got')
async function upload () {
await pipeline(fs.createReadStream(filename), got.stream.post('http://localhost:4000/upload'))
}
I have tried with different servers (raw http and express) and clients (raw node, request, node fetch). Ive also tried multipart with busboy. Same problem.
Im trying this running node v14.4.0 on a Mac.
I tried transferring a 10 gb file on my computer and it worked for me.
Server example:
const http = require('http');
const fs = require('fs');
const PORT = 8333;
const server = http.createServer((req, res) => {
const writeStream = fs.createWriteStream('./result.txt');
req.pipe(writeStream);
req.on('end', () => {
console.log('The file was successfully written.');
res.end('OK');
});
});
server.listen(PORT, () => {
console.log(`Server has been started on port ${server.address().port}`);
});
Client example
const http = require('http');
const fs = require('fs');
function upload() {
const readStream = fs.createReadStream('./test.txt');
const request = http.request('http://localhost:8333', { method: 'POST' }, (res) => console.log(`STATUS: ${res.statusCode}`));
readStream.pipe(request);
readStream.on('end', () => request.end());
}
upload();
I am not saying that this code is very correct, but you can try it, since it worked for me.
I ran this example on Node.js v12.4.1 and Windows 10.
I am writing a server that is meant to serve and receive files. It is written in node.js, using express.js. I also have a client, also written in node, which is meant to send a request to the server and receive the files on the server.
Server-side
const express = require("express");
const app = express();
const file = "./samplefiles/Helloworld.txt";
app.get("/", (res)=>{
res.download(file);
});
module.exports = app; //this exports to server.js
const http = require("http");
const app = require("./app.js);
const port = 8080;
const server = http.createServer(app);
server.listen(port, () => {
console.clear();
console.log("server running");
})
Client-side
const request = require("request");
request.get("http://localhost:8080/", (req, body) => {
console.log(body);
console.log(res);
});
If I try to access it by my browser I am asked what I want to do with the file, it works. However, Is I run my client-side code it prints the body and the res(being null). I expected the file name and it's content to be in the body but only the content of the file was in the body.
I want to receive the whole file, is possible, or at least get the name of it so that I can "make" a copy of it on the client-side.
Change code your server side to:
const port = 8080;
const express = require("express");
const app = express();
const path = require('path');
app.get("/", function(req, res){
res.sendFile(path.join(__dirname, 'app.js'));
});
app.listen(port, () => {
console.clear();
console.log("server running");
});
Change code your client-side to:
var request = require('request');
request('http://localhost:8080/', function (error, response, body) {
console.log('error:', error); // Print the error if one occurred
console.log('statusCode:', response && response.statusCode); // Print the response status code if a response was received
console.log('body:', body); // Print data of your file
});
You need to install request npm i request for client side
You can serve up any files you want with express static method:
app.use(express.static('public'))
in this case just put all the files you want to serve in folder called public and then you can access it by localhost:8080/Helloworld.txt.
I ended up working around it.
I sent the file name as a header and was thus able to create a replica of the file I wanted to download using the body info and the filenameheader.
I want to be able to proxy a remote icecast stream to client. I've been fiddling around a lot in the past few days to no avail.
Use case:
Be able to extract analyser data out of an <audio> tag src without running into CORS issues.
My solution so far
In order to address CORS issues preventing me to create an leverage sound data directly out of the <audio>'s source, I've tried to write a tiny proxy which would pipe requests to a specific stream and return statics in any other case.
Here is my code:
require('dotenv').config();
const http = require('http');
const express = require('express');
const app = express();
const PORT = process.env.PORT || 4000;
let target = 'http://direct.fipradio.fr/live/fip-midfi.mp3';
// figure out 'real' target if the server returns a 302 (redirect)
http.get(target, resp => {
if(resp.statusCode == 302) {
target = resp.headers.location;
}
});
app.use(express.static('dist'));
app.get('/api', (req, res) => {
http.get(target, audioFile => {
res.set(audioFile.headers);
audioFile.addListener('data', (chunk) => {
res.write(chunk);
});
audioFile.addListener('end', () => {
res.end();
});
}).on('error', err => {
console.error(err);
});
});
app.listen(PORT);
The problem
The client receives a response from the proxy but this gets stalled to 60kb of data about and subsequent chunks are not received, in spite of being received by the proxy:
Any suggestion is welcome!
I've found a solution, use stream pipe.
const app = express();
const PORT = process.env.PORT || 4000;
let target = 'http://direct.fipradio.fr/live/fip-midfi.mp3';
// figure out 'real' target if the server returns a 302 (redirect)
http.get(target, resp => {
if(resp.statusCode == 302) {
target = resp.headers.location;
}
});
app.use(express.static('dist'));
app.get('/api', (req, res) => {
req.pipe(request.get(target)).pipe(res);
});
app.listen(PORT);