How do I send progress streams using Node and Express? - node.js

I am making a Video Resizer site and I need to send the video conversion progress to the user. I read about SSE but all tutorials use get request and listen through EventSource.
In my case it is post request which contains the video file to be resized.
My Server Side Code is
const resizeVideo = async (req, res) => {
const command = ffmpeg(path.join(rootPath, 'uploads', 'videos', 'v.mp4'));
command.size('?x480').keepDAR().on('end', () => {
res.end();
console.log("Converted");
}).on('progress', function (progress) {
console.log('Processing: ' + progress.percent + '% done');
// I want to send this progress updates to react.js client.
}).save(path.join(rootPath, 'output', '1.mp4'));
}
My Client Side code is
setIsUploading(true);
const config: AxiosRequestConfig = {
responseType: 'json',
headers: {
'Content-Type': 'multipart/form-data', 'Access-Control-Allow-Origin': 'http://localhost:3000', 'Access-Control-Allow-Methods': 'POST', 'Access-Control-Allow-Headers': 'Content-Type, Authorization'
},
};
const response = await instance.post('videoconvert/resize', formData, config);
//I want to get updates here.
setIsUploading(false);
How do I achieve that?

you cant use axios you need to use SocketIO to send update live

Related

Download Video using node js

I have one link, It's local, so this link if I write the link directly in chrome a video is downloaded.
this is the link https://127.0.0.1:7001/api/media/getDirectDownloadVideo?username=admin&password=admin*&setCookie=true&camera_id={c22128b3-6af0-79b8-764e-62234dd023fc}&pos=2023-01-23T13:30&duration=10
so I just want to implementate one function to get it, through node js, I'm using fetch, I create this function:
exports.getVideo = async (req, res) => {
const path = process.env.APP_LOCALHOST_URL + urlDirectDownloadVideo(req.query);
const response = await fetch(path,{
method: 'GET',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${req.query.token}`
},
});
return await response.json();
}
currently don't download the video, what I have to do? to downloaded it

unexpected end of data at line 1 column 1 of the JSON data in express with React

I am working on a node API with React and Express. Node retrieves the data from Postgress like this:
router.get('/getRestaurants', async(req, res) => {
console.log('Restaurants');
try {
const { rows } = await db.getAllRestaurants();
console.log(rows);
res.json(rows);
} catch(error) {
console.error(`Error ${error}`);
res.status(500).send({message: `API internal error`});
}});
The console.log it shows the data without problem and if I use Postman or Curl it seems to work fine. But when I try to retrieve the data from my frontend React I get this error:
Uncaught (in promise) SyntaxError: JSON.parse: unexpected end of data at line 1 column 1 of the JSON data
React makes the POST request like this:
useEffect(() => {
async function fetchData() {
const response = await fetch('http://172.20.0.4:3000/getRestaurants', {
method: 'GET', // *GET, POST, PUT, DELETE, etc.
mode: 'no-cors', // no-cors, *cors, same-origin
cache: 'no-cache', // *default, no-cache, reload, force-cache, only-if-cached
credentials: 'same-origin', // include, *same-origin, omit
headers: {
'Content-Type': 'application/json'
// 'Content-Type': 'application/x-www-form-urlencoded',
},
redirect: 'follow', // manual, *follow, error
referrerPolicy: 'no-referrer', // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
});
const data = await response.json();
console.log(data);
return data;
}
fetchData();
});
It's probably not hard to see but there's something I'm missing. Thank you in advance!
I think you have a problem with CORS, since you are fetching data from another origin, you need to set mode: 'cors', which means that you will fetch data across origins. when you set it to mode: 'no-cors' that mean that you don't allow cross origins and that is the cause of the problem. cos as you said. your express app has a different origin than your react app. but it will still not work until you allow your express api, the origin you are fetching from. by setting headers to: ACCESS-CONTROLLE-ALLOW-ORIGIN * and the star * means allow all kind of origins. but if you want to allow a specific origin, replace the
* with url of your react app. you can also use a node.js package that will help you at this in a clean and easy way, example using cors package https://github.com/expressjs/cors:
const cors = require("cors");
let whitelist = ["http://localhost:3000"];
// Middleware
app.use(
cors({
origin: function (origin, callback) {
if (!origin) return callback(null, true);
if (whitelist.indexOf(origin) === -1) {
var message =
"The CORS policy for this origin doesnt " +
"allow access from the particular origin.";
return callback(new Error(message), false);
}
return callback(null, true);
},
})
);

400 Bad request while fetching data in post call

I am running my React js web app in one port 3000.
For node server I am using 4000.
While calling fetch method it returns `400 Bad request'.
Error
POST http://localhost:4006/auth/admin 400 (Bad Request)
react code npm started in 3000 port
fetch('http://localhost:4000/auth/admin',
{ mode: 'no-cors',
method: "POST",
body: JSON.stringify({
username:"admin",
password:"1234"
}),
headers: {
"Content-Type": "application/json",
'Accept': 'application/json, text/plain, */*',
credentials: "omit", //
// "Content-Type": "application/x-www-form-urlencoded",
},
})
.then((response) => console.log(response));
node code running in 4000 port
const passport = require("passport");
const route = require("../constants/routeStrings");
const keys = require("../config/keys");
const processStatus = require("../constants/processStatus");
const success = {
status: processStatus.SUCCESS
};
const failute = {
status: processStatus.FAILURE
};
module.exports = app => {
app.post('/auth/admin', passport.authenticate("local"), (req, res) => {
res.send(success);
});
};
Do not stringify the body. Change from
body: JSON.stringify({
username:"admin",
password:"1234"
}),
to
body: {
username:"admin",
password:"1234"
},
The 400 response is raised by passport since it is unable to read your params. You need to tell your "node" app to parse them before your actual routes.
// Import body parser, you should read about this on their git to understand it fully
const parser = require('body-parser');
const urlencodedParser = parser.urlencoded({extended : false});
// before your routes
app.use(parser .json());
app.use(urlencodedParser) // This will parse your body and make it available for your routes to use
Then do your other calls.
Also, make sure that you are sending username and password keys, otherwise read the documentation on how to change these key names to something else
I suffered long hours, but I overcame it throw writing those lines of code blocks. I successfully send the request to the server's controller, hopefully yours: make it try.
First define a async function to make POST request:
async function _postData(url = '', data = {}) {
const response = await fetch(url, {
method: 'POST',
mode: 'cors',
cache: 'no-cache',
credentials: 'same-origin',
redirect: 'follow',
referrerPolicy: 'no-referrer',
headers: {
"Content-type": "application/json; charset=UTF-8"
},
body: JSON.stringify(data)
});
return response.json();
}
Now create a request JSON payload:
let requestPayload = {
propertyName1: 'property value1',
propertyName2: 'property value23',
propertyName3: 'property value',
So on
}
Note: Request model will be your desired model, what request payload you actually send.
Now make a request using this payload including your end point URL:
_postData('http://servername/example', requestPayload )
.then(json => {
console.log(json) // Handle success
})
.catch(err => {
console.log(err) // Handle errors
});
100% worked on my project.

Using Node/Express to stream file to user for download

I want to use a Node/Express server to stream a file to the client as an attachment. I would like to make an async request from the client to a /download endpoint and then serve an object received via API proxy to the client as a downloadable file (similar to the way res.attachment(filename); res.send(body); behaves).
For example:
fetch(new Request('/download'))
.then(() => console.log('download complete'))
app.get('/download', (req, res, next) => {
// Request to external API
request(config, (error, response, body) => {
const jsonToSend = JSON.parse(body);
res.download(jsonToSend, 'filename.json');
})
});
This will not work because res.download() only accepts a path to a file. I want to send the response from an object in memory. How is this possible with existing Node/Express APIs?
Setting the appropriate headers does not trigger a download, either:
res.setHeader('Content-disposition', 'attachment; filename=filename.json');
res.setHeader('Content-type', 'application/json');
res.send({some: 'json'});
This worked for me.
I use the content type octet-stream to force the download.
Tested on chrome the json was downloaded as 'data.json'
You can't make the download with ajax according to: Handle file download from ajax post
You can use a href / window.location / location.assign. This browser is going to detect the mime type application/octet-stream and won't change the actual page only trigger the download so you can wrap it a ajax success call.
//client
const endpoint = '/download';
fetch(endpoint, {
method: 'POST',
credentials: 'include',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
}
})
.then(res => res.json())
.then(res => {
//look like the json is good to download
location.assign(endpoint);
})
.catch(e => {
//json is invalid and other e
});
//server
const http = require('http');
http.createServer(function (req, res) {
const json = JSON.stringify({
test: 'test'
});
const buf = Buffer.from(json);
res.writeHead(200, {
'Content-Type': 'application/octet-stream',
'Content-disposition': 'attachment; filename=data.json'
});
res.write(buf);
res.end();
}).listen(8888);
You can set the header to force the download, then use res.send
see those links
Force file download with php using header()
http://expressjs.com/en/api.html#res.set

Node JS upload file streams over HTTP

I'm switching one of my projects from request over to something a bit more light-weight (such as got, axios, or fetch). Everything is going smoothly, however, I'm having an issue when attempting to upload a file stream (PUT and POST). It works fine with the request package, but any of the other three return a 500 from the server.
I know that a 500 generally means an issue on the server's end, but it is consistent only with the HTTP packages that I'm testing out. When I revert my code to use request, it works fine.
Here is my current Request code:
Request.put(`http://endpoint.com`, {
headers: {
Authorization: `Bearer ${account.token.access_token}`
},
formData: {
content: fs.createReadStream(localPath)
}
}, (err, response, body) => {
if (err) {
return callback(err);
}
return callback(null, body);
});
And here is one of the attempts using another package (in this case, got):
got.put(`http://endpoint.com`, {
headers: {
'Content-Type': 'multipart/form-data',
Authorization: `Bearer ${account.token.access_token}`,
},
body: {
content: fs.createReadStream(localPath)
}
})
.then(response => {
return callback(null, response.body);
})
.catch(err => {
return callback(err);
});
Per the got documentation, I've also tried using the form-data package in conjunction with it according to its example and I still get the same issue.
The only difference between these 2 I can gather is with got I do have to manually specify the Content-Type header otherwise the endpoint does give me a proper error on that. Otherwise, I'm not sure how the 2 packages are constructing the body with the stream, but as I said, fetch and axios are also producing the exact same error as got.
If you want any of the snippets using fetch or axios I'd be happy to post them as well.
I know this question was asked a while ago, but I too am missing the simple pipe support from the request package
const request = require('request');
request
.get("https://res.cloudinary.com/demo/image/upload/sample.jpg")
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(request.post("http://127.0.0.1:8000/api/upload/stream"))
and had to do some experimenting to find similar features from current libraries.
Unfortunately, I haven't worked with "got" but I hope the following 2 examples help someone else that are interested in working with the Native http/https libraries or the popular axios library
HTTP/HTTPS
Supports piping!
const http = require('http');
const https = require('https');
console.log("[i] Test pass-through: http/https");
// Note: http/https must match URL protocol
https.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
(imageStream) => {
console.log(" [i] Received stream");
imageStream.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
);
}
);
// Or any readable stream
fs.createReadStream('/Users/file/path/localFile.jpeg')
.pipe(
http.request("http://localhost:8000/api/upload/stream/", {
method: "POST",
headers: {
"Content-Type": imageStream.headers["content-type"],
},
})
)
Axios
Note the usage of imageStream.data and that it's being attached to data in the Axios config.
const axios = require('axios');
(async function selfInvokingFunction() {
console.log("[i] Test pass-through: axios");
const imageStream = await axios.get(
"https://res.cloudinary.com/demo/image/upload/sample.jpg",
{
responseType: "stream", // Important to ensure axios provides stream
}
);
console.log(" [i] Received stream");
const upload = await axios({
method: "post",
url: "http://127.0.0.1:8000/api/upload/stream/",
data: imageStream.data,
headers: {
"Content-Type": imageStream.headers["content-type"],
},
});
console.log("Upload response", upload.data);
})();
Looks like this was a headers issue. If I use the headers directly from FormData (i.e., headers: form.getHeaders()) and just add in my additional headers afterwards (Authorization), then this ends up working just fine.
For me just works when I added other parameters on FormData.
before
const form = new FormData();
form.append('file', fileStream);
after
const form = new FormData();
form.append('file', fileStream, 'my-whatever-file-name.mp4');
So that way I can send stream from my backend to another backend in node, waiting a file in multipart/form-data called 'file'

Resources