I'm running into a CORS issue trying to pull images from S3 (Converting Image URL to base64 - CORS issue).
I'm just using the images for a few seconds while I am generating a PDF file. Is there a way that I can have Meteor download the image and serve for just a few seconds so that I can get around the CORS problem?
I can't have Meteor just serve the images all the time since there are a ton of them and they change for the different reports.
I ended up getting around the CORS issue by doing this:
import { request } from "meteor/froatsnook:request";
Meteor.methods({
convertImage: function(imageUrl) {
try {
var result = request.getSync(imageUrl, {encoding: null});
return 'data:image/png;base64,' + new Buffer(result.body).toString('base64');
} catch(e) {
throw new Meteor.Error("cant-download", "Error: Can't download image.");
}
}
});
Related
I'm trying to implement a multiple large file uploader within a Vue.js application. The backend is a FastAPI application.
The issue is about a strange behavior for the ProgressEvent associated with an axios POST for uploading a file to the backend.
The ProgressEvent.loaded value is not incremental and resets when the file is almost entirely uploaded into the backend. It starts back from a low number of uploaded bytes and finally completes the upload. It seems like the file is uploaded twice.
I have this simple FastAPI path operation function implementing the file upload endpoint:
#router.post(
"/upload/",
status_code=status.HTTP_200_OK,
summary="Upload job-specific file",
description=(
"Accepts file uploads. Files can be uploaded in chunks to allow pausing/ resuming uploads"
),
dependencies=[Depends(get_current_user)]
)
async def upload_file_chunk(chunk: UploadFile, custom_header_job_id=Header(...), settings: Settings = Depends(get_settings)):
filename = compose_upload_filename(custom_header_job_id, chunk.filename)
#filename = '_'.join([custom_header_job_id, chunk.filename])
path_to_file = os.path.join(settings.UPLOAD_FOLDER, filename)
try:
async with aiofiles.open(path_to_file, "ab") as input_file:
while content := await chunk.read(1024):
await input_file.write(content)
except FileNotFoundError:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="File chunk not found",
)
return {"hello": "world"}
The endpoint is not completed yet, since it is supposed to do other things besides receiving the file.
The frontend request starts from a Vue component:
uploadFileChunk({ file, jobId, startByte, onProgressUpdate = undefined }) {
const chunk = file.slice(startByte);
const formData = new FormData();
formData.append("chunk", chunk, file.name);
//formData.append("jobId", jobId);
/* return axios.post(`http://localhost:1234/upload`, formData, {
headers: {
"Custom-Header-Job-Id": jobId,
"Content-Disposition": `form-data; name="chunk"; filename="${file.name}"`,
"Content-Range": `bytes=${startByte}-${startByte + chunk.size}/${
file.size
}`,
},
onUploadProgress: onProgressUpdate,
}); */
return instance.post(`/files/upload`, formData, {
headers: {
"Custom-Header-Job-Id": jobId,
"Content-Disposition": `form-data; name="chunk"; filename="${file.name}"`,
"Content-Range": `bytes=${startByte}-${startByte + chunk.size}/${
file.size
}`,
},
onUploadProgress: onProgressUpdate,
});
}
const onProgressUpdate = (progress) => {
console.log("loaded: ", progress.loaded);
const percentage = Math.round(
(progress.loaded * 100) / file.size);
};
console.log("percentage: ", percentage);
The commented request points to a different Node.js backend with a file upload endpoint exclusively made to assess if the current issue I'm facing up is dependent on the client code or the backend. Here is the implementation (not an expert in Node.js and express):
const express = require("express");
const cors = require("cors");
const multer = require("multer");
const app = express();
app.use(express.json());
app.use(cors());
const upload = multer({ dest: "uploads/" });
app.post("/upload", upload.single("chunk"), (req, res) => {
res.json({ message: "Successfully uploaded files" });
});
app.listen(1234);
console.log("listening on port 1234");
In addition, the client code is actually put into a much more articulated pattern involving XState for managing the uploader component. Nevertheless, attached snippets should be enough to have an idea of the main parts to be discussed here.
Here is a screenshot for the request the FastAPI endpoint:
FastAPI
Where we can see the file is almost entirely uploaded and then the uploaded parameter drops to a lower upload percentage, eventually finalizing the upload (not shown here).
The same experiment repeated on the Node.js endpoint does not create issues, which splits the upload in much less packets:
Node.js express
It seems like the Node.js backend works fine, whereas the FastAPI doesn't. In my opinion there are some issues with how FastAPI/ starlette manages large files. It could be something related to the spooled file that starlette creates, or maybe something happening when passing from storing the file in the main memory to the mass memory. Unfortunately, starlette UploadFile class seems very hermetic and not easy to be customized/ inspected.
DETAILS
FastAPI backend running on a Debian Bullseye docker image
FastAPI version 0.78.0
python-multipart version 0.0.5
client and server running in localhost and tested with Chrome 103.0.5060.53 (Official Build) (x86_64)
system: Mac OSX 11.3.1
Thank you so much for your help!
I have a site that shows a live broadcast from youtube
Website built from reactjs and nodejs
In frontend I use video.js and hls.js, to view a live broadcast from M3U8 links, and it works for me with a sample link:
https://multiplatformf.akamaihd.net/i/multi/will/bunny/big_buck_bunny_,640x360_400,640x360_700,640x360_1000,950x540_1500,.f4v.csmil/master.m3u8
In nodejs (backEnd) I use youtube-dl model to create or get an M3U8 link, I send it to react and put it in the src value in video html tag.
react component code:
let videoSrc = data.url;
if (Hls.isSupported()) {
var hls = new Hls();
hls.loadSource(videoSrc);
hls.attachMedia(video);
}
node js code:
let video = youtubedl(`yotubeUrl`, [])
let stream = this;
video.on('info', function (info) {
console.log(info.url)
stream.updateYoutubeStream(info.url) // personal function to update m3u8 url
}).on('error', err => {
console.log(err)
})
But he gives me a cors error
enter image description here
And unable to connect with me
What is the problem? And how can it be fixed?
Thank you very much
I'm trying to show an image in react, which is neither a local image (in the client) nor an external image from the web but an image that is in the node.js express server (and I don't want to call it as if it was an external image, because the domain could change and it just doesn't seem right).
I know I can't just import it like I do with a local image in the client because we're speaking about different localhosts. I did try this:
loadImage = async (imageUrl) => {
const response = await fetch(`/api/images/${imageUrl}`);
const data = await response.json();
this.setState({ image: data });
}
componentDidMount() {
const { imageUrl } = this.props;
try {
this.loadImage(imageUrl);
} catch(error) {
console.log("Hay un error: " + error);
}
}
render() {
const { image } = this.state;
return(
<div>
<div>
<img alt="dontknowyet" className="blog-list-image" src={image} // and so on...
{image} does receive the correct path, but the image won't load and the console throws this error:
Not allowed to load local resource: file:///C:/Users/Dafna/Desktop/adrian/proyectos/esteticand/img/t4.jpg
So how can I make it work? and in case that I need to import the image file instead of just the link, how can I do that? (I can't update the state with an image...)
In order to access the path of the image it has to be done through the express server.
For example, if the (backend) server is running on port 4500 and the image is in a folder called images, and the express variable is called app, in the server file you have to use:
app.use(express.static('images'));
and then the image can be accessed in http://localhost:4500/nameoftheimage.jpg.
Do you have the api running on the same port as the React app?
You usually would make them run on different ports. Maybe it's got something to do with it.
I am trying to upload the image on the AWS server using the multer using the following code:
//post request
var multer = require('multer');
var upload = multer({ dest: path.resolve('./public/uploads/'), });
/* POST saveblog router. */
router.post('/uploadMedia', upload.any(), function(req, res, next) {
//console.log("res",res);
console.log(req.body, 'Body');
console.log(req.files, 'files');
var myJSON = JSON.stringify(req.files[0].filename);
console.log("file name "+myJSON);
if (typeof myJSON != 'undefined'){
var obj= new Object();
obj.status=true;
obj.message="File Uploaded Successfully";
obj.type= req.files[0].mimetype;
var fileExtensionArray=(req.files[0].mimetype).split("/");
var fileExtension=fileExtensionArray[1];
res.json(obj);
}else {
var obj= new Object();
obj.status=false;
obj.message="Error while uploading file try again";
res.json(obj);
}
res.end();
});
It is working fine on the local. But when I upload that code to the server and try to hit through the API. The server stops with following logs
Change detected on path public/uploads/b690b296bfde62eb8ff527328bc8b463 for app www - restarting
PM2 | Stopping app:www id:0
As log says change detected but I am not able to find any image.
As I have searched on StackOverflow and google, I am not able to find tutorial or help to do the same.
I am getting help regarding the s3 but I don't want to upload on it.
Is it not possible to upload the image to AWS server like that and I have to use S3 or something wrong with my code?
Edit: Now I am able to get the image to the destination folder but still not able to return the response.
You're likely running PM2 in Watch mode, which is a feature intended to be used in development, to restart your application upon changes to its files. This is not intended to be used in production.
To fix this, if you're starting pm2 from the cli use the option
--no-autorestart
or the yaml config;
autorestart: false
I am using Typescript and Cordova 4.0.
I have the following sample code:
uploadImages(imageUris: Array<string>): any {
var fileTransfer = new FileTransfer();
for (var i = 0; i < imageUris.length; i++) {
fileTransfer.upload(imageUris[i], encodeURI('http://3187cf3.ngrok.com/test/photos'), (success) => {
alert('success');
}, (err) => {
alert('error');
});
}
}
This corresponds to an express route:
var router = express.Router(),
test = test.controller;
router
.post('/test/photos', bind(test.uploadPhotos, test));
Which corresponds to a controller method:
uploadPhotos(req: express.Request, res: express.Response) {
console.log(req);
}
I can't seem to figure out how to, inside of my controller, grab the "file" or image I'm posting to my server using Filetransfer. It's not on req.body or req.query, and when I look through the entire req I can't seem to locate the file. The app flow is working enough to actually make the POST request to test/photos, I just don't know how to or if I can access the file at that point.
How does Filetransfer work, and how can I access the data I need in my controller so that I can push it to Azure Blob Storage?
It looks like you have everything setup correctly to send the data through to your controller. The issue is that you need to put the file on the request since cordova's filetransfer plugin doesn't do that by default.
You can do that with a popular library multer.
npm install multer --save-dev To install multer and save it to your package.json file.
In your express config file, add something like the following:
var multer = require('multer');
app.use(multer({ dest: path.resolve(config.root, 'public/img/') }))
'public/img/' is the path you would like for your image to be saved.
Now your req will have files on it. To upload a single file, you would use req.files.file. You'll want to use this variable to send your file to azure's blob storage using something like blobSvc.createBlockBlobFromLocalFile(containerName, fileName, filePath)
Since you're using Azure for remote storage, chances are you will want to remove the local file that multer has saved. I'd recommend using fs or rimraf to remove the file stored in public/img/, or whatever you set the path to in your express config. If you are using fs, you'll want to use the .unlink command.