FastAPI: file uploaded bytes on client ProgressEvent done twice before completing upload - node.js

Question

I'm trying to implement a multiple large file uploader within a Vue.js application. The backend is a FastAPI application.
The issue is about a strange behavior for the ProgressEvent associated with an axios POST for uploading a file to the backend.
The ProgressEvent.loaded value is not incremental and resets when the file is almost entirely uploaded into the backend. It starts back from a low number of uploaded bytes and finally completes the upload. It seems like the file is uploaded twice.
I have this simple FastAPI path operation function implementing the file upload endpoint:
#router.post(
"/upload/",
status_code=status.HTTP_200_OK,
summary="Upload job-specific file",
description=(
"Accepts file uploads. Files can be uploaded in chunks to allow pausing/ resuming uploads"
),
dependencies=[Depends(get_current_user)]
)
async def upload_file_chunk(chunk: UploadFile, custom_header_job_id=Header(...), settings: Settings = Depends(get_settings)):
filename = compose_upload_filename(custom_header_job_id, chunk.filename)
#filename = '_'.join([custom_header_job_id, chunk.filename])
path_to_file = os.path.join(settings.UPLOAD_FOLDER, filename)
try:
async with aiofiles.open(path_to_file, "ab") as input_file:
while content := await chunk.read(1024):
await input_file.write(content)
except FileNotFoundError:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="File chunk not found",
)
return {"hello": "world"}
The endpoint is not completed yet, since it is supposed to do other things besides receiving the file.
The frontend request starts from a Vue component:
uploadFileChunk({ file, jobId, startByte, onProgressUpdate = undefined }) {
const chunk = file.slice(startByte);
const formData = new FormData();
formData.append("chunk", chunk, file.name);
//formData.append("jobId", jobId);
/* return axios.post(`http://localhost:1234/upload`, formData, {
headers: {
"Custom-Header-Job-Id": jobId,
"Content-Disposition": `form-data; name="chunk"; filename="${file.name}"`,
"Content-Range": `bytes=${startByte}-${startByte + chunk.size}/${
file.size
}`,
},
onUploadProgress: onProgressUpdate,
}); */
return instance.post(`/files/upload`, formData, {
headers: {
"Custom-Header-Job-Id": jobId,
"Content-Disposition": `form-data; name="chunk"; filename="${file.name}"`,
"Content-Range": `bytes=${startByte}-${startByte + chunk.size}/${
file.size
}`,
},
onUploadProgress: onProgressUpdate,
});
}
const onProgressUpdate = (progress) => {
console.log("loaded: ", progress.loaded);
const percentage = Math.round(
(progress.loaded * 100) / file.size);
};
console.log("percentage: ", percentage);
The commented request points to a different Node.js backend with a file upload endpoint exclusively made to assess if the current issue I'm facing up is dependent on the client code or the backend. Here is the implementation (not an expert in Node.js and express):
const express = require("express");
const cors = require("cors");
const multer = require("multer");
const app = express();
app.use(express.json());
app.use(cors());
const upload = multer({ dest: "uploads/" });
app.post("/upload", upload.single("chunk"), (req, res) => {
res.json({ message: "Successfully uploaded files" });
});
app.listen(1234);
console.log("listening on port 1234");
In addition, the client code is actually put into a much more articulated pattern involving XState for managing the uploader component. Nevertheless, attached snippets should be enough to have an idea of the main parts to be discussed here.
Here is a screenshot for the request the FastAPI endpoint:
FastAPI
Where we can see the file is almost entirely uploaded and then the uploaded parameter drops to a lower upload percentage, eventually finalizing the upload (not shown here).
The same experiment repeated on the Node.js endpoint does not create issues, which splits the upload in much less packets:
Node.js express
It seems like the Node.js backend works fine, whereas the FastAPI doesn't. In my opinion there are some issues with how FastAPI/ starlette manages large files. It could be something related to the spooled file that starlette creates, or maybe something happening when passing from storing the file in the main memory to the mass memory. Unfortunately, starlette UploadFile class seems very hermetic and not easy to be customized/ inspected.
DETAILS
FastAPI backend running on a Debian Bullseye docker image
FastAPI version 0.78.0
python-multipart version 0.0.5
client and server running in localhost and tested with Chrome 103.0.5060.53 (Official Build) (x86_64)
system: Mac OSX 11.3.1
Thank you so much for your help!