Since nestjs is an express app, it's possible to use any library to handle upload using nest, and since it provides Midlewares, it's also possible to use multer. My question is: What's the best way to handle file uploads using nestjs?
As informed by #Kamyl on issue https://github.com/nestjs/nest/issues/262, since v4.6.0 is possible to upload files using multer to nestjs using a common file interceptor.
import { ... , UseInterceptors, FileInterceptor, UploadedFile } from '#nestjs/common'
...
#UseInterceptors(FileInterceptor('file'))
async upload( #UploadedFile() file) {
console.log(file)
}
This way the variable file will have a buffer
Using Multer options
It's also needed the field name as the first param, then an array with Multer Options
import { ... , UseInterceptors, FileInterceptor, UploadedFile } from '#nestjs/common'
import { diskStorage } from 'multer'
import { extname } from 'path'
...
#UseInterceptors(FileInterceptor('file', {
storage: diskStorage({
destination: './uploads'
, filename: (req, file, cb) => {
// Generating a 32 random chars long string
const randomName = Array(32).fill(null).map(() => (Math.round(Math.random() * 16)).toString(16)).join('')
//Calling the callback passing the random name generated with the original extension name
cb(null, `${randomName}${extname(file.originalname)}`)
}
})
}))
async upload( #UploadedFile() file) {
console.log(file)
}
This way the variable file will have a filename, destination and path.
The destination param from the diskStorage can also be a function, with the parameters and expecting the callback the same as filename. By passing a diskStorage the file will be automatically saved to the destination informed with the filename given.
It's also possible to handle multiple files by using #UploadedFiles and FilesInterceptor (plural)
A cleaner way would be to extract the configurations to a separate file and then call it inside the interceptor method
import { extname } from 'path';
import { existsSync, mkdirSync } from 'fs';
import { diskStorage } from 'multer';
import { v4 as uuid } from 'uuid';
import { HttpException, HttpStatus } from '#nestjs/common';
// Multer configuration
export const multerConfig = {
dest: process.env.UPLOAD_LOCATION,
};
// Multer upload options
export const multerOptions = {
// Enable file size limits
limits: {
fileSize: +process.env.MAX_FILE_SIZE,
},
// Check the mimetypes to allow for upload
fileFilter: (req: any, file: any, cb: any) => {
if (file.mimetype.match(/\/(jpg|jpeg|png|gif)$/)) {
// Allow storage of file
cb(null, true);
} else {
// Reject file
cb(new HttpException(`Unsupported file type ${extname(file.originalname)}`, HttpStatus.BAD_REQUEST), false);
}
},
// Storage properties
storage: diskStorage({
// Destination storage path details
destination: (req: any, file: any, cb: any) => {
const uploadPath = multerConfig.dest;
// Create folder if doesn't exist
if (!existsSync(uploadPath)) {
mkdirSync(uploadPath);
}
cb(null, uploadPath);
},
// File modification details
filename: (req: any, file: any, cb: any) => {
// Calling the callback passing the random name generated with the original extension name
cb(null, `${uuid()}${extname(file.originalname)}`);
},
}),
};
and then call it under the interceptor like so
import { ... , UseInterceptors, FileInterceptor, UploadedFile } from '#nestjs/common'
import { diskStorage } from 'multer'
import { extname } from 'path'
import { multerOptions } from 'src/config/multer.config';
...
#Post('/action/upload')
#UseInterceptors(FileInterceptor('file', multerOptions))
async upload( #UploadedFile() file) {
console.log(file)
}
Cleanest implementation using Multer options
Thank you #VictorIvens for the best answer out of the bunch.
However, I found the following problems in the code.
the import called FileInterceptor does not exist withing #nestjs/common package int the latest version of NestJS.
the code looks a bit too cluttered to my eyes.
So, to simplify the things up, I have come up with the following solution.
storage.config.ts
export const storage = diskStorage({
destination: "./uploads",
filename: (req, file, callback) => {
callback(null, generateFilename(file));
}
});
function generateFilename(file) {
return `${Date.now()}.${extname(file.originalname)}`;
}
your-controller.controller.ts
import {
Controller,
Post,
UseInterceptors,
UploadedFile
} from "#nestjs/common";
import { FileInterceptor } from "#nestjs/platform-express";
import { diskStorage } from "multer";
import { extname } from "path";
import { storage } from "./storage.config"
#Controller()
export class YourController {
#Post("upload") // API path
#UseInterceptors(
FileInterceptor(
"file", // name of the field being passed
{ storage }
)
)
async upload(#UploadedFile() file) {
return file;
}
}
**
2021 Update
**
To do this now, you need to import FileInterceptor like this...
import { FileInterceptor } from '#nestjs/platform-express';
If you are getting the data from the user via API call, you can save the data as buffer and access the content using adm-zip. Below is the controller method implementation in nest.js.
#Post("/blackBoardUpload")
#UseInterceptors(
FileInterceptor('image', {
storage: memoryStorage(),
fileFilter: zipFileFilter,
}),
)
async uploadedFile(#UploadedFile() file) {
console.log(file)
const response = {
originalname: file.originalname,
filename: file.filename,
};
var AdmZip = require('adm-zip');
var zip = new AdmZip(file.buffer);
var zipEntries = zip.getEntries();
console.log(zipEntries.length);
return {
status: HttpStatus.OK,
message: 'Received Zip file successfully!',
data: response,
};
}
Create a helper.ts file that rename your file and contains path
export class Helper {
static customFileName(req, file, cb) {
const uniqueSuffix = Date.now() + '-' + Math.round(Math.random() * 1e9);
let fileExtension = "";
if(file.mimetype.indexOf("jpeg") > -1){
fileExtension = "jpg"
}else if(file.mimetype.indexOf("png") > -1){
fileExtension = "png";
}
const originalName = file.originalname.split(".")[0];
cb(null, originalName + '-' + uniqueSuffix+"."+fileExtension);
}
static destinationPath(req, file, cb) {
cb(null, 'uploads/')
}
}
code for controller
import { Helper } from '../service/Helper';
import { diskStorage } from 'multer';
import {FileInterceptor} from '#nestjs/platform-express'
import {Controller, Post, Body, UseInterceptors, UploadedFile} from '#nestjs/common'
#Post('upload')
#UseInterceptors(
FileInterceptor('picture', {
storage: diskStorage({
destination: Helper.destinationPath,
filename: Helper.customFileName,
}),
}),
)
uploadFile(#UploadedFile() file: Express.Multer.File) {
console.log(file);
}
A simple way is to use controllers. You need to define an upload controller and add it in your app.module, this is an example of what a controller should be (back-end):
#Controller()
export class Uploader {
#Post('sampleName')
#UseInterceptors(FileInterceptor('file'))
uploadFile(#UploadedFile() file) {
// file name selection
const path = `desired path`;
const writeStream = fs.createWriteStream(path);
writeStream.write(file.buffer);
writeStream.end();
return {
result: [res],
};
}
}
And call your controller by fetch in the front-end:
fetch('controller address', {
method: 'POST',
body: data,
})
.then((response) => response.json())
.then((success) => {
// What to do when succeed
});
})
.catch((error) => console.log('Error in uploading file: ', error));
Related
I want to register MulterModule twice in the same module because I have two types of uploads: a video and a thumbnail. I tried this implementation it did not work. Is it even possible?
video.module.ts
import { Module } from '#nestjs/common';
import { VideoController } from './video.controller';
import { MulterModule } from '#nestjs/platform-express';
import VideoDiskStorage from './multer/storage/video-disk-storage.service';
import ThumbnailDiskStorage from './multer/storage/thumbnail-disk-storage';
#Module({
imports: [
MulterModule.registerAsync({
useClass: VideoDiskStorage,
}),
MulterModule.registerAsync({
useClass: ThumbnailDiskStorage,
}),
],
controllers: [VideoController],
})
export class VideoModule {}
video-disk-storage.ts
import multer, { StorageEngine } from 'multer';
import { nanoid } from 'nanoid';
import path from 'node:path';
import { BadRequestException, Injectable } from '#nestjs/common';
import { ensureDir } from 'fs-extra';
import {
MulterModuleOptions,
MulterOptionsFactory,
} from '#nestjs/platform-express';
#Injectable()
class VideoDiskStorage implements MulterOptionsFactory {
private dir: string;
createMulterOptions(): Promise<MulterModuleOptions> | MulterModuleOptions {
return {
storage: this.diskStorage(),
limits: {
fileSize: 1024 * 1024 * 1024 * 10,
},
};
}
private diskStorage(): StorageEngine {
return multer.diskStorage({
filename: async (req, file, cb) => {
// eslint-disable-next-line #typescript-eslint/ban-ts-comment
// #ts-ignore
const userId = req.session?.getUserId();
if (!userId) {
return cb(new BadRequestException('Invalid user'), null);
}
const mimeType = file.mimetype.split('/')[0];
if (mimeType !== 'video') {
return cb(
new BadRequestException('only video files are allowed'),
null,
);
}
const fileName = nanoid() + path.extname(file.originalname);
const dir = path.join(
process.cwd(),
'tmp',
'uploads',
'videos',
'original',
userId,
);
await ensureDir(dir);
this.dir = dir;
cb(null, fileName);
},
destination: this.dir,
});
}
}
export default VideoDiskStorage;
thumbnail-disk-storage.ts
import multer, { StorageEngine } from 'multer';
import { nanoid } from 'nanoid';
import path from 'node:path';
import { BadRequestException, Injectable } from '#nestjs/common';
import { ensureDir } from 'fs-extra';
import {
MulterModuleOptions,
MulterOptionsFactory,
} from '#nestjs/platform-express';
#Injectable()
class ThumbnailDiskStorage implements MulterOptionsFactory {
private dir: string;
createMulterOptions(): Promise<MulterModuleOptions> | MulterModuleOptions {
return {
storage: this.diskStorage(),
limits: {
fileSize: 1024 * 2,
},
};
}
private diskStorage(): StorageEngine {
return multer.diskStorage({
filename: async (req, file, cb) => {
console.log('file', file);
// eslint-disable-next-line #typescript-eslint/ban-ts-comment
// #ts-ignore
const userId = req.session?.getUserId();
if (!userId) {
return cb(new BadRequestException('Invalid user'), null);
}
const mimeType = file.mimetype.split('/')[0];
console.log('mimeType', mimeType);
if (mimeType !== 'img') {
return cb(
new BadRequestException('only images files are allowed'),
null,
);
}
const thumbnailName = nanoid() + path.extname(file.originalname);
const dir = path.join(
process.cwd(),
'tmp',
'thumbnails',
'original',
userId,
);
await ensureDir(dir);
this.dir = dir;
cb(null, thumbnailName);
},
destination: this.dir,
});
}
}
export default ThumbnailDiskStorage;
The problem with this implementation is first registered MulterModule is overwritten by the second. So how can I use both of them?
I have created a route for uploading files to an S3 bucket, which is working perfectly. However, when I try and add it into my Accommodation controller with additional logic it is causing the error Cannot read properties of undefined (reading 'map'). I am using the exact same request in postman for each route.
Can anyone spot why this is happening?
My original logic for the upload controller:
import { Request, Response } from "express";
import catchBlock from "../utils/catchBlock";
import { s3Uploadv2 } from "../utils/s3Service";
const UploadController = {
Upload: async (req: Request, res: Response) => {
const files = req.files as Express.Multer.File[];
try {
const results = await s3Uploadv2(files);
res.send({ success: "successful", results });
} catch (e: unknown) {
catchBlock(e, res);
}
},
};
export default UploadController;
My accommodation upload:
UploadImages: async (req: Request, res: Response) => {
const accommodationId = req.params.id;
const accommodation = await accommodationSchema.findOne({
_id: accommodationId,
});
const files = req.files as Express.Multer.File[];
try {
const results = await s3Uploadv2(files);
console.log(results);
if (accommodation) {
results.forEach((file) => accommodation.photos.push(file.Location));
await accommodation.save();
res.send({ success: "successful", accommodation });
} else {
res.status(400).send("No such accommodation");
}
res.send({ success: "successful", results });
} catch (e: unknown) {
catchBlock(e, res);
}
},
S3 service:
import { S3 } from "aws-sdk";
import { v4 as uuid } from "uuid";
export interface Param {
Bucket: string;
Key: string;
Body: Buffer;
}
export const s3Uploadv2 = async (files: Express.Multer.File[]) => {
const s3 = new S3();
const params: Param[] = files.map((file) => {
return {
Bucket: process.env.AWS_BUCKET_NAME,
Key: `uploads/${uuid()}-${file.originalname}`,
Body: file.buffer,
};
});
const results = await Promise.all(
params.map((param) => s3.upload(param).promise())
);
return results;
};
Multer service:
import multer, { FileFilterCallback, MulterError } from "multer";
import { Request } from "express";
const storage = multer.memoryStorage();
const fileFilter = (
req: Request,
file: Express.Multer.File,
cb: FileFilterCallback
) => {
if (file.mimetype.split("/")[0] === "image") {
cb(null, true);
} else {
cb(new MulterError("LIMIT_UNEXPECTED_FILE"));
}
};
export const upload = multer({
storage,
fileFilter,
limits: { fileSize: 1000000, files: 5 },
});
Accommodation Route:
import express from "express";
import AccommodationController from "../controllers/accommodation";
const accommodationRouter = express.Router();
accommodationRouter.get("/", AccommodationController.All);
accommodationRouter.post(
"/create",
AccommodationController.CreateAccommodation
);
accommodationRouter.get(
"/users-accommodation",
AccommodationController.UsersAccommodation
);
accommodationRouter.post("/delete/:id", AccommodationController.Delete);
accommodationRouter.post("/upload/:id", AccommodationController.UploadImages);
export default accommodationRouter;
Upload Route:
import express from "express";
import UploadController from "../controllers/upload";
import { upload } from "../utils/multer";
const uploadRouter = express.Router();
uploadRouter.post("/", upload.array("file", 5), UploadController.Upload);
export default uploadRouter;
realised my mistake was not putting my upload middleware on the accommodation route.
changed this:
accommodationRouter.post("/upload/:id", AccommodationController.UploadImages);
to this:
accommodationRouter.post(
"/upload/:id",
upload.array("file", 5),
AccommodationController.UploadImages
);
I use multer to manage uploaded file:
#Post('upload') #UseInterceptors(FilesInterceptor("images", 10, {
dest: "./uploads",
}))
uploadMultiple(#UploadedFiles() files) {
console.log(files, 'test');
}
I try to add a file extension to my uploaded files as:
#Post('upload') #UseInterceptors(FilesInterceptor("images", 10, {
dest: "./uploads",
filename: function (req, file, cb) {
cb(null, Date.now() + '.jpg') //Appending .jpg
}
}))
But when I do this I get an error:
TS2345: Argument of type '{ dest: string; filename: (req: any, file: any, cb: any) => void; }' is not assignable to parameter of type 'MulterOptions'. Object literal may only specify known properties, and 'filename' does not exist in type 'MulterOptions'
How to specify file extension to my uploaded files?
You can do it dynamically using the extname import from path using the diskStorage to get your filename extension.
import { extname } from 'path';
import { diskStorage } from 'multer';
export const exampleDiskStorage = diskStorage({
destination: './public/img/users',
filename: (req, file, cb) => {
return cb(null, `${Date.now()}${extname(file.originalname)}`);
}
});
In your module, you only need to import the diskStorage to MulterModule.register
import { MulterModule } from '#nestjs/platform-express';
#Module({
imports: [
MulterModule.register({
storage: exampleDiskStorage,
}),
],
});
I am creating a interceptor Inside that I want to use FileInterceptor but I am getting error Declaration Expected after FileInterceptor
import { CallHandler, ExecutionContext, Injectable, NestInterceptor, UseInterceptors } from '#nestjs/common';
import { Observable } from 'rxjs';
import { FileInterceptor } from '#nestjs/platform-express';
import { diskStorage } from 'multer';
import { extname } from 'path';
#Injectable()
export class UploaderInterceptor implements NestInterceptor {
intercept(context: ExecutionContext, next: CallHandler): Observable<any> {
#UseInterceptors(FileInterceptor('file', {
storage: diskStorage({
destination: './uploads'
, filename: (req, file, cb) => {
// Generating a 32 random chars long string
const randomName = Array(32).fill(null).map(() => (Math.round(Math.random() * 16)).toString(16)).join('')
//Calling the callback passing the random name generated with the original extension name
cb(null, `${randomName}${extname(file.originalname)}`)
}
})
}));
return next.handle();
}
}
FileInterceptor is a mixin meaning it is a function that returns a class. This function takes in a configuration for the interceptor to actually use. What you can do instead of trying to make a class that makes use of the interceptor under the hood or extends it, is essentially make an alias for the configuration like so:
export const UploadInterceptor = FileInterceptor('file', {
storage: diskStorage({
destination: './uploads',
filename: (req, file, cb) => {
// Generating a 32 random characters long string
const randomName = Array(32).fill(null).map(() => (Math.round(Math.random() * 16)).toString(16)).join('')
//Calling the callback passing the random name generated with the original extension name
cb(null, `${randomName}${extname(file.originalname)}`)
}
})
})
Now with this exported constant (that is actually a class), you can then make use of the interceptor as such:
#Controller('upload')
export class UploadController {
#UseInterceptors(UploadInterceptor)
#Post()
uploadFile() {
// do logic;
}
}
And everything should work out for you.
I am trying to upload a file to my nest.js server but I am getting an error:
Error: Unsupported Media Type: multipart/form-data; boundary=--------------------------140603536005099484714904
I followed this documentation.
Angular 6 Code
public async Upload<TReponse>(file: File, path) {
try {
const formData = new FormData();
formData.append("file", file);
//>> function to upload and applying header as null
const result = await this.http.post(this.baseUrl + path, formData, { headers: null }).pipe(map((response: any) => response)).toPromise();
var response = result as ApiResponseObject<TReponse>;
return response;
}
catch (ex) {
var result = new ApiResponseObject<TReponse>()
result.message = ex.message;
result.isSuccess = false;
return result;
}
finally {
}
}
NestJs Code Module Code
import { Module, NestModule, MiddlewareConsumer, MulterModule } from '#nestjs/common';
import { UserService } from './services/user.service';
import { UserController } from './user.controller';
import { TypeOrmModule } from '#nestjs/typeorm';
import { User } from './dto/user.entity';
import * as multer from 'multer';
#Module({
imports: [TypeOrmModule.forFeature([User]),
MulterModule.register({
dest: '/public',
fileFilter: (req, file, cb) => {
let extension = (file.originalname.split('.').pop())
//Put here your custom validation for file extensións.
// To accept the file pass `true`, like so:
cb(null, true);
// To reject this file pass `false` or throw Exception, like so:
//cb(new HttpException ("File format is not valid", HttpStatus.BAD_REQUEST), false)
},
limits: {
fileSize: 2097152 //2 Megabytes
},
storage: multer.diskStorage({
destination(req, file, cb) {
cb(null, './public');
},
filename(req, file, cb) {
cb(null, "usman_" + file.originalname);
},
})
}),
],
providers: [UserService],
controllers: [UserController],
})
NestJs Code Component Code
#Post("upload")
#UseInterceptors(FileInterceptor('file'))
async upload(#UploadedFile() file, #Request() req) {
console.log(file);
console.log(req.files);
}
I assume you are using the FastifyAdapter instead of express.
In the documentation it says:
Multer will not process any form which is not multipart
(multipart/form-data). Besides, this package won't work with the
FastifyAdapter.
So if possible, switch to express. Otherwise, you have to use Fastify-Multipart until it is natively supported by nest.js.