Storing and serving images in NestJS - nestjs

I'm figuring out how to enable a local upload folder for a profile image.
Now, i've come as far as storing a file in the uploads folder. However, when I return that file it has no extension and its a JFIF binary file:
���� JFIF �� cmp3.10.3.3Lq4 0xa61382cc�� C
// etcetera...
The controller I've created is:
./src/controllers/user-profile.controller.ts
#Controller('user-profile')
export class UserProfileController {
#Post('/:id/upload-photo')
#UseInterceptors(FileInterceptor('image', { dest: './uploads' }))
uploadSinglePhoto(
#Param('id', ParseIntPipe) id: number,
#UploadedFile() image
) {
console.log('===> ', image);
return this.userProfileService.saveUserProfilePhotoLocation(id, image.path);
}
#Get('/:id/profile-photo')
getUserProfilePhoto(
): any {
// '15c924f42ffaa67b3f14a5be05f0a312' is the file name that is created by the upload
return new StreamableFile(createReadStream(join(process.cwd(), 'uploads', '15c924f42ffaa67b3f14a5be05f0a312')))
}
}
The image object in the console log is:
===> {
fieldname: 'image',
originalname: '526dad4edd250b689eeb1394c3c6eb41.jpg',
encoding: '7bit',
mimetype: 'image/jpeg',
destination: './uploads',
filename: '590b454b4315009660273deac082b4ed',
path: 'uploads\\590b454b4315009660273deac082b4ed',
size: 45842
}
Then I store the path to the database.
./src/services/user-profile.service.ts
#Injectable()
export class UserProfileService {
constructor(
#InjectRepository(UserProfileEntity)
private userProfileRepostory: Repository<UserProfileEntity>
) {}
async saveUserProfilePhotoLocation(id, path): Promise<UserProfileEntity> {
const result = await this.userProfileRepostory.createQueryBuilder()
.update(UserProfileEntity)
.set({
photo: path
})
.where({id})
.returning('*')
.execute();
return result.raw;
}
}
And finally I've configured the express server to be able to serve static files from the uploads folder:
./src/main.ts
async function bootstrap() {
const app = await NestFactory.create<NestExpressApplication>(AppModule);
app.useStaticAssets(join(__dirname, '..', 'uploads'), {
index: false,
prefix: 'uploads',
});
await app.listen('3000');
}
bootstrap();
Now when I do a get on the end point /:id/profile-photo it does not return me a rendered image. Ïnstead it returns a file with a random set of characters in it which is probably because of the encoding.
What should I do so that I can serve an jpeg file to my application?
What is happening here?

When setting the content header (thanks Jay McDoniel) to 'image/jpeg' and creating a readable stream it returns the image:
#Get('/:id/profile-photo')
getUserProfilePhoto(
#Res({ passthrough: true }) res: Response
): StreamableFile {
res.set({'Content-Type': 'image/jpeg'});
const imageLocation = join(process.cwd(), 'uploads', '15c924f42ffaa67b3f14a5be05f0a312');
const file = createReadStream(imageLocation);
return new StreamableFile(file);
}

Related

Nestjs throw error and http status 400 when fileFilter in fileinterceptor finds an unsupported file extension

I am having difficulty handling an error in my application.
I want to create a FileInterceptor (multer) which checks if the file has an allowed file extension. The documentation shows this:
const allowedFileExtensions = ['.jpg', '.png'];
FileInterceptor(
'image',
{
dest: './uploads',
fileFilter: (req, file, callback) => {
const extension = path.extname(file.originalname);
if (allowedFileExtensions.includes(extension)) {
callback(null, true);
} else {
// gives the 500 error
callback(new Error('Only images are allowed'), false);
}
}
}
This kinda works. But it has two flaws in my opinion. First, it returns a 500 error:
{
"statusCode": 500,
"message": "Internal server error"
}
It would be better to return a 400 error with a error message which explains why it failed.
Second, in the console of the Nest application it shows the stacktrace. I'd rather use the logging for this instead. (The application has middleware in place to pick up NestJS errors and log them automagically.)
So what I am trying is the following:
const allowedFileExtensions = ['.jpg', '.png'];
FileInterceptor(
'image',
{
dest: './uploads',
fileFilter: (req, file, callback) => {
const extension = path.extname(file.originalname);
if (!allowedFileExtensions.includes(extension)) {
// crashes the application
throw new BadRequestException('Only images are allowed', `Bad request. Accepted file extensions are: ${allowedFileExtensions.toString()}`);
}
callback(null, true)
}
}
But this crashes the application. I am not able to upload a different image anymore after this.
Any idea?
Ah found it. The BadRequestException cannot be thrown outside the body of the function.
const allowedFileExtensions = ['.jpg', '.png'];
enum FileValidationErrors {
UNSUPPORTED_FILE_TYPE
}
#Post('/:id/upload-photo')
#UseInterceptors(
FileInterceptor(
'image',
{
dest: './uploads',
fileFilter: (req, file, callback) => {
const extension = path.extname(file.originalname);
if (allowedFileExtensions.includes(extension)) {
callback(null, true);
} else {
// provide the validation error in the request
req.fileValidationError = FileValidationErrors.UNSUPPORTED_FILE_TYPE
callback(null, false);
}
}
}
)
)
uploadSinglePhoto(
#Param('id', ParseIntPipe) id: number,
#UploadedFile() image,
#Req() req // add the request property
): Promise<UserProfileEntity> {
// check here for a potential error
if (req?.fileValidationError === FileValidationErrors.UNSUPPORTED_FILE_TYPE) {
// if so, throw the BadRequestException
throw new BadRequestException('Only images are allowed', `Bad request. Accepted file extensions are: ${allowedFileExtensions.toString()}`);
}
this.logger.verbose(`Adding image ${image}`);
const imageUrlLocation = `${image.destination.substring(1)}/${image.filename}`;
return this.userProfileService.saveUserProfilePhotoLocation(id, imageUrlLocation);
}

Image uploaded works in localhost but not on server (s3)

Well, I have an option in the system where I can see some documents sent by the user. But the images appear as broken icons.
As "localhost" everything works great, but I can't see the images when I run straight from the website. (On the s3 everything is working normally, I can see the images.)
In the brownser console, an error is thrown about the route that the request is being made "api.mysite.com/the-image-example.png" with status code 404.
What I don't understand is why the request is being made there and not directly on amazon s3 (there I have the images loading normally). Someone cal help me with that??
How images appear:
This is the code
Upload config:
const tmpFolder = path.resolve(__dirname, '..', '..', 'tmp');
export const mimeTypesPhotos = [
'image/png',
'image/jpeg',
'image/bmp',
'image/webp',
];
export const mimeTypesVideos = ['video/webm', 'video/ogg', 'video/mp4'];
export const mimeTypesPng = ['application/pdf'];
const pngRoutes = ['/requests/test', '/test/:id/confirm'];
interface IUploadConfig {
driver: 's3' | 'disk';
tmpFolder: string;
uploadsFolder: string;
multer: multer.Options;
image: {
height: number;
width: number;
};
config: {
disk: unknown;
aws: {
bucket: string;
};
};
}
export default {
driver: process.env.STORAGE_DRIVER,
tmpFolder,
uploadsFolder: path.resolve(tmpFolder, 'uploads'),
multer: {
storage: multer.diskStorage({
destination: tmpFolder,
filename: (request, file, callback) => {
const fileHash = crypto.randomBytes(10).toString('hex');
const fileName = `${fileHash}-${file.originalname}`;
return callback(null, fileName);
},
}),
fileFilter: (request, file, callback) => {
const url = matchId(request.originalUrl);
const mimetypes = [...mimeTypesPhotos];
if (pngRoutes.includes(url)) {
mimetypes.push(...mimeTypesPng);
}
if (!mimetypes.includes(file.mimetype)) {
return callback(new Error("File doesn't supported"));
}
return callback(null, true);
},
},
image: {
width: Number(process.env.MAX_IMAGE_SIZE || '1024'),
height: Number(process.env.MAX_IMAGE_SIZE || '1024'),
},
config: {
disk: {},
aws: {
bucket: process.env.AWS_BUCKET || 'mybucket',
},
},
} as IUploadConfig;
Router file (where is calling the route i mean):
app.use('/files', express.static(uploadConfig.uploadsFolder));
The Media Entity:
#Column({ type: 'varchar' })
type: 'photo';
#Column()
media: string;
#Expose({ name: 'mediaUrl' })
getMediaUrl(): string | null {
if (!this.media) {
return null;
}
switch (uploadConfig.driver) {
case 'disk':
return `${process.env.APP_API_URL}/files/${this.media}`;
case 's3':
return `https://${uploadConfig.config.aws.bucket}.s3.amazonaws.com/${this.media}`;
default:
return null;
}
}
Step-1
we need to Enable CORS in the API gateway, so click on the resource and then click on the Action button and Enable CORS. In CORS settings the value of Access-Control-Allow-Headers is '' and Access-Control-Allow-Origin is '' , and leave other settings as it is. Click on Enable CORS and replace existing CORS headers.
Step-2
Now, go to the settings of that API on which you are working. In settings enable Binary Media Types. Set the value of that field like this / , and Save the changes.

multer: How to process and manipulate files before saving?

I want to process images using sharp before saving so I don't need to save the file twice (one by multer and one by sharp). The best way I found is to save the file in memory with initializing multer with no argument:
const upload = multer()
and then giving the file buffer to sharp in the route handler:
await sharp(req.file.buffer)
.resize(500)
.jpeg({ quality: 50 })
.toFile(path)
);
If is there a better way let me know.
The better question would be:
Is there something like a hook in multer that calls before saving? so I can change the file content before saving it.
the process is start from fileFilter method then fileName of multer so you should start resize it in fileFilter save it to temporary directory then reszie image after resize you can use method .ToFile of sharp package it will save the destination inside .toFile("./upload")
file-mapper.ts (use for return another object)
export const fileMapper = ({ file, req }: FileMapper) => {
if (!file) {
return null!;
}
const fileName = editFilename(req.file);
const image_url = `${req.protocol}://${req.headers.host}/${req.file.destination}/${fileName}`;
resizeImage(req, fileName);
return {
filename: file.filename,
image_url,
path: file.path,
};
};
my resize-image.ts
import { Request } from 'express';
import path from 'path';
import sharp from 'sharp';
import fs from 'fs';
export const resizeImage = async (req: Request, fileName: string) => {
await sharp(req.file.path)
.resize({ height: 700, width: 700, position: 'center' })
.jpeg({ quality: 90, chromaSubsampling: '4:4:4' })
.toFile(path.resolve(req.file.destination, fileName))
.then((sharpOutPut) => {
if (sharpOutPut) fs.unlinkSync(req.file.path);
})
.catch((err) => {
throw err;
});
};
disk-storage.ts
const storage= diskStorage({
destination: 'upload/product',
filename: editFileName,
}),
fileFilter: imageFileFilter,
})
use sharp in fileFilter of multer
I'm using fileMapper because I want to receive new object for my db

how to store images in unique folder(using primary key) for each upload using multer in nest js

I have an application where i want to store multipart form data along with image. I want images to be stored in unique folder which is primary id. How can i do it.
Here i am saving multiform data in database. I want to use primary id return by below function to create folder to save its respective images.
async createFaceDetection(faceDetectionReqDto: FaceDetectionReqDto): Promise<Object> {
try {
const newFaceDetection = new face_detection({
Name: faceDetectionReqDto.Name
})
const savedMediaUpload = await this.FaceDetectionRepository.save(newFaceDetection);
return savedMediaUpload
}
catch (err) {
throw err;
}
}
how can i use primary id fetched by "savedMediaUpload" variable in multer
my component.controller.ts code
#UseInterceptors(
FilesInterceptor('Images', 20,
{
storage: diskStorage({
destination: async function (req, file, cb) {
const filePath = path.join(ROOT_DIR, FACE_DETECTION_PATH, {Primary-Id}) // (primary id
generated by above function.)
if (!existsSync(`${filePath}`)) { mkdirSync(filePath, { recursive: true }) }
cb(null, filePath);
},
filename: (_req, file, cb) => {
return cb(null, file.originalname);
},
}),
}),
)
I've done it like this, by appending generated UUID to the file name - may not be perfect but works good enough for me so far:
Controller:
#UseInterceptors(
FilesInterceptor('asset', 10, {
storage: diskStorage({
destination: UPLOADS_DIR,
filename: getUniqueFileName,
}),
limits: {
fileSize: MAX_FILE_SIZE_BYTES,
},
fileFilter: validateImageFile,
}),
)
#Post()
async createAsset(
#Body() assetCreateRequest: AssetCreateRequest,
#UploadedFile() asset: UploadedFileModel,
): Promise<AssetResponse[]> {
....
}
And getUniqueFileName looks like this:
import { extname } from 'path';
import { v4 as uuidv4 } from 'uuid';
import { UploadedFileModel } from '../../dto/internal/file-upload';
export const getUniqueFileName = (
_request: unknown,
file: UploadedFileModel,
callback: (error: Error | null, fileName: string) => void,
) => {
const name = file.originalname.split('.')[0];
const extension = extname(file.originalname);
const randomUuid = uuidv4();
callback(null, `${name}-${randomUuid}${extension}`);
};
Later on in the database, I have separate Asset and e.g. User tables where User references a record in the Asset table by foreign key, and Asset record contains all the necessary info - file name, path, URL etc.

How to get rid of "TypeError: req.pipe is not a function" nestjs and fastify

I tried to upload file with NestJS/Fastify and typescript
this is main.ts
async function bootstrap() {
//file upload with fastify
const fastifyAdapter = new FastifyAdapter();
fastifyAdapter.register(fmp, {
limits: {
fieldNameSize: 100, // Max field name size in bytes
fieldSize: 1000000, // Max field value size in bytes
fields: 10, // Max number of non-file fields
fileSize: 100, // For multipart forms, the max file size
files: 1, // Max number of file fields
headerPairs: 2000, // Max number of header key=>value pairs
},
});
const app = await NestFactory.create<NestFastifyApplication>(
AppModule,
fastifyAdapter,
);
await app.listen(3000);
Logger.log('application started on http://localhost:3000', 'Bootstrap');
}
bootstrap();
and this is file.controller.ts
#Post()
#UseInterceptors(FileInterceptor('image'))
#ApiConsumes('multipart/form-data')
#ApiBody({
description: 'logo',
type: UploadFileDto,
})
uploadedFile(#UploadedFile() file) {
const response = {
originalname: file.originalname,
filename: file.filename,
};
return response;
}
after uploading a file to this action, code throw an exception like this
TypeError: req.pipe is not a function
at multerMiddleware (D:\R.Khodabakhshi\Repository\raimun-web\node_modules\multer\lib\make-middleware.js:176:9)
at Promise (D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\platform-express\multer\interceptors\file.interceptor.js:15:81)
at new Promise ()
at MixinInterceptor.intercept (D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\platform-express\multer\interceptors\file.interceptor.js:15:19)
at D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\interceptors\interceptors-consumer.js:22:36
at Object.handle (D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\interceptors\interceptors-consumer.js:20:56)
at LoggingInterceptor.intercept (D:\R.Khodabakhshi\Repository\raimun-web\dist\shared\logging.interceptor.js:28:21)
at D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\interceptors\interceptors-consumer.js:22:36
at InterceptorsConsumer.intercept (D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\interceptors\interceptors-consumer.js:24:24)
at D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\router\router-execution-context.js:45:60
[Nest] 10928 - 2020-02-06 10:10:49 [ExceptionFilter] undefined undefined +587529ms
TypeError: req.pipe is not a function
at multerMiddleware (D:\R.Khodabakhshi\Repository\raimun-web\node_modules\multer\lib\make-middleware.js:176:9)
at Promise (D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\platform-express\multer\interceptors\file.interceptor.js:15:81)
at new Promise ()
at MixinInterceptor.intercept (D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\platform-express\multer\interceptors\file.interceptor.js:15:19)
at D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\interceptors\interceptors-consumer.js:22:36
at Object.handle (D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\interceptors\interceptors-consumer.js:20:56)
at LoggingInterceptor.intercept (D:\R.Khodabakhshi\Repository\raimun-web\dist\shared\logging.interceptor.js:28:21)
at D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\interceptors\interceptors-consumer.js:22:36
at InterceptorsConsumer.intercept (D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\interceptors\interceptors-consumer.js:24:24)
at D:\R.Khodabakhshi\Repository\raimun-web\node_modules#nestjs\core\router\router-execution-context.js:45:60
how can I fix the problem???
You cannot use the FastifyAdapter with the FileInterceptor. It says so in the beginning of the docs. If you want to use Fastify and a file upload, you'll need to create your own interceptor for it.
Problem solved, as Jay McDaniel mentioned we couldn't use the FastifyAdapter with the FileInterceptor.
I resolved the problem with this little code.
import {
Controller,
Logger,
Post,
Req,
Res,
} from '#nestjs/common';
import * as fs from 'fs';
import * as path from 'path';
import * as pump from 'pump';
const logger = new Logger('FileController');
#ApiTags('File')
#Controller('api/file')
export class FileController {
#Post()
upload(#Req() req: any, #Res() reply: any): void {
const mp = req.multipart(
(field: any, file: any, filename: any, encoding: any, mimeType: any) => {
console.log('save file from request ---- ', field, filename, mimeType);
file.on('limit', () => logger.error('SIZE_LIMITED'));
const filePath = path.resolve('./'+filename);
const writeStream = fs.createWriteStream(filePath);
pump(file, writeStream);
writeStream.on('finish', () => {
reply.code(200).send();
});
},
(error: any) => {
if (error) {
logger.error(error);
reply.code(500).send();
}
},
);
mp.on('partsLimit', () => logger.error('MAXIMUM_NUMBER_OF_FORM_PARTS'));
mp.on('filesLimit', () => logger.error('MAXIMUM_NUMBER_OF_FILES'));
mp.on('fieldsLimit', () => logger.error('MAXIMUM_NUMBER_OF_FIELD'));
}
}
I hope this will help you too...

Resources