Image uploaded works in localhost but not on server (s3) - node.js

Well, I have an option in the system where I can see some documents sent by the user. But the images appear as broken icons.
As "localhost" everything works great, but I can't see the images when I run straight from the website. (On the s3 everything is working normally, I can see the images.)
In the brownser console, an error is thrown about the route that the request is being made "api.mysite.com/the-image-example.png" with status code 404.
What I don't understand is why the request is being made there and not directly on amazon s3 (there I have the images loading normally). Someone cal help me with that??
How images appear:
This is the code
Upload config:
const tmpFolder = path.resolve(__dirname, '..', '..', 'tmp');
export const mimeTypesPhotos = [
'image/png',
'image/jpeg',
'image/bmp',
'image/webp',
];
export const mimeTypesVideos = ['video/webm', 'video/ogg', 'video/mp4'];
export const mimeTypesPng = ['application/pdf'];
const pngRoutes = ['/requests/test', '/test/:id/confirm'];
interface IUploadConfig {
driver: 's3' | 'disk';
tmpFolder: string;
uploadsFolder: string;
multer: multer.Options;
image: {
height: number;
width: number;
};
config: {
disk: unknown;
aws: {
bucket: string;
};
};
}
export default {
driver: process.env.STORAGE_DRIVER,
tmpFolder,
uploadsFolder: path.resolve(tmpFolder, 'uploads'),
multer: {
storage: multer.diskStorage({
destination: tmpFolder,
filename: (request, file, callback) => {
const fileHash = crypto.randomBytes(10).toString('hex');
const fileName = `${fileHash}-${file.originalname}`;
return callback(null, fileName);
},
}),
fileFilter: (request, file, callback) => {
const url = matchId(request.originalUrl);
const mimetypes = [...mimeTypesPhotos];
if (pngRoutes.includes(url)) {
mimetypes.push(...mimeTypesPng);
}
if (!mimetypes.includes(file.mimetype)) {
return callback(new Error("File doesn't supported"));
}
return callback(null, true);
},
},
image: {
width: Number(process.env.MAX_IMAGE_SIZE || '1024'),
height: Number(process.env.MAX_IMAGE_SIZE || '1024'),
},
config: {
disk: {},
aws: {
bucket: process.env.AWS_BUCKET || 'mybucket',
},
},
} as IUploadConfig;
Router file (where is calling the route i mean):
app.use('/files', express.static(uploadConfig.uploadsFolder));
The Media Entity:
#Column({ type: 'varchar' })
type: 'photo';
#Column()
media: string;
#Expose({ name: 'mediaUrl' })
getMediaUrl(): string | null {
if (!this.media) {
return null;
}
switch (uploadConfig.driver) {
case 'disk':
return `${process.env.APP_API_URL}/files/${this.media}`;
case 's3':
return `https://${uploadConfig.config.aws.bucket}.s3.amazonaws.com/${this.media}`;
default:
return null;
}
}

Step-1
we need to Enable CORS in the API gateway, so click on the resource and then click on the Action button and Enable CORS. In CORS settings the value of Access-Control-Allow-Headers is '' and Access-Control-Allow-Origin is '' , and leave other settings as it is. Click on Enable CORS and replace existing CORS headers.
Step-2
Now, go to the settings of that API on which you are working. In settings enable Binary Media Types. Set the value of that field like this / , and Save the changes.

Related

Storing and serving images in NestJS

I'm figuring out how to enable a local upload folder for a profile image.
Now, i've come as far as storing a file in the uploads folder. However, when I return that file it has no extension and its a JFIF binary file:
���� JFIF �� cmp3.10.3.3Lq4 0xa61382cc�� C
// etcetera...
The controller I've created is:
./src/controllers/user-profile.controller.ts
#Controller('user-profile')
export class UserProfileController {
#Post('/:id/upload-photo')
#UseInterceptors(FileInterceptor('image', { dest: './uploads' }))
uploadSinglePhoto(
#Param('id', ParseIntPipe) id: number,
#UploadedFile() image
) {
console.log('===> ', image);
return this.userProfileService.saveUserProfilePhotoLocation(id, image.path);
}
#Get('/:id/profile-photo')
getUserProfilePhoto(
): any {
// '15c924f42ffaa67b3f14a5be05f0a312' is the file name that is created by the upload
return new StreamableFile(createReadStream(join(process.cwd(), 'uploads', '15c924f42ffaa67b3f14a5be05f0a312')))
}
}
The image object in the console log is:
===> {
fieldname: 'image',
originalname: '526dad4edd250b689eeb1394c3c6eb41.jpg',
encoding: '7bit',
mimetype: 'image/jpeg',
destination: './uploads',
filename: '590b454b4315009660273deac082b4ed',
path: 'uploads\\590b454b4315009660273deac082b4ed',
size: 45842
}
Then I store the path to the database.
./src/services/user-profile.service.ts
#Injectable()
export class UserProfileService {
constructor(
#InjectRepository(UserProfileEntity)
private userProfileRepostory: Repository<UserProfileEntity>
) {}
async saveUserProfilePhotoLocation(id, path): Promise<UserProfileEntity> {
const result = await this.userProfileRepostory.createQueryBuilder()
.update(UserProfileEntity)
.set({
photo: path
})
.where({id})
.returning('*')
.execute();
return result.raw;
}
}
And finally I've configured the express server to be able to serve static files from the uploads folder:
./src/main.ts
async function bootstrap() {
const app = await NestFactory.create<NestExpressApplication>(AppModule);
app.useStaticAssets(join(__dirname, '..', 'uploads'), {
index: false,
prefix: 'uploads',
});
await app.listen('3000');
}
bootstrap();
Now when I do a get on the end point /:id/profile-photo it does not return me a rendered image. Ïnstead it returns a file with a random set of characters in it which is probably because of the encoding.
What should I do so that I can serve an jpeg file to my application?
What is happening here?
When setting the content header (thanks Jay McDoniel) to 'image/jpeg' and creating a readable stream it returns the image:
#Get('/:id/profile-photo')
getUserProfilePhoto(
#Res({ passthrough: true }) res: Response
): StreamableFile {
res.set({'Content-Type': 'image/jpeg'});
const imageLocation = join(process.cwd(), 'uploads', '15c924f42ffaa67b3f14a5be05f0a312');
const file = createReadStream(imageLocation);
return new StreamableFile(file);
}

Nestjs throw error and http status 400 when fileFilter in fileinterceptor finds an unsupported file extension

I am having difficulty handling an error in my application.
I want to create a FileInterceptor (multer) which checks if the file has an allowed file extension. The documentation shows this:
const allowedFileExtensions = ['.jpg', '.png'];
FileInterceptor(
'image',
{
dest: './uploads',
fileFilter: (req, file, callback) => {
const extension = path.extname(file.originalname);
if (allowedFileExtensions.includes(extension)) {
callback(null, true);
} else {
// gives the 500 error
callback(new Error('Only images are allowed'), false);
}
}
}
This kinda works. But it has two flaws in my opinion. First, it returns a 500 error:
{
"statusCode": 500,
"message": "Internal server error"
}
It would be better to return a 400 error with a error message which explains why it failed.
Second, in the console of the Nest application it shows the stacktrace. I'd rather use the logging for this instead. (The application has middleware in place to pick up NestJS errors and log them automagically.)
So what I am trying is the following:
const allowedFileExtensions = ['.jpg', '.png'];
FileInterceptor(
'image',
{
dest: './uploads',
fileFilter: (req, file, callback) => {
const extension = path.extname(file.originalname);
if (!allowedFileExtensions.includes(extension)) {
// crashes the application
throw new BadRequestException('Only images are allowed', `Bad request. Accepted file extensions are: ${allowedFileExtensions.toString()}`);
}
callback(null, true)
}
}
But this crashes the application. I am not able to upload a different image anymore after this.
Any idea?
Ah found it. The BadRequestException cannot be thrown outside the body of the function.
const allowedFileExtensions = ['.jpg', '.png'];
enum FileValidationErrors {
UNSUPPORTED_FILE_TYPE
}
#Post('/:id/upload-photo')
#UseInterceptors(
FileInterceptor(
'image',
{
dest: './uploads',
fileFilter: (req, file, callback) => {
const extension = path.extname(file.originalname);
if (allowedFileExtensions.includes(extension)) {
callback(null, true);
} else {
// provide the validation error in the request
req.fileValidationError = FileValidationErrors.UNSUPPORTED_FILE_TYPE
callback(null, false);
}
}
}
)
)
uploadSinglePhoto(
#Param('id', ParseIntPipe) id: number,
#UploadedFile() image,
#Req() req // add the request property
): Promise<UserProfileEntity> {
// check here for a potential error
if (req?.fileValidationError === FileValidationErrors.UNSUPPORTED_FILE_TYPE) {
// if so, throw the BadRequestException
throw new BadRequestException('Only images are allowed', `Bad request. Accepted file extensions are: ${allowedFileExtensions.toString()}`);
}
this.logger.verbose(`Adding image ${image}`);
const imageUrlLocation = `${image.destination.substring(1)}/${image.filename}`;
return this.userProfileService.saveUserProfilePhotoLocation(id, imageUrlLocation);
}

Vite plugin for SvelteKit: Generated files aren't added to the build output

I am currently working on another plugin, that should in the end generate the webmanifest and all images and splash screens needed for a PWA (minus the service worker). I am planning on making this a plugin for vite (rollup), with a special focus on sveltekit, because that's where I plan on using it.
I currently have this setup as a package that exports both mjs and cjs, and should for all I know have a working version to test with. Sadly, the output emitted using this.emitFiles doesn't appear in the build output, even though prior function returns an assetId that resolves to a URL.
Code
index.ts
import { Plugin } from 'vite'
import { PluginOptions } from './types.js'
import { readFileSync } from 'fs'
import { generateResizedWebpIcon, generateResizedJpegIcon } from './utils.js';
export default (options: PluginOptions): Plugin => {
const iconResolutions = [16, 48, 128, 512]
return {
name: 'vite-plugin-pwa',
async transformIndexHtml() {
// add images and manifest to build output
// generate icons and emit them, store the urls
const icon = readFileSync(options.image.src)
let icons = await Promise.all(iconResolutions.map(async res => {
const resolveID = this.emitFile({
type: 'asset',
name: `icon-${res}x${res}.webp`,
source: await generateResizedWebpIcon({...})
})
return {
type: 'image/webp',
sizes: `${res}x${res}`,
src: this.getFileName(resolveID)
}
}, this))
if (options.image.output?.jpeg) {
icons.push(...await Promise.all(iconResolutions.map(async res => {
const resolveID = this.emitFile({
type: 'asset',
name: `icon-${res}x${res}.jpeg`,
source: await generateResizedJpegIcon({...})
})
return {
type: 'image/jpeg',
sizes: `${res}x${res}`,
src: this.getFileName(resolveID)
}
}, this)))
}
const packageInfo = JSON.parse(readFileSync('package.json').toString())
const manifest = {
name: packageInfo.name || 'name',
description: packageInfo.description || 'description',
...options.manifest || {},
icons
};
const manifestUrl = this.getFileName(
this.emitFile({
type: 'asset',
name: 'manifest.json',
source: Buffer.from(JSON.stringify({
manifest
}))
})
)
// generate manifest with icons, save the url
// generate apple splashes and emit them, save the urls
// add links to manifest and apple meta tags
return [
{
tag: 'link',
attrs: {
rel: 'manifest',
href: manifestUrl
},
injectTo: 'head'
}
]
},
}
}
In this example, the <link rel="manifest" href="_app/manifest.webmanifest"> turns up in the html and chrome tries to fetch it. But the server returns a 404 Not Found code. It appears vite emits the file, but it is somehow overwritten by the sveltekit build process?
Does anyone know how to make this emit a file that also turns up in the final build output?

how to store images in unique folder(using primary key) for each upload using multer in nest js

I have an application where i want to store multipart form data along with image. I want images to be stored in unique folder which is primary id. How can i do it.
Here i am saving multiform data in database. I want to use primary id return by below function to create folder to save its respective images.
async createFaceDetection(faceDetectionReqDto: FaceDetectionReqDto): Promise<Object> {
try {
const newFaceDetection = new face_detection({
Name: faceDetectionReqDto.Name
})
const savedMediaUpload = await this.FaceDetectionRepository.save(newFaceDetection);
return savedMediaUpload
}
catch (err) {
throw err;
}
}
how can i use primary id fetched by "savedMediaUpload" variable in multer
my component.controller.ts code
#UseInterceptors(
FilesInterceptor('Images', 20,
{
storage: diskStorage({
destination: async function (req, file, cb) {
const filePath = path.join(ROOT_DIR, FACE_DETECTION_PATH, {Primary-Id}) // (primary id
generated by above function.)
if (!existsSync(`${filePath}`)) { mkdirSync(filePath, { recursive: true }) }
cb(null, filePath);
},
filename: (_req, file, cb) => {
return cb(null, file.originalname);
},
}),
}),
)
I've done it like this, by appending generated UUID to the file name - may not be perfect but works good enough for me so far:
Controller:
#UseInterceptors(
FilesInterceptor('asset', 10, {
storage: diskStorage({
destination: UPLOADS_DIR,
filename: getUniqueFileName,
}),
limits: {
fileSize: MAX_FILE_SIZE_BYTES,
},
fileFilter: validateImageFile,
}),
)
#Post()
async createAsset(
#Body() assetCreateRequest: AssetCreateRequest,
#UploadedFile() asset: UploadedFileModel,
): Promise<AssetResponse[]> {
....
}
And getUniqueFileName looks like this:
import { extname } from 'path';
import { v4 as uuidv4 } from 'uuid';
import { UploadedFileModel } from '../../dto/internal/file-upload';
export const getUniqueFileName = (
_request: unknown,
file: UploadedFileModel,
callback: (error: Error | null, fileName: string) => void,
) => {
const name = file.originalname.split('.')[0];
const extension = extname(file.originalname);
const randomUuid = uuidv4();
callback(null, `${name}-${randomUuid}${extension}`);
};
Later on in the database, I have separate Asset and e.g. User tables where User references a record in the Asset table by foreign key, and Asset record contains all the necessary info - file name, path, URL etc.

Query parameters in express 4 give me a "No default engine was specified" error

I am developing a web app using the MEAN stack (no Mongo for now)
I am trying to pass the name of a file on the server using a query paramerer, the error happens when i get :
"localhost:8080/api/result?filename=for-debug-file-name"
It is working well if I remove the console.log() right below
But when I get the query parameter it gets me the "Error: No default engine was specified and no extension was provided”.
(This route correspond to api/result)
var express = require('express');
var router = express.Router();
router.get('/', function(req, res) {
console.log(req.query('filename')); // ERROR
res.status(200).json({ "json-test": 42 });
})
module.exports = router;
Here are my angular routes :
const appRoutes: Routes = [
{
path: 'result',
component: ResultComponent,
},
{
path: 'upload',
component: UploaderComponent,
},
{
path: '',
redirectTo: '/upload',
pathMatch: 'full'
}];
And here is my ResultComponent.ts :
ngOnInit() {
this.getParsedDocumentData('for-debug-file-name');
}
getParsedDocumentData(fileName: string): Observable<string[]> {
let params = new URLSearchParams();
params.append('filename', fileName);
let options = new RequestOptions({ params: params });
return this.http.get('http://localhost:8080/api/result/', options)
.map(res => res.json())
.catch(this.handleError);
}
private handleError (error: any) {
return Observable.throw(error);
}
I would really appreciate your help as I have been stuck for 4 hours.
Thanks.
query method in request object does not exists. Instead use query property to access filename parameter.
console.log(req.query.filename);
Reference

Resources