Earlier I was using rest API to upload my files to the server. I had input type file and I was sending the form data to the server with that API. It was working fine but now I have to use graphql mutation for the same. We have an apollo server running app where I have used the same API to upload our images but it seems that file object was not in the same format as it was expected by rest API. So my question is how can we create same file object like we have in browser from nodejs.
We are using RESTDataSource to make HTTP calls in graphql app
My schema
type Mutation {
uploadFile(file: Upload!): Boolean!
}
Here is my resolver
import { createWriteStream, createReadStream } from 'fs';
import FormData from 'form-data';
async uploadFile(file: any): Promise<any> {
const url = new URL(`https://dev.dummber.com/portal/file/v1/uploadFile`);
const { createReadStream: apolloFileStream, mimetype, filename } = await file;
const tempFilename = `${filename}`;
const stream = apolloFileStream();
const wstream = createWriteStream(tempFilename);
await new Promise((resolve, reject) => {
stream
.pipe(wstream)
.on('finish', () => resolve())
.on('error', () => reject());
});
const rstream = createReadStream(tempFilename);
const formData = new FormData();
formData.append('uploadingFile', rstream);
const response = await this.post(url.toString(),
{
data: formData
}
, {
headers: {
'Content-Type': 'multipart/form-data'
}
});
return true
}
Related
I am attempting to use a form in Remix to add a file and then upload that file to WhatsApp using their Cloud API Media Upload endpoint. Below is my initial code within the action. The current error I am receiving is message: '(#100) The parameter messaging_product is required.. I feel like this error may be misleading based off the form data I have appended with the "messaging_product".
export async function action({ request, params }: ActionArgs) {
const uploadHandler = unstable_composeUploadHandlers(
async ({ name, contentType, data, filename }) => {
const whatsAppPhoneId = process.env.WHATSAPP_PHONE_ID;
const whatsAppToken = process.env.WHATSAPP_ACCESS_TOKEN;
const dataArray1 = [];
for await (const x of data) {
dataArray1.push(x);
}
const file1 = new File(dataArray1, filename, { type: contentType });
const graphApiUrl = `https://graph.facebook.com/v15.0/${whatsAppPhoneId}/media`;
const formData = new FormData();
formData.append("file", file1);
formData.append("messaging_product", "whatsapp");
formData.append("type", contentType);
try {
const imageMediaResponse = await fetch(graphApiUrl, {
method: "POST",
headers: {
Authorization: `Bearer ${whatsAppToken}`,
"Content-Type": "multipart/form-data",
},
body: formData,
});
const imageMedia = await imageMediaResponse.json();
return imageMedia?.id;
} catch (error) {
console.error(error);
}
const whatsAppMediaId = await uploadWhatsAppImageMedia(
whatsAppPhoneId,
whatsAppToken,
data,
filename,
contentType
);
}
);
const formData = await unstable_parseMultipartFormData(
request,
uploadHandler
);
}
I am trying to upload mp4 files using typescript, node js and mongodb but it is not working.
I am using busboy library for the file upload
I am using postman to test my api
when I send the file I receive the file on busboy
I can display the file info like the filename
but the file does not move to the new location
video.controller.ts
const MIME_TYPES = ["video/mp4"];
function getPath({
videoId,
extension,
}: {
videoId: Video["videoId"];
extension: Video["extension"];
}) {
return `${process.cwd()}/videos/${videoId}.${extension}`;
}
export async function uploadVideoHandler(req: Request, res: Response) {
const bb = busboy({ headers: req.headers });
const user = res.locals.user;
const video = await createVideo({ owner: user._id });
console.log(video);
bb.on("file", async (_, file, info) => {
const extension = info.mimeType.split("/")[1];
const filePath = getPath({
videoId: video.videoId,
extension,
});
video.extension = extension;
const { filename } = info;
console.log("filename ", filename);
await video.save();
console.log(filePath);
const stream = fs.createWriteStream(filePath);
file.pipe(stream);
});
bb.on("close", () => {
res.writeHead(StatusCodes.CREATED, {
Connection: "close",
"Content-Type": "application/json",
});
res.write(JSON.stringify(video));
res.end();
});
return req.pipe(bb);
}
Api route
const router = express.Router();
router.post("/", requireUser, uploadVideoHandler);
export default router;
New to SvelteKit and working to adapt an endpoint from a Node/Express server to make it more generic so as to be able to take advantage of SvelteKit adapters. The endpoint downloads files stored in a database via node-postgresql.
My functional endpoint in Node/Express looks like this:
import stream from 'stream'
import db from '../utils/db'
export async function download(req, res) {
const _id = req.params.id
const sql = "SELECT _id, name, type, data FROM files WHERE _id = $1;"
const { rows } = await db.query(sql, [_id])
const file = rows[0]
const fileContents = Buffer.from(file.data, 'base64')
const readStream = new stream.PassThrough()
readStream.end(fileContents)
res.set('Content-disposition', `attachment; filename=${file.name}`)
res.set('Content-Type', file.type)
readStream.pipe(res)
}
Here's what I have for [filenum].json.ts in SvelteKit so far...
import stream from 'stream'
import db from '$lib/db'
export async function get({ params }): Promise<any> {
const { filenum } = params
const { rows } = await db.query('SELECT _id, name, type, data FROM files WHERE _id = $1;', [filenum])
if (rows) {
const file = rows[0]
const fileContents = Buffer.from(file.data, 'base64')
const readStream = new stream.PassThrough()
readStream.end(fileContents)
let body
readStream.pipe(body)
return {
headers: {
'Content-disposition': `attachment; filename=${file.name}`,
'Content-type': file.type
},
body
}
}
}
What is the correct way to do this with SvelteKit without creating a dependency on Node? Per SvelteKit's Endpoint docs,
We don't interact with the req/res objects you might be familiar with from Node's http module or frameworks like Express, because they're only available on certain platforms. Instead, SvelteKit translates the returned object into whatever's required by the platform you're deploying your app to.
UPDATE: The bug was fixed in SvelteKit. This is the updated code that works:
// src/routes/api/file/_file.controller.ts
import { query } from '../_db'
type GetFileResponse = (fileNumber: string) => Promise<{
headers: {
'Content-Disposition': string
'Content-Type': string
}
body: Uint8Array
status?: number
} | {
status: number
headers?: undefined
body?: undefined
}>
export const getFile: GetFileResponse = async (fileNumber: string) => {
const { rows } = await query(`SELECT _id, name, type, data FROM files WHERE _id = $1;`, [fileNumber])
if (rows) {
const file = rows[0]
return {
headers: {
'Content-Disposition': `attachment; filename="${file.name}"`,
'Content-Type': file.type
},
body: new Uint8Array(file.data)
}
} else return {
status: 404
}
}
and
// src/routes/api/file/[filenum].ts
import type { RequestHandler } from '#sveltejs/kit'
import { getFile } from './_file.controller'
export const get: RequestHandler = async ({ params }) => {
const { filenum } = params
const fileResponse = await getFile(filenum)
return fileResponse
}
I've been unable to figure out in NodeJS how to:
create a "file" in memory from a raw string; and,
how to POST that data to another server that expects a multipart/form-data payload.
Seems you cannot use the Blob or File classes in NodeJS.
I've read the pattern should be to use the Buffer class.
I still cannot get it to work with Buffers.
My GQL Datasoruce class looks something like:
const { RESTDataSource } = require('apollo-datasource-rest');
const FormData = require('form-data');
export default class MyDatasource extends RESTDataSource {
async postFileToServer({ string }) {
const inMemoryFile = Buffer.from(string, 'utf-8');
const myForm = new FormData();
myForm.append('file', inMemoryFile, 'file.txt');
const url = 'http://examnple.com';
const opts = { headers: { 'Content-Type': 'multipart/form-data' } };
return await this.post(url, myForm, opts);
}
}
The endpoint I want to hit works fine when I use Postman to make the API call with a file from my local machine. However, I need the GQL server to create the file from a raw string to afterwards call the example.com endpoint that is expecting a multipart/form-data.
The above example code always gives me an error of Status 400 and SyntaxError: Unexpected token - in JSON at position 0
File upload works for me using the apollo-datasource-rest package. Here is an example:
server.ts:
import { ApolloServer, gql } from 'apollo-server';
import MyDatasource from './datasource';
const typeDefs = gql`
type Query {
dummy: String
}
type Mutation {
upload: String
}
`;
const resolvers = {
Mutation: {
upload(_, __, { dataSources }) {
return dataSources.uploadAPI.postFileToServer({ str: '1234' });
},
},
};
const server = new ApolloServer({
typeDefs,
resolvers,
dataSources: () => {
return {
uploadAPI: new MyDatasource(),
};
},
});
const port = 3001;
server.listen(port).then(({ url }) => console.log(`🚀 Server ready at ${url}`));
datasource.ts:
import { RESTDataSource } from 'apollo-datasource-rest';
import FormData from 'form-data';
export default class MyDatasource extends RESTDataSource {
public async postFileToServer({ str }) {
const inMemoryFile = Buffer.from(str, 'utf-8');
const myForm = new FormData();
myForm.append('file', inMemoryFile, 'file.txt');
const url = 'http://localhost:3000/upload';
return this.post(url, myForm);
}
}
uploadServer.ts:
import multer from 'multer';
import express from 'express';
import path from 'path';
const upload = multer({ dest: path.resolve(__dirname, 'uploads/') });
const app = express();
const port = 3000;
app.post('/upload', upload.single('file'), (req, res) => {
console.log(req.file);
console.log(req.body);
res.sendStatus(200);
});
app.listen(port, () => {
console.log(`upload server is listening on http://localhost:${port}`);
});
The logs printed in the controller of /upload API:
{
fieldname: 'file',
originalname: 'file.txt',
encoding: '7bit',
mimetype: 'text/plain',
destination: '/Users/ldu020/workspace/github.com/mrdulin/apollo-graphql-tutorial/src/stackoverflow/63181608/uploads',
filename: '3cba4dded6089479ad495e2fb2daac21',
path: '/Users/ldu020/workspace/github.com/mrdulin/apollo-graphql-tutorial/src/stackoverflow/63181608/uploads/3cba4dded6089479ad495e2fb2daac21',
size: 4
}
[Object: null prototype] {}
source code: https://github.com/mrdulin/apollo-graphql-tutorial/tree/master/src/stackoverflow/63181608
I'm trying to download image using axios and fs
When i run it using node app.js it gives me an error saying pipe
can't be defined
"TypeError: Cannot read property 'pipe' of undefined".
const axios = require('axios');
const fs = require('fs');
const Path = require('path');
async function download()
{
const url ='https://www.google.co.in/imgres'
const path = Path.resolve(__dirname,'files','image1.jpg')
const response = axios
(
{
method : 'GET',
url:url,
responseType:'stream'
}
)
response.data.pipe(fs.createWriteStream(path))
return new Promise((resolve,reject)=>{
response.data.on('end',()=>{
resolve()
})
response.data.on('error',err=>{
reject(err);
})
}).catch();
}
download().then(()=>{
console.log('download finished');
})
Don't you need to wait for axios promise to complete?
See Axios API
...
const response = axios
(
{
method : 'GET',
url:url,
responseType:'stream'
}
).then(function(response) {
response.data.pipe(fs.createWriteStream(path))
return new Promise((resolve,reject)=>{
response.data.on('end',()=>{
resolve()
})
...
Depending on script level you could do this with async/await too I guess, but I'm no Axios expert.