I have a Node function that uploads a file to an endpoint using FormData:
const uploadFile = (filepath) => {
return new Promise((resolve, reject) => {
const formData = new FormData();
// Add file
formData.append("file", fs.createReadStream(filepath));
formData.submit(`${API_URL}/v2/upload_file`, (error, res) => {
if (error) {
return reject(error);
}
if (res.statusCode !== 201) {
return reject(res.statusMessage);
}
return resolve(res);
});
});
};
This works fine. The problem is that after an update on the endpoint, a custom header with a token is now required:
'x-session-token': 'abc123'
I can't find a way to add this token. formData.append() only adds key-values to the body. I was reading the node source code but can't find a method or a way to add options like headers.
I also tried using fetch() adding the headers, but is not parsing the multipart/form-data body data in a valid format.
Any ideas?
From the documentation...
In case you need to also send custom HTTP headers with the POST request, you can use the headers key in first parameter of form.submit()
const url = new URL(`${API_URL}/v2/upload_file`);
formData.submit({
headers: { "x-session-token": "abc123" },
host: url.hostname,
path: url.pathname,
port: url.port,
protocol: url.protocol,
}, (error, res) => {
// ...
});
IMO this is pretty clunky, particularly having to parse the URL into its components. It also doesn't support query parameters.
I would prefer to use fetch()
const uploadFile = async (filepath) => {
const formData = new FormData();
// Add file
formData.append("file", fs.createReadStream(filepath));
const res = await fetch(`${API_URL}/v2/upload_file`, {
method: "POST",
body: formData,
headers: {
...formData.getHeaders(),
"x-session-token": "abc123",
},
});
if (!res.ok) {
throw new Error(res.statusText);
}
return res.json(); // or res.text()
};
Related
I am attempting to use a form in Remix to add a file and then upload that file to WhatsApp using their Cloud API Media Upload endpoint. Below is my initial code within the action. The current error I am receiving is message: '(#100) The parameter messaging_product is required.. I feel like this error may be misleading based off the form data I have appended with the "messaging_product".
export async function action({ request, params }: ActionArgs) {
const uploadHandler = unstable_composeUploadHandlers(
async ({ name, contentType, data, filename }) => {
const whatsAppPhoneId = process.env.WHATSAPP_PHONE_ID;
const whatsAppToken = process.env.WHATSAPP_ACCESS_TOKEN;
const dataArray1 = [];
for await (const x of data) {
dataArray1.push(x);
}
const file1 = new File(dataArray1, filename, { type: contentType });
const graphApiUrl = `https://graph.facebook.com/v15.0/${whatsAppPhoneId}/media`;
const formData = new FormData();
formData.append("file", file1);
formData.append("messaging_product", "whatsapp");
formData.append("type", contentType);
try {
const imageMediaResponse = await fetch(graphApiUrl, {
method: "POST",
headers: {
Authorization: `Bearer ${whatsAppToken}`,
"Content-Type": "multipart/form-data",
},
body: formData,
});
const imageMedia = await imageMediaResponse.json();
return imageMedia?.id;
} catch (error) {
console.error(error);
}
const whatsAppMediaId = await uploadWhatsAppImageMedia(
whatsAppPhoneId,
whatsAppToken,
data,
filename,
contentType
);
}
);
const formData = await unstable_parseMultipartFormData(
request,
uploadHandler
);
}
I use nuxt3/node.js with multer ,
i can't store or receive file in my server and i can't see req.file in root, its empty all time.
server Code:
const upload = multer({ dest: './uploads/' })
router.post('/sendNewQuestionCSV', upload.single('csv'),adminControler.sendNewQuestionCSV.bind(adminControler))
and my FrontEnd Code with nuxt3:
async function fileSelected(e) {
let formData = new FormData();
formData.append("csv", e.target.files[0]);
const res = await $postFetch("/admin/sendNewQuestionCSV", formData, {
"Content-Type": "multipart/form-data",
});
}
note:$postFetch is an method i made my self use fetch and third argument is "headers". its a nuxt3 Plugin
this Plugin code:
export default defineNuxtPlugin(async () => {
return {
provide: {
postFetch: async (url, body,headers={}) => {
let token = useCookie('token');
return await $fetch(url, {
method: 'post',
body: { token: token.value, ...body },
baseURL: useRuntimeConfig().API_BASE,
...headers
}).catch((error) => error.data)
}
}
}
})
try using .append to add the token:
postFetch: async (url, body,headers={}) => {
let token = useCookie('token');
body.append('token', token.value);
return await $fetch(url, {
method: 'post',
body: body,
baseURL: useRuntimeConfig().API_BASE
}).catch((error) => error.data)
}
EDIT
also try removing headers
I'm trying to download zip release from a private repository, i've tried many solutions but none seems to work.
Here my code :
function Download(url, path, options) {
updateHeader(options.stageTitles.Downloading)
let received_bytes = 0;
let total_bytes = 0;
// eslint-disable-next-line
console.log('-----------------')
// eslint-disable-next-line
console.log(url, path)
console.log('-----------------')
var req = request(
{
method: 'GET',
uri: url,
headers: {
Authorization: `token ${options.gitRepoToken}`,
'Accept': 'application/octet-stream',
'User-Agent': 'request module',
},
//encoding: null
}
);
// eslint-disable-next-line
console.log(req)
var out = fs.createWriteStream(path);
req.pipe(out);
req.on('response', data => {
// eslint-disable-next-line
console.log(data.headers, data)
total_bytes = parseInt(data.headers['content-length']);
});
req.on('data', chunk => {
received_bytes += chunk.length;
showProgress(received_bytes, total_bytes);
});
req.on('end', () => {
Install(options)
});}
url variable equal something like : https://github.com/SolberLight/PRIVATE_REPO_NAME/releases/download/1.0/MY_ZIP_NAME.zip
Looks like I'm always getting a content size of 9 bytes with a "not found" response so I'm guessing that my headers aren't correct ?
thanks for your help !
EDIT 1 :
Made some progress but now it seems that the blob is not the right size
(async () => {
//get latest release
var response = await axios.get(
`https://api.github.com/repos/${repoWithOwner}/releases/latest`,
{
headers: authHeaders,
}
);
var assets = response.data.assets;
for (var i = 0; i < assets.length; i++) {
console.log(assets[i].url);
response = await axios({
method: "get",
url: assets[i].url,
responseType: "blob",
headers: {
//Accept: "application/octet-stream",
...authHeaders,
},
});
// eslint-disable-next-line
console.log(response.data, path)
let reader = new FileReader()
reader.onload = function() {
if (reader.readyState == 2) {
var buffer = new Buffer(reader.result)
console.log(`Saving ${JSON.stringify({ fileName: 'myfile.zip', size: response.data.size })}`)
outputFile(path, buffer, err => {
if (err) {
// eslint-disable-next-line
console.log(err.message)
} else {
// eslint-disable-next-line
console.log(path)
}
})
}
}
reader.readAsArrayBuffer(response.data)
}
})();
You can use the Download repository archive (zip) endpoint, documented here
Take into consideration the following:
You will perform a GET request to /repos/{owner}/{repo}/zipball/{ref}
The first request will send a redirect (302 response), you need to follow the redirect properly, in this case, since you're using axios, you will need to handle it manually. The response will include a zipball_url you will use in the following request.
Since you are downloading a private repo release, take into consideration these links are temporary and expire after five minutes.
The first request accepts a user or installation token (in case you're consuming the API as a GitHub Application). Read more about the difference between a GitHub App and a GitHub OAuth App here
If you are using a GitHub OAuth application, ensure you have the repo scope, for GitHub Application, ensure you have the Contents permissions
Check out the following working example:
Download a zip release from GitHub
View in Fusebit
const owner = 'stack-overflow-demos';
// For the purpose of this example, we have 1 release, so it will be always the first item.
const releases = await installationClient.rest.repos.listReleases({
owner: 'stack-overflow-demos',
repo: 'private-demo'
});
// Get release information (e.g zipball URL)
const release = await installationClient.rest.repos.getRelease({
owner: 'stack-overflow-demos',
repo: 'private-demo',
release_id: releases.data[0].id,
});
const { token } = await installationClient.auth();
// According to official GitHub docs https://docs.github.com/en/rest/reference/repos#download-a-repository-archive-zip
// You will get a zipball URL you can use to download the zip file, you need to handle properly the redirects since a second GET
// request is needed, this time with an authenticated URL.
const { zipball_url, tag_name } = release.data;
const response = await getZipFile(zipball_url, token);
// Handle redirects properly if a responseUrl is returned that means you need to perform the redirect,
// you don't need to send the access token to this request, since is a signed URL
// Note: For private repositories, these links are temporary and expire after five minutes.
const {
request: {
res: { responseUrl },
},
} = response;
if (responseUrl) {
await downloadZipFile(responseUrl, `/tmp/'${tag_name}.zip`);
}
// Get information about the zip file you're about to download
const getZipFile = async (url, token) => {
return await axios({
redirect: 'manual',
method: 'get',
url,
headers: {
Authorization: `token ${token}`,
Accept: 'application/vnd.github.v3+json',
},
});
};
// Download the zip file from the temporary secure URL returned from the first request
const downloadZipFile = async (url, filePath) => {
const response = await axios({
method: 'get',
url,
responseType: 'stream',
headers: {
Accept: 'application/octet-stream',
},
});
return new Promise((resolve, reject) => {
console.log(`writing to ${filePath}`);
const dest = fs.createWriteStream(filePath);
let progress = 0;
response.data
.on('end', () => {
console.log('🎉 Done downloading file.');
resolve(filePath);
})
.on('error', (err) => {
console.error('🚫 Error downloading file.');
reject(err);
})
.on('data', (d) => {
progress += d.length;
console.log(`🕛 Downloaded ${progress} bytes`);
})
.pipe(dest);
});
};
This example is using Octokit
In my node application, I want to get a file from one server, and then upload it into another server. I have the following code:
const axios = require("axios");
const FormData = require("form-data");
const { createWriteStream, createReadStream } = require("fs");
const response = await axios({
url: "https://first-server/image.png",
method: "GET",
responseType: "stream",
});
await new Promise((res) => {
response.data.pipe(
createWriteStream("someFile.png").on("finish", () => {
res();
})
);
});
const form = new FormData();
form.append("file", createReadStream("./someFile.png"));
const postHeaders = {
headers: {
Authorization: "Bearer " + env("APY_KEY"),
...form.getHeaders(),
},
data: form,
};
axios.post("https://second-server.com/api", form, postHeaders)
.then((response) => {
console.log(JSON.stringify(response.data));
})
This code works, but I think it's not the right way to do this, since it writes the retrieved file into the local disc before posting it again into the second server. I need to be able to upload the file without writing it into the local disc. Is there any way?
Just replace form.append("file", createReadStream("./someFile.png")); with
form.append("file", response.data);
Both response.data and createReadStream("./someFile.png") are readable stream.
Note: You can directly transfer returned stream data without any need to create temporary file.
const axios = require("axios");
const FormData = require("form-data");
axios({
url: "http://localhost:3000/temp.png",
method: "GET",
responseType: "stream",
}).then(response => {
response.data.on("data", function(data) {
const form = new FormData();
form.append("file", data);
const postHeaders = {
headers: {
// Authorization: "Bearer " + env("APY_KEY"),
...form.getHeaders(),
},
data: form,
};
axios.post("http://localhost:8000/api", form, postHeaders)
.then((response) => {
// console.log(JSON.stringify(response.data));
})
.catch(function(error){
console.log(error)
});
});
})
.catch(function(error){
console.log(error)
});
Here is my code of put request
requestMethodPutWithFormData: function (url, form_data, header) {
return new Promise((resolve, reject) => {
//SET ALL THESE PARATMETER TO MAKE REQUEST
request.put({url: url, formData: form_data, headers: header}, function (error, response, body) {
var responseData = JSON.parse(body);
resolve(responseData);
}
});
});
}
And my setting my form_data like this
var form_data = {
image_file: {
value: fs.createReadStream(files.file.path),
options: {
filename: files.file.name
}
},
store_id: fields.store_id,
image_file_id: fields.image_file_id,
token_type: g_token_type,
account_id: g_account_id
};
My problem is the data of formData is not receiving on the requested URL it is showing data null, but can not understand the point that this code is absolutely working fine with post request but not with put request data is not receiving at the endpoint.
Please help me to overcome this problem