I have the following code. Issue is, the file isn't always getting caught by my server (hosted on DigitalOcean, not that that matters so much as I often couldn't even get it to work on my local vagrant machine).
It seems to work with smaller files, smaller images, however if I send a larger file (such as a full screen screenshot) my server is telling me that the request has been made but no 'image' has been found.
I had a look around for a way for createReadStream to return some kind of promise, in hopes that it would be as simple as it is still trying to read the image from the hard drive, but I've had no luck.
Worth mentioning that the image is being generated just before this happens, but this is being called in a callback so that shouldn't matter.
...
const capture = fs.createReadStream(filePath);
request.post({
url: UPLOAD_URL,
method: 'POST',
formData: {
'image': capture,
'token': this.token
},
headers: {
'User-Agrent': 'request',
'Content-Type': 'multipart/form-data'
}
}, (err, response, body) => {
...
Related
I am trying to write a NodeJS app that grabs a image from private URL using Axios and passes it to the Azure Face SDK (documentation) to detect faces in the image and get attributes of those faces - in this case, emotions and head pose.
I have gotten a modified version of the quickstart example code here working, which makes a call to the detectWithUrl() method. However, the image that I have in my code is a ArrayBuffer, so I thought I would try calling detectWithStream() instead. The documentation for this method says it needs to be passed something of type msRest.HttpRequestBody - I found some documentation for this type, which looks like it wants to be a Blob, string, ArrayBuffer, or ArrayBufferView. The problem is, I don't really understand what those are or how I might get from a arraybuffer image to an HttpRequestBody of that type. I have worked with HTTP requests before, but I don't quite understand why one is being passed to this method, or how to make it.
const rqs = {
responseType: 'arraybuffer',
url: `${fileUrl}`,
method: 'GET',
headers: {
'Content-Type': `image/jpeg`,
'Cache-Control': 'no-cache'
},
httpsAgent: agent
};
axios.request(rqs)
.then((ret) => {
// ret.data is a ArrayBuffer type;
let detected_faces = client.face.detectWithStream(ret.data, {
returnFaceAttributes: ["Accessories", "Age", "Blur", "Emotion", "Exposure", "FacialHair", "Glasses", "Hair", "HeadPose", "Makeup", "Noise", "Occlusion", "Smile", "QualityForRecognition"],
detectionModel: "detection_01",
recognitionModel: "recognition_03"
})
.then(ok => {
console.log(ok);
console.log("face(s) detected from image.");
console.log("Face attributes for face(s):");
})
.catch(err => {
console.error(err.message);
});
console.log('ok');
resolve(ret.data);
})
.catch((err) => {
console.error('nok');
console.error(err.message);
reject(false)
});
And I had receive a message error Only absolute URLs are supported
Only absolute URLs are supported -
a. it may be due to passing relative URL to Axios in line 3, try checking the fileUrl.
b. check the face SDK is provided with the proper endpoint
relative URL - "./a/b"
absolute URL - "https://www.x.com/a/b"
how I might get from an array buffer image?
Array buffer is not needed. Looking at face SDK code Ref (hope this is right sdk), we just need streams.
we can get this by changing the responseType value to stream
responseType: 'arraybuffer', // current
responseType: 'stream', // updated - we can update it in rqs and reponse will be stream and you can pass res.data to detectWithStream function
-------------------------------------Hope the code works----------------------------------------
Recommendation:
You are creating a callback hell by using nested .then chains what is call back hell.
Try using async/await syntax to achieve the same functionality in a simplified way - Ref.
Regards,
Muhamed
I'm querying a data service with a restful URL. The server (also node) sends about 955k of JSON data.
1) I can CURL the data to get the correct result, ie, it passes JSON.parse().
2) From node, I can exec("curl ..."); and also get the correct result.
3) Using both Request and Axios, I get about 600k of data. The precise number of characters changes each time.
4) Using Axios, I streamed the data into a file and got many 'data' events which I concatenated into a file. It was also incorrect.
5) It works fine with a smaller payload.
Experts Unite!! I am at your mercy. I will supplicate and offer praise and thanks for your aid.
Without your help, I will have a production application that uses CURL from NodeJS and evil will win.
Sincerely,
TQ White II
UPDATE: I was asked for a code snippet. Here it is:
const datGetterWORKS_FOR_SMALL_DATA_LOADS=(element, next)=>{
const localCallback=sendToTransformerCallback(element, next);
const {url, headers}=networkSpecs.connection;
axios.get(url + element.urlSegment, {
method: 'get',
responseType: 'json',
headers: headers,
maxContentLength: 6000000,
})
.then(function (response) {
localCallback('', response, response.data)
});
}
Note that this is give to a require('async').each() process.
I retrieve this URL from using some previous logic - https://bots.rocket.chat/file-upload/ZSzGtWdXyP8DZwrQ9/Audio%20record.mp3
Now I need to download it for further processing, but I tried using the download, axios.request, and other packages available. My best guess for them failing is that the actual URL of the audio is different then what I retrieved.
Something like (It changes after a while) -https://s3.amazonaws.com/uploads.use1.cloud.rocket.chat/KqCQiHfeFaKdSEGvy/uploads/GENERAL/XmYemgXfudSzBQstR/ZSzGtWdXyP8DZwrQ9?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAILPK6SHTK5RJZLHQ%2F20190518%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20190518T115900Z&X-Amz-Expires=120&X-Amz-Signature=8d793d14fbb33d78b084a5ed153106ba19440feb7bf05525aeed530b5a0907f9&X-Amz-SignedHeaders=host
Also, I'm able to download the file using the same code when I provide it with the end URL (the longer one).
So need to figure out how to reach that URL in node.
Also to access the first URL you need to make an account on https://bots.rocket.chat
This is the code I have currently. The url passed to getFile function is the is a relative path which I append to the prefix url to form the above mentioned shorter URL, and call axios and this doesn't work.
But when I replace the url with a long url which is similar to the longer one posted above the code works just fine.
async function getFile(url) {
var fs = require('fs');
return await axios.request({
responseType: 'arraybuffer',
followAllRedirects: true,
url: `https://bots.rocket.chat${url}`,
method: 'get',
headers: {
'Content-Type': 'audio/mpeg',
},
}).then((result) => {
const outputFilename = 'file2.mp3';
fs.writeFileSync(outputFilename, result.data);
return outputFilename;
})
.catch(err => console.log(err));
;
}
I'm trying to send a multipart/form-data image to another server, using unirest. Their .attach() works with a fs.createReadStream(), however, I have not been able to convert the buffer to an image. The logical step is to convert the buffer to Uint8Array first, then creating a read stream from it. However, this throws up an error message saying that the array must not contain null values. Removing 0 entries from the array will almost certainly break the image.
Image is not null, has all the bytes, and even sending the image data as a giant string works.
Here's what I tried:
imageBytes = new Uint8Array(image.buffer)
unirest
.post(someURL)
.headers(headers)
.attach("image", fs.createReadStream(imageBytes))
.end(response => {
console.log(response.body);
});
The alternatives are:
1. Attaching the buffer directly, which sends the raw data as a form field. Not ideal, and might run into image size restrictions.
2. Write the file to storage instead of keeping it in memory. This would be handling some sensitive information, thus would require auto-deletion post a certain amount of time, leading to more work.
EDIT: I ended up switching to request, as that allowed inline 'files' from buffers. The code to do so is below:
request({
uri: someURL,
method: "POST",
formData: {
"image": {
value: image.buffer,
options: {
filename: image.originalname,
contentType: image.mimetype
}
}
}
}, (err, resp, body) => {
if (err) {
console.log("ERROR -> " + err)
}
if (body) {
console.log(body)
}
})
EDIT2: Please also put in encoding: null in the request options if you follow this. Don't be like me and spend a day tracking down why your returned data is of an alien format. :)
I have this GET request I built to call Google APIS dot com to get an image of a house at a given address. It's all working fine. In Postman it displays the image from the body of the request. All good!
I converted the code to NodeJS REQUEST. Put that code into my project. It all works though the data returned is all �����JFIF������ like this in the BODY returned.
Can you point me to some resources or can you tell me in NODEJS how I get that into a Image type variable. I want to then display it using JSON code into Messenger Bot. I have the JSON code to send a IMAGE type back to Messenger - I just need to get the results of the GET above into a format in NODEJS that will work - like a PNG or JPG format.
This is the code I used from Postman CODE:
var options = { method: 'GET',
url: 'https://maps.googleapis.com/maps/api/streetview',
qs:
{ size: '450x450',
location: 'N108W15303%20Bel%20Aire%20Ln%2053022',
fov: '90',
heading: '235',
pitch: '10',
key: 'xxx' },
headers:
{ 'Postman-Token': 'xxx',
'Cache-Control': 'no-cache',
Accept: 'application/json' } };
requestGoogle(options, function (error, response, body) {
if (error) throw new Error(error);
console.log(body);
});
The BODY var is is all ����JFIF��C
It displays great in the Postman App. so you are somehow converting it to display it - what I am looking for.
Any help would be appreciated - or direct me to a resource that can help that would be great.