I'm trying to send a multipart/form-data image to another server, using unirest. Their .attach() works with a fs.createReadStream(), however, I have not been able to convert the buffer to an image. The logical step is to convert the buffer to Uint8Array first, then creating a read stream from it. However, this throws up an error message saying that the array must not contain null values. Removing 0 entries from the array will almost certainly break the image.
Image is not null, has all the bytes, and even sending the image data as a giant string works.
Here's what I tried:
imageBytes = new Uint8Array(image.buffer)
unirest
.post(someURL)
.headers(headers)
.attach("image", fs.createReadStream(imageBytes))
.end(response => {
console.log(response.body);
});
The alternatives are:
1. Attaching the buffer directly, which sends the raw data as a form field. Not ideal, and might run into image size restrictions.
2. Write the file to storage instead of keeping it in memory. This would be handling some sensitive information, thus would require auto-deletion post a certain amount of time, leading to more work.
EDIT: I ended up switching to request, as that allowed inline 'files' from buffers. The code to do so is below:
request({
uri: someURL,
method: "POST",
formData: {
"image": {
value: image.buffer,
options: {
filename: image.originalname,
contentType: image.mimetype
}
}
}
}, (err, resp, body) => {
if (err) {
console.log("ERROR -> " + err)
}
if (body) {
console.log(body)
}
})
EDIT2: Please also put in encoding: null in the request options if you follow this. Don't be like me and spend a day tracking down why your returned data is of an alien format. :)
Related
I am getting a 400 error (bad request) from this POST:
siteData.myPost = async function (url, data) {
let options = {
method: 'POST',
headers: {
'Content-Type': 'application/json;charset=utf-8'
},
body: JSON.stringify(data)
}
try {
const response = await fetch(quilkinUrlBase() + url,options);
return response.json();
}
catch (error) {
qPopup.Alert("Error with web request: " + error);
}
};
(the fetch address computes to "http://localhost:1337/UpdateSiteVisits").
This can't, of course, convert 'response' to json because the response is a DOC containing the 400 error message.
in server.js I have the following callback definition for the URL specified in the fetch:
app.post("/UpdateSiteVisits", sites.updateVisits);
in the debugger, the code never reaches updateVisits().
I have a very similar construct with a GET:
app.get("/GetSitesForType/:type", sites.findByType);
which works fine, so I assume the problem is with the 'options' object (which the GET doesn't use). In the POST case the data is just an integer ID, and if I change the data format to plain text (instead of json), the post works fine.
If leave the format as json, and send an object, rather than a simple integer, the post also works fine. So it looks like the system just doesn't like converting a single integer to json.
I needed to send the integer as json, so instead of using the method like this:
siteData.myPost("UpdateSiteVisits", siteID)
I needed to use
siteData.myPost("UpdateSiteVisits", { siteId: siteID })
[ later ]
Please ignore my question and answer, I was just getting mixed up with types. Such is the result from me converting from strongly-typed C# to untyped javascript!
I am trying to create a function which return the latest blocked domain from my pihole server.
I first created a JSON call, since the first call I needed was in JSON format, this is all working and I get the data needed.
However, the second function I need is to fetch plain text data and that one doesn't work, it simply returns an empty body [].
This is the function
socket.on("pihole_last", function() {
setInterval(function() {
let settings = {
method: "Get",
headers: {
"Accept": "text/html"
}
};
fetch('http://domain/admin/api.php?recentBlocked', settings)
.then(res => res.text())
.then((data) => {
console.log(data);
}).catch(error => {
return error;
});;
}, 1000)
});
The JSON function which works looks pretty much the same, the only real different is the header accept and the res.text() which should fetch the data in plain text?
The data returned from the URL is a plain text domain, no tags, no nothing.
According to this issue from the pi-hole GIT, you should provide some form of authentication. The question which you linked in your comment is 5 years old, at that time this was an unintended behaviour.
If I understand correctly the API description one way to authorize should be working with this url:
http://domain/admin/api.php?recentBlocked?auth=YOUR_TOKEN
The YOUR_TOKEN should be in:
Authorization & Token required (see WEBPASSWORD in /etc/pihole/setupVars.conf)
I am trying to write a NodeJS app that grabs a image from private URL using Axios and passes it to the Azure Face SDK (documentation) to detect faces in the image and get attributes of those faces - in this case, emotions and head pose.
I have gotten a modified version of the quickstart example code here working, which makes a call to the detectWithUrl() method. However, the image that I have in my code is a ArrayBuffer, so I thought I would try calling detectWithStream() instead. The documentation for this method says it needs to be passed something of type msRest.HttpRequestBody - I found some documentation for this type, which looks like it wants to be a Blob, string, ArrayBuffer, or ArrayBufferView. The problem is, I don't really understand what those are or how I might get from a arraybuffer image to an HttpRequestBody of that type. I have worked with HTTP requests before, but I don't quite understand why one is being passed to this method, or how to make it.
const rqs = {
responseType: 'arraybuffer',
url: `${fileUrl}`,
method: 'GET',
headers: {
'Content-Type': `image/jpeg`,
'Cache-Control': 'no-cache'
},
httpsAgent: agent
};
axios.request(rqs)
.then((ret) => {
// ret.data is a ArrayBuffer type;
let detected_faces = client.face.detectWithStream(ret.data, {
returnFaceAttributes: ["Accessories", "Age", "Blur", "Emotion", "Exposure", "FacialHair", "Glasses", "Hair", "HeadPose", "Makeup", "Noise", "Occlusion", "Smile", "QualityForRecognition"],
detectionModel: "detection_01",
recognitionModel: "recognition_03"
})
.then(ok => {
console.log(ok);
console.log("face(s) detected from image.");
console.log("Face attributes for face(s):");
})
.catch(err => {
console.error(err.message);
});
console.log('ok');
resolve(ret.data);
})
.catch((err) => {
console.error('nok');
console.error(err.message);
reject(false)
});
And I had receive a message error Only absolute URLs are supported
Only absolute URLs are supported -
a. it may be due to passing relative URL to Axios in line 3, try checking the fileUrl.
b. check the face SDK is provided with the proper endpoint
relative URL - "./a/b"
absolute URL - "https://www.x.com/a/b"
how I might get from an array buffer image?
Array buffer is not needed. Looking at face SDK code Ref (hope this is right sdk), we just need streams.
we can get this by changing the responseType value to stream
responseType: 'arraybuffer', // current
responseType: 'stream', // updated - we can update it in rqs and reponse will be stream and you can pass res.data to detectWithStream function
-------------------------------------Hope the code works----------------------------------------
Recommendation:
You are creating a callback hell by using nested .then chains what is call back hell.
Try using async/await syntax to achieve the same functionality in a simplified way - Ref.
Regards,
Muhamed
I have the following code. Issue is, the file isn't always getting caught by my server (hosted on DigitalOcean, not that that matters so much as I often couldn't even get it to work on my local vagrant machine).
It seems to work with smaller files, smaller images, however if I send a larger file (such as a full screen screenshot) my server is telling me that the request has been made but no 'image' has been found.
I had a look around for a way for createReadStream to return some kind of promise, in hopes that it would be as simple as it is still trying to read the image from the hard drive, but I've had no luck.
Worth mentioning that the image is being generated just before this happens, but this is being called in a callback so that shouldn't matter.
...
const capture = fs.createReadStream(filePath);
request.post({
url: UPLOAD_URL,
method: 'POST',
formData: {
'image': capture,
'token': this.token
},
headers: {
'User-Agrent': 'request',
'Content-Type': 'multipart/form-data'
}
}, (err, response, body) => {
...
Yeah i kinda didn't know how to type the title well...
I've a node server which recives an image via post form. I then want to send this image to Microsoft vision and the same Google service in order to gether information from both, do some stuff, and return a result to the user that has accessed my server.
My problem is: how do i send the actual data?
This is the actual code that cares of that:
const microsofComputerVision = require("microsoft-computer-vision");
module.exports = function(req, res)
{
var file;
if(req.files)
{
file = req.files.file;
// Everything went fine
microsofComputerVision.analyzeImage(
{
"Ocp-Apim-Subscription-Key": vision_key,
"content-type": "multipart/form-data",
"body": file.data.toString(),
"visual-features":"Tags, Faces",
"request-origin":"westcentralus"
}).then((result) =>
{
console.log("A");
res.write(result);
res.end();
}).catch((err)=>
{
console.log(err);
res.writeHead(400, {'Content-Type': 'application/json'});
res.write(JSON.stringify({error: "The request must contain an image"}));
res.end();
});
}
else
{
res.writeHead(400, {'Content-Type': 'application/octet-stream'});
res.write(JSON.stringify({error: "The request must contain an image"}));
res.end();
}
}
If instead of calling "analyzeImage" i do the following
res.set('Content-Type', 'image/jpg')
res.send(file.data);
res.end();
The browser renders the image correctly, which made me think "file.data" contains the actual file (considered it's of type buffer).
But apparently Microsoft does not agree with that, because when i send the request to computer vision i get the following response:
"InvalidImageFormat"
The only examples i found are here, and the "data" that is used in that example comes from a file system read, not stright from a request. But saving the file to load it and then delete it to me looks like an horrible workaround, so i'd rather like to know in what form and how should i work on the "file" that i have to send it correctly for the APIs call.
Edit: if i use file.data (which i thought was the most correct since it would be sending the raw image as the body) i get an error which says that i must use a string or a buffer as content. So apparently that file.data is not a buffer in the way "body" requires O.o i'm not understanding honestly.
Solved, the error was quite stupid. In the "then" part, res.write(result) did not accept result as argument. This happened when i actually used the corret request (file.data which is a buffer). The other errors occurred everytime i tryed using toString() on file.data, in that case the request wasn't accepted.
Solved, the request asked for a buffer, and file.data is indeed a buffer. After chacking file.data type in any possible way i started looking for other problems. The error was much easier and, forgive my being stupid, too stupid to be evident. The result was a json, and res.write didn't accept a json as argument.
This is how I did it with Amazon Recognition Image Classifier, I know its not the same service your using - hoping this helps a little thou:
const imagePath = `./bat.jpg`;
const bitmap = fs.readFileSync(imagePath);
const params = {
Image: { Bytes: bitmap },
MaxLabels: 10,
MinConfidence: 50.0
};
route.post('/', upload.single('image'), (req, res) => {
let params = getImage();
rekognition.detectLabels(params, function(err, data) {
if (err) {
console.log('error');
}else {
console.log(data);
res.json(data);
}
});
});