I have a simple code that's supposed to download images from slack messages.
var url = message.file.private_url;
var destination_path = './tmp/uploaded';
var opts = {
method: 'GET',
url: url,
headers: {
Authorization: 'Bearer ' + process.env.botToken,
}
};
request(opts, function(err, res, body) {
console.log('FILE RETRIEVE STATUS',res.statusCode);
}).pipe(fs.createWriteStream(destination_path));
The code worked fine for a while, but now I'm getting this error:
An error occured in the receive middleware: TypeError: Cannot read property 'private_url' of undefined
Any help would be appreciated!
Are you using the events API?
Several changes have been made to the API recently (both Events and Web APIs). See here: https://api.slack.com/changelog/2018-05-file-threads-soon-tread
If you describe the API you're using, I might be able to provide more specific help but I suspect the issue (as described in the link above) is that the file attribute attached to messages has been replaced with a new files field (an array). The files in the array are also in a different format.
Check the JSON payload. It probably contains a files array.
Related
We have a problem with the fleet.ls.hereapi.com when uploading a new layer for geofencing.
const myId = 'MYLAYER'; // just a id to check
zip.file('data.wkt', `NAME\tWKT\n${myId}\t${wkt}`);
const content = await zip.generateAsync({ type: 'nodebuffer' });
const formData = new FormData();
formData.append('zipfile', content);
await axios.post(config.HERE_MAPS_UPLOAD_ENDPOINT, formData, {
headers: {
'content-type': 'multipart/form-data',
},
params: {
apiKey: config.HERE_MAPS_REST_API_KEY,
layer_id: myId,
},
});
We get a bad request without a message and do not know what the problem is. The same implementation works in the Frontend (with 'blob' as zip type). Is there a parameter to get a better error message from the api?
We got the instructions how to implement it from this tutorial: https://www.youtube.com/watch?v=Ayw9GcS1V-8 and as I mentioned it works fine in the frontend. Also it works if I write a file in node and upload it via curl. Thank you for any help in advance!
Edit: I'm getting the following issue from the API: 'Multipart should contain exactly one part but contains 0'
I fixed it!
The problem was that the api needed a filename for the form data. This filename can be provided as third parameter as described here.
So I basically changed formData.append('zipfile', content); to formData.append('zipfile', content, zipfile.zip); and it worked.
Hope this will help somebody in the future!
I want to send multiple files to Github repository via nodejs. Tried several approaches and end up using node-rest-client module. Tried below code send a sample file to repository called 'metadata'. But after post I am getting error message "Request forbidden by administrative rules. Please make sure your request has a User-Agent header"...please let me know if anyone faced this error before and get rid of it.
convertval = "somedata";
var dataObj = {
"message": "my commit message",
"committer": {
"name": "Scott Chacon",
"email": "ravindra.devagiri#gmail.com"
},
"content": "bXkgbmV3IGZpbGUgY29udGVudHM="
}
debugger;
var Client = require('node-rest-client').Client;
var client = new Client()
var args = {
data: dataObj,
headers: { 'Content-Type': 'application/json' },
};
client.post("https://api.github.com/repos/metadata/contents", args, function (data, response) {
console.log("file send: True : " + data);
});
According to the REST API:
All API requests MUST include a valid User-Agent header. Requests with
no User-Agent header will be rejected.
First of all, you need to define 'User-Agent' with value 'request' in your request header. Refer to this link.
Second, endpoint you are trying to call might require authentication. Generate a personal token from here, add that token in your request header, 'Authorization': 'token '.
If you're using Git extensively in your code, I suggest you to use this - Nodegit.
Edit:
I don't think sending multiple files in a single request is possible in 'Contents' endpoints group (link).
You can checkout Git Data API (as discussed here).
I'm still new enough with Node that HTTP requests trip me up. I have checked all the answers to similar questions but none seem to address my issue.
I have been dealt a hand in the Wild of having to go after JSON files in an API. I then parse those JSON files to separate them out into rows that populate a SQL database. The API has one JSON file with an ID of 'keys.json' that looks like this:
{
"keys":["5sM5YLnnNMN_1540338527220.json","5sM5YLnnNMN_1540389571029.json","6tN6ZMooONO_1540389269289.json"]
}
Each array element in the keys property holds the value of one of the JSON data files in the API.
I am having problems getting either type of file returned to me, but I figure if I can learn what is wrong with the way I am trying to get 'keys.json', I can leverage that knowledge to get the individual JSON data files represented in the keys array.
I am using the npm modules 'request' and 'request-promise-native' as follows:
const request = require('request');
const rp = require('request-promise-native');
My URL is constructed with the following elements, as follows (I have used the ... to keep my client anonymous, but other than that it is a direct copy:
let baseURL = 'http://localhost:3000/Users/doug5solas/sandbox/.../server/.quizzes/'; // this is the development value only
let keysID = 'keys.json';
Clearly the localhost aspect will have to go away when we deploy but I am just testing now.
Here is my HTTP call:
let options = {
method: 'GET',
uri: baseURL + keysID,
headers: {
'User-Agent': 'Request-Promise'
},
json: true // Automatically parses the JSON string in the response
};
rp(options)
.then(function (res) {
jsonKeysList = res.keys;
console.log('Fetched', jsonKeysList);
})
.catch(function (err) {
// API call failed
let errMessage = err.options.uri + ' ' + err.statusCode + ' Not Found';
console.log(errMessage);
return errMessage;
});
Here is my console output:
http://localhost:3000/Users/doug5solas/sandbox/.../server/.quizzes/keys.json 404 Not Found
It is clear to me that the .catch() clause is being taken and not the .then() clause. But I do not know why that is because the data is there at that spot. I know it is because I placed it there manually.
Thanks to #Kevin B for the tip regarding serving of static files. I revamped the logic using express.static and served the file using that capability and everything worked as expected.
I have an Excel file (.xlsx), which when I have it already in my OneDrive, I can use a REST command like this, to modify it:
/v1.0/me/drive/root:/SpreadSheetName.xlsx:/workbook/worksheets/content_stats/tables('RawStats')/Rows
When I modify my code to first upload the file to OneDrive (rather than using the file that is already there), and I use the REST API, I get the error:
Open navigation properties are not supported on OpenTypes. Property name: 'tables'.
I have searched the web for this message, and cannot find anything related to what I am doing. The REST call for modifying the file which was just uploaded is nearly identical, although I do reference the file by ID instead, as that is what is returned by the upload API. This is the URL I use to modify the file which was just uploaded.
/v1.0/me/drive/items:/<RealExcelSpreadsheetID>:/workbook/worksheets/content_stats/tables('RawStats')/Rows
Both are doing a POST. Exact same file, only difference is it is being uploaded first, rather than already being in OneDrive. The file was definitely uploaded correctly, as when I go through the OneDrive web interface I do find it and can view it online. This is a business account.
It was uploaded as MIME type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet.
It uses these scopes delegated via OAuth2:
User.Read
User.ReadWrite
Files.Read
Files.ReadWrite
Files.ReadWrite.All
Sites.ReadWrite.All
Using Node.js and JavaScript, although that should't matter.
Here is the code used to upload the file:
function copyTemplateInOneDrive(res, queryParam, officeAccessToken, callback) {
var fs = require('fs');
var excelExt = ".xlsx";
var excelSpreadsheetFilenameStart = "stats";
var uploadUrl = "https://graph.microsoft.com/v1.0/me/drive/root:/" +
excelSpreadsheetFilename + dateNowFull() + excelExt + ":/content";
var xlsxMimeType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
fs.readFile(excelSpreadsheetTemplateFilename, function read(error, fileContent) {
var request = require('request');
var options = {
url: uploadUrl,
headers: {
'Accept': 'application/json',
'Authorization': 'Bearer ' + officeAccessToken,
'Content-Type': xlsxMimeType,
},
body: fileContent
};
request.put(options, function (error, response, body) {
var result = JSON.parse(body);
if ('id' in result) {
console.log("Successfully uploaded template file to OneDrive");
res.write("Successfully uploaded template file to OneDrive");
var excelSpreadsheetID = result.id;
excelSpreadsheetUrl = result.webUrl;
} else {
console.log("ERROR: unable to upload template file to OneDrive " + body);
res.write("Error: unable to upload template file to OneDrive" + body);
return;
}
callback(null, res, queryParam, officeAccessToken, excelSpreadsheetID);
});
});
}
It uses the async module from node.js (which makes use of callback). It also saves the ID returned, and later passes it into the call to the Microsoft Graph.
Your issue here is the URL you're calling with the id is using syntax expecting the file path (i.e. folder\filename.ext) rather than the id. This is why switching to the file name started working for you.
There are two ways to address a file stored in OneDrive:
drive/items/{item-id}
/drive/root:/path/to/file (note the :)
You correctly switched your URI from drive/root to drive/items but by leaving the : in place you are telling OneDrive to address the file by it's path rather than it's id. In other words, it's looking for a file named "{some-id}".
For addressing a file by it's path, your URL is correct:
/drive/root:/{file-path}:/workbook/worksheets/content_stats/tables('RawStats')/Rows
For addressing a file by it's id however, you need to drop the ::
/drive/items/{file-id}/workbook/worksheets/content_stats/tables('RawStats')/Rows
You can read about how files are addressed in the documentation for DriveItem.
I am trying to get data from the Bing search API, and since the existing libraries seem to be based on old discontinued APIs I though I'd try myself using the request library, which appears to be the most common library for this.
My code looks like
var SKEY = "myKey...." ,
ServiceRootURL = 'https://api.datamarket.azure.com/Bing/Search/v1/Composite';
function getBingData(query, top, skip, cb) {
var params = {
Sources: "'web'",
Query: "'"+query+"'",
'$format': "JSON",
'$top': top, '$skip': skip
},
req = request.get(ServiceRootURL).auth(SKEY, SKEY, false).qs(params);
request(req, cb)
}
getBingData("bookline.hu", 50, 0, someCallbackWhichParsesTheBody)
Bing returns some JSON and I can work with it sometimes but if the response body contains a large amount of non ASCII characters JSON.parse complains that the string is malformed. I tried switching to an ATOM content type, but there was no difference, the xml was invalid. Inspecting the response body as available in the request() callback actually shows bad code.
So I tried the same request with some python code, and that appears to work fine all the time. For reference:
r = requests.get(
'https://api.datamarket.azure.com/Bing/Search/v1/Composite?Sources=%27web%27&Query=%27sexy%20cosplay%20girls%27&$format=json',
auth=HTTPBasicAuth(SKEY,SKEY))
stuffWithResponse(r.json())
I am unable to reproduce the problem with smaller responses (e.g. limiting the number of results) and unable to identify a single result which causes the issue (by stepping up the offset).
My impression is that the response gets read in chunks, transcoded somehow and reassembled back in a bad way, which means the json/atom data becomes invalid if some multibyte character gets split, which happens on larger responses but not small ones.
Being new to node, I am not sure if there is something I should be doing (setting the encoding somewhere? Bing returns UTF-8, so this doesn't seem needed).
Anyone has any idea of what is going on?
FWIW, I'm on OSX 10.8, node is v0.8.20 installed via macports, request is v2.14.0 installed via npm.
i'm not sure about the request library but the default nodejs one works well for me. It also seems a lot easier to read than your library and does indeed come back in chunks.
http://nodejs.org/api/http.html#http_http_request_options_callback
or for https (like your req) http://nodejs.org/api/https.html#https_https_request_options_callback (the same really though)
For the options a little tip: use url parse
var url = require('url');
var params = '{}'
var dataURL = url.parse(ServiceRootURL);
var post_options = {
hostname: dataURL.hostname,
port: dataURL.port || 80,
path: dataURL.path,
method: 'GET',
headers: {
'Content-Type': 'application/json; charset=utf-8',
'Content-Length': params.length
}
};
obviously params needs to be the data you want to send
I think your request authentication is incorrect. Authentication has to be provided before request.get.
See the documentation for request HTTP authentication. qs is an object that has to be passed to request options just like url and auth.
Also you are using same req for second request. You should know that request.get returns a stream for the GET of url given. Your next request using req will go wrong.
If you only need HTTPBasicAuth, this should also work
//remove req = request.get and subsequent request
request.get('http://some.server.com/', {
'auth': {
'user': 'username',
'pass': 'password',
'sendImmediately': false
}
},function (error, response, body) {
});
The callback argument gets 3 arguments. The first is an error when applicable (usually from the http.Client option not the http.ClientRequest object). The second is an http.ClientResponse object. The third is the response body String or Buffer.
The second object is the response stream. To use it you must use events 'data', 'end', 'error' and 'close'.
Be sure to use the arguments correctly.
You have to pass the option {json:true} to enable json parsing of the response