I am trying to post an event to my IBM Social Business SmartCloud account. I have been able to grant access to the application and get the access and refresh tokens. but when posting the new even I get a 401 error "No 'Access-Control-Allow-Origin' header is present on the requested resource."
function postEvent(){
var postString = '{'+
'"actor": {'+
'"id": "#me"'+
'},'+
'"verb": "post",'+
'"title": "${share}",'+
'"content":"This event is my <b>first content</b>",'+
'"updated": "2012-01-01T12:00:00.000Z",'+
'"object": {'+
'"summary": "My Summary",'+
'"objectType": "note",'+
'"id": "someid",'+
'"displayName": "My displayName",'+
'"url": "mydomain.com"'+
'}}';
$.ajax({
url: 'https://apps.na.collabserv.com/connections/opensocial/basic/rest/activitystreams/#me/#all?format=json&access_token=<access_token>',
data: postString,
contentType: 'application/json',
method: 'POST',
dataType: 'json',
headers: {
// Set any custom headers here.
// If you set any non-simple headers, your server must include these
// headers in the 'Access-Control-Allow-Headers' response header.
'Content-Type: application/json',
'Origin': 'https://mydomain.com/',
'Access-Control-Allow-Headers' :'*',
'Access-Control-Allow-Origin': '*'
}
}).done(function(data) {
console.log(data);
});
}
so this is the proxy method using file_get_contents I used to get it working in php, cURL did not work.
$post = file_get_contents('https://apps.na.collabserv.com/connections/opensocial/basic/rest/activitystreams/#me/#all?format=json',FALSE,stream_context_create(array(
'http' => array(
'method' => 'POST',
'header' => "Authorization: Bearer $access_token\r\n".
"Content-type: application/json\r\n".
"Content-length: " . strlen($json_data) . "\r\n",
'content' => $json_data,
),
)));
now the issues is other people can not see my posting even though we're following each other.
got the json structure for embedding a webpage with og tags for a video into my ibm sb activity stream. so the video opens directly in my stream with a thumbnail without linking out in a new window
$json_data = '{"content":"https://mydomain.com/somevideo/",
"attachments":[{"objectType":"link",
"displayName":"My Display Name",
"url":"https://mydomain.com/somevideo/",
"summary":"My summary",
"image":{
"url":"{thumbnail}/api/imageProxy?url=http%3a%2f%2fmydomain.com%2fsomevideo%2fthumbnail.jpg",
"alt":"My Display Name"
},
"connections":{
"video":{
"connections":{"mime-type":"application/x-shockwave-flash"},
"width":"853",
"height":"480",
"url":"https://mydomain.com/somevideo/"
}
}
}
]
}';
and you post to this url:
https://apps.na.collabserv.com/connections/opensocial/rest/ublog/#me/#all?format=json
As long as your code is hosted on a different domain than apps.na.collabserv.com you won't be able to access the REST API with JavaScript alone
In this situation, it is the browser blocking you. The cross origin header won't work, because the backend is not configured to enable CORS requests.
You can work around this by accessing the REST API trough an ajax proxy, deployed on the same domain as your page
Related
I'm using scrape-it in my node.js scraping tool (for identifying proper keyword usage) but being identified as a bot by some websites and not getting any content. Is there a way to configure a known user agent header for the GET request to bypass the block?
You can set the headers, including User-agent, by passing an options object to scrape-it:
scrapeIt({
url: "http://example.com"
, headers: { "User-agent": "known-user-agent-of-choice" }
},
{
// some scrapeHTML options ...
})
.then(
// some code ...
);
I want to send multiple files to Github repository via nodejs. Tried several approaches and end up using node-rest-client module. Tried below code send a sample file to repository called 'metadata'. But after post I am getting error message "Request forbidden by administrative rules. Please make sure your request has a User-Agent header"...please let me know if anyone faced this error before and get rid of it.
convertval = "somedata";
var dataObj = {
"message": "my commit message",
"committer": {
"name": "Scott Chacon",
"email": "ravindra.devagiri#gmail.com"
},
"content": "bXkgbmV3IGZpbGUgY29udGVudHM="
}
debugger;
var Client = require('node-rest-client').Client;
var client = new Client()
var args = {
data: dataObj,
headers: { 'Content-Type': 'application/json' },
};
client.post("https://api.github.com/repos/metadata/contents", args, function (data, response) {
console.log("file send: True : " + data);
});
According to the REST API:
All API requests MUST include a valid User-Agent header. Requests with
no User-Agent header will be rejected.
First of all, you need to define 'User-Agent' with value 'request' in your request header. Refer to this link.
Second, endpoint you are trying to call might require authentication. Generate a personal token from here, add that token in your request header, 'Authorization': 'token '.
If you're using Git extensively in your code, I suggest you to use this - Nodegit.
Edit:
I don't think sending multiple files in a single request is possible in 'Contents' endpoints group (link).
You can checkout Git Data API (as discussed here).
I've been playing around with some web scraping but I've run into an issue I can't figure out; Using a nodejs server (on my local computer) I cannot get passed a permission error barring me from accessing the data. What is confusing to me most is that using the chrome extension "Postman" I don't run into the permission errors, but using the code generated by postman, I do (as well as fiddling with variations of my own scratch code).
Do I have to be using a live server? Do I need to include some extra items in the headers that aren't being put there by Postman? Is there some layer of security around the API that for some reason Postman has access do that a local machine doesnt?
Any light that can be shed would be of use. Note that there is no public documentation of the SmithsFoodAndDrug API (that I can find), so there aren't necessarily APIKeys that are going to be used. But the fact that Postman can access the information makes me think I should be able to on a node server without any special authentication set up.
In Summary:
I'm looking at SmithsFoodAndDrug product information, and found the API where they are grabbing information from.
I figured out the headers needed in order to get local price information on products (on top of the json body format for the POST request)
Using postman I can generate the POST request and retrieve the desired API results
Using nodejs (and the code generated by postman to replicate the request) with both 'request' module and standard 'http' module request module I receive permission errors from the server.
Details: (assume gathering data on honeycrisp apples (0000000003283) with division-id of 706 and store-id of 00144)
http://www.smithsfoodanddrug.com/products/api/products/details
Headers are 'division-id' and 'store-id'. Body is in format of {"upcs":["XXX"],"filterBadProducts":false} where XXX is the specific product code.
Here are the Request Headers in postman. Here are the Request Body settings in postman. The following is a portion of the json response (which is what I want).
{"products": [
{
"brandName": null,
"clickListItem": true,
"countryOfOrigin": "Check store for country of origin details",
"customerFacingSize": "price $2.49/lb",
...
"calculatedPromoPrice": "2.49",
"calculatedRegularPrice": "2.99",
"calculatedReferencePrice": null,
"displayTemplate": "YellowTag",
"division": "706",
"minimumAdvertisedPrice": null,
"orderBy": "Unit",
"regularNFor": "1",
"referenceNFor": "1",
"referencePrice": null,
"store": "00144",
"endDate": "2018-09-19T00:00:00",
"priceNormal": "2.55",
"priceSale": "2.12",
"promoDescription": "About $2.12 for each",
"promoType": null,
...
"upc": "0000000003283",
...
}
],
"coupons": {},
"departments": [],
"priceHasError": false,
"totalCount": 1 }
When using the code given by postman to replicate the request, I get the error saying 'You don't have permission to access "http://www.smithsfoodanddrug.com/products/api/products/details" on this server.
Reference #18.1f3de93f.1536955806.1989a2b1.' .
// Code given by postman
var request = require("request");
var options = { method: 'POST',
url: 'http://www.smithsfoodanddrug.com/products/api/products/details',
headers:
{ 'postman-token': 'ad9638c1-1ea5-1afc-925e-fe753b342f91',
'cache-control': 'no-cache',
'store-id': '00144',
'division-id': '706',
'content-type': 'application/json' },
body: { upcs: [ '0000000003283' ], filterBadProducts: false },
json: true };
request(options, function (error, response, body) {
if (error) throw new Error(error);
console.log(body);
});
change headers
headers:
{
'store-id': '00144',
'division-id': '706'
//'content-type': 'application/json'
}
I try to communicate with a REST api via xmls. I generate an xml and test it with Restlet Client (a chrome extenson for testing REST APIs).
It works fine, I receive the response XML.
If I try to send with request-promise, I receive
'<html><head><title>Error</title></head><body>Internal Server Error</body></html>'
in err.response.body.
The response from the API server is always XML, so I do not get why it is 'html'.
Here is the code
options = {
uri: uri,
header: {
'Content-Type': 'application/xml ',
'Accept' : 'application/xml '
},
body: xml // I use the same xml in Restlet Client's body
}
rp.post(options).then(response=>{
console.log('------SUCCESS-----')
console.log(response)
}).catch(err=>{
console.log('------ERROR-----')
console.log(err.response)
})
If you need any plus information, please let me know.
I am using nodejs and the REST API to interact with bigquery. I am using the google-oauth-jwt module for JWT signing.
I granted a service account write permission. So far I can list projects, list datasets, create a table and delete a table. But when it comes to upload a file via multipart POST, I ran into two problems:
gzipped json file doesn't work, I get an error saying "end boundary missing"
when I use uncompressed json file, I get a 401 unauthorized error
I don't think this is related to my machine's time being out of sync since other REST api calls worked as expected.
var url = 'https://www.googleapis.com/upload/bigquery/v2/projects/' + projectId + '/jobs';
var request = googleOauthJWT.requestWithJWT();
var jobResource = {
jobReference: {
projectId: projectId,
jobId: jobId
},
configuration: {
load: {
sourceFormat: 'NEWLINE_DELIMITED_JSON',
destinationTable: {
projectId: projectId,
datasetId: datasetId,
tableId: tableId
},
createDisposition: '',
writeDisposition: ''
}
}
};
request(
{
url: url,
method: 'POST',
jwt: jwtParams,
headers: {
'Content-Type': 'multipart/related'
},
qs: {
uploadType: 'multipart'
},
multipart: [
{
'Content-Type':'application/json; charset=UTF-8',
body: JSON.stringify(jobResource)
},
{
'Content-Type':'application/octet-stream',
body: fileBuffer.toString()
}
]
},
function(err, response, body) {
console.log(JSON.parse(body).selfLink);
}
);
Can anyone shine some light on this?
P.S. the documentation on bigquery REST api is not up to date on many things, wish the google guys can keep it updated
Update 1:
Here is the full HTTP request:
POST /upload/bigquery/v2/projects/239525534299/jobs?uploadType=multipart HTTP/1.1
content-type: multipart/related; boundary=71e00bd1-1c17-4892-8784-2facc6998699
authorization: Bearer ya29.AHES6ZRYyfSUpQz7xt-xwEgUfelmCvwi0RL3ztHDwC4vnBI
host: www.googleapis.com
content-length: 876
Connection: keep-alive
--71e00bd1-1c17-4892-8784-2facc6998699
Content-Type: application/json
{"jobReference":{"projectId":"239525534299","jobId":"test-upload-2013-08-07_2300"},"configuration":{"load":{"sourceFormat":"NEWLINE_DELIMITED_JSON","destinationTable":{"projectId":"239525534299","datasetId":"performance","tableId":"test_table"},"createDisposition":"CREATE_NEVER","writeDisposition":"WRITE_APPEND"}}}
--71e00bd1-1c17-4892-8784-2facc6998699
Content-Type: application/octet-stream
{"practiceId":2,"fanCount":5,"mvp":"Hello"}
{"practiceId":3,"fanCount":33,"mvp":"Hello"}
{"practiceId":4,"fanCount":71,"mvp":"Hello"}
{"practiceId":5,"fanCount":93,"mvp":"Hello"}
{"practiceId":6,"fanCount":92,"mvp":"Hello"}
{"practiceId":7,"fanCount":74,"mvp":"Hello"}
{"practiceId":8,"fanCount":100,"mvp":"Hello"}
{"practiceId":9,"fanCount":27,"mvp":"Hello"}
--71e00bd1-1c17-4892-8784-2facc6998699--
You are most likely sending duplicate content-type headers to the Google API.
I don't have the capability to effortlessly make a request to Google BigQuery to test, but I'd start with removing the headers property of your options object to request().
Remove this:
headers: {
'Content-Type': 'multipart/related'
},
The Node.js request module automatically detects that you have passed in a multipart array, and it adds the appropriate content-type header. If you provide your own content-type header, you most likely end up with a "duplicate" one, which does not contain the multipart boundary.
If you modify your code slightly to print out the actual headers sent:
var req = request({...}, function(..) {...});
console.log(req.headers);
You should see something like this for your original code above (I'm using the Node REPL):
> req.headers
{ 'Content-Type': 'multipart/related',
'content-type': 'multipart/related; boundary=af5ed508-5655-48e4-b43c-ae5be91b5ae9',
'content-length': 271 }
And the following if you remove the explicit headers option:
> req.headers
{ 'content-type': 'multipart/related; boundary=49d2371f-1baf-4526-b140-0d4d3f80bb75',
'content-length': 271 }
Some servers don't deal well with multiple headers having the same name. Hopefully this solves the end boundary missing error from the API!
I figured this out myself. This is one of those silly mistakes that would have you stuck for the whole day and at the end when you found the solution you would really knock on your own head.
I got the 401 by typing the selfLink URL in the browser. Of course it's not authorized.