There are a few topics and articles on Solr authentication & authorization, but I cannot get it to work (the way I like).
I followed these tutorials / information sources:
https://cwiki.apache.org/confluence/display/solr/Authentication+and+Authorization+Plugins
and
https://lucidworks.com/blog/2015/08/17/securing-solr-basic-auth-permission-rules/
Then I created this security.json and I confirmed it is active in Zookeeper:
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{
"solr":"...",
"admin":"...",
"monitor":"...",
"data_import":"..."},
"":{"v":8}},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"permissions":[
{
"name":"security-edit",
"role":"adminRole"},
{
"name":"security-read",
"role":"adminRole"},
{
"name":"schema-edit",
"role":"adminRole"},
{
"name":"schema-read",
"role":"collectionRole"},
{
"name":"config-edit",
"role":"adminRole"},
{
"name":"config-read",
"role":"collectionRole"},
{
"name":"collection-admin-edit",
"role":"adminRole"},
{
"name":"collection-admin-read",
"role":"collectionRole"},
{
"name":"update",
"role":"dataImportRole"},
{
"name":"read",
"role":"dataImportRole"}],
"user-role":{
"solr":[
"adminRole",
"collectionRole",
"dataImportRole"],
"admin":[
"adminRole",
"collectionRole",
"dataImportRole"],
"monitor":[
"collectionRole",
"dataImportRole"],
"data_import":["dataImportRole"]}}}
I now have a security.json that works for curl requests from command line:
curl "http://localhost:8983/solr/admin/authorization"
Unauthorized request, Response code: 401
curl --user solr:<pwd> "http://localhost:8983/solr/admin/authorization"
Normal response with the info
So far so good.
Now I try and select something from a collection, which shouldn't work anonymously according to my security.json, however it still works
curl "http://localhost:8983/solr/outlets_shard1_replica1/select?q=*%3A*&wt=json&indent=true"
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"indent":"true",
"q":"*:*",
"wt":"json"}},
"response":{"numFound":2000,"start":0,"d.. }
This is the first thing that vexes me. I probably can create some custom path permission for /select, but having the read right assigned to a specific role should do the trick right? but [1] How can I disable all anonymous access?
Continuing on, probably related, it bothers me that the Solr Admin UI(http://solrurl:8983/solr/#) is still accessible. In previous Solr installations (with tomcat) I remember that even this interface was secured. It also seems that I still have complete access to the entire core (reload worked) and I can also inspect cloud configuration.[2] How can I restrict access to Solr Admin UI?
The only stuff that actually seems to be secure is all the /solr/admin related commands
Which brings me to the 3rd thing I can't seem to figure out: How do I configure solr.in.sh so that solr authentication is passed with /bin/solr commands
I see the SOLR_AUTHENTICATION_CLIENT_CONFIGURER and SOLR_AUTHENTICATION_OPTS options, but I have no clue how to modify those to feed basic realm authentication into solr commandline. So [3] How do I keep all access from commandline to Solr (and Zookeeper) authorized & authenticated?
eg. solr status now returns
Found 1 Solr nodes:
Solr process 15931 running on port 8983
ERROR: Failed to get system information from http://localhost:8983/solr due to: org.apache.http.client.ClientProtocolException: Expected JSON response from server but received: <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 401 Unauthorized request, Response code: 401</title>
</head>
<body><h2>HTTP ERROR 401</h2>
<p>Problem accessing /solr/admin/collections. Reason:
<pre> Unauthorized request, Response code: 401</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>
</body>
</html>
I've tested with
SOLR_AUTHENTICATION_OPTS="-DinternalAuthCredentialsBasicAuthUsername=solr -DinternalAuthCredentialsBasicAuthPassword=<pass>"
To no avail
I also faced the same issue and then I looked at the source code.
The read permission in RuleBasedAuthorizationPlugin is defined as :
read :{" +
path:['/update/*', '/get']}," +
Which will never work.
I have raised an issue:
https://issues.apache.org/jira/browse/SOLR-8439
Now, to lock down your admin ui completely, you need to define a new permission, with path="/", which will going to solve your issue, something like this:
curl --user solr:SolrRocks http://localhost:8983/solr/admin/authorization -H 'Content-type:application/json' -d '{
"set-permission" : {"name":"admin-ui",
"path":"/",
"before":"update",
"role":"admin"}}'
Did you forget setting the blockUnknown to true?
Your authentication block in security.json should be:
"authentication":{
"blockUnknown": true,
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"..."}
},
If you don't set it, it will allow all anonymous access! It is strange but here is the source:
'blockUknown:true' means that unauthenticated requests are not allowed to pass through [1]
Start by using the default user/Pw given in the Solr tutorials.
The passwords are hashed sha512 with a salt. Solr provides the encryption from the plaintext passwords when you create new authenticated users, but the pw in the Lucidworks instructions is already encrypted for the plaintext value: solrRocks (or similar) - use that account to create others, give them appropriate permissions, then remove the solr:solrRocks account.
Related
So, this happen on my windows 10 machine, CouchDb 3.1.0 and Postman 7.29.1.
During a DB course class i couldn't do the same as my teacher and access:
http://127.0.0.1:5984/_all_dbs
was returning a 401 error:
{
"error": "unauthorized",
"reason": "You are not authorized to access this db."
}
I did some search but couldnt find nothing, than i start to messing things around.
** ATTENTION **
This solution is for local and unsecure enviroment and has by any mean the proposity to solve any other problem.
In the CouchDB/etc, open default.ini, and change the following lines:
; When true, system databases _users and _replicator are created immediately
; on startup if not present.
single_node = true
; prevent non-admins from accessing /_all_dbs
admin_only_all_dbs = false
Than restart Apach CouchDB service (or restart computer). and voila!
Hope that can help others and if my response is duplicated admins feel free to remove it.
Thanks all.
(sry for (m)any spelling/grammatic mistake)
Obs: Another thing that can happen is 401 on new DB. Go to DB permissions and remove _Admin from Members - Roles.
I have used the Crossref REST API where I just send it a query in a browser address bar, which then returns results in JSON.
So I send the following URL:
https://api.crossref.org/works?query.bibliographic=Randomized trial of intensive early intervention for children with pervasive developmental disorder&query.author=Groen&rows=1
I was hoping to so the same with the Microsoft REST API, but if I send it:
I get "Access denied due to invalid subscription key".
Can I pass my key via the URL? If so how?
Or is it not that simple.
Does it need other code as well - I can code in PHP if needed, or use jQuery.
Something like:
$(document).ready(function() {
$.ajax({
type:'GET',
url:'https://api.crossref.org/works?query.bibliographic=<?php echo $title ?>&query.author=<?php echo $author ?>&rows=1&select=is-referenced-by-count,author,title,DOI,issn-type,volume,issue,link,page,abstract',
success:function(result) {
var total_results = result.message["total-results"];
}
});
But again, with the Microsoft API - how would I send it the keys?
Thank you.
What is "the Microsoft REST API"? Do you mean Azure API Management? "Invalid subscription key" implies that this is an API hosted in API Management which uses this term to describe its authentication model. The following answer assumes this is the case.
You would need to be registered as a user and be given a subscription key. There is a self-service portal for doing this, which is described by this documentation. It is up to the API's administrator whether you are permitted to self-service or not, so you may or may not be able to do this yourself, or you may have to request the administrator to register you. Further documentation describes a bit how to use the portal.
When you have it, you would apply it as an http header named Ocp-Apim-Subscription-Key, but the name it expects is also configurable and may have been changed, which the administrator would have to tell you.
I finally found the documentation for what I needed (the Microsoft documentation around Azure is a horrendous maze with a lot of stuff that is outdated and broken).
So the documentation is here:
https://msr-apis.portal.azure-api.net/docs/services/academic-search-api/operations/565d753be597ed16ac3ffc03?
I modified the Jacascript example at the bottom of page and came up with:
<!DOCTYPE html>
<head>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
<script type="text/javascript">
$(document).ready(function() {
$.ajax({
type:'GET',
url:'https://api.labs.cognitive.microsoft.com/academic/v1.0/evaluate?expr=Composite(AA.AuN==%27jaime%20teevan%27)&count=2&attributes=Ti,Y,CC,AA.AuN,AA.AuId',
beforeSend: function(xhrObj){
// Request headers
xhrObj.setRequestHeader("Ocp-Apim-Subscription-Key","xxxxmykeyxxxxx");
},
success:function(result) {
alert(result.entities[0].Ti);
}
});
});
</script>
</head>
</html>
The query returns all the titles for the author "jamie teevan".
Though in this example I am just outputting the first title via 'alert'. I haven't coded the rest of it yet - just wanted to know if it worked or not at this point.
Now all I need to do is work out the expression to return all the citing documents for a given title + author!! :-/ wish me luck.
I have registred a sandbox user. Now I am trying to use it, but I recieve "The access_token provided is invalid".
Sample request:
https://api.instagram.com/v1/users/searchq=abc&count=2&client_id=7b67cff1a7ab462881298434c08b5ab8
It was working in live mode, but I can't find why it isn't working in sandbox.
You need to check for requirements Scope for API you are using and if it's need and Authentications "Valid access Token". where also had some changes.
basic - to read a user’s profile info and media
public_content to read any public profile info and media on a user’s behalf
follower_list to read the list of followers and followed-by users
comments - to post and delete comments on a user’s behalf
relationships - to follow and unfollow accounts on a user’s behalf
likes - to like and unlike media on a user’s behalf
And take look about **Sandbox API behavior of your application not life yet :**
API Behavior
The behavior of the API when you are in sandbox mode is the same as when your app is live, but comes with the following restrictions:
Data is restricted to sandbox users and the 20 most recent media from each sandbox user
Reduced API rate limits
The first point is important and it means that the API behaves as if the only users on Instagram were your sandbox users, and the only media ever posted were the last 20 for each of these users.
For example, if you query the /users/{user-id}/ endpoint and the {user-id} is a sandbox user, then you will get the normal API response; but if the {user-id} is not a sandbox user, you will get a APINotFound error.
Good luck
For me the access token Instagram generate is too short: they gave me something like this:
d2c387d768ec4d619306807c53bbf92b
it should be:
2940736713.1677ed0.31bb22b2b0f84befacc79c6afd025134
I used this site to generate it:
http://instagram.pixelunion.net
I found it when solving the same problem with access by access_token
Here is official faq how to get access_token
You just should go through following steps (link to Instagram API documentation https://www.instagram.com/developer/authentication/):
1.Request the CODE
https://api.instagram.com/oauth/authorize/?client_id=CLIENT-ID&redirect_uri=REDIRECT-URI&response_type=code
change CLIENT_ID to you client id (you could get it here https://www.instagram.com/developer/clients/manage/ )
change REDIRECT-URI to your redirect_uri from tab security in client settings(open link above and tap to 'manage' button). I Used this one https://meyerweb.com/eric/tools/dencoder/ Encode url here and paste te result instead REDIRECT-URI
After you prepared it cope ready link to adress bar in your browser and run
2. Get the CODE
Browser ask you a premissons and redirect you to callback url with the code in the end. It will be look like this in adress bar:
http://yoursite.some/?code=d8af5619af6853d4ad11b4dd5f1ef17e
In this example your code is d8af5619af6853d4ad11b4dd5f1ef17e
Save it. You should use it in access_token request below
3. Make an access_token request
Open terminal and use curl:
curl -F 'client_id=CLIENT_ID' \
-F 'client_secret=CLIENT_SECRET' \
-F 'grant_type=authorization_code' \
-F 'redirect_uri=AUTHORIZATION_REDIRECT_URI' \
-F 'code=CODE' \
https://api.instagram.com/oauth/access_token
Change CLIENT_ID, CLIENT_SECRET, AUTHORIZATION_REDIRECT_URI and CODE to your own data and run it.
Curl return you JSON with access_token like this:
{
"access_token": "fb2e77d.47a0473320504cb3ab4a1f626d174d2d",
"user": {
"id": "1576583",
"username": "some",
"full_name": "Some",
"profile_picture": "..."
}
}
Well done! In this example fb2e77d.47a0473320504cb3ab4a1f626d174d2d is your access_token. Copy access_token value, use it in your app and enjoy you coding! :)
My issue was using v1 of Instafeed.js, rather than v2.
Once I switched to v2, everything started working.
<script src="/instafeed/dist/instafeed.min.js"></script>
<script src="https://ig.instant-tokens.com/users/.../instagram/.../token.js?userSecret=..."></script>
<div id="instafeed" class="instagram-feed">
</div>
<script>
(function(){
console.log(InstagramToken);
window.addEventListener('load', function () {
var feed = new Instafeed({
accessToken: InstagramToken,
});
feed.run();
});
})();
</script>
You haven't specified an access_token for your request.
I'm trying to use the Power BI REST API, using an access token acquired with the "client credentials" method, but I keep getting 403 Forbidden on my requests.
My code follows the pattern demonstrated in this AzureAD sample. In fact, to isolate this problem, I'm running that sample code (with my own values in the parameters.json, of course):
{
expiresIn: 3599,
tokenType: 'Bearer',
expiresOn: Tue Sep 01 2015 16:56:07 GMT-0500 (CDT),
resource: '00000002-0000-0000-c000-000000000000',
accessToken: 'eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Ik1uQ19WWmNBVGZNNXBPWWlKSE1iYTlnb0VLWSIsImtpZCI6Ik1uQ19WWmNBVGZNNXBPWWlKSE1iYTlnb0VLWSJ9.eyJhdWQiOiIwMDAwMDAwMi0wMDAwLTAwMDAtYzAwMC0wMDAwMDAwMDAwMDAiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8xM2QxNzIwNC0wZGU2LTQ1NzQtOTgzYS05NjFhYjk0M2M3Y2UvIiwiaWF0IjoxNDQxMTQwNjcwLCJuYmYiOjE0NDExNDA2NzAsImV4cCI6MTQ0MTE0NDU3MCwidmVyIjoiMS4wIiwidGlkIjoiMTNkMTcyMDQtMGRlNi00NTc0LTk4M2EtOTYxYWI5NDNjN2NlIiwib2lkIjoiYzM1ZWQyYTktYTYzZS00YzAwLThmYmYtY2FlYjlmZjYwMjYwIiwic3ViIjoiYzM1ZWQyYTktYTYzZS00YzAwLThmYmYtY2FlYjlmZjYwMjYwIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvMTNkMTcyMDQtMGRlNi00NTc0LTk4M2EtOTYxYWI5NDNjN2NlLyIsImFwcGlkIjoiNDQ2Y2Y5OTItMDQzYS00YjgxLWJhYzQtY2RlZWYyNGFhNzFjIiwiYXBwaWRhY3IiOiIxIn0.YTGJfdW1wP09bDHwwsv3FPAmEpmQdc_kifvgY-1KjhkZWANfYtd050wfeZdNgMUeSPZyFdWnoBjnJ4xrlDtnsADwV1Grr6TXYcymPLofbY-xy0cjyvzxTmM11DJ9XN8A4tkgvK0jtR-YyIjPw5EKJSKyeEbD9U3mWsE_gu7IzKzXl8e-dfVAqRYS6WHZy6_0FaNmppPDls5s_QIPOHofFSiWVISw41Mz0fQnP2QEGyceOCvKYJtrUOCDwfVuwFS-gSLmYvEGOJfmIjftP3srda0JPirVzBeU0IFJJ1KW81kE5cfKw1KkBB04VVetRUs_7HqloYaKKiTybauhXAodRQ',
isMRRT: true,
_clientId: '[snip]',
_authority: 'https://login.windows.net/[snip]'
}
When I use that access token in a curl request, as follows, I get a 403:
curl -vv -X GET https://api.powerbi.com/v1.0/myorg/datasets -H"Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6Ik1uQ19WWmNBVGZNNXBPWWlKSE1iYTlnb0VLWSIsImtpZCI6Ik1uQ19WWmNBVGZNNXBPWWlKSE1iYTlnb0VLWSJ9.eyJhdWQiOiIwMDAwMDAwMi0wMDAwLTAwMDAtYzAwMC0wMDAwMDAwMDAwMDAiLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC8xM2QxNzIwNC0wZGU2LTQ1NzQtOTgzYS05NjFhYjk0M2M3Y2UvIiwiaWF0IjoxNDQxMTQwNjcwLCJuYmYiOjE0NDExNDA2NzAsImV4cCI6MTQ0MTE0NDU3MCwidmVyIjoiMS4wIiwidGlkIjoiMTNkMTcyMDQtMGRlNi00NTc0LTk4M2EtOTYxYWI5NDNjN2NlIiwib2lkIjoiYzM1ZWQyYTktYTYzZS00YzAwLThmYmYtY2FlYjlmZjYwMjYwIiwic3ViIjoiYzM1ZWQyYTktYTYzZS00YzAwLThmYmYtY2FlYjlmZjYwMjYwIiwiaWRwIjoiaHR0cHM6Ly9zdHMud2luZG93cy5uZXQvMTNkMTcyMDQtMGRlNi00NTc0LTk4M2EtOTYxYWI5NDNjN2NlLyIsImFwcGlkIjoiNDQ2Y2Y5OTItMDQzYS00YjgxLWJhYzQtY2RlZWYyNGFhNzFjIiwiYXBwaWRhY3IiOiIxIn0.YTGJfdW1wP09bDHwwsv3FPAmEpmQdc_kifvgY-1KjhkZWANfYtd050wfeZdNgMUeSPZyFdWnoBjnJ4xrlDtnsADwV1Grr6TXYcymPLofbY-xy0cjyvzxTmM11DJ9XN8A4tkgvK0jtR-YyIjPw5EKJSKyeEbD9U3mWsE_gu7IzKzXl8e-dfVAqRYS6WHZy6_0FaNmppPDls5s_QIPOHofFSiWVISw41Mz0fQnP2QEGyceOCvKYJtrUOCDwfVuwFS-gSLmYvEGOJfmIjftP3srda0JPirVzBeU0IFJJ1KW81kE5cfKw1KkBB04VVetRUs_7HqloYaKKiTybauhXAodRQ"
Wondering if that curl request was flawed somehow, I snooped out an access token “the wrong way” via browser webtools, and the above works fine, returning a 200 and a JSON response listing my datasets.
I did also notice that the return code is 403 (forbidden), not 401 (unauthorized), so I wondered if the authorization was okay but the permissions on the Power BI side were wrong. But I also get 403 when I use any garbage text for the access token (e.g., Authorization: Bearer foo), so I discarded that theory.
So. I think I have a valid test, and I’m getting what I think is a valid access token (from that client-credentials-sample.js code), but it’s still not working. What am I missing?
With the assistance of some Microsoft folks (thanks, Jon Gallant & Josh Caplan), I've learned that authenticating with an OAuth client-credentials flow, as I was doing with that JavaScript sample, provides insufficient access. To use Power BI, authentication needs to be based on a particular user.
I tried using:
the similar JavaScript sample username-password-sample.js
a resource value of https://analysis.windows.net/powerbi/api (thanks, slugslog)
adding username and password to the parameters.json
That got me closer, but I was still getting a 400 response: "error_description":"AADSTS90014: The request body must contain the following parameter: 'client_secret or client_assertion'. …".
A hack to the adal-node library (hardcoding the client secret, i.e., oauthParameters[OAuth2Parameters.CLIENT_SECRET] = "my-client-secret"; after line 217 of token-request.js) was enough to get back an access token which works in the Authorization header for my original curl call.
Of course hardcoding that value in there isn't my final solution. I don't plan to use the adal-node library, anyway. But as far as this proof-of-concept for this authentication case goes, that's the answer I came to.
Make sure that your app that you registered with AAD has the read write all datasets permission. That should solve the problem.
This is not an answer but one step forward in the debug process. I think the resource for which the token is requested should be "https://analysis.windows.net/powerbi/api". I've seen these in multiple references; one of them is linked below. Even after changing this, I still get a 403. As the OP mentioned if we use the accessToken from the powerBI portal, everything works.
So I tried this with my own app, the following command works (for me):
curl -vv -X GET https://api.powerbi.com/v1.0/myorg/datasets -H"Authorization: Bearer ey....qqqq"
BTW, the extra "v" after -v seems redundant.
So what I can conclude is that your application is missing the required permissions to call Power BI's APIs.
One thing you might try is grab one of our samples, create a new application in AAD for it, and then see if the authorization token works for it. Here's a good one to try: https://github.com/PowerBI/Integrate-a-tile-into-an-app
I'm building a webapp with google appengine, node.js and socket.io, and I'm currently trying to set up a google compute instance to use the google cloud datastore api following this tutorial. So far, I've completed steps 1 and 2, but when running the downloaded adams.js file locally, I get this:
Error: No access or refresh token is set.
Stuff I've tried:
gcloud auth login (this logs me in google, but doesn't set my token locally)
changing the way I'm exporting my DATASTORE_SERVICE_ACCOUNT and DATASTORE_PRIVATE_KEY_FILE values, as strings, plain text, etc.
logging my credentials on the line before the error (I'm definitely missing a token)
creating a new service account and going through key creation steps again
ran
curl "http://metadata/computeMetadata/v1/instance/service-accounts/default/token" -D "Metadata-Flavor: Google" to get more info.
that command gives me this:
<html>
<head><title>301 Moved Permanently</title></head>
<body bgcolor="white">
<center><h1>301 Moved Permanently</h1></center>
<hr><center>nginx</center>
</body>
</html>
so I'm definitely missing a token, but for some reason I cannot figure out how I am supposed to be getting it. Or where I need to store it when my app is deployed and wants to access the API online. It feels like this should be a really easy thing to find, but for some reason all the documentation I'm reading is leading me into walls. Any help would be awesome.
If your code is running on a Google Compute Instance, and the instance has the correct scopes, you don't need to set any environment variables.
You can confirm the scopes by looking at the instance in the Developers Console or by asking the metadata server:
curl http://metadata/computeMetadata/v1/instance/service-accounts/default/scopes -H "Metadata-Flavor: Google"
To use Cloud Datastore, both the datastore and userinfo.email scopes must be present.