On occasion I need to email all Jenkins users, for example warning them Jenkins will be offline for maintenance. The script below gives me email addresses for all people that Jenkins knows about, but it includes people that don’t have accounts, these are people that have committed changes that triggered builds. I’d like to remove them from the email list, that is the list should only include the users that you’d see on the securityRealm page. I’m looking for help modifying the script. Thanks.
import hudson.model.User
import hudson.tasks.Mailer
def users = User.getAll()
for (User u : users) {
def mailAddress = u.getProperty(Mailer.UserProperty.class).getAddress()
print(mailAddress + "; ")
}
First of all you should know that Jenkins will not always be able to tell you whether the user exists or not. From Jenkins' javadoc:
This happens, for example, when the security realm is on top of the servlet implementation, there's no way of even knowing if an user of a given name exists or not.
I found two solutions.
Solution 1
HudsonPrivateSecurityRealm.html#getAllUsers() returns all users who can login to the system. And this works for me:
import hudson.model.User
import hudson.tasks.Mailer
import jenkins.model.Jenkins
def realm = Jenkins.getInstance().getSecurityRealm()
def users = realm.getAllUsers()
for (User u : users) {
def mailAddress = u.getProperty(Mailer.UserProperty.class).getAddress()
print(mailAddress + "; ")
}
Note: this depends on the Jenkins config and may not work on the system where used another (not a HudsonPrivateSecurityRealm) security realm.
Solution 2
SecurityRealm#loadUserByUsername returns user details if user exists and throws UsernameNotFoundException otherwise:
import hudson.model.User
import hudson.tasks.Mailer
import jenkins.model.Jenkins
import org.acegisecurity.userdetails.UsernameNotFoundException
def realm = Jenkins.getInstance().getSecurityRealm()
def users = User.getAll()
for (User u : users) {
try {
realm.loadUserByUsername(u.getId()) // throws UsernameNotFoundException
def mailAddress = u.getProperty(Mailer.UserProperty.class).getAddress()
print(mailAddress + "; ")
} catch (UsernameNotFoundException e) { }
}
This is tricky one, but should work with all security realms as we use the method that exists in top level abstract class (SecurityRealm).
Related
I'm using Django 3.2 and djangorestframework==3.12.2. I recently added this to my settings file because I want to add some secured endpoints to my application ...
REST_FRAMEWORK = {
'DEFAULT_PERMISSION_CLASSES': [
'rest_framework.permissions.IsAuthenticated',
'rest_framework.permissions.IsAdminUser',
],
'DEFAULT_AUTHENTICATION_CLASSES': (
'rest_framework_jwt.authentication.JSONWebTokenAuthentication',
)
}
JWT_AUTH = {
# how long the original token is valid for
'JWT_EXPIRATION_DELTA': datetime.timedelta(hours=1),
}
However, this seems to have caused all my endpoints to require authentication. For example, I had this view set up in my views.py file
class CoopList(APIView):
"""
List all coops, or create a new coop.
"""
def get(self, request, format=None):
contains = request.GET.get("contains", "")
if contains:
coops = Coop.objects.find(
partial_name=contains,
enabled=True
)
else:
partial_name = request.GET.get("name", "")
enabled_req_param = request.GET.get("enabled", None)
enabled = enabled_req_param.lower() == "true" if enabled_req_param else None
city = request.GET.get("city", None)
zip = request.GET.get("zip", None)
street = request.GET.get("street", None)
state = request.GET.get("state", None)
coop_types = request.GET.get("coop_type", None)
types_arr = coop_types.split(",") if coop_types else None
coops = Coop.objects.find(
partial_name=partial_name,
enabled=enabled,
street=street,
city=city,
zip=zip,
state_abbrev=state,
types_arr=types_arr
)
serializer = CoopSearchSerializer(coops, many=True)
return Response(serializer.data)
accessible in my urls.py file using
path('coops/', views.CoopList.as_view()),
But now when I try and call that I get the below response
{"detail":"Authentication credentials were not provided."}
I only want certain views/endpoints secured. How do I make the default that all views are accessible and only specify some views/endpoints to be validated using a provided JWT?
'DEFAULT_PERMISSION_CLASSES' is conventiently applied to all views, unless manually overridden. In your case both listed permissions require the user to be authenticated. FYI, the list is evaluated in an OR fashion.
If you want to allow everyone by default and only tighten down specific views, you want to set
'DEFAULT_PERMISSION_CLASSES': ['rest_framework.permissions.AllowAny']
which does not require the user to be authenticated. Then set more strict permissions explicitly on the view (e.g. permissions_classes = [IsAuthenticated]) The DEFAULT_AUTHENTICATION_CLASS can stay as is.
NOTE: It is generally advisable to do it the other way round. It's very easy to accidentally expose an unsecured endpoint like this and potentially create a security breach in your API. The default should be secure and then exceptions should be be manually lifted.
Set the below configuration in settings.py
'DEFAULT_PERMISSION_CLASSES': [
'rest_framework.permissions.IsAuthenticated',
]
For class based views you can set permission class to empty list.
class CoopList(APIView):
permission_classes = []
def get(self, request, format=None):
pass
For Function based views add the decorator #permission_classes
from rest_framework.decorators import permission_classes
#permission_classes([])
def CoopList(request, format=None):
pass
I am trying to create an event using Google Calendar API in Python 3. I also want to generate a Google Meet conference link for the event. I am using the documentations provided here:
https://developers.google.com/calendar/quickstart/python
https://developers.google.com/calendar/v3/reference/events#conferenceData
https://developers.google.com/calendar/create-events
The event is created without a problem. However, it is missing the conference link. My code so far is as follows:
from pathlib import Path
from pickle import load
from pickle import dump
from google.auth.transport.requests import Request
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from uuid import uuid4
from typing import Dict, List
from oauth2client import file, client, tools
class EventPlanner:
def __init__(self, guests: Dict[str, str], schedule: Dict[str, str]):
guests = [{"email": email} for email in guests.values()]
service = self._authorize()
self.event_states = self._plan_event(guests, schedule, service)
#staticmethod
def _authorize():
scopes = ["https://www.googleapis.com/auth/calendar"]
credentials = None
token_file = Path("./calendar_creds/token.pickle")
if token_file.exists():
with open(token_file, "rb") as token:
credentials = load(token)
if not credentials or not credentials.valid:
if credentials and credentials.expired and credentials.refresh_token:
credentials.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file('calendar_creds/credentials.json', scopes)
credentials = flow.run_local_server(port=0)
with open(token_file, "wb") as token:
dump(credentials, token)
calendar_service = build("calendar", "v3", credentials=credentials)
return calendar_service
#staticmethod
def _plan_event(attendees: List[Dict[str, str]], event_time, service: build):
event = {"summary": "test meeting",
"start": {"dateTime": event_time["start"]},
"end": {"dateTime": event_time["end"]},
"attendees": attendees,
"conferenceData": {"createRequest": {"requestId": f"{uuid4().hex}",
"conferenceSolutionKey": {"type": "hangoutsMeet"}}},
"reminders": {"useDefault": True}
}
event = service.events().insert(calendarId="primary", sendNotifications=True, body=event, conferenceDataVersion=1).execute()
return event
if __name__ == "__main__":
plan = EventPlanner({"test_guest": "test.guest#gmail.com"}, {"start": "2020-07-31T16:00:00",
"end": "2020-07-31T16:30:00"})
print(plan.event_states)
I suspect that the problem is with where I have passed conferenceDataVersion but the docs are not exactly clear about where it has to be passed other than that it must be passed. I also tried putting it in the body of the event or in createRequest. It always creates the event but not the conference. Unfortunately, I could not find anything about this online anywhere. Maybe I'm actually that bad at searching, but I have been testing different things and searching for a solution for several days! If anyone knows what I am missing, I will truly appreciate their assistance.
Thanks to #Tanaike, I found what was the problem. The token which is generated the first time the API is authenticated is very specific. The problem I was having turned out to be just with that. As soon as I removed the token and had it get generated again, the problem was solved. That being said, I have no idea why the problem appeared in the first place. I will update the response if I find the reason behind it. But for now, if you are having the same problem, just remove the token and regenerate it.
With this code we can create the conference (Google meet link) for me it works
"conferenceData": {
"createRequest": {
"requestId": "SecureRandom.uuid"
}
}
I am trying to use keycloak with apache superset. I have spent hours on the links below but unable to replace the current login.
Using OpenID/Keycloak with Superset
2.Using KeyCloak(OpenID Connect) with Apache SuperSet
Using OpenID/Keycloak with Superset
I am using apache superset 0.34.5. While above links use 0.28 and below.
i am confused at inital step. let me explain the steps and see what i am missing.
I install superset using pip.
The structure i have is, i have config.py and security.py at the same level (i dont have security folder)
I renamed the security to oid_security.
I created a security.py with the following content.
from flask_appbuilder.security.manager import AUTH_OID
from superset.security import SupersetSecurityManager
from flask_oidc import OpenIDConnect
from flask_appbuilder.security.views import AuthOIDView
from flask_login import login_user
from urllib.parse import quote
from flask_appbuilder.views import ModelView, SimpleFormView, expose
import logging
class AuthOIDCView(AuthOIDView):
#expose('/login/', methods=['GET', 'POST'])
def login(self, flag=True):
sm = self.appbuilder.sm
oidc = sm.oid
#self.appbuilder.sm.oid.require_login
def handle_login():
user = sm.auth_user_oid(oidc.user_getfield('email'))
if user is None:
info = oidc.user_getinfo(['preferred_username', 'given_name', 'family_name', 'email'])
user = sm.add_user(info.get('preferred_username'), info.get('given_name'), info.get('family_name'), info.get('email'), sm.find_role('Gamma'))
login_user(user, remember=False)
return redirect(self.appbuilder.get_url_for_index)
return handle_login()
#expose('/logout/', methods=['GET', 'POST'])
def logout(self):
oidc = self.appbuilder.sm.oid
oidc.logout()
super(AuthOIDCView, self).logout()
redirect_url = request.url_root.strip('/') + self.appbuilder.get_url_for_login
return redirect(oidc.client_secrets.get('issuer') + '/protocol/openid-connect/logout?redirect_uri=' + quote(redirect_url))
class OIDCSecurityManager(SupersetSecurityManager):
authoidview = AuthOIDCView
def __init__(self,appbuilder):
super(OIDCSecurityManager, self).__init__(appbuilder)
if self.auth_type == AUTH_OID:
self.oid = OpenIDConnect(self.appbuilder.get_app)
I then created custom manager with the following
from flask_appbuilder.security.manager import AUTH_OID
from flask_appbuilder.security.sqla.manager import SecurityManager
from flask_oidc import OpenIDConnect
class OIDCSecurityManager(SecurityManager):
def __init__(self, appbuilder):
super(OIDCSecurityManager, self).__init__(appbuilder)
if self.auth_type == AUTH_OID:
self.oid = OpenIDConnect(self.appbuilder.get_app)
self.authoidview = AuthOIDCView
I created client secret.json with my credentials.
I edited config file as below.
from superset.security import OIDCSecurityManager
AUTH_TYPE = AUTH_OID
OIDC_CLIENT_SECRETS = 'client_secret.json'
OIDC_ID_TOKEN_COOKIE_SECURE = False
OIDC_REQUIRE_VERIFIED_EMAIL = False
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = 'Gamma'
CUSTOM_SECURITY_MANAGER = OIDCSecurityManager
One thing to mention here is have manager py in security folder in flask appbuilder which has Abstract Security Manager cls. I am getting an error security py
It says cannot import name SupersetSecurityManager from superset - security
anyone please?
I suggest you start afresh and follow the steps that worked for me:
Create a virtual environment within your superset directory and activate it.
Install the flask-oidc and superset plugins within your virtual environment. pip install flask-oidc
Have a oidc_security.py file with the script you pasted above i.e. security.py in your setup.
Have a client_secret.json file with your keycloak config.
Have a superset_config.py with the script you pasted above.
Add all three of these files to your pythonpath.
Run superset db upgrade & superset init commands.
Finally, execute superset run. After the initialization completes, navigate to http://localhost:8088 on your browser. Expected behaviour: you'll be redirected to keycloak to login/register. After successful sign in, you'll be redirected to superset app.
I hope this helps. Do post back incase you succeed or face an error.
I then created custom manager with the following
where to update this??
from flask_appbuilder.security.manager import AUTH_OID
from flask_appbuilder.security.sqla.manager import SecurityManager
from flask_oidc import OpenIDConnect
class OIDCSecurityManager(SecurityManager):
def __init__(self, appbuilder):
super(OIDCSecurityManager, self).__init__(appbuilder)
if self.auth_type == AUTH_OID:
self.oid = OpenIDConnect(self.appbuilder.get_app)
self.authoidview = AuthOIDCView
I have an app in the Google App Engine Python 3 Standard Environment. I have it set up to group log entries by their request, as described in Writing Application Logs (in the "Viewing related request log entries" section).
That page notes:
The highest severity from the "child" log entries does not automatically apply to the top-level entry. If that behavior is desired, manually set the highest severity in the top-level entry.
The top-level entry in question is the request log that App Engine creates automatically. It's always at log level "Any". I'm not seeing how to change its log level/severity.
Here's (one branch of) the code I have now:
import os
from flask import request
from google.cloud import logging as gcp_logging
from google.cloud.logging.resource import Resource
client = gcp_logging.Client()
logger = client.logger('my_custom_log_type')
trace_id = (f"projects/{os.environ['GOOGLE_CLOUD_PROJECT']}/traces/"
f"{request.headers['X-Cloud-Trace-Context'].split('/')[0]}")
res = Resource(type='gae_app',
labels={'project_id': os.environ['GOOGLE_CLOUD_PROJECT'],
'module_id': os.environ['GAE_SERVICE'],
'version_id': os.environ['GAE_VERSION']})
logger.log_text('Sample log text', resource=res, trace=trace_id, severity='INFO')
It's working perfectly to group the logs with their requests, but the parent (request) log, e.g.,
> * 2019-11-19 15:54:56.613 EST GET 200 1.21 KiB 390ms Chrome 78 /
is displaying with the "Any" log level. I'd like it to be "Info" or "Error" as appropriate. How can I make this happen?
I've not found a way to tinker with the auto-generated log.
I'm still interested in hearing if there's a better way, so I'm not marking this answer as accepted.
But I have found a workaround that pretty much meets my needs, by adding my own request log and ignoring the automatically generated one.
For each child log, I use the same idea as in the question's code, but update a new attribute on flask's g object to keep track of the most severe log level so far within the request.
from flask import g
LOG_LEVELS = ('DEFAULT', 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL')
previous_level = g.get('log_level', 0)
g.log_level = max(previous_level, new_level)
(where new_level is the integer index of the level in LOG_LEVELS)
Then, to generate the new request log, I use a scaled-down version of this solution:
def log_request(response):
logger = client.logger('my_request_logger')
severity = LOG_LEVELS[g.get('log_level', 0)]
request_info = {
'requestMethod': request.method,
'requestUrl': request.url,
'status': response.status_code,
'userAgent': request.headers.get('USER-AGENT'),
'responseSize': response.content_length,
'latency': g.request_duration(),
'remoteIp': request.remote_addr
}
if request.method == 'POST':
payload = request.get_json() or json.loads(request.data.decode())
else:
payload = {}
logger.log_struct(payload,
trace=trace_id,
http_request=request_info,
severity=severity)
The above was in a my_logging module. Then, just a couple of additions to main.py to make this work:
import time
from flask import g
import my_logging
#app.before_request
def setup_timing():
g.request_start_time = time.time()
g.request_duration = lambda: f'{(time.time() - g.request_start_time):.5f}s'
#app.after_request
def log_request(response):
my_logging.log_request(response)
return response
This is working reasonably well - I'd love to hear any ideas to make it better.
Is there a recommended way to test the security setup in a Pyramid application? More specifically I'm using routes and custom routes factories. With fine grained ACLs the security setup is splitted in different spots: the config setup, factories, permission set in the #view_config, and event explicit check of permissions inside views.
The page on unit and functionnal testing (http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/testing.html) does not seem to indicate a way to test if user A can only see and modify data he is allowed to.
This is functional testing. Webtest can preserve session cookies so you can use it to login and visit various pages as a user.
myapp = pyramid.paster.get_app('testing.ini')
app = TestApp(myapp)
resp = app.post('/login', params={'login': 'foo', 'password': 'seekrit'})
# this may be a redirect in which case you may want to follow it
resp = app.get('/protected/resource')
assert resp.status_code == 200
As far as testing just certain parts of your app, you can override the authentication policy with something custom (or just use a custom groupfinder).
def make_test_groupfinder(principals=None):
def _groupfinder(u, r):
return principals
return _groupfinder
You can then use this function to simulate various principals. This doesn't handle the userid though, if your app also relies on authenticated_userid(request) anywhere. For that, you'll have to replace the authentication policy with a dummy one.
class DummyAuthenticationPolicy(object):
def __init__(self, userid, extra_principals=()):
self.userid = userid
self.extra_principals = extra_principals
def authenticated_userid(self, request):
return self.userid
def effective_principals(self, request):
principals = [Everyone]
if self.userid:
principals += [Authenticated]
principals += list(self.extra_principals)
return principals
def remember(self, request, userid, **kw):
return []
def forget(self, request):
return []
I think both the question and answer might be old at this point: with current versions of Pyramid, there's a testing_securitypolicy method (docs here) that allows easy access to setting things like authenticated_userid, effective_principals, results of remember and forget, etc.
Here's an example of usage if need was to set authenticated_userid on a request.
from pyramid.testing import (setUp, tearDown, DummyRequest)
def test_some_view():
config = setUp()
config.testing_securitypolicy(userid='mock_user') # Sets authenticated_userid
dummy_request = DummyRequest()
print(dummy_request.authenticated_userid) # Prints 'mock_user'
# Now ready to test something that uses request.authenticated_userid
from mypyramidapp.views.secure import some_auth_view
result = some_auth_view(dummy_request)
expected = 'Hello mock_user!'
assert result == expected
# Finally, to avoid security changes leaking to other tests, use tearDown
tearDown() # Undo the effects of pyramid.testing.setUp()