I have a API Gateway service based on FastAPI and some specific services (like plugins) to connect with it. One of them - Auth service dealing with user accounts and access-tokens.
For example Auth service wants to tell AG about new functionality he provides and to register new endpoint in AG at runtime.
I see the following steps:
Auth creates new endpoint in AG, /new_endpoint for example;
All the traffic going to http://AG/new_endpoint will be redirected to http://Auth/...
I looked at the method FastAPI.add_api_route to add new endpoint. It works at runtime - I checked using curl.
There is no effect after refreshing http://AG/docs page because OpenAPI schema is cached.
I would like to re-generate OpenAPI schema and see /new_endpoint on the OpenAPI page.
I think I found the solution how to re-generate OpenAPI schema.
Drop cache app.openapi_schema = None
Re-generate schema app.setup()
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel
app = FastAPI()
class NewEndpointResponse(BaseModel):
status: str
method: str
url_path: str
async def catch_all(request: Request) -> JSONResponse:
"""
Your new endpoint handler
"""
# some logic to interact with Auth-service
# like: requests.get("http://Auth/...")
res = NewEndpointResponse(status="OK", method=request.method, url_path=request.url.path)
return JSONResponse(res.dict(), status_code=200)
class EndpointRegisterDTO(BaseModel):
endpoint: str = "/new_endpoint"
method: str = "GET"
name: str = "Extra Functionality"
#app.post("/register/endpoint")
async def add_endpoint(request: EndpointRegisterDTO):
"""
Adds new endpoint at runtime
"""
app.add_api_route(
request.endpoint,
catch_all,
methods=[request.method],
name=request.name,
response_model=NewEndpointResponse)
app.openapi_schema = None
app.setup()
return {"status": "OK"}
Open http://AG/docs. Only one endpoint is available.
Press "Try it out" and do POST /register/endpoint with suggested parameters.
Refresh http://AG/docs - now you can see /new_endpoint.
Call GET /new_endpoint and check that response is correct.
The solution is ugly a bit, but it works!
I think it's bloody hard to debug it!
Related
I used this documentation to secure some endpoints in my fastapi app.
It seemed to be working fine, and I was also able to run some tests (using pytest):
...
client = TestClient(app=app)
response = client.post(
"/my_end_point",
json={'data':'model'},
headers={"Authorization": f"Bearer {token}"},
)
...
My problem is, I want to connect my app to a Gitlab Webhook, and when doing that, Gitlab sends this header:
{'x-gitlab-token':'some-token-that-can-be-defined-when-setting-up-the-webhook'}
That means, even if I configure a valid token in the Gitlab Webhook configuration, it is not accepted by my FastAPI app, which returns 401 Not authorized error.
I guess my question is how to instruct FastApi to take the token from 'x-gitlab-token' key and not from 'Authorization' key
There's an example of how you can use Header in a dependency function in FastAPI's user guide.
A self-contained example:
import uvicorn
from fastapi import (
Depends,
FastAPI,
Header,
HTTPException,
)
def authenticate_gitlab(x_gitlab_token: str = Header(...)):
if x_gitlab_token != 'magic':
raise HTTPException(status_code=403)
return x_gitlab_token
app = FastAPI()
#app.get("/")
async def req(authenticated_with: str = Depends(authenticate_gitlab)):
return {'authenticated_with': authenticated_with}
if __name__ == "__main__":
uvicorn.run("foo:app", host="127.0.0.1", port=5000, log_level="info")
This accepts any request with X-Gitlab-Token set to magic, while refusing other keys:
λ curl http://localhost:5000 -H "X-GitLab-Token: foo"
{"detail":"Forbidden"}
λ curl http://localhost:5000 -H "X-GitLab-Token: magic"
{"authenticated_with":"magic"}
You can use the dependencies argument when creating an APIRouter to have the dependency run before every route in a given router (which you can composition together as you need from multiple routers):
authenticated_router = APIRouter(dependencies=[Depends(authenticate_gitlab)])
How do I set this up correctly? The description seems straight-forward
import os
import json
from ibm_watson import SpeechToTextV1
from ibm_cloud_sdk_core.authenticators import IAMAuthenticator
api_key='XYZ'
service_endpoint='https://api.eu-gb.speech-to-text.watson.cloud.ibm.com'
authenticator = IAMAuthenticator(api_key)
speech2text = SpeechToTextV1(authenticator=authenticator)
speech_to_text.set_service_url(service_endpoint)
speech_models = speech_to_text.list_models().get_result()
print(json.dumps(speech_models, indent=2))
Which API key do I have to pass to the 'IAMAuthenticator'?
I don't quite understand if I first have to create an Instance/Resource (https://cloud.ibm.com/catalog/services/speech-to-text) and use that API key, or only create and use an IBM Cloud API key (https://cloud.ibm.com/iam/apikeys). I have tried different combinations of the above, but cannot connect successfully, raising ApiException: Error: Provided API key could not be found, Code: 400
Actually, you use IAMAuthenticator with the API key. Don't forget to import it.
See the example in the API reference and more information about the SDK in the README. Can you make a call with curl with the same API key and URL
(The BasicAuthenticator is specifically for username and password auth.)
To use the APIKEY directly you must use the BasicAuthenticator instead of the IAMAuthenticator from ibm_cloud_sdk_core.authenticators. See the code snippet below:
from ibm_watson import SpeechToTextV1, ApiException
from ibm_cloud_sdk_core.authenticators import BasicAuthenticator
...
# Basic Authentication with Watson STT API
stt_authenticator = BasicAuthenticator(
'apikey',
'<insert_your_watson_stt_apikey_here>'
)
# Construct a Watson STT client with the authentication object
stt = SpeechToTextV1(
authenticator=stt_authenticator
)
# Set the URL endpoint for your Watson STT client
stt.set_service_url(
'<insert_your_watson_stt_endpoint_url_here>'
)
# Read audio file and call Watson STT API:
with open(
os.path.join(
os.path.dirname(__file__), './.',
'audio_sample.flac'
), 'rb'
) as audio_file:
# Transcribe the audio.flac with Watson STT
# Recognize method API reference:
# https://cloud.ibm.com/apidocs/speech-to-text?code=python#recognize
stt_result = stt.recognize(
audio=audio_file,
content_type='audio/flac',
model='pt-BR_BroadbandModel'
).get_result()
# Print STT API call results
print(json.dumps(stt_result, indent=2))
...
I hope this answer helps you.
How do we post a GraphQL request through AWS AppSync using boto?
Ultimately I'm trying to mimic a mobile app accessing our stackless/cloudformation stack on AWS, but with python. Not javascript or amplify.
The primary pain point is authentication; I've tried a dozen different ways already. This the current one, which generates a "401" response with "UnauthorizedException" and "Permission denied", which is actually pretty good considering some of the other messages I've had. I'm now using the 'aws_requests_auth' library to do the signing part. I assume it authenticates me using the stored /.aws/credentials from my local environment, or does it?
I'm a little confused as to where and how cognito identities and pools will come into it. eg: say I wanted to mimic the sign-up sequence?
Anyways the code looks pretty straightforward; I just don't grok the authentication.
from aws_requests_auth.boto_utils import BotoAWSRequestsAuth
APPSYNC_API_KEY = 'inAppsyncSettings'
APPSYNC_API_ENDPOINT_URL = 'https://aaaaaaaaaaaavzbke.appsync-api.ap-southeast-2.amazonaws.com/graphql'
headers = {
'Content-Type': "application/graphql",
'x-api-key': APPSYNC_API_KEY,
'cache-control': "no-cache",
}
query = """{
GetUserSettingsByEmail(email: "john#washere"){
items {name, identity_id, invite_code}
}
}"""
def test_stuff():
# Use the library to generate auth headers.
auth = BotoAWSRequestsAuth(
aws_host='aaaaaaaaaaaavzbke.appsync-api.ap-southeast-2.amazonaws.com',
aws_region='ap-southeast-2',
aws_service='appsync')
# Create an http graphql request.
response = requests.post(
APPSYNC_API_ENDPOINT_URL,
json={'query': query},
auth=auth,
headers=headers)
print(response)
# this didn't work:
# response = requests.post(APPSYNC_API_ENDPOINT_URL, data=json.dumps({'query': query}), auth=auth, headers=headers)
Yields
{
"errors" : [ {
"errorType" : "UnauthorizedException",
"message" : "Permission denied"
} ]
}
It's quite simple--once you know. There are some things I didn't appreciate:
I've assumed IAM authentication (OpenID appended way below)
There are a number of ways for appsync to handle authentication. We're using IAM so that's what I need to deal with, yours might be different.
Boto doesn't come into it.
We want to issue a request like any regular punter, they don't use boto, and neither do we. Trawling the AWS boto docs was a waste of time.
Use the AWS4Auth library
We are going to send a regular http request to aws, so whilst we can use python requests they need to be authenticated--by attaching headers.
And, of course, AWS auth headers are special and different from all others.
You can try to work out how to do it
yourself, or you can go looking for someone else who has already done it: Aws_requests_auth, the one I started with, probably works just fine, but I have ended up with AWS4Auth. There are many others of dubious value; none endorsed or provided by Amazon (that I could find).
Specify appsync as the "service"
What service are we calling? I didn't find any examples of anyone doing this anywhere. All the examples are trivial S3 or EC2 or even EB which left uncertainty. Should we be talking to api-gateway service? Whatsmore, you feed this detail into the AWS4Auth routine, or authentication data. Obviously, in hindsight, the request is hitting Appsync, so it will be authenticated by Appsync, so specify "appsync" as the service when putting together the auth headers.
It comes together as:
import requests
from requests_aws4auth import AWS4Auth
# Use AWS4Auth to sign a requests session
session = requests.Session()
session.auth = AWS4Auth(
# An AWS 'ACCESS KEY' associated with an IAM user.
'AKxxxxxxxxxxxxxxx2A',
# The 'secret' that goes with the above access key.
'kwWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxgEm',
# The region you want to access.
'ap-southeast-2',
# The service you want to access.
'appsync'
)
# As found in AWS Appsync under Settings for your endpoint.
APPSYNC_API_ENDPOINT_URL = 'https://nqxxxxxxxxxxxxxxxxxxxke'
'.appsync-api.ap-southeast-2.amazonaws.com/graphql'
# Use JSON format string for the query. It does not need reformatting.
query = """
query foo {
GetUserSettings (
identity_id: "ap-southeast-2:8xxxxxxb-7xx4-4xx4-8xx0-exxxxxxx2"
){
user_name, email, whatever
}}"""
# Now we can simply post the request...
response = session.request(
url=APPSYNC_API_ENDPOINT_URL,
method='POST',
json={'query': query}
)
print(response.text)
Which yields
# Your answer comes as a JSON formatted string in the text attribute, under data.
{"data":{"GetUserSettings":{"user_name":"0xxxxxxx3-9102-42f0-9874-1xxxxx7dxxx5"}}}
Getting credentials
To get rid of the hardcoded key/secret you can consume the local AWS ~/.aws/config and ~/.aws/credentials, and it is done this way...
# Use AWS4Auth to sign a requests session
session = requests.Session()
credentials = boto3.session.Session().get_credentials()
session.auth = AWS4Auth(
credentials.access_key,
credentials.secret_key,
boto3.session.Session().region_name,
'appsync',
session_token=credentials.token
)
...<as above>
This does seem to respect the environment variable AWS_PROFILE for assuming different roles.
Note that STS.get_session_token is not the way to do it, as it may try to assume a role from a role, depending where it keyword matched the AWS_PROFILE value. Labels in the credentials file will work because the keys are right there, but names found in the config file do not work, as that assumes a role already.
OpenID
In this scenario, all the complexity is transferred to the conversation with the openid connect provider. The hard stuff is all the auth hoops you jump through to get an access token, and thence using the refresh token to keep it alive. That is where all the real work lies.
Once you finally have an access token, assuming you have configured the "OpenID Connect" Authorization Mode in appsync, then you can, very simply, drop the access token into the header:
response = requests.post(
url="https://nc3xxxxxxxxxx123456zwjka.appsync-api.ap-southeast-2.amazonaws.com/graphql",
headers={"Authorization": ACCESS_TOKEN},
json={'query': "query foo{GetStuff{cat, dog, tree}}"}
)
You can set up an API key on the AppSync end and use the code below. This works for my case.
import requests
# establish a session with requests session
session = requests.Session()
# As found in AWS Appsync under Settings for your endpoint.
APPSYNC_API_ENDPOINT_URL = 'https://vxxxxxxxxxxxxxxxxxxy.appsync-api.ap-southeast-2.amazonaws.com/graphql'
# setup the query string (optional)
query = """query listItemsQuery {listItemsQuery {items {correlation_id, id, etc}}}"""
# Now we can simply post the request...
response = session.request(
url=APPSYNC_API_ENDPOINT_URL,
method='POST',
headers={'x-api-key': '<APIKEYFOUNDINAPPSYNCSETTINGS>'},
json={'query': query}
)
print(response.json()['data'])
Building off Joseph Warda's answer you can use the class below to send AppSync commands.
# fileName: AppSyncLibrary
import requests
class AppSync():
def __init__(self,data):
endpoint = data["endpoint"]
self.APPSYNC_API_ENDPOINT_URL = endpoint
self.api_key = data["api_key"]
self.session = requests.Session()
def graphql_operation(self,query,input_params):
response = self.session.request(
url=self.APPSYNC_API_ENDPOINT_URL,
method='POST',
headers={'x-api-key': self.api_key},
json={'query': query,'variables':{"input":input_params}}
)
return response.json()
For example in another file within the same directory:
from AppSyncLibrary import AppSync
APPSYNC_API_ENDPOINT_URL = {YOUR_APPSYNC_API_ENDPOINT}
APPSYNC_API_KEY = {YOUR_API_KEY}
init_params = {"endpoint":APPSYNC_API_ENDPOINT_URL,"api_key":APPSYNC_API_KEY}
app_sync = AppSync(init_params)
mutation = """mutation CreatePost($input: CreatePostInput!) {
createPost(input: $input) {
id
content
}
}
"""
input_params = {
"content":"My first post"
}
response = app_sync.graphql_operation(mutation,input_params)
print(response)
Note: This requires you to activate API access for your AppSync API. Check this AWS post for more details.
graphql-python/gql supports AWS AppSync since version 3.0.0rc0.
It supports queries, mutation and even subscriptions on the realtime endpoint.
The documentation is available here
Here is an example of a mutation using the API Key authentication:
import asyncio
import os
import sys
from urllib.parse import urlparse
from gql import Client, gql
from gql.transport.aiohttp import AIOHTTPTransport
from gql.transport.appsync_auth import AppSyncApiKeyAuthentication
# Uncomment the following lines to enable debug output
# import logging
# logging.basicConfig(level=logging.DEBUG)
async def main():
# Should look like:
# https://XXXXXXXXXXXXXXXXXXXXXXXXXX.appsync-api.REGION.amazonaws.com/graphql
url = os.environ.get("AWS_GRAPHQL_API_ENDPOINT")
api_key = os.environ.get("AWS_GRAPHQL_API_KEY")
if url is None or api_key is None:
print("Missing environment variables")
sys.exit()
# Extract host from url
host = str(urlparse(url).netloc)
auth = AppSyncApiKeyAuthentication(host=host, api_key=api_key)
transport = AIOHTTPTransport(url=url, auth=auth)
async with Client(
transport=transport, fetch_schema_from_transport=False,
) as session:
query = gql(
"""
mutation createMessage($message: String!) {
createMessage(input: {message: $message}) {
id
message
createdAt
}
}"""
)
variable_values = {"message": "Hello world!"}
result = await session.execute(query, variable_values=variable_values)
print(result)
asyncio.run(main())
I am unable to add a comment due to low rep, but I just want to add that I tried the accepted answer and it didn't work. I was getting an error saying my session_token is invalid. Probably because I was using AWS Lambda.
I got it to work pretty much exactly, but by adding to the session token parameter of the aws4auth object. Here's the full piece:
import requests
import os
from requests_aws4auth import AWS4Auth
def AppsyncHandler(event, context):
# These are env vars that are always present in an AWS Lambda function
# If not using AWS Lambda, you'll need to add them manually to your env.
access_id = os.environ.get("AWS_ACCESS_KEY_ID")
secret_key = os.environ.get("AWS_SECRET_ACCESS_KEY")
session_token = os.environ.get("AWS_SESSION_TOKEN")
region = os.environ.get("AWS_REGION")
# Your AppSync Endpoint
api_endpoint = os.environ.get("AppsyncConnectionString")
resource = "appsync"
session = requests.Session()
session.auth = AWS4Auth(access_id,
secret_key,
region,
resource,
session_token=session_token)
The rest is the same.
Hope this Helps Everyone
import requests
import json
import os
from dotenv import load_dotenv
load_dotenv(".env")
class AppSync(object):
def __init__(self,data):
endpoint = data["endpoint"]
self.APPSYNC_API_ENDPOINT_URL = endpoint
self.api_key = data["api_key"]
self.session = requests.Session()
def graphql_operation(self,query,input_params):
response = self.session.request(
url=self.APPSYNC_API_ENDPOINT_URL,
method='POST',
headers={'x-api-key': self.api_key},
json={'query': query,'variables':{"input":input_params}}
)
return response.json()
def main():
APPSYNC_API_ENDPOINT_URL = os.getenv("APPSYNC_API_ENDPOINT_URL")
APPSYNC_API_KEY = os.getenv("APPSYNC_API_KEY")
init_params = {"endpoint":APPSYNC_API_ENDPOINT_URL,"api_key":APPSYNC_API_KEY}
app_sync = AppSync(init_params)
mutation = """
query MyQuery {
getAccountId(id: "5ca4bbc7a2dd94ee58162393") {
_id
account_id
limit
products
}
}
"""
input_params = {}
response = app_sync.graphql_operation(mutation,input_params)
print(json.dumps(response , indent=3))
main()
I need to connect to azure service bus using SAS token(generate and connect).
I don't see anything for the python implementation.
This link provides the implementation for Eventhubs -
https://learn.microsoft.com/en-us/rest/api/eventhub/generate-sas-token#python
Not sure where I can find the python implementation for servicebus.
I have found a way where you can do it for the ServiceBusService Class.
After running "pip install azure.servicebus", I imported it as:
from azure.servicebus.control_client import ServiceBusService
The ServiceBusService constructor takes an argument called "authentication", which isn't specified by default.
If you got into the ServiceBusService init file, you can see how authentication is handled in more detail.
if authentication:
self.authentication = authentication
else:
if not account_key:
account_key = os.environ.get(AZURE_SERVICEBUS_ACCESS_KEY)
if not issuer:
issuer = os.environ.get(AZURE_SERVICEBUS_ISSUER)
if shared_access_key_name and shared_access_key_value:
self.authentication = ServiceBusSASAuthentication(
shared_access_key_name,
shared_access_key_value)
elif account_key and issuer:
self.authentication = ServiceBusWrapTokenAuthentication(
account_key,
issuer)
If you don't pass a custom authentication object, it will then try to use the ServiceBusSASAuthentication class, which is the default if you populate the shared_access_key_name and shared_access_key_value.
So if you jump into the ServiceBusSASAuthentication class, you'll notice something useful.
class ServiceBusSASAuthentication:
def __init__(self, key_name, key_value):
self.key_name = key_name
self.key_value = key_value
self.account_key = None
self.issuer = None
def sign_request(self, request, httpclient):
request.headers.append(
('Authorization', self._get_authorization(request, httpclient)))
def _get_authorization(self, request, httpclient):
uri = httpclient.get_uri(request)
uri = url_quote(uri, '').lower()
expiry = str(self._get_expiry())
to_sign = uri + '\n' + expiry
signature = url_quote(_sign_string(self.key_value, to_sign, False), '')
auth_format = 'SharedAccessSignature sig={0}&se={1}&skn={2}&sr={3}' # <----awww, yeah
auth = auth_format.format(signature, expiry, self.key_name, uri)
return auth # <--after inserting values into string, the SAS Token is just returned.
def _get_expiry(self): # pylint: disable=no-self-use
'''Returns the UTC datetime, in seconds since Epoch, when this signed
request expires (5 minutes from now).'''
return int(round(time.time() + 300))
The sign_request function is the only function that will be directly referenced by the ServiceBusService class when it does authentication, but you'll notice that all its doing is adding an authentication header to a request that...IS IN THE FORMAT OF A SAS TOKEN.
So at this point I had all the information I needed to make my own authentication class. I made one that looked exactly like this.
class ServiceBusSASTokenAuthentication:
def __init__(self, sas_token):
self.sas_token = sas_token
# this method is the one used by ServiceBusService for authentication, need to leave signature as is
# even though we don't use httpClient like the original.
def sign_request(self, request, httpclient):
request.headers.append(
('Authorization', self._get_authorization())
)
def _get_authorization(self):
return self.sas_token
I probably could get rid of the _get_auth function all together, but I haven't polished everything up yet.
So now if you call this class like so in the ServiceBusService constructor with a valid SAS Token, it should work.
subscription_client = ServiceBusService(
authentication=ServiceBusSASTokenAuthentication(sas_token=sas_token),
service_namespace=service_namespace
)
Once you create the Service Bus using Azure Portal, ServiceBusService object enables you to work with queues.
Follow this document for more information on creating the queue, sending message to queue, receiving message from a queue using python to programmatically access the Service Bus.
I am working on a project with the following architecture:
UI: React on client and server-side rendering via a Node server, Apollo Client for GraphQL,
API: Django handles GraphQL queries through Graphene.
I use Auth0 (JWT based) for my frontend authentication. I would like to use the token I get to authenticate my user in the context of the GraphQL queries API side.
[Edit2]
To pass the token to my API, I use:
const idToken = cookie.load('idToken') || null;
networkInterface.use([{
applyMiddleware(req, next) {
if (!req.options.headers) {
req.options.headers = {}; // Create the header object if needed.
}
req.options.headers.authorization = `Bearer ${idToken}`;
next();
}
}]);
Then I need to retrieve it in Django: I use django-jwt-auth and the code proposed by #Craig Ambrose.
My authorization header is received and decoded (I can get the payload) but there is a problem when verifying the signature: I get "Error decoding signature."
This is strange since the signature is verified when I test it on jwt.io.
How can I authenticate on Django side ?
I've just done this using django-jwt-auth (not using Auth0)
That package provides a JSONWebTokenAuthMixin that you can combine with the GraphQLView from graphene_django, for example.
from jwt_auth.mixins import JSONWebTokenAuthMixin
class AuthGraphQLView(JSONWebTokenAuthMixin, GraphQLView):
pass
urlpatterns = [
url(r'^graphql', csrf_exempt(AuthGraphQLView.as_view(schema=schema))),
url(r'^graphiql', include('django_graphiql.urls')),
]
This works, but I found that graphiql stopped working, because it wasn't sending to token. I wanted to keep using cookie based auth for that, for dev purposes, so changed it to the following.
from jwt_auth.mixins import JSONWebTokenAuthMixin
class OptionalJWTMixin(JSONWebTokenAuthMixin):
def dispatch(self, request, *args, **kwargs):
auth = get_authorization_header(request)
if auth:
return super(OptionalJWTMixin, self).dispatch(request, *args, **kwargs)
else:
return super(JSONWebTokenAuthMixin, self).dispatch(request, *args, **kwargs)
class AuthGraphQLView(OptionalJWTMixin, GraphQLView):
pass
urlpatterns = [
url(r'^graphql', csrf_exempt(AuthGraphQLView.as_view(schema=schema))),
url(r'^graphiql', include('django_graphiql.urls')),
]
My setting is working now:
I have used code from #Craig Ambrose with django-jwt-auth. I had to fork the package on Github to handle the Audience 'aud' payload present in Auth0 Token.
def jwt_get_user_id_from_payload_handler(payload):
sub = payload.get('sub')
Auth0User = import_string('project.models.Auth0User')
auth0_user = Auth0User.objects.filter(auth0_id=sub)[0]
user_id = auth0_user.user.id
return user_id
JWT_PAYLOAD_GET_USER_ID_HANDLER = jwt_get_user_id_from_payload_handler
auth0_key = '<MyAuth0SECRET>'
JWT_SECRET_KEY = base64.b64decode(auth0_key.replace("_","/").replace("-","+"))
JWT_VERIFY = True
JWT_AUTH_HEADER_PREFIX = 'Bearer'
JWT_AUDIENCE = '<MyAuth0CLIENT_ID>'
With Aut0User a model with OnoToOne relation with classic Django user and a field with auth0_id.