Currently trying to connect sftp server using user credential from AWS secret manager, and password contains double quote special character, which causing the issue. Below is sample code,
import sys
import boto3
import base64
from botocore.exceptions import ClientError
import hashlib
import pysftp
secret_name = "SFTP_TEST"
region_name = "eu-central-1"
_SFTP_DETAILS = {}
pass1= "E?%/?s\"N1sS#OnXN"
cnopts = pysftp.CnOpts()
cnopts.hostkeys = None
cnopts.log = True
basepath ='/test/'
def get_connect(secret_name,region_name):
session = boto3.session.Session()
client = session.client(service_name='secretsmanager', region_name=region_name.strip())
if secret_name.strip() not in _SFTP_DETAILS:
try:
get_secret_value_response = client.get_secret_value(SecretId=secret_name.strip())
except Exception as e:
raise e
else:
if 'SecretString' in get_secret_value_response:
secret = get_secret_value_response['SecretString']
print("Secret value Original ==>",secret)
secretValue = json.loads(secret)
awsValue = secretValue.get(secret_name.strip())
sftpStrValue = awsValue.replace("“","\"").replace("”","\"")
print("Secrete Value After JSON loader ==>",sftpStrValue)
sftpValues = json.loads(sftpStrValue)
_HOST_NAME = sftpValues.get("url")
_USER_NAME = sftpValues.get("username")
_PASSWORD = sftpValues.get("password")
print("Secrete Password:::" + _PASSWORD)
_PORT = sftpValues.get("port")
with pysftp.Connection(_HOST_NAME, username=_USER_NAME, password=_PASSWORD, port=int('22'), cnopts=cnopts) as sftp:
print("I am in SFTP SERVER")
for attr in sftp.listdir_attr(basepath):
print("listdir is",attr)
_SFTP_DETAILS[secret_name] = [_HOST_NAME.strip(),_USER_NAME.strip(),_PASSWORD.strip(),_PORT.strip()]
return _SFTP_DETAILS[secret_name.strip()]
get_connect()
Here we are fetching password (_PASSWORD) from AWS secret manager and passing to pysftp.Connection function, but unable to connect.
Here if I am hard coded password i.e. pass1 in above code then its working fine and able connect. Unable to get the issue is from python or AWS Secrets Manager.
Could you please let me know why password from AWS secret manager is not working while hard coded is working correctly. Here requirement to keep password in AWS Secrets Manager.
Any help on this appreciated.
I have this as my secret in AWS Secret Manager Console:
Secret Key | Secret Value
TEST_KEY | afgvbq3tg"afsvgqag"af.qw/asffq3gvd13
If I get the secret_value by:
secret = client.get_secret_value(SecretId="test_secret_ron")
and print the secret["SecretString"], the result will look like:
'{"TEST_KEY":"afgvbq3tg\\"afsvgqag\\"af.qw/asffq3gvd13"}'
once you turn this string to dictionary by:
json.loads(secret["SecretString"])
the expected dictionary will reflect the correct format of the string:
{'TEST_KEY': 'afgvbq3tg"afsvgqag"af.qw/asffq3gvd13'}
enter image description here
Related
I'm trying to use the AssumeRole in such a way that i'm traversing multiple accounts and retrieving assets for those accounts. I've made it to this point:
import boto3
stsclient = boto3.client('sts')
assumedRoleObject = sts_client.assume_role(
RoleArn="arn:aws:iam::account-of-role-to-assume:role/name-of-role",
RoleSessionName="AssumeRoleSession1")
Great, i have the assumedRoleObject. But now i want to use that to list things like ELBs or something that isn't a built-in low level resource.
How does one go about doing that? If i may ask - please code out a full example, so that everyone can benefit.
Here's a code snippet from the official AWS documentation where an s3 resource is created for listing all s3 buckets. boto3 resources or clients for other services can be built in a similar fashion.
# create an STS client object that represents a live connection to the
# STS service
sts_client = boto3.client('sts')
# Call the assume_role method of the STSConnection object and pass the role
# ARN and a role session name.
assumed_role_object=sts_client.assume_role(
RoleArn="arn:aws:iam::account-of-role-to-assume:role/name-of-role",
RoleSessionName="AssumeRoleSession1"
)
# From the response that contains the assumed role, get the temporary
# credentials that can be used to make subsequent API calls
credentials=assumed_role_object['Credentials']
# Use the temporary credentials that AssumeRole returns to make a
# connection to Amazon S3
s3_resource=boto3.resource(
's3',
aws_access_key_id=credentials['AccessKeyId'],
aws_secret_access_key=credentials['SecretAccessKey'],
aws_session_token=credentials['SessionToken'],
)
# Use the Amazon S3 resource object that is now configured with the
# credentials to access your S3 buckets.
for bucket in s3_resource.buckets.all():
print(bucket.name)
To get a session with an assumed role:
import botocore
import boto3
import datetime
from dateutil.tz import tzlocal
assume_role_cache: dict = {}
def assumed_role_session(role_arn: str, base_session: botocore.session.Session = None):
base_session = base_session or boto3.session.Session()._session
fetcher = botocore.credentials.AssumeRoleCredentialFetcher(
client_creator = base_session.create_client,
source_credentials = base_session.get_credentials(),
role_arn = role_arn,
extra_args = {
# 'RoleSessionName': None # set this if you want something non-default
}
)
creds = botocore.credentials.DeferredRefreshableCredentials(
method = 'assume-role',
refresh_using = fetcher.fetch_credentials,
time_fetcher = lambda: datetime.datetime.now(tzlocal())
)
botocore_session = botocore.session.Session()
botocore_session._credentials = creds
return boto3.Session(botocore_session = botocore_session)
# usage:
session = assumed_role_session('arn:aws:iam::ACCOUNTID:role/ROLE_NAME')
ec2 = session.client('ec2') # ... etc.
The resulting session's credentials will be automatically refreshed when required which is quite nice.
Note: my previous answer was outright wrong but I can't delete it, so I've replaced it with a better and working answer.
You can assume role using STS token, like:
class Boto3STSService(object):
def __init__(self, arn):
sess = Session(aws_access_key_id=ARN_ACCESS_KEY,
aws_secret_access_key=ARN_SECRET_KEY)
sts_connection = sess.client('sts')
assume_role_object = sts_connection.assume_role(
RoleArn=arn, RoleSessionName=ARN_ROLE_SESSION_NAME,
DurationSeconds=3600)
self.credentials = assume_role_object['Credentials']
This will give you temporary access key and secret keys, with session token. With these temporary credentials, you can access any service. For Eg, if you want to access ELB, you can use the below code:
self.tmp_credentials = Boto3STSService(arn).credentials
def get_boto3_session(self):
tmp_access_key = self.tmp_credentials['AccessKeyId']
tmp_secret_key = self.tmp_credentials['SecretAccessKey']
security_token = self.tmp_credentials['SessionToken']
boto3_session = Session(
aws_access_key_id=tmp_access_key,
aws_secret_access_key=tmp_secret_key, aws_session_token=security_token
)
return boto3_session
def get_elb_boto3_connection(self, region):
sess = self.get_boto3_session()
elb_conn = sess.client(service_name='elb', region_name=region)
return elb_conn
with reference to the solution by #jarrad which is not working as of Feb 2021, and as a solution that does not use STS explicitly please see the following
import boto3
import botocore.session
from botocore.credentials import AssumeRoleCredentialFetcher, DeferredRefreshableCredentials
def get_boto3_session(assume_role_arn=None):
session = boto3.Session(aws_access_key_id="abc", aws_secret_access_key="def")
if not assume_role_arn:
return session
fetcher = AssumeRoleCredentialFetcher(
client_creator=_get_client_creator(session),
source_credentials=session.get_credentials(),
role_arn=assume_role_arn,
)
botocore_session = botocore.session.Session()
botocore_session._credentials = DeferredRefreshableCredentials(
method='assume-role',
refresh_using=fetcher.fetch_credentials
)
return boto3.Session(botocore_session=botocore_session)
def _get_client_creator(session):
def client_creator(service_name, **kwargs):
return session.client(service_name, **kwargs)
return client_creator
the function can be called as follows
ec2_client = get_boto3_session(role_arn='my_role_arn').client('ec2', region_name='us-east-1')
If you want a functional implementation, this is what I settled on:
def filter_none_values(kwargs: dict) -> dict:
"""Returns a new dictionary excluding items where value was None"""
return {k: v for k, v in kwargs.items() if v is not None}
def assume_session(
role_session_name: str,
role_arn: str,
duration_seconds: Optional[int] = None,
region_name: Optional[str] = None,
) -> boto3.Session:
"""
Returns a session with the given name and role.
If not specified, duration will be set by AWS, probably at 1 hour.
If not specified, region will be left unset.
Region can be overridden by each client or resource spawned from this session.
"""
assume_role_kwargs = filter_none_values(
{
"RoleSessionName": role_session_name,
"RoleArn": role_arn,
"DurationSeconds": duration_seconds,
}
)
credentials = boto3.client("sts").assume_role(**assume_role_kwargs)["Credentials"]
create_session_kwargs = filter_none_values(
{
"aws_access_key_id": credentials["AccessKeyId"],
"aws_secret_access_key": credentials["SecretAccessKey"],
"aws_session_token": credentials["SessionToken"],
"region_name": region_name,
}
)
return boto3.Session(**create_session_kwargs)
def main() -> None:
session = assume_session(
"MyCustomSessionName",
"arn:aws:iam::XXXXXXXXXXXX:role/TheRoleIWantToAssume",
region_name="us-east-1",
)
client = session.client(service_name="ec2")
print(client.describe_key_pairs())
import json
import boto3
roleARN = 'arn:aws:iam::account-of-role-to-assume:role/name-of-role'
client = boto3.client('sts')
response = client.assume_role(RoleArn=roleARN,
RoleSessionName='RoleSessionName',
DurationSeconds=900)
dynamodb_client = boto3.client('dynamodb', region_name='us-east-1',
aws_access_key_id=response['Credentials']['AccessKeyId'],
aws_secret_access_key=response['Credentials']['SecretAccessKey'],
aws_session_token = response['Credentials']['SessionToken'])
response = dynamodb_client.get_item(
Key={
'key1': {
'S': '1',
},
'key2': {
'S': '2',
},
},
TableName='TestTable')
print(response)
#!/usr/bin/env python3
import boto3
sts_client = boto3.client('sts')
assumed_role = sts_client.assume_role(RoleArn = "arn:aws:iam::123456789012:role/example_role",
RoleSessionName = "AssumeRoleSession1",
DurationSeconds = 1800)
session = boto3.Session(
aws_access_key_id = assumed_role['Credentials']['AccessKeyId'],
aws_secret_access_key = assumed_role['Credentials']['SecretAccessKey'],
aws_session_token = assumed_role['Credentials']['SessionToken'],
region_name = 'us-west-1'
)
# now we make use of the role to retrieve a parameter from SSM
client = session.client('ssm')
response = client.get_parameter(
Name = '/this/is/a/path/parameter',
WithDecryption = True
)
print(response)
Assuming that 1) the ~/.aws/config or ~/.aws/credentials file is populated with each of the roles that you wish to assume and that 2) the default role has AssumeRole defined in its IAM policy for each of those roles, then you can simply (in pseudo-code) do the following and not have to fuss with STS:
import boto3
# get all of the roles from the AWS config/credentials file using a config file parser
profiles = get_profiles()
for profile in profiles:
# this is only used to fetch the available regions
initial_session = boto3.Session(profile_name=profile)
# get the regions
regions = boto3.Session.get_available_regions('ec2')
# cycle through the regions, setting up session, resource and client objects
for region in regions:
boto3_session = boto3.Session(profile_name=profile, region_name=region)
boto3_resource = boto3_session.resource(service_name='s3', region_name=region)
boto3_client = boto3_session.client(service_name='s3', region_name=region)
[ do something interesting with your session/resource/client here ]
Credential Setup (boto3 - Shared Credentials File)
Assume Role Setup (AWS)
After a few days of searching, this is the simplest solution I have found. explained here but does not have a usage example.
import boto3
for profile in boto3.Session().available_profiles:
boto3.DEFAULT_SESSION = boto3.session.Session(profile_name=profile)
s3 = boto3.resource('s3')
for bucket in s3.buckets.all():
print(bucket)
This will switch the default role you will be using. To not make the profile the default, just do not assign it to boto3.DEFAULT_SESSION. but instead, do the following.
testing_profile = boto3.session.Session(profile_name='mainTesting')
s3 = testing_profile.resource('s3')
for bucket in s3.buckets.all():
print(bucket)
Important to note that the .aws credentials need to be set in a specific way.
[default]
aws_access_key_id = default_access_id
aws_secret_access_key = default_access_key
[main]
aws_access_key_id = main_profile_access_id
aws_secret_access_key = main_profile_access_key
[mainTesting]
source_profile = main
role_arn = Testing role arn
mfa_serial = mfa_arn_for_main_role
[mainProduction]
source_profile = main
role_arn = Production role arn
mfa_serial = mfa_arn_for_main_role
I don't know why but the mfa_serial key has to be on the roles for this to work instead of the source account which would make more sense.
Here's the code snippet I used
sts_client = boto3.client('sts')
assumed_role_object = sts_client.assume_role(
RoleArn=<arn of the role to assume>,
RoleSessionName="<role session name>"
)
print(assumed_role_object)
credentials = assumed_role_object['Credentials']
session = Session(
aws_access_key_id=credentials['AccessKeyId'],
aws_secret_access_key=credentials['SecretAccessKey'],
aws_session_token=credentials['SessionToken']
)
self.s3 = session.client('s3')
I am running a function in Jupyter notebook to test my connection to a Database
Here is the code:
from azure.identity import DefaultAzureCredential
from azure.appconfiguration import AzureAppConfigurationClient
from azure.keyvault.secrets import SecretClient
import requests
config_url = '#####'
credential = DefaultAzureCredential()
app_slug = 'XXXXX'
key = '*******'
label=None
key = '{}:{}'.format(app_slug, key)
config_client = AzureAppConfigurationClient(base_url=config_url, credential=credential)
fetched_config = config_client.get_configuration_setting(key=key, label=label)
fetched_config.value
It will connect for a coworker but not mine. I am wondering if anyone has run into this problem before and could help
The error seems to be in this line:
fetched_config = config_client.get_configuration_setting(key=key, label=label)
but I cannot understand why
I have hosted an API to detect intent in Dialogflow CX using python Flask. I have the environment variable GOOGLE_APPLICATION_CREDENTIALS correctly set to the service account credential json. While calling the API, it results in Default Credential error. If I try to print the value set in GOOGLE_APPLICATION_CREDENTIALS within the python code, it shows None. Whereas, if I check the environment variable in the server using printenv GOOGLE_APPLICATION_CREDENTIALS, I get the location of my json file. What could have went wrong? Could anybody help on this please...
UPDATE
Now, I tried by adding the environment variable in the code. This time, detect_intent doesn't give any response. It just times out after 300 seconds
Here's the code:
from flask import make_response, jsonify, request
from flask_restful import Resource, reqparse
import os
from google.cloud import dialogflowcx_v3
_KEY = 'abcd-chat-bot-e440cw2t611c.json'
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = _KEY
class DetectIntentLatest(Resource):
#classmethod
def post(cls):
parser = reqparse.RequestParser()
parser.add_argument('text')
parser.add_argument('session_id')
args = parser.parse_args()
text = args['text']
session_id = args['session_id']
project_id = "abcd-chat-bot"
location_id = "global"
agent_id = "grt31207-18q3-1a22-6dqa-44856u783364"
agent = f"projects/{project_id}/locations/{location_id}/agents/{agent_id}"
language_code = "en-us"
session_path = f"{agent}/sessions/{session_id}"
print(f"Session path: {session_path}\n")
client_options = None
client = dialogflowcx_v3.SessionsClient()
# Initialize request argument(s)
query_input = dialogflowcx_v3.QueryInput()
query_input.text.text = text
query_input.language_code = language_code
print("_____________________ QUERY INPUT _____________________")
print(query_input)
request = dialogflowcx_v3.DetectIntentRequest(
session=session_path,
query_input=query_input,
)
# Make the request
response = client.detect_intent(request=request)
# Handle the response
print("_____________________ RESPONSE _____________________")
print(response)
return make_response(jsonify(status=1, result='success'), 200)
I am trying to write a python script that uses watchdog to look for file creation and upload that to s3 using boto3. However, my boto3 credentials expire after every 12hrs, So I need to renew them. I am storing my boto3 credentials in ~/.aws/credentials. So right now I am trying to catch the S3UploadFailedError, renew the credentials, and write them to ~/.aws/credentials. But though the credentials are getting renewed and I am calling boto3.client('s3') again its throwing exception.
What am I doing wrong? Or how can I resolve it?
Below is the code snippet
try:
s3 = boto3.client('s3')
s3.upload_file(event.src_path,'bucket-name',event.src_path)
except boto3.exceptions.S3UploadFailedError as e:
print(e)
get_aws_credentials()
s3 = boto3.client('s3')
I have found a good example to refresh the credentials within this link:
https://pritul95.github.io/blogs/boto3/2020/08/01/refreshable-boto3-session/
but there this a little bug inside. Be careful about that.
Here is the corrected code:
from uuid import uuid4
from datetime import datetime
from time import time
from boto3 import Session
from botocore.credentials import RefreshableCredentials
from botocore.session import get_session
class RefreshableBotoSession:
"""
Boto Helper class which lets us create refreshable session, so that we can cache the client or resource.
Usage
-----
session = RefreshableBotoSession().refreshable_session()
client = session.client("s3") # we now can cache this client object without worrying about expiring credentials
"""
def __init__(
self,
region_name: str = None,
profile_name: str = None,
sts_arn: str = None,
session_name: str = None,
session_ttl: int = 3000
):
"""
Initialize `RefreshableBotoSession`
Parameters
----------
region_name : str (optional)
Default region when creating new connection.
profile_name : str (optional)
The name of a profile to use.
sts_arn : str (optional)
The role arn to sts before creating session.
session_name : str (optional)
An identifier for the assumed role session. (required when `sts_arn` is given)
session_ttl : int (optional)
An integer number to set the TTL for each session. Beyond this session, it will renew the token.
50 minutes by default which is before the default role expiration of 1 hour
"""
self.region_name = region_name
self.profile_name = profile_name
self.sts_arn = sts_arn
self.session_name = session_name or uuid4().hex
self.session_ttl = session_ttl
def __get_session_credentials(self):
"""
Get session credentials
"""
session = Session(region_name=self.region_name, profile_name=self.profile_name)
# if sts_arn is given, get credential by assuming given role
if self.sts_arn:
sts_client = session.client(service_name="sts", region_name=self.region_name)
response = sts_client.assume_role(
RoleArn=self.sts_arn,
RoleSessionName=self.session_name,
DurationSeconds=self.session_ttl,
).get("Credentials")
credentials = {
"access_key": response.get("AccessKeyId"),
"secret_key": response.get("SecretAccessKey"),
"token": response.get("SessionToken"),
"expiry_time": response.get("Expiration").isoformat(),
}
else:
session_credentials = session.get_credentials().__dict__
credentials = {
"access_key": session_credentials.get("access_key"),
"secret_key": session_credentials.get("secret_key"),
"token": session_credentials.get("token"),
"expiry_time": datetime.fromtimestamp(time() + self.session_ttl).isoformat(),
}
return credentials
def refreshable_session(self) -> Session:
"""
Get refreshable boto3 session.
"""
# get refreshable credentials
refreshable_credentials = RefreshableCredentials.create_from_metadata(
metadata=self.__get_session_credentials(),
refresh_using=self.__get_session_credentials,
method="sts-assume-role",
)
# attach refreshable credentials current session
session = get_session()
session._credentials = refreshable_credentials
session.set_config_variable("region", self.region_name)
autorefresh_session = Session(botocore_session=session)
return autorefresh_session
According to the documentation, the client looks in several locations for credentials and there are other options that are also more programmatic-friendly that you might want to consider instead of the .aws/credentials file.
Quoting the docs:
The order in which Boto3 searches for credentials is:
Passing credentials as parameters in the boto.client() method
Passing credentials as parameters when creating a Session object
Environment variables
Shared credential file (~/.aws/credentials)
AWS config file (~/.aws/config)
Assume Role provider
In your case, since you are already catching the exception and renewing the credentials, I would simply pass the new ones to a new instance of the client like so:
client = boto3.client(
's3',
aws_access_key_id=NEW_ACCESS_KEY,
aws_secret_access_key=NEW_SECRET_KEY,
aws_session_token=NEW_SESSION_TOKEN
)
If instead you are using these same credentials elsewhere in the code to create other clients, I'd consider setting them as environment variables:
import os
os.environ['AWS_ACCESS_KEY_ID'] = NEW_ACCESS_KEY
os.environ['AWS_SECRET_ACCESS_KEY'] = NEW_SECRET_KEY
os.environ['AWS_SESSION_TOKEN'] = NEW_SESSION_TOKEN
Again, quoting the docs:
The session key for your AWS account [...] is only needed when you are using temporary credentials.
Here is my implementation which only generates new credentials if existing credentials expire using a singleton design pattern
import boto3
from datetime import datetime
from dateutil.tz import tzutc
import os
import binascii
class AssumeRoleProd:
__credentials = None
def __init__(self):
assert True==False
#staticmethod
def __setCredentials():
print("\n\n ======= GENERATING NEW SESSION TOKEN ======= \n\n")
# create an STS client object that represents a live connection to the
# STS service
sts_client = boto3.client('sts')
# Call the assume_role method of the STSConnection object and pass the role
# ARN and a role session name.
assumed_role_object = sts_client.assume_role(
RoleArn=your_role_here,
RoleSessionName=f"AssumeRoleSession{binascii.b2a_hex(os.urandom(15)).decode('UTF-8')}"
)
# From the response that contains the assumed role, get the temporary
# credentials that can be used to make subsequent API calls
AssumeRoleProd.__credentials = assumed_role_object['Credentials']
#staticmethod
def getTempCredentials():
credsExpired = False
# Return object for the first time
if AssumeRoleProd.__credentials is None:
AssumeRoleProd.__setCredentials()
credsExpired = True
# Generate if only 5 minutes are left for expiry. You may setup for entire 60 minutes by catching botocore ClientException
elif (AssumeRoleProd.__credentials['Expiration']-datetime.now(tzutc())).seconds//60<=5:
AssumeRoleProd.__setCredentials()
credsExpired = True
return AssumeRoleProd.__credentials
And then I am using singleton design pattern for client as well which would generate a new client only if new session is generated. You can add region as well if required.
class lambdaClient:
__prodClient = None
def __init__(self):
assert True==False
#staticmethod
def __initProdClient():
credsExpired, credentials = AssumeRoleProd.getTempCredentials()
if lambdaClient.__prodClient is None or credsExpired:
lambdaClient.__prodClient = boto3.client('lambda',
aws_access_key_id=credentials['AccessKeyId'],
aws_secret_access_key=credentials['SecretAccessKey'],
aws_session_token=credentials['SessionToken'])
return lambdaClient.__prodClient
#staticmethod
def getProdClient():
return lambdaClient.__initProdClient()
I would like to read my secret from a pod with python.
I try with this:
import os
import hvac
f = open('/var/run/secrets/kubernetes.io/serviceaccount/token')
jwt = f.read()
client = hvac.Client()
client = hvac.Client(url='https://vault.mydomain.internal')
client.auth_kubernetes("default", jwt)
print(client.read('secret/pippo/pluto'))
I'm sure that secret/pippo/pluto exists.
I'm sure that I'm properly authenticated
But I always receive "None" in answer to my print.
Where can I look to solve this ?
Thx a lot
If you read KV value from Vault, you need the Mount Point and the Path.
Example:
vault_client.secrets.kv.v1.read_secret(
path=path,
mount_point=mount_point
)
i've tried the method you provided in my k8s Python3 pod, i can get Vault secret data successfully.
You need to specify the correct vault token parameter in your hvac.Client and disable client.auth_kubernetes method.
Give it a shot and remember your code should run in k8s Python container instead of your host machine.
import hvac
f = open('/var/run/secrets/kubernetes.io/serviceaccount/token')
jwt = f.read()
print("jwt:", jwt)
f.close()
client = hvac.Client(url='http://vault:8200', token='your_vault_token')
# res = client.auth_kubernetes("envelope-creator", jwt)
res = client.is_authenticated()
print("res:", res)
hvac_secrets_data_k8s = client.read('secret/data/compliance')
print("hvac_secrets_data_k8s:", hvac_secrets_data_k8s)
Below is the result:
92:qfedu shawn$ docker exec -it 202a119367a4 bash
airflow#airflow-858d8c6fcf-bgmwn:~$ ls
airflow-webserver.pid airflow.cfg config dags logs test_valut_in_webserver.py unittests.cfg webserver_config.py
airflow#airflow-858d8c6fcf-bgmwn:~$ python test_valut_in_webserver.py
jwt: eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia
res: True
hvac_secrets_data_k8s: {'request_id': '80caf0cb-8c12-12d2-6517-530eecebd1e0', 'lease_id': '', 'renewable': False, 'lease_duration': 0, 'data': {'data': {'s3AccessKey': 'XXXX', 's3AccessKeyId': 'XXXX', 'sftpPassword': 'XXXX', 'sftpUser': 'XXXX'}, 'metadata': {'created_time': '2020-02-07T14:04:26.7986128Z', 'deletion_time': '', 'destroyed': False, 'version': 4}}, 'wrap_info': None, 'warnings': None, 'auth': None}
As #shawn mentioned above, below commands work for me as well
import hvac
vault_url = 'https://<vault url>:8200/'
vault_token = '<vault token>'
ca_path = '/run/secrets/kubernetes.io/serviceaccount/ca.crt'
secret_path = '<secret path in vault>'
client = hvac.Client(url=vault_url,token=vault_token,verify= ca_path)
client.is_authenticated()
read_secret_result = client.read(secret_path)
print(read_secret_result)
print(read_secret_result['data']['username'])
print(read_secret_result['data']['password'])
Note: ca_path is where the pod stores k8s CA and usually it should be found under "/run/secrets/kubernetes.io/serviceaccount/ca.crt"
I found it easier to use hvac for authentication, and then use the API directly
Can skip this and use root/dev token for testing
import hvac as h
client = h.Client(url='https://<vault url>:8200/')
username = input("username")
import getpass
password = getpass.getpass()
print(client.token)
del username,password
Get the list of mounts
import requests,json
vault_url = 'https://<vault url>:8200/'
vault_token = '<vault token>'
headers = {
'X-Vault-Token': vault_token
}
response = requests.get(vault_url+'v1/sys/mounts', headers=headers)
json.loads(response.text).keys() #The ones ending with / is your mount name
Then get the password (have to create one fist)
mount = '<mount name>'
secret = '<secret name>'
response = requests.get(vault_url+'v1/'+mount+'/'+secret, headers=headers)
response.text
For the username/password to get access to password created by root, you have to add path in the JSON under Policies.