Getting access key age AWS Boto3 - python-3.x

I am trying to figure out a way to get a users access key age through an aws lambda function using Python 3.6 and Boto 3. My issue is that I can't seem to find the right api call to use if any exists for this purpose. The two closest that I can seem to find are list_access_keys which I can use to find the creation date of the key. And get_access_key_last_used which can give me the day the key was last used. However neither or others I can seem to find give simply the access key age like is shown in the AWS IAM console users view. Does a way exist to get simply the Access key age?

This simple code do the same stuff without converting a lot of time etc:
import boto3
from datetime import date
client = boto3.client('iam')
username = "<YOUR-USERNAME>"
res = client.list_access_keys(UserName=username)
accesskeydate = res['AccessKeyMetadata'][0]['CreateDate'].date()
currentdate = date.today()
active_days = currentdate - accesskeydate
print (active_days.days)

There is no direct way. You can use the following code snippet to achieve what you are trying:
import boto3, json, time, datetime, sys
client = boto3.client('iam')
username = "<YOUR-USERNAME>"
res = client.list_access_keys(UserName=username)
accesskeydate = res['AccessKeyMetadata'][0]['CreateDate'] ### Use for loop if you are going to run this on production. I just wrote it real quick
accesskeydate = accesskeydate.strftime("%Y-%m-%d %H:%M:%S")
currentdate = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime())
accesskeyd = time.mktime(datetime.datetime.strptime(accesskeydate, "%Y-%m-%d %H:%M:%S").timetuple())
currentd = time.mktime(datetime.datetime.strptime(currentdate, "%Y-%m-%d %H:%M:%S").timetuple())
active_days = (currentd - accesskeyd)/60/60/24 ### We get the data in seconds. converting it to days
print (int(round(active_days)))
Let me know if this works as expected.

Upon further testing, I've come up with the following which runs in Lambda. This function in python3.6 will email users if their IAM keys are 90 days or older.
Pre-requisites
all IAM users have an email tag with a proper email address as the value.
Example;
IAM user tag key: email
IAM user tag value: someone#gmail.com
every email used, needs to be confirmed in SES
import boto3, os, time, datetime, sys, json
from datetime import date
from botocore.exceptions import ClientError
iam = boto3.client('iam')
email_list = []
def lambda_handler(event, context):
print("All IAM user emails that have AccessKeys 90 days or older")
for userlist in iam.list_users()['Users']:
userKeys = iam.list_access_keys(UserName=userlist['UserName'])
for keyValue in userKeys['AccessKeyMetadata']:
if keyValue['Status'] == 'Active':
currentdate = date.today()
active_days = currentdate - \
keyValue['CreateDate'].date()
if active_days >= datetime.timedelta(days=90):
userTags = iam.list_user_tags(
UserName=keyValue['UserName'])
email_tag = list(filter(lambda tag: tag['Key'] == 'email', userTags['Tags']))
if(len(email_tag) == 1):
email = email_tag[0]['Value']
email_list.append(email)
print(email)
email_unique = list(set(email_list))
print(email_unique)
RECIPIENTS = email_unique
SENDER = "AWS SECURITY "
AWS_REGION = os.environ['region']
SUBJECT = "IAM Access Key Rotation"
BODY_TEXT = ("Your IAM Access Key need to be rotated in AWS Account: 123456789 as it is 3 months or older.\r\n"
"Log into AWS and go to your IAM user to fix: https://console.aws.amazon.com/iam/home?#security_credential"
)
BODY_HTML = """
AWS Security: IAM Access Key Rotation: Your IAM Access Key need to be rotated in AWS Account: 123456789 as it is 3 months or older. Log into AWS and go to your https://console.aws.amazon.com/iam/home?#security_credential to create a new set of keys. Ensure to disable / remove your previous key pair.
"""
CHARSET = "UTF-8"
client = boto3.client('ses',region_name=AWS_REGION)
try:
response = client.send_email(
Destination={
'ToAddresses': RECIPIENTS,
},
Message={
'Body': {
'Html': {
'Charset': CHARSET,
'Data': BODY_HTML,
},
'Text': {
'Charset': CHARSET,
'Data': BODY_TEXT,
},
},
'Subject': {
'Charset': CHARSET,
'Data': SUBJECT,
},
},
Source=SENDER,
)
except ClientError as e:
print(e.response['Error']['Message'])
else:
print("Email sent! Message ID:"),
print(response['MessageId'])

Using the above methods you will only get the age of the access keys. But as a best practice or a security approach, you need to check the rotation period, when the keys are last rotated. If the keys rotation age is more than 90 days you could alert your team.
The only way to get the rotation age of the access keys is by using the credentials report from IAM. Download it, parse it, and calculate the age.

Related

AWS Cognito 90 day automated Password rotation

I have a requirement to create an automated password reset script. I created a custom field in order to try and track this and also hope I can access some of the standard fields. This script should find users with the following criteria:
The latest of any of the following 3 dates that are >= 90 days ago : Sign_Up, Forgot_Password, or custom:pwdCreateDate
I can't seem to find any boto3 cognito client ways of getting the information on this except for forgot password which shows up in admin_list_user_auth_events and that response doesn't include username in the response. I suppose since you provide username to get the events you can figure out a way to find the latest forgot password from the events and tie it to the username.
Has anyone else implemented any boto3 automation to set the account to force password reset based on any of these fields?
here is where i landed, take it with the understanding that coginito has some limitations which make true flawless password rotation difficult. Also know if you can make the script more efficient you should because in lambda you probably time out with more than about 350 users due to the 5RPS on the admin API.
Prerequisites : set the lambda function to 5 concurrency or you will exceed the limit of 5RPS. 1 mutable field in your cognito userpool attributes to put a date in. a custom lambda zip file that includes pandas saved to s3.
import os
import sys
# this adds the parent directory of bin so we can find the module
parent_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir))
sys.path.append(parent_dir)
#This addes venv lib/python2.7/site-packages/ to the search path
mod_path = os.path.abspath(parent_dir+"/lib/python"+str(sys.version_info[0])+"."+str(sys.version_info[1])+"/site-packages/")
sys.path.append(mod_path)
import boto3
import datetime
import pandas as pd
import time
current_path = os.path.dirname(os.path.realpath(__file__))
# Use this one for the parent directory
ENV_ROOT = os.path.abspath(os.path.join(current_path, os.path.pardir))
# Use this one for the current directory
#ENV_ROOT = os.path.abspath(os.path.join(current_path))
sys.path.append(ENV_ROOT)
#if __name__ == "__main__":
def lambda_handler(event, context):
user_pool_id = os.environ['USER_POOL_ID']
idp_client = boto3.client('cognito-idp')
users_list = []
page_token = None
dateToday = datetime.datetime.today().date()
def update_user(user) :
idp_client.admin_update_user_attributes(
UserPoolId = user_pool_id,
Username = user,
UserStatus = 'RESET_REQUIRED',
UserAttributes = [
{
'Name': 'custom:pwdCreateDate',
'Value': str(dateToday)
}
]
)
users = idp_client.list_users(
UserPoolId = user_pool_id
)
for user in users['Users']: users_list.append(user['Username'])
page_token = users['PaginationToken']
while 'PaginationToken' in users :
users = idp_client.list_users(
UserPoolId = user_pool_id,
PaginationToken = page_token
)
for user in users["Users"]: users_list.append(user["Username"])
if 'PaginationToken' in users :
page_token = users['PaginationToken']
attrPwdDates = []
for i in range(len(users_list)) :
userAttributes = idp_client.admin_get_user(
UserPoolId = user_pool_id,
Username = users_list[i]
)
for a in userAttributes['UserAttributes'] :
if a['Name'] == 'custom:pwdCreateDate' :
attrPwdDates.append(datetime.datetime.strptime(a['Value'], '%Y-%m-%d %H:%M:%S.%f').date())
time.sleep(1.0)
list_of_userattr_tuples = list(zip(users_list, attrPwdDates))
df1 = pd.DataFrame(list_of_userattr_tuples,columns = ['Username','Password_Last_Set'])
authPwdDates = []
for i in range(len(users_list)) :
authEvents = idp_client.admin_list_user_auth_events(
UserPoolId = user_pool_id,
Username = users_list[i]
)
for event in authEvents['AuthEvents'] :
if event['EventType'] == 'ForgotPassword' and event['EventResponse'] == 'Pass' :
authPwdDates.append(event['CreationDate'].date())
break
time.sleep(1.0)
list_of_userauth_tuples = list(zip(users_list, authPwdDates))
df2 = pd.DataFrame(list_of_userauth_tuples,columns = ['Username','Password_Last_Forgot'])
df3 = df1.merge(df2,how='left', on = 'Username')
df3[['Password_Last_Set','Password_Last_Forgot']] = df3[['Password_Last_Set','Password_Last_Forgot']].apply(pd.to_datetime)
cols = ['Password_Last_Set','Password_Last_Forgot']
df4 = df3.loc[df3[cols].max(axis=1)<=pd.Timestamp.now() - pd.Timedelta(90, unit='d'), 'Username']
for i,r in df4.iterrows() :
update_user(r['Username'])

Run Python Script within Python & check if value is outputted - If statement

I have a Python3 script which basically runs through a list of Amazon AWS Account numbers (Uses Boto3), checks to see if their access keys are older than x number of days and report on it.
I'd like to make my report nice by checking to see if the output has a user(s) or not and output this into a file for SNS to email to me.
Here is the code I've already tried:
if not os.system("python3 ListUsersWithAccessKeysOlderThan90Days.py " + accountNumber):
print("No Content", file=reportName)
else:
print("Content", file=reportName)
I've already tried this too:
if os.system("python3 ListUsersWithAccessKeysOlderThan90Days.py " + accountNumber) == " ":
print("No Content", file=reportName)
else:
print("Content", file=reportName)
But I only seem to get this in my output file:
Running on account accountNumber - accountLabel - accountEnvironment
No Content
Running on account accountNumber - accountLabel - accountEnvironment
No Content
Running on account accountNumber - accountLabel - accountEnvironment
No Content
Ideally, I'd like it to look like this:
Running on account accountNumber - accountLabel - accountEnvironment
No Content
Running on account accountNumber - accountLabel - accountEnvironment
Content
Running on account accountNumber - accountLabel - accountEnvironment
No Content
No Content = No access keys need rotating.
Content = User needs their key rotating.
I can achieve this in Bash, but I wouldn't mind trying to get it working in Python3.
Here is my Bash example:
if [[ -z "$(python3 ListUsersWithAccessKeysOlderThan90Days.py ${ACCOUNT})" ]]; then
echo -e "$ACCOUNT ($LABEL) is up to date no need to report\n" >> $REPORT
else
echo -e "$ACCOUNT Need keys rotating" >> $REPORT
fi
Any help would be most appreciated.
Thanks,
You can get the status of IAM users and credentials from the AWS Credentials Report. That would probably satisfy most needs.
If you prefer Python, then I've written a basic script that can be used to print out all IAM users in an account whose access keys are over 90 days old (regardless of when they last used these keys).
import sys
import boto3
from datetime import datetime, timedelta, timezone
DAYS = 90
iam = boto3.client('iam')
sts = boto3.client('sts')
identity = sts.get_caller_identity()
account = identity['Account']
header_printed = False
count = 0
today = datetime.now(timezone.utc)
# Get all IAM users in this AWS account
for user in iam.list_users()['Users']:
arn = user['Arn']
username = user['UserName']
# Get all access keys for this IAM user
keys = iam.list_access_keys(UserName=username)
# Test each key's age and print those that are too old
for key in keys['AccessKeyMetadata']:
akid = key['AccessKeyId']
created = key['CreateDate']
created_delta = today - created
# if this access key is older than DAYS
if created + timedelta(days=DAYS) < today:
count += 1
response = iam.get_access_key_last_used(AccessKeyId=akid)
akid_last_used = response['AccessKeyLastUsed']
if not header_printed:
header_printed = True
print(f'Account, Username, Access Key, Age, Last Used')
print(f'{account}, {username}, {akid}, {created_delta.days} ', end = '')
# Only keys that have actually been used will have last used date
if 'LastUsedDate' in akid_last_used:
last_used = akid_last_used['LastUsedDate']
last_used_delta = today - last_used
print(flast_used_delta.days)
else:
print('none')
sys.exit(count)
This will print out a list of access keys over 90 days, in a CSV format. For example:
Account, Username, Access Key, Age, Last Used
123456784321, james, AKIAJ7PL4POLWNEXAMPLE, 91, 1
123456784321, frank, AKIAL2CV9LKWEXAMPLE, 200, 100
123456784321, mary, AKIAYTWHD3BNMLEXAMPLE, 97, none
The Age is how many days old the access key is. The Last Used is how many days it has been since the credential was last used. Hope this proves to be helpful.
The script's exit code is the count of keys older than 90 days, so you can use this exit code in a shell script to decide what to do next. For example:
#!/bin/bash
python3 scripts_older_than_90days.py > oldkeys.csv
count=$?
if [ $count -eq 0 ]
then
echo "All access keys good"
else
echo "Count of old keys" $count
fi

How can I return a string from a Google BigQuery row iterator object?

My task is to write a Python script that can take results from BigQuery and email them out. I've written a code that can successfully send an email, but I am having trouble including the results of the BigQuery script in the actual email. The query results are correct, but the actual object I am returning from the query (results) always returns as a Nonetype.
For example, the email should look like this:
Hello,
You have the following issues that have been "open" for more than 7 days:
-List issues here from bigquery code
Thanks.
The code reads in contacts from a contacts.txt file, and it reads in the email message template from a message.txt file. I tried to make the bigquery object into a string, but it still results in an error.
from google.cloud import bigquery
import warnings
warnings.filterwarnings("ignore", "Your application has authenticated using end user credentials")
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from string import Template
def query_emailtest():
client = bigquery.Client(project=("analytics-merch-svcs-thd"))
query_job = client.query("""
select dept, project_name, reset, tier, project_status, IssueStatus, division, store_number, top_category,
DATE_DIFF(CURRENT_DATE(), in_review, DAY) as days_in_review
from `analytics-merch-svcs-thd.MPC.RESET_DETAILS`
where in_review IS NOT NULL
AND IssueStatus = "In Review"
AND DATE_DIFF(CURRENT_DATE(), in_review, DAY) > 7
AND ready_for_execution IS NULL
AND project_status = "Active"
AND program_name <> "Capital"
AND program_name <> "SSI - Capital"
LIMIT 50
""")
results = query_job.result() # Waits for job to complete.
return results #THIS IS A NONETYPE
def get_queryresults(results): #created new method to put query results into a for loop and store it in a variable
for i,row in enumerate(results,1):
bq_data = (i , '. ' + str(row.dept) + " " + row.project_name + ", Reset #: " + str(row.reset) + ", Store #: " + str(row.store_number) + ", " + row.IssueStatus + " for " + str(row.days_in_review)+ " days")
print (bq_data)
def get_contacts(filename):
names = []
emails = []
with open(filename, mode='r', encoding='utf-8') as contacts_file:
for a_contact in contacts_file:
names.append(a_contact.split()[0])
emails.append(a_contact.split()[1])
return names, emails
def read_template(filename):
with open(filename, 'r', encoding='utf-8') as template_file:
template_file_content = template_file.read()
return Template(template_file_content)
names, emails = get_contacts('mycontacts.txt') # read contacts
message_template = read_template('message.txt')
results = query_emailtest()
bq_results = get_queryresults(query_emailtest())
import smtplib
# set up the SMTP server
s = smtplib.SMTP(host='smtp-mail.outlook.com', port=587)
s.starttls()
s.login('email', 'password')
# For each contact, send the email:
for name, email in zip(names, emails):
msg = MIMEMultipart() # create a message
# bq_data = get_queryresults(query_emailtest())
# add in the actual person name to the message template
message = message_template.substitute(PERSON_NAME=name.title())
message = message_template.substitute(QUERY_RESULTS=bq_results) #SUBSTITUTE QUERY RESULTS IN MESSAGE TEMPLATE. This is where I am having trouble because the Row Iterator object results in Nonetype.
# setup the parameters of the message
msg['From']='email'
msg['To']='email'
msg['Subject']="This is TEST"
# body = str(get_queryresults(query_emailtest())) #get query results from method to put into message body
# add in the message body
# body = MIMEText(body)
#msg.attach(body)
msg.attach(MIMEText(message, 'plain'))
# query_emailtest()
# get_queryresults(query_emailtest())
# send the message via the server set up earlier.
s.send_message(msg)
del msg
Message template:
Dear ${PERSON_NAME},
Hope you are doing well. Please find the following alert for Issues that have been "In Review" for greater than 7 days.
${QUERY_RESULTS}
If you would like more information, please visit this link that contains a complete dashboard view of the alert.
ISE Services
The BQ result() function returns a generator, so I think you need to change your return to yield from.
I'm far from a python expert, but the following pared-down code worked for me.
from google.cloud import bigquery
import warnings
warnings.filterwarnings("ignore", "Your application has authenticated using end user credentials")
def query_emailtest():
client = bigquery.Client(project=("my_project"))
query_job = client.query("""
select field1, field2 from `my_dataset.my_table` limit 5
""")
results = query_job.result()
yield from results # NOTE THE CHANGE HERE
results = query_emailtest()
for row in results:
print(row.field1, row.field2)

Python - Error querying Solarwinds N-Central via SOAP

I'm using python 3 to write a script that generates a customer report for Solarwinds N-Central. The script uses SOAP to query N-Central and I'm using zeep for this project. While not new to python I am new to SOAP.
When calling the CustomerList fuction I'm getting the TypeError: __init__() got an unexpected keyword argument 'listSOs'
import zeep
wsdl = 'http://' + <server url> + '/dms/services/ServerEI?wsdl'
client = zeep.CachingClient(wsdl=wsdl)
config = {'listSOs': 'true'}
customers = client.service.CustomerList(Username=nc_user, Password=nc_pass, Settings=config)
Per the perameters below 'listSOs' is not only a valid keyword, its the only one accepted.
CustomerList
public com.nable.nobj.ei.Customer[] CustomerList(String username, String password, com.nable.nobj.ei.T_KeyPair[] settings) throws RemoteException
Parameters:
username - MSP N-central username
password - Corresponding MSP N-central password
settings - A list of non default settings stored in a T_KeyPair[]. Below is a list of the acceptable Keys and Values. If not used leave null
(Key) listSOs - (Value) "true" or "false". If true only SOs with be shown, if false only customers and sites will be shown. Default value is false.
I've also tried passing the dictionary as part of a list:
config = []
key = {'listSOs': 'true'}
config += key
TypeError: Any element received object of type 'str', expected lxml.etree._Element or builtins.dict or zeep.objects.T_KeyPair
Omitting the Settings value entirely:
customers = client.service.CustomerList(Username=nc_user, Password=nc_pass)
zeep.exceptions.ValidationError: Missing element Settings (CustomerList.Settings)
And trying zeep's SkipValue:
customers = client.service.CustomerList(Username=nc_user, Password=nc_pass, Settings=zeep.xsd.SkipValue)
zeep.exceptions.Fault: java.lang.NullPointerException
I'm probably missing something simple but I've been banging my head against the wall off and on this for awhile I'm hoping someone can point me in the right direction.
Here's my source code from my getAssets.py script. I did it in Python2.7, easily upgradeable though. Hope it helps someone else, N-central's API documentation is really bad lol.
#pip2.7 install zeep
import zeep, sys, csv, copy
from zeep import helpers
api_username = 'your_ncentral_api_user'
api_password='your_ncentral_api_user_pw'
wsdl = 'https://(yourdomain|tenant)/dms2/services2/ServerEI2?wsdl'
client = zeep.CachingClient(wsdl=wsdl)
response = client.service.deviceList(
username=api_username,
password=api_password,
settings=
{
'key': 'customerId',
'value': 1
}
)
# If you can't tell yet, I code sloppy
devices_list = []
device_dict = {}
dev_inc = 0
max_dict_keys = 0
final_keys = []
for device in response:
# Iterate through all device nodes
for device_properties in device.items:
# Iterate through each device's properties and add it to a dict (keyed array)
device_dict[device_properties.first]=device_properties.second
# Dig further into device properties
device_properties = client.service.devicePropertyList(
username=api_username,
password=api_password,
deviceIDs=device_dict['device.deviceid'],
reverseOrder=False
)
prop_ind = 0 # This is a hacky thing I did to make my CSV writing work
for device_node in device_properties:
for prop_tree in device_node.properties:
for key, value in helpers.serialize_object(prop_tree).items():
prop_ind+=1
device_dict["prop" + str(prop_ind) + "_" + str(key)]=str(value)
# Append the dict to a list (array), giving us a multi dimensional array, you need to do deep copy, as .copy will act like a pointer
devices_list.append(copy.deepcopy(device_dict))
# check to see the amount of keys in the last item
if len(devices_list[-1].keys()) > max_dict_keys:
max_dict_keys = len(devices_list[-1].keys())
final_keys = devices_list[-1].keys()
print "Gathered all the datas of N-central devices count: ",len(devices_list)
# Write the data out to a CSV
with open('output.csv', 'w') as csvfile:
fieldnames = final_keys
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
for csv_line in devices_list:
writer.writerow(csv_line)

Get list of commits by user with the GitLab API

I can successfully access info about a user with this command:
curl http://gitlab.$INTERNAL_SERVER.com/api/v3/\
users/$USER_ID\?private_token\=$GITLAB_TOKEN
However, I can not find the API endpoint for getting a list of the commits that the user has pushed to the GitLab server. Does a URL with this info exist?
To the best of my knowledge, such an API endpoint does not exist. Essentially the best I've been able to come up with is this flow:
find all the projects the user is involved with (not 100% simple in itself)
then get commits for that project
THEN filter those commits based on useremail.
I am using java-gitlab-api to access the Gitlab server, so don't have curl samples handy (sorry!).
It looks like you can get a list of commits by using the Events endpoint
data = requests.get(host + "/api/v4/users/{id}/events".format(id=user_id),
params={"action": "pushed"})
And you can chain that by updating params to
params.update({"before": before_date})
Where before date can be the last element in data, and you can loop continuously to get all commits by user from a specific date
I have written a Python script that does what #demaniak suggest. Enjoy
import requests
import ujson as json
header ={...}
def get_all_commits_gitlab(project_id, username):
json_loads_of_commit = []
f_date = "2022-01-01T00:00:42.000+01:00"
params = {"until": f_date}
url_p = "https://gitlab.xxx.xx/api/v4/projects/%d/\
repository/commits" % project_id
r = requests.get(url_p, params, headers=header)
c = 0
while r.status_code == 200:
jsLoad = json.loads(r.content)
newDate = jsLoad[-1]["committed_date"]
if (params["until"] == newDate):
break
user_commits = []
for cm in jsLoad:
if cm["author_name"] == username:
user_commits.append(cm)
c += 1
json_loads_of_commit.append(user_commits)
params["until"] = newDate
r = requests.get(url_p, params, headers=header)
print("project %d: %d commits by user %s, \
the first one %s" % (project_id, c, username, newDate))
return json_loads_of_commit

Resources