I have a requirement to create an automated password reset script. I created a custom field in order to try and track this and also hope I can access some of the standard fields. This script should find users with the following criteria:
The latest of any of the following 3 dates that are >= 90 days ago : Sign_Up, Forgot_Password, or custom:pwdCreateDate
I can't seem to find any boto3 cognito client ways of getting the information on this except for forgot password which shows up in admin_list_user_auth_events and that response doesn't include username in the response. I suppose since you provide username to get the events you can figure out a way to find the latest forgot password from the events and tie it to the username.
Has anyone else implemented any boto3 automation to set the account to force password reset based on any of these fields?
here is where i landed, take it with the understanding that coginito has some limitations which make true flawless password rotation difficult. Also know if you can make the script more efficient you should because in lambda you probably time out with more than about 350 users due to the 5RPS on the admin API.
Prerequisites : set the lambda function to 5 concurrency or you will exceed the limit of 5RPS. 1 mutable field in your cognito userpool attributes to put a date in. a custom lambda zip file that includes pandas saved to s3.
import os
import sys
# this adds the parent directory of bin so we can find the module
parent_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir))
sys.path.append(parent_dir)
#This addes venv lib/python2.7/site-packages/ to the search path
mod_path = os.path.abspath(parent_dir+"/lib/python"+str(sys.version_info[0])+"."+str(sys.version_info[1])+"/site-packages/")
sys.path.append(mod_path)
import boto3
import datetime
import pandas as pd
import time
current_path = os.path.dirname(os.path.realpath(__file__))
# Use this one for the parent directory
ENV_ROOT = os.path.abspath(os.path.join(current_path, os.path.pardir))
# Use this one for the current directory
#ENV_ROOT = os.path.abspath(os.path.join(current_path))
sys.path.append(ENV_ROOT)
#if __name__ == "__main__":
def lambda_handler(event, context):
user_pool_id = os.environ['USER_POOL_ID']
idp_client = boto3.client('cognito-idp')
users_list = []
page_token = None
dateToday = datetime.datetime.today().date()
def update_user(user) :
idp_client.admin_update_user_attributes(
UserPoolId = user_pool_id,
Username = user,
UserStatus = 'RESET_REQUIRED',
UserAttributes = [
{
'Name': 'custom:pwdCreateDate',
'Value': str(dateToday)
}
]
)
users = idp_client.list_users(
UserPoolId = user_pool_id
)
for user in users['Users']: users_list.append(user['Username'])
page_token = users['PaginationToken']
while 'PaginationToken' in users :
users = idp_client.list_users(
UserPoolId = user_pool_id,
PaginationToken = page_token
)
for user in users["Users"]: users_list.append(user["Username"])
if 'PaginationToken' in users :
page_token = users['PaginationToken']
attrPwdDates = []
for i in range(len(users_list)) :
userAttributes = idp_client.admin_get_user(
UserPoolId = user_pool_id,
Username = users_list[i]
)
for a in userAttributes['UserAttributes'] :
if a['Name'] == 'custom:pwdCreateDate' :
attrPwdDates.append(datetime.datetime.strptime(a['Value'], '%Y-%m-%d %H:%M:%S.%f').date())
time.sleep(1.0)
list_of_userattr_tuples = list(zip(users_list, attrPwdDates))
df1 = pd.DataFrame(list_of_userattr_tuples,columns = ['Username','Password_Last_Set'])
authPwdDates = []
for i in range(len(users_list)) :
authEvents = idp_client.admin_list_user_auth_events(
UserPoolId = user_pool_id,
Username = users_list[i]
)
for event in authEvents['AuthEvents'] :
if event['EventType'] == 'ForgotPassword' and event['EventResponse'] == 'Pass' :
authPwdDates.append(event['CreationDate'].date())
break
time.sleep(1.0)
list_of_userauth_tuples = list(zip(users_list, authPwdDates))
df2 = pd.DataFrame(list_of_userauth_tuples,columns = ['Username','Password_Last_Forgot'])
df3 = df1.merge(df2,how='left', on = 'Username')
df3[['Password_Last_Set','Password_Last_Forgot']] = df3[['Password_Last_Set','Password_Last_Forgot']].apply(pd.to_datetime)
cols = ['Password_Last_Set','Password_Last_Forgot']
df4 = df3.loc[df3[cols].max(axis=1)<=pd.Timestamp.now() - pd.Timedelta(90, unit='d'), 'Username']
for i,r in df4.iterrows() :
update_user(r['Username'])
Related
I'm a new Django programmer. I write a API call with rest_framework in Django. when call this API, my program connect to KUCOIN and get list of all cryptocurrency. I want save symbol and name this cryptocurrencies in database. For save data to database, I use 'for loop' and in every for loop iteration, I query to database and save data. my code :
for currencie in currencies:
name = currencie['name']
symbol = currencie['symbol']
active = (False, True)[symbol.endswith('USDT')]
oms = 'kucoin'
try:
obj = Instrument.objects.get(symbol=symbol, oms=oms)
setattr(obj, 'name', name)
setattr(obj, 'active', active)
obj.save()
except Instrument.DoesNotExist:
obj = Instrument(name=name, symbol=symbol,
active=active, oms=oms)
obj.save()
query to database in every for loop iteration have problem ,How can I solve this problem?
Exist any way in Django to save data in database in with one query.
All my code:
class getKucoinInstrument(APIView):
def post(self, request):
try:
person = Client.objects.filter(data_provider=True).first()
person_data = ClientSerializer(person, many=False).data
api_key = person_data['api_key']
api_secret = person_data['secret_key']
api_passphrase = person_data['api_passphrase']
client = kucoin_client(api_key, api_secret, api_passphrase)
currencies = client.get_symbols()
for currencie in currencies:
name = currencie['name']
symbol = currencie['symbol']
active = (False, True)[symbol.endswith('USDT')]
oms = 'kucoin'
try:
obj = Instrument.objects.get(symbol=symbol, oms=oms)
setattr(obj, 'name', name)
setattr(obj, 'active', active)
obj.save()
except Instrument.DoesNotExist:
obj = Instrument(name=name, symbol=symbol,
active=active, oms=oms)
obj.save()
return Response({'response': 'Instruments get from kucoin'}, status=status.HTTP_200_OK)
except Exception as e:
print(e)
return Response({'response': 'Internal server error'}, status=status.HTTP_500_INTERNAL_SERVER_ERROR)
Thank you for you help.
Yes! Take a look at bulk_create() documentation. https://docs.djangoproject.com/en/4.0/ref/models/querysets/#bulk-create
If you have a db that supports ignore_conflicts parameter (all do, except Oracle), you can do this:
new_currencies = []
for currencie in currencies:
name = currencie['name']
symbol = currencie['symbol']
active = (False, True)[symbol.endswith('USDT')]
oms = 'kucoin'
new_currencies.append(Instrument(name=name, symbol=symbol,
active=active, oms=oms))
Instrument.objects.bulk_create(new_currencies, ignore_conflicts=True)
1-liner:
Instrument.objects.bulk_create(
[
Instrument(
name=currencie['name'], symbol=currencie['symbol'],
active=currencie['symbol'].endswith('USDT'), oms='kucoin'
)
for currencie in currencies
],
ignore_conflicts=True
)
I'm doing a web scraping data university research project. I started working on a ready GitHub project, but this project does not retrieve all the data.
The project works like this:
Search Google using keywords: example: (accountant 'email me at' Google)
Extract a snippet.
Retrieve data from this snippet.
The issue is:
The snippets extracted are like this: " ... marketing division in 2009. For more information on career opportunities with our company, email me: vicki#productivedentist.com. Neighborhood Smiles, LLCĀ ..."
The snippet does not show all, the "..." hides information like role, location... How can I retrieve all the information with the script?
from googleapiclient.discovery import build #For using Google Custom Search Engine API
import datetime as dt #Importing system date for the naming of the output file.
import sys
from xlwt import Workbook #For working on xls file.
import re #For email search using regex.
if __name__ == '__main__':
# Create an output file name in the format "srch_res_yyyyMMdd_hhmmss.xls in output folder"
now_sfx = dt.datetime.now().strftime('%Y%m%d_%H%M%S')
output_dir = './output/'
output_fname = output_dir + 'srch_res_' + now_sfx + '.xls'
search_term = sys.argv[1]
num_requests = int(sys.argv[2])
my_api_key = "replace_with_you_api_key" #Read readme.md to know how to get you api key.
my_cse_id = "011658049436509675749:gkuaxghjf5u" #Google CSE which searches possible LinkedIn profile according to query.
service = build("customsearch", "v1", developerKey=my_api_key)
wb=Workbook()
sheet1 = wb.add_sheet(search_term[0:15])
wb.save(output_fname)
sheet1.write(0,0,'Name')
sheet1.write(0,1,'Profile Link')
sheet1.write(0,2,'Snippet')
sheet1.write(0,3,'Present Organisation')
sheet1.write(0,4,'Location')
sheet1.write(0,5,'Role')
sheet1.write(0,6,'Email')
sheet1.col(0).width = 256 * 20
sheet1.col(1).width = 256 * 50
sheet1.col(2).width = 256 * 100
sheet1.col(3).width = 256 * 20
sheet1.col(4).width = 256 * 20
sheet1.col(5).width = 256 * 50
sheet1.col(6).width = 256 * 50
wb.save(output_fname)
row = 1 #To insert the data in the next row.
#Function to perform google search.
def google_search(search_term, cse_id, start_val, **kwargs):
res = service.cse().list(q=search_term, cx=cse_id, start=start_val, **kwargs).execute()
return res
for i in range(0, num_requests):
# This is the offset from the beginning to start getting the results from
start_val = 1 + (i * 10)
# Make an HTTP request object
results = google_search(search_term,
my_cse_id,
start_val,
num=10 #num value can be 1 to 10. It will give the no. of results.
)
for profile in range (0, 10):
snippet = results['items'][profile]['snippet']
myList = [item for item in snippet.split('\n')]
newSnippet = ' '.join(myList)
contain = re.search(r'[\w\.-]+#[\w\.-]+', newSnippet)
if contain is not None:
title = results['items'][profile]['title']
link = results['items'][profile]['link']
org = "-NA-"
location = "-NA-"
role = "-NA-"
if 'person' in results['items'][profile]['pagemap']:
if 'org' in results['items'][profile]['pagemap']['person'][0]:
org = results['items'][profile]['pagemap']['person'][0]['org']
if 'location' in results['items'][profile]['pagemap']['person'][0]:
location = results['items'][profile]['pagemap']['person'][0]['location']
if 'role' in results['items'][profile]['pagemap']['person'][0]:
role = results['items'][profile]['pagemap']['person'][0]['role']
print(title[:-23])
sheet1.write(row,0,title[:-23])
sheet1.write(row,1,link)
sheet1.write(row,2,newSnippet)
sheet1.write(row,3,org)
sheet1.write(row,4,location)
sheet1.write(row,5,role)
sheet1.write(row,6,contain[0])
print('Wrote {} search result(s)...'.format(row))
wb.save(output_fname)
row = row + 1
print('Output file "{}" written.'.format(output_fname))
Please I need help here. In the code below, I want the FOR loop to loop through all the accounts with the associated API KEYs and SECRETs listed and send bitcoins from them to the recipient email address one after the other as long as their balance is greater than zero:
#!/data/data/com.termux/files/usr/bin/python3.8
from coinbase.wallet.client import Client
import json
api_key1 = '<key>'
api_secret1 = '<secret>'
api_key2 = '<key>'
api_secret2 = '<secret>'
api_key3 = '<key>'
api_secret3 = '<secret>'
api_key4 = '<key>'
api_secret4 = '<secret>'
api_key5 = '<key>'
api_secret5 = '<secret>'
client = Client(api_key, api_secret)
accounts = client.get_accounts()['data']
for account in accounts:
sending_currency = account['currency']
if float(account['balance']['amount']) > 0:
#Send to other account
sending_account = client.get_account(account['id'])
sending_amount = account['balance']['amount']
print('sending %s %s from SENDER_EMAIL_ADDRESS' %(sending_amount, sending_currency))
sending_account.send_money(to = 'RECEPIENT_EMAIL_ADDRESS', amount = sending_amount, currency = sending_currency)
In order to attain what you have described, I would use a list of dictionaries, this will also be iterable and hence usable in the for loop. in order to do this I'd rewrite your code as such :
#!/data/data/com.termux/files/usr/bin/python3.8
from coinbase.wallet.client import Client
import json
credentials = [
{'api_key':'<key1>', 'api_secret':'<secret1>'},
{'api_key':'<key2>', 'api_secret':'<secret2>'},
{'api_key':'<key3>', 'api_secret':'<secret3>'}
........
]
while True:
for credential in credentials:
client = Client(credential['api_key'], credential['api_secret'])
accounts = client.get_accounts()['data']
for account in accounts:
sending_currency = account['currency']
if float(account['balance']['amount']) > 0:
#Send to other account
sending_account = client.get_account(account['id'])
sending_amount = account['balance']['amount']
print('sending %s %s from SENDER_EMAIL_ADDRESS' %(sending_amount, sending_currency))
sending_account.send_money(to = 'RECEPIENT_EMAIL_ADDRESS', amount = sending_amount, currency = sending_currency)
else:
........
I am using flask session to achieve multi-user concurrency in my chatbot which is embedded in FB messenger and backend implemented in python.
Below is code:
#app.route("/", methods=['GET','POST'])
def webhook():
state = session.get('state')
session.modified = True
print("state:",state)
q_number = session.get('q_number')
age = session.get('age')
print('age:',age)
sex = session.get('sex')
symptoms = session.get('symptoms', [])
prev = session.get('prev')
conditions = session.get('conditions')
data = request.get_json()
##pprint(data["entry"])
sender_id = data["entry"][0]["messaging"][0]["sender"]["id"]
# get user info
r = requests.get('https://graph.facebook.com/v3.1/'+sender_id+
'?fields=first_name,last_name&access_token='+ACCESS_TOKEN)
if data["object"] == "page":
user_input = data['entry'][0]['messaging'][0]['message']['text']
#print(type(user_input))
user_input = user_input.lower()
#print(user_input)
question = get_question(sender_id,user_input,state, q_number, age, sex, symptoms, prev, conditions)
send_message(sender_id, question)
return "ok", 200
session.get is not working in flask and i am unable to fetch the values. so i cant able to send the state,age,sex,q_number,symptoms,prev,conditions to other function.. Please help me to overcome this.
I am trying to figure out a way to get a users access key age through an aws lambda function using Python 3.6 and Boto 3. My issue is that I can't seem to find the right api call to use if any exists for this purpose. The two closest that I can seem to find are list_access_keys which I can use to find the creation date of the key. And get_access_key_last_used which can give me the day the key was last used. However neither or others I can seem to find give simply the access key age like is shown in the AWS IAM console users view. Does a way exist to get simply the Access key age?
This simple code do the same stuff without converting a lot of time etc:
import boto3
from datetime import date
client = boto3.client('iam')
username = "<YOUR-USERNAME>"
res = client.list_access_keys(UserName=username)
accesskeydate = res['AccessKeyMetadata'][0]['CreateDate'].date()
currentdate = date.today()
active_days = currentdate - accesskeydate
print (active_days.days)
There is no direct way. You can use the following code snippet to achieve what you are trying:
import boto3, json, time, datetime, sys
client = boto3.client('iam')
username = "<YOUR-USERNAME>"
res = client.list_access_keys(UserName=username)
accesskeydate = res['AccessKeyMetadata'][0]['CreateDate'] ### Use for loop if you are going to run this on production. I just wrote it real quick
accesskeydate = accesskeydate.strftime("%Y-%m-%d %H:%M:%S")
currentdate = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime())
accesskeyd = time.mktime(datetime.datetime.strptime(accesskeydate, "%Y-%m-%d %H:%M:%S").timetuple())
currentd = time.mktime(datetime.datetime.strptime(currentdate, "%Y-%m-%d %H:%M:%S").timetuple())
active_days = (currentd - accesskeyd)/60/60/24 ### We get the data in seconds. converting it to days
print (int(round(active_days)))
Let me know if this works as expected.
Upon further testing, I've come up with the following which runs in Lambda. This function in python3.6 will email users if their IAM keys are 90 days or older.
Pre-requisites
all IAM users have an email tag with a proper email address as the value.
Example;
IAM user tag key: email
IAM user tag value: someone#gmail.com
every email used, needs to be confirmed in SES
import boto3, os, time, datetime, sys, json
from datetime import date
from botocore.exceptions import ClientError
iam = boto3.client('iam')
email_list = []
def lambda_handler(event, context):
print("All IAM user emails that have AccessKeys 90 days or older")
for userlist in iam.list_users()['Users']:
userKeys = iam.list_access_keys(UserName=userlist['UserName'])
for keyValue in userKeys['AccessKeyMetadata']:
if keyValue['Status'] == 'Active':
currentdate = date.today()
active_days = currentdate - \
keyValue['CreateDate'].date()
if active_days >= datetime.timedelta(days=90):
userTags = iam.list_user_tags(
UserName=keyValue['UserName'])
email_tag = list(filter(lambda tag: tag['Key'] == 'email', userTags['Tags']))
if(len(email_tag) == 1):
email = email_tag[0]['Value']
email_list.append(email)
print(email)
email_unique = list(set(email_list))
print(email_unique)
RECIPIENTS = email_unique
SENDER = "AWS SECURITY "
AWS_REGION = os.environ['region']
SUBJECT = "IAM Access Key Rotation"
BODY_TEXT = ("Your IAM Access Key need to be rotated in AWS Account: 123456789 as it is 3 months or older.\r\n"
"Log into AWS and go to your IAM user to fix: https://console.aws.amazon.com/iam/home?#security_credential"
)
BODY_HTML = """
AWS Security: IAM Access Key Rotation: Your IAM Access Key need to be rotated in AWS Account: 123456789 as it is 3 months or older. Log into AWS and go to your https://console.aws.amazon.com/iam/home?#security_credential to create a new set of keys. Ensure to disable / remove your previous key pair.
"""
CHARSET = "UTF-8"
client = boto3.client('ses',region_name=AWS_REGION)
try:
response = client.send_email(
Destination={
'ToAddresses': RECIPIENTS,
},
Message={
'Body': {
'Html': {
'Charset': CHARSET,
'Data': BODY_HTML,
},
'Text': {
'Charset': CHARSET,
'Data': BODY_TEXT,
},
},
'Subject': {
'Charset': CHARSET,
'Data': SUBJECT,
},
},
Source=SENDER,
)
except ClientError as e:
print(e.response['Error']['Message'])
else:
print("Email sent! Message ID:"),
print(response['MessageId'])
Using the above methods you will only get the age of the access keys. But as a best practice or a security approach, you need to check the rotation period, when the keys are last rotated. If the keys rotation age is more than 90 days you could alert your team.
The only way to get the rotation age of the access keys is by using the credentials report from IAM. Download it, parse it, and calculate the age.