I am generating a file where it should output 2 various requests and 2 various responses from different SOAP Request steps which are:
TestRegion
TestRules
However I noticed the file is just generating TestRegion request and response 2 times each over. For each request it displays TestRegion request and for each response it's displaying TestRegion response. Why is it doing this and how can I get the correct request and responses to show? it does display the hard coded strings correct like TESTREGION REQUEST: and then later on TEST REGION RESPONSE, but the request and response is incorrect.
def testRegionRequest = context.expand( '${${TestRegion}#Request}' )
def testRegionResponse = context.expand( '${${TestRegion}#Response}' )
def testRulesRequest = context.expand( '${${TestRules}#Request}' )
def testRulesResponse = context.expand( '${${TestRules}#Response}' )
def fileName = "XXX.txt"
def logFile = new File(fileName)
//Draws a line
def drawLine(def letter = '=', def count = 70) { letter.multiply(count)}
def testResult = new StringBuffer()
testResult.append drawLine('-', 60) + "\n\n"
testResult.append "\n\nTEST REGION REQUEST:\n\n"
testResult.append(testRegionRequest.toString())
testResult.append "\n\n" + drawLine('-', 60) + "\n\n"
testResult.append "\n\nTEST REGION RESPONSE:\n\n"
testResult.append(testRegionResponse.toString())
testResult.append "\n\n" + drawLine('-', 60) + "\n\n"
testResult.append "\n\nTEST RULES REQUEST:\n\n"
testResult.append(testRulesRequest.toString())
testResult.append "\n\n" + drawLine('-', 60) + "\n\n"
testResult.append "\n\nTEST RULES RESPONSE:\n\n"
testResult.append(testRulesResponse.toString())
// Now create and record the result file
logFile.write(testResult.toString())
It appears that you have incorrect values used in the context.expand
Change from:
${${SearchRegion}#Request}
To:
${SearchRegion#Request}
And the same is applicable to other properties as well.
Related
I am trying to send an email through amazon SES without the SDK so I can send emails asynchronously in Python. I am using amazon's v4 signing method on their site here: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html, but I'm not having any luck sending emails without the SDK. The output is:
RESPONSE++++++++++++++++++++++++++++++++++++
Response code: 403
<ErrorResponse xmlns="http://ses.amazonaws.com/doc/2010-12-01/">
<Error>
<Type>Sender</Type>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.</Message>
</Error>
<RequestId>a19a5fa1-3228-11e9-b2bc-ddb6d8a1cb1c</RequestId>
</ErrorResponse>
Process finished with exit code 0
Here is the block of code generating that response:
import datetime
import hashlib
import hmac
import urllib.parse
import requests
method = 'GET'
service = 'ses'
host = 'email.us-east-1.amazonaws.com'
region = 'us-east-1'
endpoint = 'https://email.us-east-1.amazonaws.com/'
def sign(key, msg):
return hmac.new(key, msg.encode('utf-8'), hashlib.sha256).digest()
def getSignatureKey(key, dateStamp, regionName, serviceName):
kDate = sign(('AWS4' + key).encode('utf-8'), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, 'aws4_request')
return kSigning
access_key = '<my access_key here>'
secret_key = '<my secret_key here>'
my_email = 'my email here'
t = datetime.datetime.utcnow()
amz_date = t.strftime('%Y%m%dT%H%M%SZ')
datestamp = t.strftime('%Y%m%d')
canonical_uri = '/'
canonical_headers = 'host:' + host + '\n'
signed_headers = 'host'
algorithm = 'AWS4-HMAC-SHA256'
credential_scope = datestamp + '/' + region + '/' + service + '/' + 'aws4_request'
canonical_querystring = '''Action=SendEmail
&Source=%s%40gmail.com
&Destination.ToAddresses.member.1=%s%40gmail.com
&Message.Subject.Data=This%20is%20the%20subject%20line.
&Message.Body.Text.Data=Hello.%20I%20hope%20you%20are%20having%20a%20good%20day''' % (my_email, my_email)
canonical_querystring += '&X-Amz-Algorithm=AWS4-HMAC-SHA256'
canonical_querystring += '&X-Amz-Credential=' + urllib.parse.quote_plus(access_key + '/' + credential_scope)
canonical_querystring += '&X-Amz-Date=' + amz_date
canonical_querystring += '&X-Amz-Expires=30'
canonical_querystring += '&X-Amz-SignedHeaders=' + signed_headers
payload_hash = hashlib.sha256(('').encode('utf-8')).hexdigest()
canonical_request = method + '\n' + canonical_uri + '\n' + canonical_querystring + '\n' + canonical_headers + '\n' + signed_headers + '\n' + payload_hash
string_to_sign = algorithm + '\n' + amz_date + '\n' + credential_scope + '\n' + hashlib.sha256(canonical_request.encode('utf-8')).hexdigest()
signing_key = getSignatureKey(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, (string_to_sign).encode("utf-8"), hashlib.sha256).hexdigest()
canonical_querystring += '&X-Amz-Signature=' + signature
request_url = endpoint + "?" + canonical_querystring
print('\nBEGIN REQUEST++++++++++++++++++++++++++++++++++++')
print('Request URL = ' + request_url)
r = requests.get(request_url)
print('\nRESPONSE++++++++++++++++++++++++++++++++++++')
print('Response code: %d\n' % r.status_code)
print(r.text)
This is basically Amazon's code for generating a signature copied and pasted from their docs. Does anyone know what I'm doing wrong in signing my requests to amazon?
Edit* I changed the canonical_querystring to be alphabetical like so:
canonical_querystring = '''Action=SendEmail
&Destination.ToAddresses.member.1={}%40gmail.com
&Message.Body.Text.Data=Hello.%20I%20hope%20you%20are%20having%20a%20good%20day
&Message.Subject.Data=This%20is%20the%20subject%20line.
&Source={}%40gmail.com'''.format(my_email, my_email)
It is still giving me the same error though. Everything else in the query string is alphabetized, or "canonically ordered".
Your canonical query string is not canonical.
You have Action... Source... Destination... Message but all of the parameters need to be lexically ordered.
Step 3: Create the canonical query string.
[...]
The parameters must be sorted by name.
This ordering is the reason why the value is called canonical. It's necessary because the relative positions of query string parameters aren't necessarily guaranteed. The parameters don't need to be sorted in the actual query string accompanying the request, but they do need to be sorted here, for signing.
Since a given request can have only one possible valid signature, the parameters are sorted before signing to remove the ambiguity that might otherwise arise if a user agent or proxy rearranged the query parameters or ordered them arbitrarily when building the URL (as might be expected if, for example, the parameters are passed to the UA as an unordered hash/dictionary structure).
I have created a program and from what I understand from the error shown below and from other posts on Stack, I need to encode the object before it can be hashed.
I have tried several ways to do this but still keep getting the same error message. provided below is my code and also a list of changes I have tried.
I understand what needs to be done but I guess I'm putting the code in the wrong place or the syntax is wrong as what I am trying isn't working.
any help is much appreciated.
Error Message
ha1 = hashlib.md5(user + ':' + realm + ':' + password.strip()).hexdigest()
TypeError: Unicode-objects must be encoded before hashing
Code
import sys
import requests
import hashlib
realm = "Pentester Academy"
lines = [line.rstrip('\n') for line in open('wordl2.txt')]
print (lines)
for user in ['nick', 'admin']:
get_response = requests.get("http://pentesteracademylab.appspot.com/lab/webapp/digest2/1")
test_creds = get_response
print (test_creds)
for password in lines:
# not the correct way but works for this challenge
snounce = test_creds.headers.get('www-authenticate').split('"')
uri = "/lab/webapp/digest2/1"
# create the HTTP Digest
ha1 = hashlib.md5(user + ':' + realm + ':' + password.strip()).hexdigest()
ha2 = hashlib.md5("GET:" + uri).hexdigest()
response = hashlib.md5(ha1 + ':' + snounce + ':' + ha2).hexdigest()
header_string = 'Digest username="%s", realm="%s", nonce="%s", uri="%s", response="%s"' % (user, realm, snounce, uri, response)
headers = { 'Authorization' : header_string }
test_creds = requests.get("http://pentesteracademylab.appspot.com/lab/webapp/digest2/1", headers = headers)
if test_creds.status_code == 200:
print ("CRACKED: %s:%s" % (user,password))
break
elif test_creds.status_code == 401:
print ("FAILED: %s:%s" % (user,password))
else:
print ("unexpected Status code: %d " % test_creds.status_code)
Attempted Changes
password.encode(utf-8)
----------------------
hashlib.md5().update(password.encode(lines.encoding))
---------------
lines = [line.rstrip('\n') for line in open('wordl2.txt', "rb")]
I have managed to solve my own problem using the line
pass2 = str.encode(password)
just inside the password for loop
This code acts as an early warning system for ADFS failures, which works fine when run locally. Problem is that when I run it in Lambda, it loops non stop.
In short:
lambda_handler() runs pagecheck()
pagecheck() produces the info needed then passes 2 lists (msgdet_list, error_list) and an int (error_count) to notification().
notification() collates and prints the output. The output is two key variables (notificationheader and notificationbody).
I've #commentedOut the SNS piece which would usually email the info, and am using print() to instead send the info to CloudWatch logs until I can get the loop sorted. Logs:
CloudWatch logs
If I run this locally, it produces a clean single output. In Lambda, the function will loop until it times out. It's almost like every time the lists are updated, they're passed to the notification() module and it's run. I can limit the function time, but would rather fix the code!
Cheers,
tac
# This python/boto3/lambda script sends a request to an Office 365 landing page, parses return details to confirm a successful redirect to /
# the organisation ADFS homepage, authenticates homepge is correct, raises any errors, and sends a consolodated report to /
# an AWS SNS topic.
# Run once to produce pageserver and htmlchar values for global variables.
# Import required modules
import boto3
import urllib.request
from urllib.request import Request, urlopen
from datetime import datetime
import time
import re
import sys
# Global variables to be set
url = "https://outlook.com/CONTOSSO.com"
adfslink = "https://sts.CONTOSSO.com/adfs/ls/?client-request-id="
# Input after first run
pageserver = "Microsoft-HTTPAPI/2.0 Microsoft-HTTPAPI/2.0"
htmlchar = 18600
# Input AWS SNS ARN
snsarn = 'arn:aws:sns:ap-southeast-2:XXXXXXXXXXXXX:Daily_Check_Notifications_CONTOSSO'
sns = boto3.client('sns')
def pagecheck():
# Present the request to the webpage as if coming from a user in a browser
user_agent = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64)'
values = {'name' : 'user'}
headers = { 'User-Agent' : user_agent }
data = urllib.parse.urlencode(values)
data = data.encode('ascii')
# "Null" the Message Detail and Error lists
msgdet_list = []
error_list = []
request = Request(url)
req = urllib.request.Request(url, data, headers)
response = urlopen(request)
with urllib.request.urlopen(request) as response:
# Get the URL. This gets the real URL.
acturl = response.geturl()
msgdet_list.append("\nThe Actual URL is:")
msgdet_list.append(str(acturl))
if adfslink not in acturl:
error_list.append(str("Redirect Fail"))
# Get the HTTP resonse code
httpcode = response.code
msgdet_list.append("\nThe HTTP code is: ")
msgdet_list.append(str(httpcode))
if httpcode//200 != 1:
error_list.append(str("No HTTP 2XX Code"))
# Get the Headers as a dictionary-like object
headers = response.info()
msgdet_list.append("\nThe Headers are:")
msgdet_list.append(str(headers))
if response.info() == "":
error_list.append(str("Header Error"))
# Get the date of request and compare to UTC (DD MMM YYYY HH MM)
date = response.info()['date']
msgdet_list.append("The Date is: ")
msgdet_list.append(str(date))
returndate = str(date.split( )[1:5])
returndate = re.sub(r'[^\w\s]','',returndate)
returndate = returndate[:-2]
currentdate = datetime.utcnow()
currentdate = currentdate.strftime("%d %b %Y %H%M")
if returndate != currentdate:
date_error = ("Date Error. Returned Date: ", returndate, "Expected Date: ", currentdate, "Times in UTC (DD MMM YYYY HH MM)")
date_error = str(date_error)
date_error = re.sub(r'[^\w\s]','',date_error)
error_list.append(str(date_error))
# Get the server
headerserver = response.info()['server']
msgdet_list.append("\nThe Server is: ")
msgdet_list.append(str(headerserver))
if pageserver not in headerserver:
error_list.append(str("Server Error"))
# Get all HTML data and confirm no major change to content size by character lenth (global var: htmlchar).
html = response.read()
htmllength = len(html)
msgdet_list.append("\nHTML Length is: ")
msgdet_list.append(str(htmllength))
msgdet_list.append("\nThe Full HTML is: ")
msgdet_list.append(str(html))
msgdet_list.append("\n")
if htmllength // htmlchar != 1:
error_list.append(str("Page HTML Error - incorrect # of characters"))
if adfslink not in str(acturl):
error_list.append(str("ADFS Link Error"))
error_list.append("\n")
error_count = len(error_list)
if error_count == 1:
error_list.insert(0, 'No Errors Found.')
elif error_count == 2:
error_list.insert(0, 'Error Found:')
else:
error_list.insert(0, 'Multiple Errors Found:')
# Pass completed results and data to the notification() module
notification(msgdet_list, error_list, error_count)
# Use AWS SNS to create a notification email with the additional data generated
def notification(msgdet_list, error_list, errors):
datacheck = str("\n".join(msgdet_list))
errorcheck = str("\n".join(error_list))
notificationbody = str(errorcheck + datacheck)
if errors >1:
result = 'FAILED!'
else:
result = 'passed.'
notificationheader = ('The daily ADFS check has been marked as ' + result + ' ' + str(errors) + ' ' + str(error_list))
if result != 'passed.':
# message = sns.publish(
# TopicArn = snsarn,
# Subject = notificationheader,
# Message = notificationbody
# )
# Output result to CloudWatch logstream
print('Response: ' + notificationheader)
else:
print('passed')
sys.exit()
# Trigger the Lambda handler
def lambda_handler(event, context):
aws_account_ids = [context.invoked_function_arn.split(":")[4]]
pagecheck()
return "Successful"
sys.exit()
Your CloudWatch logs contain the following error message:
Process exited before completing request
This is caused by invoking sys.exit() in your code. Locally your Python interpreter will just terminate when encountering such a sys.exit().
AWS Lambda on the other hand expects a Python function to just return and handles sys.exit() as an error. As your function probably got invoked asynchronously AWS Lambda retries to execute it twice.
To solve your problem, you can replace the occurences of sys.exit() with return or even better, just remove the sys.exit() calls, as there would be already implicit returns in the places where you use sys.exit().
I have a SOAP request which has dynamic values generated by random method. How to capture these generated values to log?
My SOAP request:
<soap:Envelope xmlns:soap="http://www.w3.org/2003/05/soap-envelope" xmlns:web="http://www.webserviceX.NET/">
<soap:Header/>
<soap:Body>
<web:ChangeLengthUnit>
<web:LengthValue>${=(int)(Math.random()*9999)}</web:LengthValue>
<web:fromLengthUnit>Inches</web:fromLengthUnit>
<web:toLengthUnit>Centimeters</web:toLengthUnit>
</web:ChangeLengthUnit>
</soap:Body>
</soap:Envelope>
Groovy script:
import com.eviware.soapui.support.GroovyUtils;
def prj = testRunner.testCase.testSuite.project.workspace.getProjectByName("Project1")
tCase = prj.testSuites['TestSuite'].testCases['TestCase']
tStep = tCase.getTestStepByName("ChangeLengthUnit")
def stepReq = tStep.getProperty("Request").getValue()
def runner = tStep.run(testRunner, context)
log.info ("runner status ....... : " + runner.hasResponse())
log.info stepReq
for( assertion in tStep.assertionList )
{
log.info "Assertion [" + assertion.label + "] has status [" + assertion.status + "]"
for( e in assertion.errors )
log.info "-> Error [" + e.message + "]"
}
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
def holder = groovyUtils.getXmlHolder( stepReq.toString() )
log.info holder.getNodeValues( "//web:LengthValue" ).toString()
Above groovy script is giving output as below:
Wed Oct 18 14:55:13 SGT 2017:INFO:[${=(int)(Math.random()*9999)}]
Actual value of LengthValue tag = 3490, How to get this value ?
It can easily read from RawRequest property using Script Assertion for the same request step.
assert context.rawRequest, 'Request is empty or null'
def xml = new XmlSlurper().parseText(context.rawRequest)
def actualValueInTheRequest = xml.'**'.find{it.name() == 'LengthValue'}?.text()
log.info "Value for LengthValue in the actual request is : $actualValueInTheRequest"
I think what you're doing is getting the LengthValue from the request 'template' not what SoapUI has actually passed to the web service.
In your example, you have...
def stepReq = tStep.getProperty("Request").getValue()
The request is what you see in SoapUI that includes your vars, e.g. ${myVar}. I think what you're after is the 'raw request'.
In SoapUI, run the request and after it has ran, you should see a tab labelled 'Raw Request'. This is what SoapUI actually sends to the web service, in here you'll see that the vars have been evaluated and will show the value, which is what you are after.
I've taken your example and made some mods it. I haven't tested it, but it should help you.
import com.eviware.soapui.support.GroovyUtils;
def prj = testRunner.testCase.testSuite.project.workspace.getProjectByName("Project1")
tCase = prj.testSuites['TestSuite'].testCases['TestCase']
tStep = tCase.getTestStepByName("ChangeLengthUnit")
def stepReq = tStep.getProperty("Request").getValue()
def runner = tStep.run(testRunner, context)
// CHA MOD - Can only get raw request after running the step.
def rawRequest = context.expand( '${ChangeLengthUnit#RawRequest}' )
//CHA MOD - Quick look at the rawRequest...
log.info(rawRequest);
log.info ("runner status ....... : " + runner.hasResponse())
log.info stepReq
for( assertion in tStep.assertionList )
{
log.info "Assertion [" + assertion.label + "] has status [" + assertion.status + "]"
for( e in assertion.errors )
log.info "-> Error [" + e.message + "]"
}
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
def holder = groovyUtils.getXmlHolder( stepReq.toString() )
log.info holder.getNodeValues( "//web:LengthValue" ).toString()
//CHA MOD - Get the length from the raw request...
holder = groovyUtils.getXmlHolder( rawRequest.toString() )
log.info holder.getNodeValues( "//web:LengthValue" ).toString()
You are getting value from the actual request. Try to get the data from RawRequest
use below code
def holder = groovyUtils.getXmlHolder(mentionThenameofthestep#RawRequest)
The above code will point to Raw request of your teststep which contains actual value passed during execution instead of "${=(int)(Math.random()*9999)}"
I am trying to capture the raw response for REST (POST) API call into a file using groovy Script.
I can see the response as below in RAW, but when file is produced it is blank.
REST Response:
HTTP/1.1 401 Unauthorized
content-length: 0
Date: Tue 12 jul 2016 12:12:12gmt
WWW-Autheticate: OAuth
Server: Jetty (8.1.17V20150415)
I am using SOAP UI version 5.2.
Any help appreciated.
Groovy Script:
def Date startTime = new Date()
File it=new File("Result")
def cur_Time = startTime.getMonth()+1 + "_" + startTime.getDate()
cur_Time = cur_Time + "_" + startTime.getHours() + startTime.getMinutes() +startTime.getSeconds()
def fileName = it.name + "_" + cur_Time
//Request File
def myXmlRequest="C:\\ConnectivityResults\\"+ "Rest_Request" + fileName+".xml"
def request=context.expand('${Testcasename#Request}')
def req = new File (myXmlRequest)
req.write(request,"UTF-8")
//Response File
def myXmlResponse="C:\\ConnectivityResults\\"+ "Rest_Response" + fileName+".xml"
def response=context.expand('${Testcasename#Response}')
def res = new File (myXmlResponse)
res.write(response,"UTF-8")
The problem isn't probably in your Groovy script, the problem is simply that your request is incorrect and nothing is returned as response. Based on the http-headers you show in the question:
HTTP/1.1 401 Unauthorized
content-length: 0
Date: Tue 12 jul 2016 12:12:12gmt
WWW-Autheticate: OAuth
Server: Jetty (8.1.17V20150415)
You're receiving an 401 Unauthorized response instead of 200 OK, and based on the Content-lenght which is 0. It's normal that your response is blank, so there is no content to save in file.
EDIT BASED ON COMMENT
If you want also to save the http-headers in a file, you can add the follow snippet to your Groovy script:
def fileName = ...
// http-headers file
def httpHeadersFilePath ="C:/ConnectivityResults/Rest_Request${fileName}.txt"
def ts = testRunner.testCase.getTestStepByName('Testcasename')
def headers = ts.getTestRequest().response.responseHeaders
def httpHeaderFile = new File(httpHeadersFilePath)
httpHeaderFile.text = ''
headers.each { key, value ->
httpHeaderFile.append("${key}:${value}\n",'UTF-8')
}
Hope it helps,
Sorry about the late...
There's a simple way to take it and record in a file, using Groovy Script on your SoapUI:
#Take the Raw Request into a variable "request":
def request = context.expand( '${Request 1#RawRequest}' )
#Take the Raw Response into a variable "response":
def response = context.expand( '${Request 1#Response}' )
#Create and fill a file "MyFile.json" whit the variables values:
new File( "C:/foo/bar/MyFile.json" ).write( request + response, "UTF-8" )
Hope that's useful.