Invoking sagemaker endpoint using AWS-Lamda throwing parameter validation error - python-3.x

Question

I trained an ML model in AWS Sagemaker and created an endpoint. I want to invoke it using AWS-Lambda. My model has 30 predictor variables. So I passed them into test event of Lambda as dict type as mentioned below
{
"Time": "10 ",
"V1": "1.449043781 ",
"V2": "-1.176338825 ",
"V3": "0.913859833 ",
"V4": "-1.375666655 ",
"V5": "-1.971383165 ",
"V6": "-0.629152139 ",
"V7": "-1.423235601 ",
"V8": "0.048455888 ",
"V9": "-1.720408393 ",
"V10": "1.626659058 ",
"V11": "1.19964395 ",
"V12": "-0.671439778 ",
"V13": "-0.513947153 ",
"V14": "-0.095045045 ",
"V15": "0.230930409 ",
"V16": "0.031967467 ",
"V17": "0.253414716 ",
"V18": "0.854343814 ",
"V19": "-0.221365414 ",
"V20": "-0.387226474 ",
"V21": "-0.009301897 ",
"V22": "0.313894411 ",
"V23": "0.027740158 ",
"V24": "0.500512287 ",
"V25": "0.251367359 ",
"V26": "-0.129477954 ",
"V27": "0.042849871 ",
"V28": "0.016253262 ",
"Amount": "7.8"
}
Now I ran below mentioned code in AWS Lambda
import json
import os
import csv
import boto3
import io
import codecs
endpoint_name = os.environ['ENDPOINT_NAME']
runtime = boto3.client('runtime.sagemaker')
def lambda_handler(event, context):
print("received event: "+json.dumps(event,indent=2))
data = json.loads(json.dumps(event))
payload = data["Time"]+data["V1"]+data["V2"]+data["V3"]+data["V4"]+data["V5"]+data["V6"]+data["V7"]+data["V8"]+data["V9"]+data["V10"]+data["V11"]+data["V12"]+data["V13"]+data["V14"]+data["V15"]+data["V16"]+data["V17"]+data["V18"]+data["V19"]+data["V20"]+data["V21"]+data["V22"]+data["V23"]+data["V24"]+data["V25"]+data["V26"]+data["V27"]+data["V28"]+data["Amount"]
payload = payload.split(" ")
payload = [codecs.encode(i,'utf-8') for i in payload]
payload=[bytearray(i) for i in payload]
print(payload)
response = runtime.invoke_endpoint(EndpointName=endpoint_name,ContentType='text/csv',Body=payload)
print(response)
result=json.loads(response['Body'].decode())
pred = int(float(response))
predicted_label = 'fraud' if pred==1 else 'not fraud'
return predicted_label
This code is throwing below this error
[ERROR] ParamValidationError: Parameter validation failed:
Invalid type for parameter Body, value: [bytearray(b'10'), bytearray(b'1.449043781'), bytearray(b'-1.176338825'), bytearray(b'0.913859833'), bytearray(b'-1.375666655'), bytearray(b'-1.971383165'), bytearray(b'-0.629152139'), bytearray(b'-1.423235601'), bytearray(b'0.048455888'), bytearray(b'-1.720408393'), bytearray(b'1.626659058'), bytearray(b'1.19964395'), bytearray(b'-0.671439778'), bytearray(b'-0.513947153'), bytearray(b'-0.095045045'), bytearray(b'0.230930409'), bytearray(b'0.031967467'), bytearray(b'0.253414716'), bytearray(b'0.854343814'), bytearray(b'-0.221365414'), bytearray(b'-0.387226474'), bytearray(b'-0.009301897'), bytearray(b'0.313894411'), bytearray(b'0.027740158'), bytearray(b'0.500512287'), bytearray(b'0.251367359'), bytearray(b'-0.129477954'), bytearray(b'0.042849871'), bytearray(b'0.016253262'), bytearray(b'7.8')], type: <class 'list'>, valid types: <class 'bytes'>, <class 'bytearray'>, file-like object
I understand that somehow I need to pass my 30 features into Lambda function such that data type of payload is compatible with ContentType for respnse to work. Can someone please explain how to do it?
edit: I'm trying this problem by looking at this aws blog. I don't quite understand how the author of above mentioned blog did it.

Answer 1

Pretty sure your error is coming from:
response = runtime.invoke_endpoint(EndpointName=endpoint_name,ContentType='text/csv',Body=payload)
but just FYI, it's very helpful if when you ask the a question about an exception, you point out the line causing it :)
The invoke_endpoint() docs specify that Body has to be:
bytes or seekable file-like object
and that Body
provides input data, in the format specified in the ContentType request header.
Your content type is text/csv so the endpoint will be expecting bytes which represent a csv string. The question then is what is it getting instead that's making it unhappy?
payload = data["Time"]+data["V1"]+ ...
At this point, payload is a string
payload = payload.split(" ")
Now it's a list of strings
payload = [codecs.encode(i,'utf-8') for i in payload]
Now it's a list of strings encoded in utf-8 (this step might be unnecessary)
payload=[bytearray(i) for i in payload]
Now it's a list of bytearrays.
And that list is what you're passing as Body.
So, to get it to work, you'll need to change your logic to:
Make your original input into a CSV
Convert that CSV into one single byte array
Pass that array as part of your invocation
A note on style: passing the input numbers as strings, with trailing spaces so you can split(" ") is pretty hacky. Just pass the input as plain numbers (remove the " surrounding them), and find a way to do what you need with appending then splitting (these are effectively inverse operations, they cancel each other out).