Errno 20 Not a directory: '/tmp/tmp6db1q_sn/pyproject.toml/egg-info' - python-3.x

I'm wrapping my colleague's code in an AWS SAM. I made a lambda to just call existing code with a run method which I did like this.
import json
from sentance_analyser_v2.main import run
def lambda_handler(event, context):
if (event['path'] == '/analyse'):
return RunAnalyser(event, context)
else:
return {
"statusCode": 200,
"body": json.dumps({
"message": "Hello"
}),
}
def RunAnalyser(event, context):
source = event['queryStringParameters']['source']
output = run(source)
return {
"statusCode": 200,
"body": json.dumps({
"output": json.dumps(output)
})
}
After running into numerous package problems I realized I forget to use the sam build --use-container command and I was hoping after a build it would solve some of my package errors but I ran into this error code -
Error: PythonPipBuilder:ResolveDependencies - [Errno 20] Not a directory: '/tmp/tmp6db1q_sn/pyproject.toml/egg-info'
After trying to understand what the freak that even is I started pulling my hairs out because there is nothing on this...
Here is the basic template that I currently use just so I can test that my wrapping worked. I'm still very new to AWS & SAM so I don't want to overcomplicate things just yet with the template.
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
sam-app
Sample SAM Template for sam-app
Globals:
Function:
Timeout: 30
Tracing: Active
Api:
TracingEnabled: True
Resources:
MainLambda:
Type: AWS::Serverless::Function
Properties:
CodeUri: main_lambda/
Handler: app.lambda_handler
Runtime: python3.9
Architectures:
- x86_64
Events:
Analyser:
Type: Api
Properties:
Path: /analyse
Method: get
Outputs:
HelloEfsApi:
Description: "API Gateway endpoint URL for Prod stage for Hello EFS function"
Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/hello/"

Related

Not able to attach EventBridge Schedule to Lambda Function in AWS SAM

I'm unable to attach an EventBridge Schedule to a lambda function in AWS SAM.
Here is the section of my SAM template that intends to attach a schedule to the Lambda function:
TestLambda:
Type: AWS::Serverless::Function
Properties:
FunctionName: TestLambda
CodeUri: test_directory/
Handler: test_file.lambda_handler
Runtime: python3.8
Role: !Ref LambdaApiRole
Events:
TestLambdaCron:
Type: Schedule
Properties:
Schedule: rate(1 day)
This is the error I get while deploying the template:
Embedded stack <stack ARN> was not successfully updated. Currently in UPDATE_ROLLBACK_IN_PROGRESS with reason: The following resource(s) failed to create: [TestLambdaTestLambdaCron].
A solution would be much appreciated, thanks in advance!
PS: LambdaApiRole is defined in the template and can be referenced in the function (working fine), it's when TestLambdaCron is added when the issues occur
The AWS SAM CLI version I am working on is: SAM CLI, version 1.37.0
Here is simple template that I have it and it works.
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
Sample SAM Template for demo
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 3
Resources:
TestFunctionLambda:
Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
Properties:
CodeUri: app/
Handler: app.lambda_handler
Runtime: python3.9
Architectures:
- x86_64
Role: arn:aws:iam::631249183025:role/mycstomrole
Events:
CWSchedule:
Type: Schedule
Properties:
Schedule: "rate(1 minute)"
Name: TestSchedule
Description: test schedule
Enabled: true

Falcon application can not render Openapi(swagger) specification

Hello everyone we are running a Falcon application that uses the falcon-apispec library to generate OpenAPI specifications.
Here is our code that initializes the definition:
import falcon
from apispec import APISpec
from falcon_apispec import FalconPlugin
from kubernetes import config
from api.admission_response import AdmissionResponse
from api.health import Health
from api.k8s_config_validator import K8sConfigValidator
from api.middleware.json import RequireJSON, JSONTranslator
from api.apidocs import ApiDocs
def create_app(config_validator):
api = falcon.API(middleware=[
RequireJSON(),
JSONTranslator(),
])
resources ={
'/': AdmissionResponse(config_validator),
'/api-docs': ApiDocs(),
'/health': Health()
}
for r in resources:
api.add_route(r, resources[r])
setup_swagger_documentation(api, resources)
# initialize k8s client
config.load_incluster_config()
return api
def get_app():
return create_app(K8sConfigValidator())
def setup_swagger_documentation(api, resources):
spec = APISpec(
title='Admission Controller API',
version='latest',
openapi_version='2.0',
plugins=[
FalconPlugin(api)
],
info=dict(description="Admission Controller API"),
)
for r in resources:
spec.path(resource=resources[r])
with open('./api/config/openapi/openapi_spec.yaml', 'w') as f:
f.write(spec.to_yaml())
Here is our openapi-spec defintion defined:
openapi: 3.0.0
info:
description: Admission Controller API
title: Admission Controller API
version: latest
paths:
/:
post:
tags:
- "API"
parameters:
- in: "query"
name: "body"
description: "List of user object"
required: true
schema:
type: string
responses:
"200":
description: "Success"
/api-docs:
get:
tags:
- "API Doc Endpoints"
responses:
"200":
description: "Success"
/health:
get:
tags:
- "Health Endpoints"
responses:
"200":
description: "Success"
And here is one of the classes that defines what should be done on a post:
class AdmissionResponse(object):
def __init__(self, k8s_config_validator):
self.k8s_config_validator = k8s_config_validator
#falcon.before(validate_schema)
def on_post(self, req, resp):
"""
---
tags: ['API']
parameters:
- in: "query"
name: "body"
description: "List of user object"
required: true
type: string
responses:
"200":
description: "Success"
"""
admission_review = AdmissionReview(req.context['doc'])
errors = self.k8s_config_validator.validate(admission_review)
if errors:
resp.context['result'] = ResponseBuilder(admission_review).not_allowed(errors)
api.logger.info("Validations for %s of kind %s in %s failed with %s", admission_review.name(), admission_review.kind(), admission_review.namespace(), errors)
else:
resp.context['result'] = ResponseBuilder(admission_review).allowed()
api.logger.info("Validations for %s of kind %s in %s passed", admission_review.name(), admission_review.kind(), admission_review.namespace())
Whenever we try to hit our hosted swagger-ui we run into this error:
Unable to render this definition The provided definition does not specify a valid version field.
Please indicate a valid Swagger or OpenAPI version field. Supported version fields are swagger: "2.0" and those that match openapi: 3.0.n (for example, openapi: 3.0.0).
Does anyone know how we could resolve this? When we paste in our openapi specification to the swagger editor located here: https://editor.swagger.io/ it works just fine. Any help would be awesome!
When calling APISpec, not all strings are supported.
Try something like "0.0.1" (you supplied 'latest').
Also, using falcon_swagger_ui, an error may be reported if the openapi_version format is not correct (in such case, examples of valid formats are provided).
This works for me:
spec = APISpec(
title="My APP",
version="0.0.1",
openapi_version='3.0.0',
plugins=[FalconPlugin(api)]
)

Failed to parse Dialogflow response into AppResponse because of invalid platform response (with Youtube API)

I am trying to build a YouTube entertainment app using Google Assistant, following this tutorial: here. I have followed every step precisely, copying code verbatim, but when I run the test, I get this error:
MalformedResponse Failed to parse Dialogflow response into AppResponse because of invalid platform response: Could not find a RichResponse or SystemIntent in the platform response for agentId: ~~ and intentId: ~~. WebhookStatus: code: 2 message: "Webhook call failed. Error: UNKNOWN." ..
I'm not really well versed in DialogFlow, so I'm not sure what's happening. If anyone has any advice, I'd really appreciate it!
Edit: So, here's what happens that triggers the error. I follow the tutorial all the way to the end. I run the test and type in their test request 'rahman'. The response I get back from the test is the above error. I'm not sure what other details I can add, but if there's anything else I can provide, please let me know!
Edit 2: Following the next comment I received, I opened the cloud functions up in the GCP console and found that a new function was made called dialogflowFirebaseFulfillment. I checked the logs for the 'youtube' function I made, and found this notification:
{
insertId: "..."
labels: {
execution_id: ""
}
logName: "projects/<name of project>/logs/cloudfunctions.googleapis.com%2Fcloud-functions"
receiveTimestamp: "<time>"
resource: {
labels: {…}
type: "cloud_function"
}
severity: "ERROR"
textPayload: "Warning, estimating Firebase Config based on GCLOUD_PROJECT. Intializing firebase-admin may fail"
timestamp: "<time>"
}
I then checked out the new function that was made without me knowing and saw it didn't deploy, having an error: "Function failed on loading user code. Error message: Node.js module defined by file index.js is expected to export function named dialogflowFirebaseFulfillment". I checked the logs and found this:
{
insertId: "<id>"
logName: "projects/<project name>/logs/cloudaudit.googleapis.com%2Factivity"
operation: {
id: "operations/<id>"
last: true
producer: "cloudfunctions.googleapis.com"
}
protoPayload: {
#type: "type.googleapis.com/google.cloud.audit.AuditLog"
authenticationInfo: {
principalEmail: "<email>"
}
methodName: "google.cloud.functions.v1.CloudFunctionsService.UpdateFunction"
requestMetadata: {
destinationAttributes: {
}
requestAttributes: {
}
}
resourceName: "projects/<project name>/locations/us-central1/functions/dialogflowFirebaseFulfillment"
serviceName: "cloudfunctions.googleapis.com"
status: {
code: 3
message: "INVALID_ARGUMENT"
}
}
receiveTimestamp: "<time>"
resource: {
labels: {…}
type: "cloud_function"
}
severity: "ERROR"
timestamp: "<time>"
I know this isn't a good sign, but I also don't know how to really interpret where I should go fix the error. Any ideas would be appreciated, thanks!

Python Step Functions API: get_activity_task seems to always timeout

I've got a lambda function like this:
import boto3
import os
import json
step_functions = boto3.client('stepfunctions')
workers_topic = boto3.resource('sns').Topic(os.environ.get("WORKERS_TOPIC_ARN"))
def test_push_to_workers_sns(event, context):
activity_response = \
step_functions.get_activity_task(
activityArn=os.environ.get("ACKNOWLEDGE_ACTIVITY_ARN"),
workerName='test_push_to_workers_sns'
)
task_token, input_ = activity_response['taskToken'], activity_response['input']
print(f"Task token is {task_token}")
print(f"Input is {input}")
if not task_token:
print("No activity found")
return
workers_topic.publish(Message="blah blah")
When I set off an execution of the step function I have and it reaches the activity, I've repeatedly checked that running aws stepfunctions get-activity-task --activity-arn <ACKNOWLEDGE_ACTIVITY_ARN> on my terminal returns a taskToken and input that are both correct. However this lambda function seems to always time out regardless of whether or not the activity is running (I've got my timeout value set to 1 min 15 secs on the lambda function, and the activity state on the step function's timeout at 1 hour)
I checked this case using following CloudFormation template and it works:
AWSTemplateFormatVersion: "2010-09-09"
Description: Stack creating AWS Step Functions state machine and lambda function calling GetActivityTask.
Resources:
LambdaFunction:
Type: AWS::Lambda::Function
Properties:
Handler: "index.handler"
Role: !GetAtt LambdaExecutionRole.Arn
Code:
ZipFile: |
import boto3
import os
import json
step_functions = boto3.client('stepfunctions')
workers_topic = boto3.resource('sns').Topic(os.environ.get("WORKERS_TOPIC_ARN"))
def handler(event, context):
activity_response = step_functions.get_activity_task(
activityArn=os.environ.get("ACKNOWLEDGE_ACTIVITY_ARN"),
workerName='test_push_to_workers_sns'
)
if 'taskToken' not in activity_response:
return
task_token, task_input = activity_response['taskToken'], activity_response['input']
print(f"Task token is {task_token}")
print(f"Input is {input}")
workers_topic.publish(Message="blah blah")
step_functions.send_task_success(
taskToken=task_token,
output=task_input
)
Runtime: "python3.6"
Timeout: 25
Environment:
Variables:
WORKERS_TOPIC_ARN: !Ref WorkersTopic
ACKNOWLEDGE_ACTIVITY_ARN: !Ref AcknowledgeActivity
StateMachine:
Type: AWS::StepFunctions::StateMachine
Properties:
RoleArn: !GetAtt StatesExecutionRole.Arn
DefinitionString: !Sub
- >
{
"Comment": "State Machine for GetActivityTask testing purposes.",
"StartAt": "FirstState",
"States": {
"FirstState": {
"Type": "Task",
"Resource": "${ACKNOWLEDGE_ACTIVITY_ARN}",
"End": true
}
}
}
- ACKNOWLEDGE_ACTIVITY_ARN: !Ref AcknowledgeActivity
AcknowledgeActivity:
Type: AWS::StepFunctions::Activity
Properties:
Name: !Sub ${AWS::AccountId}-AcknowledgeActivity
WorkersTopic:
Type: AWS::SNS::Topic
LambdaExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Path: "/"
Policies:
- PolicyName: StepFunctionsAccess
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- states:GetActivityTask
- states:SendTaskFailure
- states:SendTaskSuccess
Resource: arn:aws:states:*:*:*
- PolicyName: SNSAccess
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- SNS:Publish
Resource: arn:aws:sns:*:*:*
StatesExecutionRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- !Sub states.${AWS::Region}.amazonaws.com
Action: sts:AssumeRole
Path: "/"
Policies: []
I created execution manually from Step Functions console and executed Lambda manually from Lambda console.
Keep in mind that after you take taskToken using GetActivityTask the execution related with it is waiting for response (SendTaskSuccess or SendTaskFailure) until it reaches timeout (default is very long). So if you have taken token before then that execution isn't available for GetAcitivtyTask. You can lookup status of execution in Step Functions console looking at events for specific execution.
You should call SendTaskSuccess or SendTaskFailure from your code after getting token from GetActivityTask (otherwise execution will be hanging until it reaches timeout or is stopped).
Aside from original question: GetActivityTask is not designed to be called from Lambda. You can pass Lambda Function as resource to state machine (instead of activity) and it will be called when execution reaches specified state (event in handler will contain execution state). Activities should be used only for long-running jobs on dedicated machines (EC2, ECS). I should also point that there are service limits for GetActivityTask calls (25 RPS with bucket of size 1000) and Lambda-based states are limited basically only by transition count limit (400 per second with bucket of size 800). You can read more about step function limits here: https://docs.aws.amazon.com/step-functions/latest/dg/limits.html

Setting HTTP Proxy on AWS API Gateway via Cloudformation

Thanks for the help in advance.
Currently using cloudformation templates to deploy a simple API to AWS as part of a POC for moving from Azure to AWS API management.
I have got everything working except i have not been able to figure out the YAML AWS extension for setting the HTTP proxy checkbox for the HTTP request.
Sample YAML below. I know this will not set that checkbox (as i have tested it and it worked minus that problem), but on this page
https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-swagger-extensions-integration.html
i cannot see a extension that sets this option? Has AWS not done this yet
AWSTemplateFormatVersion: '2010-09-09'
Resources:
PlayersAPI:
Type: AWS::ApiGateway::RestApi
Properties:
Name: RAH API
Description: A demo API for testing
Body:
swagger: '2.0'
info:
title: test api
description: test api
version: 1.0.1
contact:
name: SH
email: test#mailinator.com
paths:
"/heartbeat":
get:
description: Checks the API is working
produces:
- application/json
responses:
'200':
description: API Response information
x-amazon-apigateway-integration:
type: http
responses:
default:
statusCode: '200'
httpMethod: GET
uri: https://api.example.com
This works for me:
resources:
Resources:
ProxyResource:
Type: AWS::ApiGateway::Resource
Properties:
ParentId:
Fn::GetAtt:
- ApiGatewayRestApi # our default Rest API logical ID
- RootResourceId
PathPart: "{proxy+}" # the endpoint in your API that is set as proxy
RestApiId:
Ref: ApiGatewayRestApi
ProxyMethod:
Type: AWS::ApiGateway::Method
Properties:
ResourceId:
Ref: ProxyResource
RestApiId:
Ref: ApiGatewayRestApi
HttpMethod: GET # the method of your proxy. Is it GET or POST or ... ?
MethodResponses:
- StatusCode: 200
Integration:
IntegrationHttpMethod: GET
Type: HTTP_PROXY
Uri: http://bucket.mybucket.co.s3.eu-west-1.amazonaws.com/{proxy} # the URL you want to set a proxy to
IntegrationResponses:
- StatusCode: 200
AuthorizationType: NONE

Resources