Lambda#edge not triggered by an origin-request event - amazon-cloudfront

Question

I'm trying to configure Lambda#edge functions using CloudFormation. After deploying the template everything looks find in the console, however the lambda functions listening to origin-request events are not being triggered.
Strange enough, a viewer-request event does manage to trigger a function.
What am I doing wrong? How can I make the origin-request event to work?
Here's my template:
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: Deployment of Lambda#edge functions
Parameters:
Stage:
Type: String
AllowedValues:
- staging
- production
Default: staging
Description: Stage that can be added to resource names
CodeBucket:
Type: String
Description: The S3 Bucket name for latest code upload
CodeKey:
Type: String
Description: The S3 Key for latest code upload
# Mappings:
# AliasMap:
# staging:
# Alias: "staging-app.achrafsouk.com"
# production:
# Alias: "app.achrafsouk.com"
Resources:
CFDistribution:
Type: AWS::CloudFront::Distribution
Properties:
DistributionConfig:
Enabled: true
Logging:
Bucket: XXX.s3.amazonaws.com
IncludeCookies: false
Prefix: !Sub ${Stage}
PriceClass: PriceClass_100
Comment:
!Sub "Lambda#Edge - ${Stage}"
# Aliases:
# - !FindInMap [AliasMap,
# Ref: Stage, Alias]
Origins:
- CustomOriginConfig:
OriginProtocolPolicy: https-only
DomainName: !Sub "${Stage}.defaultOrigin.example.com"
Id: defaultOrigin
OriginCustomHeaders:
- HeaderName: X-Node-Env
HeaderValue: !Sub "${Stage}"
- CustomOriginConfig:
OriginProtocolPolicy: https-only
DomainName: !Sub "${Stage}.another.example.com"
Id: rateLimitsOrigin
OriginCustomHeaders:
- HeaderName: X-Node-Env
HeaderValue: !Sub "${Stage}"
DefaultCacheBehavior:
ForwardedValues:
Headers:
- CloudFront-Viewer-Country
- X-Request-Host
- User-Agent
QueryString: true
ViewerProtocolPolicy: https-only
TargetOriginId: defaultOrigin
DefaultTTL: 0
MaxTTL: 0
LambdaFunctionAssociations:
# - EventType: viewer-request
# LambdaFunctionARN:
# Ref: ViewerRequestLambdaFunction.Version
- EventType: origin-request
LambdaFunctionARN:
Ref: OriginRequestLambdaFunction.Version
CacheBehaviors:
- ForwardedValues:
Headers:
- CloudFront-Viewer-Country
- X-Request-Host
- User-Agent
QueryString: true
ViewerProtocolPolicy: https-only
DefaultTTL: 60
MaxTTL: 60
TargetOriginId: rateLimitsOrigin
PathPattern: "/rate-limit*"
LambdaFunctionAssociations:
- EventType: origin-request
LambdaFunctionARN:
Ref: GetRateLimitLambdaFunction.Version
# ViewerRequestLambdaFunction:
# Type: AWS::Serverless::Function
# Properties:
# CodeUri:
# Bucket: !Sub ${CodeBucket}
# Key: !Sub ${CodeKey}
# Role: !GetAtt LambdaEdgeFunctionRole.Arn
# Runtime: nodejs10.x
# Handler: src/functions/viewerRequest.handler
# Timeout: 5
# AutoPublishAlias: live
OriginRequestLambdaFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri:
Bucket: !Sub ${CodeBucket}
Key: !Sub ${CodeKey}
Role: !GetAtt LambdaEdgeFunctionRole.Arn
Runtime: nodejs10.x
Handler: src/functions/originRequest.handler
Timeout: 5
AutoPublishAlias: live
GetRateLimitLambdaFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri:
Bucket: !Sub ${CodeBucket}
Key: !Sub ${CodeKey}
Role: !GetAtt LambdaEdgeFunctionRole.Arn
Runtime: nodejs10.x
Handler: src/functions/getRateLimit.handler
Timeout: 5
AutoPublishAlias: live
LambdaEdgeFunctionRole:
Type: "AWS::IAM::Role"
Properties:
Path: "/"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
AssumeRolePolicyDocument:
Version: "2012-10-17"
Statement:
- Sid: "AllowLambdaServiceToAssumeRole"
Effect: "Allow"
Action:
- "sts:AssumeRole"
Principal:
Service:
- "lambda.amazonaws.com"
- "edgelambda.amazonaws.com"
Outputs:
# ViewerRequestLambdaFunctionVersion:
# Value: !Ref ViewerRequestLambdaFunction.Version
OriginRequestLambdaFunctionVersion:
Value: !Ref OriginRequestLambdaFunction.Version
GetRateLimitLambdaFunctionVersion:
Value: !Ref GetRateLimitLambdaFunction.Version
CFDistribution:
Description: Cloudfront Distribution Domain Name
Value: !GetAtt CFDistribution.DomainName

Answer 1

So, after painful hours of trial and error the solution was staring me in the face:
CloudFront distribution was emitting and error saying it could not connect to the origin, and it was right - because I've specified dummy domains in the origins definition of the CloudFront distribution.
My logic was that if my lambda#edge functions intercept the requests before reaching the origin, then it shouldn't matter if the origin's domains is real or not.
As it turns out, it's partially true. Because it seams that:
viewer-request event is triggered before trying to connect to the origin, and there for it can by fake.
origin-request event is triggered only after trying to connect to the origin, so you'd need to specify something real.
Hope my wasited hours will contribute to someone else exploring the same path of reasoning.

Answer 2

In case it helps someone in future with this problem - I noticed that origin-response (which I accidentally selected instead of origin-request) events were (mostly) working for me, but there were still a large number of OriginConnectError in the logs, and the "Percentage of Viewer Requests by Result Type" report was showing a high error rate.
It turns out it was only working at all because I had my origin connection set to https-only, but my origin (an empty s3 bucket) only accepted http connections - and for some reason the events were still being triggered.
Changing the origin connection to the correct http-only completely stopped things working for origin-response, until I also corrected the event type to origin-request
So, in general, check the OriginProtocolPolicy is http-only if you're using an s3 bucket origin. If it's wrong, things might still appear to work sometimes, but not always, and you'll end up with high error rates.