Was not able to find any security groups for AWS Lambda.
Is there a way to allow access from AWS Lambda to RDS without alowing all IPs (0.0.0.0/0) and without allowing all Amazon IP Range?
As #user5919440 suggests, now that this new feature is out:
https://aws.amazon.com/blogs/aws/new-access-resources-in-a-vpc-from-your-lambda-functions/
...you simply need to tell AWS Lambda which VPC subnets to bind to your function. The function then can communicate with any AWS service that also has access to that subnet.
This means that you should be able to add a security group in your RDS that allows traffic from the same internal subnet (10.x.x.x) that your Lambda function is bound to.
This feature is out as of yesterday
https://aws.amazon.com/blogs/aws/new-access-resources-in-a-vpc-from-your-lambda-functions/
There currently isn't, and a moment's reflection suggests that if there were, if would be a false sense of security -- the traffic wouldn't be assured to be from your Lambda functions... but from anybody's -- the IP addresses are pooled.
There have been hints of a future mechanism to allow a cleaner trust relationship between Lambda and VPC, perhaps implemented with the VPC Endpoint feature (currently available only with S3), or perhaps differently... no details have been forthcoming, so far.
Related
My lambda is getting ETIMEDOUT for hostname s3.amazonaws.com. I figured out that is is probably caused by the fact that my lambda is inside a VPC.
I suspect that I need to use an AWS Endpoint to reach S3. I know that we have endpoint set up and in AWS Console I can see and endpoint associated with this VPC called dev-s3-gateway with service name com.amazonaws.us-east-1.s3
How do I tell my lambda to use this endpoint?
Inside a VPC, your Lambda can be in either a private subnet or a public subnet.
Lambda in Public Subnets CANNOT access the internet.
Having Lambda access the internet
Being in a Private Subnet, your Lambda cannot have "direct" access to the internet. You need to provision a Public Subnet in your VPC and have both an Internet Gateway as well as a NAT Gateway in that subnet.
Route the traffic to 0.0.0.0/0 from your Lambda's subnet via the NAT Gateway.
From the routing table of the Public Subnet, route traffic to 0.0.0.0/0 via the Internet Gateway.
Your Lambda should now have access to the internet (and to S3).
Feel free to also check out this more detailed guide.
Accessing AWS services through VPC endpoints
Several AWS services offer VPC endpoints. These allow you to connect and interact with the respective services without your traffic ever leaving the AWS network.
For more information on them, please check out their documentation.
EDIT: Expanding a bit on your specific S3 use case.
S3 offers Gateway and Interface VPC Endpoints. Based on the endpoint name you provided in the question, I'm going to guess this is a Gateway VPC Endpoint. Once you set up the endpoint in your VPC, the Security Group(s) associated with your Lambda must allow outbound traffic to the endpoint.
You have two options.
First and simplest (but perhaps less secure - depending on your use case), you allow outbound traffic to 0.0.0.0/0. This will effectively allow you to call anything. However, if the Lambda is in a public subnet, it won't be able to access the internet, as explained above, but rather only the Gateway VPC Endpoint ranges.
The second option is to allow outbound traffic to the known Prefix List of the regional Amazon S3 endpoint. You can retrieve the PrefixList ID (looks like pl-xxxxxx) by invoking the DescribePrefixLists and looking for that Prefix List for service com.amazonaws.us-east-1.s3. Once you have the ID of the Prefix List, you can add it to the destination of an outbound rule of your Security Group(s).
The https://docs.aws.amazon.com/general/latest/gr/ssm.html website lists all of the SSM endpoints that need to be whitelisted over port 443/tcp (https) in order for AWS SSM agents to communicate; however, after about 2-3 weeks of troubleshooting, I just came to the conclusion that AWS SSM does not actually communicate via FQDN, which causes problems for proxies that only whitelist FQDNs and not IP addresses.
Since the IP addresses of thoes FQDNs don't point back to those FQDNs, this leaves the AWS SSM tool broken in the particular environment I'm using it.
Is there a way to either "force" AWS SSM to use FQDNs when communicating outbound?
AWS does not use or allow FQDNs in security groups or routing unless you are remaining on their network (AWS service to AWS service communication). You must use IPv4 or IPv6 addresses in all other cases.
This AWS article is good, but I have found that the SSM Messages and EC2 Messages endpoints are also required, which is addressed here.
Basically, you need to create 3 VPC endpoints (SSM, SSM Messages, and EC2 Messages) and route the 443 traffic to them while allowing the traffic on the affected security groups, route tables and NACLs. This has the advantage of not having the traffic leave AWS' networks and the traffic will not be affected by your firewalls/proxies in most architectures. Beware that the AWS documentation is not good and conflicts with itself, as shown in the 2 articles linked here.
i want to send mail using node mailer in NodeJS if my lambda function is developed in default VPC because I have to access RDS too from the lambda function.
I am unable to send success mail for data successfully inserted in RDS if I deployed my lambda function in default VPC WHAT changes I need to do so I can send.
IF I choose NO vpc then I am unable to set data to database.
From https://docs.aws.amazon.com/lambda/latest/dg/configuration-vpc.html,
When you connect a function to a VPC in your account, it does not have access to the internet unless your VPC provides access.
I take this to mean that if you wish to access both RDS and the internet from lambda from within your VPC, you need a NAT gatway (or to spin up your own instance). In other words, lambda does not support internet access with a public IP through an Internet Gateway, which is the mechanism of internet access within your vpc.
If you don't mind the cost, about 4.5 cents an hour plus data transfer last I checked, the simplest solution is probably:
add another subnet to your VPC.
Add a NAT Gateway to your VPC.
Add a route table to the subnet that routes through the NAT Gateway
put your lambda in that subnet
This essentially creates a connection to the internet in that VPC without your lambda holding a Public IP address.
Even when I create EC2 instances in a private subnet, they must be able to send traffic to the Internet if I want to register them to a ECS cluster.
I am using a NAT gateway to do this, but I still feel insecure that the instances can send private information to anywhere in case of takeover.
What would be the most compact CIDR range that I can use for the instances' security group, instead of 0.0.0.0/0?
For the moment, you may have to rely on the list of public IP address ranges for AWS, allowing traffic bound for all the CIDR blocks associated with your region.
Part of design for resiliency of much of what AWS does relies on the ability of their service endpoints not to depend on static address assigments and instead to use DNS... but their service endpoints should always be on addresses associated with your region, since very few services violate their practice strict regional separation of service infrastructure.
(CloudFront, Route 53, and IAM do, maybe others, but these are provisioning endpoints, not operational ones. These provisioning-only endpoints do not need to be accessible for most applications to function normally.)
A super common pattern here is to create a proxy in a narrow CIDR and only allow outbound traffic to flow through the proxy. White-list AWS endpoints and log all traffic flowing through the proxy for monitoring/auditing.
AWS Endpoints = https://docs.aws.amazon.com/general/latest/gr/rande.html
The AWS docs are almost useless when trying to describe an entire system. Is there any resource or compiled list of all the resources that can belong to a security group and the different types of security groups?
Here is what I have so far:
EC2-Classic instance
EC2-VPC instance
RDS
ElasticCache
Anything else I'm missing? Any really good doc resource I'm missing?
The main concept to understand about an AWS Security Group is that it determines what traffic is permitted in/out of a resource on a virtual network.
Therefore, think about what can be launched "into" a virtual network:
Amazon EC2 instances
Services that launch EC2 instances:
AWS Elastic Beanstalk
Amazon Elastic MapReduce
Services that use EC2 instances (without appearing directly in the EC2 service):
Amazon RDS (Relational Database Service)
Amazon Redshift
Amazon ElastiCache
Amazon CloudSearch
Elastic Load Balancing
Lambda
Resources do not "belong" to a security group. Rather, one or more Security Groups are associated to a resource. This is often a difficult concept to understand since Security Groups have similar abilities to firewalls, and firewalls generally "encase" a number of devices. Rather than "belonging to", or "being encased by", a security group, the virtual network simply uses the definitions contained within a security group to determine what traffic to permit in/out of the resource.
For example, imagine two EC2 instances that are associated with a "Web" security group and the security group is configured to permit incoming traffic on port 80. While both instances are associated to the same security group, they cannot communicate with each other. This is because they do not "belong" to the security group, and are not "within" the security group. Rather, the security group definition is used to filter traffic in/out of the instances. The security group can, of course, be configured to permit incoming traffic from the security group itself (a self-reference), which really means that incoming traffic is permitted from any resource that is, itself, associated with the security group. (See, I told you that it's a difficult concept grasp!)
Also, a security group is not actually associated with an EC2 instance within a VPC. Rather, the security group is associated with the Elastic Network Interface (ENI) that is attached to an EC2 instance. Think of the ENI as a "network card" that links an instance to a VPC subnet. An instance can have multiple ENIs and can therefore connect to multiple subnets. Each ENI can have its own association with security groups. Thus, the actual security groups being used depends upon where the traffic is flow in/out of the instance, rather than actually being associated with the instance.
There are only two "types" of security groups:
EC2 Classic (the legacy network configuration)
EC2 VPC (the modern private network configuration)
Either type of security group can be associated with any other resource, as long as they are in the same network type (classic or VPC).
A Lambda Function can also be associated with a Security Group. That might not have been the case in 2015, when the original answer was written.
Fargate tasks can also be assigned to security groups.
AWS EFS (Elastic File System) needs a security group attached.
From the AWS document:
To connect your Amazon EFS file system to your Amazon EC2 instance,
you must create two security groups: one for your Amazon EC2 instance
and another for your Amazon EFS mount target.
Reference: https://docs.aws.amazon.com/efs/latest/ug/accessing-fs-create-security-groups.html
interface endpoints can also be associated with security groups, this is a good question and so far not easy to find on AWS documentation.