As in the title, I have uploaded an image to ECR and I would like to deploy this to EKS, is this possible using Terraform?
I have already done the following with Terraform
Launched running EKS Cluster
Pushed my image to ECR
But I cannot seem to find an example of changing EKS to use the ECR image.
Related
I have a spark application and want to deploy this on a Kubernetes cluster.
Following the below documentation I have managed to create an empty Kubernetes cluster, generated docker image using the Dockerfile provided under kubernetes/dockerfiles/spark/Dockerfile and deployed this on the cluster using spark-submit in a Dev environment.
https://spark.apache.org/docs/latest/running-on-kubernetes.html
However, in a 'proper' environment we have a managed Kubernetes cluster (bespoke unlike EKS etc.) and will have to provide pod configuration files to get deployed.
I believe you can supply Pod template file as an argument to the spark-submit command.
https://spark.apache.org/docs/latest/running-on-kubernetes.html#pod-template
How can I do this without spark-submit? And are there any example yaml files?
PS: we have limited access to this cluster, e.g. we can install Helm charts but not operator or controller.
You could try to use k8s Spark CRD https://github.com/GoogleCloudPlatform/spark-on-k8s-operator and provide a pod configuration through it.
I am trying to add the tags to my existing VPC.
I am trying to solve this problem using AWS cli for multiple VPC's.
But could not find the appropriate command to add the tag using AWSCLI.
Can someone please help. TIA.
You can add tags to the existing resources using create-tags
Adds or overwrites only the specified tags for the specified Amazon EC2 resource or resources. When you specify an existing tag key, the value is overwritten with the new value.
https://docs.aws.amazon.com/cli/latest/reference/ec2/create-tags.html
aws ec2 create-tags --resources vpc-1234567 --tags Key=Stack,Value=production
Or multiple tags
aws ec2 create-tags \
--resources vpc-1234567 i-1234567890abcdef0 \
--tags Key=webserver,Value= Key=stack,Value=Production
Make sure your default region is correct for VPC or add region to the cli command.
I have few ec2 machines running created from aws console manually. They are not under terrafrom.
If i want to use terrafrom now to create new VPC and ec2 machine does it will delete my old machines ?
No, Terraform will not delete old machines in aws (created by aws console). That is because whenever you run terraform script to create something, it will create state file which acts as reference copy for terraform. In your case if you create any resources (like ec2 VMs) in AWS, you will end up having both machines (the one created by manually and second set created by terraform)
Read here more about terraform
https://learn.hashicorp.com/tutorials/terraform/infrastructure-as-code?in=terraform/aws-get-started
i'd like use the aws cli to list images available here: https://gallery.ecr.aws/lambda/nodejs
when i try the following command:
aws ecr-public describe-image-tags --repository-name lambda/nodejs
i get this error:
User: arn:aws:sts::<my-session> is not authorized to perform: \
ecr-public:DescribeImageTags on resource: arn:aws:ecr-public::<my-account>:repository/lambda/nodejs
how do i specify the aws public repository, and not one of my own?
the api docs mention a --registry-id option, to supply an account ID, or if left blank "..the default public registry is assumed", but it seems like my account info is still being inserted.
Q: can someone provide a working example of a command that lists each of (or the first pagination of) the tags in the Image tags panel of a repo in the Amazon ECR Public Gallery?
At time of writing, this can only be found by visiting the ECR Public gallery. There is no way to programmatically get a list of tags in ECR Public for a repository you haven't been explicitly granted API access to. Here's the relevant roadmap item https://github.com/aws/containers-roadmap/issues/1262
Actually AWS CLI supports the Public Repositories you created, which are linked with your account. Like I created a repository named randomname and I was able to list images under that.
$ aws ecr-public describe-images --repository-name randomname
{
"imageDetails": [...]
}
Amazon ECR public registries
You can use your public registry to manage public image repositories consisting of Docker and Open Container Initiative (OCI) images. Each AWS account is provided with a default public and private Amazon ECR registry
To be more specific this part you see in the AWS Console
For future reference, the tags listing API is now available in ECR Public: https://github.com/aws/containers-roadmap/issues/1262
I'm looking to set up a local Docker instance of AWS Secrets Manager.
I've been scouring the web for an image or anything of the sort that I can use. I can only find documentation for AWS ECS secrets management.
Does anyone have any experience with setting up AWS Secrets Manager for local testing through Docker? Thanks!
Good question!
You could run localstack [1] inside a docker container. It mocks some of the AWS services for testing purposes. AWS Secrets Manager is supported at http://localhost:4584 by default.
There are some useful blog posts covering localstack. [2][3]
However, I could not find any blog post covering AWS Secrets Manager on localstack. I guess you have to try it out yourself.
References
[1] https://github.com/localstack/localstack
[2] https://medium.com/#andyalky/developing-aws-apps-locally-with-localstack-7f3d64663ce4
[3] https://medium.com/pareture/localstack-for-local-aws-dev-22775e483e3d
You can setup local AWS SecretManager inside LocalStack using the following command:
aws --endpoint-url=http://localhost:4566 secretsmanager create-secret --name my_secret --secret-string '[{"my_uname":"username","my_pwd":"password"}]'
Output:
{
"ARN": "arn:aws:secretsmanager:us-east-1:000000000000:secret:my_secret-denusf",
"Name": "my_secret",
"VersionId": "e168cdf1-5c94-493d-bafd-791779a7515d"
}