I want to create copy of my instance programmatically using javascript, and I also want to mount my S3 bucket to the newly created instance.
Is there a way to do "Launch more like this" using javascript.
Things I tried:
Created an AMI
Using that I created an instance.
But it is not copying the contents of Original Instance into the newly created Instance. And also it is not mounting the S3 bucket.
Launch More Like This is an AWS Console UI functionality that copies over all settings of the current instance like AMI, Storage, Security Groups, AZs, Subnets etc, but still gives you an opportunity to make modifications before launching. This can be easily reproduced by coping over the the output of DescribeInstances API and applying them to the RunInstances API.
It does not copy over the contents/data of the existing machine. If you need to copy over the contents, create an AMI of the existing instance and then launch the new EC2 instance using the new AMI.
To attach an S3 Bucket as a volume to your EC2 instance, you can use S3FS/Fuse You may want to install this as part of your AMI, so you don't need to install it each time you launch your instance. You can run the mount scripts as part of the init scripts, where you can specify or configure the S3 bucket to be mounted.
Hope this helps.
Related
I am building an application where the user can upload images, for this I am using S3 as files storage.
In other area of the application there is some process deployed on EC2 that need to use the uploaded images.
This process need the images multiple times (it generate some report with it) and it's in part of multiple EC2 - using elastic beanstalk.
The process doesn't need all the images at once, but need some subset of it every job it gets (depend the parameters it gets).
Every ec2 instance is doing an independent job - they are not sharing file between them but they might need the same uploaded images.
What I am doing now is to download all the images from s3 to the EC2 machine because it's need the files locally.
I have read that EFS can be mounted to an EC2 and then I can access it like it was a local storage.
I did not found any example of uploading directly to EFS with nodejs (or other lang) but I found a way to transfer file from S3 to EFS - "DataSync".
https://docs.aws.amazon.com/efs/latest/ug/transfer-data-to-efs.html
So I have 3 questions about it:
It is true that I can't upload directly to EFS from my application? (nodesjs + express)
After I move files to EFS, will I able to use it exactly like it in the local storage of the ec2?
Is it a good idea to move file from s3 to efs all the time or there is other solution to the problem I described?
For this exact situation, we use https://github.com/kahing/goofys
It's very reliable, and additionally, offers the ability to mount S3 buckets as folders on any device - windows, mac as well as of course Linux.
Works outside of the AWS cloud 'boundary' too - great for developer laptops.
Downside is that it does /not/ work in a Lambda context, but you can't have everything!
Trigger a Lambda to call an ECS task when the file is uploaded to s3. The ECS task starts and mounts the EFS volume and copies the file from s3 to the EFS.
This wont run into problems with really large files with Lambda getting timed out.
I dont have the code but would be interested if someone already has this solution coded.
I have an AWS Windows Server 2016 VM. This VM has a bunch of libraries/software installed (dependencies).
I'd like to, using python3, launch and deploy multiple clones of this instance. I want to do this so that I can use them almost like batch compute nodes in Azure.
I am not very familiar with AWS, but I did find this tutorial.
Unfortunately, it shows how to launch an instance from the store, not an existing configured one.
How would I do what I want to achieve? Should I create an AMI from my configured VM and then just launch that?
Any up-to-date links and/or advice would be appreciated.
Yes, you can create an AMI from the running instance, then launch N instances from that AMI. You can do both using the AWS console or you could call boto3 create_image() and run_instances(). Alternatively, look at Packer for creating AMIs.
You don't strictly need to create an AMI. You could simply the bootstrap each instance as it launches via a user data script or some form of CM like Ansible.
I am trying to launch my own AMI using user-data so that it can run a script and then terminate.
So I launched an Ec2 Windows Base and configure it to have all the tools I need (NodeJS etc) and saved my script to C:\Projects\index.js.
I then saved it as an Image.
So I then used the console to launch an EC2 from my new AMI with the user-data of
node C:\Projects\index.js --uuid=1
</powershell>
If I run that command having RDP into the EC2 it works, so it seems that the userdata did not run when the Image was started.
Having read some of the other questions and answers it could be because the AMI created was made from an Instance that started already. So the userdata did not persist.
Can anyone advise me on how I can launch my AMI with a custom userdata each time? (as the UUID will change)
Thanks
Another solution that worked for me is to run Sysprep with EC2Launch.
The issue is that AWS doesn't reestablish the route to the profile service (169.254.169.254) in your custom AMI. See response by SanjitPatel in this post. So when I tried to use my custom AMI to create spot requests, my new instances were failing to find user data.
Shutting down with Sysprep, essentially forces AWS re-do all setup work on the instance, as if it were run for the first time. So when you create your instance, shut it down with Sysprep and then create your custom AMI, AWS will setup the profile service route correctly for the new instances and execute your user data. This also avoids manually changing Windows Tasks and executing user data on subsequent boots, as persist tag does.
Here is a quick step-by-step:
1.Create an instance using one of the AWS Windows AMIs (Windows Server 2016 Nano Server doesn't support Sysprep) and passing your desired user data (this may be optional, but good to make sure AWS wires setup scripts correctly to handle user data).
2.Customize your instance as needed.
3.Shut down your instance with Sysprep. Just open EC2LaunchSettings application and click "Shutdown with Sysprep".
4.Create your custom AMI from the instance you just shut down.
5.Use your custom AMI to create other instances, passing user data on instance creation. User data will be executed on instance launch. In my case, I used Spot Request screen, which had a User Data text box.
Hope this helps!
We have a large RDS instance in AWS (100GB). We're creating a script which will clear out old data in the database at regular intervals. In order to test this fully we'd like to setup a copy of the existing production RDS instance which we can use to test the script on (so we don't lose data from the production instance).
Is there a way to create a standalone duplicate RDS instance based on another instance? I had thought I could do this by using a snapshot, but it appears you can only restore an instance from a snapshot.
With a snapshot you can restore to a different/new instance name. E.g. create a snapshot of database-one then restore the snapshot to a different instance name database-one-copy.
I use this method programmatically to create a development database on a nightly schedule or as needed using the python AWS SDK Boto3 and this method https://github.com/airsciences/aws-rds-persist.
I have a customer who wishes me to do some customisations of the erp system opentaps, which they used via opentaps Amazon Elastic Computing Cloud (EC2) images, I've only worked with it on a normal server and don't know anything about images in the cloud. When I ssh in with the details the client gave me there is no sign of the erp installation directory I'd expect to see. I did originally expect that the image wouldn't be accessible, but the client assured me it was. I suppose they could be confused.
Would one have to create a new image and swap it out or is there a way to alter the source and rebuild like on a normal server?
Something is not quite clear to me here. First of all EC2 images running in the cloud are just like normal virtual servers, so If you have an access to the running instance there is no difference between instance in the cloud and instance on another pc in your home for example.
You have to find out how opentaps are installed on the provided amis, then do your modifications, create an image from the modified instance and save it to s3 for backup if necessary.
If you want to start with fresh instance, you can start up any linux/windows distro on the EC2, install opentaps yourself your way and you are done.