So you have a running EC2 instance. It works great, except it’s one of the ephemeral, kill-it-and-you-lose-everything kind. An EBS-backed instance is the logical choice, so how do you convert it? Easy:
Step 1: Create the EBS volume
Just do it in the web interface. You could use the command-line tools, but why? While you’re there, attach it to your running EC2 instance, making note of the volume-id and device it’s connected to, eg: vol-abcd1234 and /dev/sdf
While you’re in the web interface, make a note of the ramdisk and kernel your running instance is using. This will be important later. They’ll be something like “ari-12345678” and “aki-abcdef12”, respectively.
Step 2: Sync your running instance
If you have things like mysql running, shut them down. It’ll save you hassles later. Then create a FS on your EBS volume:
# mkfs.ext3 /dev/sdf
Next, mount it:
# mkdir /mnt/ebs
# mount /dev/sdf /mnt/ebs
Now, use rsync to copy everything over to the EBS volume:
# rsync -a –delete –progress -x / /mnt/ebs
You won’t have /dev/sda2 for your /mnt partition on EBS, so you need to remove it from the copied fstab in /mnt/ebs/etc/fstab. Comment it out, remove it, whatever.
(Added 11/2010 thanks to Mark Smithson in the comments)
You may need to create some device files on the new EBS volume. If console, zero or null don’t exist in /mnt/ebs/dev, create them using some or all of these:
# MAKEDEV -d /mnt/ebs/dev -x console
# MAKEDEV -d /mnt/ebs/dev -x zero
# MAKEDEV -d /mnt/ebs/dev -x null
Unmount the EBS volume:
# umount /mnt/ebs
Step 3: Get your keys in order
You’ll need an EC2 X.509 cert and private key. You get these through the web interface’s “Security Credentials” area. This is NOT the private key you use to SSH into an instance. You can have as many as you want, just keep track of the private key because Amazon doesn’t keep it for you. If you lose it, it’s gone for good. Once you have the files, set some environment variables to make it easy:
# export EC2_CERT=`pwd`/cert-*.pem
# export EC2_PRIVATE_KEY=`pwd`/pk-*.pem
Step 4: Make your AMI
Now you can make a snapshot of your EBS volume. This is the basis of the AMI you’ll be creating. Whatever you copied to the EBS volume in step 2 will be there — user accounts, database data, etc. First, the snapshot (using the volume-id from step 1):
# ec2-create-snapshot vol-abcd1234
That’ll give you a snapshot-id back. You then need to wait for the snapshot to finish. Keep running this until it says it’s “completed”:
# ec2-describe-snapshots snap-1234abcd
Finally, you can register the snapshot as an AMI:
# ec2-register –snapshot snap-1234abcd –description “your description here” –name “something-significant-here” –ramdisk ari-12345678 –kernel aki-abcdef12
(The arguments to ec2-register should be normal Unix-style long options: “-“, “-“, “snapshot”; “-“, “-“, “kernel”. WordPress seems to be displaying those as an mdash instead. It needs to be a double-dash.)
Step 5: Launch!
At this point, you should see your EBS volume, the snapshot, and your AMI in their respective areas of the web interface. Launch an instance from the AMI and you’ll find it pretty much exactly where you left your original instance.
May 28, 2010 at 3:34 pm
Fantastic walkthrough – very intuitive… Thanks for the time you put into this!
June 13, 2010 at 2:13 am
I have successfully converted my running instance store AMI into an EC2 EBS AMI. However, after creating an instance using the newly created EBS AMI, I am getting a timeout error when sshing into the instance. Any suggestion on what I could have done wrong during the conversion, given the fact that ssh was working fine on the old instance store AMI. How can I fix this issue
July 1, 2010 at 12:05 pm
I had a similar issue.
Needed to create the console, zero and null devices in the EBS Snapshot.
adding these commands before umount and snapshot worked for me.
MAKEDEV -d /mnt/ebs/dev -x console
MAKEDEV -d /mnt/ebs/dev -x zero
MAKEDEV -d /mnt/ebs/dev -x null
November 24, 2010 at 5:50 pm
thank you so much for this comment. I had the same problem.
December 4, 2010 at 7:46 pm
FYI, for Debian-based machines you need to put something like:
mknod /mnt/ebs/dev/console c 5 1
mknod /mnt/ebs/dev/tty c 5 0
mknod /mnt/ebs/dev/zero c 1 5
mknod /mnt/ebs/dev/null c 1 3
August 10, 2010 at 9:45 am
Just a note: if you want to register an 64bit AMI, you have to specify
–architecture x86_64
in your ec2-register statement. Otherwise, it will register as the default i386. Likewise, you may be able to ommit kernel id and ramdisk and your AMI might run fine, but omitting the parameters might also screw your AMI.
BTW, thanks for the great post. Its the most concise explanation for this topic available.
September 22, 2010 at 3:03 am
THis is inaccurate. You can’t create an AMI from a snapshot. I wasn’t able to follow this. You can register an ami from an existing manifest but not create an ami just from a snapshot
September 23, 2010 at 1:59 pm
Well, in fact you CAN create an AMI from a snapshot. ec2-register works the way described above.
Maybe you screwed something up between EBS backed AMIs and S3-backed AMIs?
May 10, 2012 at 11:40 pm
I followed these instructions as well and ended up with an unbootable AMI.
[ 21.712803] dracut Warning: No root device “block:/dev/disk/by-label/\x2f” found
May 11, 2012 at 12:38 am
According to the ec2-register doco, when registering an EBS backed AMI, you now have to include –root-device-name /dev/sda1 in the ec2-register command.
September 28, 2010 at 6:19 pm
Great article.
Why can’t every documentation be as good as this?
@Mark Smithson & Steffen Müller : Thanks. Both your tips helped me as well.
Tks
Ash RS
December 17, 2010 at 8:26 pm
Thanks for the succinct instructions!
Two updates:
* on some AKI’s you need to “tune2fs -L ‘/’ /dev/sdf” at the end of step 3 in order to get it to boot.
* newer kernels (like in the Amazon Linux AMI) use the Xen names once it’s booted (xvdf instead of sdf).
March 2, 2011 at 11:05 am
What changes did you have to do in order to get this working with Amazon’s Linux AMIs?
February 10, 2011 at 11:10 am
Thank you very much for publishing a very helpful guide. I could successfully create my EBS AMI using instance store ami. I can log in to the instance that I created but when I rebooted the machine It wont come up. Gives me
“ssh: connect to host ec2-184-73-22-220.compute-1.amazonaws.com port 22: Connection timed out”
I tried creating the /dev/ above too but no luck.
I commented out the irrelevant partition from the fstab.
By the way image I worked on is redhat5.3 64bit. Please advice gentleman.
February 10, 2011 at 11:12 am
Thank you very much for publishing a very helpful guide. I could successfully create my EBS AMI using instance store ami. I can log in to the instance that I created but when I rebooted the machine It wont come up. Gives me
“ssh: connect to host ec2-184-73-22-220.compute-1.amazonaws.com port 22: Connection timed out”
I tried creating the /dev/ above too but no luck.
I commented out the irrelevant partition from the fstab.
By the way image I worked on is redhat5.3 64bit. Please advice gentleman.
May 13, 2011 at 1:15 pm
First of all, like to thank the author for creating this excellent guide.
I had the same problem using Alestic built ubuntu lucid ami. I checked the console via Firefox ElasticFox and realized that there is a boot up problem. The error message was: “Give up waiting for root device”. I followed the instruction by other posts on the web to change /boot/grub/menu.lst. For all references to root=LABEL=uec-rootfs, change to root=/dev/sda1.
I now have no problem login to my EBS ami.
May 29, 2011 at 9:53 am
That is not necessary, since device names can changes with time (now I have xvd* instead of sd*, for example).
For LABEL=uec-rootfs syntax to work, you need to add label to your newly created FS (after you performed mkfs):
sudo tune2fs -L uec-rootfs /dev/sdf
Note that label is the same as of your original volume. Otherwise boot loader does not see the volume and does no mount it as root.
February 12, 2011 at 1:41 pm
For Ubuntu ec2-register command did not work for me. But finally I found the problem. It was because of ec2-api-tool new version. Please follow the link:
https://help.ubuntu.com/community/EC2APITools
March 6, 2011 at 2:06 pm
Thank you! This post so much simplified the process.
I’ve switched two ephemeral instances to EBS and made 4 EBS base AMIs in the last 24 hours.
One thing though. When I cut and paste commands, the places that should be — (e.g., –kernel) come across as -kernel, and the ec2 don’t complain, but don’t work right until you redo it with –.
March 6, 2011 at 2:27 pm
Yeah…Wordpress seems to have started substituting an mdash for the double-minus. I’ll make a note in the post. Thanks!
April 29, 2011 at 10:16 pm
To get your EC2 X.509 cert and private key, you will need to create a new pair. The default cert will not provide you a private key.
May 9, 2011 at 7:35 am
Pay attention to this line:
“If you have things like mysql running, shut them down. It’ll save you hassles later.”
I didn’t. Mysql acted really weird, I kept being disconnected randomly and had to re-install mysql-server from scratch and then re-import all the data.
June 9, 2011 at 6:49 pm
Hi, this is great guide.
I was just wondering would the new instance still be backed by the ebs volume or would I have to repeat these steps?
Thanks in Advance KW
June 10, 2011 at 5:50 am
The new image when launch will be EBS backed. If you have Ubuntu instances to convert, this will help also: http://www.capsunlock.net/2009/12/create-ebs-boot-ami.html
June 13, 2011 at 5:08 am
I’ve went through this many times and believe what is most problematic is the ec2-register part. The syntax that at least gets me a booting instance is this:
ec2-register -s snap-6cf87202 -name “MoreMyFun3″ -kernel aki-407d9529 –root-device-name /dev/sda1 -b”/dev/sda=snap-6cf87202”
But the (“Get System Log”) results are:
…
Creating root device.
Mounting root filesystem.
mount: could not find filesystem ‘/dev/root’
Setting up other filesystems.
setuproot: moving /dev failed: No such file or directory
setuproot: error mounting /proc: No such file or directory
setuproot: error mounting /sys: No such file or directory
Mount failed for selinuxfs on /selinux: No such file or directory
Switching to new root and running init.
switchroot: mount failed: No such file or directory
Booting has failed.
June 13, 2011 at 11:57 pm
Just getting back to this… I was reviewing the EC2 User Guide (API Version 2011-05-15) and found the section on “Creating Amazon EBS-Backed AMIs” which points to “ec2-create-image” –which is so much easier than going through the method above. What is the difference between the instructions posted above and using ec2-create-image?
Cheers,
–kkruzich
June 14, 2011 at 1:13 pm
It’s very possible Amazon has created a more streamlined process for doing what I document here. This was the state-of-the-art 18 months ago, but a lot can change in that time.
June 20, 2011 at 9:25 am
I have selected default ramdisk and kernel. Any ideas how to find there ids?
June 20, 2011 at 2:25 pm
I am unable to complete the last instruction of step 4. Whatever I do, I get the same error message
SYNOPSIS
ec2reg (ec2-register)
ec2reg [GENERAL OPTIONS] MANIFEST
GENERAL NOTES
Any command option/parameter may be passed a value of ‘-‘ to indicate
that values for that option should be read from stdin.
DESCRIPTION
Register an AMI manifest for use with EC2.
The MANIFEST parameter is the manifest (in S3) to register.
e.g. mybucketname/image.manifest
GENERAL OPTIONS
-K, –private-key KEY
Specify KEY as the private key to use. Defaults to the value of the
EC2_PRIVATE_KEY environment variable (if set). Overrides the default.
-C, –cert CERT
Specify CERT as the X509 certificate to use. Defaults to the value
of the EC2_CERT environment variable (if set). Overrides the default.
-U, –url URL
Specify URL as the web service URL to use. Defaults to the value of
‘https://ec2.amazonaws.com’ or to that of the EC2_URL environment
variable (if set). Overrides the default.
–region REGION
Specify REGION as the web service region to use.
This option will override the URL specified by the “-U URL” option and EC2_URL environment variable.
-v, –verbose
Verbose output.
-?, –help
Display this help.
-H, –headers
Display column headers.
–debug
Display additional debugging information.
–show-empty-fields
Indicate empty fields.
–connection-timeout TIMEOUT
Specify a connection timeout TIMEOUT (in seconds).
–request-timeout TIMEOUT
Specify a request timeout TIMEOUT (in seconds).
June 21, 2011 at 11:55 am
Just to log my progress:
After spending a day or so, I figured that my EC2 API Toolkit was very old (2009) then it took me hours to find out how to and where to install it. I finally managed to complete the last instruction on Step 4. Now I am getting
ssh: connect to host ec2-72-44-50-141.compute-1.amazonaws.com port 22: Operation timed out
Next to sort this out!
Its not easy when you are not a techie, and every command had to be learned without really knowing what it does!
June 20, 2011 at 2:32 pm
@Manoj: Try a few variations. I think this one worked for me. Try with/without –root-device-name Please post what works for you.
ec2-register -s snap-6cf87123 -name “MyAMI″ -kernel aki-407d9123 –root-device-name /dev/sda1
June 21, 2011 at 11:57 am
I actually used
ec2reg -s snapshotid -n edocrAMItest -a x86_64
which created the AMI
June 29, 2011 at 11:43 pm
How do you know which Kernel and ramdisk to use?
June 30, 2011 at 5:22 am
@Marcus: In the AWS console or by way of other tools (cli, api) you can get a “Description” on an instance. In the console it’s the bottom half of the “My Instances” section –here you can find “Kernel ID” Without specifying ramdisk a default is used.
Check out some of the comments above for details. Keep in mind that your instance may already be EBS. If so, you can probably just use “ec2-create-image.”
July 7, 2011 at 6:52 am
I am also getting connection time out error. i followed all above mention procedure.
ami was created , but when i launched instance through it, ec2 instance was running but i cant connect to my instance. (Error: network connection time out).
below commands i used to creat snapshot and ami .
1) ec2-create-snapshot –region ap-southeast-1 vol-05598a78
this will give me snap id
2) ec2-register -n lg_stpi -d size20gb -a x86_64 –root-device-name /dev/sda1 –snapshot snap-a55d09c3 –region ap-southeast-1 –kernel aki-1df58a4f –ramdisk ari-35f58a67
Any idea regarding this?
Thanks in advance.
July 7, 2011 at 5:38 pm
@Gaurav: If you instance is already EBS bound then just use “ec2-create-image” instead. See my notes above.
December 8, 2011 at 12:30 am
You can now create an AMI from a running instance directly from the web console by selecting the instance and choosing “Create Image (EBS AMI)” from the actions menu.
The instance gets shutdown (not terminated), a EBS snapshot taken and converted to an AMI which is registered against your account, and then the instance is rebooted and comes back online.
It’s very streamlined, and much easier than the command line methods that used to be required.
July 6, 2012 at 11:46 pm
That only works if the instance is already running on EBS.
If it’s an older instance storage setup, you still need to follow this guide.
December 23, 2011 at 8:45 pm
every time I run this command ec2-register –snapshot snap-1ad8a07a –architecture x86_64 –name “live streaming server” –description “This is my personal imsage” –root-device-name /dev/sdf –kernel aki-e78edda2
I keep an error that saying: client.invalidmanifest Invalid maifest path
December 29, 2011 at 12:46 am
What about swap space?
All of my S3-backed instances have a swap partition at /dev/sda3. How do I account for that in the new EBS-backed AMI? Anyone know?
January 21, 2012 at 8:36 pm
Thanks for the detailed guide. I made a AMI from an instance store snapshot, now I launched a new instance with the AMI however i cannot connect to SSH (Timeout). Below is the console output:
Linux version 2.6.16-xenU (root@ip-10-204-118-8) (gcc version 4.0.2 20051125 (Red Hat 4.0.2-8)) #14 SMP Wed Nov 23 08:48:06 EST 2011
BIOS-provided physical RAM map:
Xen: 0000000000000000 – 000000006d400000 (usable)
1020MB HIGHMEM available.
727MB LOWMEM available.
NX (Execute Disable) protection: active
Built 1 zonelists
Kernel command line: root=/dev/sda1 ro 4
Enabling fast FPU save and restore… done.
Enabling unmasked SIMD FPU exception support… done.
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 65536 bytes)
Xen reported: 2327.502 MHz processor.
Console: colour dummy device 80×25
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Software IO TLB disabled
vmalloc area: ee000000-f53fe000, maxmem 2d7fe000
Memory: 1759232k/1789952k available (1970k kernel code, 21364k reserved, 628k data, 156k init, 1044488k highmem)
Checking if this processor honours the WP bit even in supervisor mode… Ok.
Calibrating delay using timer specific routine.. 4658.00 BogoMIPS (lpj=23290029)
Mount-cache hash table entries: 512
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 6144K
Checking ‘hlt’ instruction… OK.
Brought up 1 CPUs
migration_cost=0
Grant table initialized
NET: Registered protocol family 16
Enabling SMP…
Initializing CPU#1
migration_cost=646
Brought up 2 CPUs
xen_mem: Initialising balloon driver.
highmem bounce pool size: 64 pages
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered (default)
io scheduler deadline registered
io scheduler cfq registered
i8042.c: No controller found.
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
Xen virtual console successfully installed as tty1
Event-channel device installed.
netfront: Initialising virtual ethernet driver.
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
NET: Registered protocol family 2
Registering block device major 8
netfront: device eth0 has copying receive path.
IP route cache hash table entries: 65536 (order: 6, 262144 bytes)
TCP established hash table entries: 262144 (order: 9, 2097152 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
TCP bic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
NET: Registered protocol family 15
Using IPI No-Shortcut mode
XENBUS: Device with no driver: device/console/0
md: Autodetecting RAID arrays.
md: autorun …
md: … autorun DONE.
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 156k freed
lxcmount stop/pre-start, process 674
libudev: udev_monitor_new_from_netlink_fd: error getting socket: Invalid argument
mountall:mountall.c:3666: Assertion failed in main: udev_monitor = udev_monitor_new_from_netlink (udev, “udev”)
General error mounting filesystems.
A maintenance shell will now be started.
CONTROL-D will terminate this shell and reboot the system.
Give root password for maintenance
(or type Control-D to continue):
February 15, 2012 at 3:06 am
AMI The and their fast delivery technologies can be the best treatments for ED
February 22, 2012 at 4:01 am
Very excellent information for the complete process. I was able to convert one of the official Virtualmin AMIs into an EBS-backed equivalent. I shared the resulting AMI in case anyone want to compare results.
More about the Virtualmin AMIs at https://www.virtualmin.com/documentation/aws/virtualmin_gpl_ami
(In my case ami-9129eff8 was converted to EBS and shared publically as ami-5485573d when I finished.)
March 11, 2012 at 6:12 pm
I’ve tried this guide many times over and I’ve added the devices too and still i can’t log in to my new ebs instance 😦
May 11, 2012 at 12:42 am
Check your console log to see why it’s not booting.
August 11, 2012 at 4:32 pm
Thank ou for this! Well done. I only had one trip up on doing the conversion to EBS store, but not the fault of your instructions. On my instance, I had a mounted volume for my wordpress that didn’t get copied with the rsync. I just had to detach it from the instance store and attached it to the new EBS store.