From Zero to Cloud: Setting up an EC2 Sandbox, Part 2

This is the second post in a three part series focused on setting up EC2 as a sandbox for application developers. I assume you have an AWS account with Amazon; if you don’t, please read Part 1 to learn how easy it is to sign up. In this installment we’ll see how to configure our development environment and install an Amazon Machine Instance (AMI) to run our applications.

Configuring your Development Environment

Before we get too carried away, let’s make sure our local client is ready to go. Amazon provides a rich set of command line tools that wraps the AWS Management Console nicely. If you’ve made it this far, chances are you prefer the command line.

First we need to download our X.509 Certificate, which is a Public Key that allows us to make secure SOAP requests to Amazon, and our corresponding Private Key. You can download your X.509 Certificate at any point, but Amazon only gives you your Private Key once and doesn’t store it. Make sure you keep it, do not share it, and keep in mind that there is no way to recover it if it ever gets lost.

Click on Account in the top menu; this will take us to your account page. Then, click Security Credentials from the right column. Scroll down to Access Credentials, select the X.509 tab and click Create a new Certificate. A dialog will appear that has links to the two aforementioned files, your Private Key and the X.509 Certificate (Public Key). Remember that this is your only chance to download your Private Key. Be sure to do so, and store it in a secure location.

X.509 Certificate Popup

Next, we need to download the Amazon EC2 API Tools which is the client interface to Amazon’s EC2 web services. Go to http://aws.amazon.com/developertools/.

AWS Developer Tools page

Locate and click on Amazon EC2 API Tools. Unzip the download and we should have the following files.

Unzipped EC2 API Tools folder

Next, let’s open a shell and get our workstation configured to talk to Amazon and interact with our EC2 environment.

First, create the hidden .ec2 directory in your home directory.

Last login: Wed Aug 17 22:34:19 on ttys000
MacBook-Pro:~ john2$ mkdir ~/.ec2

Next, move the X.509 Certificate, the Private Key file, and the ec2-api-tools folder into ~/.ec2.

MacBook-Pro:~ john2$ cd .ec2
MacBook-Pro:.ec2 john2$ mv ../Downloads/*.pem .
MacBook-Pro:.ec2 john2$ mv ../Downloads/ec2-api-tools-1.4.4.0 .
MacBook-Pro:.ec2 john2$ ls -las
total 16
0 drwxr-xr-x   5 john2  staff  170 Aug 17 22:38 .
0 drwxr-xr-x+ 16 john2  staff  544 Aug 17 22:35 ..
8 -rw-r--r--@  1 john2  staff  916 Aug 17 22:28 cert-{YOUR-HASH}.pem
0 drwxr-xr-x@  7 john2  staff  238 Aug  3 04:24 ec2-api-tools-1.4.4.0
8 -rw-r--r--@  1 john2  staff  924 Aug 17 22:28 pk-{YOUR-HASH}.pem
MacBook-Pro:.ec2 john2$ ls -las ec2-api-tools-1.4.4.0/
total 120
 0 drwxr-xr-x@   7 john2  staff    238 Aug  3 04:24 .
 0 drwxr-xr-x    5 john2  staff    170 Aug 17 22:38 ..
96 -rw-r--r--@   1 john2  staff  46468 Aug  3 04:19 THIRDPARTYLICENSE.TXT
 0 drwxr-xr-x@ 510 john2  staff  17340 Aug  3 04:24 bin
 0 drwxr-xr-x@  33 john2  staff   1122 Aug  3 04:24 lib
16 -rw-r--r--@   1 john2  staff   4852 Aug  3 04:19 license.txt
 8 -rw-r--r--@   1 john2  staff    539 Aug  3 04:19 notice.txt

Next, it’s a good idea to edit ~/.bash_profile to provide our machine with the necessary EC2 paths.

MacBook-Pro:.ec2 john2$ vi ~/.bash_profile
export EC2_HOME=~/.ec2/ec2-api-tools-1.4.4.0
export PATH=$PATH:$EC2_HOME/bin
export EC2_PRIVATE_KEY=pk-{YOUR-HASH}.pem
export EC2_CERT=cert-{YOUR-HASH}.pem
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/

Afterwards, be sure to source ~/.bash_profile to make use of these variables in the current session.

MacBook-Pro:.ec2 john2$ source ~/.bash_profile

Now we have our local development environment configured to talk to EC2 and issue remote commands via the ec2 client interface. The next step is to create a key pair that we can use to launch instances; and, once launched, connect via ssh. This is a new and different set of keys than we created when configuring our SOAP client to Amazon EC2 itself.

MacBook-Pro:.ec2 john2$ ec2-create-keypair john2.kp > john2.kp
MacBook-Pro:.ec2 john2$ ec2-describe-keypairs
KEYPAIR	john2.kp	42:a8:19:7d:85:fc:aa:e7:a8:13:cc:c2:2b:92:b9:ee:c6:6e:07:f5
MacBook-Pro:.ec2 john2$ chmod 600 john2.kp
MacBook-Pro:.ec2 john2$ cat john2.kp 
KEYPAIR	john2.kp	42:a8:19:7d:85:fc:aa:e7:a8:13:cc:c2:2b:92:b9:ee:c6:6e:07:f5
-----BEGIN RSA PRIVATE KEY-----
SUPER SECRET STUFF
-----END RSA PRIVATE KEY-----

If you run into any ec2-xxx-xxxx command not found messages, be sure that ~/.bash_profile has been sourced in that particular terminal session and that the EC2-specific paths therein are correct.

Finding an AMI

The next step is to find an Amazon Machine Instance (AMI). An AMI is a ready-to-run image of an operating system instance. You can find a variety of AMI providers for many operating systems from Windows to Red Hat. A good place to start is http://aws.amazon.com/amis.

If you are interested in running Ubuntu, Alestic provides reliable Ubuntu-based AMI’s. Simply go to http://alestic.com/, choose your region and browse the available images.

Ubuntu AMIs on alestic

I’ll choose us-east-1 Ubuntu 11.04 Natty EBS boot server 64-bit. We can use ec2-describe-images to ensure Amazon recognizes our image ID and review the details of the image.

MacBook-Pro:.ec2 john2$ ec2-describe-images ami-1aad5273
IMAGE	ami-1aad5273	099720109477/ebs/ubuntu-images/ubuntu-natty-11.04-amd64-server-20110426	099720109477	available	public		x86_64	machine	aki-427d952b			ebs	paravirtual	xen
BLOCKDEVICEMAPPING	/dev/sda1		snap-7d8f0f12	8

Now we’re ready to launch our first instance! We’ll use the key we created earlier by specifying the -k parameter. This is important because this same key will be used to connect to our instance via ssh in subsequent steps. The --instance-type argument will ultimately save a lot of money; we are building a development sandbox and we shouldn’t need more resources than a micro instance provides. Finally, we pass the ID of the chosen AMI and we have a running Ubuntu server in Amazon’s EC2 Cloud!

MacBook-Pro:.ec2 john2$ ec2-run-instances -k john2.kp --instance-type t1.micro ami-1aad5273
RESERVATION	r-5be89134	582155754520	default
INSTANCE	i-c795bfa6	ami-1aad5273			pending	john2.kp	0		t1.micro	2011-08-22T00:21:03+0000	us-east-1b	aki-427d952b			monitoring-disabled					ebs				paravirtual	xen		sg-f5a0899c	default
MacBook-Pro:.ec2 john2$ ec2-describe-instances
RESERVATION	r-03751f6c	582155754520	default
INSTANCE	i-c97328a8	ami-1aad5273			terminated	john2.kp	0		t1.micro	2011-08-20T00:48:15+0000	us-east-1a	aki-427d952b			monitoring-disabled					ebs			paravirtual	xen		sg-f5a0899c	default
RESERVATION	r-5be89134	582155754520	default
INSTANCE	i-c795bfa6	ami-1aad5273	ec2-75-101-238-254.compute-1.amazonaws.com	ip-10-245-202-88.ec2.internal	running	john2.kp	0		t1.micro	2011-08-22T00:21:03+0000	us-east-1b	aki-427d952b			monitoring-disabled	75.101.238.254	10.245.202.88			ebs					paravirtual	xen		sg-f5a0899c	default
BLOCKDEVICE	/dev/sda1	vol-58923732	2011-08-22T00:21:26.000Z

Before we connect via ssh, let’s stop our instance, start it up again, and while doing so have a look at the AWS Management Console.

When we started our instance via ec2-run-instances, Amazon returned a unique instance ID. In the case above, it returned i-c795bfa6. This instance ID is what we use to invoke future instance specific actions, for example starting, stopping, creating AMIs, and terminating (deleting) our instance. Spend a moment here stopping and starting your instance and watching the Status column in the AWS Management Console for this instance transition from running to stopping to stopped.

AWS Management Console

MacBook-Pro:.ec2 john2$ ec2-stop-instances i-c795bfa6
INSTANCE	i-c795bfa6	running	stopping

In the future, you won’t need to stop and start the instance via the command line, but it is helpful to be familiar with the process. It also helps to understand the distinction between the two key pair families. The former, X.509, is for interacting with EC2 from the command line. These key pairs give us access to the 124 ec2 commands. The latter, the john2.kp keys give us access to the instance itself. Why did we only see one key from the latter’s family? We saw the Private Key, Amazon kept the Public Key. In the future, however, we can execute many of the actions provided by the command line client with the AWS Management Console. To invoke these actions select the checkbox of the instance you want to perform an action on and then find the appropriate action in the Instance Actions dropdown at the top of the My Instances section.

Let’s start our instance back up and connect via ssh for the first time. Remember the Private Key Amazon gave us when we ran ec2-create-keypair and that we used to launch our instance via ec2-run-instances? We need that to connect to our instance via ssh. We’ll also need the public DNS entry for this instance. You can find this in the EC2-Instance details section of a selected instance within the AWS Management Console. Copy and paste this into your ssh command. We need to use the -i (identity file) flag to connect to our instance for the first time. Different AMIs have different rules for connecting for the first time, but the Ubuntu image we chose allows the user ubuntu to connect.

Before connecting via ssh, we need to open the necessary ports using ec2-authorize.

MacBook-Pro:.ec2 john2$ ec2-authorize default -p 22
MacBook-Pro:.ec2 john2$ ec2-authorize default -p 80
MacBook-Pro:.ec2 john2$ ssh -i john2.kp ubuntu@ec2-50-19-132-116.compute-1.amazonaws.com

This concludes Part 2 of our 3 part series on setting up a development sandbox on Amazon EC2. We have gone from a brand new account to a running Ubuntu instance. Stay tuned to enjoy the payoff of running servers at Amazon. Next we will cover setting up a LAMP stack and creating our own images!

Image via Lightspring / Shutterstock

Free book: Jump Start HTML5 Basics

Grab a free copy of one our latest ebooks! Packed with hints and tips on HTML5's most powerful new features.