Eric Hammond on Deploying Linux on EC2

By SCALE Guest Blogger Matthew Sacks, The Bitsource

Eric Hammond is an expert in the practical aspects of running Ubuntu on Amazon EC2. Since 2007, Eric has supported this platform creating public Amazon Machine Images (AMI's), publishing articles, guiding users in public forums, and occasional private consulting.

Eric has built and led successful technology for a number of early stage Internet startups in the Los Angeles area including Citysearch.com, Stamps.com, Rent.com, ThisNext.com, and his current startup, CampusExplorer.com, which runs in Ubuntu on Amazon EC2.

Eric is leading an EC2 beginners workshop in the Try It Lab at the Southern California Linux Expo (SCALE), February 19-21, in Los Angeles, CA. Eric spoke with me and gives some insights into his upcoming training lab.

Q: Why is EC2 an ideal platform for running Linux servers?

Eric Hammond: Though no single platform is ideal for all server situations, Amazon AWS and EC2 certainly deliver many benefits for a large segment of existing applications and open up the potential for new applications which may not have been previously possible or reasonable to consider.

Amazon AWS/EC2 provides on demand, self service, pay as you go, computing infrastructure. You don't have to plan ahead for how much hardware you might need. You don't have to keep paying for hardware after you no longer need it. You don't have to talk to any humans to provision the hardware. You don't have to maintain the system or network hardware.

If hardware fails, you simply throw it away and switch to new hardware with no additional cost. We just did this with our production database server. While we were at it, we brought the new server up in a newer data center (availability zone). How many startups can control these levels of resources using command line programs on a laptop?

EC2 eliminates many of the mundane chores and concerns which exist in so many other infrastructure environments. Amazon has built out solid hardware, software technology, processes, and staff to take care of an entire layer of your technology architecture so that you can focus on what is really unique and pertinent to your business.

EC2 has changed how I personally approach problems. One example is that instead of working in serial, sequential steps, I try to think in parallel. If I am working with a big set of data and don't know which of three possible architectures is going to get me the answer the fastest, I'll fire up a number of EC2 servers and try multiple approaches in parallel. When one of them is the clear winner, the other servers can be discarded without a second thought.

The coolest thing about Amazon EC2 for beginners is that it's easy and cheap to take it for a spin. You don't have to make a big decision and commit a lot up front just to try it out. There's nothing quite like the actual experience of typing a command or clicking a button and having one or a dozen high powered servers at your fingertips in under a minute.

In the EC2 beginners workshop at SCALE, we will help participants take this first step in experiencing how to run an EC2 server, use it a bit, and shut it down--all for about a dime.

Q: Do you recommend using pre-built images for deploying Linux-based EC2 instances?

Eric Hammond: Short answer: Yes. Long answer: It depends on a number of factors including how long it takes to configure a new server for your application, your level of trust in the persons or companies who built the public images, your level of Linux experience, and more.

Beginners are going to want to start with a public image published by a trusted individual or company. I recommend the official Ubuntu AMIs published by Canonical or the ones I publish myself. The Ubuntu 9.10 Karmic AMIs are the most up-to-date and run with consistent, modern kernels. When Ubuntu releases 10.4 Lucid in April, it will be immediately available for use on Amazon EC2 at the same time it is available for download. Other Linux distributions on EC2 tend to only be available in older releases or run with older/incompatible kernels, though there are still a lot of companies building EC2 applications with Debian, CentOS, Fedora, and more.

After running a public AMI and configuring the server, the next step many folks take is to re-bundle the running server, saving it as a private AMI to be run later. This is ok, but can create maintenance headaches when you realize you need to switch between architectures or upgrade to a new base image.

Many experienced users run public AMIs and have the servers install the desired software and configuration automatically on startup. This is the method I recommend to keep in mind as an end goal, even if you don't get there right away.

Some advanced users build an AMI from scratch, which has benefits of knowing exactly what is going into the image and being able to reproduce the process when you need to update the AMIs.

Q: Do many Linux users find EC2 as an ideal platform for learning Linux?

Eric Hammond: It's difficult to get a comprehensive view of who is using EC2 and for what purposes. There are hundreds of thousands of AWS customers and each person has their own level of experience, involvement, and reasons for using Linux and EC2.

However, based on some of the questions I've seen in communities like the EC2 Forum and the ec2ubuntu Google group, I believe that there is a significant number of users new to Linux getting their early experience with Linux on EC2, even though they're needing to learn two major technologies at the same time instead of just one.

Reasons for this might include the fact that you don't have to replace any operating system on your existing desktops or server hardware in order to try Linux on EC2; simply rent a system by the hour and try it out with a web browser and an ssh client. Plus, it's cheap; you can fool around with Linux on EC2 for most of a workday and end up paying under a dollar.

One of the benefits of EC2 is that each server starts out fresh; if you mess one up you can terminate it and start a new one, repeating until you get it just right. This can be handy even to more experienced Linux users who are experimenting with new software configurations.

Not only is it possible to try out Linux in the server environment on EC2, but users can also run a Linux desktop on EC2 and interact with it through their own Windows, Mac, (or Linux) computer as if it were local. This can be an alternative to "live CDs" like those provided by Ubuntu, and some users end up keeping their desktop persistent on EC2 so that they can log in to it from anywhere or share it with a user across the globe.

Q: Are there specific considerations a user must take when deploying Linux to EC2?

Eric Hammond: I would recommend a slow, gradual approach to EC2 deployments. As long as the need is not urgent, take your time, learn about EC2 and the other services in AWS, start running tests with your applications and figure out what works and what approaches need to be adapted.

Amazon has published a lot of documentation from the high architectural level, addressing concerns like security and reliability, down to the low command line options and API spec levels for developers. Read all you can and then don't hesitate to ask questions. The communities I mentioned earlier are great resources with lots of helpful folks who have gone through the same learning experiences you are about to enjoy.

Q: Will users need to bring their own laptop to take your tutorial at SCALE?

Eric Hammond: A number of Internet connected Ubuntu workstations will be provided in the Try It Lab for use by workshop participants. If there is extra room and the wifi is working well, then folks are also welcome to bring in laptops and follow along.

There is some preparation required for participation in the EC2 lab: Each participant needs to sign up for an AWS account well in advance of the SCALE event so that Amazon has enough time to authorize the account.

There is no commitment or cost for creating the account, so go ahead and take the steps if you are even thinking that you might like to participate in the introductory EC2 workshop.

More Resources: Follow Eric on Twitter: http://twitter.com/esh

Eric's technical blog: http://alestic.com/

Amazon EC2: http://aws.amazon.com/ec2/