Amazon Web Services Tutorial

By: Jinesh Lalan Department of Computer Science San Francisco State University 9th December, 2010

Amazon EC2 Tutorial:

The purpose of this tutorial is to make it easier for a novice user experienced in basic computer science technologies to use Amazon Cloud computing service. The tutorial here is written take into consideration that the user is proficient at the following skills: Experience with a cloud computing services such as Amazons Amazon Web Services. Understanding X.509 Certificates and Public key cryptography. Knowledge of the concept of Virtualization Perl Linux

Following is the summary of the steps that you need to follow to get started using Amazon Cloud Services: 1. AWS Sign-Up 2. EC2 Sign-Up 3. Installing AWS command line tools 4. Securing your interactions with the AWS Cloud System 5. Creating a Machine Image (Amazon Machine Image) in order to launch your Instances

Step 1: AWS Sign-Up In order to get started using the service, the first thing one has to do is go to the AWS website (http://aws.amazon.com/) and signup.

Sign-in Page If one already has an account with Amazon (the account through which one buys ones books), you dont need to sign up, you can use the same account here. If one does not have an account with Amazon, one can enter his/her Email-id and click on the button Sign in using secure server and fill in the details. Once one has done or signed in for the first time one will get the following message.

Account Created

Step 2: EC2 Sign-Up

Once you sign in to AWS, the next thing one needs to do is, go to the following link (http://aws.amazon.com/ec2/) and click on Sign Up for Amazon EC2. One may need to signin here again. Then if one scrolls down at the bottom of the page it asks one to enter your credit card details. If one already has been using Amazon services and provided ones credit card information before it wont ask one, but one can provide a new credit card if one wants to.

Once you click on continue, you will need to verify your identity. You can enter your phone number and click on the button Call me now. Once you click Call me now you will see a pin number on your screen which you need to type it out on your phone once you pick up the phone. Once you have done that you will see the message Your identity has been verified successfully. Then when you click on continue it asks you to click on complete sign -up button. Once you click on that the next web page will tell you that We will send you a confirmation on your email when the service has been activated. It may take some time while you receive your confirmation email.

Next click go to the following link (http://aws.amazon.com/s3/). Click on the sign-up button. You should have the following displayed.

You may follow the current AWS tutorial for step1 and step2: http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/SignUp.html

Step 3: Installing AWS command line tools:

Setting JAVA_HOME environment variable: Installing command line tools require java version 5 or later. You can download it from the following site http://java.sun.com/j2se/1.5.0/ . Once you have the appropriate java version, the next thing you do is set the JAVA_HOME environment variable.

On your linux machine you can check your Java Path by typing which java on your terminal. Should give you something like /usr/bin/java. To set the JAVA_HOME variable type export JAVA_HOME=/usr on your terminal. Downloading EC2 AMI tools: You can download the tools using the following command: wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip Wget is the command to download a file and the second part is the link for downloading the AMI tools. You can then unzip the tools by typing the following command: unzip ec2-ami-tools.zip The while is extracted to the directory you are in or the directory that you provided. The extracted directory name is something like ec2-ami-tools-versionNumber.

Setting EC2_HOME environment variable: The command line tools rely on the EC2_HOME environment variable. Type in the following commands on your terminal: export EC2_HOME=/opt/ec2-api-tools-versionNumber export PATH=$PATH:$EC2_HOME/bin/

Setting the Private Key and X.509 Certificate key: The command line tools need access to your private key file and the certificate file that you downloaded earlier and hence you will need to set the environment variables for the same. export EC2_PRIVATE_KEY=~/.ec2/pk-XXX.pem export EC2_CERT=~/.ec2/cert-XXX.pem

Step 4: Securing your interactions with the AWS Cloud System Access to applications and services within AWS cloud is secure and protected in multiple ways. Accessing those applications and services requires the use of special credentials that are associated with your account. There are three types of credentials currently offered by AWS [AWS]. For our purpose we only need one of those three credentials i.e. Access Credentials, which includes three things: Access Keys X.509 Certificates Key Pairs

Access Keys:

Go to the following link (http://aws.amazon.com/), which is AWSs home page. Click on the Accounts tab. In the drop down menu click on Security Credentials on the left. Under the Access Credentials section, you can see the Access keys tabs which contain your access key id and your secret access key. You should take care that these credentials dont get into wrong hands, as it may give them access to your account. We will get to know a bit later as to how and when to use these credentials

X .509 certificate: X.509 certificates are based on the concept of public key cryptography. The following wikipedia links will explain you its use. (http://en.wikipedia.org/wiki/Public_Key_Cryptography). Click on X.509 Certificates tab and click on Create a new Certificate. It will give you the option of downloading the private key and the X.509 certificate. Download both of them and save it somewhere on your machine. You will need the X.509 keys if you plan to use the command line tools.

You can rename them as per your convenience.

Generate Key Pair:

You must create a public/private key pair to ensure that only you have access to instances that you launch.

After you generate a key pair, the public key is stored in Amazon EC2 using the key pair name you selected. Whenever you launch an instance using the key pair name, the public key is copied to the instance metadata. This allows you to access the instance securely using your private key.

Go to your AWS home page and click on Accounts tab. Select Amazon EC2 from the drop down and click on Sign in to the AWS Console. Once you sign in, you can see your EC2 dashboard.

Key Pairs Click on the Key Pairs link under the section Networking and Security. Click on Create a new Keypair. You will be asked to download the keypair to your machine. It will a .pem" file.

Following link will lead you to an updated version of the tutorial for setting up your security credentials: http://docs.amazonwebservices.com/AWSEC2/latest/DeveloperGuide/using-credentials.html

Step 5: Creating a Machine Image (Amazon Machine Image) in order to launch your Instances: Following steps need to be followed: Find a suitable AMI Launch an Instance of that AMI Access the AMI through the terminal using SSH Compress (Bundle) the AMI for Uploading it on S3 Upload the AMI to S3

Find a suitable Amazon Machine Image Log in to your AWS console as shown above and select click on Amazon EC2 tab on the top.Amazons Data Centers are spread across United States and that there are multiple locations from where you can run your AMIs from. If you look at the navigation menu on the left, you can see the drop down menu, where you can select the region for your AMIs. I would suggest selecting a region near you. Say if you are working from San Francisco, it is better you select your AMIs in the region US West (N. California).

Click on the AMIs link under the Images section on the navigation bar on the left.Select Public Images in the Viewing drop down. You should a list of AMIs with different operating systems and applications bundled in together.

Select an AMI for the platform you want to run your applications on. For our research we selected an AMI whose root device is Instance Store. There are two ways in which you can store your machine images. The first one is instance store i.e. on Simple Storage Service (S3) and second is Elastic Block Storage (EBS). You can check out the following link that explains the difference between the two

(http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/)

Launching an Instance of that AMI Once you select an AMI by clicking on the checkbox provided for that AMI, click on Launch.

Next type in the number of instances you want to create. If you are doing it for the 1 st time, I guess create only one instance. You can select any Availability Zone from the drop down, or let them decide the zone for you by keeping it as No Preference and click on Continue. On the next page you are required to choose a kernel id and a ram disk id. Let these option be use default. Click on continue.

Next you land on the keypair page.Here you need to select the keypair that you created earlier. If you only have a single keypair created it will be preselected for you. You can have multiple keypair created each with a different name. Click on continue.

Next you land on the configure firewall page. Use the default security group provided by the AWS. Click on continue.

This is the final page where you see a summary of your configuration for the instance that you are going to create. Hit Launch if you feel everything seems good, else you can click on back to make any changes.

Click on the instances link on the left navigation area. You should see your instance starting which is indicated by a yello solid circle below the status field .

After a while you should see your status as running indicated by a green solid circle under the status field. You can now access your instances via SSH as explained below

Access the AMI through the terminal using SSH If you are on a Linux system, open your terminal and type in the following command: ssh -i id_keypair_file root@ec2-67-202-51-223.compute-1.amazonaws.com

Here ssh is the network protocol through which you will be able to interact with your instances. Id_keypair_file is your key pair (if your key pair is in a directory other than your current directory, then you should include the key pair file along with its path) file that you downloaded earlier. root is the user you will be logging in as. The part after the @ is your public DNS of your instance. You can get it from you EC2 management console. Click on the checkbox of your

instance on the console and then scroll down. You can see the details of your instance. Among those will the field Public DNS. Copy the field value and paste it right beside the @ and press enter and you will be logged into your instance and do whatever you want to with that instance. Once you are inside the instance you will be able to install all your applications and upload any files to it. Talk abt instance storage. Uploading your Certificate and Private Key File to your AMI: You need to upload your private key file and certificate file from your local machine to your AMI in order to to ficilitate the next step i.e. Uploading your AMI to S3. You can use the scp command of the SSH family to copy files to a remote server.

scp -i id_keypair_file /path/private-key /path/certificate root@publicDNS

Compress (Bundle) the AMI for uploading it on S3: Every time you launch an instance and make changes to your instance you need to bundle your instance and upload the image to S3. In order to bundle and upload your instances you first need to use the AMI tools. AMI tools are already installed inside your AMI. "The creation (bundling) process for an AMI that uses an instance store(S3) as its root device does the following: Compresses the image to minimize bandwidth usage and storage requirements

Encrypts and signs the compressed image to ensure confidentiality and authenticates the image against its creator

Splits the encrypted image into manageable parts for upload Creates a manifest file that contains a list of the image parts with their checksums" [AWS]

Command to bundle your AMI: ec2-bundle-vol -k <private_keyfile> -c <certificate_file> -u <user_id> where private_keyfile is the path to your private key file on your AMI, same for the certificate_file and finally the last parameter is your account user_id as we saw earlier. e.g. ec2-bundle-vol -d /mnt -k /.ec2/pk-XXXX.pem -c /.ec2/cert-XXX.pem -u XXXX-XXXXXXXX

Uploading the AMI to S3: "You must upload the bundled AMI to Amazon S3 before it can be accessed by Amazon EC2. Use ec2-upload-bundle to upload the bundled AMI that you created earlier. Amazon S3 stores data objects in buckets, which are similar to directories. Buckets must have globally unique names. The ec2-upload-bundle utility uploads the bundled AMI to a specified bucket. If the specified bucket does not exist, it will be created. If the

specified bucket exists and belongs to another AWS account, the ec2-upload-bundle command will fail." [AWS]

Command to Upload the bundled AMI: ec2-upload-bundle -b <bucket> -m image.manifest.xml -a <access_key> -s <secret_key> where bucket is the name of the bucket, image.manifest.xml is the manifest file as described above and is written as it is.

Registering your AMI Once all the image parts are uploaded onto S3, the next thing you need to do is register the uploaded AMI. You can run the following command on your terminal using the ec3-register command. ec2-register <Bucket-Name>/image.manifest.xml n image-name