Which instances are available in my region?

Have you wondered what the best way is to quickly list which instances are available in an AWS region? When the t2.nano was added to the Sydney region this week, I thought exactly this.

First place you would look would be the pricing page and select your region. However this only lists the latest instance types, you have to click through to another page to see the older generations. As a technical person you think "surely this can be done from the CLI"?

First port of call would be the trusty AWS CLI which I just love, its so handy. However there is no command which lets you extract the instances types. The closest you can get is to list the currently available reserved instances, for example

aws ec2 describe-reserved-instances-offerings --query "ReservedInstancesOfferings[?AvailabilityZone=='ap-southeast-2a'] [InstanceType]" --output text --region "ap-southeast-2" | sort -u

Problem with this command is that there may be an instance type which does not have an RI available. When I was doing this t2.nano and hi1.4xlarge were not available for RI in Sydney. So the CLI is probably not the best solution here.

The only other possibility is the relatively new pricing file. This was created to provide a programatic interface to the pricing data instead of the nasty scraping hacks people previously performed. The pricing file for EC2 lists all of the instances available and looks something like this.

{
  "formatVersion" : "v1.0",
  "disclaimer" : "This pricing list [...]",
  "offerCode" : "AmazonEC2",
  "version" : "20160126001708",
  "publicationDate" : "2016-01-26T00:17:08Z",
  "products" : {
    "DQ578CGN99KG6ECF" : {
      "sku" : "DQ578CGN99KG6ECF",
      "productFamily" : "Compute Instance",
      "attributes" : {
        "servicecode" : "AmazonEC2",
        "location" : "US East (N. Virginia)",
        "locationType" : "AWS Region",
        "instanceType" : "hs1.8xlarge",
        "currentGeneration" : "No",
        "instanceFamily" : "Storage optimized",
        "vcpu" : "17",
        "physicalProcessor" : "Intel Xeon E5-2650",
        "clockSpeed" : "2 GHz",
        "memory" : "117 GiB",
        "storage" : "24 x 2000",
        "networkPerformance" : "10 Gigabit",
        "processorArchitecture" : "64-bit",
        "tenancy" : "Shared",
        "operatingSystem" : "Windows",
        "licenseModel" : "License Included",
        "usagetype" : "BoxUsage:hs1.8xlarge",
        "operation" : "RunInstances:0002",
        "preInstalledSw" : "NA"
      }
    },

So with some use of jq magic to manipulate the json data we can extract only the instance flavors. Notice I am filtering for the region I am interested in.

curl -s  https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/index.json | jq -r '.products[].attributes | select(.location == "Asia Pacific (Sydney)" and .tenancy == "Shared") | .instanceType' | sort -u
c1.medium
c1.xlarge
c3.2xlarge
c3.4xlarge
c3.8xlarge
c3.large
c3.xlarge
c4.2xlarge
c4.4xlarge
c4.8xlarge
c4.large
c4.xlarge
d2.2xlarge
d2.4xlarge
d2.8xlarge
d2.xlarge
g2.2xlarge
g2.8xlarge
hi1.4xlarge
hs1.8xlarge
i2.2xlarge
i2.4xlarge
i2.8xlarge
i2.xlarge
m1.large
m1.medium
m1.small
m1.xlarge
m2.2xlarge
m2.4xlarge
m2.xlarge
m3.2xlarge
m3.large
m3.medium
m3.xlarge
m4.10xlarge
m4.2xlarge
m4.4xlarge
m4.large
m4.xlarge
r3.2xlarge
r3.4xlarge
r3.8xlarge
r3.large
r3.xlarge
t1.micro
t2.large
t2.medium
t2.micro
t2.nano
t2.small

If you are just after the instance families a little more manipulation will get that too.

curl -s  https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/index.json | jq -r '.products[].attributes | select(.location == "Asia Pacific (Sydney)" and .tenancy == "Shared") | .instanceType' | sort -u | cut -f1 -d. | sort -u
c1
c3
c4
d2
g2
hi1
hs1
i2
m1
m2
m3
m4
r3
t1
t2

Note you have to use the text descriptor of the region names and not the codes such as "ap-southeast-2". If you want to list all the region descriptors you can do.

curl -s  https://pricing.us-east-1.amazonaws.com/offers/v1.0/aws/AmazonEC2/current/index.json | jq -r '.products[].attributes.location | select(. != null)' | sort -u
Asia Pacific (Seoul)
Asia Pacific (Singapore)
Asia Pacific (Sydney)
Asia Pacific (Tokyo)
AWS GovCloud (US)
EU (Frankfurt)
EU (Ireland)
South America (Sao Paulo)
US East (N. Virginia)
US West (N. California)
US West (Oregon)

There you go, a nice and easy way to pull instance types for a region. Well its easy if you cut and paste, its a relatively long command.

Enjoy.

Rodos

P.S. Thanks to the Solution Architects in the Sydney team at AWS who thought of using describe-reserved-instances-offerings in the CLI, turns out it is not as fruitful as the pricing file though.

Killer interview question on success

I do a lot of interviewing, close to 400 of them over my 3 years working at Amazon. I am also in the process of becoming one of those mysterious "bar raisers" which you can read a description of in this WSJ article. Whilst I feel I still have a lot to learn about interviewing I find it a fascinating topic.

Previously I wrote some advise on interviews but figured I should write some more. Lifehacker.com.au likes to write articles on Killer Interview Questions and there was one this month that I thought was a good one.

One good question to ask is “Who succeeds in this position?” or, to phrase it carefully, “How would you define success for this position?”

What a great question. You are probably thinking "Great with the answer to this question I can just tell them what they want to hear! For the rest of the interview or for subsequent interviews I will reinforce my alignment to these". If that's what you are thinking you have missed a great insight.

What insight should you gain from this question? The insight you will gain is if you want to work for this company. Often people forget that interviews are a two way thing. The interviewers are trying to understand you and if you will be able to perform the role and be a good cultural fit for the company. Likewise you are trying to figure out if this role and company is something that you are willing to commit a portion of your life too. The answer to this question is going to give you good insight into the role and how to be successful. If characteristics expressed do not align with your personal goals/desires or how you want to work then you really need to consider is this the right job for you.

For example, what if part of the answer was, "To be successful in this role you really need to be curious and have a deep desire to learn new things and experiment on your own with new technologies and concepts. The people who have done very well in this role are like sponges when it comes to technology." That might sound fantastic to you and describe just how you like to operate in your personal and work life. On the other hand it might make you really uncomfortable. Maybe you just finished your MBA after previously doing years of university and you really want to find opportunities to practise your learning rather than embarking on something that is going to require you to learn and develop lots of new knowledge. Maybe you are the type of person that likes formal training and structure and experimenting on your own is just not your thing.

Another example, what if part of the answer was, "Success is easily defined here, if you don't hit your monthly quota you get zero commission. If you miss three in a row you will be moved onto a performance plan. We are a results driven company and there is no easier way to measure success than hitting your quota". Does that sound fantastic to you? Is that an environment where you think you will thrive? Some people might love that environment, they are result driven and like clear measurements. They have a track record of results to know they can achieve the task and they like it when everyone around them is held to the same bar. Of course many people, including me, would not find such an environment a cultural fit. This answer would give me some good insight to determine if this was a role I would be good at and enjoy.

Remember the questions you ask in the interview are important. As advised in the previously mentioned article, have some good questions prepared. But it's important to know why you are asking the question and what you are going to do with the answer. You want to gain insight into the role and the company. Some questions are better at achieving this than others.

Happy interviewing!

Rodos

P.S. Shameless plug. Remember Amazon in always hiring. See http://amazon.jobs/ for open roles in Australia. If you apply for a role in Solution Architecture you may end up having an interview with me! Wouldn't that be fun!