Last week Soliant Consulting joined 11,000 attendees at the 2017 AWS Summit in Chicago. This is the third year someone from my team has attended the event. Much like moving to the cloud, AWS Summit Chicago has grown significantly. Last year, less than 8,000 people registered for the conference. I noticed some visible growing pains as several sessions filled up very quickly and organizers struggle to manage the influx of attendees on the first day.
An Overview of AWS Summit Chicago
The AWS Summit included many informational sessions about all types of services focused on both developers and system administrators. Every session has an associated skill level, allowing attendees to quickly determine if it’s a good fit for them. For example, the 200 level topics are aimed at someone with no real experience with the technology subject. In addition to these sessions, AWS Summit included “The Hub,” where sponsors and vendors showcased their products and services in everything from infrastructure and monitoring to backups and security.
AWS Keynote Insights
The AWS Summit Chicago keynote included two examples of how using the right tools in AWS minimizes costs and maximizes performance.
Redshift Spectrum
The first example showed loading one exabyte (1,000,000 terabytes) of structured data into s3 and then running a complex query on that data. The presenters started running the data through Apache Hive with 1,000 nodes, let it run for a while, and then determined the process would take five years. They then ran the same data through Redshift Spectrum, and the process took 155 seconds. That’s more than 6,451 TB/S! Keep in mind, though, Hive can run on unstructured data but Spectrum can’t.
Lamba
Morning Star’s CTO, Mitch Shue, provided the next example on his company’s migration to AWS. Their first stage of the process was a like for like migration. Their team moved from Elastic Compute Cloud (EC2) instances and started processing data using S3 and Lambda, a change that shortened its nightly import time to only two hours. As a result, Morning Star decreased costs by 97%.
Lambda works by running small jobs on AWS. Think of it as moving the line where AWS is in charge of the infrastructure. For EC2, AWS is in charge through the hyper visor. With Lambda AWS also manages the OS; you only need to worry about your code and only pay for the seconds and memory your code requires to run. Lambda is the optimal choice for jobs that don’t run all of the time or for parallel work, such as nightly imports.
Want to hear more? Watch the full AWS Summit Chicago keynote here.
AWS Elastic File System (EFS)
One of my favorite talks was the deep dive on AWS EFS given by Darryl Osborne. He discussed EFS, S3, Elastic Block Store (EBS), and their differences. For example, EBS can only be accessed by one EC2 instance, much like a hard drive in a physical computer. It can, however, have very high IOPS and decent throughput, depending on the type of storage you pick.
With S3, you use APIs to access data. You will probably need to rewrite your programs to access your data, but you’ll benefit from inexpensive storage and very high throughput. EFS has slower throughput than S3; its throughput is related to the amount of data you have stored in EFS. EFS, much like S3, is elastic storage, so you only pay for what you need. EBS requires you to provision the disk for a particular size. During the demo, Darryl showed how a T2.micro can perform better with EBS than an m4.2xlarge over a short period of time due to T2’s burst performance capabilities.
Looking forward to AWS Summit Chicago 2018
I strongly recommend both developers and system administrators attend next year’s event, as the conference includes many topics for both groups. I enjoyed attending on behalf of Soliant Consulting for the 2017 event and look forward to attending for our team’s fourth year in 2018. If you have any questions about the event, please ask in a comment below. I’m happy to provide answers and additional insights about the conference.
