CloudOps at AWS Re-Invent

downloadre-invent

This is a little delayed because we’ve been busy deploying clients on AWS and cloud.ca.

There were plenty of highlights from last month’s AWS re:Invent 2014 conference held in Las Vegas, NV. Plenty of good news, a few surprises, and a few disappointments. We are excited to share the news and improvements with our managed AWS cloud customers and the blogosphere.

Starting with the biggest news for us – Aurora.

Aurora is AWS’s new massively scalable MySQL managed database service, available through RDS. The traditional RDS MySQL has proved a disappointment for several use cases for many years now. Size limits, backup snapshots, scalability limits, and read-replicas only get you so far. Fortunately Aurora addresses most of these issues increasing the size of the datastore up to 64TB! As a bonus it auto-expands in 10gb chunks, so you only pay for what you need. In addition, AWS has included built-in data redundancy, the ability to add more servers who utilize the same shared block storage, and replication between up to 3 availability zones.

Overall fairly impressive and a great drop-in replacement for many customers needing more juice out of MySQL. This should cause other vendor offerings such as Percona Cluster, MariaDB, and Clustrix some concern.

That said, we continue to see some requirements for specific MySQL workloads not addressed by Aurora (or other current AWS solutions). The system is not true multi-master, utilizing shared disk access instead. Moreover the replication is not synchronous and many applications are either sensitive to, or not known to be read-after-write safe. Finally the service is only currently available in one region (though we suspect AWS will address this limitation quickly as usual) and it remains to be seen if multi-region implementation will be possible.

In summary, Aurora is another strong addition to the current MySQL and database offerings from AWS.

Highly popular AWS EC2 Container Service & Lambda

After the AWS announcement of the AWS EC2 Container service (Docker compatible), all session tracks related to Docker support had a significant lineup and standing room only. The AWS EC2 container service brings all the features you would expect when using AWS compute – scaling, scheduling, and more built in. We don’t have much more to say about the benefits until we test this so we’ll be doing a separate Docker post soon.

What’s the downside? Things are getting crowded on AWS. It’s starting to become a complex choice between Beanstalk, OpsWorks, CloudFormation, and now Lambda and Containers.

Lambda appears to have already developed a bit of a cult following. Lambda falls somewhere around Beanstalk in terms of usefulness, but without the cost overhead. Useful for high volume events from code snippets or infrequently run workflow helpers. The downside? It’s javascript only, and you won’t be using it for streams or long polling.

What else? Consistent deploys!

We’ve seen for some time that deployments in AWS can be a mess, regardless of what and how you deploy your application today (OpsWorks, Beanstalk, Cloudformation, or manually in some fashion). One person runs git pull, another uses jenkins to push code, someone else uses bamboo or hudson, and the next person produces fully-fledged rpm and deb packages.

Every customer we manage requires a slightly different deployment model. It’s painful, it’s home grown, and we work to accommodate it as best we can – everyone has something special about the way they deploy upgrades.

The 3-way release from AWS appears to hold promise of an end to end solution to this mess. They’ve also realized that nobody is ditching their current toolchains overnight as every single component supports integrations. CodeCommit gives an alternative private Git repo option, CodePipeline provides a way of delivering and packaging, providing workflows and integrations, and finally CodeDeploy provides a way to push all that code into test/dev, and finally release to production.

We are very optimistic that this should bring some sanity to deploying customized applications into AWS, regardless of how you choose to manage them.

Bigger, better resources.

Finally we are excited, though not entirely surprised, to see the C4 instance family released along with larger EBS drives.

The fact that AWS was able to gain exclusive use of the Intel-made Xeon E5-2666 v3 was amazing, but was not implausible given AWS’ scale. We expect to see more hardware exclusivity deals in the future if the C4 catches on.

Larger EBS volumes came as absolutely no surprise. Our customers have been begging AWS to release larger drives for years now. Larger drives make it easier to grab a consistent snapshot (RAID/LVM/DB tablespaces), make things easier to manage, and in some cases decrease licensing costs (for per volume licensing). The fact that the volumes happen to be available up to 16TB (hey, we just wanted 4TB!) and they more than double the baseline IOPS (10k!) and PIOPS (up to 20k!) is simply icing on the cake.

Overall a great Re-Invent conference. We look forward to seeing how these new features will integrate with our customers’ environments over the next several months.

Share Your Thoughts!

420 Guy Street
Montreal, QC H3J 1S6
CANADA
19/20 Commercial Road
Melbourne, VIC 3004
AUSTRALIA
201 Sommerville Ave.
Sommerville, MA 02143
USA
530 Broadway, 9th Floor
New York, NY 10012
USA