Amazon Web Services Korea LLC (AWS Korea) opened an office in Seoul, South Korea in 2012. This office has educated and supported many customers from startups to large enterprises. Owing to high customer demand, we launched our Asia Pacific (Seoul) Region with 2 Availability Zones and two edge locations in January 2016. This Region has given AWS customers in Korea low-latency access to our suite of AWS infrastructure services.

Andy Jassy, CEO of Amazon Web Services announced to launch Seoul Region in AWS Cloud 2016.

Following this launch, Amazon CloudFront announced two new Edge locations and one Edge cache: the third in May 2016, and the fourth in Feb 2018. CloudFront’s expansion across Korea further improves the availability and performance of content delivery to users in the region.

Today I am happy to announce that AWS added a third Availability Zone (AZ) to the AWS Asia Pacific (Seoul) Region to support the high demand of our growing Korean customer base. This third AZ provides customers with additional flexibility to architect scalable, fault-tolerant, and highly available applications in AWS Asia Pacific (Seoul), and will support additional AWS services in Korea. This launch brings AWS’s global AZ total to 66 AZs within 21 geographic Regions around the world. AZs located in AWS Regions consist of one or more discrete data centers, each with redundant power, networking, and connectivity, and each housed in separate facilities.

Now AWS serves tens of thousands of active customers in Korea, ranging from startups and enterprises to educational institutions. One of the examples that reflects this demand is AWS Summit Seoul 2019, a part of our commitment to investing in education. More than 16,000 builders attended, a greater than tenfold increase from the 1,500 attendees of our first Summit in 2015.

AWS Summit 2018 – a photo of keynote by Dr. Werner Vogels, CTO of Amazon.com

So, how have Korean customers migrated to the AWS Cloud and what has motivated them? They have learned that the AWS Cloud is the new normal in the IT industry and quick adoption to their business has allowed them to regain global competitiveness.

Let us look at some examples of how our customers are utilizing the benefit of the broad and deep AWS Cloud platform in the global market by replicating their services in Korea.

Do you know Korean Wave?
The Korean Wave represents the increase in global popularity of South Korean culture such as Korean Pop and Drama. The top three broadcasting companies in Korea (KBS, MBC, and SBS) use AWS. They co-invested to found Content Alliance Platform (CAP) that launched POOQ, which offers real-time OTT broadcasting to 600,000+ subscribers for TV programs including popular K-Dramas and has been able to reduce the buffer times on its streaming services by 20 percents. CAP also used AWS’s video processing and delivery services to stream Korea’s largest sports event, the PyeongChang 2018 Olympic Winter Games.

Lots of K-Pop fans from KCON Concert 2016 in France – Wikipedia

SM Entertainment, a South Korean entertainment company to lead K-Pop influences with NCT 127, EXO, Super Junior, and Girls’ Generation. The company uses AWS to deliver its websites and mobile applications. By using AWS, the company was able to scale to support more than 3 million new users of EXO-L mobile app in three weeks. The company also developed its mobile karaoke app, Everysing, on AWS, saving more than 50 percent in development costs. The scalability, flexibility, and pay-as-you-go pricing of AWS encouraged them to develop more mobile apps.

Global Enterprises on the Cloud
Korean Enterprises rapidly adopted AWS cloud to offer scalable global scale services as well as focus on their own business needs.

Samsung Electronics uses the breadth of AWS services to save infrastructure costs and achieve rapid deployments, which provides high availability to customers and allows them to scale their services globally to support Galaxy customers worldwide. For example, Samsung Electronics increased reliability and reduced costs by 40 percent within a year after migrating its 860TB Samsung Cloud database to AWS. Samsung chose Amazon DynamoDB for its stability, scalability, and low latency to maintain the database used by 300 million Galaxy smartphone users worldwide.

LG Electronics has selected AWS to run its mission-critical services for more than 35 million LG Smart TVs across the globe to handle the dramatic instant traffic peaks that come with broadcasting live sports events such as the World Cup and Olympic Games. Also, it built a new home appliance IoT platform called ThinQ. LG Electronics uses a serverless architecture and secure provisioning on AWS to reduce the development costs for this platform by 80 percent through increased efficiency in managing its developer and maintenance resources.

Recently Korean Air decided to move its entire infrastructure to AWS over the next three years – including its website, loyalty program, flight operations, and other mission-critical operations — and will shut down its data centers after this migration. “This will enable us to bring new services to market faster and more efficiently, so that customer satisfaction continues to increase.” said Kenny Chang, CIO of Korean Air.

AWS Customers in Korea – From Startups to Enterprises in each industries

AI/ML on Traditional Manufacturers
AWS is helping Korean manufacturing companies realize the benefits of digitalization and regain global competitiveness by leveraging over collective experience gained from working with customers and partners around the world.

Kia Motors produces three million vehicles a year to customers worldwide. It uses Amazon Rekognition and Amazon Polly to develop a car log-in feature using face analysis and voice services. Introduced in CES 2018, this system welcomes drivers and adjusts settings such as seating, mirrors and in-vehicle infotainment based on individual preferences to create a personalized driving experience.

Coway, a Korean home appliance company uses AWS for IoCare, its IoT service for tens of thousands of air & water purifiers. It migrated IoCare from on-premises to AWS for speed and efficiency to handle increasing traffic as their business grew. Coway uses AWS managed services such as AWS IoT, Amazon Kinesis, Amazon DynamoDB, AWS Lambda, Amazon RDS, and Amazon ElastiCache, which also integrated Alexa Skills with AWS Lambda with their high-end air purifier Airmega for the global market.

Play Amazing Games
AWS has transformed the nature of Korean gaming companies, allowing them to autonomously launch and expand their businesses globally without help from local publishers. As a result, the top 15 gaming companies in Korea are currently using AWS, including Nexon, NC Soft, Krafton, Netmarble, and KaKao Games.

Krafton is the developer of the hit video game Player Unknown’s Battle Grounds (PUBG), which was developed on AWS in less than 18 months. The game uses AWS Lambda, Amazon SQS, and AWS CodeDeploy for its core backend service, Amazon DynamoDB as its primary game database, and Amazon Redshift as its data analytics platform. PUBG broke records upon release, with more than 3 million concurrent players connected to the game.

Nexon, a top Korean gaming company to produce top mobile games such as Heroes of Incredible Tales (HIT). They achieved cost savings of more than 30 percent for global infrastructure management and can now launch new games quicker by using AWS. Nexon uses Amazon DynamoDB for its game database and first started using AWS to respond to unpredictable spikes in user demand.

Startups to go Global
Lots of hot startups in Korea are using AWS to grow the local market, but here are great examples to go global although they are based on Korea.

Azar is Hyperconnect’s video-based social discovery mobile app recorded 300 million downloads and now widely accessible in over 200 countries around the world with 20 billion cumulative matches in last year. Overcoming complex matching issues for reliable video chats between users, Hyperconnect utilizes various AWS services efficiently, which uses Amazon EC2, Amazon RDS, and Amazon SES to save cost managing global infra, and Amazon S3 and Amazon CloudFront to store and deliver service data to global users faster. They also use Amazon EMR to manage the vast amount of data generated by 40 million matches per day.

SendBird provides chat APIs and messaging SDK in more than 10 thousand apps globally processing about 700 million messages per month. It uses AWS global regions to provide a top-class customer experience by keeping low latency under 100 ms everywhere in the world. Amazon ElastiCache is currently used to handle large volumes of chat data, and all the data are stored in the encrypted Amazon Aurora for integrity and reliability. Server log data are analyzed and processed using the Amazon Kinesis Data Firehose as well as Amazon Athena.

Freedom to Local Financial Industry
We also see Korean enterprises in the financial services industry leverage AWS to digitally transform their businesses by using data analytics, fintech, and digital banking initiatives. Financial services companies in Korea are leveraging AWS to deliver an enhanced customer experience, and examples of these customers include Shinhan Financial Group, KB Kookmin Bank, Kakao Pay, Mirae Asset, and Yuanta Securities.

Shinhan Financial Group achieved a 50 percent cost reduction and a 20 percent response-time reduction after migrating its North American and Japanese online banking services to AWS. Shinhan’s new Digital Platform unit now uses Amazon ECS, Amazon CloudFront, and other services to reduce development time for new applications by 50 percent. Shinhan is currently pursuing an all-in migration to AWS including moving more than 150 workloads.

Hyundai Card, a top Korean credit card company and a financial subsidiary of the Hyundai Kia Motor Group, built a dev/test platform called Playground on AWS to prototype new software and services by the development team. The customer uses Amazon EMR, AWS Glue, and Amazon Kinesis for cost and architecture optimization. It allowed quick testing of new projects without waiting for resource allocation from on-premises infrastructure, reducing the development period by 3-4 months

Security and Compliance
At AWS, the security, privacy, and protection of customer data always come first, which AWS provides local needs as well as global security and compliances. Our most recent example of this commitment is that AWS became the first global cloud service provider to achieve the Korea-Information Security Management System certification (K-ISMS) in December 2017. With this certification, enterprises and organizations across Korea are able to meet its compliance requirements more effectively and accelerate business transformation by using best-in-class technology delivered from the highly secure and reliable AWS Cloud. AWS also completed its first annual surveillance audit for the K-ISMS certification in 2018.

In April 2019, AWS achieved the Multi-Tier Cloud Security Standard (MTCS) Level-3 certification for Seoul region. AWS is also the first cloud service provider in Korea to do so. With the MTCS, FSI customers in Korea can accelerate cloud adoption by no longer having to validate 109 controls, as required in the relevant regulations (Financial Security Institute’s Guideline on Use of Cloud Computing Services in Financial Industry and the Regulation on Supervision on Electronic Financial Transactions (RSEFT). AWS also published a workbook for Korean FSI customer, covering those and 32 additional controls from the RSEFT.

What to support and enable Korean customers
AWS Korea has made significant investments in education and training in Korea. Tens of thousands of people including IT professionals, developers, and students have been trained in AWS cloud skills over the last two years. AWS Korea also supports community-driven activities to enhance the developer ecosystem of cloud computing in Korea. To date, the AWS Korean User Group has tens of thousands of members, who hold hundreds of meetups across Korea annually.

AWS Educate program is expected to accelerate Korean students’ capabilities in cloud computing skills, helping them acquire cloud expertise that is becoming increasingly relevant for their future employment. Tens of universities including Sogang University, Yonsei University, and Seoul National University have joined this program with thousands of students participating in AWS-related classes and non-profit e-learning programs such as Like a Lion, a non-profit organization that teaches coding to students.

AWS is building a vibrant cloud ecosystem with hundreds of partners ― Systems Integrator (SI) partners include LG CNS, Samsung SDS, Youngwoo Digital, Saltware, NDS, and many others. Among them, Megazone, GS Neotek, and Bespin Global are AWS Premier Consulting Partners. Independent Software Vendor (ISV) partners include AhnLab, Hancom, SK Infosec, SendBird, and IGAWorks. They help our customers to enable AWS services in their workloads to migrate from on-premise or launch new services.

The customer’s celebration whiteboard for 5th anniversary of AWS Summit Seoul

Finally, I want to introduce lots of customer’s feedback in our whiteboard of AWS Summit 2019 although they were written in Korean. Here is one voice from them ― “It made me decide to become an AWS customer voluntary to climb on the shoulders of the giant to see the world.” We always will hear customer’s voices and build the broadest and deepest cloud platform for them to leverage ours and be successful in both Korea and global market.

– Channy Yun;

This article was translated into Korean(한국어) in AWS Korea Blog.

Last week we made a fairly quiet (too quiet, in fact) announcement of our plan to slowly and carefully deprecate the path-based access model that is used to specify the address of an object in an S3 bucket. I spent some time talking to the S3 team in order to get a better understanding of the situation in order to write this blog post. Here’s what I learned…

We launched S3 in early 2006. Jeff Bezos’ original spec for S3 was very succinct – he wanted malloc (a key memory allocation function for C programs) for the Internet. From that starting point, S3 has grown to the point where it now stores many trillions of objects and processes millions of requests per second for them. Over the intervening 13 years, we have added many new storage options, features, and security controls to S3.

Old vs. New
S3 currently supports two different addressing models: path-style and virtual-hosted style. Let’s take a quick look at each one. The path-style model looks like either this (the global S3 endpoint):

https://s3.amazonaws.com/jbarr-public/images/ritchie_and_thompson_pdp11.jpeg
https://s3.amazonaws.com/jeffbarr-public/classic_amazon_door_desk.png

Or this (one of the regional S3 endpoints):

https://s3-us-east-2.amazonaws.com/jbarr-public/images/ritchie_and_thompson_pdp11.jpeg
https://s3-us-east-2.amazonaws.com/jeffbarr-public/classic_amazon_door_desk.png

In this example, jbarr-public and jeffbarr-public are bucket names; /images/ritchie_and_thompson_pdp11.jpeg and /jeffbarr-public/classic_amazon_door_desk.png are object keys.

Even though the objects are owned by distinct AWS accounts and are in different S3 buckets (and possibly in distinct AWS regions), both of them are in the DNS subdomain s3.amazonaws.com. Hold that thought while we look at the equivalent virtual-hosted style references (although you might think of these as “new,” they have been around since at least 2010):

https://jbarr-public.s3.amazonaws.com/images/ritchie_and_thompson_pdp11.jpeg
https://jeffbarr-public.s3.amazonaws.com/classic_amazon_door_desk.png

These URLs reference the same objects, but the objects are now in distinct DNS subdomains (jbarr-public.s3.amazonaws.com and jeffbarr-public.s3.amazonaws.com, respectively). The difference is subtle, but very important. When you use a URL to reference an object, DNS resolution is used to map the subdomain name to an IP address. With the path-style model, the subdomain is always s3.amazonaws.com or one of the regional endpoints; with the virtual-hosted style, the subdomain is specific to the bucket. This additional degree of endpoint specificity is the key that opens the door to many important improvements to S3.

Out with the Old
In response to feedback on the original deprecation plan that we announced last week, we are making an important change. Here’s the executive summary:

Original Plan – Support for the path-style model ends on September 30, 2020.

Revised Plan – Support for the path-style model continues for buckets created on or before September 30, 2020. Buckets created after that date must be referenced using the virtual-hosted model.

We are moving to virtual-hosted references for two reasons:

First, anticipating a world with billions of buckets homed in many dozens of regions, routing all incoming requests directly to a small set of endpoints makes less and less sense over time. DNS resolution, scaling, security, and traffic management (including DDoS protection) are more challenging with this centralized model. The virtual-hosted model reduces the area of impact (which we call the “blast radius” internally) when problems arise; this helps us to increase availability and performance.

Second, the team has a lot of powerful features in the works, many of which depend on the use of unique, virtual-hosted style subdomains. Moving to this model will allow you to benefit from these new features as soon as they are announced. For example, we are planning to deprecate some of the oldest security ciphers and versions (details to come later). The deprecation process is easier and smoother (for you and for us) if you are using virtual-hosted references.

In With the New
As just one example of what becomes possible when using virtual-hosted references, we are thinking about providing you with increased control over the security configuration (including ciphers and cipher versions) for each bucket. If you have ideas of your own, feel free to get in touch.

Moving Ahead
Here are some things to know about our plans:

Identifying Path-Style References – You can use S3 Access Logs (look for the Host Header field) and AWS CloudTrail Data Events (look for the host element of the requestParameters entry) to identify the applications that are making path-style requests.

Programmatic Access – If your application accesses S3 using one of the AWS SDKs, you don’t need to do anything, other than ensuring that your SDK is current. The SDKs already use virtual-hosted references to S3, except if the bucket name contains one or more “.” characters.

Bucket Names with Dots – It is important to note that bucket names with “.” characters are perfectly valid for website hosting and other use cases. However, there are some known issues with TLS and with SSL certificates. We are hard at work on a plan to support virtual-host requests to these buckets, and will share the details well ahead of September 30, 2020.

Non-Routable Names – Some characters that are valid in the path component of a URL are not valid as part of a domain name. Also, paths are case-sensitive, but domain and subdomain names are not. We’ve been enforcing more stringent rules for new bucket names since last year. If you have data in a bucket with a non-routable name and you want to switch to virtual-host requests, you can use the new S3 Batch Operations feature to move the data. However, if this is not a viable option, please reach out to AWS Developer Support.

Documentation – We are planning to update the S3 Documentation to encourage all developers to build applications that use virtual-host requests. The Virtual Hosting documentation is a good starting point.

We’re Here to Help
The S3 team has been working with some of our customers to help them to migrate, and they are ready to work with many more.

Our goal is to make this deprecation smooth and uneventful, and we want to help minimize any costs you may incur! Please do not hesitate to reach out to us if you have questions, challenges, or concerns.

— Jeff;

PS – Stay tuned for more information on tools and other resources.

Amazon’s Customer Obsession leadership principle says:

Leaders start with the customer and work backwards. They work vigorously to earn and keep customer trust. Although leaders pay attention to competitors, they obsess over customers.

Starting from the customer and working backwards means that we do not invent in a vacuum. Instead, we speak directly to our customers (both external and internal), ask detailed questions, and pay attention to what we learn. On the AWS side, we often hear about new use cases that help us to get a better understanding of what our customers are doing with AWS. For example, large-scale EC2 users provide us with another set of interesting data points, often expressed in terms of ratios between dollars, vCPUs, memory size, storage size, and networking throughput.

We launched the I3 instances (Now Available – I3 Instances for Demanding, I/O Intensive Workloads) just about two years ago. Our customers use them to host distributed file systems, relational & NoSQL databases, in-memory caches, key-value stores, data warehouses, and MapReduce clusters. Because our customers are always (in Jeff Bezos’ words) “divinely discontent”, they want I/O-optimized instances with even more power & storage. To be specific, they have asked us for:

• A lower price per TB of storage
• Increased storage density to allow consolidation of workloads and scale-up processing
• A higher ratio of network bandwidth and instance storage to vCPUs

The crucial element here is that our customers were able to express their needs in a detailed and specific manner. Simply asking for something to be better, faster, and cheaper does not help us to make well-informed decisions.

New I3en Instances
Today I am happy to announce the I3en instances. Designed to meet these needs and to do an even better job of addressing the use cases that I mentioned above, these instances are powered by AWS-custom Intel Xeon Scalable (Skylake) processors with 3.1 GHz sustained all-core turbo performance, up to 60 TB of fast NVMe storage, and up to 100 Gbps of network bandwidth. Here are the specs:

In comparison to the I3 instances, the I3en instances offer:

• A cost per GB of SSD instance storage that is up to 50% lower
• Storage density (GB per vCPU) that is roughly 2.6x greater
• Ratio of network bandwidth to vCPUs that is up to 2.7x greater

You will need HVM AMIs with the NVMe 1.0e and ENA drivers. You can also make use of the new Elastic Fabric Adapter (EFA) if you are using the i3en.24xlarge (read my recent post to learn more).

Now Available
You can launch I3en instances today in the US East (N. Virginia), US West (Oregon), US West (N. California), Europe (Ireland), Asia Pacific (Tokyo), Asia Pacific (Singapore), and Europe (Frankfurt) Regions in On-Demand and Spot form. Reserved Instances, Dedicated Instances, and Dedicated Hosts are available.

— Jeff;

Join us this May to learn about AWS services and solutions. The AWS Online Tech Talks are live, online presentations that cover a broad range of topics at varying technical levels. These tech talks, led by AWS solutions architects and engineers, feature technical deep dives, live demonstrations, customer examples, and Q&A with AWS experts. Register Now!

Note – All sessions are free and in Pacific Time.

Tech talks this month:

Compute

May 30, 2019 | 11:00 AM – 12:00 PM PT – Your First HPC Cluster on AWS – Get a step-by-step walk-through of how to set up your first HPC cluster on AWS.

Containers

May 20, 2019 | 11:00 AM – 12:00 PM PT – Managing Application Deployments with the AWS Service Operator – Learn how the AWS Service Operator helps you provision AWS resources and your applications using kubectl CLI.

May 22, 2019 | 9:00 AM – 10:00 AM PT – Microservice Deployment Strategies with AWS App Mesh – Learn how AWS App Mesh makes the process of deploying microservices on AWS easier and more reliable while providing you with greater control and visibility to support common deployment patterns.

Data Lakes & Analytics

May 20, 2019 | 9:00 AM – 10:00 AM PT – EKK is the New ELK: Aggregating, Analyzing and Visualizing Logs – Learn how to aggregate, analyze, & visualize your logs with Amazon Elasticsearch Service, Amazon Kinesis Data Firehose, and Kibana

May 22, 2019 | 11:00 AM – 12:00 PM PT – Building a Data Streaming Application Leveraging Apache Flink – Learn how to build and manage a real-time streaming application with AWS using Java and leveraging Apache Flink.

Databases

May 20, 2019 | 1:00 PM – 2:00 PM PT – Migrating to Amazon DocumentDB – Learn how to migrate MongoDB workloads to Amazon DocumentDB, a fully managed MongoDB compatible database service designed from the ground up to be fast, scalable, and highly available.

May 29, 2019 | 1:00 PM – 2:00 PM PT – AWS Fireside Chat: Optimize Your Business Continuity Strategy with Aurora Global Database – Join us for a discussion with the General Manager of Amazon Aurora to learn how you can use Aurora Global Database to build scalable and reliable apps.

DevOps

May 21, 2019 | 9:00 AM – 10:00 AM PT – Infrastructure as Code Testing Strategies with AWS CloudFormation – Learn about CloudFormation testing best practices, including what tools to use and when, both while authoring code and while testing in a continuous integration and continuous delivery pipeline.

End-User Computing

May 23, 2019 | 1:00 PM – 2:00 PM PT – Managing Amazon Linux WorkSpaces at Scale using AWS OpsWorks for Chef Automate – Learn how to simplify your Amazon Linux Workspaces management at scale by integrating with AWS OpsWorks for Chef Automate.

Enterprise & Hybrid

May 28, 2019 | 11:00 AM – 12:00 PM PT – What’s New in AWS Landing Zone – Learn how the AWS Landing Zone can automate the setup of best practice baselines when setting up new AWS environments.

IoT

May 29, 2019 | 11:00 AM – 12:30 PM PT – Customer Showcase: Extending Machine Learning to Industrial IoT Applications at the Edge – Learn how AWS IoT customers are building smarter products and bringing them to market quickly with Amazon FreeRTOS and AWS IoT Greengrass.

Machine Learning

May 28, 2019 | 9:00 AM – 10:00 AM PT – Build a Scalable Architecture to Automatically Extract and Import Form Data – Learn how to build a scalable architecture to process thousands of forms or documents with Amazon Textract.

May 30, 2019 | 1:00 PM – 2:00 PM PT – Build Efficient and Accurate Recommendation Engines with Amazon Personalize – Learn how to build efficient and accurate recommendation engines with high impact to the bottom line of your business.

Mobile

May 21, 2019 | 1:00 PM – 2:00 PM PT – Deploying and Consuming Serverless Functions with AWS Amplify – Learn how to build and deploy serverless applications using React and AWS Amplify.

Networking & Content Delivery

May 31, 2019 | 1:00 PM – 2:00 PM PT – Simplify and Scale How You Connect Your Premises to AWS with AWS Direct Connect on AWS Transit Gateway – Learn how to use multi account Direct Connect gateway to interface your on-premises network with your AWS network through a AWS Transit Gateway.

Security, Identity, & Compliance

May 30, 2019 | 9:00 AM – 10:00 AM PT – Getting Started with Cross-Account Encryption Using AWS KMS, Featuring Slack Enterprise Key Management – Learn how to manage third-party access to your data keys with AWS KMS.

May 31, 2019 | 9:00 AM – 10:00 AM PT – Authentication for Your Applications: Getting Started with Amazon Cognito – Learn how to use Amazon Cognito to add sign up and sign in to your web or mobile application.

Serverless

May 22, 2019 | 1:00 PM – 2:00 PM PT – Building Event-Driven Serverless Apps with AWS Event Fork Pipelines – Learn how to use Event Fork Pipelines, a new suite of open-source nested applications in the serverless application repository, to easily build event driven apps.

Storage

May 28, 2019 | 1:00 PM – 2:00 PM PT – Briefing: AWS Hybrid Cloud Storage and Edge Computing – Learn about AWS hybrid cloud storage and edge computing capabilities.

May 23, 2019 | 11:00 AM – 12:00 PM PT – Managing Tens to Billions of Objects at Scale with S3 Batch Operations – Learn about S3 Batch Operations and how to manage billions of objects with a single API request in a few clicks.

SAP SAPPHIRE NOW 2019 takes place this week in Florida! Many of my AWS colleagues will be there, and they would love to talk to you. Today, I would like to share some customer success stories and give you a brief update on the work that we have been doing to make sure that AWS is the best place for you to run your SAP-powered OLTP and OLAP enterprise workloads.

Customer Update
Let’s start with a quick recap of some recent customer success stories. Here are just a few of the many customers that are using SAP on AWS in production today:

Fiat Chrysler Automotive – After exploring multiple options and vendors, FIAT decided to deploy SAP on AWS with Capgemini as a partner:

Engie – Read the case study to learn how this international energy provider has been able to Transform and Streamline their Financial Processes and drastically reduced the ramp-up time for new users from three days to one day by running SAP S/4HANA on AWS:

AIG – Watch the video to learn how AIG migrated 13 SAP landscapes from an on-premises environment to SAP HANA on AWS in 13 months, while reducing their infrastructure cost by $8M:

Sumitomo Chemical – Read this case study to learn how Sumitomo Chemical runs a huge number of SAP ERP batch jobs on AWS, cutting job processing time by around 40%:

There are additional SAP on AWS Case Studies for your reading pleasure!

AWS customers are making great use of the 6, 9, and 12 TB EC2 High Memory instances that we launched last year. They are building many different SAP Solutions on AWS, taking advantage of the SAP Rapid Migration Test Program, and working with members of the SAP Competency Partner Network.

What’s New
Our customers are building ever-larger SAP installations, using both scale-up (larger instances) or scale-out (more instances) models. We have been working with SAP to certify two additional scale-out options:

48 TB Scale-Out (S/4HANA) – When we launched the EC2 High Memory instances with 12 TB of memory last year, they were certified by SAP to run OLTP and OLAP HANA workloads in scale-up configurations. These instances now support additional configuration choices for your OLTP workloads. You can now use up to four of these 12 TB High Memory instances to run an OLTP S/4HANA solution in scale-out mode, while meeting all SAP requirements.

This is the first ever SAP-certified scale-out certification of S/4HANA on cloud instances. SAP recommends (SAP OSS Note 2408419) the use of bare metal platforms with a minimum of 8 CPUs and 6 TB of memory for running S/4HANA in scale-out. Since the EC2 High Memory instances with 12 TB memory is an EC2 bare metal instance that combines the benefits of the cloud with the performance characteristics of a bare metal platform, it is able to support SAP-certified scale-out configurations for S/4HANA in the cloud. To learn more, read Announcing support for extremely large S/4HANA deployments on AWS and review the certification.

100 TB Scale-Out (BW4/HANA, BW on HANA, Datamart) – You can now use up to 25 x1e.32xlarge EC2 instances (thanks to TDI Phase 5) to create an OLAP solution that scales to 100 TB, again while meeting all SAP requirements. You can start with as little as 244 GB of memory and scale out to 100 TB; review the certification to learn more.

The 48 TB OLTP solution and the 100 TB OLAP solution are the largest SAP-certified solutions available from any cloud provider.

We also have a brand-new S/4HANA Quick Start to help you get going in minutes. It sets up a VPC that spans two Availability Zones, each with a public and private subnet, and a pair of EC2 instances. One instance hosts the primary copy of S/4HANA and the other hosts the secondary. Read the Quick Start to learn more:

What’s Next
Ok, still with me? I hope so, since I have saved the biggest news for last!

We are getting ready to extend our lineup of EC2 High Memory instances, and will make them available with 18 TB and 24 TB of memory in the fall of 2019. The instances will use second-generation Intel^® Xeon^® Scalable processors, and will be available in bare metal form. Like the existing EC2 High Memory instances, you will be able to run them in the same Virtual Private Cloud (VPC) that hosts your cloud-based business applications, and you will be able to make use of important EC2 features such as Elastic Block Store and advanced networking. You can launch, manage, and even resize these EC2 instances using the AWS Command Line Interface (CLI) and the AWS SDKs.

Here are screen shots of SAP HANA Studio running on 18 TB and 24 TB instances that are currently in development:

And here is the output from top on those instances:

Here is a handy reference to all of your scale-up and scale-out SAP HANA on AWS options:

If you want to learn more or you want to gain early access to the new instances, go ahead and contact us.

— Jeff;

Guest post by AWS Serverless Hero Forrest Brazeal. Forrest is a senior cloud architect at Trek10, Inc., host of the Think FaaS serverless podcast at Trek10, and a regular speaker at workshops and events in the serverless community.

Events and serverless go together like baked beans and barbecue. The serverless mindset says to focus on code and configuration that provide business value. It turns out that much of the time, this means working with events: structured data corresponding to things that happen in the outside world. Rather than maintaining long-running server tasks that chew up resources while polling, I can create serverless applications that do work only in response to event triggers.

I have lots of options when working with events in AWS: Amazon Kinesis Data Streams, Amazon Simple Notification Service (SNS), Amazon Simple Queue Service (SQS), and more, depending on my requirements. Lately, I’ve been using a service more often that has the word ‘event’ right in the name: Amazon CloudWatch Events.

CloudWatch Events: The best-kept secret in serverless event processing

I first knew CloudWatch as the service that collects my Lambda logs and lets me run functions on a schedule. But CloudWatch Events also lets me publish my own custom events using the CloudWatch API. It has similar pricing and delivery guarantees to SNS, and supports a whole bunch of AWS services as targets.

Best of all, I don’t even have to provision the event bus—it’s just there in the CloudWatch console. I can publish an event now, using the boto3 AWS SDK for Python:

import boto3
cw = boto3.client('cloudwatch')
cw.put_events(
    Entries=[
        {
            'Source': 'my.app.event',
            'DetailType': 'MY_EVENT_TYPE',
            'Detail': '{"my_data":"As a JSON string"}'
        }
    ]
)

In short, CloudWatch Events gives me a fully managed event pipe that supports an arbitrary number of consumers, where I can drop any kind of JSON string that I want. And this is super useful for building serverless apps.

Event-driven architectures in action

I build cloud-native solutions for clients at Trek10 daily. I frequently use event-driven architectures as a powerful way to migrate legacy systems to serverless, enable easier downstream integrations, and more. Here are just a couple of my favorite patterns:
• Strangling legacy databases
• Designing event-sourced applications

Strangling legacy databases

The “strangler pattern” hides a legacy system behind a wrapper API, while gradually migrating users to the new interface. Trek10 has written about this before. Streaming changes to the cloud as events is a great way to open up reporting and analytics use cases while taking load off a legacy database. The following diagram shows writing a legacy database to events.

This pattern can also work the other way: I can write new data to CloudWatch Events, consume it into a modern data source, and create a second consumer that syncs the data back to my legacy system.

Designing event-sourced applications

Event sourcing simply means treating changes in the system state as events, publishing them on a ledger or bus where they can be consumed by different downstream applications.

Using CloudWatch Events as a centralized bus, I can make a sanitized record of events available as shown in the following event validation flow diagram.

The validation function ensures that only events that match my application’s standards get tagged as “valid” and are made available to downstream consumers. The default bus handles lots of events (remember, service logs go here!), so it’s important to set up rules that only match the events that I care about.

CloudWatch Events simplifies these patterns by providing a single bus where I can apply filters and subscribe consumers, all without having to provision or maintain any infrastructure. And that’s just the beginning.

Use case: Multi-account event processing with CloudWatch Events

CloudWatch Events gets most interesting when I start connecting multiple AWS accounts. It’s easy to set up a trust relationship between CloudWatch Event buses in different accounts, using filtering rules to choose which events get forwarded.

As an example, imagine a widget processing system for a large enterprise, AnyCompany. AnyCompany has many different development teams, each using their own AWS account. Some services are producing information about widgets as they check into warehouses or travel across the country. Others need that data to run reports or innovate new products.

Suppose that Service A produces information about new widgets, Service B wants to view aggregates about widgets in real time, and Service C needs historical data about widgets for reporting. The full event flow might look like the following diagram.

1. Service A publishes the new widget event to CloudWatch Events in their AWS account with the following event body:

{
            'Source': 'cwi.servicea',
            'DetailType': 'NEW_WIDGET',
            'Detail': '{"widget_id":"abc123"}'
}

2. A filtering rule forwards events tagged with cwi.servicea to the central event processing account. Using CloudFormation, they could define the rule as follows:

CentralForwardingRule:
    Type: AWS::Events::Rule
    Properties:
      Description: Rule for sending events to central account
      EventPattern:
        source:
          - cwi.servicea
      Targets:
        - Arn: !Sub arn:aws:events:${CENTRAL_ACCOUNT_REGION}:${CENTRAL_ACCOUNT_ID}:event-bus/default
          Id: CentralTarget
          RoleArn: <IAM ROLE WITH ACCESS TO PUT CW EVENTS> 

3. The event is validated according to their standards.

4. The valid event is republished on the central event bus with a new source, valid.cw.servicea. This is important because, to avoid infinite loops, an individual event can only be forwarded one time.

5. A filtering rule forwards the valid event to Service B’s AWS account, where it updates a DynamoDB table connected to an AWS AppSync API.

6. A second rule forwards the same event to the Service C account, where it flows through Kinesis Data Firehose to an Amazon S3 bucket for analysis using Amazon Athena.

What CloudWatch Events provides here is a decoupled system that uses mostly “plug-and-play” services, and yet opens up flexibility for future innovation.

Taking full advantage of the cloud

The biggest reason I love CloudWatch Events? It’s a fantastically cloud-native service. There’s little code required, and no operational responsibilities beyond watching AWS service limits. I don’t even have to deploy anything to start using it. And yet, under the hood, I’m using a powerful piece of AWS infrastructure that can support complex distributed applications without breaking a sweat.

That’s pretty close to the platonic ideal of serverless apps. Anytime I’m cooking up an event-driven application, I make sure to consider what CloudWatch Events can bring to the table.

Last year I showed you how to Use an AWS Transit Gateway to Simplify Your Network Architecture. As I said at the time:

You can connect your existing VPCs, data centers, remote offices, and remote gateways to a managed Transit Gateway, with full control over network routing and security, even if your VPCs, Active Directories, shared services, and other resources span multiple AWS accounts. You can simplify your overall network architecture, reduce operational overhead, and gain the ability to centrally manage crucial aspects of your external connectivity, including security. Last but not least, you can use Transit Gateways to consolidate your existing edge connectivity and route it through a single ingress/egress point.

In that post I also promised you support for AWS Direct Connect, and I’m happy to announce that this support is available today for use in the US East (N. Virginia), US East (Ohio), US West (N. California), and US West (Oregon) Regions. The applications that you run in the AWS Cloud can now communicate with each other, and with your on-premises applications, at speeds of up to 10 Gbps per Direct Connect connection. You can set it up in minutes (assuming that you already have a private or hosted connection running at 1 Gbps or more) and start using it right away.

Putting it all together, you get a lot of important benefits from today’s launch:

Simplification – You can simplify your network architecture and your network management overhead by creating a hub-and-spoke model that spans multiple VPCs, regions, and AWS accounts. If you go this route, you may also be in a position to cut down on the number of AWS VPN connections that you use.

Consolidation – You have the opportunity to reduce the number of private or hosted connections, saving money and avoiding complexity in the process. You can consolidate your connectivity so that it all flows across the same BGP session.

Connectivity – You can reach your Transit Gateway using your connections from any of the 90+ AWS Direct Connect locations (except from AWS Direct Connect locations in China).

Using Transit Gateway & Direct Connect
I will use the freshly updated Direct Connect Console to set up my Transit Gateway for use with Direct Connect. The menu on the left lets me view and create the resources that I will need:

My AWS account already has access to a 1 Gbps connection (MyConnection) to TierPoint in Seattle:

I create a Direct Connect Gateway (MyDCGateway):

I create a Virtual Interface (VIF) with type Transit:

I reference my Direct Connect connection (MyConnection) and my Direct Connect Gateway (MyDCGateway) and click Create virtual interface:

When the state of my new VIF switches from pending to down I am ready to proceed:

Now I am ready to create my transit gateway (MyTransitGW). This is a VPC component; clicking on Transit gateways takes me to the VPC console. I enter a name, description, and ASN (which must be distinct from the one that I used for the Direct Connect Gateway), leave the other values as-is, and click Create Transit Gateway:

The state starts out as pending, and transitions to available:

With all of the resources ready, I am ready to connect them! I return to the Direct Connect Console, find my Transit Gateway, and click Associate Direct Connect gateway:

I associate the Transit Gateway with a Direct Connect Gateway in my account (using another account requires the ID of the gateway and the corresponding AWS account number), and list the network prefixes that I want to advertise to the other side of the Direct Connect connection. Then I click Associate Direct Connect gateway to make it so:

The state starts out as associating and transitions to associated. This can take some time, so I will take Luna for a walk:

By the time we return, the Direct Connect Gateway is associated with the Transit Gateway, and we are good to go!

In a real-world situation you would spend more time planning your network topology and addressing, and you would probably use multiple AWS accounts.

Available Now
You can use this new feature today to interface with your Transit Gateways hosted in four AWS regions.

— Jeff;

Trust is a wonderful thing, and is the basis for almost every business and personal relationship or transaction. In some cases, trust is built up over an extended period of time, reinforced with each successful transaction and seen as an integral part of the relationship. In other situations, there’s no time to accumulate trust and other mechanisms must be used instead. The parties must find a way to successfully complete the transaction in the absence of trust. Today, emerging blockchain technologies such as Hyperledger Fabric and Ethereum fill this important need, allowing parties to come to consensus regarding the validity of a proposed transaction and create an unalterable digital record (commonly known as a ledger) of each transaction in the absence of trust.

Amazon Managed Blockchain
We announced Amazon Managed Blockchain at AWS re:Invent 2018 and invited you to sign up for a preview. I am happy to announce that the preview is complete and that Amazon Managed Blockchain is now available for production use in the US East (N. Virginia) Region. You can use it to create scalable blockchain networks that use the Hyperledger Fabric open source framework, with Ethereum in the works. As you will see in a minute, you can create your network in minutes. Once created, you can easily manage and maintain your blockchain network. You can manage certificates, invite new members, and scale out peer node capacity in order to process transactions more quickly.

The blockchain networks that you create with Amazon Managed Blockchain can span multiple AWS accounts so that a group of members can execute transactions and share data without a central authority. New members can easily launch and configure peer nodes that process transaction requests and store a copy of the ledger.

Using Amazon Managed Blockchain
I can create my own scalable blockchain network from the AWS Management Console, AWS Command Line Interface (CLI) (aws managedblockchain create-network), or API (CreateNetwork). To get started, I open the Amazon Managed Blockchain Console and click Create a network:

I need to choose the edition (Starter or Standard) for my network. The Starter Edition is designed for test networks and small production networks, with a maximum of 5 members per network and 2 peer nodes per member. The Standard Edition is designed for scalable production use, with up to 14 members per network and 3 peer nodes per member (check out the Amazon Managed Blockchain Pricing to learn more about both editions). I also enter a name and a description for my network:

Then I establish the voting policy for my network, and click Next to move ahead (read Work with Proposals to learn more about creating and voting on proposals):

Now, I need to create the first member of my network. Each member is a distinct identity within the network, and is visible within the network. I also set up a user name and password for my certificate authority, and click Next:

I review my choices, and click Create network and member:

My network enters the Creating status, and I take a quick break to walk my dog! When I return, my network is Available:

Inviting Members
Now that my network is available, I can invite members by clicking the Members tab:

I can see the current members of my network, both those I own and those owned by others. I click on Propose invitation to invite a new member:

Then I enter the AWS account number of the proposed member and click Create:

This creates a proposal (visible to me and to the other members of the network). I click on the ID to proceed:

I review the proposal, select my identity (block-wizard), and then click Yes to vote:

After enough Yes votes have been received to pass the threshold that I specified when I created the network, the invitation will be extended to the new member, and will be visible in the Invitations section:

If you are building a blockchain network for testing purposes and don’t have access to multiple AWS accounts, you can even invite your own account. After you do this (and vote to let yourself in), you will end up with multiple members in the same account.

Using the Network
Now that the network is running, and has some members, the next step is to create an endpoint in the Virtual Private Cloud (VPC) where I will run my blockchain applications (this feature is powered by AWS PrivateLink). Starting from the detail page for my network, I click Create VPC endpoint:

I choose the desired VPC and the subnets within it, pick a security group, and click Create:

My applications can use the VPC endpoint to communicate with my blockchain network:

The next step is to build applications that make use of the blockchain. To learn how to do this, read Build and deploy an application for Hyperledger Fabric on Amazon Managed Blockchain. You can also read Get Started Creating a Hyperledger Fabric Blockchain Network Using Amazon Managed Blockchain.

Things to Know
As usual, we have a healthy roadmap for this new service. Stay tuned to learn more!

— Jeff;

PS – Check out the AWS Blockchain Pub to see a novel use for Amazon Managed Blockchain and AWS DeepLens.

We announced Elastic Fabric Adapter (EFA) at re:Invent 2018 and made it available in preview form at the time. During the preview, AWS customers put EFA through its paces on a variety of tightly-coupled HPC workloads, providing us with valuable feedback and helping us to fine-tune the final product.

Now Available
Today I am happy to announce that EFA is now ready for production use in multiple AWS regions. It is ready to support demanding HPC workloads that need lower and more consistent network latency, along with higher throughput, than is possible with traditional TCP communication. This launch lets you apply the scale, flexibility, and elasticity of the AWS Cloud to tightly-coupled HPC apps and I can’t wait to hear what you do with it. You can, for example, scale up to thousands of compute nodes without having to reserve the hardware or the network ahead of time.

All About EFA
An Elastic Fabric Adapter is an AWS Elastic Network Adapter (ENA) with added capabilities (read my post, Elastic Network Adapter – High Performance Network Interface for Amazon EC2, to learn more about ENA). An EFA can still handle IP traffic, but also supports an important access model commonly called OS bypass. This model allows the application (most commonly through some user-space middleware) access the network interface without having to get the operating system involved with each message. Doing so reduces overhead and allows the application to run more efficiently. Here’s what this looks like (source):

The MPI Implementation and libfabric layers of this cake play crucial roles:

MPI – Short for Message Passing Interface, MPI is a long-established communication protocol that is designed to support parallel programming. It provides functions that allow processes running on a tightly-coupled set of computers to communicate in a language-independent way.

libfabric – This library fits in between several different types of network fabric providers (including EFA) and higher-level libraries such as MPI. EFA supports the standard RDM (reliable datagram) and DGRM (unreliable datagram) endpoint types; to learn more, check out the libfabric Programmer’s Manual. EFA also supports a new protocol that we call Scalable Reliable Datagram; this protocol was designed to work within the AWS network and is implemented as part of our Nitro chip.

Working together, these two layers (and others that can be slotted in instead of MPI), allow you to bring your existing HPC code to AWS and run it with little or no change.

You can use EFA today on c5n.18xlarge and p3dn.24xlarge instances in all AWS regions where those instances are available. The instances can use EFA to communicate within a VPC subnet, and the security group must have ingress and egress rules that allow all traffic within the security group to flow. Each instance can have a single EFA, which can be attached when an instance is started or while it is stopped.

You will also need the following software components:

EFA Kernel Module – The EFA Driver is in the Amazon GitHub repo, and in the Amazon Linux & Amazon Linux 2 AMIs. We are working to add it to AMIs for other Linux distributions.

Libfabric Network Stack – You will need to use an AWS-custom version (already present in the Amazon Linux and Amazon Linux 2 AMIs) for now. We are working to get our changes into the next release (1.8) of libfabric.

MPI or NCCL Implementation – You can use Open MPI 3.1.3 (or later) or NCCL (2.3.8 or later) plus the OFI driver for NCCL. We also also working on support for the Intel MPI library.

You can launch an instance and attach an EFA using the CLI, API, or the EC2 Console, with CloudFormation support coming in a couple of weeks. If you are using the CLI, you need to include the subnet ID and ask for an EFA, like this (be sure to include the appropriate security group):

$ aws ec2 run-instances ... 
  --network-interfaces DeleteOnTermination=true,DeviceIndex=0,SubnetId=SUBNET,InterfaceType=efa

After your instance has launched, run lspci | grep efa0 to verify that the EFA device is attached. You can (but don’t have to) launch your instances in a Cluster Placement Group in order to benefit from physical adjacency when every light-foot counts. When used in this way, EFA can provide one-way MPI latency of 15.5 microseconds.

You can also create a Launch Template and use it to launch EC2 instances (either directly or as part of an EC2 Auto Scaling Group) in On-Demand or Spot Form, launch Spot Fleets, and to run compute jobs on AWS Batch.

Learn More
To learn more about EFA, and to see some additional benchmarks, be sure to watch this re:Invent video (Scaling HPC Applications on EC2 w/ Elastic Fabric Adapter):

AWS Customer CFD Direct maintains the popular OpenFOAM platform for Computational Fluid Dynamics (CFD) and also produces CFD Direct From the Cloud (CFDDC), an AWS Marketplace offering that makes it easy for you to run OpenFOAM on AWS. They have been testing and benchmarking EFA and recently shared their measurements in a blog post titled OpenFOAM HPC with AWS EFA. In the post, they report on a pair of simulations:

External Aerodynamics Around a Car – This simulation scales extra-linearly to over 200 cores, gradually declining to linear scaling at 1000 cores (about 100K simulation cells per core).

Flow Over a Weir with Hydraulic Jump – This simulation (1000 cores and 100M cells) scales at between 67% and 72.6%, depending on a “data write” setting.

Read the full post to learn more and to see some graphs and visualizations.

In the Works
We plan to add EFA support to additional EC2 instance types over time. In general, we plan to provide EFA support for the two largest sizes of “n” instances of any given type, and also for bare metal instances.

— Jeff;