Amazon MSK Serverless Now Generally Available–No More Capacity Planning for Your Managed Kafka Clusters

This post was originally published on this site

Today we are making Amazon MSK Serverless generally available to help you reduce even more the operational overhead of managing an Apache Kafka cluster by offloading the capacity planning and scaling to AWS.

In May 2019, we launched Amazon Managed Streaming for Apache Kafka to help our customers stream data using Apache Kafka. Apache Kafka is an open-source platform that enables customers to capture streaming data like clickstream events, transactions, and IoT events. Apache Kafka is a common solution for decoupling applications that produce streaming data (producers) from those consuming the data (consumers). Amazon MSK makes it easy to ingest and process streaming data in real time with fully managed Apache Kafka clusters.

Amazon MSK reduces the work needed to set up, scale, and manage Apache Kafka in production. With Amazon MSK, you can create a cluster in minutes and start sending data. Apache Kafka runs as a cluster on one or more brokers. Brokers are instances with a given compute and storage capacity distributed in multiple AWS Availability Zones to create high availability. Apache Kafka stores records on topics for a user-defined period of time, partitions those topics, and then replicates these partitions across multiple brokers. Data producers write records to topics, and consumers read records from them.

When creating a new Amazon MSK cluster, you need to decide the number of brokers, the size of the instances, and the storage that each broker has available. The performance of an MSK cluster depends on these parameters. These settings can be easy to provide if you already know the workload. But how will you configure an Amazon MSK cluster for a new workload? Or for an application that has variable or unpredictable data traffic?

Amazon MSK Serverless
Amazon MSK Serverless automatically provisions and manages the required resources to provide on-demand streaming capacity and storage for your applications. It is the perfect solution to get started with a new Apache Kafka workload where you don’t know how much capacity you will need or if your applications produce unpredictable or highly variable throughput and you don’t want to pay for idle capacity. Also, it is great if you want to avoid provisioning, scaling, and managing resource utilization of your clusters.

Amazon MSK Serverless comes with a lot of secure features out of the box, such as private connectivity. This means that the traffic doesn’t leave the AWS backbone, AWS Identity and Access Management (IAM) access control, and encryption of your data at rest and in transit, which keeps it secure.

An Amazon MSK Serverless cluster scales capacity up and down instantly based on the application requirements. When Apache Kafka clusters are scaled horizontally (that is, more brokers are added), you also need to move partitions to these new brokers to make use of the added capacity. With Amazon MSK Serverless, you don’t need to scale brokers or do partition movement.

Each Amazon MSK Serverless cluster provides up to 200 MBps of write-throughput and 400 MBps of read-throughput. It also allocates up to 5 MBps of write-throughput and 10 MBps of read-throughput per partition.

Amazon MSK Serverless pricing is based on throughput. You can learn more on the MSK’s pricing page.

Let’s see it in action
Imagine that you are the architect of a mobile game studio, and you are about to launch a new game. You invested in the game’s marketing, and you expect it will have a lot of new players. Your games send clickstream data to your backend application. The data is analyzed in real time to produce predictions on your players’ behaviors. With these predictions, your games make real-time offers that suit the current player’s behavior, encouraging them to stay in the game longer.

Your games send clickstream data to an Apache Kafka cluster. As you are using an Amazon MSK Serverless cluster, you don’t need to worry about scaling the cluster when the new game launches, as it will adjust its capacity to the throughput.

In the following image, you can see a graph of the day of the launch of the new game. It shows in orange the metric MessagesInPerSec that the cluster is consuming. And you can see that the number of messages per second is increasing first from 100, which is our base number before the launch. Then it increases to 300, 600, and 1,000 messages per second, as our game is getting downloaded and played by more and more players. You can feel confident that the volume of records can keep increasing. Amazon MSK Serverless is capable of ingesting all the records as long as your application throughput stays within the service limits.

Graph of messages in per second to the cluser

How to get started with Amazon MSK Serverless
Creating an Amazon MSK Serverless cluster is very simple, as you don’t need to provide any capacity configuration to the service. You can create a new cluster on the Amazon MSK console page.

Choose the Quick create cluster creation method. This method will provide you with the best-practice settings to create a starter cluster and input a name for your cluster.

Create a cluster

Then, in the General cluster properties, choose the cluster type. Choose the Serverless option to create an Amazon MSK Serverless cluster.

General cluster properties

Finally, it shows all the cluster settings that it will configure by default. You cannot change most of these settings after the cluster is created. If you need different values for these settings, you might need to create the cluster using the Custom create method. If the default settings work for you, then create the cluster.

Cluster settings page

Creating the cluster will take you a few minutes, and after that, you see the Active status on the Cluster summary page.

Cluster information page

Now that you have the cluster, you can start sending and receiving records using an Amazon Elastic Compute Cloud (Amazon EC2) instance. For doing that, the first step is to create a new IAM policy and IAM role. The instances need to authenticate using IAM in order to access the cluster from the instances.

Amazon MSK Serverless integrates with IAM to provide fine-grained access control to your Apache Kafka workloads. You can use IAM policies to grant least privileged access to your Apache Kafka clients.

Create the IAM policy
Create a new IAM policy with the following JSON. This policy will give permissions to connect to the cluster, create a topic, send data, and consume data from the topic.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:Connect"
            ],
            "Resource": "arn:aws:kafka:<REGION>:<ACCOUNTID>:cluster/msk-serverless-tutorial/cfeffa15-431c-4af4-8725-42636fab9937-s3"
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:DescribeTopic",
                "kafka-cluster:CreateTopic",
                "kafka-cluster:WriteData",
                "kafka-cluster:ReadData"
            ],
            "Resource": "arn:aws:kafka:<REGION>:<ACCOUNTID>:topic/msk-serverless-tutorial/cfeffa15-431c-4af4-8725-42636fab9937-s3/msk-serverless-tutorial"
        },
        {
            "Effect": "Allow",
            "Action": [
                "kafka-cluster:AlterGroup",
                "kafka-cluster:DescribeGroup"
            ],
            "Resource": "arn:aws:kafka:<REGION>:<ACCOUNTID>:group/msk-serverless-tutorial/cfeffa15-431c-4af4-8725-42636fab9937-s3/*"
        }
    ]
}

Make sure that you replace the Region and account ID with your own. Also, you need to replace the cluster, topic, and group ARN. To get these ARNs, you can go to the cluster summary page and get the cluster ARN. The topic ARN and the group ARN are based on the cluster ARN. Here, the cluster and the topic are named msk-serverless-tutorial.

"arn:aws:kafka:<REGION>:<ACCOUNTID>:cluster/msk-serverless-tutorial/cfeffa15-431c-4af4-8725-42636fab9937-s3"
"arn:aws:kafka:<REGION>:<ACCOUNTID>:topic/msk-serverless-tutorial/cfeffa15-431c-4af4-8725-42636fab9937-s3/msk-serverless-tutorial"
"arn:aws:kafka:<REGION>:<ACCOUNTID>:group/msk-serverless-tutorial/cfeffa15-431c-4af4-8725-42636fab9937-s3/*"

Then create a new role with the use case EC2 and attach this policy to the role.

Create a new role

Create a new EC2 instance
Now that you have the cluster and the role, create a new Amazon EC2 instance. Add the instance to the same VPC, subnet, and security group as the cluster. You can find that information on your cluster properties page in the networking settings. Also, when configuring the instance, attach the role that you just created in the previous step.

Cluster networking configuration

When you are ready, launch the instance. You are going to use the same instance to produce and consume messages. To do that, you need to set up Apache Kafka client tools in the instance. You can follow the Amazon MSK developer guide to get your instance ready.

Producing and consuming records
Now that you have everything configured, you can start sending and receiving records using Amazon MSK Serverless. The first thing you need to do is to create a topic. From your EC2 instance, go to the directory where you installed the Apache Kafka tools and export the bootstrap server endpoint.

cd kafka_2.13-3.1.0/bin/
export BS=boot-abc1234.c3.kafka-serverless.us-east-2.amazonaws.com:9098

As you are using Amazon MSK Serverless, there is only one address for this server, and you can find it in the client information on your cluster page.

Viewing client information

Run the following command to create a topic with the name msk-serverless-tutorial.

./kafka-topics.sh --bootstrap-server $BS 
--command-config client.properties 
--create --topic msk-serverless-tutorial --partitions 6

Now you can start sending records. If you want to see the service work under a high throughput, you can use the Apache Kafka producer performance test tool. This tool allows you to send many messages at the same time to the MSK cluster with a defined throughput and specific size. Experiment with this performance test tool, change the number of messages per second and the record size and see how the cluster behaves and adapts its capacity.

./kafka-topics.sh --bootstrap-server $BS 
--command-config client.properties 
--create --topic msk-serverless-tutorial --partitions 6

Finally, if you want to receive the messages, open a new terminal, connect to the same EC2 instance, and use the Apache Kafka consumer tool to receive the messages.

cd kafka_2.13-3.1.0/bin/
export BS=boot-abc1234.c3.kafka-serverless.us-east-2.amazonaws.com:9098
./kafka-console-consumer.sh 
--bootstrap-server $BS 
--consumer.config client.properties 
--topic msk-serverless-tutorial --from-beginning

You can see how the cluster is doing on the monitoring page of the Amazon MSK Serverless cluster.

Cluster metrics page

Availability
Amazon MSK Serverless is available in US East (Ohio), US East (N. Virginia), US West (Oregon), Europe (Frankfurt), Europe (Ireland), Europe (Stockholm), Asia Pacific (Singapore), Asia Pacific (Sydney), and Asia Pacific (Tokyo).
Learn more about this service and its pricing on the Amazon MSK Serverless feature page.

Marcia

Skyline Advisor Pro Proactive Findings – April Edition

This post was originally published on this site

Tweet VMware Skyline releases new Proactive Findings every month. Findings are prioritized by trending issues in VMware Support, issues raised through post escalation review, security vulnerabilities, and issues raised from VMware engineering, and customers. For the month of April, we released 21 new Findings. Of these, there are 18 Findings based on trending issues, 1 … Continued

The post Skyline Advisor Pro Proactive Findings – April Edition appeared first on VMware Support Insider.

New – Storage-Optimized Amazon EC2 Instances (I4i) Powered by Intel Xeon Scalable (Ice Lake) Processors

This post was originally published on this site

Over the years we have released multiple generations of storage-optimized Amazon Elastic Compute Cloud (Amazon EC2) instances including the HS1 (2012) , D2 (2015), I2 (2013) , I3 (2017), I3en (2019), D3/D3en (2020), and Im4gn/Is4gen (2021). These instances are used to host high-performance real-time relational databases, distributed file systems, data warehouses, key-value stores, and more.

New I4i Instances
Today I am happy to introduce the new I4i instances, powered by the latest generation Intel Xeon Scalable (Ice Lake) Processors with an all-core turbo frequency of 3.5 GHz.

The instances offer up to 30 TB of NVMe storage using AWS Nitro SSD devices that are custom-built by AWS, and are designed to minimize latency and maximize transactions per second (TPS) on workloads that need very fast access to medium-sized datasets on local storage. This includes transactional databases such as MySQL, Oracle DB, and Microsoft SQL Server, as well as NoSQL databases: MongoDB, Couchbase, Aerospike, Redis, and the like. They are also an ideal fit for workloads that can benefit from very high compute performance per TB of storage such as data analytics and search engines.

Here are the specs:

Instance Name vCPUs
Memory (DDR4) Local NVMe Storage
(AWS Nitro SSD)
Sequential Read Throughput
(128 KB Blocks)
Bandwidth
EBS-Optimized
Network
i4i.large 2 16 GiB 468 GB 350 MB/s Up to 10 Gbps Up to 10 Gbps
i4i.xlarge 4 32 GiB 937 GB 700 MB/s Up to 10 Gbps Up to 10 Gbps
i4i.2xlarge 8 64 GiB 1,875 GB 1,400 MB/s Up to 10 Gbps Up to 12 Gbps
i4i.4xlarge 16 128 GiB 3,750 GB 2,800 MB/s Up to 10 Gbps Up to 25 Gbps
i4i.8xlarge 32 256 GiB 7,500 GB
(2 x 3,750 GB)
5,600 MB/s 10 Gbps 18.75 Gbps
i4i.16xlarge 64 512 GiB 15,000 GB
(4 x 3,750 GB)
11,200 MB/s 20 Gbps 37.5 Gbps
i4i.32xlarge 128 1024 GiB 30,000 GB
(8 x 3,750 GB)
22,400 MB/s 40 Gbps 75 Gbps

In comparison to the Xen-based I3 instances, the Nitro-powered I4i instances give you:

  • Up to 60% lower storage I/O latency, along with up to 75% lower storage I/O latency variability.
  • A new, larger instance size (i4i.32xlarge).
  • Up to 30% better compute price/performance.

The i4i.16xlarge and i4.32xlarge instances give you control over C-states, and the i4i.32xlarge instances support non-uniform memory access (NUMA). All of the instances support AVX-512, and use Intel Total Memory Encryption (TME) to deliver always-on memory encryption.

From Our Customers
AWS customers and AWS service teams have been putting these new instances to the test ahead of today’s launch. Here’s what they had to say:

Redis Enterprises powers mission-critical applications for over 8,000 organizations. According to Yiftach Shoolman (Co-Founder and CTO of Redis):

We are thrilled with the performance we are seeing from the Amazon EC2 I4i instances which use the new low latency AWS Nitro SSDs. Our testing shows I4i instances delivering an astonishing 2.9x higher query throughput than the previous generation I3 instances. We have also tested with various read and write mixes, and observed consistent and linearly scaling performance.

ScyllaDB is a high performance NoSQL database that can take advantage of high performance cloud computing instances.
Avi Kivity (Co-Founder and CTO of ScyllaDB) told us:


When we tested I4i instances, we observed up to 2.7x increase in throughput per vCPU compared to I3 instances for reads. With an even mix of reads and writes, we observed 2.2x higher throughput per vCPU, with a 40% reduction in average latency than I3 instances. We are excited for the incredible performance and value these new instances will enable for our customers going forward.

Amazon QuickSight is a business intelligence service. After testing,
Tracy Daugherty (General Manager, Amazon Quicksight) reported that:

I4i instances have demonstrated superior performance over previous generation I instances, with a 30% improvement across operations. We look forward to using I4i to further elevate performance for our customers.

Available Now

You can launch I4i instances today in the AWS US East (N. Virginia), US East (Ohio), US West (Oregon), and Europe (Ireland) Regions (with more to come) in On-Demand and Spot form. Savings Plans and Reserved Instances are available, as are Dedicated Instances and Dedicated Hosts.

In order to take advantage of the performance benefits of these new instances, be sure to use recent AMIs that include current ENA drivers and support for NVMe 1.4.

To learn more, visit the I4i instance home page.

Jeff;

AA22-117A: 2021 Top Routinely Exploited Vulnerabilities

This post was originally published on this site

Original release date: April 27, 2022

Summary

This joint Cybersecurity Advisory (CSA) was coauthored by cybersecurity authorities of the United States, Australia, Canada, New Zealand, and the United Kingdom: the Cybersecurity and Infrastructure Security Agency (CISA), National Security Agency (NSA), Federal Bureau of Investigation (FBI), Australian Cyber Security Centre (ACSC), Canadian Centre for Cyber Security (CCCS), New Zealand National Cyber Security Centre (NZ NCSC), and United Kingdom’s National Cyber Security Centre (NCSC-UK). This advisory provides details on the top 15 Common Vulnerabilities and Exposures (CVEs) routinely exploited by malicious cyber actors in 2021, as well as other CVEs frequently exploited.

U.S., Australian, Canadian, New Zealand, and UK cybersecurity authorities assess, in 2021, malicious cyber actors aggressively targeted newly disclosed critical software vulnerabilities against broad target sets, including public and private sector organizations worldwide. To a lesser extent, malicious cyber actors continued to exploit publicly known, dated software vulnerabilities across a broad spectrum of targets. 

The cybersecurity authorities encourage organizations to apply the recommendations in the Mitigations section of this CSA. These mitigations include applying timely patches to systems and implementing a centralized patch management system to reduce the risk of compromise by malicious cyber actors.

Click here for a PDF version of this report. 

Technical Details

Key Findings

Globally, in 2021, malicious cyber actors targeted internet-facing systems, such as email servers and virtual private network (VPN) servers, with exploits of newly disclosed vulnerabilities. For most of the top exploited vulnerabilities, researchers or other actors released proof of concept (POC) code within two weeks of the vulnerability’s disclosure, likely facilitating exploitation by a broader range of malicious actors.

To a lesser extent, malicious cyber actors continued to exploit publicly known, dated software vulnerabilities—some of which were also routinely exploited in 2020 or earlier. The exploitation of older vulnerabilities demonstrates the continued risk to organizations that fail to patch software in a timely manner or are using software that is no longer supported by a vendor.

Top 15 Routinely Exploited Vulnerabilities

Table 1 shows the top 15 vulnerabilities U.S., Australian, Canadian, New Zealand, and UK cybersecurity authorities observed malicious actors routinely exploiting in 2021, which include:

  • CVE-2021-44228. This vulnerability, known as Log4Shell, affects Apache’s Log4j library, an open-source logging framework. An actor can exploit this vulnerability by submitting a specially crafted request to a vulnerable system that causes that system to execute arbitrary code. The request allows a cyber actor to take full control over the system. The actor can then steal information, launch ransomware, or conduct other malicious activity.[1] Log4j is incorporated into thousands of products worldwide. This vulnerability was disclosed in December 2021; the rapid widespread exploitation of this vulnerability demonstrates the ability of malicious actors to quickly weaponize known vulnerabilities and target organizations before they patch.
  • CVE-2021-26855, CVE-2021-26858, CVE-2021-26857, CVE-2021-27065. These vulnerabilities, known as ProxyLogon, affect Microsoft Exchange email servers. Successful exploitation of these vulnerabilities in combination (i.e., “vulnerability chaining”) allows an unauthenticated cyber actor to execute arbitrary code on vulnerable Exchange Servers, which, in turn, enables the actor to gain persistent access to files and mailboxes on the servers, as well as to credentials stored on the servers. Successful exploitation may additionally enable the cyber actor to compromise trust and identity in a vulnerable network.
  • CVE-2021-34523, CVE-2021-34473, CVE-2021-31207. These vulnerabilities, known as ProxyShell, also affect Microsoft Exchange email servers. Successful exploitation of these vulnerabilities in combination enables a remote actor to execute arbitrary code. These vulnerabilities reside within the Microsoft Client Access Service (CAS), which typically runs on port 443 in Microsoft Internet Information Services (IIS) (e.g., Microsoft’s web server). CAS is commonly exposed to the internet to enable users to access their email via mobile devices and web browsers. 
  • CVE-2021-26084. This vulnerability, affecting Atlassian Confluence Server and Data Center, could enable an unauthenticated actor to execute arbitrary code on vulnerable systems. This vulnerability quickly became one of the most routinely exploited vulnerabilities after a POC was released within a week of its disclosure. Attempted mass exploitation of this vulnerability was observed in September 2021.

Three of the top 15 routinely exploited vulnerabilities were also routinely exploited in 2020: CVE-2020-1472, CVE-2018-13379, and CVE-2019-11510. Their continued exploitation indicates that many organizations fail to patch software in a timely manner and remain vulnerable to malicious cyber actors.

Table 1: Top 15 Routinely Exploited Vulnerabilities in 2021

CVE

Vulnerability Name

Vendor and Product

Type

CVE-2021-44228

Log4Shell

Apache Log4j

Remote code execution (RCE)

CVE-2021-40539

 

Zoho ManageEngine AD SelfService Plus

RCE

CVE-2021-34523

ProxyShell

Microsoft Exchange Server

Elevation of privilege

CVE-2021-34473

ProxyShell

Microsoft Exchange Server

RCE

CVE-2021-31207

ProxyShell

Microsoft Exchange Server

Security feature bypass

CVE-2021-27065

ProxyLogon

Microsoft Exchange Server

RCE

CVE-2021-26858

ProxyLogon

Microsoft Exchange Server

RCE

CVE-2021-26857

ProxyLogon

Microsoft Exchange Server

RCE

CVE-2021-26855

ProxyLogon

Microsoft Exchange Server

RCE

CVE-2021-26084

 

 

Atlassian Confluence Server and Data Center

Arbitrary code execution

CVE-2021-21972

 

VMware vSphere Client

RCE

CVE-2020-1472

ZeroLogon

Microsoft Netlogon Remote Protocol (MS-NRPC)

Elevation of privilege

CVE-2020-0688

 

Microsoft Exchange Server

RCE

CVE-2019-11510

 

Pulse Secure Pulse Connect Secure

Arbitrary file reading

CVE-2018-13379

 

Fortinet FortiOS and FortiProxy

Path traversal

Additional Routinely Exploited Vulnerabilities

In addition to the 15 vulnerabilities listed in table 1, U.S., Australian, Canadian, New Zealand, and UK cybersecurity authorities identified vulnerabilities, listed in table 2, that were also routinely exploited by malicious cyber actors in 2021. 

These vulnerabilities include multiple vulnerabilities affecting internet-facing systems, including Accellion File Transfer Appliance (FTA), Windows Print Spooler, and Pulse Secure Pulse Connect Secure. Three of these vulnerabilities were also routinely exploited in 2020: CVE-2019-19781, CVE-2019-18935, and CVE-2017-11882.

Table 2: Additional Routinely Exploited Vulnerabilities in 2021

CVE

Vendor and Product

Type

CVE-2021-42237

Sitecore XP

RCE

CVE-2021-35464

ForgeRock OpenAM server

RCE

CVE-2021-27104

Accellion FTA

OS command execution

CVE-2021-27103

Accellion FTA

Server-side request forgery

CVE-2021-27102

Accellion FTA

OS command execution

CVE-2021-27101

Accellion FTA

SQL injection

CVE-2021-21985

VMware vCenter Server

RCE

CVE-2021-20038

SonicWall Secure Mobile Access (SMA)

RCE

CVE-2021-40444

Microsoft MSHTML

RCE

CVE-2021-34527

Microsoft Windows Print Spooler

RCE

CVE-2021-3156

Sudo

Privilege escalation

CVE-2021-27852

Checkbox Survey

Remote arbitrary code execution

CVE-2021-22893

Pulse Secure Pulse Connect Secure

Remote arbitrary code execution

CVE-2021-20016

SonicWall SSLVPN SMA100

Improper SQL command neutralization, allowing for credential access

CVE-2021-1675

Windows Print Spooler

RCE

CVE-2020-2509

QNAP QTS and QuTS hero

Remote arbitrary code execution

CVE-2019-19781

Citrix Application Delivery Controller (ADC) and Gateway

Arbitrary code execution

CVE-2019-18935

Progress Telerik UI for ASP.NET AJAX

Code execution

CVE-2018-0171

Cisco IOS Software and IOS XE Software

Remote arbitrary code execution

CVE-2017-11882

Microsoft Office

RCE

CVE-2017-0199

Microsoft Office

RCE

Mitigations

Vulnerability and Configuration Management

  • Update software, operating systems, applications, and firmware on IT network assets in a timely manner. Prioritize patching known exploited vulnerabilities, especially those CVEs identified in this CSA, and then critical and high vulnerabilities that allow for remote code execution or denial-of-service on internet-facing equipment. For patch information on CVEs identified in this CSA, refer to the appendix. 
    • If a patch for a known exploited or critical vulnerability cannot be quickly applied, implement vendor-approved workarounds.
  • Use a centralized patch management system.
  • Replace end-of-life software, i.e., software that is no longer supported by the vendor. For example, Accellion FTA was retired in April 2021.
  • Organizations that are unable to perform rapid scanning and patching of internet-facing systems should consider moving these services to mature, reputable cloud service providers (CSPs) or other managed service providers (MSPs). Reputable MSPs can patch applications—such as webmail, file storage, file sharing, and chat and other employee collaboration tools—for their customers. However, as MSPs and CSPs expand their client organization’s attack surface and may introduce unanticipated risks, organizations should proactively collaborate with their MSPs and CSPs to jointly reduce that risk. For more information and guidance, see the following resources.

Identity and Access Management

  • Enforce multifactor authentication (MFA) for all users, without exception.
  • Enforce MFA on all VPN connections. If MFA is unavailable, require employees engaging in remote work to use strong passwords. 
  • Regularly review, validate, or remove privileged accounts (annually at a minimum).
  • Configure access control under the concept of least privilege principle.
    • Ensure software service accounts only provide necessary permissions (least privilege) to perform intended functions (non-administrative privileges).

Note: see CISA Capacity Enhancement Guide – Implementing Strong Authentication and ACSC guidance on Implementing Multi-Factor Authentication for more information on hardening authentication systems.

Protective Controls and Architecture 

  • Properly configure and secure internet-facing network devices, disable unused or unnecessary network ports and protocols, encrypt network traffic, and disable unused network services and devices. 
    • Harden commonly exploited enterprise network services, including Link-Local Multicast Name Resolution (LLMNR) protocol, Remote Desktop Protocol (RDP), Common Internet File System (CIFS), Active Directory, and OpenLDAP.
    • Manage Windows Key Distribution Center (KDC) accounts (e.g., KRBTGT) to minimize Golden Ticket attacks and Kerberoasting.
    • Strictly control the use of native scripting applications, such as command-line, PowerShell, WinRM, Windows Management Instrumentation (WMI), and Distributed Component Object Model (DCOM).
  • Segment networks to limit or block lateral movement by controlling access to applications, devices, and databases. Use private virtual local area networks. 
  • Continuously monitor the attack surface and investigate abnormal activity that may indicate lateral movement of a threat actor or malware.
    • Use security tools, such as endpoint detection and response (EDR) and security information and event management (SIEM) tools. Consider using an information technology asset management (ITAM) solution to ensure your EDR, SIEM, vulnerability scanner etc., are reporting the same number of assets.
    • Monitor the environment for potentially unwanted programs.
  • Reduce third-party applications and unique system/application builds; provide exceptions only if required to support business critical functions.
  • Implement application allowlisting. 

Resources

Disclaimer

The information in this report is being provided “as is” for informational purposes only. CISA, the FBI, NSA, ACSC, CCCS, NZ NCSC, and NCSC-UK do not endorse any commercial product or service, including any subjects of analysis. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply endorsement, recommendation, or favoring.

Purpose 

This document was developed by U.S., Australian, Canadian, New Zealand, and UK cybersecurity authorities in furtherance of their respective cybersecurity missions, including their responsibilities to develop and issue cybersecurity specifications and mitigations.

References

[1] CISA’s Apache Log4j Vulnerability Guidance

Appendix: Patch Information and Additional Resources for  Top Exploited Vulnerabilities

CVE

Vendor

Affected Products

Patch Information

Resources

CVE-2021-42237

Sitecore

Sitecore XP 7.5.0 – Sitecore XP 7.5.2

Sitecore XP 8.0.0 – Sitecore XP 8.2.7

Sitecore Security Bulletin SC2021-003-499266

ACSC Alert Active Exploitation of vulnerable Sitecore Experience Platform Content Management Systems

 

CVE-2021-35464

ForgeRock

Access Management (AM) 5.x, 6.0.0.x, 6.5.0.x, 6.5.1, 6.5.2.x and 6.5.3

OpenAM 9.x, 10.x, 11.x, 12.x and 13.x

ForgeRock AM Security Advisory #202104

ACSC Advisory Active exploitation of ForgeRock Access Manager / OpenAM servers

CCCS ForgeRock Security Advisory

CVE-2021-27104

Accellion

FTA 9_12_370 and earlier

Accellion Press Release: Update to Recent FTA Security Incident

Joint CSA Exploitation of Accellion File Transfer Appliance

ACSC Alert Potential Accellion File Transfer Appliance compromise

 

 

CVE-2021-27103

FTA 9_12_411 and earlier

CVE-2021-27102

FTA versions 9_12_411 and earlier

CVE-2021-27101

FTA 9_12_370 and earlier

 

CVE-2021-21985

VMware

vCenter Server 7.0, 6.7, 6.5

Cloud Foundation (vCenter Server) 4.x and 3.x

VMware Advisory VMSA-2021-0010

CCCS VMware Security Advisory

CVE-2021-21972

VMware

vCenter Server 7.0, 6.7, 6.5

Cloud Foundation (vCenter Server) 4.x and 3.x

VMware Advisory VMSA-2021-0002

ACSC Alert VMware vCenter Server plugin remote code execution vulnerability

CCCS VMware Security Advisory

CCCS Alert APT Actors Target U.S. and Allied Networks – Update 1

CVE-2021-20038

SonicWall

SMA 100 Series (SMA 200, 210, 400, 410, 500v), versions 10.2.0.8-37sv, 10.2.1.1-19sv, 10.2.1.2-24sv

SonicWall Security Advisory SNWLID-2021-0026

ACSC Alert Remote code execution vulnerability present in SonicWall SMA 100 series appliances

CCCS SonicWall Security Advisory

 

CVE-2021-44228

Apache

Log4j, all versions from 2.0-beta9 to 2.14.1

For other affected vendors and products, see CISA’s GitHub repository.

Log4j: Apache Log4j Security Vulnerabilities

For additional information, see joint CSA: Mitigating Log4Shell and Other Log4j-Related Vulnerabilities

CISA webpage Apache Log4j Vulnerability Guidance

CCCS Active exploitation of Apache Log4j vulnerability – Update 7

CVE-2021-40539

Zoho ManageEngine

ADSelfService Plus version 6113 and prior

Zoho ManageEngine: ADSelfService Plus 6114 Security Fix Release

Joint CSA APT Actors Exploiting Newly Identified Vulnerability in ManageEngine ADSelfService Plus

CCCS Zoho Security Advisory

CVE-2021-40444

Microsoft

Multiple Windows products; see Microsoft Security Update Guide: MSHTML Remote Code Execution Vulnerability, CVE-2021-40444

Microsoft Security Update Guide: MSHTML Remote Code Execution Vulnerability, CVE-2021-40444

 

CVE-2021-34527

Microsoft

Multiple Windows products; see Microsoft Security Update Guide: Windows Print Spooler Remote Code Execution Vulnerability, CVE-2021-34527

Microsoft Security Update Guide: Windows Print Spooler Remote Code Execution Vulnerability, CVE-2021-34527

Joint CSA Russian State-Sponsored Cyber Actors Gain Network Access by Exploiting Default Multifactor Authentication Protocols and “PrintNightmare” Vulnerability

CCCS Alert Windows Print Spooler Vulnerability Remains Unpatched – Update 3

CVE-2021-34523

Microsoft

Microsoft Exchange Server 2013 Cumulative Update 23

Microsoft Exchange Server 2016 Cumulative Updates 19 and 20

Microsoft Exchange Server 2019 Cumulative Updates 8 and 9

Microsoft Security Update Guide: Microsoft Exchange Server Elevation of Privilege Vulnerability, CVE-2021-34523

Joint CSA Iranian Government-Sponsored APT Cyber Actors Exploiting Microsoft Exchange and Fortinet Vulnerabilities in Furtherance of Malicious Activities

ACSC Alert Microsoft Exchange ProxyShell Targeting in Australia

 

CVE-2021-34473

Microsoft

Multiple Exchange Server versions; see: Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-34473

Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-34473

CVE-2021-31207

Microsoft

Multiple Exchange Server versions; see Microsoft Update Guide: Microsoft Exchange Server Security Feature Bypass Vulnerability, CVE-2021-31207

Microsoft Update Guide: Microsoft Exchange Server Security Feature Bypass Vulnerability, CVE-2021-31207

CVE-2021-3156

Sudo

Sudo before 1.9.5p2

Sudo Stable Release 1.9.5p2

 

CVE-2021-27852

Checkbox Survey

Checkbox Survey versions prior to 7

 

 

CVE-2021-27065

Microsoft Exchange Server

Multiple versions; see: Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-27065

Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-27065

CISA Alert: Mitigate Microsoft Exchange Server Vulnerabilities

ACSC Advisory Active exploitation of Vulnerable Microsoft Exchange servers

CCCS Alert Active Exploitation of Microsoft Exchange Vulnerabilities – Update 4

CVE-2021-26858

Microsoft

Exchange Server, multiple versions; see Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-26858

Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-26858

CVE-2021-26857

Microsoft

Exchange Server, multiple versions; see Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-26857

Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-26857

CVE-2021-26855

Microsoft

Exchange Server, multiple versions; see Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-26855

Microsoft Security Update Guide: Microsoft Exchange Server Remote Code Execution Vulnerability, CVE-2021-26855

CVE-2021-26084

 

Jira Atlassian

Confluence Server and Data Center, versions 6.13.23, from version 6.14.0 before 7.4.11, from version 7.5.0 before 7.11.6, and from version 7.12.0 before 7.12.5.

Jira Atlassian: Confluence Server Webwork OGNL injection – CVE-2021-26084

ACSC Alert Remote code execution vulnerability present in certain versions of Atlassian Confluence

CCCS Atlassian Security Advisory

CVE-2021-22893

Pulse Secure

PCS 9.0R3/9.1R1 and Higher

Pulse Secure SA44784 – 2021-04: Out-of-Cycle Advisory: Multiple Vulnerabilities Resolved in Pulse Connect Secure 9.1R11.4

CCCS Alert  Active Exploitation of Pulse Connect Secure Vulnerabilities – Update 1

CVE-2021-20016

SonicWall

SMA 100 devices (SMA 200, SMA 210, SMA 400, SMA 410, SMA 500v)

SonicWall Security Advisory SNWLID-2021-0001

 

CVE-2021-1675

Microsoft

Multiple Windows products; see Microsoft Security Update Guide Windows Print Spooler Remote Code Execution Vulnerability, CVE-2021-1675

Microsoft Security Update Guide: Windows Print Spooler Remote Code Execution Vulnerability, CVE-2021-1675

CCCS Alert Windows Print Spooler Vulnerability Remains Unpatched – Update 3

CVE-2020-2509

QNAP

QTS, multiple versions; see QNAP: Command Injection Vulnerability in QTS and QuTS hero

QuTS hero h4.5.1.1491 build 20201119 and later

QNAP: Command Injection Vulnerability in QTS and QuTS hero

 

CVE-2020-1472

Microsoft

Windows Server, multiple versions; see Microsoft Security Update Guide: Netlogon Elevation of Privilege Vulnerability, CVE-2020-1472

Microsoft Security Update Guide: Netlogon Elevation of Privilege Vulnerability, CVE-2020-1472

ACSC Alert Netlogon elevation of privilege vulnerability (CVE-2020-1472)

Joint CSA APT Actors Chaining Vulnerabilities Against SLTT, Critical Infrastructure, and Elections Organizations

CCCS Alert Microsoft Netlogon Elevation of Privilege Vulnerability – CVE-2020-1472 – Update 1

CVE-2020-0688

Microsoft

Exchange Server, multiple versions; see Microsoft Security Update Guide: Microsoft Exchange Validation Key Remote Code Execution Vulnerability, CVE-2020-0688

Microsoft Security Update Guide: Microsoft Exchange Validation Key Remote Code Execution Vulnerability, CVE-2020-0688

CISA Alert Chinese Ministry of State Security-Affiliated Cyber Threat Actor Activity

Joint CSA Russian State-Sponsored Cyber Actors Target Cleared Defense Contractor Networks to Obtain Sensitive U.S. Defense Information and Technology

CCCS Alert Microsoft Exchange Validation Key Remote Code Execution Vulnerability

CVE-2019-19781

Citrix

ADC and Gateway version 13.0 all supported builds before 13.0.47.24

NetScaler ADC and NetScaler Gateway, version 12.1 all supported builds before 12.1.55.18; version 12.0 all supported builds before 12.0.63.13; version 11.1 all supported builds before 11.1.63.15; version 10.5 all supported builds before 10.5.70.12

SD-WAN WANOP appliance models 4000-WO, 4100-WO, 5000-WO, and 5100-WO all supported software release builds before 10.2.6b and 11.0.3b

Citrix Security Bulletin CTX267027

Joint CSA APT Actors Chaining Vulnerabilities Against SLTT, Critical Infrastructure, and Elections Organizations

CISA Alert Chinese Ministry of State Security-Affiliated Cyber Threat Actor Activity

CCCS Alert Detecting Compromises relating to Citrix CVE-2019-19781

 

 

 

CVE-2019-18935

Progress Telerik

UI for ASP.NET AJAX through 2019.3.1023

Telerik UI for ASP.NET AJAX Allows JavaScriptSerializer Deserialization

ACSC Alert Active exploitation of vulnerability in Microsoft Internet Information Services

 

CVE-2019-11510

Pulse Secure

Pulse Connect Secure 8.2 before 8.2R12.1, 8.3 before 8.3R7.1, and 9.0 before 9.0R3.4

Pulse Secure: SA44101 – 2019-04: Out-of-Cycle Advisory: Multiple vulnerabilities resolved in Pulse Connect Secure / Pulse Policy Secure 9.0RX

CISA Alert Continued Exploitation of Pulse Secure VPN Vulnerability

CISA Alert Chinese Ministry of State Security-Affiliated Cyber Threat Actor Activity

ACSC Advisory Recommendations to mitigate vulnerability in Pulse Connect Secure VPN Software

Joint CSA APT Actors Chaining Vulnerabilities Against SLTT, Critical Infrastructure, and Elections Organizations

CCCS Alert APT Actors Target U.S. and Allied Networks – Update 1

CVE-2018-13379

Fortinet

FortiProxy 2.0.2, 2.0.1, 2.0.0, 1.2.8, 1.2.7, 1.2.6, 1.2.5, 1.2.4, 1.2.3, 1.2.2, 1.2.1, 1.2.0, 1.1.6

Fortinet FortiGuard Labs: FG-IR-20-233

Joint CSA Russian State-Sponsored Cyber Actors Target Cleared Defense Contractor Networks to Obtain Sensitive U.S. Defense Information and Technology

Joint CSA Iranian Government-Sponsored APT Cyber Actors Exploiting Microsoft Exchange and Fortinet Vulnerabilities in Furtherance of Malicious Activities

Joint CSA APT Actors Chaining Vulnerabilities Against SLTT, Critical Infrastructure, and Elections Organizations

ACSC Alert APT exploitation of Fortinet Vulnerabilities

CCCS Alert Exploitation of Fortinet FortiOS vulnerabilities (CISA, FBI) – Update 1

CVE-2018-0171

Cisco

See Cisco Security Advisory: cisco-sa-20180328-smi2

Cisco Security Advisory: cisco-sa-20180328-smi2

CCCS Action Required to Secure the Cisco IOS and IOS XE Smart Install Feature

CVE-2017-11882

Microsoft

Office, multiple versions; see Microsoft Security Update Guide: Microsoft Office Memory Corruption Vulnerability, CVE-2017-11882

Microsoft Security Update Guide: Microsoft Office Memory Corruption Vulnerability, CVE-2017-11882

CCCS Alert Microsoft Office Security Update

CVE-2017-0199

Microsoft

Multiple products; see Microsoft Security Update Guide: Microsoft Office/WordPad Remote Code Execution Vulnerability w/Windows, CVE-2017-0199

Microsoft Security Update Guide: Microsoft Office/WordPad Remote Code Execution Vulnerability w/Windows, CVE-2017-0199

CCCS Microsoft Security Updates

Contact Information

U.S. organizations: all organizations should report incidents and anomalous activity to CISA 24/7 Operations Center at report@cisa.gov or (888) 282-0870 and/or to the FBI via your local FBI field office or the FBI’s 24/7 CyWatch at (855) 292-3937 or CyWatch@fbi.gov. When available, please include the following information regarding the incident: date, time, and location of the incident; type of activity; number of people affected; type of equipment used for the activity; the name of the submitting company or organization; and a designated point of contact. For NSA client requirements or general cybersecurity inquiries, contact Cybersecurity_Requests@nsa.gov. Australian organizations: visit cyber.gov.au or call 1300 292 371 (1300 CYBER 1) to report cybersecurity incidents and access alerts and advisories. Canadian organizations: report incidents by emailing CCCS at contact@cyber.gc.ca. New Zealand organizations: report cyber security incidents to incidents@ncsc.govt.nz or call 04 498 7654. United Kingdom organizations: report a significant cyber security incident: ncsc.gov.uk/report-an-incident (monitored 24 hours) or, for urgent assistance, call 03000 200 973.

Revisions

  • April 27, 2022: Initial Version

This product is provided subject to this Notification and this Privacy & Use policy.

MITRE ATT&CK v11 – a small update that can help (not just) with detection engineering, (Wed, Apr 27th)

This post was originally published on this site

MITRE ATT&CK has long been the de facto standard for sharing TTPs of different threat actors and for planning and executing various threat emulation exercises. However, especially in the last few years, I’ve seen more and more Security Operations Centers start using it as well, for mapping of their defensive capabilities, detection use cases and SIEM correlation rules.

AWS Week in Review – April 25, 2022

This post was originally published on this site

This post is part of our Week in Review series. Check back each week for a quick roundup of interesting news and announcements from AWS!

The first in this year’s series of AWS Summits took place in San Francisco this past week and we had a bunch of great announcements. Let’s take a closer look…

Last Week’s Launches
Here are some launches that caught my eye this week:

AWS Migration Hub Orchestrator – Building on AWS Migration Hub (launched in 2017), this service helps you to reduce migration costs by automating manual tasks, managing dependencies between tools, and providing better visibility into the migration progress. It makes use of workflow templates that you can modify and extend, and includes a set of predefined templates to get you started. We are launching with support for applications based on SAP NetWeaver with HANA databases, along with support for rehosting of applications using AWS Application Migration Service (AWS MGN). To learn more, read Channy’s launch post: AWS Migration Hub Orchestrator – New Migration Orchestration Capability with Customizable Workflow Templates.

Amazon DevOps Guru for Serverless – This is a new capability for Amazon DevOps Guru, our ML-powered cloud operations service which helps you to improve the availability of your application using models informed by years of Amazon.com and AWS operational excellence. This launch helps you to automatically detect operational issues in your Lambda functions and DynamoDB tables, giving you actionable recommendations that help you to identify root causes and fix issues as quickly as possible, often before they affect the performance of your serverless application. Among other insights you will be notified of concurrent executions that reach the account limit, lower than expected use of provisioned concurrency, and reads or writes to DynamoDB tables that approach provisioned limits. To learn more and to see the full list of insights, read Marcia’s launch post: Automatically Detect Operational Issues in Lambda Functions with Amazon DevOps Guru for Serverless.

AWS IoT TwinMaker – Launched in preview at re:Invent 2021 (Introducing AWS IoT TwinMaker), this service helps you to create digital twins of real-world systems and to use them in applications. There’s a flexible model builder that allows you to create workspaces that contain entity models and visual assets, connectors to bring in data from data stores to add context, a console-based 3D scene composition tool, and plugins to help you create Grafana and Amazon Managed Grafana dashboards. To learn more and to see AWS IoT TwinMaker in action, read Channy’s post, AWS IoT TwinMaker is now Generally Available.

AWS Amplify Studio – Also launched in preview at re:Invent 2021 (AWS Amplify Studio: Visually build full-stack web apps fast on AWS), this is a point-and-click visual interface that simplifies the development of frontend and backends for web and mobile applications. During the preview we added integration with Figma so that to make it easier for designers and front-end developers to collaborate on design and development tasks. As Steve described in his post (Announcing the General Availability of AWS Amplify Studio), you can easily pull component designs from Figma, attach event handlers, and extend the components with your own code. You can modify default properties, override child UI elements, extend collection items with additional data, and create custom business logic for events. On the visual side, you can use Figma’s Theme Editor plugin to make UI components to your organization’s brand and style.

Amazon Aurora Serverless v2Amazon Aurora separates compute and storage, and allows them to scale independently. The first version of Amazon Aurora Serverless was launched in 2018 as a cost-effective way to support workloads that are infrequent, intermittent, or unpredictable. As Marcia shared in her post (Amazon Aurora Serverless v2 is Generally Available: Instant Scaling for Demanding Workloads), the new version is ready to run your most demanding workloads, with instant, non-disruptive scaling, fine-grained capacity adjustments, read replicas, Multi-AZ deployments, and Amazon Aurora Global Database. You pay only for the capacity that you consume, and can save up to 90% compared to provisioning for peak load.

Amazon SageMaker Serverless InferenceAmazon SageMaker already makes it easy for you to build, train, test, and deploy your machine learning models. As Antje descibed in her post (Amazon SageMaker Serverless Inference – Machine Learning Inference without Worrying about Servers), different ML inference use cases pose varied requirements on the infrastructure that is used to host the models. For example, applications that have intermittent traffic patterns can benefit from the ability to automatically provision and scale compute capacity based on the volume of requests. The new serverless inferencing option that Antje wrote about provides this much-desired automatic provisioning and scaling, allowing you to focus on developing your model and your inferencing code without having to manage or worry about infrastructure.

Other AWS News
Here are a few other launches and news items that caught my eye:

AWS Open Source News and Updates – My colleague Ricardo Sueiras writes this weekly open-source newsletter where he highlights new open source projects, tools, and demos from the AWS community. Read edition #109 here.

Amazon Linux AMI – An Amazon Linux 2022 AMI that is optimized for Amazon ECS is now available. Read the What’s New to learn more.

AWS Step Functions – AWS Step Functions now supports over 20 new AWS SDK integrations and over 1000 new AWS API actions. Read the What’s New to learn more.

AWS CloudFormation Registry – There are 35 new resource types in the AWS CloudFormation Registry, including AppRunner, AppStream, Billing Conductor, ECR, EKS, Forecast, Lightsail, MSK, and Personalize. Check out the full list in the What’s New.

Upcoming AWS Events
AWS SummitThe AWS Summit season is in full swing – The next AWS Summits are taking place in London (on April 27), Madrid (on May 4-5), Korea (online, on May 10-11), and Stockholm (on May 11). AWS Global Summits are free events that bring the cloud computing community together to connect, collaborate, and learn about AWS. Summits are held in major cities around the world. Besides in-person summits, we also offer a series of online summits across the regions. Find an AWS Summit near you, and get notified when registration opens in your area.

.NET Enterprise Developer Day EMEA .NET Enterprise Developer Day EMEA 2022 is a free, one-day virtual conference providing enterprise developers with the most relevant information to swiftly and efficiently migrate and modernize their .NET applications and workloads on AWS. It takes place online on April 26. Attendees can also opt-in to attend the free, virtual DeveloperWeek Europe event, taking place April 27-28.

AWS Innovate - Data EditionAWS Innovate – Data Edition Americas AWS Innovate Online Conference – Data Edition is a free virtual event designed to inspire and empower you to make better decisions and innovate faster with your data. You learn about key concepts, business use cases, and best practices from AWS experts in over 30 technical and business sessions. This event takes place on May 11.

That’s all for this week. Check back again next week for the another AWS Week in Review!

Jeff;

Simple PDF Linking to Malicious Content, (Mon, Apr 25th)

This post was originally published on this site

Last week, I found an interesting piece of phishing based on a PDF file. Today, most of the PDF files that are delivered to end-user are not malicious, I mean that they don’t contain an exploit to trigger a vulnerability and infect the victim’s computer. They are just used as a transport mechanism to deliver more malicious content. Yesterday, Didier analyzed the same kind of Word document[1]. They are more and more common because they are (usually) not blocked by common filters at the perimeter.

Are Roku Streaming Devices Safe from Exploitation?, (Sat, Apr 23rd)

This post was originally published on this site

I have noticed in the past several weeks random scans specifically for Roku streaming devices (and likely other types) captured by my honeypot. If they can be compromised, what can be gain? Settings like stored payment information, personal information (email/password), subscription, App selected, etc. Like any other devices, it is important to keep the OS and Apps up-to-date. 

Amazon Aurora Serverless v2 is Generally Available: Instant Scaling for Demanding Workloads

This post was originally published on this site

Today we are very excited to announce that Amazon Aurora Serverless v2 is generally available for both Aurora PostgreSQL and MySQL. Aurora Serverless is an on-demand, auto-scaling configuration for Amazon Aurora that allows your database to scale capacity up or down based on your application’s needs.

Amazon Aurora is a MySQL- and PostgreSQL-compatible relational database built for the cloud. It is fully managed by Amazon Relational Database Service (RDS), which automates time-consuming administrative tasks, such as hardware provisioning, database setup, patches, and backups.

One of the key features of Amazon Aurora is the separation of compute and storage. As a result, they scale independently. Amazon Aurora storage automatically scales as the amount of data in your database increases. For example, you can store lots of data, and if one day you decide to drop most of the data, the storage provisioned adjusts.

How Amazon Aurora works - compute and storage separation
However, many customers said that they need the same flexibility in the compute layer of Amazon Aurora since most database workloads don’t need a constant amount of compute. Workloads can be spiky, infrequent, or have predictable spikes over a period of time.

To serve these kinds of workloads, you need to provision for the peak capacity you expect your database will need. However, this approach is expensive as database workloads rarely run at peak capacity. To provision the right amount of compute, you need to continuously monitor the database capacity consumption and scale up resources if consumption is high. However, this requires expertise and often incurs downtime.

To solve this problem, in 2018, we launched the first version of Amazon Aurora Serverless. Since its launch, thousands of customers have used Amazon Aurora Serverless as a cost-effective option for infrequent, intermittent, and unpredictable workloads.

Today, we are making the next version of Amazon Aurora Serverless generally available, which enables customers to run even the most demanding workload on serverless with instant and nondisruptive scaling, fine-grained capacity adjustments, and additional functionality, including read replicas, Multi-AZ deployments, and Amazon Aurora Global Database.

Aurora Serverless v2 is launching with the latest major versions available on Amazon Aurora. Versions supported: Aurora PostgreSQL-compatible edition with PostgreSQL 13 and Aurora MySQL-compatible edition with MySQL 8.0.

Main features of Aurora Serverless v2
Aurora Serverless v2 enables you to scale your database to hundreds of thousands of transactions per second and cost-effectively manage the most demanding workloads. It scales database capacity in fine-grained increments to closely match the needs of your workload without disrupting connections or transactions. In addition, you pay only for the exact capacity you consume, and you can save up to 90 percent compared to provisioning for peak load.

If you have an existing Amazon Aurora cluster, you can create an Aurora Serverless v2 instance within the same cluster. This way, you’ll have a mixed configuration cluster where both provisioned and Aurora Serverless v2 instances can coexist within the same cluster.

It supports the full breadth of Amazon Aurora features. For example, you can create up to 15 Amazon Aurora read replicas deployed across multiple Availability Zones. Any number of these read replicas can be Aurora Serverless v2 instances and can be used as failover targets for high availability or for scaling read operations.

Similarly, with Global Database, you can assign any of the instances to be Aurora Serverless v2 and only pay for minimum capacity when idling. These instances in secondary Regions can also scale independently to support varying workloads across different Regions. Check out the Amazon Aurora user guide for a comprehensive list of features.

Aurora Serverless compute and storage scaling

How Aurora Serverless v2 scaling works
Aurora Serverless v2 scales instantly and nondisruptively by growing the capacity of the underlying instance in place by adding more CPU and memory resources. This technique allows for the underlying instance to increase and decrease capacity in place without failing over to a new instance for scaling.

For scaling down, Aurora Serverless v2 takes a more conservative approach. It scales down in steps until it reaches the required capacity needed for the workload. Scaling down too quickly can prematurely evict cached pages and decrease the buffer pool, which may affect the performance.

Aurora Serverless capacity is measured in Aurora capacity units (ACUs). Each ACU is a combination of approximately 2 gibibytes (GiB) of memory, corresponding CPU, and networking. With Aurora Serverless v2, your starting capacity can be as small as 0.5 ACU, and the maximum capacity supported is 128 ACU. In addition, it supports fine-grained increments as small as 0.5 ACU which allows your database capacity to closely match the workload needs.

Aurora Serverless v2 scaling in action
To show Aurora Serverless v2 in action, we are going to simulate a flash sale. Imagine that you run an e-commerce site. You run a marketing campaign where customers can purchase items 50 percent off for a limited amount of time. You are expecting a spike in traffic on your site for the duration of the sale.

When you use a traditional database, if you run those marketing campaigns regularly, you need to provision for the peak load you expect. Or, if you run them now and then, you need to reconfigure your database for the expected peak of traffic during the sale. In both cases, you are limited to your assumption of the capacity you need. What happens if you have more sales than you expected? If your database cannot keep up with the demand, it may cause service degradation. Or when your marketing campaign doesn’t produce the sales you expected? You are unnecessarily paying for capacity you don’t need.

For this demo, we use Aurora Serverless v2 as the transactional database. An AWS Lambda function is used to call the database and process orders during the sale event for the e-commerce site. The Lambda function and the database are in the same Amazon Virtual Private Cloud (VPC), and the function connects directly to the database to perform all the operations.

To simulate the traffic of a flash sale, we will use an open-source load testing framework called Artillery. It will allow us to generate varying load by invoking multiple Lambda functions. For example, we can start with a small load and then increase it rapidly to observe how the database capacity adjusts based on the workload. This Artillery load test runs on an Amazon Elastic Compute Cloud (Amazon EC2) instance inside the same VPC.

Architecture diagram
The following Amazon CloudWatch dashboard shows how the database capacity behaves when the order count increases. The dashboard shows the orders placed in blue and the current database capacity in orange.

At the beginning of the sale, the Aurora Serverless v2 database starts with a capacity of 5 ACUs, which was the minimum database capacity configured. For the first few minutes, the orders increase, but the database capacity doesn’t increase right away. The database can handle the load with the starting provisioned capacity.

However, around the time 15:55, the number of orders spikes to 12,000. As a result, the database increases the capacity to 14 ACUs. The database capacity increases in milliseconds, adjusting exactly to the load.

The number of orders placed stays up for some seconds, and then it goes dramatically down by 15:58. However, the database capacity doesn’t adjust exactly to the drop in traffic. Instead, it decreases in steps until it reaches 5 ACUs. The scaling down is done more conservatively to avoid prematurely evicting cached pages and affecting performance. This is done to prevent any unnecessary latency to spiky workloads, and also so the caches and buffer pools are not aggressively purged.

Cloudwatch dashboard

Get started with Aurora Serverless v2 with an existing Amazon Aurora cluster
If you already have an Amazon Aurora cluster and you want to try Aurora Serverless v2, the fastest way to get started is by using mixed configuration clusters that contain both serverless and provisioned instances. Start by adding a new reader into the existing cluster. Configure the reader instance to be of the type Serverless v2.

Adding a serverless reader

Test the new serverless instance with your workload. Once you have confirmation that it works as expected, you can start a failover to the serverless instance, which will take less than 30 seconds to finish. This option provides a minimal downtime experience to get started with Aurora Serverless v2.

Failover to the serverless instance

How to create a new Aurora Serverless v2 database
To get started with Aurora Serverless v2, create a new database from the RDS console. The first step is to pick the engine type: Amazon Aurora. Then, pick which database engine you want it to be compatible with: MySQL or PostgreSQL. Open the filters under Engine version and select the filter Show versions that support Serverless v2. Then, you see that the Available versions dropdown list only shows options that are supported by Aurora Serverless v2.

Engine options
Next, you need to set up the database. Specify credential settings with a username and password for the administrator of the database.

Database settings
Then, configure the instance for the database. You need to select what kind of instance class you want. This allocates the computational, network, and memory capacity for the database instance. Select Serverless.

Then, you need to define the capacity range. Aurora Serverless v2 capacity scales up and down within the minimum and maximum configuration. Here you can specify the minimum and maximum database capacity for your workload. The minimum capacity you can specify is 0.5 ACUs, and the maximum is 128 ACUs. For more information on Aurora Serverless v2 capacity units, see the Instant autoscaling documentation.

Capacity configuration
Next, configure connectivity by creating a new VPC and security group or use the default. Finally, select Create database.

Connectivity configuration

Creating the database takes a couple of minutes. You know your database is ready when the status switches to Available.

Database list

You will find the connection details for the database on the database page. The endpoint and the port, combined with the user name and password for the administrator, are all you need to connect to your new Aurora Serverless v2 database.

Database details page

Available Now!
Aurora Serverless v2 is available now in US East (Ohio), US East (N. Virginia), US West (N. California), US West (Oregon), Asia Pacific (Hong Kong), Asia Pacific (Mumbai), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), Europe (Frankfurt), Europe (Ireland), Europe (London), Europe (Paris), Europe (Stockholm), and South America (São Paulo).

Visit the Amazon Aurora Serverless v2 page for more information about this launch.

Marcia