Amazon File Cache – A High Performance Cache On AWS For Your On-Premises File Systems

This post was originally published on this site

I am pleased to announce today the availability of Amazon File Cache, a new high-speed cache service on AWS designed for processing file data stored in disparate locations—including on premises. File Cache accelerates and simplifies your most demanding cloud bursting and hybrid workflows by giving your applications access to files using a fast and familiar POSIX interface, no matter if the original files live on premises on any file system that can be accessed through NFS v3 or on Amazon Simple Storage Service (Amazon S3).

Imagine you have a large data set on on-premises storage infrastructure, and your end-of-month reporting typically takes two to three days to run. You want to move that occasional workload to the cloud to run it on larger machines with more CPU and memory to reduce the processing time. But you’re not ready to move the data set to the cloud yet.

Imagine another scenario where you have access to a large data set on Amazon Simple Storage Service (Amazon S3), spread across multiple Regions. Your application that wants to exploit this data set is coded for traditional (POSIX) file system access and uses command line tools like awk, sed, pipes, and so on. Your application requires file access with sub-millisecond latencies. You cannot update the source code to use the S3 API.

File Cache helps to address these use cases and many others, think about management and transformation of video files, AI/ML data sets, and so on. File Cache creates a file system–based cache in front of either NFS v3 file systems or S3 buckets in one or more Regions. It transparently loads file content and metadata (such as the file name, size, and permissions) from the origin and presents it to your applications as a traditional file system. File Cache automatically releases the less recently used cached files to ensure the most active files are available in the cache for your applications.

You can link up to eight NFS file systems or eight S3 buckets to a cache, and they will be exposed as a unified set of files and directories. You can access the cache from a variety of AWS compute services, such as virtual machines or containers. The connection between File Cache and your on-premises infrastructure uses your existing network connection, based on AWS Direct Connect and/or Site-to-Site VPN.

When using File Cache, your applications benefit from consistent, sub-millisecond latencies, up to hundreds of GB/s of throughput, and up to millions of operations per second. Just like with other storage services, such as Amazon Elastic Block Store (Amazon EBS), the performance depends on the size of the cache. The cache size can be expanded to petabyte scale, with a minimum size of 1.2 TiB.

Let’s See How It Works
To show you how it works, I create a file cache on top of two existing Amazon FSx for OpenZFS file systems. In a real-world scenario, it is likely you will create caches on top of on-premises file systems. I choose FSx for OpenZFS for the demo because I don’t have an on-premises data center at hand (I should maybe invest in seb-west-1). Both demo OpenZFS file systems are accessible from a private subnet in my AWS account. Finally, I access the cache from an EC2 Linux instance.

I open my browser and navigate to the AWS Management Console. I search for “Amazon FSx” in the console search bar and click on Caches in the left navigation menu. Alternatively, I go directly to the File Cache section of the console. To get started, I select Create cache.

Amazon File Cache consoleI enter a Cache name for my cache (AWSNewsBlog for this demo) and a Cache storage capacity. The storage capacity is expressed in tebibytes. The minimum value is 1.2 TiB or increments of 2.4 TiB. Notice that the Throughput capacity increases as you choose large cache sizes.

Amazon File Cache create 1I check and accept the default values provided for Networking and Encryption. For networking, I might select a VPC, subnet, and security group to associate with my cache network interface. It is recommended to deploy the cache in the same subnet as your compute service to minimize the latency when accessing files. For encryption, I might use an AWS KMS-managed key (the default) or select my own.

Then, I create Data Repository Association. This is the link between the cache and a data source. A data source might be an NFS file system or an S3 bucket or prefix. I might create up to eight data repository associations for one cache. All Data Repository Associations for a cache have the same type: they are all NFS v3 or all S3. If you need both, you can create two caches.

In this demo, I choose to link two OpenZFS file systems on my AWS account. You can link to any NFS v3 servers, including the ones you already have on premises. Cache path allows you to choose where the source file system will be mounted in the cache. The Data repository path is the URL to your NFS v3 or S3 data repository. The format is nfs://hostname/path or s3://bucketname/path.

The DNS server IP addresses allows File Cache to resolve the DNS name of your NFS server. This is useful when DNS resolution is private, like in my example. When you are associating NFS v3 servers deployed in a VPC, and when using the AWS-provided DNS server, the DNS server IP address of your VPC is the VPC Range + two. In my example, my VPC CIDR range is, hence the DNS server IP address is

Do not forget to click on the Add button! Otherwise, your input is ignored. You can repeat the operation to add more data repositories.

File Cache - Create new Data Repository Association- dataset one File Cache - Create new Data Repository Association- dataset two

Once I have entered my two data repositories, I select Next, and I review my choices. When I am ready, I select Create cache.

File Cache - review choices

After a few minutes, the cache status becomes ✅ Available.

Amazon File cache status is available

The last part is to mount the cache on the machine where my workload is deployed. File Cache uses Lustre behind the scene. I have to install the Lustre client for Linux first, as explained in our documentation. Once done, I select the Attach button on the console to receive the instructions to download and install the Lustre client and to mount the cache file system.File Cache Attach To do so, I connect to an EC2 instance running in the same VPC. Then I type:

sudo mount -t lustre -o relatime,flock file_cache_dns_name@tcp:/mountname /mnt

This command mounts my cache with two options:

  • relatime – Maintains atime (inode access times) data, but not for each time that a file is accessed. With this option enabled, atime data is written to disk only if the file has been modified since the atime data was last updated (mtime) or if the file was last accessed more than a certain amount of time ago (one day by default). relatime is required for automatic cache eviction to work properly.
  • flock – Enables file locking for your cache. If you don’t want file locking enabled, use the mount command without flock.

Once mounted, processes running on my EC2 instance can access files in the cache as usual. As I defined at cache creation time, the first ZFS file system is available inside the cache at /dataset1, and the second ZFS file system is available as /dataset2.

$ echo "Hello File Cache World" > /mnt/zsf1/greetings

$ sudo mount -t lustre -o relatime,flock /mnt/cache

$ ls -al /mnt/cache
total 98
drwxr-xr-x 5 root root 33280 Sep 21 14:37 .
drwxr-xr-x 2 root root 33280 Sep 21 14:33 dataset1
drwxr-xr-x 2 root root 33280 Sep 21 14:37 dataset2

$ cat /mnt/cache/dataset1/greetings
Hello File Cache World

I can observe and measure the activity and the health of my caches using Amazon CloudWatch metrics and AWS CloudTrail log monitoring.

CloudWatch metrics for a File Cache resource are organized into three categories:

  • Front-end I/O metrics
  • Backend I/O metrics
  • Cache front-end utilization metrics

As usual, I can create dashboards or define alarms to be informed when metrics reach thresholds that I defined.

Things To Keep In Mind
There are a couple of key points to keep in mind when using or planning to use File Cache.

First, File Cache encrypts data at rest and supports encryption of data in transit. Your data is always encrypted at rest using keys managed in AWS Key Management Service (AWS KMS). You can use either service-owned keys or your own keys (customer-managed CMKs).

Second, File Cache provides two options for importing data from your data repositories to the cache: lazy load and preload. Lazy load imports data on demand if it’s not already cached, and preload imports data at user request before you start your workload. Lazy loading is the default. It makes sense for most workloads since it allows your workload to start without waiting for metadata and data to be imported to the cache. Pre loading is helpful when your access pattern is sensitive to first-byte latencies.

Pricing and Availability
There are no upfront or fixed-price costs when using File Cache. You are billed for the provisioned cache storage capacity and metadata storage capacity. The pricing page has the details. In addition to File Cache itself, you pay for S3 request costs, AWS Direct Connect charges, and the usual data transfer charges for inter-AZ, inter-Region, and internet egress traffic between File Cache and the data sources.

File Cache is available in US East (Ohio), US East (N. Virginia), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Canada (Central), Europe (Frankfurt), Europe (Ireland), and Europe (London).

Now go build and create your first file cache today!

— seb

Amazon WorkSpaces Introduces Ubuntu Desktops

This post was originally published on this site

At AWS, we love to give our customers choices: the choice of infrastructure to deploy your workloads, to store your most important data, or the operating systems for your virtual desktops.

Many of you choose Amazon Workspaces to provision and distribute virtual desktops securely and at scale to your workforce. Our customers choose Workspaces when choosing a solution to enable secure remote workers or when they want to quickly provision desktop for contractors, just to name two use cases. When using Workspaces, you’ve had the choice between two operating systems: Microsoft Windows or Amazon Linux.

Starting today, you can also provision Ubuntu desktops for your developers, engineers, or data scientists. This allows virtual desktop users to have more choices and to embrace new categories of workloads. It also gives virtual desktop administrators a consistent experience whether you are managing Windows or Linux-based desktops for your workforce. You can now have one process and set of tools to manage the majority of your desktop operating systems.

According to the HackerEarth 2020 Developer Survey, Ubuntu is the most widely used operating system among professional developers (66 percent Ubuntu, 61 percent Windows, and 57 percent macOS). Many developers or DevOps engineers have a battery of scripts, tools, or libraries well tested on their Ubuntu desktops or laptops. Providing Ubuntu desktop on Workspaces gives developers and engineers a familiar and compatible environment allowing them to work from anywhere, with access to a wealth of open-source tools and libraries in cutting-edge fields like data science, AI/ML, cloud, and IoT. All Ubuntu WorkSpaces benefit from Ubuntu Pro, which includes support for expanded security patching for 10 years.

Having Ubuntu on Amazon Workspaces is important for you, but also for us, AWS, and for Canonical, the publisher of Ubuntu.

We’ve brought Ubuntu Desktop to WorkSpaces so developers can streamline the design, coding, pipelines, and deployment of Ubuntu-based workloads, whether instances or containers, all within the AWS environment,” said
Alex Gallagher, VP Cloud for
Canonical. “
Also, Ubuntu virtual desktops on WorkSpaces enable IT organizations to quickly and easily provision high-performance Ubuntu Desktop instances, delivered as a fully managed AWS service. In the face of constant and increasing pressure to support the security and productivity needs of hybrid workers, that’s a win for IT organizations and their end users.

Why a Virtual Desktop for Developers And Engineers?
There are several benefits to providing virtual cloud desktops to your technical workforce.

First, the security: all your files are securely stored on cloud-based volumes within the security perimeter of your AWS account. Application assets such as source code or design documents are not locally stored on end-user machines, and no data is at risk in case of a laptop being lost or stolen.

Second, the ease of provisioning at scale. Providing new desktops, including with the latest generation of hardware, is a matter of minutes. Onboarding new team members or contractors is accelerated, and can even be automated.

Third, the reduction in costs by paying on demand, either monthly or hourly, just for the time the resources are used.

Fourth, end-users may access their desktop from anywhere. They can work with the same desktop from on premises, from home, or when traveling.

And fifth, virtual desktop administrators now have a common administration experience across Windows and two distributions of Linux (Ubuntu and Amazon Linux).

How to Get Started
For IT administrators, provisioning Ubuntu desktops is no different than provisioning Windows or Amazon Linux desktops. You may choose the hardware bundle, with various sizes of CPUs and memory, depending on the end-user needs. Workspaces are bound to an Active Directory. You may use your on-premises AD or a fully managed directory service in the cloud.

For this blog post, I choose to show you the end-user experience instead. As a developer or engineer, once my desktop is provisioned, I receive an email like this one:

Workspaces provisioning emailI select the link and enter my password. Then, I download and install a client from

Workspaces create password

For this demo, I am using the Windows Workspaces client. After downloading and installing it, I enter the registration code I received by email.

Workspaces enter code

A few seconds later, I have access to my Ubuntu desktop.

Ubuntu desktop from Windows

Imagine I later travel to a family member’s place, and I have to access my work desktop. I can borrow a web browser and reconnect from the web client. Not surprisingly, I find my desktop in the exact state I left it—even the mouse pointer is in the same place. When closing a session, I am confident that I did not leave any confidential files behind me. All my files, messages, and applications are securely stored within my company AWS account.

Workspaces web client

Things You Need To Know
The service team is actively listening to your feedback and building new capabilities into Workspaces.

For this launch, we chose to use version 22.04 LTS (Jammy Jellyfish). Canonical has optimized the base image for cloud development use cases. We have preinstalled AWS CLI and SDK.

You may select from a selection of Value, Standard, Performance, Power, or Power Pro bundles. These bundles provide you with computer sizes starting at 1 vCPU and 2 GB RAM, up to 8 vCPU and 32 GB RAM.

There are two features of Workspaces that are not present today for the launch of Ubuntu Workspaces. They will be added in the very near future. First, you may connect today using the Windows or web-based Workspaces clients. Linux and macOS clients will be supported as a quick follow-up to this launch. Second, the GPU-based Graphics.g4dn, and GraphicsPro.g4dn bundles will be available in the coming weeks.

Availability and Pricing
Ubuntu Workspaces are available in all AWS Regions where Workspaces is available, except a Region in China: China (Ningxia). Again, we’re working hard to add support for China (Ningxia) as a quick follow-up after launch.

Workspace pricing is either per month, for users that primarily use Workspaces as their main desktop, or per hour for those in your workforce making occasional use of the workspace. Ubuntu Workspace prices start at $23 per month and per desktop or $0.19 per hour. Our pricing page has the details.

Now go build and start your first Ubuntu Workspaces today.

— seb

AWS IoT FleetWise Now Generally Available – Easily Collect Vehicle Data and Send to the Cloud

This post was originally published on this site

Today we announce the general availability of AWS IoT FleetWise, a fully managed AWS service that makes it easier to collect, transform, and transfer vehicle data to the cloud. Last AWS re:Invent 2021, we previewed AWS IoT FleetWise, heard customer feedback, and improved features for various use cases of near-real-time vehicle data processing.

With AWS IoT FleetWise, automakers, fleet operators, and automotive suppliers can take the complex variability out of collecting data from vehicle fleets at scale. You can access standardized fleet-wide vehicle data and avoid developing custom data collection systems, or you can integrate AWS IoT FleetWise to enhance your existing systems. AWS IoT FleetWise enables intelligent data collection that sends the exact data you need from the vehicle to the cloud. You can use the data to analyze vehicle fleet health to more quickly identify potential maintenance issues or make in-vehicle infotainment systems smarter. Furthermore, you can use it to train machine learning (ML) models that improve autonomous driving and advanced driver assistance systems (ADAS).

For example, electric vehicle (EV) battery temperature is a critical metric that should be continuously analyzed for the entire vehicle fleet. In order to avoid costly continuous data ingestion, you may want to optimize the data collection by setting a threshold on EV battery temperature. The results of this analysis would be provided to the automaker’s quality engineering department, enabling fast assessment of the criticality and possible root causes of any issues identified at certain temperatures. Based on the root cause analysis, the automaker can then take short-term actions to support the driver affected by the issue, as well as midterm actions to improve vehicle quality.

How AWS IoT FleetWise Works
AWS IoT FleetWise provides a vehicle modeling framework that you can use to model your vehicle and its sensors and actuators in the cloud. To enable secure communication between your vehicle and the cloud, AWS IoT FleetWise also provides the AWS IoT FleetWise Edge Agent application that you can use to download and install in-vehicle electronic control units (ECUs) such as the gateway, in-vehicle infotainment controller, etc. You define data collection schemes in the cloud and deploy them to your vehicle.

The AWS IoT FleetWise Edge Agent running in your vehicle uses data collection schemes to control what data to collect and when to transfer it to the cloud. Data collected and ingested through AWS IoT FleetWise Edge Agent software goes directly into your Amazon Timestream table or Amazon Simple Storage Service (Amazon S3) repositories via AWS IoT Core.

AWS IoT FleetWise Features
To get started with AWS IoT FleetWise, you can register your account and configure the settings via the AWS console. AWS IoT FleetWise automatically registers your AWS account, IAM role, and Amazon Timestream resources.

The Edge Agent software is a C++ application distributed as source code and is available on GitHub to collect, decode, normalize, cache, and ingest vehicle data to AWS. It supports multiple deployment options, such as vehicle gateways, infotainment systems, telematics control units (TCUs), or aftermarket devices. When vehicles are connected to the cloud, the Edge Agent continually receives data collection schemes and collects, decodes, normalizes and ingests the transformed vehicle data to AWS.

Let’s see the benefits and features of AWS IoT FleetWise:

Signal catalog
A signal catalog contains a collection of vehicle signals. Signals are fundamental structures that you define to contain vehicle data and its metadata. A signal can be a sensor and its status, an attribute as static information of the manufacturer, a branch to represent a nested structure such as Vehicle.Powertrain.combustionEngine expression, or an actuator such as the state of a vehicle device. For example, you can create a sensor to receive in-vehicle temperature values and store its metadata, including a sensor name, a data type, and a unit.

Signals in a signal catalog can be used to model vehicles that use different protocols and data formats. For example, there are two cars made by different automakers: one uses the Controller Area Network (CAN) to transmit the in-vehicle temperature data and the other uses On-board Diagnostic (OBD) protocol.

You can define a sensor in the signal catalog to receive in-vehicle temperature values. This sensor can be used to represent the thermocouples in both cars, irrespective of how this temperature data is available within the vehicle networks. For more information, see Create and manage signal catalogs in the AWS documentation.

Vehicle models
Vehicle models are virtual declarative representations that standardize the format of your vehicles and define relationships between signals in the vehicles. Vehicle models enforce consistent information across multiple vehicles of the same type so that you can quickly configure and create a vehicle fleet. In each vehicle model, you can add signals, including attributes, branches (signal hierarchies), sensors, and actuators.

You can define condition-based schemes to control what data to collect, such as data in-vehicle temperature values that are greater than 40 degrees. You can also define time-based schemes to control how often to collect data. For more information, see Create and manage vehicle models in the AWS documentation.

When a decoder manifest is associated with a vehicle model, you can create a vehicle. Each vehicle corresponds to an AWS IoT thing. You can use an existing AWS IoT thing to create a vehicle or set AWS IoT FleetWise to automatically create an AWS IoT thing for your vehicle. For more information, see Provision vehicles in the AWS documentation. After you create vehicles, you can create campaigns for them.

A campaign gives the AWS IoT FleetWise Edge Agent instructions on how to select, collect, and transfer data to the cloud. You can make a campaign with vehicle attributes that you added when creating vehicles, and a data collection scheme. You can manually define the data collection scheme either condition-based logical expressions such as $variable.myVehicle.InVehicleTemperature > 40.0, or time-based data collection in milliseconds such as from 10000 – 60000 milliseconds. To learn more, see Create a campaign in the AWS documentation.

After you create and approve the campaign, AWS IoT FleetWise automatically deploys the campaign to the listed vehicles. The AWS IoT FleetWise Edge Agent software doesn’t start collecting data until a running campaign is deployed to the vehicle. If you want to pause collecting data from vehicles connected to the campaign, on the Campaign summary page, choose Suspend. To resume collecting data from vehicles connected to the campaign, choose Resume.

Demo – Visualizing Vehicle Data
Here is a demo that aims to show how AWS IoT FleetWise can make it easy to collect vehicle data and use it to build visualizing applications. In this demo, you can simulate two kinds of vehicles, an NXP GoldBox powered by an Automotive Grade Linux distribution that runs the AWS IoT FleetWise agent as an AWS IoT Greengrass component or a completely virtual vehicle implemented as an AWS Graviton ARM-based Amazon EC2 instance. To learn more, see the getting started guide and source code in the GitHub repository.

The vehicle in CARLA Simulator can self-drive or be driven with a game steering wheel connected to your desktop. You can watch a live demo video.

Data is collected by AWS IoT FleetWise and stored in the Amazon Timestream table, and visualized on a Grafana Dashboard.

Customer and Partner Voices
During the preview period, we heard lots of feedback from our customers and partners in automotive industry such as automakers, fleet operators, and automotive suppliers.

For example, Hyundai Motor Group (HMG) is a global vehicle manufacturer that offers consumers a technology-rich lineup of cars, sport utility vehicles, and electrified vehicles. HMG has used AWS services, such as using Amazon SageMaker, to reduce its ML model training time for autonomous driving models.

Hae Young Kwon, vice president and head of the infotainment development group at HMG, said:

“As a leading global vehicle manufacturer, we have come to appreciate the breadth and depth of AWS services to help create new connected vehicle capabilities. With more data available from our expanding global fleet of connected cars, we look forward to leveraging AWS IoT FleetWise to discover how we can build more personalized ownership experiences for our customers.”

LG CNS is a global IT service provider and AWS Premier Consulting Partner that is transforming smart transportation services by building an advanced transportation system that is convenient and safe by maximizing the operational efficiency of multiple modes of transport, including buses, subways, taxis, railways, and airplanes.

Jae Seung Lee, vice president at LG CNS, said:

“At LG CNS, we are committed to advancing the technology that is powering the future of transportation. By using AWS IoT FleetWise, we are creating a new data platform that allows us to ingest, analyze, and simulate vehicle conditions in real-time. With these advanced insights, our customers can gain a better understanding of their vehicles and, as a result, improve decision-making about their fleets.”

Bridgestone is a global leader in tires and rubber building on its expertise to provide solutions for safe and sustainable mobility. Bridgestone has worked with AWS for several years to develop a system that delivers insights derived from the interaction between a tire and a vehicle using advanced machine learning capabilities on Amazon SageMaker.

Brian Goldstine, president of mobility solutions and fleet management at Bridgestone Americas Inc. said:

“Bridgestone has been working with AWS to transform the digital services we provide to our automotive manufacturer, fleet, and retail customers. We look forward to exploring how AWS IoT FleetWise will make it easier for our customers to collect detailed tire data, which can provide new insights for their products and applications.”

Renesas Electronics Corporation is a global leader in microcontrollers, analog, power, and system on chips (SoC) products. Renesas launched cellular-to-cloud IoT development platforms and its cloud development kits to run on AWS IoT Core and FreeRTOS.

Yusuke Kawasaki, director at Renesas Electronics Corporation, said:

“The volume of connected vehicle data is forecast to increase dramatically over the next few years, driven by new and evolving customer expectations. As a result, Renesas is focused on addressing the needs of automotive engineers facing increasing system complexity. Incorporating AWS IoT FleetWise into our vehicle gateway solution will enable our customers to enjoy our market-ready approach for large-scale data collection and accelerate their cloud development strategy. We look forward to further collaborating with AWS to provide a better and simpler development environment for our customers.”

By working with AWS IoT FleetWise Partners, you can take advantage of solutions to streamline your IoT projects, reduce the risk of your efforts, and accelerate time to value. To learn more how AWS accelerates the automotive industry’s digital transformation, see AWS for Automotive.

Now Available
AWS IoT FleetWise is now generally available in the US East (N. Virginia) and Europe (Frankfurt) Regions. You pay for the vehicles you have created and messages per vehicle per month. Additional services used alongside AWS IoT FleetWise, such as AWS IoT Core and Amazon Timestream, are billed separately. For more detail, see the AWS IoT FleetWise pricing page.

To learn more, see the AWS IoT FleetWise resources page including documentations, videos, and blog posts. Please send feedback to AWS re:Post for AWS IoT FleetWise or through your usual AWS support contacts.


AWS Week In Review — September 26, 2022

This post was originally published on this site

It looks like my travel schedule is coupled with this Week In Review series of blog posts. This week, I am traveling to Fort-de-France in the French Caribbean islands to meet our customers and partners. I enjoy the travel time when I am offline. It gives me the opportunity to reflect on the past or plan for the future.

Last Week’s Launches
Here are some of the launches that caught my eye last week:

Amazon SageMaker Autopilothas added a new Ensemble training mode powered by AutoGluon that is 8X faster than the current Hyper parameter Optimization Mode and supports a wide range of algorithms, including LightGBM, CatBoost, XGBoost, Random Forest, Extra Trees, linear models, and neural networks based on PyTorch and FastAI.

AWS Outposts and Amazon EKSYou can now deploy both the worker nodes and the Kubernetes control plane on an Outposts rack. This allows you to maximize your application availability in case of temporary network disconnection on premises. The Kubernetes control plane continues to manage the worker nodes, and no pod eviction happens when on-premises network connectivity is reestablished.

Amazon Corretto 19 – Corretto is a no-cost, multiplatform, production-ready distribution of OpenJDK. Corretto is distributed by Amazon under an open source license. This version supports the latest OpenJDK feature release and is available on Linux, Windows, and macOS. You can download Corretto 19 from our downloads page.

Amazon CloudWatch Evidently – Evidently is a fully-managed service that makes it easier to introduce experiments and launches in your application code. Evidently adds support for Client Side Evaluations (CSE) for AWS Lambda, powered by AWS AppConfig. Evidently CSE allows application developers to generate feature evaluations in single-digit milliseconds from within their own Lambda functions. Check the client-side evaluation documentation to learn more.

Amazon S3 on AWS OutpostsS3 on Outposts now supports object versioning. Versioning helps you to locally preserve, retrieve, and restore each version of every object stored in your buckets. Versioning objects makes it easier to recover from both unintended user actions and application failures.

Amazon PollyAmazon Polly is a service that turns text into lifelike speech. This week, we announced the general availability of Hiujin, Amazon Polly’s first Cantonese-speaking neural text-to-speech (NTTS) voice. With this launch, the Amazon Polly portfolio now includes 96 voices across 34 languages and language variants.

X in Y – We launched existing AWS services in additional Regions:

Other AWS News
Introducing the Smart City Competency program – The AWS Smart City Competency provides best-in-class partner recommendations to our customers and the broader market. With the AWS Smart City Competency, you can quickly and confidently identify AWS Partners to help you address Smart City focused challenges.

An update to IAM role trust policy behavior – This is potentially a breaking change. AWS Identity and Access Management (IAM) is changing an aspect of how role trust policy evaluation behaves when a role assumes itself. Previously, roles implicitly trusted themselves. AWS is changing role assumption behavior to always require self-referential role trust policy grants. This change improves consistency and visibility with regard to role behavior and privileges. This blog post shares the details and explains how to evaluate if your roles are impacted by this change and what to modify. According to our data, only 0.0001 percent of roles are impacted. We notified by email the account owners.

Amazon Music Unifies Music QueuingThe Amazon Music team published a blog post to explain how they created a unified music queue across devices. They used AWS AppSync and AWS Amplify to build a robust solution that scales to millions of music lovers.

Upcoming AWS Events
Check your calendar and sign up for an AWS event in your Region and language:

AWS re:Invent – Learn the latest from AWS and get energized by the community present in Las Vegas, Nevada. Registrations are open for re:Invent 2022 which will be held from Monday, November 28 to Friday, December 2.

AWS Summits – Come together to connect, collaborate, and learn about AWS. Registration is open for the following in-person AWS Summits: Bogotá (October 4), and Singapore (October 6).

Natural Language Processing (NLP) Summit – The AWS NLP Summit 2022 will host over 25 sessions focusing on the latest trends, hottest research, and innovative applications leveraging NLP capabilities on AWS. It is happening at our UK headquarters in London, October 5–6, and you can register now.

AWS Innovate for every app – This regional online conference is designed to inspire and educate executives and IT professionals about AWS. It offers dozens of technical sessions in eight languages (English, Spanish, French, German, Italian, Japanese, Korean, and Indonesian). Register today: Americas, September 28; Europe, Middle-East, and Africa, October 6; Asia Pacific & Japan, October 20.

AWS Innovate for every app

AWS Community DaysAWS Community Day events are community-led conferences to share and learn with one another. In September, the AWS community in the US will run events in Arlington, Virginia (September 30). In Europe, Community Day events will be held in October. Join us in Amersfoort, Netherlands (October 3), Warsaw, Poland (October 14), and Dresden, Germany (October 19).

AWS Tour du Cloud – The AWS Team in France has prepared a roadshow to meet customers and partners with a one-day free conference in seven cities across the country (Aix en Provence, Lille, Toulouse, Bordeaux, Strasbourg, Nantes, and Lyon), and in Fort-de-France, Martinique. Tour du Cloud France

AWS Fest – This third-party event will feature AWS influencers, community heroes, industry leaders, and AWS customers, all sharing AWS optimization secrets (this week on Wednesday, September). You can register for AWS Fest here.

Stay Informed
That is my selection for this week! To better keep up with all of this news, please check out the following resources:

— seb
This post is part of our Week in Review series. Check back each week for a quick roundup of interesting news and announcements from AWS!

AA22-265A: Control System Defense: Know the Opponent

This post was originally published on this site

Original release date: September 22, 2022


Traditional approaches to securing OT/ICS do not adequately address current threats.

Operational technology/industrial control system (OT/ICS) assets that operate, control, and monitor day-to-day critical infrastructure and industrial processes continue to be an attractive target for malicious cyber actors. These cyber actors, including advanced persistent threat (APT) groups, target OT/ICS assets to achieve political gains, economic advantages, or destructive effects. Because OT/ICS systems manage physical operational processes, cyber actors’ operations could result in physical consequences, including loss of life, property damage, and disruption of National Critical Functions.

OT/ICS devices and designs are publicly available, often incorporate vulnerable information technology (IT) components, and include external connections and remote access that increase their attack surfaces. In addition, a multitude of tools are readily available to exploit IT and OT systems. As a result of these factors, malicious cyber actors present an increasing risk to ICS networks.

Traditional approaches to securing OT/ICS do not adequately address current threats to those systems. However, owners and operators who understand cyber actors’ tactics, techniques, and procedures (TTPs) can use that knowledge when prioritizing hardening actions for OT/ICS.

This joint Cybersecurity Advisory, which builds on previous NSA and CISA guidance to stop malicious ICS activity and reduce OT exposure [1] [2], describes TTPs that malicious actors use to compromise OT/ICS assets. It also recommends mitigations that owners and operators can use to defend their systems. NSA and CISA encourage OT/ICS owners and operators to apply the recommendations in this CSA.

Download the PDF version of this report: pdf, 538.12 kb

Technical Details

OT/ICS assets operate, control, and monitor industrial processes throughout U.S. critical infrastructure. Traditional ICS assets are difficult to secure due to their design for maximum availability and safety, coupled with their use of decades-old systems that often lack any recent security updates. Newer ICS assets may be able to be configured more securely, but often have an increased attack surface due to incorporating Internet or IT network connectivity to facilitate remote control and operations. The net effect of the convergence of IT and OT platforms has increased the risk of cyber exploitation of control systems. [3]

Today’s cyber realm is filled with well-funded malicious cyber actors financed by nation-states, as well as less sophisticated groups, independent hackers, and insider threats. Control systems have been targeted by a variety of these malicious cyber actors in recent years to achieve political gains, economic advantages, and possibly destructive effects. [4] [5] [6] [7] [8] More recently, APT actors have also developed tools for scanning, compromising, and controlling targeted OT devices. [9] 

Malicious actors’ game plan for control system intrusions

Cyber actors typically follow these steps to plan and execute compromises against critical infrastructure control systems:

  1. Establish intended effect and select a target.
  2. Collect intelligence about the target system.
  3. Develop techniques and tools to navigate and manipulate the system.
  4. Gain initial access to the system.
  5. Execute techniques and tools to create the intended effect.

Leveraging specific expertise and network knowledge, malicious actors such as nation-state actors can conduct these steps in a coordinated manner, sometimes concurrently and repeatedly, as illustrated by real world cyber activity. [5] [10]

Establish intended effect and select a target

Cyber actors, from cyber criminals to state-sponsored APT actors, target critical infrastructure to achieve a variety of objectives. Cyber criminals are financially motivated and target OT/ICS assets for financial gain (e.g., data extortion or ransomware operations). State-sponsored APT actors target critical infrastructure for political and/or military objectives, such as destabilizing political or economic landscapes or causing psychological or social impacts on a population. The cyber actor selects the target and the intended effect—to disrupt, disable, deny, deceive, and/or destroy—based on these objectives. For example, disabling power grids in strategic locations could destabilize economic landscapes or support broader military campaigns. Disrupting water treatment facilities or threatening to destroy a dam could have psychological or social impacts on a population. [11] [12]

Collect intelligence about the target system

Once the intent and target are established, the actor collects intelligence on the targeted control system. The actor may collect data from multiple sources, including:

  • Open-source research: A great deal of information about control systems and their designs is publicly available. For example, solicitation information and employment advertisements may indicate components and—list specific model numbers.
  • Insider threats: The actor may also leverage trusted insiders, even unwitting ones, for collecting information. Social engineering often elicits a wealth of information from people looking for a new job or even just trying to help.
  • Enterprise networks: The actor may compromise enterprise IT networks and collect and exfiltrate ICS-related information. Procurement documents, engineering specifications, and even configurations may be stored on corporate IT networks.

In addition to OT-specific intelligence, information about IT technologies used in control systems is widely available. Knowledge that was once limited to control system engineers and OT operators has become easily available as IT technologies move into more of the control system environment. Control system vendors, in conjunction with the owner/operator community, have continually optimized and reduced the cost of engineering, operating, and maintaining control systems by incorporating more commodity IT components and technologies in some parts of OT environments. These advancements sometimes can make information about some systems easily available, thereby increasing the risk of cyber exploitation. 

Develop techniques and tools

Using the intelligence collected about the control system’s design, a cyber actor may procure systems that are similar to the target and configure them as mock-up versions for practice purposes. Nation-state actors can easily obtain most control system equipment. Groups with limited means can still often acquire control systems through willing vendors and secondhand resellers.

Access to a mock-up of the target system enables an actor to determine the most effective tools and techniques. A cyber actor can leverage resident system utilities, available exploitation tools; or, if necessary, develop or purchase custom tools to affect the control system. Utilities that are already on the system can be used to reconfigure settings and may have powerful troubleshooting capabilities. 

As the control system community has incorporated commodity IT and modernized OT, the community has simplified the tools, techniques, scripts, and software packages used in control systems. As a result, a multitude of convenient tools are readily available to exploit IT and OT systems.

Actors may also develop custom ICS-focused malware based on their knowledge of the control systems. For example, TRITON malware was designed to target certain versions of Triconex Tricon programmable logic controllers (PLCs) by modifying in-memory firmware to add additional programming. The extra functionality allows an actor to read/modify memory contents and execute custom code, disabling the safety system. [13] APT actors have also developed tools to scan for, compromise, and control certain Schneider Electric PLCs, OMRON Sysmac NEX PLCs, and Open Platform Communications Unified Architecture (OPC UA) servers. [9] 

With TTPs in place, a cyber actor is prepared to do virtually anything that a normal system operator can, and potentially much more.

Gain initial access to the system

To leverage the techniques and tools that they developed and practiced, cyber actors must first gain access to the targeted system. 

Most modern control systems maintain remote access capabilities allowing vendors, integrators, service providers, owners, and operators access to the system. Remote access enables these parties to perform remote monitoring services, diagnose problems remotely, and verify warranty agreements. 

However, these access points often have poor security practices, such as using default and maintenance passwords. Malicious cyber actors can leverage these access points as vectors to covertly gain access to the system, exfiltrate data, and launch other cyber activities before an operator realizes there is a problem. Malicious actors can use web-based search platforms, such as Shodan, to identify these exposed access points. 

Vendor access to control systems typically use connections that create a bridge between control system networks and external environments. Often unknown to the owner/operator, this bridge provides yet another path for cyber exploitation and allows cyber actors to take advantage of vulnerabilities in other infrastructure to gain access to the control system. 

Remote access points and methodologies use a variety of access and communication protocols. Many are nothing more than vendor-provided dial-up modems and network switches protected only by obscurity and passwords. Some are dedicated devices and services that communicate via more secure virtual private networks (VPNs) and encryption. Few, if any, offer robust cybersecurity capabilities to protect the control system access points or prevent the transmission of acquired data outside the relatively secure environment of the isolated control system. This access to an ostensibly closed control system can be used to exploit the network and components.

Execute techniques and tools to create the intended effects

Once an actor gains initial access to targeted OT/ICS system, the actor will execute techniques, tools, and malware to achieve the intended effects on the target system. To disrupt, disable, deny, deceive, and/or destroy the system, the malicious actor often performs, in any order or in combination, the following activities:

  1. Degrade the operator’s ability to monitor the targeted system or degrade the operator’s confidence in the control system’s ability to operate, control, and monitor the targeted system. Functionally, an actor could prevent the operator’s display (human machine interface, or HMI) from being updated and selectively update or change visualizations on the HMI, as witnessed during the attack on the Ukraine power grid. [5] (Manipulation of View [T0832] )
  2. Operate the targeted control system. Functionally, this includes the ability to modify analog and digital values internal to the system (changing alarms and adding or modifying user accounts), or to change output control points — this includes abilities such as altering tap changer output signals, turbine speed demand, and opening and closing breakers. (Manipulation of Control [T0831])
  3. Impair the system’s ability to report data. Functionally, this is accomplished by degrading or disrupting communications with external communications circuits (e.g., ICCP , HDLC , PLC , VSAT, SCADA radio, other radio frequency mediums), remote terminal units (RTUs) or programmable logic controllers (PLCs), connected business or corporate networks, HMI subnetworks, other remote I/O, and any connected Historian/bulk data storage. (Block Reporting Message [T0804], Denial of View [T0815])
  4. Deny the operator’s ability to control the targeted system. Functionally, this includes the ability to stop, abort, or corrupt the system’s operating system (OS) or the supervisory control and data acquisition (SCADA) system’s software functionality. (Denial of Control [T0813])
  5. Enable remote or local reconnaissance on the control system. Functionally, an actor could obtain system configuration information to enable development of a modified system configuration or a custom tool. (Collection [TA0100], Theft of Operational Information [T0882])

Using these techniques, cyber actors could cause various physical consequences. They could open or close breakers, throttle valves, overfill tanks, set turbines to over-speed, or place plants in unsafe operating conditions. Additionally, cyber actors could manipulate the control environment, obscuring operator awareness and obstructing recovery, by locking interfaces and setting monitors to show normal conditions. Actors can even suspend alarm functionality, allowing the system to operate under unsafe conditions without alerting the operator. Even when physical safety systems should prevent catastrophic physical consequences, more limited effects are possible and could be sufficient to meet the actor’s intent. In some scenarios though, if an actor simultaneously manipulates multiple parts of the system, the physical safety systems may not be enough. Impacts to the system could be temporary or permanent, potentially even including physical destruction of equipment. 


The complexity of balancing network security with performance, features, ease-of-use, and availability can be overwhelming for owner/operators. This is especially true where system tools and scripts enable ease-of-use and increase availability or functionality of the control network; and when equipment vendors require remote access for warranty     compliance, service obligations, and financial/billing functionality. However, with the increase in targeting of OT/ICS by malicious actors, owner/operators should be more cognizant of the risks when making these balancing decisions. Owner/operators should also carefully consider what information about their systems needs to be publicly available and determine if each external connection is truly needed. [1] 

System owners and operators cannot prevent a malicious actor from targeting their systems. Understanding that being targeted is not an “if” but a “when” is essential context for making ICS security decisions. By assuming that the system is being targeted and predicting the effects that a malicious actor would intend to cause, owner/operators can employ and prioritize mitigation actions.

However, the variety of available security solutions can also be intimidating, resulting in choice paralysis. In the midst of so many options, owner/operators may be unable to incorporate simple security and administrative strategies that could mitigate many of the common and realistic threats. Fortunately, owner/operators can apply a few straightforward ICS security best practices to counter adversary TTPs. 

Limit exposure of system information

Operational and system information and configuration data is a key element of critical infrastructure operations. The importance of keeping such data confidential cannot be overstated. To the extent possible, avoid disclosing information about system hardware, firmware, and software in any public forum. Incorporate information protection education into training for personnel. Limit information that is sent out from the system.

Document the answers to the following questions:

  1. From where and to where is data flowing?
  2. How are the communication pathways documented and how is the data secured/encrypted?
  3. How is the data used and secured when it arrives at its destination?
  4. What are the network security standards at the data destination, whether a vendor/regulator or administrator/financial institution? 
  5. Can the data be shared further once at its destination? Who has the authority to share this data?

Eliminate all other data destinations. Share only the data necessary to comply with applicable legal requirements, such as those contractually required by vendors—nothing more. Do not allow other uses of the data and other accesses to the system without strict administrative policies designed specifically to protect the data. Prevent new connections to the control system using strict administrative accountability. Ensure strict agreements are in place with outside systems/vendors when it comes to sharing, access, and use. Have strong policies for the destruction of such data. Audit policies and procedures to verify compliance and secure data once it gets to its destination, and determine who actually has access to it. 

Identify and secure remote access points

Owner/operators must maintain detailed knowledge of all installed systems, including which remote access points are—or could be—operating in the control system network. Creating a full “connectivity inventory” is a critical step in securing access to the system.

Many vendor-provided devices maintain these access capabilities as an auxiliary function and may have services that will automatically ‘phone home’ in an attempt to register and update software or firmware. A vendor may also have multiple access points to cover different tasks. 

Once owner/operators have identified all remote access points on their systems, they can implement the following recommendations to improve their security posture:

  • Reduce the attack surface by proactively limiting and hardening Internet-exposed assets. See CISA’s Get Your Stuff Off Search page for more information.
  • Establish a firewall and a demilitarized zone (DMZ) between the control system and the vendor’s access points and devices. Do not allow direct access into the system; use an intermediary service to share only necessary data and only when required. For more information see CISA’s infographic Layering Network Security Through Segmentation. [14]
  • Consider using virtual private networks (VPNs) at specific points to and from the system rather than allowing separate access points for individual devices or vendors.
  • Utilize jump boxes to isolate and monitor access to the system.
  • Ensure that data can only flow outward from the system – administratively and physically. Use encrypted links to exchange data outside of the system.
  • Enforce strict compliance with policies and procedures for remote access, even if personnel complain that it is too difficult.
  • If the system does not use vendor access points and devices, ensure that none are active. Use strict hardware, software, and administrative techniques to prevent them from becoming covertly active.
  • Do not allow vendor-provided system access devices and software to operate continuously in the system without full awareness of their security posture and access logs.
  • Install and keep current all vendor-provided security systems associated with the installed vendor access points.
  • Review configurations to ensure they are configured securely. Operators typically focus on necessary functionality, so properly securing the configurations and remote access may be overlooked. 
  • Consider penetration testing to validate the system’s security posture and any unknown accesses or access vulnerabilities. 
  • Add additional security features to the system as needed. Do not assume that one vendor has a monopoly on the security of their equipment; other vendors may produce security features to fill gaps. 
  • Change all default passwords throughout the system and update any products with hard-coded passwords, especially in all remote access and security components.
  • Patch known exploited vulnerabilities whenever possible. Prioritize timely patching of all remote access points. Keep operating systems, firewalls, and all security features up-to-date.
  • Continually monitor remote access logs for suspicious accesses. Securely aggregate logs for easier monitoring.

Restrict tools and scripts 

Limit access to network and control system application tools and scripts to legitimate users performing legitimate tasks on the control system. Removing the tools and scripts entirely and patching embedded control system components for exploitable vulnerabilities is often not feasible. Thus, carefully apply access and use limitations to particularly vulnerable processes and components to limit the threat.

The control system and any accompanying vendor access points may have been delivered with engineering, configuration, and diagnostic tools pre-installed. Engineers use these tools to configure and modify the system and its processes as needed. However, such tools can also be used by a malicious actor to manipulate the system, without needing any special additional tools. Using the system against itself is a powerful cyber exploitation technique. Mitigations strategies include:

  1. Identify any engineering, configuration, or diagnostic tools.
  2. Securely store gold copies of these tools external to the system if possible.
  3. Remove all non-critical tools.
  4. Prevent these tools from being reinstalled.
  5. Perform routine audits to check that these tools have not been reinstalled.

Conduct regular security audits

The owner/operator of the control system should consider performing an independent security audit of the system, especially of third-party vendor access points and systems. The owner/operator cannot solely depend on the views, options, and guidance of the vendor/integrator that designed, developed, or sold the system. The goal of such an audit is to identify and document system vulnerabilities, practices, and procedures that should be eliminated to improve the cyber defensive posture, and ultimately prevent malicious cyber actors from being able to cause their intended effects. Steps to consider during an audit include the following:

  1. Validate all connections (e.g., network, serial, modem, wireless, etc.).
  2. Review system software patching procedures.
  3. Confirm secure storage of gold copies (e.g., OS, firmware, patches, configurations, etc.).
  4. Verify removal from the system of all non-critical software, services, and tools.
  5. Audit the full asset inventory. 
  6. Implement CISA ICS mitigations and best practices. [15] [16]
  7. Monitor system logs and intrusion detection system (IDS) logs.

Implement a dynamic network environment

Static network environments provide malicious actors with persistent knowledge of the system. A static network can provide cyber actors the opportunity to collect bits of intelligence about the system over time, establish long-term accesses into the system, and develop the tools and TTPs to affect the control system as intended. 

While it may be unrealistic for the administrators of many OT/ICS environments to make regular non-critical changes, owner/operators should consider periodically making manageable network changes. A little change can go a long way to disrupt previously obtained access by a malicious actor. Consider the following:

  1. Deploy additional firewalls and routers from different vendors.
  2. Modify IP address pools.
  3. Replace outdated hardware (e.g., workstations, servers, printers, etc.).
  4. Upgrade operating systems.
  5. Install or upgrade commercially available security packages for vendor access points and methodologies.

Planning these changes with significant forethought can help minimize the impact on network operation.

Owner/operators should familiarize themselves with the risks to the system as outlined by the product vendor. These may be described in manuals as the system using insecure protocols for interoperability or certain configurations that may expose the system in additional ways. Changes to the system to reduce these risks should be considered and implemented when feasible.


The combination of integrated, simplified tools and remote accesses creates an environment ripe for malicious actors to target control systems networks. New IT-enabled accesses provide cyber actors with a larger attack surface into cyber-physical environments. It is vital for OT/ICS defenders to anticipate the TTPs of cyber actors combining IT expertise with engineering know-how. Defenders can employ the mitigations listed in this advisory to limit unauthorized access, lock down tools and data flows, and deny malicious actors from achieving their desired effects. 

Disclaimer of endorsement

The information and opinions contained in this document are provided “as is” and without any warranties or guarantees. Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer, or otherwise, does not constitute or imply its endorsement, recommendation, or favoring by the United States Government, and this guidance shall not be used for advertising or product endorsement purposes.


This advisory was developed by NSA and CISA in furtherance of their cybersecurity missions, including their responsibilities to develop and issue cybersecurity specifications and mitigations. This information may be shared broadly to reach all appropriate stakeholders.

Contact Information

For NSA client requirements or general cybersecurity inquiries, contact To report incidents and anomalous activity or to request incident response resources or technical assistance related to these threats, contact CISA at

Media Inquiries / Press Desk: 



  • Initial Release: September 22, 2022

This product is provided subject to this Notification and this Privacy & Use policy.

Phishing Campaigns Use Free Online Resources, (Wed, Sep 21st)

This post was originally published on this site

A phishing campaign needs some resources: bandwidth, CPU, storage, … For a very long time, a lot of phishing kits have been hosted on compromised servers. The most popular are CMS with weak configurations or outdated. I think that WordPress is the number one in this category. By careful, it does not mean that WordPress is a bad CMS. Most vulnerabilities are introduced through plugins. Once compromised, the phishing kit files are copied on the server and usually are reachable via the /wp-content/ or /wp-plugin/ directories.

Skyline Insights API – How to get Results using Python

This post was originally published on this site

Tweet In the past, we have shown you how to leverage Skyline Insights API using Curl with shell scripting and Powershell.  Here let’s look at the same task using Python.  The formula is still the same.  You will need to use your API token to get your Access Token. Then you can query Skyline for … Continued

The post Skyline Insights API – How to get Results using Python appeared first on VMware Support Insider.