AWS

AWS Glue 6.0 now available with 30% lower price and full Apache Iceberg v3 support

Leave a comment
This post was originally published on this site

Today, we are announcing the general availability of AWS Glue 6.0, delivering 30% lower pricing than previous AWS Glue versions and introducing full support for Apache Iceberg v3 features. AWS Glue 6.0 is built on a fully modernized runtime, Apache Spark 4.1, Python 3.12, and Scala 2.13, delivering faster performance.

With this release, AWS Glue provides the most complete Iceberg v3 implementation on any fully serverless managed Spark service, along with new capabilities that simplify ETL authoring, improve PySpark performance, and enable real-time streaming with single-digit millisecond latency.

What is new in AWS Glue 6.0
AWS Glue 6.0 delivers the complete Apache Iceberg v3 specification, built on Iceberg 1.11.0. The headline feature is the VARIANT data type with shredding support, which achieves faster query read performance compared to traditional string data type columns for semi-structured data.

With VARIANT shredding, you can store and query JSON, logs, and event data without flattening schemas, eliminating duplicate data copies, custom parsing code, and pipeline breakage when schemas change. This capability transforms how teams handle semi-structured data at scale.

Additional Iceberg v3 capabilities include:

  • Geometry and Geography data types: Enable native spatial processing for GIS analytics, location intelligence, and geospatial data pipelines directly on managed Spark.
  • Nanosecond-precision timestamps: Support IoT sensor data, scientific computing, and high-frequency financial workloads that require precision beyond standard milliseconds.
  • Unknown type handling: Process data with unexpected or evolving schemas without pipeline failures, providing resilience against upstream schema changes.

AWS Glue 6.0 also includes most significant upgrade in Spark 4.1, the modern runtime engine:

  • Spark declarative pipelines: Spark Declarative Pipelines introduces a simplified approach to ETL authoring. Data engineers declare transformations, specifying what data should look like, while the engine automatically determines execution order and optimization. This reduces the complexity of pipeline development and eliminates manual orchestration overhead.
  • Arrow-native Python UDFs and UDTFs: AWS Glue 6.0 introduces Arrow-native execution for Python User-Defined Functions (UDFs) and User-Defined Table Functions (UDTFs). This eliminates serialization overhead between Python and the JVM, improving PySpark performance for complex transformations.
  • Real-time streaming mode: For stateless streaming use cases, AWS Glue 6.0 introduces a real-time streaming mode that achieves single-digit millisecond latency. Built on Spark 4.1’s Real-Time Mode with Glue-optimized execution, this capability supports real-time event processing, low-latency data transformation pipelines, and time-sensitive data routing.

Getting started with AWS Glue 6.0
No API changes are required to use AWS Glue 6.0. You can select the new version using the existing --glue-version parameter in the create-job or update-job APIs through AWS Command Line Interface (AWS CLI), AWS SDK, AWS Glue Studio, Amazon SageMaker Unified Studio, and your preferred IDE.

To get started with AWS Glue 6.0 jobs in the AWS Glue Studio console, open the AWS Glue job and on the Job Details tab, choose the version Glue 6.0 – Supports Spark 4.1, Scala 2, Python 3. You can create new AWS Glue jobs on AWS Glue 6.0 to get the benefit from the improvements, or migrate your existing AWS Glue jobs.

To start using AWS Glue 6.0 on an AWS Glue Studio notebook or an interactive session through a Jupyter notebook, set 6.0 in the %glue_version magic. You can also upgrade existing jobs to Glue 6.0 using the Spark upgrade agent on AWS Glue Studio or use the auto-upgrade feature in their existing Glue jobs to automatically upgrade them to Glue 6.0.

To learn more, visit the AWS Glue 6.0 version detail and Migrating AWS Glue for Spark jobs to AWS Glue version 6.0 in the AWS documentation.

Now available
AWS Glue 6.0 is generally available today in all AWS Regions where AWS Glue operates. For Regional availability and a future roadmap, visit the AWS Capabilities by Region. If you want to call APIs, search documentation, find regional availability, and check troubleshooting about this new feature, try using the AWS MCP Server and plugins with your preferred AI tool.

You pay an hourly rate, billed by the second, for crawlers (discovering data) and extract, transform, and load (ETL) jobs (processing and loading data). For the AWS Glue Data Catalog, you pay a simplified monthly fee for storing and accessing the metadata. The first million objects stored are free, and the first million accesses are free. To learn more, visit AWS Glue Pricing page.

Give it a try in the AWS Glue Studio console, and send feedback to AWS re:Post for AWS Glue or through your usual AWS support contacts.

— Channy

Security

Using Microsoft Graph and Powershell to Mine for Information – Stale Accounts and Licenses, (Thu, Aug 20th)

Leave a comment
This post was originally published on this site

Microsoft Graph is a newer API that is meant to replace several others.  OK, it's at version 2.3.9, so it's not all that new, but it's new enough that lots of folks (and commercial tools) aren't using it yet.   It allows you to Get and Set info from/to M365, Entra Users and Entra managed machines for starters.  Let's dig in!

Security

Simple Scans for Cloud Metadata Service, (Wed, Aug 19th)

Leave a comment
This post was originally published on this site

Cloud providers typically expose a REST API at 169.254.169.254 that allows code running on virtual machines to retrieve machine-specific data. Some of the data is more or less harmless, such as the region the machine is running in or its MAC and IP addresses. However, the service may also be used to retrieve credentials for IAM roles and service account tokens.

AWS

AWS Weekly Roundup: EC2 application status checks, IAM role manager, OpenAI Daybreak on Bedrock, and more (August 17, 2026)

Leave a comment
This post was originally published on this site

Last week, the OpenSearch and Valkey teams visited Seoul to meet open source developers and contributors in the Open Source Summit Korea 2026 and MCP DevSummit Seoul 2026. At the four-day event, community leaders and users of open source projects and emerging agent AI gathered to share knowledge, collaborate on solutions, and push the projects forward.

Leaders of the Korean OpenSearch communities volunteered to participate in the booth, and also had time to network and interact in the user group meetup.

OpenSearch is an open source, enterprise-grade search and observability suite that brings order to unstructured data at scale. On June 9, 2026, OpenSearch 3.7 introduced new tools designed to query, alert, and track SLOs across logs, traces, and metrics through a single interface and retrieve vectors up to 5.5x faster for improved search performance. Since July 30, 2026, you can run OpenSearch version 3.7 on Amazon OpenSearch Service for improvements in vector search performance, search relevance, and Query Insights.

Valkey is an open source high-performance key/value datastore that supports a variety of workloads such as caching, message queues, and it can act as a primary database. On May 19, 2026, Valkey 9.1 introduced a redesigned I/O threading model that improves throughput by up to 17% and reduces memory usage for strings under 128 bytes by up to 20%. Since June 23, 2026, you can run Valkey 9.1 in Amazon ElastiCache for node-based clusters, delivering higher throughput, improved memory efficiency, and stronger access control for multi-tenant workloads.

You can meet our open source teams at upcoming OpenSearch and Valkey events.

Last week’s launches
Here are some launches that got my attention:

For a full list of AWS announcements, be sure to keep an eye on the What’s New with AWS page.

Other AWS news
Here are some additional projects and news items you may find interesting:

  • The deprecation of email validation in AWS Certificate Manager: ACM will discontinue support for email-validated public certificates by September 30, 2027. If you use email validation for your ACM public certificates, you need to migrate to DNS validation before that date. For Amazon CloudFront distributions, HTTP validation is also available.
  • The next-generation AWS VPN Client with CLI support and admin controls: You can use a new AWS VPN Client built on OpenVPN3. With the new client, you get full backward compatibility with existing AWS Client VPN endpoints while delivering the automation capabilities and security posture that enterprise networking teams have been asking for.
  • Oracle Exadata on Exascale for Oracle AI Database@AWS: ExaDB-XS brings Exadata-class performance and availability through a consumption-based model. With ExaDB-XS, you can scale compute and storage independently in small increments and pay only for what you consume.

For a full list of AWS blog posts, be sure to keep an eye on the AWS Blogs page.

Learn more about AWS, browse and join upcoming AWS-led in-person and virtual events, startup events, and developer-focused events including AWS Summits and AWS Community Days. Join the AWS Builder Center to connect with builders, share solutions, and access content that supports your development.

That is all for this week. Check back next Monday for another Weekly Roundup!

— Channy

Security

Apple Screen Sharing Security, (Mon, Aug 17th)

Leave a comment
This post was originally published on this site

About 20 years ago, with macOS 10.5 (Leopard), Apple introduced screen sharing. Apple did not invent a new protocol for screen sharing. Instead, it used the established VNC protocol. VNC is a pretty simple, unencrypted protocol using TCP port 5900. Historically, the protocol used a simple global password for authentication. Apple adapted the protocol for its own use, but overall, left the VNC protocol itself alone.

A couple of weeks ago, two severe vulnerabilities exposed issues Apple introduced when it bolted on its own modifications to VNC. Currently, these vulnerabilities are being exploited, and a system with screen sharing exposed should be considered compromised. But here are some tips to improve screen sharing security.

One weakness exposed by these recent vulnerabilities is Apple's support for both "regular" VNC authentication and authentication via Apple's own macOS authentication system.  

Apple does allow old-fashioned VNC authentication by defining a VNC password. If this authentication scheme is used, a VNC client is prompted only for a password, not a username. The client may then ask for permission to use the screen, or they will be presented with an OS login prompt. This can be useful if you are trying to provide remote support to a logged-in user. But it does provide access to the system without any strong authentication. Access should still be secured by local user credentials, but the process already runs with elevated privileges to allow access for any user who logs in. This contributed to a recent vulnerability.

Next, you can restrict which users can remotely access the system. This should be restricted to allow only users who need remote access to connect.

Access to screen sharing can also be controlled via macOS's built-in firewall. But the settings are not always clear. Just enabling the firewall is not sufficient.

If "Automatically allow built-in software" is enabled, the firewall will allow access to screen sharing. The same is true for "Automatically allow downloaded signed software". Even if "stealth mode" is enabled, screen sharing is still available. You may also select "Block all incoming connections", which will block everything, even applications you approved in the past.

Here are a few command-line tips to secure the system (this is for macOS 26; prior versions use slightly different syntax)

# use this to check the current firewall state
# /usr/libexec/ApplicationFirewall/socketfilterfw --getglobalstate
# turn firewall on
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setglobalstate on
# turn stealth mode on to not respond to pings
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setstealthmode on
# do not allow signed binaries
sudo /usr/libexec/ApplicationFirewall/socketfilterfw --setallowsigned off
# disable filesharing
sudo launchctl disable system/com.apple.smbd
# disable screensharing
sudo launchctl disable system/com.apple.screensharing

A script like this is handy if you need to switch from your internal network to a public one. VNC access should always happen via a VPN. SSH forwarding works well with VNC. Other solutions, like Tailscale, are easy to use if you need VNC for remote support.

—
Johannes B. Ullrich, Ph.D. , Dean of Research, SANS.edu
Twitter|

(c) SANS Internet Storm Center. https://isc.sans.edu Creative Commons Attribution-Noncommercial 3.0 United States License.

Security

Using Gemma4 with Ollama – Testing File Hash Analysis and Recommendations with AI, (Wed, Aug 12th)

Leave a comment
This post was originally published on this site

In the past few weeks, I have been using Gemma4 as a Large Language Model (LLM) to see how useful it can be to analyze some of the malware hashes uploaded to the DShield sensor over the past 30 days and figure out how its recommendation can be considered useful about the activity my DShield sensor is collecting and tracking. The model I use for this testing is gemma4:e4b [2] using two sites to compare the data against VirusTotal and CyberGordon.