This post was originally published on this site

PowerShell 7.2 Preview 2

Today we are proud to announce the second preview release of PowerShell 7.2.
This preview is still based on .NET 5 as we wait for the first preview of .NET 6 which we expect PowerShell 7.2 to be based upon.

This preview includes many changes including code cleanup, bug fixes, and a few new features.

Code cleanup

The community has made significant contributions to code cleanup
which is a focus early in a new release.
Approximately two thirds of the 120 pull requets were for code cleanup!

Thanks to all the community members involved in submitting pull requests and reviewing them!

Notable bug fixes

Although we appreciate all bug fixes from the community, there are a few I believe have a broader impact and worth mentioning.

Correct handling of Windows invalid reparse points

On Windows, reparse points are a collection of user-defined data that define specific filesystem behaviors.
For example, symbolic links, OneDrive files, and Microsoft installed applications use reparse points.
Due to a bug introduced in PowerShell 7.1, if you try to use an executable on a drive that isn’t NTFS, you’ll get an Incorrect Function error.
This can be a local USB drive or a network share, for example.

Thanks to our community maintainer Ilya Sazonov for the fix.

We expect to backport this fix to PowerShell 7.1 for the next servicing release.

Breaking changes

-PipelineVariable common parameter

The -PipelineVariaable common parameter
now correctly contains all the objects passed in from the pipeline making script cmdlets work the same as C# cmdlets instead of just the first input object.

You can see an example of the change in behavior in the original issue.

Thanks to Joel Sallow for the fix.

New features

$PSStyle automatic variable for ANSI rendering

When working in the console with a modern terminal, color and text effects can help
make text information more interesting and useful.

This experimental feature called PSAnsiRendering exposes a new $PSStyle automatic variable that can be used for two different purposes.

The first is to make it easier to author text content that contains ANSI escape codes which control
text decorations like color, bold, italics, etc…

This example simply dumps the contents of $PSStyle and shows you the members you can use and their effect on text as well as the actual ANSI escape sequence.
Note that the custom formatting for this variable includes nested types like Formatting, Foreground, and Background.

You can use multiple ANSI escape sequences together.
In this example, I’ve set warning messages to have bold and italicized yellow text on a magenta background:

There are also FromRgb() methods available to make use of full 24-bit color if your terminal supports it:

C# module authors can also leverage $PSStyle by using the PSStyle singleton class in the System.Management.Automation namespace:

string text = $"{PSStyle.Instance.Reverse}{PSStyle.Instance.Foreground.Green}PowerShell{PSStyle.Instance.Foreground.Yellow} Rocks!{PSStyle.Instance.Reset}";

You can control how PowerShell outputs strings that contain ANSI escape sequences by setting $PSStyle.OutputRendering:

• Automatic
  This is the default and currently will output the text as-is whether it is to the host or through the pipeline if the
  terminal supports ANSI escape sequences (otherwise the output will be plaintext). This is similar behavior to what you
  would get on Linux.
• Ansi
  This value will output the text as-is whether it is to the host or through the pipeline.
• PlainText
  This value will remove ANSI escape sequences from any text output whether it is to the host or through the pipeline.
• Host
  This value will output the text as-is if sent to the host if ANSI escape sequences are supported, but will output plaintext
  if the output is sent through the pipeline or redirected. This is similar behavior to what you would get on macOS.

As this is an experimental feature, we encourage feedback on this before we make a decision to take it out of experimental.
See the original issue for additional details, but open new issues if you have any problems or
suggestions on how to improve this feature.

We very much appreciate on going feedback on our preview releases so we can make adjustments before the release is finalized.
Please participate in on going discussions or create new issues in our repo.

Thanks again to the PowerShell community and all the amazing contributors!

Steve Lee
Pricipal Software Engineer Manager
PowerShell Team

The post PowerShell 7.2 Preview 2 release appeared first on PowerShell.

This post was originally published on this site

 

The PowerShell gallery experienced downtime on October 30th 2020. This report will give context as to what caused the downtime, what actions were taken to mitigate the issue, and what steps we are taking to improve the PowerShell gallery experience moving forward.

Downtime Impact

The downtime was declared at 2020-10-30 03:29 PDT, and was mitigated about 12 hours later at 2020-10-30 15:39 PDT. During this time packages were not available from the gallery, and the web interface was not accessible.

Root Cause of the Downtime

The downtime was a result of an attempt to fix ongoing statistics errors with the gallery. For roughly 3 weeks the PowerShell gallery was experiencing many server errors (roughly 100-200 per minute) due to a key that had reached a max int value (total downloads reached over 2 billion) and was causing persistent int overflow errors on the gallery. This prevented new entries from being added to the ‘PackageStatistics’ table (required for the intermediary processing of statistics). The int overflow first occurred on 9/18/2020.

After an attempt to perform database migrations failed due to the persistent errors manual updates were made to the database to fix inflated package statistics numbers.
These changes triggered a series of deadlock and timeout errors which consumed all our available cloud resources.
This caused a spike in DTU/CPU utilization for the database which inversely correlated with the availability for the service. The availability for the gallery was so low that it was non-functional and declared down.

Mitigating the Downtime

The first mitigation step was to restore the gallery database (DB) to a previous timestamp. It was believed that an error in the attempted fix of gallery statistics caused the DB to get into a bad state and thus restoring the DB reverted those changes. This initial error was likely due to a trigger on the database that we did not account for. Unfortunately, reverting the DB caused additional issues. Checking the PowerShell gallery backend logs, we saw that the service had trouble connecting to the DB with an error that user credentials were wrong. This indicated that the user had been orphaned by the restore so we re-created the user in the DB. After this step, checking the PowerShell gallery backend logs again, the service had additional trouble connecting to the DB with an error that login was failing. We determined that this error was caused by the DB restore dropping the DB from the gallery’s failover group. The next mitigation step was to re-add the DB to the gallery’s failover group. The final mitigation step was to restart the cloud services so they could re-connect to the failover group. At this point the gallery started working again. We validated these fixes with customers, as well as with our own testing and continued to closely monitor the DTU/CPU utilization and service availability.

Statistics Errors

The gallery has had ongoing issues with the package statistics since August 2020.

These errors came from the gallery reaching a scale (more than 2 billion installations) that was not supported by the design of the statistics pipeline. The impact of this has been both incorrect and unavailable package statistics. The package statistics from 2020-09-18 through 2020-10-07 were never recorded, which meant we were unable to recover statistics from this period.

Restoring Statistics

We restored statistics in two ways, first we repaired statistics for individual packages (surfaced on a package’s page), and then we repaired aggregated statistics (surfaced on the gallery homepage and statistics page).

In ordered to repair package statistics we updated values in our main database and within the code base itself, that referenced a key for package statistics from an integer to a bigint/long. There was some pending data that was dropped when the int overflow error first appeared. We retrieved specific ‘lost’ data from a restored database, but were unfortunately unable to recover some data (mentioned in Statistics Errors).

To repair the aggregated statistics, we then made parallel changes to our data warehouse.

PowerShell Gallery Repairs

Our repair items are focused on 3 categories: detect, diagnose, and fix. By focusing on these three areas, we hope to not only improve the overall performance of the gallery but also, more quickly find and mitigate issues as they arise.

• Detect:
  • Add more notifications to the production database
  • Create alerts for when critical metrics are reached in the DB
  • Improve post-deployment validation so that we can quickly roll back undesirable changes
• Diagnose:
  • Send database logs to a central location outside of the service so that logs are more easily available
• Fix:
  • Improve the deployment process for gallery cloud services
  • Better document (internal) procedures for recovery and communication during an outage

 

We are also in the process of designing architectural changes to the PowerShell gallery, to ensure this is a reliable, performant, and supportable service moving forward.

Expectations going forward

In conjunction with these repairs, we are working to set and monitor Service Level Objectives (SLOs). Look forward to a future post detailing these expectations and how gallery users can track our progress against these objectives.

Reporting Issues

If you notice any issues with the PowerShell gallery please open an issue in our GitHub repository.

If you are a package owner and have an issue with your package please use our support alias: cgadmin@microsoft.com.

We continue to update the status of the PowerShell gallery at: aka.ms/PSGalleryStatus.

Sydney

PowerShell Team

 

The post Incident Report – PowerShell Gallery Downtime October 30, 2020 appeared first on PowerShell.