Tag Archives: Microsoft

Announcing PowerShell Crescendo Preview.2

This post was originally published on this site

We are pleased to announce the second preview of PowerShell Crescendo, a framework to rapidly
develop PowerShell cmdlets for native commands, regardless of platform.

The updated preview releases are now available for download on the PowerShell Gallery:

To install Microsoft.PowerShell.Crescendo:

Install-Module Microsoft.PowerShell.Crescendo -AllowPrerelease

For more information on Microsoft.PowerShell.Crescendo, check out these previous blog posts:

Crescendo Preview 2 Updates

This update to Crescendo adds elevation support for native commands, and generation of a module
manifest when exporting a Crescendo module. Read the full list of changes below:

  • Added support for native command elevation on Windows and Linux platforms. (Issue #50)
  • Export-CrescendoModule now exports a module manifest (psd1) along with the module file (psm1). (Issue #51)
  • Added support for generating aliases for Crescendo cmdlets (Issue #52)
  • Added support for SupportsShouldProcess. Thanks jdhitsolutions! (Issue #59)

Native Command elevation

Native commands may require administrative elevation to perform the requested operation. The
Crescendo schema has been extended to support elevation on Windows and Linux/macOS platforms. The schema
supports the new keyword Elevation that has two properties Command and Arguments.

  • Command: Defines the elevation mechanism to be used to elevate the native command. The function
    Invoke-WindowsNativeAppWithElevation has been included to aid with elevation on Windows. sudo
    has been tested as well.
  • Arguments: These are the parameters to be used in conjunction with the elevation command. This can
    be a collection of parameters.

    • OriginalName: This is the parameter to be used with the elevation command.
    • DefaultValue: The default parameter value.

Windows Native Command Elevation

Elevation on Windows is supported using the built-in function
Invoke-WindowsNativeAppWithElevation. Invoke-WindowsNativeAppWithElevation uses Start-Process
with a PSCredential to invoke the native command and captures the output and errors of the native
command which are returned to the user. This function is inserted into the Crescendo module you
export. In the example below, the PowerShell cmdlet Get-Credential will prompt for credentials
necessary to elevate the native command when the exported Crescendo function is executed.

"Elevation": {
        "Command": "Invoke-WindowsNativeAppWithElevation",
        "Arguments": [
            {
                "OriginalName" : "-Credential",
                "DefaultValue": "(get-credential)"
            }
        ]
    },

In automation scenarios that require elevation without user interaction, Crescendo supports
retrieving credentials from secured vaults. In the example below, credentials for elevation are
retrieved from SecretManagement and SecretStore.

For more information about storing and managing secrets, see [SecretManagement Announcement](https://devblogs.microsoft.com/powershell/secretmanagement-and-secretstore-are-generally-available/

"Elevation": {
        "Command": "Invoke-WindowsNativeAppWithElevation",
        "Arguments": [
            {
                "OriginalName" : "-Credential",
                "DefaultValue": "(get-secret admin)"
            }
        ]
    },

Elevation can be defined for each cmdlet. When authoring a Crescendo json configuration, include
the elevation definition before the parameters are defined. In the example below, the native command
netsh.exe requires elevation to enable and disable the Windows Firewall.

{
    "$schema": "./Microsoft.PowerShell.Crescendo.Schema.json",
    "Verb": "Set",
    "Noun": "WinFirewall",
    "Platform": ["Windows"],
    "OriginalCommandElements": ["advfirewall", "set", "allprofiles", "state"],
    "OriginalName": "$env:Windir/system32/netsh.exe",
    "Elevation": {
        "Command": "Invoke-WindowsNativeAppWithElevation",
        "Arguments": [
            {
                "OriginalName" : "-Credential",
                "DefaultValue": "(get-credential)"
            }
        ]
    },
    "DefaultParameterSetName": "Enable",
    "Parameters": [
        {
            "OriginalName": "on",
            "Name": "On",
            "ParameterType": "switch",
            "ParameterSetName": ["Enable"]
        },
        {
            "OriginalName": "off",
            "Name": "Off",
            "ParameterType": "switch",
            "ParameterSetName": ["Disable"]
        }
    ]
}

Linux Native Command Elevation

Native command elevation for Linux and macOS is handled through the command sudo. To enable
elevation for Crescendo, include sudo as the Command value. In the example below, when the
function Get-TimeServer is executed, the user is prompted for the sudo password. The native
command is elevated with sudo privileges. In automation scenarios that require elevation without
user interaction, configure the sudoers file.

{
    "$schema": "../src/Microsoft.PowerShell.Crescendo.Schema.json",
    "Verb": "Get",
    "Noun": "TimeServer",
    "Elevation": {
        "Command": "sudo"
    },
    "OriginalCommandElements": ["-getnetworktimeserver"],
    "OriginalName": "/usr/sbin/systemsetup",
    "Platform": ["MacOS"],
    "OutputHandlers": [
        {
            "ParameterSetName": "Default",
            "Handler": "$args|%{[pscustomobject]@{ TimeServer = $_.Split(':')[1].Trim()}}"
        }
    ]
}

SupportsShouldProcess

Cmdlets that cause change to the system, such as files and service configurations, create a risk
that could negatively impact operations. To help mitigate risk, cmdlets support common parameters
-WhatIf and -Confirm. These parameters provide greater detail about the target of each
change, and provides a confirmation mechanism allowing the user to approve each change. Crescendo
supports this functionality when wrapping native commands though the schema keyword
SupportsShouldProcess.

In the example below, The keyword SupportsShouldProcess accepts a boolean to enable the common
parameters -WhatIf and -Confirm.

{
    "$schema": "../src/Microsoft.PowerShell.Crescendo.Schema.json",
    "Verb": "Remove",
    "Noun": "DockerImage",
    "OriginalName": "docker",
    "SupportsShouldProcess": true,
    "OriginalCommandElements": [
        "image",
        "rm"
    ],
    "Parameters": [
        {
            "Name": "ID",
            "OriginalName": "",
            "Mandatory": true,
            "ValueFromPipelineByPropertyName": true
        }
    ]
}

Executing Remove-DockerImage with the -WhatIf parameter supported by SupportsShouldProcess.

Get-DockerImage | Where-Object {$_.id -match "d70eaf7277ea"} | Remove-DockerImage -WhatIf

When SupportsShouldProcess is set True, the following expected output will result.

What if: Performing the operation "Remove-DockerImage" on target "docker image rm d70eaf7277ea".

Aliases

Aliases for Crecendo wrapped native commands are now supported in the Crescendo schema with the
keyword Aliases. In the snippet below, the cmdlet definition Set-WinFirewall includes the alias
definition SFW. When exported, the Crescendo created module will include the Set-WinFirewall
function and SFW alias.

{
    "$schema": "./Microsoft.PowerShell.Crescendo.Schema.json",
    "Verb": "Set",
    "Noun": "WinFirewall",
    "Platform": ["Windows"],
    "OriginalCommandElements": ["advfirewall", "set", "allprofiles", "state"],
    "OriginalName": "$env:Windir/system32/netsh.exe",
    "Aliases": ["SFW"],
    "Elevation": {
        "Command": "Invoke-WindowsNativeAppWithElevation",
        "Arguments": [
            {
                "OriginalName" : "-Credential",
                "DefaultValue": "(get-credential)"
            }
        ]
    },

Future plans

Next/Future plans for preview.3 will be based on feedback and include a few items under investigation:

  • Improved OutputHandler support for building objects from native command string output.
  • Support for multiple cmdlet definitions in a single json configuration.

Our goal is to make it easier to convert your native commands to PowerShell cmdlets and receive the
benefits that PowerShell provides. We value your ideas and feedback and hope you will give Crescendo
a try, then stop by our GitHub repository and let us know of any issues or features you would like
added.

For more information about PowerShell Crescendo issues and features, see:
Crescendo on GitHub

Jason Helmick

Program Manager, PowerShell

The post Announcing PowerShell Crescendo Preview.2 appeared first on PowerShell Team.

Optimizing your $Profile

This post was originally published on this site

Optimizing your $Profile

Your PowerShell Profile allows you to customize
your PowerShell session and runs at startup.
Complex profiles can cause a significant delay in the startup of PowerShell as it is a script that needs to be executed before the prompt first
shows up.

Spoiler: I’ll show how I got my profile loading time from 1465 ms to 217 ms!

It’s important to note that many of the optmizations I cover here are micro-optimizations.
They won’t have significant overall impact and, in general, may not be the best way to write your scripts.
In the case of a profile, I’m optimizing for speed and, in some cases, making the script slightly harder to read as a tradeoff.
For production scripts used in automation, the micro performance improvement may not be worthwhile as maintainability of the script
may be more important.

Using GitHub to store my profile.ps1

As part of working on PowerShell on GitHub, I have a macBook Pro, a Windows desktop, and also a Linux desktop I use frequently.
I like to keep my PowerShell environment customizations in sync across these devices without having to manually update the profile should I make changes.
The solution I decided upon was to publish my profile as a GitHub Gist.

NOTE: that the current version on GitHub contains all the optimizations covered in this blog post, you can look at the revisions on GitHub
to see how my profile has changed over time.
I would NOT recommend using my profile directly as I may make changes that break you or are not applicable to your daily usage of PowerShell.
Use it more as an example.

The key parts of the code that does this is a ThreadJob that makes a REST call to GitHub
to retrieve the latest version of my profile and compare with the current version.
Creating the ThreadJob does take time, but because I’m making a networking call which can lead to variable execution time, it’s a worthwhile
tradeoff.
Also, because that scriptblock is running as a separate thread from the main startup thread, I don’t have to worry about performance optimizations
of that ThreadJob scriptblock.

My profile contains a # Version x.y.z comment near the top of the script and this version string is used to determine if the one on GitHub
is newer than the one that’s currently running.
A regular expression is used to parse this from the profile.
If the ThreadJob determines there is a newer version, it saves that version string to a file that I can easily check at the start of my profile.
At the start of my profile, I check the current loaded version against this file and will prompt to install the latest version if there is one.
This means that on startup, I won’t know that a newer version exists until the next startup, but it’s a worthwhile tradeoff.

Getting a baseline for pwsh

Before we start making changes, we want to get a baseline so we know what impact, if any, our changes are affecting the startup performance.
I wanted to separate the startup time of pwsh itself from the execution of my profile.
I got an average startup, in milliseconds, for pwsh starting up without loading any profile.

Here I use the Measure-Command cmdlet to make this easy.
However, I do a loop of 100 times to make sure any variance of my computer are accounted for and calculate the average time:

$p = 0
1..100 | ForEach-Object {
    Write-Progress -Id 1 -Activity 'pwsh' -PercentComplete $_
    $p += (Measure-Command {
        pwsh -noprofile -command 1
    }).TotalMilliseconds 
}
Write-Progress -id 1 -Activity 'profile' -Completed
$p = $p/100
$p

I’m using the variable $p to store the result as I want to subtract that time from my profile measurements.
Since running this 100 times can take some time, I like to know how far it’s progressed, so I’m using Write-Progress as a visual indicator of
how many more need to be run.
Since the writing of progress is not in the scriptblock used by Measure-Command, it has no impact on the measured time.
pwsh -noprofile -command 1 will ensure that when PowerShell starts, it doesn’t load my profile, and the command 1 simply has PowerShell
emit the number 1 and exit.

For my baseline, I got a time of 1176 ms for the startup of pwsh.

Getting a baseline for my profile

To get the average start time of my profile, the script is quite similar:

$a = 0
1..100 | ForEach-Object {
    Write-Progress -Id 1 -Activity 'profile' -PercentComplete $_
    $a += (Measure-Command {
        pwsh -command 1
    }).TotalMilliseconds
}
Write-Progress -id 1 -activity 'profile' -Completed
$a/100 - $p

The only major difference here is not using -noprofile so that my profile is loaded and also subtracting the startup time of pwsh $p from the result.
I got a time of 1465 ms for the startup of my profile.

Measure-Script

Mathias Jessen published a great profiling tool for scripts called PSProfiler
that I decided to use against my profile to see which lines were taking the most time.

The module doesn’t require PowerShell 7, but if you use it with PowerShell 7.2, then you get some coloring to help identify the top lines that
are taking the longest execution time:

Measure-Script image

This screenshot shows a few lines I should focus on optimizing first.
At the top, you can see that reading contents of a file was identified as a performance hot spot.

Get-Content

One thing that is not obvious is that when you use the Get-Content cmdlet, it will add some additional note properties to the output identifying
the original source of the content.

For my usage in the profile, I have no need for this metadata.
The cmdlet has a -ReadCount parameter that specifies how to batch lines sent through the pipeline.
However, it also tells the cmdlet not to add the additional source information note properties.
Since I’m not using the pipeline to process the contents, using -ReadCount 0 will help avoid incurring the cost of the cmdlet adding the note properties.

This saved a very tiny amount of milliseconds although a good percentage improvement.
Using Get-Content $profile -Raw was 0.807888 ms, Get-Content $profile -Raw -ReadCount 0 was 0.651742 ms.
Instead of relying on the cmdlet, I could simply call the equivalent .NET API directly.
Since I’m not using the features of the cmdlet and simply getting the contents of the file, I could use [System.IO.File]::ReadAllText($profile) which took 0.185826 ms.

Now this is a much more significant improvement.
Also, these measurements were taken when the Get-Content cmdlet from the Management module was already loaded.
So for profile startup where the module is not loaded, this would be even more significant.

Changing cmdlets to using .NET APIs

Using cmdlets can make writing scripts much easier and potentially easier to read.
However, there is a startup time impact of loading the module (which itself is a series of complex steps), having the cmdlet process the input, and finally wrapping
the result in a PSObject and emitting to the pipeline.
Since my profile was not using any of the advanced capabilities of many cmdlets, I proceeded to change most of my cmdlet usage to .NET APIs.

In the screenshot, you can see that cmdlets like Join-Path and Resolve-Path are another line that is relatively slow and my profile uses these two cmdlets quite often.
Join-Path can be replaced by [System.IO.Path]::Combine().
I was using Resolve-Path ~ as Windows doesn’t understand that the tilde is supposed to point to the user’s home directly.
However, PowerShell already has a $HOME variable for that purpose, so I simply removed the use of Resolve-Path and used $HOME directly instead.

Taking another measurement

With all the changes from using cmdlets to .NET APIs, I was able to drop the start time to 1404 ms from 1465 ms.
This was pretty good for minimal work, but not enough of an improvement that I would notice on a daily basis.

Using Measure-Script again, I can see that the new hot spots were with Get-Command to see if some commands were available before doing some work.
There isn’t an equivalent .NET API to Get-Command, however, I could just try executing a command or cmdlet and catch the CommandNotFoundException.

Get-Command import-foo -ErrorAction Ignore takes about 15.8 ms.
However, try { import-foo } catch [System.Management.Automation.CommandNotFoundException] { } only takes about 10 ms.

With a few more changes I would also be reduce the profile start time to 1360 ms.
This was about a 100 ms improvement, but still not something I would notice on a daily basis.
To make an improvement that was noticeable I needed to take a different approach.

A new hope

It occurred to me that lots of the script I had in my profile were only ever intended for when I used PowerShell interactively.
This includes customizations to PSReadLine, ensuring dotnet was in my path, creating PSDrives as shortcuts to my test folder and git folders.
So the big change was to move all of that into a new private function called Initialize-Profile.
I would then update my prompt function to check if this function was ever called and if not, then call it to complete setting up my environment.

This meant that the majority of the script was moved to Initialize-Profile which was only ever run if there was an interactive prompt.
With this change, my profile startup reduced to 217 ms!

In a way, this is cheating a little bit as it will delay the first time I see a prompt when PowerShell starts up.
However, it’s a definite psycological impact not seeing the message from PowerShell telling me my profile was taking a large amount of the overall startup time.
There are other improvements I could make to defer more initialization until needed.
For example, only adding some tools to my $env:PATH when my file location is in a git repo.

Closing

  • Use tools like PSProfiler to identify hot spots in your scripts to know where to focus optimization efforts.
  • Understand trade offs for when to use cmdlets vs .NET APIs.
  • Micro-performance improvements do not necessarily translate to real world benefits.
  • Defer some initialization in your profile to the prompt if you can separate parts of your profile for interactive use vs automation.

The post Optimizing your $Profile appeared first on PowerShell Team.

SecretStore Release Candidate 3

This post was originally published on this site

The SecretStore release candidate 3 (RC3) module is now available on the PowerShell Gallery. This contains an exciting new feature which allows users to non-interactively create, and configure a SecretStore. This feature was added to support CI systems and other automated scenarios.

SecretStore is an extension vault module, for PowerShell SecretManagement, which works over all supported PowerShell platforms on Windows, Linux, and macOS. For more context on this module and the SecretManagement module refer to the previous blog posts:

For more context on these modules check out these previous blog posts:

Before installing this module, please uninstall the current preview versions of the module and restart your PowerShell session.

To install these updates run the following commands:

Uninstall-Module Microsoft.PowerShell.SecretStore -Force
# Restart your PowerShell session
Install-Module -Name Microsoft.PowerShell.SecretStore -Repository PSGallery
Register-SecretVault -Name SecretStore -ModuleName Microsoft.PowerShell.SecretStore -DefaultVault -AllowClobber

SecretStore Updates

Previously, Set-SecretStoreConfiguration required manual password confirmation to make changes to SecretStore configuration. This update adds a -Password parameter to Set-SecretStoreConfiguration to endable automated creation and configuration of a SecretVault.

Breaking Change

  • The -Force parameter was removed from the Set-SecretStoreConfiguration command, and instead the -Confirm:$false should be used to suppress PowerShell confirmation prompting in automation scripts.

New Feature

  • Set-SecretStoreConfiguration command now takes a -Password parameter so that there is no need to prompt for a password

How to non-interactively create and configure a SecretStore

This is an example of automation script that installs and configures the Microsoft.PowerShell.SecretStore module without user prompting. The configuration requires a password and sets user interaction to None, so that SecretStore will never prompt the user. The configuration also requires a password, and the password is passed in as a SecureString object. The -Confirm:false parameter is used so that PowerShell will not prompt for confirmation.

The SecretStore password must be provided in a secure fashion. Here the password is being imported from an encrypted file using Windows Data Protection API, but this is a Windows only solution. Another option is to use a CI system mechanism such as secure variables.

Next, the SecretManagement module is installed and the SecretStore module registered so that the SecretStore secrets can be managed.

The Unlock-SecretStore cmdlet is used to unlock the SecretStore for this session. The password timeout was configured for 1 hour and SecretStore will remain unlocked in the session for that amount of time, after which it will need to be unlocked again before secrets can be accessed.

Install-Module -Name Microsoft.PowerShell.SecretStore -Repository PSGallery -Force
$password = Import-CliXml -Path $securePasswordPath

Set-SecretStoreConfiguration -Scope CurrentUser -Authentication Password -PasswordTimeout 3600 -Interaction None -Password $password -Confirm:$false

Install-Module -Name Microsoft.PowerShell.SecretManagement -Repository PSGallery -Force
Register-SecretVault -Name SecretStore -ModuleName Microsoft.PowerShell.SecretStore -DefaultVault

Unlock-SecretStore -Password $password

General Availability (GA)

This is a “go live” release, which means that we feel that this RC is feature complete and supported in production.

If no bugs are identified through this release, we will increment the versioning and declare the module as GA in late March. If any high-risk bugs are identified we will continue to release RCs until the quality bar is met for a GA release.

Feedback and Support

Community feedback has been essential to the iterative development of these modules. Thank you to everyone who has contributed issues, and feedback thus far! To file issues or get support for the SecretManagement interface or vault development experience please use the SecretManagement repository. For issues which pertain specifically to the SecretStore and its cmdlet interface please use the SecretStore repository.

Sydney Smith

PowerShell Team

 

 

The post SecretStore Release Candidate 3 appeared first on PowerShell Team.

Updating help for the PSReadLine module in Windows PowerShell 5.1

This post was originally published on this site

Back in November of 2020 I
posted instructions
about how to work around the problem of updating the help for the PSReadLine module.

The version of the PSReadline module that shipped in Windows PowerShell 5.1 used a lowercase
letter in the name. The name of the module was changed for the release of PowerShell 6. It now uses
a capital L in the name. And even after following the instructions in the previous post, you still
get the same error when you try to update help for Windows PowerShell 5.1.

Failed to update Help for the module(s) ‘PSReadline

The root problem in Windows PowerShell 5.1 is that the module is automatically loaded at startup
using the PSReadline name (with the lowercase l). Use the following steps workaround this
problem and update the help.

  1. Close all powershell.exe processes. Then rename the PSReadline module folder name to
    PSReadLine.
  2. Open an powershell.exe session in Administrator privilege.
  3. Run Remove-Module PSReadLine to remove the auto-loaded PSReadLine module.
  4. Run Import-Module PSReadLine (use PSReadLine with capital L).
  5. Then run Update-Help -Module PSReadLine -Force

These steps load the module in your session using the new name, which allows Update-Help to use the
correct name to find the help content.

The post Updating help for the PSReadLine module in Windows PowerShell 5.1 appeared first on PowerShell Team.

Announcing PSReadLine 2.2-beta.2 with Dynamic help

This post was originally published on this site

We are pleased to announce the new feature Dynamic Help in
PSReadLine 2.2.0-beta.2.
Features like TabCompletetion and the Help system make PowerShell easier to use. Expanding on that
theme, Dynamic Help provides just-in-time help at the command line. This experience allows you to
stay focused on your work in the terminal without losing your place typing on the pipeline.

Supported Releases

Dynamic Help is implemented and presented through the PSReadLine module. In an effort to bring the
benefits of Dynamic Help to the most users, Dynamic Help is supported down-level to Windows
PowerShell 5.1.

PSReadLine 2.2.0-beta2, including Dynamic Help, is supported in the following versions:

  • Windows PowerShell 5.1
  • PowerShell 7.0+

Install PSReadLine with Dynamic Help

To receive the benefits of Dynamic Help, download and install the PSReadLine module from PSGallery.

Install-Module PSReadLine -AllowPrerelease

Getting Cmdlet Help

Dynamic Help provides a view of full cmdlet help shown on an alternative screen buffer using a cross
platform Pager utility. PSReadLine maps the function ShowCommandHelp to the F1 key.

  • When the cursor is at the end of a fully expanded cmdlet, pressing F1 displays the help for
    that cmdlet.
  • When the cursor is at the end of a fully expanded parameter, pressing F1 displays the help
    beginning at the parameter.

Image DynHelp1

The Pager in PSReadLine supports viewing content one screen at a time. To navigate the displayed
help content, press the up and down arrow keys to scroll the screen. Pressing Q exits the
alternative screen buffer and returns to the current cursor location in the the primary screen
buffer.

The Pager is a work in progress and we welcome your feedback and suggestions. To provide feedback
and for more information, see the Pager repository on GitHub.

Focused Parameter Help

Pressing the Alt-h key combination provides dynamic help for parameters. The help is shown below
the current command line like MenuComplete. The cursor must be at the end of the fully-expanded
parameter name when you press the Alt-h key.

Image DynHelp Alt H

Note


The PSReadLine function ShowParameterHelp is bound to Alt-h.
The Alt bindings do not currently work on macOS as detailed in this
issue. Mac users will need to map
ShowParameterHelp function to an available key combination.

Set-PSReadLineKeyHandler -chord "Ctrl-l" -Function ShowParameterHelp

Selection of Arguments

To rapidly select and change the arguments of a cmdlet without disturbing your syntax, press
Alt-a. Selection of arguments is scoped within a script block. Based on the cursor position, it
searches from the innermost script block to the outmost script block, and stops when it finds any
arguments in a script block scope.

Image dynhelpA

Note


The PSReadLine function SelectCommandArgument is bound to
Alt-a. The Alt bindings do not currently work on macOS as detailed in this
issue. Mac users will need to map
SelectCommandArgument function to an available key combination.

Set-PSReadLineKeyHandler -chord "Ctrl-k" -function SelectCommandArgument

Feedback and Support

Community feedback is essential to the iterative development of PSReadLine and features like Dynamic
Help. Thank you for your continued support and engagement. To file issues for PSReadLine or Dynamic
Help, please visit the PSReadLine repository.

Jason Helmick

PowerShell Team

The post Announcing PSReadLine 2.2-beta.2 with Dynamic help appeared first on PowerShell Team.

PowerShell for Visual Studio Code Updates – February 2021

This post was originally published on this site

 

We are excited to announce that updates to our PowerShell extension and PowerShell Preview extension are now available on the Visual Studio Code marketplace. This blog will explain what is new in these releases as well as what you can expect from the extension in the coming months.

What’s new in the PowerShell Extension release

This incremental release incorporates changes from four preview releases! Some highlights of the release include:

For the full list of updates please refer to the changelog. Further goals of this release are well discussed on GitHub.

What’s new in PowerShell Preview release

This preview release contains updates to our build infrastructure, bug fixes, and updates to our language server client. For the full list of updates please refer to the changelog.

This release contains a breaking change which removes PowerShell notebook mode. This feature, which was only available on Visual Studio Code insiders in the PowerShell preview extension, was removed due to changes to the preview notebook APIs breaking the functionality of the feature. We have chosen to prioritizes fixes which we believe will improve the stability and reliability of the extension overall in the short term and hope to re-invest in PowerShell extension integration with the Visual Studio code notebook APIs once they stabilize.

A PowerShell notebook experience in Visual Studio Code insiders is still available through the .NET Interactive Notebooks extension.

What’s been happening since the last release

This is our first stable release of the PowerShell extension since June 2020. The time between these releases was longer than we anticipated and would have liked. We recognize that in the time since users have had to deal with longstanding bugs and performance deficiencies. This gap in releases reflects competing priorities across the PowerShell engineering team but does not reflect a shift in investment or commitment to Visual Studio Code as the premier free development environment for PowerShell.

In January 2021 we were also excited to welcome Andy to the PowerShell extension development team. With their support we plan to increase the cadence of improvements for the extension in the coming months.

What’s next for the extensions

Over the coming months we plan to improve the extension with the following areas of focus:

We are also currently investigating new feature areas for the extension like Predictive IntelliSense integrations for the editor, GitHub Codespaces, and notebook integrations. You can track the progress on all of these projects in our GitHub repository.

Getting support and giving feedback

If you encounter any issues with the PowerShell extension in Visual Studio Code or have feature requests, the best place to get support is through our GitHub repository.

Sydney Smith, PowerShell Team

 

The post PowerShell for Visual Studio Code Updates – February 2021 appeared first on PowerShell Team.

Announcing PowerShell Community Blog

This post was originally published on this site

Announcing PowerShell Community Blog

We want to share the exciting news about the new
PowerShell Community Blog. Since the retirement of the Scripting
Guy (Ed Wilson) the Scripting blog has had fewer new posts. With the rapid ongoing growth of
PowerShell, this means fewer community members finding the help and answers they need to be
successful.

Community members, along with some partner teams within Microsoft, have taken up the challenge to
refresh and revitalize the blog. Now focused exclusively on PowerShell and with new
features/technology driven by the community.

We are happy to help get the word out and announce the new
PowerShell Community Blog. Focused on PowerShell with new and
relevant content from the community.

Wait! If you’re wondering about all that valuable content on the
original Scripting blog, don’t worry! All the work
from the original blog will remain intact and available to you. In fact, some of the
older-but-popular content will get updated and posted on the new community blog.

Visit the new blog, and
while you’re there, share your problems and knowledge with the community and help everyone automate
solutions with PowerShell.

Jason Helmick

PowerShell Team

The post Announcing PowerShell Community Blog appeared first on PowerShell Team.

SecretManagement and SecretStore Release Candidates

This post was originally published on this site

The SecretManagement and SecretStore release candidate (RC) modules are now available on the PowerShell Gallery.

The SecretManagement module helps users manage secrets by providing a common set of cmdlets to interface with secrets across vaults. This module supports an extensible model where local and remote vaults can be registered and unregistered for use in accessing and retrieving secrets. SecretStore is a cross-platform local extension vault for use with SecretManagement. We designed this vault as a best attempt at creating a vault that is available where PowerShell is, usable in popular PowerShell scenarios (like automation and remoting) and utilizes common security practices.

For more information on these modules check out these previous blog posts:

Before installing these modules, please uninstall the current preview versions of the modules and restart your PowerShell session.

To install these updates run the following commands:

Uninstall-Module Microsoft.PowerShell.SecretManagement -Force 
Uninstall-Module Microsoft.PowerShell.SecretStore -Force 
# Restart your PowerShell session 
Install-Module -Name Microsoft.PowerShell.SecretManagement -Repository PSGallery 
Install-Module -Name Microsoft.PowerShell.SecretStore -Repository PSGallery 
Register-SecretVault -Name SecretStore -ModuleName Microsoft.PowerShell.SecretStore -DefaultVault -AllowClobber

SecretManagement Updates

  • Register-SecretVault no longer emits error when strict language mode is set
  • Set-DefaultVault cmdlet has been renamed to Set-SecretVaultDefault

General Availability (GA)

This is a “go live” release, which means that we feel that this RC is feature complete and of GA quality. If no bugs are identified through this release, we will increment the versioning and declare the modules as GA in early February. If any high-risk bugs are identified we will continue to release RCs until the quality bar is met for a GA release.

The Extension Vault Ecosystem

To find other SecretManagement extension vault modules, search the PowerShell Gallery for the “SecretManagement” tag. Some community vault extensions that are available:

Thank you to everyone who has created vaults thus far!

Feedback and Support

Community feedback has been essential to the iterative development of these modules. Thank you to everyone who has contributed issues, and feedback thus far! To file issues or get support for the SecretManagement interface or vault development experience please use the SecretManagement repository. For issues which pertain specifically to the SecretStore and its cmdlet interface please use the SecretStore repository.

Sydney Smith

PowerShell Team

 

The post SecretManagement and SecretStore Release Candidates appeared first on PowerShell.

PowerShell 7.2 Preview 2 release

This post was originally published on this site

PowerShell 7.2 Preview 2

Today we are proud to announce the second preview release of PowerShell 7.2.
This preview is still based on .NET 5 as we wait for the first preview of .NET 6 which we expect PowerShell 7.2 to be based upon.

This preview includes many changes including code cleanup, bug fixes, and a few new features.

Code cleanup

The community has made significant contributions to code cleanup
which is a focus early in a new release.
Approximately two thirds of the 120 pull requets were for code cleanup!

Thanks to all the community members involved in submitting pull requests and reviewing them!

Notable bug fixes

Although we appreciate all bug fixes from the community, there are a few I believe have a broader impact and worth mentioning.

Correct handling of Windows invalid reparse points

On Windows, reparse points are a collection of user-defined data that define specific filesystem behaviors.
For example, symbolic links, OneDrive files, and Microsoft installed applications use reparse points.
Due to a bug introduced in PowerShell 7.1, if you try to use an executable on a drive that isn’t NTFS, you’ll get an Incorrect Function error.
This can be a local USB drive or a network share, for example.

Thanks to our community maintainer Ilya Sazonov for the fix.

We expect to backport this fix to PowerShell 7.1 for the next servicing release.

Breaking changes

-PipelineVariable common parameter

The -PipelineVariaable common parameter
now correctly contains all the objects passed in from the pipeline making script cmdlets work the same as C# cmdlets instead of just the first input object.

You can see an example of the change in behavior in the original issue.

Thanks to Joel Sallow for the fix.

New features

$PSStyle automatic variable for ANSI rendering

When working in the console with a modern terminal, color and text effects can help
make text information more interesting and useful.

This experimental feature called PSAnsiRendering exposes a new $PSStyle automatic variable that can be used for two different purposes.

The first is to make it easier to author text content that contains ANSI escape codes which control
text decorations like color, bold, italics, etc…

This example simply dumps the contents of $PSStyle and shows you the members you can use and their effect on text as well as the actual ANSI escape sequence.
Note that the custom formatting for this variable includes nested types like Formatting, Foreground, and Background.

$PSStyle variable

You can use multiple ANSI escape sequences together.
In this example, I’ve set warning messages to have bold and italicized yellow text on a magenta background:

Warning message style customization

There are also FromRgb() methods available to make use of full 24-bit color if your terminal supports it:

24-bit color text

C# module authors can also leverage $PSStyle by using the PSStyle singleton class in the System.Management.Automation namespace:

string text = $"{PSStyle.Instance.Reverse}{PSStyle.Instance.Foreground.Green}PowerShell{PSStyle.Instance.Foreground.Yellow} Rocks!{PSStyle.Instance.Reset}";

You can control how PowerShell outputs strings that contain ANSI escape sequences by setting $PSStyle.OutputRendering:

  • Automatic
    This is the default and currently will output the text as-is whether it is to the host or through the pipeline if the
    terminal supports ANSI escape sequences (otherwise the output will be plaintext). This is similar behavior to what you
    would get on Linux.
  • Ansi
    This value will output the text as-is whether it is to the host or through the pipeline.
  • PlainText
    This value will remove ANSI escape sequences from any text output whether it is to the host or through the pipeline.
  • Host
    This value will output the text as-is if sent to the host if ANSI escape sequences are supported, but will output plaintext
    if the output is sent through the pipeline or redirected. This is similar behavior to what you would get on macOS.

As this is an experimental feature, we encourage feedback on this before we make a decision to take it out of experimental.
See the original issue for additional details, but open new issues if you have any problems or
suggestions on how to improve this feature.

We very much appreciate on going feedback on our preview releases so we can make adjustments before the release is finalized.
Please participate in on going discussions or create new issues in our repo.

Thanks again to the PowerShell community and all the amazing contributors!

Steve Lee
Pricipal Software Engineer Manager
PowerShell Team

The post PowerShell 7.2 Preview 2 release appeared first on PowerShell.

Incident Report – PowerShell Gallery Downtime October 30, 2020

This post was originally published on this site

 

The PowerShell gallery experienced downtime on October 30th 2020. This report will give context as to what caused the downtime, what actions were taken to mitigate the issue, and what steps we are taking to improve the PowerShell gallery experience moving forward.

Downtime Impact

The downtime was declared at 2020-10-30 03:29 PDT, and was mitigated about 12 hours later at 2020-10-30 15:39 PDT. During this time packages were not available from the gallery, and the web interface was not accessible.

Root Cause of the Downtime

The downtime was a result of an attempt to fix ongoing statistics errors with the gallery. For roughly 3 weeks the PowerShell gallery was experiencing many server errors (roughly 100-200 per minute) due to a key that had reached a max int value (total downloads reached over 2 billion) and was causing persistent int overflow errors on the gallery. This prevented new entries from being added to the ‘PackageStatistics’ table (required for the intermediary processing of statistics). The int overflow first occurred on 9/18/2020.

After an attempt to perform database migrations failed due to the persistent errors manual updates were made to the database to fix inflated package statistics numbers.
These changes triggered a series of deadlock and timeout errors which consumed all our available cloud resources.
This caused a spike in DTU/CPU utilization for the database which inversely correlated with the availability for the service. The availability for the gallery was so low that it was non-functional and declared down.

Mitigating the Downtime

The first mitigation step was to restore the gallery database (DB) to a previous timestamp. It was believed that an error in the attempted fix of gallery statistics caused the DB to get into a bad state and thus restoring the DB reverted those changes. This initial error was likely due to a trigger on the database that we did not account for. Unfortunately, reverting the DB caused additional issues. Checking the PowerShell gallery backend logs, we saw that the service had trouble connecting to the DB with an error that user credentials were wrong. This indicated that the user had been orphaned by the restore so we re-created the user in the DB. After this step, checking the PowerShell gallery backend logs again, the service had additional trouble connecting to the DB with an error that login was failing. We determined that this error was caused by the DB restore dropping the DB from the gallery’s failover group. The next mitigation step was to re-add the DB to the gallery’s failover group. The final mitigation step was to restart the cloud services so they could re-connect to the failover group. At this point the gallery started working again. We validated these fixes with customers, as well as with our own testing and continued to closely monitor the DTU/CPU utilization and service availability.

Statistics Errors

The gallery has had ongoing issues with the package statistics since August 2020.

These errors came from the gallery reaching a scale (more than 2 billion installations) that was not supported by the design of the statistics pipeline. The impact of this has been both incorrect and unavailable package statistics. The package statistics from 2020-09-18 through 2020-10-07 were never recorded, which meant we were unable to recover statistics from this period.

Restoring Statistics

We restored statistics in two ways, first we repaired statistics for individual packages (surfaced on a package’s page), and then we repaired aggregated statistics (surfaced on the gallery homepage and statistics page).

In ordered to repair package statistics we updated values in our main database and within the code base itself, that referenced a key for package statistics from an integer to a bigint/long. There was some pending data that was dropped when the int overflow error first appeared. We retrieved specific ‘lost’ data from a restored database, but were unfortunately unable to recover some data (mentioned in Statistics Errors).

To repair the aggregated statistics, we then made parallel changes to our data warehouse.

Our repair items are focused on 3 categories: detect, diagnose, and fix. By focusing on these three areas, we hope to not only improve the overall performance of the gallery but also, more quickly find and mitigate issues as they arise.

  • Detect:
    • Add more notifications to the production database
    • Create alerts for when critical metrics are reached in the DB
    • Improve post-deployment validation so that we can quickly roll back undesirable changes
  • Diagnose:
    • Send database logs to a central location outside of the service so that logs are more easily available
  • Fix:
    • Improve the deployment process for gallery cloud services
    • Better document (internal) procedures for recovery and communication during an outage

 

We are also in the process of designing architectural changes to the PowerShell gallery, to ensure this is a reliable, performant, and supportable service moving forward.

Expectations going forward

In conjunction with these repairs, we are working to set and monitor Service Level Objectives (SLOs). Look forward to a future post detailing these expectations and how gallery users can track our progress against these objectives.

Reporting Issues

If you notice any issues with the PowerShell gallery please open an issue in our GitHub repository.

If you are a package owner and have an issue with your package please use our support alias: cgadmin@microsoft.com.

We continue to update the status of the PowerShell gallery at: aka.ms/PSGalleryStatus.

Sydney

PowerShell Team

 

The post Incident Report – PowerShell Gallery Downtime October 30, 2020 appeared first on PowerShell.