Microsoft’s PowerShell Core Offers Cross-Platform Automation – InApps 2022

The Microsoft PowerShell task automation and configuration management framework has long been associated with core Microsoft workloads; it provides access to everything from Azure Stack to Windows 10. But since mid-2016, it’s also been a cross-platform open source shell and scripting tool, one uniquely suited to the emerging world of APIs and structured objects.

Increasingly, Microsoft’s customers don’t just use Windows and Windows Server; they use Linux and Macs (plus Raspberry Pi at one end of the scale and multiple cloud services at the other). Supporting this diversity, PowerShell Core runs on Windows 7, 8.1, and 10, Windows Server 2008 R2, 2012 R2, 2016, Windows Server Semi-Annual Channel, Ubuntu 14.04, 16.04, and 17.04, Debian 8.7+, and 9, CentOS 7, Red Hat Enterprise Linux 7, OpenSUSE 42.2, Fedora 25, 26 and macOS 10.12+, with unsupported packages for AppImage, Arch and Kali Linux, and experimental versions for Raspbian and Windows on ARM.

“More and more of the Linux world is becoming API based and returning structured objects, JSON objects which we convert to text and can do all sorts of wonderful things on without any parsing.” — Jeffrey Snover

The Linux version of PowerShell is no longer a separate code base; along with macOS and what looks like an increasing number of platforms including IoT, it’s part of the main PowerShell project, with common features and functionality. “The company is becoming a cross-OS company. I like to say that ‘the sea refuses no river’; we want to be the company for everybody, no matter what platform you’re using,” said Microsoft Fellow Jeffrey Snover.

But the general availability of PowerShell Core 6 for Windows, macOS and Linux doesn’t just mean Microsoft’s solution for managing the heterogeneous mix of systems now common in business is ready to use in production. It also brings some major changes to PowerShell on Windows, because the cross-platform, open source, community-driven, “get-it-from-GitHub” Core version of PowerShell is where all new development will happen.

Truly Cross-Platform

What we’ve been calling PowerShell for the last decade is now named Windows PowerShell, because it’s built on the .NET Framework that’s only available on Windows. PowerShell Core builds on Microsoft’s (also open source and cross-platform) .NET strategy, using the .NET Core 2.0 runtime and the .NET Standard 2.0 common libraries and APIs.

“What we’re doing with PowerShell Core is we’re hitting the refresh button,” Snover told us. “We’re keeping all the things that were great and we have the courage to leave behind the things that don’t bring us to where we want to be. And where we want to be is a tool that allows companies to manage anything from anywhere.”

That means little things like PowerShell Core now handling common Linux filename characters like colons and commas that Windows doesn’t support, as well as paths with forward or backslash separators. The default encoding is UTF-8 without a Byte Order Mark; the default on Linux and macOS and increasingly common on Windows.

Logging support was added for GA, but it uses a different set of group policy objects than the Windows PowerShell settings (which will simplify using both side by side, but is another change to make once the .ADMX list is published). On macOS, PowerShell uses the native os_log APIs to log to Apple’s unified logging system; on Linux, it uses Syslog. For remoting, PowerShell Core uses SSH as a native transport protocol. Register PowerShell with an OpenSSH-based SSH server and existing SSH-based authenticate mechanisms like passwords or private keys work with the traditional PSSession semantics.

PowerShell Core isn’t attempting to replace Bash, Python or any existing tools; what it’s doing is bringing the automation and handling of structured data to Linux, at a time when that is becoming increasingly important.

“More and more of the Linux world is becoming API based and returning structured objects, JSON objects which we convert to text and can do all sorts of wonderful things on without any parsing,” Snover pointed out, claiming that “Linux is moving to a place where PowerShell’s sweet spot is. On Linux, we’re a tool, we’re another shell just like any other shell and there’s a gazillion shells out there. If your world is just text parsing we do that fine but we’ve no big advantage over some of the others. But if you have structured data, there’s where we pull ahead by quite a huge distance.”

PowerShell also offers Linux some advantages in security. “We now have the world’s first security transparent shell running on Linux. We took as much of the security stuff [from Windows PowerShell] as we could run on Linux, so you can configure things to have full transparent script block logging and so on, and Linux has never had that before,” Snover said.

PowerShell program manager Joey Aiello confirmed that structured data parsing is popular with Linux PowerShell users, as is cloud orchestration.

“In general, I think people are using a lot of cloud orchestration modules; we have a Core version of AzureRM and VMware and AWS have published some modules that people are using. But I also see very often PowerShell being used as part of test orchestration and general REST API automation. We have PSSwagger that makes it easier to generate PowerShell module from open API documents that describe REST APIs and that’s made it very easy to script against REST APIs, which we see as the next level of APIs to wrangle. We’ve moved up a layer in the stack; PowerShell has really done well as a funnel a vacuum for all these disparate APIs, and we think it has a lot of opportunity to do that again in in the cloud and REST API world.”

Side by Side

For Windows PowerShell users, the implications of PowerShell Core are slightly different, because they already have a tool they’re used to and have invested effort in learning, as well as this new tool which has slightly different features that isn’t installed automatically. In fact, it’s only in the last two months that Windows use of PowerShell Core has become significant; “before that the vast bulk of usage was Linux,” Snover told us (to his surprise).

PowerShell 5.1 can run side by side with PowerShell Core, which has its own pwsh.exe executable and comes as a ZIP package so admins can install multiple versions, and scripts can be pinned to specific versions that they require. Windows PowerShell it will continue to be shipped and supported as part of Windows 10 and Windows Server, and it will get bug fixes. But it won’t get new features, like the enhancements in the JSON and Web cmdlets to simplify writing scripts against web services, or the new Docker support that lets you use docker pull microsoft/powershell to get PowerShell Core.

Read More:   An Interview with Dirk Hohndel, VMware’s New Chief Open Source Officer – InApps Technology 2022

“Windows PowerShell 5.1 is a very strong, very capable tool that continues to be supported and will be supported, my guess is forever, but it not going to be enhanced,” Snover told us. “I’m confident that anybody who needs to do something in a Windows-only way will be perfectly fine with Windows PowerShell. But we’re going to put all our investment in the new cross-platform tool.” In other words, Windows PowerShell isn’t going away, in much the same way that CMD.EXE has stayed in Windows for those who find it useful.

That’s still caused some concern among Windows PowerShell users, especially about compatibility for modules and tools, as well as differences in functionality. The goal is to be as compatible as possible. “In order to be cross-platform. there were quite a few little changes that we had to make to better fit into the UNIX world and the reality is those things were breaking changes, so there is a difference between Core PowerShell and Windows PowerShell,” Snover noted. “In reality, the vast majority of peoples’ things are just going to work when the existing cmdlets work on .NET core but there are some changes.”

If the .NET features that a PowerShell module depends on are in .NET Standard 2.0, that module will work with PowerShell Core, he explained. “The WindowsPSModulePath module changes your module path and brings in the existing modules and the reality is that most of those work just fine; some of them don’t but most of them do. More importantly, though, the team [that built that module] hasn’t signed up to support them, they haven’t done full testing.”

Most of the modules that ship as part of Windows (like Hyper-V, NetTCPIP and Storage), or with other Microsoft products like including Azure and Office have not been explicitly ported to PowerShell Core yet and aren’t supported but many seem to work. Many of the built-in modules that come with PowerShell itself have bug fixes and new capabilities — but sometimes they also lose functionality if .NET Core doesn’t support it.

Snap-ins — the older, rarely used version of modules — aren’t supported at all, which means the ActiveDirectory and DnsClient modules on Windows don’t work yet (the PowerShell and Windows Server teams are working on the AD module already). PowerShell Workflow is built on Windows Workflow Foundation, which needs .NET Framework, so that stays Windows only; “we asked the community and we didn’t hear a resounding ‘this is critical’,” Snover told us. The most popular workflow use is parallelism so Microsoft is planning to support concurrency for PowerShell Core, although it won’t come until PowerShell Core 6.1.

There’s a Windows PowerShell Compatibility Pack planned to bring back some Windows PowerShell features missing in PowerShell Core like aliases, WMI and perf cmdlest, as well as automatically setting the module path. WMI v1 cmdlets like Get-WmiObject don’t work anymore; they’ve all been replaced by WMI v2 cmdlets that often add extra features, but that’s still a change to make. Third parties will also need to update their modules — if those vendors are still in business.

There’s only a short list of compatible modules on the PowerShell Gallery so far (of the 30 three are deprecated and only a third aren’t from Microsoft) but when the PowerShell Standard 5.1 and Core 6.0 libraries come out later this month it will be easier for developers to create modules that are compatible with both.

Although Snover couldn’t comment on when various Microsoft teams would formally support PowerShell Core for Microsoft workloads, he noted that “Not only will Windows people be able to manage your stuff, so will Linux people; when a cross-platform value proposition is important, people [at Microsoft] are totally in. Azure is totally in and is going to be there pretty quickly.”

The Azure PowerShell team has several pre-release modules for using Azure Resource Manager with PowerShell Core in the PowerShell Galley, and the semantics for PowerShell cmdlets for Azure is being simplified. PowerShell Core support is “on the roadmap” for Azure Automation, but don’t expect to see it until a critical mass of cmdlets has been tested or updated.

In the long run, the change brings big advantages of course. “The big challenge right now is that cmdlets need to be supported in both frameworks; if you link against the .NET Standard library then you work both with full .NET and .NET Core. That means is cmdlets work with Windows PowerShell and PowerShell Core on Windows, and If you take that code onto Linux they run there as well, so we need to get the full community to support .NET Standard.”

Microsoft’s Desired State Configuration is getting the same deep refresh as PowerShell, although it’s not as far along. “Desired State Configuration has always been cross-platform from the beginning, it’s been available on Windows and Linux, on the agent side, but in reality, we have one code base Linux and another for Windows and we’re going to unify that codebase,” Snover said.

“We are rolling out Desired State Configuration widely through our Azure estate. If you look at the Azure portal, you’ll see we have configuration management there and we’re going to make that ubiquitous throughout Azure. And as we do that we’re picking up a new set of requirements and we’ll be coming out with a new design. It will be a hierarchal Desired State Configuration tool; in other words, there will be a well-defined interface that will be able to talk first to a native code agent that will be like the bootstrap agent and so a bunch of things. From there, it will be able to bootstrap into a .NET Core-based agent and from there a full managed code agent.”

The Just Enough Admin (JEA) PowerShell security controls work on Windows in PowerShell Core and will also be coming to Linux, Snover confirmed. “There’s a number of things we need to add to the Linux side but that’s our intent. In fact, wherever Linux allows something to be done, we’ll try and do it with PowerShell Core.”

PowerShell is shaping up to be another Microsoft open source success, with half of the changes in the new release coming from the PowerShell community. That’s where to look for the direction of PowerShell he told us. “We are now a community-based project and what that means it is completely open. We have a set of projects and priorities in our GitHub repo; there’s no second set of books, those are our books and the community gets to say ‘we think you’re wrong on this’. It’s a full open kimono. The community gets to take this wherever they need to take it.”

Microsoft is a sponsor of InApps.

Feature image via Pixabay.

InApps is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.