It’s been a while since AWS released support for running Powershell in Lambda Functions. Up until now all of the Lambda functions I’ve worked with have been either python or NodeJS, but we recently had a project that needed to update a database from inside of a deployment pipeline.
We’re trying to retire an old admin box where users would run Powershell management scripts to do things like provision servers, configure customer accounts, etc. Rather than trust a user to remember which script to run, pick the right parameters, check the output, etc, etc, we wanted all of that to happen as part of a CICD pipeline.
I know what you’re thinking: “Why would you drag along the powershell scripts? It’s probably a zillion line single script that’s impossible to debug or manage.” But the codebase is pretty well designed — it’s modular, implements defensive programming, has tons of business logic in it, and most of it has >50% unit test coverage with Pester. So there’s a lot of good stuff there that would take time to rebuild in another language.
So what’s our plan?
Our goals are to have a Lambda function that runs some .NET Core powershell code. The code needs have unit tests that can be run in a CI environment, and be deployed from a pipeline.
The .NET core requirement is interesting. As a windows user I’m still on Powershell 5.1 on my laptop and I use AWSPowershellTools (the old monolithic Powershell module). That’s not to say I couldn’t make the jump, but I’d rather not risk breaking my other tooling for this one project. So we’ll setup a docker environment to do our local development. Let’s get started!
It’s probably obvious, but I’ll be assuming you’re working on a windows machine.
Setting up a Docker image for the build
The first thing we need is a docker image we can use to run our local builds and deploys in.
My first instinct was to use an image from the Powershell repo, but I had a terrible time getting the right version of .NET Core installed. I was getting errors running the Powershell commandlets for building the lambda functions, .NET core install errors, and the list went on.
After struggling for a while I flipped to the mcr.microsoft.com/dotnet/core/sdk run time, and it worked much better. After reading several of the tutorials I decided to use the 2.1-bionic tag because
- Lambda functions are going to run in a linux environment
- It’s smaller than running a windows container
- it worked! 🙂
The first step is to get into the container and see what tools we’ll need. So I ran
mcr.microsoft.com/dotnet/core/sdk:3.1-bionic
And of course because this is a linux container our interpreter is bash off the bad. Installing powershell is pretty easy
dotnet tool install --global PowerShell
That gives us the pwsh program we can use to get into Powershell on the linux container. To see this running try
export PATH="$PATH:/root/.dotnet/tools"
pwsh
gci # prints your current directory
exit # returns you to bash
Next we need to install the zip package.
apt-get update \ && apt-get install zip -y
This will take several minutes while your container downloads You can run these as separate commands too if you’d like. When you see the AWS CLI print it’s version you know you’re setup!
Let’s we need to install the AWSLambdaPSCore into the Powershell environment. To do that you’ll run
pwsh "Install-Module AWSLambdaPSCore -Confirm:\$false -force; Import-Module AWSLambdaPSCore;"
You can break that apart too if you want to. As long as you don’t see an error importing the Powershell Module you should be good to go! Let’s put all of that work into a docker file
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-bionic
RUN dotnet tool install --global PowerShell \
&& apt-get update \
&& apt-get install zip -y
# python3 python3-pip -y \
# && pip3 install awscli --upgrade
RUN export PATH="$PATH:/root/.dotnet/tools" \
&& export PATH=~/.local/bin:$PATH \
&& pwsh -command "Install-Module AWSLambdaPSCore -Confirm:\$false -force; Import-Module AWSLambdaPSCore;"
ENTRYPOINT [ "pwsh" ]
ENV PATH="~/.local/bin:/root/.dotnet/tools:${PATH}"
You can build this by running this command inside of the directory with your Dockerfile
docker build -t lambda-dotnet .
Project Structure
Now that we have a working Dockerfile we need to create our project structure it will look like
Update-MetaDatawill hold our powershell handler functionCommonModuleswill hold our testable powershell code and unit testsscriptswill hold deployment toolsdocker-compose.ymlwe’ll dig into later
Creating the Module and Unit test
Let’s create our common module first. You’ll need to create a new Powershell module manifest with New-ModuleManifest.
New-ModuleManifest CommonModules\Common.psd1 -RootModule Common.psm1 -FunctionsToExport "*" -moduleversion "0.0.1" -Description "Testable module"
This tells powershell to create a new module definition, points to the Common.psm1 powershell module, and exports all functions. You can get a lot fancier with your psd1 files, but those are the basics to get started.
Next we’ll create a simple function in our common module
# Function to insert some data
function Set-AppData {
write-host "working on inserting data";
return $true;
}
Export-ModuleMEmber -function 'Set-AppData';
And add a single unit test for it in a tests\Common.tests.ps1 file
$module = "Common.psm1";
if(test-path ".\$module") {
Import-Module ".\$module";
} elseif(test-path "..\$module") {
import-module "..\$module";
}
# Test all of the citrix common functions
Describe "Common Functions tests" {
InModuleScope Common {
It "returns true" {
Set-AppData | Should be $true;
}
}
}
# Cleanup the module so we can test new changes
remove-module Common;
The boiler plate code the top lets you execute your tests from either the CommonModules directory or the CommonModules\tests directory. We should be able to invoke this and see our tests pass
Powershell Handler
Our Powershell handler is going to be hard to be a little hard to unit test. It seems like it has to be a ps1 file, and AWS Lambda is going to run the script from top to bottom. That makes it a little hard to inject unit tests if we have a lot of logic in that handler script.
That’s why we’re keeping our handler thin (and using the common modules we created above. Let’s look at a slim implementation of the Lambda Handler
# PowerShell script file to be executed as a AWS Lambda function.
# When executing in Lambda the following variables will be predefined.
# $LambdaInput - A PSObject that contains the Lambda function input data.
# $LambdaContext - An Amazon.Lambda.Core.ILambdaContext object that contains information about the currently running Lambda environment.
# The last item in the PowerShell pipeline will be returned as the result of the Lambda function.
# To include PowerShell modules with your Lambda function, like the AWSPowerShell.NetCore module, add a "#Requires" statemen
# indicating the module and version.
#Requires -Modules @{ModuleName='AWSPowerShell.NetCore';ModuleVersion='3.3.335.0'},@{ModuleName='Common';ModuleVersion='0.0.1'}
write-host "Inserting new app data"
Import-Module Common;
# Use function from CommonModules to insert new CMT app workers
Set-AppData;
The end of this script looks like pretty standard powershell, we import a module and call a function.
The top of the script is comments which get generated when you call New-AWSPowerShelLambda .
The middle of this script is interesting, though.
#Requires -Modules @{ModuleName='AWSPowerShell.NetCore';ModuleVersion='3.3.335.0'},@{ModuleName='Common';ModuleVersion='0.0.1'}
write-host "Inserting new app data"
Import-Module Common;
This line tells the AWS tools how to pull together the modules that your function requires. And here’s where we hit an interesting part of our build: we need a place for the lambda tools to find both your modules, and the AWSPowerShell.NetCore module.
Walking through the build
This script will run inside of the container. We’ll start by creating a new temp directory, and registering it as a local powershell repo.
new-item -itemtype Directory -Path /tmp/ -Name "localpsrepo";
Register-PSRepository -Name LocalPSRepo `
-SourceLocation '/tmp/localpsrepo' `
-ScriptSourceLocation '/tmp/localpsrepo' `
-InstallationPolicy Trusted;
Now we’ll publish our common modules to it, and save the AWSPowerShell.NetCore module to it as well.
publish-module -Name ‘/lambda/CommonModules/Common.psd1’ `
-Repository LocalPSRepo `
-NuGetApiKey ‘AnyStringWillDo’;
save-package -name “AWSPowerShell.NetCore” `
-Provider NuGet `
-source https://www.powershellgallery.com/api/v2 `
-RequiredVersion 3.3.335.0 `
-Path /tmp/localpsrepo;
That gives our call to Publish-AWSPowerShellLambda a local repo to pull from.
Publish-AWSPowerShellLambda -scriptpath /lambda/Update-MetaData/Set-AppData.ps1 `
-StagingDirectory /tmp/lambda `
-ProfileName sandbox `
-region us-east-1 `
-ModuleRepository LocalPSRepo `
-Name Set-AppData`
-IamRoleArn arn:aws:iam::*********:role/lambda_basic_execution
That’s the build, lastly let’s pull it all together with a docker-compose file.
Docker Compose File
Docker compose is a tool for building and running multiple services at a time in a docker development environment. Among other things it lets you call out volumes to map into your container, which is our main use.
Our goal is to map the powershell script, modules, and our AWS credentials into the container so we can run the build using the .NET Core container.
Your docker-compose file will look like this
version: '3.7'
services:
lambda-dotnet:
build: "."
volumes:
- .:/lambda
- "${USER_HOME}/.aws:/root/.aws"
command: /lambda/scripts/Deploy-Lambda.ps1
stdin_open: true
tty: true
NOTE: There are some tricks to sharing files between a windows host and a linux container. There are some tips in my post here.
Lastly you’ll need to create a .env file (reference here) that looks like this
USER_HOME=c:\users\bolson\
And that’s it! You can build and deploy your lambda function with
docker-compose run lambda-dotnet
Rolling Websphere 