Things n' Stuff Thoughts about the universe in general

Cloud-init is a Linux technology that allows easy setup and automation of virtual machines. The concept is very simple – the VM infrastructure provides some way of setting some custom data for each virtual machine (many providers call this “user data”), and when the operating system starts the cloud-init service reads that configuration, loads a bunch of modules to handle various parts and let them configure the system. As a user it is very convenient – you write a setup scenario using the variety of tools offered by cloud-init, you can store the scenario in a source control to allow to develop the scenario further, then just launch a bunch of machines with the specified scenario and watch them configure themselves.

The situation is much worse on the MS-Windows side of the fence: want to have an MS-Windows server configured and ready to go? Start a virtual machine, connect to is using RDP and Next, Next, Finish until your fingers are sore. Need to deploy a new version? either retrofit an existing image (again, manually) and risk deployment side effects, or do the whole process again from scratch.

Here’s a script to try to help a bit with the problem – at least on Amazon Web Services: a poor man’s cloud-init-like for MS-Windows server automation.

The reason this thing can work on AWS’s EC2 service, is that Amazon’s base MS-Windows images come included with a per-installed service called EC2Config. That service allows you to write batch and PowerShell scripts in the user data and it will run these. Apparently, anything written outside the “script tags” that it uses to delimit the script content it just being ignored. This allows us to write a script that can be embedded in a cloud-init YAML configuration file and use that to bootstrap the process.

In the following examples I’m going to use the PowerShell scripting language to automate the setup. It’s true that I hate PowerShell, it is still many orders of magnitude better than the other scripting language available on MS-Windows, and there is a large community that writes modules for it – not to mention that Microsoft itself offers many PowerShell features to manipulate MS-Windows facilities, in a way that almost¹ competes with the capabilities on Linux.

The basic setup is very simple – we use XML-like <powershell> and </powershell> “tags” to delimit the script. Because EC2Config is blind to anything other than that text between the “tags”, we can write something that is runnable by EC2 but also parses as YAML. A “Hello World” might look like this:

#cloud-config
script: |
 <powershell>
 echo "Hello World" | Set-Content -Path c:\hello.txt
 </powershell>

(no point to echo to the console if no one is going to see it)

Once we have a YAML configuration file that runs a PowerShell script, we can bootstrap by installing a few modules from the internet (using the PowerShell’s equivalent of curl|sh) and re-reading our user data into PowerShell:

#cloud-config
script: |
 <powershell>
 Try {
 (new-object Net.WebClient).DownloadString("http://psget.net/GetPsGet.ps1") | iex
 import-module PsGet
 Install-Module PowerYaml
 $userdata = (Get-Yaml (Invoke-RestMethod http://instance-data/latest/user-data))
 }
 Catch
 {
   Set-Content C:/setup-error.txt -Value $_.Exception.Message
   throw
 }  
 } *> c:/setup.log
 </powershell>

What happened here:

1. PSGet is a Ruby gems / Node Package Manager / NUGet -like tool for PowerShell – once it is loaded you can easily load all kinds of PowerShell script and binary modules available from the PSGet repository. So we first download and run their installation script
2. After installing the PSGet module, we need to load it into the current shell (I’m not sure if PSGet sets up auto-loading for future shells, but to get it into the running shell that just installed it, you need to load it explicitly
3. We then install a YAML parsing module to allow us to read the cloud-init configuration data.
4. Finally we use PowerShell built-in curl-like feature to call Amazon’s data service to get our user data and parse it using the PowerYAML module
5. Do note that the whole bootstrap script is container in a Powershell {} block – this is similar to “sub-shells” in UNIX shells and allows me to capture the output from the entire shell using a “catch-all” redirect (*>, which is similar to &> in bash) and log it to a local file. I also use a try...catch to handle exception and log the error specifically. One can then look at the server’s file system to see if the setup failed.

Now that we have the user data parsed, we can start writing cloud-init modules and run them. My current setup only supports the write_files and runcmd modules which allow you to create files on the file system, and run some commands – probably run the scripts you pushed. This looks something like this:

#cloud-config
some_meta_data:
 - that I like to add
 - and my scripts will read
 
write_files:
 - path: C:/Users/Administrator/init.ps1
   content: |
     echo "put your server initialization stuff here"

runcmd:
 - echo "Hello World"
 - & C:/Users/Administrator/init.ps1

script: |
 <powershell>
 Powershell {
 Try {
 # setup YAML support
 Set-ExecutionPolicy RemoteSigned -Force
 (new-object Net.WebClient).DownloadString("http://psget.net/GetPsGet.ps1") | iex
 import-module PsGet
 Install-Module PowerYaml
 # load user data
 $userdata = Get-Yaml (Invoke-RestMethod http://instance-data/latest/user-data)
 # imitate write_files module
 $userdata.write_files | %{
   Set-Content -Path $_.path -Value $_.content
 }
 # imitate runcmd module
 $userdata.runcmd | %{
   $_ | iex
 }
 }
 Catch
 {
   Set-Content C:/setup-error.txt -Value $_.Exception.Message
   throw
 }  
 } *> c:/setup.log
 </powershell>

And that was that! Makes MS-Windows almost usable!

As a last note, there is “real” (or less hacky, if you will) cloud-init implementation for MS-Windows from cloud computing company CloudBase that offers a lot more features and also supports OpenStack (so works not only on EC2 and compatible systems). The main down side of it, for me, is that it isn’t available on the base MS-Windows images offered by Amazon, which means I’d have to bootstrap it on there manually – or with the above little hack. Maybe I’ll look into that in the future.

1. well, not really – not even close but that is not only the fault of Microsoft: software written for MS-Windows more often than not assumes that there is an administrator sitting in front on the console [↩]

This entry was posted on Thursday, August 20th, 2015 at 20:41 and is filed under Articles, Programming, Script Day, Software, Tech. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.