Avoiding Partial Configurations - ideas?

DSC
1

As mentioned in DSC book, partial configurations should be avoided for several reasons. I tried to avoid them when setting up our DSC enviroment however I have a nagging feeling that it may not be scalable enough.

Our DSC Enviroment uses PullServers to distribute the mofs. This is how it is set up at the moment:

  • LCM Mofs are generated when new hosts are added to configurationData. For ConfigurationNames parameter, hostname is used as below:
        ConfigurationRepositoryWeb PullServer {
                ServerURL                  = $configurationData.services.DSCPullServer.DSCServerURL
                RegistrationKey            = $configurationData.services.DSCPullServer.DSCRegistrationKey
                AllowUnsecureConnection    = $false 
                ConfigurationNames         = @("$($node.NodeName)")
            }
  • There is one master Configuration file. This file uses include files for other Configurations as below:
configuration mainConfiguration {
    param(
        [Parameter(Mandatory = $false)]
        [ValidateNotNullOrEmpty()]
        [PSCredential]$credential,
    )

    Node ($AllNodes | Where-Object {$_.memberOfServices -contains "WebServer"}).NodeName
    {
      . $(Join-path $ConfigurationPath WebServer.ps1)
    }  

    Node ($AllNodes | Where-Object {$_.memberOfServices -contains "SQL"}).NodeName
    {
      . $(Join-path $ConfigurationPath SQL.ps1)
    }  

}

When this configuration is running, it generates one MOF file per host.
Having one configuration file and one configurationName per host is the only way I could avoid using partial configurations and still have scalable solution. However now I have to rerun entire configuration even if I want to apply a small change.
I would like to find out what other people have done to avoid the partial configuration but have a more flexible and modular approach?

2

What you’re running into is not a scalability problem with your approach per se, it’s a problem with how you, as the person making it all happen, can or cannot scale. This is a tooling problem, meaning you’ve found a solution that works, but you’re still implementing it manually, so it’s a PITA.

Rerunning the entire thing - even for a minor change - shouldn’t be a problem if you’ve got the right tools (scripts, maybe) to automate it for you. That’s the next place to invest - making some tooling to streamline the process.

Alternately, look into Tug, the open-source DSC pull server. This isn’t zero effort, but it would enable you to build yourself a smarter pull server that - as one example of what you COULD code up - dynamically create MOFs on the fly, look up configuration artifacts in a database (combining them on the server, on-demand - so, basically “partials, only better”), etc.

3

Thank you Don

We have a release pipeline in place and entire DSC process is pretty much automated already.When someone checks in the code, the build agents build the configuration, run tests, generate mofs files and distribute them to load balanced pull servers.
Interesting point about Tug - I will definetly investigate it.

4

Isn’t the solution just add parameter to your script, and manipulating the “AllHosts”. If you don’t want to change all hosts, pass only the hosts you want to change, but if your change… I might be missing something here.

5

Thank you Geraldo

You are absolutely right. This is the most obvious way to workaround it and you pointed me in a right direction:)

We have a ConfigurationData data stored in files as below:

├───AllNodes
│       client1.psd1
│       client2.psd1
│
├───Certificates
│   ├───Client
│   │       client1.domain.local.cer
│   │       client2.domain.local.cer
│   │
│   └───Server
│       ├───pullserver1
│       │       dsc.domain.local.cer│       │
│       └───pullserver2
│               dsc.domain.local.cer
│
├───Credentials
│       elevateduser.psd1
│ 
├───Services
│       CA.psd1
│       DSCPullServer.psd1
│
└───SiteData
        site1.psd1
        site2.psd1

Release pipeline process will work like this:

  • User edits main Configuration file ( shown in the first comment)
  • User checks out files to source control
  • Automated process builds ConfigurationData datastructure using the information from the folder structure above.
  • Automated process runs Configuration using the Configuration Data datastructure.

Now I need to think how can we can amend this workflow to be able to change content of AllNodes dynamically.