It’s probably still HKCU related, because you don’t get a proper HKCU in a remote session. It’s likely reg.exe is just hardcoded to do something that isn’t going to work in that context.
The problem that it throws an error at all, so when I build docker images it breaks the flow, how do I structure powershell call that no stderror is returned?
Huh. You could try redirecting the error pipeline into the success pipeline, and then piping the command to Out-Null. That should keep anything out of stderr.