Powershell - cannot delete registry keys

Hi there
I am trying to clean up some registry keys either locally or remotely with the following.
Remove-Item -Path HKLM:\Software\microsoft\XYZ

It does not work for me, I have tried on several machines. I also tried -Recurse and -Force.
But no change.
If I manually navigate through the registry I can delete the key. Also works if I use reg delete In a batch file.
Any ideas. I am new to powershell.

PS4
Thanks

Did you run the ISE as administrator?