Invoke-Command Performance with non-Admin credentials

I am attempting to run a set of commands on a remote computer using an account that is not a local admin on the target server
(target is 2012R2)
I have updated the default configuration using Set-PSSessionConfiguration, and in fact the test runs as expected, except for performance.

Using a local admin credential the functions run in about 2 seconds. When I execute the same test using a non-admin account it takes 11-12 seconds.

Is this performance hit expected or is there some tuning I can do to improve the response time ? Googling did not turn up anything helpful other than one person seeing the same issue back with PS V2 (no answers). Any and all thoughts appreciated.

I would think that you would need to give a specific example to get a better answer but to start with you would need to look into where the delay is occurring. If the delay is during the connection\establishing of a session it may be a difference in validating a local group vs a domain group. If the delay is during the execution of your function you would need to debug it to or add some measure commands to determine where the performance difference is. My thoughts would be to review your configuration, review event logs, and add debug logic to your code to determine where the issue is and go from there.