merge Powershell CustomObjects

[wotanshammer]

Hi, I have a question, maybe someone has an idea.

I’m reading two JSON files and would like to merge the information into one PSCustomObject but only the single values should be changed if the information already exists or a new propertiy shoud be added if the property doesent exist.

$Json1 is a global information and the values of $Json2 have the last word if there are equal keys or if additional keys musst be added.

So the information from $Json2 is leading, but should only overwrite or enrich information.

Example of JSONs.

$Json1:

{

“Hostname”: “Server1”,

“Port”: "“5001”,

“Paths”:{

“Path1”: “C:\test\123”,

“Path2”: “C:\test\456”

“Path3”: “C:\test\456”

“Pfad4”: “C:\test\456”

}

}

$Json2:

{

“Hostname”: “Server2”,

“Port”: "“5001”,

“Paths”:{

“Path1”: “E:\Prod\123”,

“Path2”: “E:\Prod\456”

}

}

I read the two files and convert it using ConvertFrom-Json

So the information is now in the variables $Json1 and $Json2

I now merge the two variables into one object.

Code:

$Object = [ordered] @{}
foreach ($Property in $Json1.PSObject.Properties) {
$Object += @{$Property.Name = $Property.Value}
}
foreach ($Property in $JSON2.PSObject.Properties) {
try{
$Object += @{$Property.Name = $Property.Value}
}catch{
$Object.$($Property.Name) = $Property.Value
}
}
}

The result of the newly created object should look as follows:

“Hostname”: “Server2”

“Port”: "“5001”,

“Paths”:@{

“Path1”: “E:\Prod\123”

“Path2”: “E:\Prod\456”

“Path3”: “C:\test\456”

“Path4”: “C:\test\456”}

But what comes out is:

“Hostname”: “Server2”

“Port”: "“5001”,

“Paths”:@{

“Path1”: “E:\Prod\123”

“Path2”: “E:\Prod\456”

}

I dont understand how to modfy only the Path1 and Path2 but keeping Path3 and 4 from the JSON1

The hostname from the JSON2 was taken over and the port remained unchanged as desired.

Only the values of path1 and path2 were changed with the JSON2 information, but path3 and path4 were deleted.

The code above deletes exactly this information path3 and path4. Everything else is replaced correctly.

So I have problems to adjust only the nested values. Everything on the first level works, but as soon as another level is added, Path-> Path1 the information of Path is completely replaced.

This is quite clear in the code, but I do not really have the idea how to solve the problem and only change the information of path1 and 2 but not deleting path3 and Path4 from the $json1

Result should look like this.

The result of the newly created object should look like this:

“Hostname”: “Server2”

“Port”: "“5001”,

“Paths”:@{

“Path1”: “E:\Prod\123”

“Path2”: “E:\Prod\456”

“Path3”: “C:\test\456”

“Path4”: “C:\test\456”}

 

I could access the path information with the name $Object.Paths.Path1, but the script should be generic because the use case is just to merge different Json files into one hash table/PSObject and if the same property is present in both Json files, the desired Json file information (json2) overwrites the values of the other one, regardless of the level depth. (Paths->;Path1…)

 

for ideas I would be thankful

 

best regards

Rolf

[rob-simmers]

Merge isn’t really the right term. Merging is typically taking two objects, say for instance the same HostName, and if you had memory in one object and cpu in another object and you want CPU and Memory associated with that hostname. Enough for semantics, it appears you want to overwrite the paths from the other record making them authoritative. Here is an example:

$Json1 = @"
{
    "Hostname": "Server1",
    "Port": "5001",
    "Paths":{
        "Path1": "C:\\test\\123",
        "Path2": "C:\\test\\456",
        "Path3": "C:\\test\\456",
        "Path4": "C:\\test\\456"
    }
}
"@ | ConvertFrom-Json

$Json2 = @"
{
    "Hostname": "Server2",
    "Port": "5001",
    "Paths":{
        "Path1": "E:\\Prod\\123",
        "Path2": "E:\\Prod\\456"
    }
}
"@ | ConvertFrom-Json

foreach ($path in $Json1.Paths.PSObject.Properties){
     $newVal = $Json2.Paths.PSObject.Properties | Where-Object {$_.Name -eq $path.Name}
     
    if ($newVal) {
        'Setting value {0} to {1} for property {2}' -f $path.Value, $newVal.Value, $path.Name
        $path.Value = $newVal.Value
    }      
}

$Json1

Output:

PS C:\Users\rasim> c:\Users\rasim\Desktop\temp.ps1
Setting value C:\test\123 to E:\Prod\123 for property Path1
Setting value C:\test\456 to E:\Prod\456 for property Path2

Hostname Port Paths
-------- ---- -----
Server1  5001 @{Path1=E:\Prod\123; Path2=E:\Prod\456; Path3=C:\test\456; Path4=C:\test\456}

[wotanshammer]

Hi, thank you for your answere…

I merge two json files into one object. Both files provide information and in case of collisions the second file (Json2) should win and modify the key or value or key/value pair.
The first part of the script builds an object with key value pairs. Because a new object is created, there are no collisions adding new keys.(dubilcate keys not possible)

The second function should trys to add all keys and values from the json2 and replace them with new values from the json2 or add new keys from the json2 if the new key from json2 are not present.

If a key is allready present the catch part will just modify the value of this key ($Object.$($Property.Name) = $Property.Value)
This works well with one level json files. (hostname, port…)

My json files have 2 levels and i haven’t always the information of the json structure…I just know that the json have a maximum deep of 2 levels…

Level1

“Hostname”: “Server2”,
“Port”: “5001”,
“Paths”:{

Level 2
“Path1”: “E:\Prod\123”,
“Path2”: “E:\Prod\456”

so …I have the problem that I have to replace the Keys/values from the second level (Paths -> Path1, Path2, Path3, …) without the names of the properties or keys ($Json2.Paths.PSObject.Properties )

The output of your solution for the Path path is that what i need…The Hostname should also changed

Thanks a lot

Rolf

[wotanshammer]

Ok, is ugly but is a frist try…mybe the right way…

$Object = [ordered] @{}
        foreach ($Property in $Json1.PSObject.Properties) {
            $Object += @{$Property.Name = $Property.Value}
         }

      foreach ($Property in $Json2.PSObject.Properties) {
        try{
            $Object += @{$Property.Name = $Property.Value}
           }catch{
                    $PropName = $($Property.Name)
                    if($Property.TypeNameOfValue -notmatch 'System.String'){
                        foreach ($path in $Object.$PropName.PSObject.Properties){
                        $newVal = $Json2.$PropName.PSObject.Properties | Where-Object {$_.Name -eq $path.Name}
                        if ($newVal) {
                            'Setting value {0} to {1} for property {2}' -f $path.Value, $newVal.Value, $path.Name
                            $path.Value = $newVal.Value
                            }       
                        }
                    }else{$Object.$($Property.Name) =  $Property.Value}
                  }
            }

 

 

[rob-simmers]

This a very unorthodox way of doing things to overwrite everything about an object with another object. This should be a recursive function if you wanted to handle any depth, but this should work:

 
$Json1 = @"
{
    "Hostname": "Server1",
    "Port": "5001",
    "Paths":{
        "Path1": "C:\\test\\123",
        "Path2": "C:\\test\\456",
        "Path3": "C:\\test\\456",
        "Path4": "C:\\test\\456"
    }
}
"@ | ConvertFrom-Json

$Json2 = @"
{
    "Hostname": "Server2",
    "Port": "5001",
    "Paths":{
        "Path1": "E:\\Prod\\123",
        "Path2": "E:\\Prod\\456"
    }
}
"@ | ConvertFrom-Json

foreach ($prop in $Json1.PSObject.Properties){
     
    if ($prop.Value -is [PSCustomObject]) {
        foreach ($subProp in $prop.Value.PSObject.Properties) {
            $newSubVal = $Json2."$($prop.Name)".PSObject.Properties | Where-Object {$_.Name -eq $subProp.Name}

            if ($newSubVal) {
                'Setting value {0} to {1} for sub property {2}' -f $subProp.Value, $newSubVal.Value, $subProp.Name
                $subProp.Value = $newSubVal.Value
            }
        }
    }
    else {
        $newVal = $Json2.PSObject.Properties | Where-Object {$_.Name -eq $prop.Name}

        if ($newVal) {
            'Setting value {0} to {1} for property {2}' -f $prop.Value, $newVal.Value, $prop.Name
            $prop.Value = $newVal.Value
        }   
    }
    
}

$Json1

Output:

PS C:\Users\rasim> c:\Users\rasim\Desktop\temp.ps1        
Setting value Server1 to Server2 for property Hostname
Setting value 5001 to 5001 for property Port
Setting value C:\test\123 to E:\Prod\123 for sub property Path1
Setting value C:\test\456 to E:\Prod\456 for sub property Path2

Hostname Port Paths
-------- ---- -----
Server2  5001 @{Path1=E:\Prod\123; Path2=E:\Prod\456; Path3=C:\test\456; Path4=C:\test\456}

[wotanshammer]

Hi, the script does not replace everything. It replace only things are present in both json files with the information of the json2 or it adds additional key/value pairs from the json2 if not present in the json1.

The use case is that the JSON1 is a kind of “Global” Parameter file and the Json2 a kind of local parameter file. Thats means that all scripts use the global json (lets call it json1) for default values like default paths, defaul server, default databases and so on. The local json (json2) have usually only additional values but for some scripts we must override some parameters deliverd from the json1 because there use maybe different paths or server or databases and so on. We talk about a of a handfull parameters that are overwritten.

In the most cases, the jsons are different and have only additional informations.

 

many thanks for yours support - it was very helpfull

If you have a better idea to handle the use case…let me know

Best regards

Rolf

 

[wotanshammer]

you don’t like my try{} Catch{}

[wotanshammer]

A better example of the jsons…my first description was a bit too general or too simple

$Json1 = @"
{
    "Hostname": "Server1",
    "FileServer": "Server2",
    "Repository": "Server3",
    "DBServer": "Server4",
    "Port": "5001",
    "SQLDriver": "{Some String}",
    "LogPaths": "E:\\SomeFileLocation\\",
    "MuchMoreProperties": "MuchMoreEntries",
    "....": "...", #means additonal properies not present in the json2
    "Paths":{
        "Path1": "C:\\test\\123",
        "Path2": "C:\\test\\456",
        "Path3": "C:\\test\\456",
        "Path4": "C:\\test\\456"
    }
}
"@ | ConvertFrom-Json
 
$Json2 = @"
{
    "Hostname": "Server2",
    "Port": "5002",
    "SQLServer": "Serverx",
    "FileExtension": ".dll",
    "Filename": "SomeName",
    "SubPath": "\\SomePath\\",
    "AndSoOn": "SomeValues",
    "....": "...", #means additonal properies not present in the json1
    "Paths":{
        "Path1": "E:\\Prod\\123",
        "Path2": "E:\\Prod\\456"
    }
}
"@ | ConvertFrom-Jso

 

Best regards

rolf

 

[rob-simmers]

Taking two objects properties and combining them into one object with properties from object 2 values being authoritative over object 1 property values.

$Json1 = @"
{
    "Hostname": "Server1",
    "FileServer": "Server2",
    "Repository": "Server3",
    "DBServer": "Server4",
    "Port": "5001",
    "SQLDriver": "{Some String}",
    "LogPaths": "E:\\SomeFileLocation\\",
    "MuchMoreProperties": "MuchMoreEntries",
    "NotInJson2": "SomeOtherValue",
    "Paths":{
        "Path1": "C:\\test\\123",
        "Path2": "C:\\test\\456",
        "Path3": "C:\\test\\456",
        "Path4": "C:\\test\\456"
    }
}
"@ | ConvertFrom-Json
 
$Json2 = @"
{
    "Hostname": "Server2",
    "Port": "5002",
    "SQLServer": "Serverx",
    "FileExtension": ".dll",
    "Filename": "SomeName",
    "SubPath": "\\SomePath\\",
    "AndSoOn": "SomeValues",
    "NotInJson1": "SomeOtherValue",
    "Paths":{
        "Path1": "E:\\Prod\\123",
        "Path2": "E:\\Prod\\456"
    }
}
"@ | ConvertFrom-Json

#Empty hashtable for new object
$newObjProps = @{}
#Find matching properties if they exist in Json1 and overwrite with Json2 values
#or add properties that only existing in Json1
foreach ($prop in $Json1.PSObject.Properties){
     
    if ($prop.Value -is [PSCustomObject]) {
        $newObjSubProps = @{}
        foreach ($subProp in $prop.Value.PSObject.Properties) {
            $newSubVal = $Json2."$($prop.Name)".PSObject.Properties | Where-Object {$_.Name -eq $subProp.Name}

            if ($newSubVal) {
                'Overwriting value {0} to {1} for sub property {2}' -f $subProp.Value, $newSubVal.Value, $subProp.Name
                $newObjSubProps.Add($subProp.Name, $newSubVal.Value)
            }
            else {
                'Adding value {0} for sub property {1}' -f $subProp.Value, $subProp.Name
                $newObjSubProps.Add($subProp.Name, $newSubVal.Value)
            }
        }

        'Adding property {0}' -f $prop.Name
        $newObjProps.Add($prop.Name, (New-Object -TypeName PSObject -Property $newObjSubProps))
    }
    else {
        $newVal = $Json2.PSObject.Properties | Where-Object {$_.Name -eq $prop.Name}

        if ($newVal) {
            'Overwriting value {0} to {1} for property {2}' -f $prop.Value, $newVal.Value, $prop.Name
            $newObjProps.Add($prop.Name, $newVal.Value)
        }   
        else {
            'Adding value {0} for property {1}' -f $prop.Value, $prop.Name
            $newObjProps.Add($prop.Name, $prop.Value)
        }
    }
}

#Identify properties in Json2 that are not in the hash table for the new object
foreach ($prop in $Json2.PSObject.Properties | Where-Object -FilterScript {$newObjProps.Keys -notcontains $_.Name}){
    $newObjProps.Add($prop.Name, $prop.Value)
}

$myNewObject = New-Object -TypeName PSObject -Property $newObjProps
$myNewObject

Output:

PS C:\Users\rasim> c:\Users\rasim\Desktop\temp.ps1
Overwriting value Server1 to Server2 for property Hostname
Adding value Server2 for property FileServer
Adding value Server3 for property Repository
Adding value Server4 for property DBServer
Overwriting value 5001 to 5002 for property Port
Adding value {Some String} for property SQLDriver
Adding value E:\SomeFileLocation\ for property LogPaths
Adding value MuchMoreEntries for property MuchMoreProperties
Adding value SomeOtherValue for property NotInJson2
Overwriting value C:\test\123 to E:\Prod\123 for sub property Path1
Overwriting value C:\test\456 to E:\Prod\456 for sub property Path2
Adding value C:\test\456 for sub property Path3
Adding value C:\test\456 for sub property Path4
Adding property Paths

NotInJson2         : SomeOtherValue
MuchMoreProperties : MuchMoreEntries
DBServer           : Server4
Repository         : Server3
Port               : 5002
AndSoOn            : SomeValues
SQLDriver          : {Some String}
FileServer         : Server2
Hostname           : Server2
FileExtension      : .dll
Filename           : SomeName
SubPath            : \SomePath\
LogPaths           : E:\SomeFileLocation\
SQLServer          : Serverx
Paths              : @{Path3=; Path1=E:\Prod\123; Path4=; Path2=E:\Prod\456}
NotInJson1         : SomeOtherValue

[wotanshammer]

Hi, nice but the output is not correct.

Paths : @{Path3=; Path1=E:</span>Prod</span>123; Path4=; Path2=E:</span>Prod</span>456}

 

Path 3/4 are empty

[wotanshammer]

function Combine-Objects {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory=$true)]
        [Object]$Json1,
        [Parameter(Mandatory=$true)]
        [Object]$Json2
    )

    $Object = [ordered] @{}
    foreach ($Property in $Json1.PSObject.Properties) {
    $Object += @{$Property.Name = $Property.Value}
                }

       foreach ($Property in $Json2.PSObject.Properties) {
            try{
                $Object += @{$Property.Name = $Property.Value}
            }catch{
                $PropName = $($Property.Name)
                if($Property.TypeNameOfValue -notmatch 'System.String'){
                    foreach ($Val in $Object.$PropName.PSObject.Properties){
                        $newVal = $Json2.$PropName.PSObject.Properties | Where-Object {$_.Name -eq $Val.Name}
                    if ($newVal) {
                        $Val.Value = $newVal.Value
                       }       
                    }
                  }else{$Object.$($Property.Name) =  $Property.Value}
                }
            }
      return [pscustomobject] $Object
        }

 

[rob-simmers]

The verbose output is correct, so line 55 should reference the $subProp, not $newSubVal, this…

$newObjSubProps.Add($subProp.Name, $newSubVal.Value)

should be:

$newObjSubProps.Add($subProp.Name, $subProp.Value)

[wotanshammer]

Thanks for you support and… a nice solution.

But the code i posted do the same…Try it, same result

function Merge-Objects {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory=$true)]
        [Object]$Json1,
        [Parameter(Mandatory=$true)]
        [Object]$Json2
    )

    $Object = [ordered] @{}
    foreach ($Property in $Json1.PSObject.Properties) {
    $Object += @{$Property.Name = $Property.Value}
                }

       foreach ($Property in $Json2.PSObject.Properties) {
            try{
                $Object += @{$Property.Name = $Property.Value}
            }catch{
                $PropName = $($Property.Name)
                if($Property.TypeNameOfValue -notmatch 'System.String'){
                    foreach ($Val in $Object.$PropName.PSObject.Properties){
                        $newVal = $Json2.$PropName.PSObject.Properties | Where-Object {$_.Name -eq $Val.Name}
                    if ($newVal) {
                        $Val.Value = $newVal.Value
                       }       
                    }
                  }else{$Object.$($Property.Name) =  $Property.Value}
                }
            }
      return [pscustomobject] $Object
        }


        $Json1 = @"
{
    "Hostname": "Server1",
    "FileServer": "Server2",
    "Repository": "Server3",
    "DBServer": "Server4",
    "Port": "5001",
    "SQLDriver": "{Some String}",
    "LogPaths": "E:\\SomeFileLocation\\",
    "MuchMoreProperties": "MuchMoreEntries",
    "NotInJson2": "SomeOtherValue",
    "Paths":{
        "Path1": "C:\\test\\123",
        "Path2": "C:\\test\\456",
        "Path3": "C:\\test\\456",
        "Path4": "C:\\test\\456"
    }
}
"@ | ConvertFrom-Json
 
$Json2 = @"
{
    "Hostname": "Server2",
    "Port": "5002",
    "SQLServer": "Serverx",
    "FileExtension": ".dll",
    "Filename": "SomeName",
    "SubPath": "\\SomePath\\",
    "AndSoOn": "SomeValues",
    "NotInJson1": "SomeOtherValue",
    "Paths":{
        "Path1": "E:\\Prod\\123",
        "Path2": "E:\\Prod\\456"
    }
}
"@ | ConvertFrom-Json


Merge-Objects -Json1 $Json1 -Json2 $Json2

rolf

 

[rob-simmers]

Glad your code is working, a couple of tips:

1. While you could argue everything is an object in Powershell, $object would normally be a PSObject. That is created with the type accelerator at the end [pscustomobject]. You're working with a hashtable or dictionary, key\value pairs. Using += is typically a performance hit and not recommended, especially when the hashtable has methods to manage (add\remove\set) the hashtable.

   PS C:\Users\rasim> $hash = [ordered] @{}    
   
   PS C:\Users\rasim> $hash.Add('Prop1','value1')
   
   PS C:\Users\rasim> $hash['Prop2'] = 'Value2'
   
   PS C:\Users\rasim> $hash['Prop2'] = 'New Value'
   
   PS C:\Users\rasim> $hash.Set_Item('Prop1','Another New Value')
   
   PS C:\Users\rasim> $hash
   
   
   Name                           Value
   ----                           -----
   Prop1                          Another New Value
   Prop2                          New Value
   
   PS C:\Users\rasim> $hash.Contains('Prop2')
   
   True
   PS C:\Users\rasim> 

2. try\catch here is a bit dangerous as anything could error and you're just executing another code block. Recommend that you use .Contains (for ordered hash) or .ContainsKey for a standard hash to see if a key exists
3. Name functions singular, not plural. Get-Process, Get-Service, etc.

Here is a modified version:

function Merge-Object {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory=$true)]
        [object]$Json1,
        [Parameter(Mandatory=$true)]
        [object]$Json2
    )

    $hash = [ordered] @{}
    foreach ($Property in $Json1.PSObject.Properties) {
        $hash[$Property.Name] = $Property.Value
    }

    foreach ($Property in $Json2.PSObject.Properties) {
        if ($hash.Contains($Property.Name)) {
            $hash[$Property.Name] = $Property.Value
        }
        else {
            $PropName = $($Property.Name)
            if ($Property.TypeNameOfValue -notmatch 'System.String'){
                foreach ($Val in $hash.$PropName.PSObject.Properties){
                    $newVal = $Json2.$PropName.PSObject.Properties | Where-hash {$_.Name -eq $Val.Name}
                    if ($newVal) {
                        $Val.Value = $newVal.Value
                    }       
                }
            }
            else{
                $hash[$Property.Name] = $Property.Value
            }
        }
    }

    [pscustomobject]$hash
}


$Json1 = @"
{
    "Hostname": "Server1",
    "FileServer": "Server2",
    "Repository": "Server3",
    "DBServer": "Server4",
    "Port": "5001",
    "SQLDriver": "{Some String}",
    "LogPaths": "E:\\SomeFileLocation\\",
    "MuchMoreProperties": "MuchMoreEntries",
    "NotInJson2": "SomeOtherValue",
    "Paths":{
        "Path1": "C:\\test\\123",
        "Path2": "C:\\test\\456",
        "Path3": "C:\\test\\456",
        "Path4": "C:\\test\\456"
    }
}
"@ | ConvertFrom-Json
 
$Json2 = @"
{
    "Hostname": "Server2",
    "Port": "5002",
    "SQLServer": "Serverx",
    "FileExtension": ".dll",
    "Filename": "SomeName",
    "SubPath": "\\SomePath\\",
    "AndSoOn": "SomeValues",
    "NotInJson1": "SomeOtherValue",
    "Paths":{
        "Path1": "E:\\Prod\\123",
        "Path2": "E:\\Prod\\456"
    }
}
"@ | ConvertFrom-Json


Merge-Object -Json1 $Json1 -Json2 $Json2

[wotanshammer]

Thanks for the Tips…I will keep it in mind.

But…The modified code delivers wrong output

@{Path1=E:\Prod\123; Path2=E:\Prod\456}

 

[rob-simmers]

Find and replace can be a pain…should be where-object…

| Where-hash {$_.Name -eq $Val.Name}

[wotanshammer]

Sometimes…i know.

But that wasn’t the problem

Same result

The first Foreach overwrites the Path result:

foreach ($Property in $Json2.PSObject.Properties) {
        if ($hash.Contains($Property.Name)) {
            $hash[$Property.Name] = $Property.Value
        }