Parse output of console (file) put in a hash table and convert JSON

pavel · June 5, 2017, 12:06pm

Hello all,
i’am trying get data from console program using PS convert it to a json format, and then push it to a zabbix.
Here is an example of raw output

Physical Device information

  Device #0
     Device is a Hard drive
     State                              : Online
     Block Size                         : 512 Bytes
     Supported                          : Yes
     Transfer Speed                     : SAS 3.0 Gb/s
     Reported Channel,Device(T:L)       : 0,0(0:0)
     Reported Location                  : Connector 0, Device 0
     Vendor                             : HITACHI
     Model                              : HUS156030VLS600
     Firmware                           : A5D0
     Serial number                      : JTWYYAAJ
     World-wide name                    : 5000CCA00F6ED187

Here is my “code” )

$f=get-content "D:\arcconf_pd.txt"

$Content=$f -replace "\s+|(-)|(Physical Device information)|(Device is a Hard drive)".split("`n") -match '\S'

$array = @()

$Content -split "`r`n" | % {

    $Delimited = $_ -split "`:"
    $object = New-Object -TypeName PSObject
    $object | Add-Member -MemberType NoteProperty -Name $Delimited[0] -Value $Delimited[1]
    $array += $object

}
$array | ConvertTo-Json

Here what i’am getting

[
    {
        "Device#0":  null
    },
    {
        "State":  "Online"
    },
    {
        "BlockSize":  "512Bytes"
    },
    {
        "Supported":  "Yes"
    },
    {
        "TransferSpeed":  "SAS3.0Gb/s"
    },
]

But Zabbbix need anothe structure of json
Here is example

{ 
 "data":[

 { "{#VMNAME}":"cp01" , "{#VMSTATE}":"Off" },
 { "{#VMNAME}":"dc01" , "{#VMSTATE}":"Running" },
 { "{#VMNAME}":"ex01" , "{#VMSTATE}":"Off" },
 { "{#VMNAME}":"fc01" , "{#VMSTATE}":"Running" },
 { "{#VMNAME}":"rdg01" , "{#VMSTATE}":"Running" },
 { "{#VMNAME}":"Windows 2012R2 G2 Template" , "{#VMSTATE}":"Off" }, <--- THIS I WANT TO REMOVE

 ]
}

Question is, how can i modify my code to get a Zabbix style json?

curtis-smith · June 5, 2017, 10:22pm

Looking at the output from your code and the output from Zabbix, you don’t have enough information.

The zabbix JSON appears to have a VMName, and the output from your code does not. Additionally, you have a lot of other information in your code output that the Zabbix JSON does not account for. You have Device, BlockSize, Supported, and TransferSpeed while Zabbix only has State.

Next you are making it difficult on your self by storing all the individual properties in separate objects.

If you move your new object outside of the loop then all of the add-member commands add the properties to the same object so it is easier to pull values from.

$f=get-content "D:\input.txt"

$Content=$f -replace "\s+|(-)|(Physical Device information)|(Device is a Hard drive)".split("`n") -match '\S'

$array = @()
$object = New-Object -TypeName PSObject
$Content -split "`r`n" | % {

    $Delimited = $_ -split "`:"
    $object | Add-Member -MemberType NoteProperty -Name $Delimited[0] -Value $Delimited[1]
}
$object

Results

Device#0                 : 
State                    : Online
BlockSize                : 512Bytes
Supported                : Yes
TransferSpeed            : SAS3.0Gb/s
ReportedChannel,Device(T : L)
ReportedLocation         : Connector0,Device0
Vendor                   : HITACHI
Model                    : HUS156030VLS600
Firmware                 : A5D0
Serialnumber             : JTWYYAAJ
Worldwidename            : 5000CCA00F6ED187

Now you can use that object to get the values you want for your JSON.
Note that I just added $computername. Since that is not actually in your data set, it will have to come from somewhere.

$f=get-content "D:\input.txt"
$computername = "mycomputer"

$Content=$f -replace "\s+|(-)|(Physical Device information)|(Device is a Hard drive)".split("`n") -match '\S'

$array = @()
$object = New-Object -TypeName PSObject
$Content -split "`r`n" | % {

    $Delimited = $_ -split "`:"
    $object | Add-Member -MemberType NoteProperty -Name $Delimited[0] -Value $Delimited[1]
}

@{
    "data" = @(
        @{"{#VMNAME}"=$computername; "{#VMSTATE}"=$object.State}
    )
} | ConvertTo-Json

Results

{
    "data":  [
                 {
                     "{#VMNAME}":  "mycomputer",
                     "{#VMSTATE}":  "Online"
                 }
             ]
}

pavel · June 21, 2017, 6:32am

Thank you, that helps a lot!
But now i fighting with another problem,
if i try to parse text info about one HDD connected to controller using a text file it’s working without any problem

----------------------------------------------------------------------

Physical Device information

  Device #0
     Device is a Hard drive
     State                              : Online
     Block Size                         : 512 Bytes
     Supported                          : Yes
     Transfer Speed                     : SAS 3.0 Gb/s
     Reported Channel,Device(T:L)       : 0,0(0:0)
     Reported Location                  : Connector 0, Device 0
     Vendor                             : HITACHI
     Model                              : HUS156030VLS600
     Firmware                           : A5D0
     Serial number                      : JTWYYAAJ
     World-wide name                    : 5000CCA00F6ED187
     Reserved Size                      : 483416 KB
     Used Size                          : 285696 MB
     Unused Size                        : 64 KB
     Total Size                         : 286168 MB
     Write Cache                        : Enabled (write-back)
     FRU                                : None
     S.M.A.R.T.                         : No
     S.M.A.R.T. warnings                : 0
     Power State                        : Full rpm
     Supported Power States             : Full rpm,Powered off
     SSD                                : No
     MaxCache Capable                   : No
     MaxCache Assigned                  : No

, but when i getting full output (this program can print only full info about all connected devices) with several hdd's it is natural that i getting error "MemberAlreadyExists" Here is a full output

----------------------------------------------------------------------
Physical Device information
----------------------------------------------------------------------
      Device #0
         Device is a Hard drive
         State                              : Online
         Block Size                         : 512 Bytes
         Supported                          : Yes
         Transfer Speed                     : SAS 3.0 Gb/s
         Reported Channel,Device(T:L)       : 0,0(0:0)
         Reported Location                  : Connector 0, Device 0
         Vendor                             : HITACHI
         Model                              : HUS156030VLS600
         Firmware                           : A5D0
         Serial number                      : JTWYYAAJ
         World-wide name                    : 5000CCA00F6ED187
         Reserved Size                      : 483416 KB
         Used Size                          : 285696 MB
         Unused Size                        : 64 KB
         Total Size                         : 286168 MB
         Write Cache                        : Enabled (write-back)
         FRU                                : None
         S.M.A.R.T.                         : No
         S.M.A.R.T. warnings                : 0
         Power State                        : Full rpm
         Supported Power States             : Full rpm,Powered off
         SSD                                : No
         MaxCache Capable                   : No
         MaxCache Assigned                  : No
      Device #1
         Device is a Hard drive
         State                              : Online
         Block Size                         : 512 Bytes
         Supported                          : Yes
         Transfer Speed                     : SAS 3.0 Gb/s
         Reported Channel,Device(T:L)       : 0,1(1:0)
         Reported Location                  : Connector 0, Device 1
         Vendor                             : HITACHI
         Model                              : HUS156030VLS600
         Firmware                           : A5D0
         Serial number                      : JTWWKE8J
         World-wide name                    : 5000CCA00F6A7BCB
         Reserved Size                      : 483416 KB
         Used Size                          : 285696 MB
         Unused Size                        : 64 KB
         Total Size                         : 286168 MB
         Write Cache                        : Enabled (write-back)
         FRU                                : None
         S.M.A.R.T.                         : No
         S.M.A.R.T. warnings                : 0
         Power State                        : Full rpm
         Supported Power States             : Full rpm,Powered off
         SSD                                : No
         MaxCache Capable                   : No
         MaxCache Assigned                  : No

I understand that i need to build some cycle for count number of connected devices, and then build objects with unique number.

#$f=get-content "C:\input.txt"
$Content=$f -replace "\s+|(-)|(Physical Device information)|(Device is a Hard drive)".split("`n") -match '\S'
#CountHDDinSystem
#$HowMuchHDDs = Select-String -InputObject $f -Pattern "Device #" -AllMatches
#$HowMuchHDDs.Matches.Count

$array = @()
$object = New-Object -TypeName PSObject
$Content -split "`r`n" | % {

    $Delimited = $_ -split "`:"
    $object | Add-Member -MemberType NoteProperty -Name $Delimited[0] -Value $Delimited[1]
    $object | Add-Member -MemberType Property -Name $Delimited[0] -Value $Delimited[1]
}


@{
    "data" = @(
        @{
        "{#HDDNumber}"=$object.'device'
        "{#Vendor}"=$object.Vendor;
        "{#Model}"=$object.Model;
        "{#Firmware}"=$object.Firmware;
        "{#SerialNumber}"=$object.Serialnumber;
        "{#TransferSpeed}"=$object.TransferSpeed;
        "{#ReservedSize}"=$object.ReservedSize;
        "{#UsedSize}"=$object.UsedSize;
        "{#UnusedSize}"=$object.UnusedSize;
        "{#TotalSize}"=$object.TotalSize;
        "{#WriteCache}"=$object.WriteCache;
        "{#FRU}"=$object.FRU;
        "{#S.M.A.R.T}"=$object.'S.M.A.R.T.';
        "{#S.M.A.R.R.Warnings}"=$object.'S.M.A.R.T.warnings';
        "{#PowerState}"=$object.PowerState;
        "{#SSD}"=$object.SSD;
      }
        
       
        
    )
} | ConvertTo-Json

Please nudge me in the right direction ) TNX!

curtis-smith · June 21, 2017, 11:13am

Yes, based on your code, that is reasonable.

You are parsing through a flat text file and creating an object, then adding properties to that object as you work down the list

Example



device0

property1

property2

device1

property1

property2

you create your object for device0, then add property1 and property2.

Then you get to device1 in your text file, but you don’t create a new object, you keep using the same one you created for device0. So when you try to add property1 again, you get the message that the member already exists.

You need to add logic to recognize that this is a new object

I would probably use a hash table to collect all my objects doing something like this

$content = @'
device#0
property1:value1
property2:value2
device#1
property1:value3
property2:value4
'@ -split "`r`n"

$devices = @{}
$content |
ForEach-Object {
    If ($_ -match "device#\d") {
        $devices[$_] = New-Object -TypeName PSObject
        $device = $_
    } Else {
        $Delimited = $_ -split "`:"
        $devices[$device] | Add-Member -MemberType NoteProperty -Name $Delimited[0] -Value $Delimited[1]
    }
}
$devices
$devices.Values

Parse output of console (file) put in a hash table and convert JSON

Physical Device information

Thank you, that helps a lot! But now i fighting with another problem, if i try to parse text info about one HDD connected to controller using a text file it’s working without any problem ---------------------------------------------------------------------- Physical Device information

Thank you, that helps a lot!
But now i fighting with another problem,
if i try to parse text info about one HDD connected to controller using a text file it’s working without any problem
----------------------------------------------------------------------
Physical Device information