Need Help with PS script

PowerShell Help
1

So I’m attempted to create a script that look for a specific string inside a .log file and export it to csv file. For example:

 

5008( 2780) 08/21/2020 02:01:53 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:01:58 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:02:03 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:02:08 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:02:14 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:02:19 JS I TypeError: task has no properties

4672( 4900) 08/21/2020 02:02:50 JS I INFO - CalcChartSvc - ------- Calculating the charts ------

4672( 4900) 08/21/2020 02:02:50 JS I INFO - CalcChartSvc - Current Operator: report.calccharts

5008( 2164) 08/21/2020 02:08:20 JRTE I Initialize successfully.

4672( 4900) 08/21/2020 02:12:52 JS I INFO - CalcChartSvc - Switching to operator: Incident.Manager, previous: report.calccharts

4672( 4900) 08/21/2020 02:12:52 RTE W ResetMandanten, operator Incident.Manager is not found

5008( 364) 08/21/2020 02:14:54 RTE A Mapping-3-globallists,server.application, SQL Query incomplete because field (server.application) not mapped in file (globallists)

5008( 364) 08/21/2020 02:14:54 RTE A SUMMARY-1 The following event has been reported 15 times in the last 30 minute(s)

5008( 3336) 08/21/2020 03:00:27 RAD I [ERROR][SX_EntityChangeV2]: Trigger entityAfterUpdate incidents RF104462 failed: Could not find incident with id=RF104462, rc=No (more) records found

5008( 3336) 08/21/2020 03:00:41 RAD I [ERROR][SX_EntityChangeV2]: Trigger entityAfterUpdate request RF110543 failed: Could not find incident for dependency RF110543, rc=No (more) records found

 

What I want to end up seeing is:

ErrorLog DATE TIME ErrorCode ErrorType Message

5008( 3336), 08/21/2020, 03:00:41, RAD I, [ERROR][SX_EntityChangeV2]:, Trigger entityAfterUpdate request RF110543 failed: Could not find incident for dependency RF110543, rc=No (more) records found

 

I’m getting stuck whether I should create a custom object with hash. Or use regex and substring method.

 

Any input will help at this point

 

 

=======

Update

So I was able to write this script to parse the log file using regex:

(Get-Content.\sm.test.txt-Raw)-split'(\s\s\d\d\d\d.\s\s\d\d\d\d.|\d\d\d\d.\s\s\d\d\d\d.|\s+\d+.\s+\d\d\d.)\s(\d\d\D\d\d\D\d\d\d\d)\s(\d{2}:\d{2}:\d{2})\s+(\w+\s\w+)\s(.*)'|
Where{$_}|
ForEach-Object{
[PSCustomObject]@{
'Col1'=$_.SubString(0,12)
'Col2'=$_.Substring(12).Trim()
}
}
But when running the code, i would get and error message :
[PSCustomObject]@{
Exception calling "Substring" with "2" argument(s): "Index and length must refer to a location within the string. (Parameter 'length')"
Something I'm missing?
2

So I was able to write up this regex to parse the log file:

(Get-Content .\sm.test.txt -Raw) -split '(\s\s\d\d\d\d.\s\s\d\d\d\d.|\d\d\d\d.\s\s\d\d\d\d.|\s+\d+.\s+\d\d\d.)\s(\d\d\D\d\d\D\d\d\d\d)\s(\d{2}:\d{2}:\d{2})\s+(\w+\s\w+)\s(.*)'  |
Where{$_} |
    ForEach-Object{
        [PSCustomObject]@{
            'Col1'=$_.SubString(0,12)
            'Col2'=$_.Substring(12).Trim()

        }
    }

But when running script, I get an error message for

Exception calling "Substring" with "2" argument(s): "Index and length must refer to a location within the string. (Parameter 'length')"

Something I’m missing?

3

Check your string lengths.

https://social.technet.microsoft.com/Forums/en-US/1afaac7f-263f-4b0e-8995-ff6a0333c8b8/exception-calling-quotsubstringquot-with-quot2quot-arguments-quotindex-and-length-must?forum=ITCG

4

I personally would not bother with using the -Raw switch here. If you read without -Raw, each line will be its own array element and can be parsed independently. Whatever parsing techniques you use will then apply to each line equally.

I would use the -match operator and named capture groups. You can name the groups what you want your column/property names to be. When you use -match against a single string and find a successful match, the $matches variable (which is a hash table) will contain each capture group name as a key and the matched text as a value.

Get-Content sm.test.txt | Where {$_} | Foreach-Object {
    if ($_ -match '(?<Elog>^\d+\(\s*\d+\))\s+(?<Date>[\d/]+)\s+(?<Time>[\d:]+)\s(?<EType>[^:]+\[ERROR\][^:]+):\s(?<Message>.*)') {
        [pscustomobject]$matches | Select-Object * -Exclude 0
    }
}

I am choosing to exclude capture group 0 because that is the full match. The downside with creating a custom object directly from $matches is the order of the properties. If you want to control the order of the properties, you will need to explicitly call out those capture group names when building your object.

Get-Content sm.test.txt | Where {$_} | Foreach-Object {
    if ($_ -match '(?<Elog>^\d+\(\s*\d+\))\s+(?<Date>[\d/]+)\s+(?<Time>[\d:]+)\s(?<EType>[^:]+\[ERROR\][^:]+):\s(?<Message>.*)') {
        [pscustomobject]@{
            ErrorLog = $matches.Elog
            Date = $matches.Date
            Time = $matches.Time
            ErrorType = $matches.EType
            Message = $matches.Message
        }
    }
}

 

5

AdminofThings always offers great advice, the same is true here. I just wanted to provide an additional way to achieve your desired results, using ConvertFrom-String.

First, we build a template

$template = @'
{ErrorLog*:5008( 3336)} {Date:08/21/2020} {Time:03:00:27} {ErrorCode:RAD I} {ErrorType:[ERROR][SX_EntityChangeV2]}: {Message:Trigger entityAfterUpdate incidents RF104462 failed: Could not find incident with id=RF104462, rc=No (more) records found}

{ErrorLog*:1234( 1234)} {Date:11/21/2019} {Time:12:55:44} {ErrorCode:RAD I} {ErrorType:[ERROR][SX_EntityChangeV2]}: {Message:Trigger entityAfterUpdate request RF110543 failed: Could not find incident for dependency RF110543, rc=No (more) records found}
'@

Now we apply that template to the data using ConvertFrom-String. You could potentially add more “training” data to exclude the additional lines, but that would be much more involved than just filtering based on the error type. Adding more training data may also have a negative impact. The key with ConvertFrom-String template is to provide just enough training data without over doing it.

@'
5008( 2780) 08/21/2020 02:01:53 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:01:58 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:02:03 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:02:08 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:02:14 JS I TypeError: task has no properties

5008( 2780) 08/21/2020 02:02:19 JS I TypeError: task has no properties

4672( 4900) 08/21/2020 02:02:50 JS I INFO - CalcChartSvc - ------- Calculating the charts ------

4672( 4900) 08/21/2020 02:02:50 JS I INFO - CalcChartSvc - Current Operator: report.calccharts

5008( 2164) 08/21/2020 02:08:20 JRTE I Initialize successfully.

4672( 4900) 08/21/2020 02:12:52 JS I INFO - CalcChartSvc - Switching to operator: Incident.Manager, previous: report.calccharts

4672( 4900) 08/21/2020 02:12:52 RTE W ResetMandanten, operator Incident.Manager is not found

5008( 364) 08/21/2020 02:14:54 RTE A Mapping-3-globallists,server.application, SQL Query incomplete because field (server.application) not mapped in file (globallists)

5008( 364) 08/21/2020 02:14:54 RTE A SUMMARY-1 The following event has been reported 15 times in the last 30 minute(s)

5008( 3336) 08/21/2020 03:00:27 RAD I [ERROR][SX_EntityChangeV2]: Trigger entityAfterUpdate incidents RF104462 failed: Could not find incident with id=RF104462, rc=No (more) records found

5008( 3336) 08/21/2020 03:00:41 RAD I [ERROR][SX_EntityChangeV2]: Trigger entityAfterUpdate request RF110543 failed: Could not find incident for dependency RF110543, rc=No (more) records found
'@ | ConvertFrom-String -TemplateContent $template | where errortype -match 'error' -OutVariable results

The output (also captured in the $results variable)

ErrorLog  : 5008( 3336)
Date      : 08/21/2020
Time      : 03:00:27
ErrorCode : RAD I
ErrorType : [ERROR][SX_EntityChangeV2]
Message   : Trigger entityAfterUpdate incidents RF104462 failed: Could not find incident with id=RF104462, rc=No (more) records found

ErrorLog  : 5008( 3336)
Date      : 08/21/2020
Time      : 03:00:41
ErrorCode : RAD I
ErrorType : [ERROR][SX_EntityChangeV2]
Message   : Trigger entityAfterUpdate request RF110543 failed: Could not find incident for dependency RF110543, rc=No (more) records found

This is how you’d do the same from a file. If the file isn’t too large, -Raw will speed it up. ConvertFrom-String will work with or without -Raw.

Get-Content $logfile -Raw | ConvertFrom-String -TemplateContent $template | where errortype -match 'error' -OutVariable results

And you can further process the data using the variable.

$results | where errorlog -like '5008*' | select date,time,message

Date       Time     Message                                                                                                                       
----       ----     -------                                                                                                                       
08/21/2020 03:00:27 Trigger entityAfterUpdate incidents RF104462 failed: Could not find incident with id=RF104462, rc=No (more) records found     
08/21/2020 03:00:41 Trigger entityAfterUpdate request RF110543 failed: Could not find incident for dependency RF110543, rc=No (more) records found


$results | Format-Table -AutoSize

ErrorLog    Date       Time     ErrorCode ErrorType                  Message                                                                                                                  
--------    ----       ----     --------- ---------                  -------                                                                                                                  
5008( 3336) 08/21/2020 03:00:27 RAD I     [ERROR][SX_EntityChangeV2] Trigger entityAfterUpdate incidents RF104462 failed: Could not find incident with id=RF104462, rc=No (more) records found
5008( 3336) 08/21/2020 03:00:41 RAD I     [ERROR][SX_EntityChangeV2] Trigger entityAfterUpdate request RF110543 failed: Could not find incident for dependency RF110543, rc=No (more) recor...