Select-String by pattern then if found actions

I am looking to search a log file for several patterns which I have working. What I would like to do to expand upon that is when it finds pattern X THEN search UP in the text log for a new pattern, down to the same pattern searched UP for to establish which lines to grab. A less technical summary would be this log has hundreds to thousands of operations in it and i want the log entries for an individual set of entries for one operation IF it contains an error. Not sure how to present the exact information being worked with without information overload.

this is the script that finds the error entries for some context to what i want to add on to…

 

$SawLog = Get-Content "C:\temp\run.log"

$PrinterLog = "C:\temp\PrinterLog.txt"

$Date = "2018-08-07"

$PrintErrors = @()

$SortMe = @()

$NoDate = @()

$Final = @()

 

If (Test-Path $PrinterLog) {Remove-Item -Path $PrinterLog -Force}

$SawLog | %{

if ($_ -match "PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078") {

if ($Matches[0] -eq "PRT0009") {

$LineIndex = $SawLog.IndexOf($_) + 1 # Get Next Line

$Output = $_, $SawLog[$LineIndex]

}

elseif ($Matches[0] -eq "\[ID: ENG0037") {

$LineIndex = $SawLog.IndexOf($_) + 3 # Get Next Line

$Output = $_, $SawLog[$LineIndex]

}

else { $Output = $_ }

$PrintErrors += $Output | Out-String

}

}

 

$SortMe += $SawLog -match "\[PRT(0004|0005|0009)|\[ID: ENG(0029|0037)|OPR0078" | Select-String -pattern $Date | %{ $List = $_ -Split '\['

$NoDate += $List[3] | Out-String  }

 

$Final += $NoDate | Group-Object -NoElement| Format-Table -AutoSize | Out-String

 

"Print Error Details `n`n" + $PrintErrors + "`n Error Summary For $Date `n" + $Final  | Set-Content -Path $PrinterLog

 

Start-Process -FilePath 'Notepad++' -ArgumentList $PrinterLog

when it finds pattern X THEN

search UP in the text log for a new pattern,

down to the same pattern searched UP for to establish which lines to grab.


Whaaaaat???

Find X

Match X

Ignore X and replace with Y, starting from the index of X

Search up for Y and extract Y

Whaaaaat ???

What happens when string are below the X index ???

Color me really confused.

Why not just group matches for the extract.

 

ok lets try this. I want to select-string all lines from “Part Started” to “Part Complete” If between those two -match “[PRT(0004|0005|0009)|[ID: ENG(0029|0037)|OPR0078” result is found. does that clarify what im up to? NOTE the number of lines between “Part Started” and “Part Complete” is not a static amount.

So what I have found is maybe a Select-String -Context but i need to ID the lines for a line number ID to make a Start point and for an end point. Then maybe a Select-Object -Index to pull that group to an array to then say IF -match is true keep it and {do something} else if false discard and move to next. This process would need to function repeatedly without duplication to process the log file maybe in a foreach loop. Does this sound plausible?

So I have been working at an answer to this on and off and so far no luck. Any -match or select-string statement works for the direct line with an error or desired piece of info but in trying to broaden the scope and gather relevant information to that error in the log with other entries the wall I run into is those other entries are an unknown number of lines from the error so I can’t just say -context to a select-string. Something I’ve used prior in a MDB code looks like maybe it’s viable but i have not worked with it in other ways does the following code snip have potential to do what I’m trying to describe?

 

$SawLog = Get-Content "C:\temp\Complete.log"

$cache = @()

$FindErrors = $SawLog -match "Part Started -"

$FindErrors.MoveFirst()

do {

if ($Matches[0] -match "PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger") {

$Cache += NotSureHowToAddAboveLineMatchesToArray

$FindErrors.MoveNext()}

Else {

$FindErrors.MoveNext() }}

until($FindErrors.EOF -eq $True)

Get-Content is going to give us an array of lines, so we just need to flag when to start gathering data. If we use a basic example with just leveraging like to start a flag and break the loop:

$test = @"
data
data
data
trigger
data i want
data i want
data i want
data i want
data i want
end trigger
data
data
data
"@

$getData = $false

$results = foreach ($line in ($test -split [environment]::NewLine)) {

    if ($line -like 'end tri*') {
        #We have all of the data, break loop
        break
    }

    if ($getData -eq $true) {
        #Return the line
        $line
    }
    
    if ($line -like 'trigg*') {
        #Start flag to gather data
        $getData = $true
        
    }
}

$results

Output:

$results


data i want
data i want
data i want
data i want
data i want

If you want the trigger lines, you can just change the order around

$results = foreach ($line in ($test -split [environment]::NewLine)) {
 
    if ($line -like 'trigg*') {
        #Start flag to gather data
        $getData = $true
        
    }

    if ($getData -eq $true) {
        #Return the line
        $line
    }
    
    if ($line -like 'end tri*') {
        #We have all of the data, break loop
        break
    }
}

$results

Output:

$results


trigger
data i want
data i want
data i want
data i want
data i want
end trigger

Also, if you had 3 segements in a single log, just remove the break and set the flag to false and you should get all of the data.

So that is a good example and I like how that is written but I don’t see how to handle the part that really has me hung up. I will reuse your example code to try and better explain…

$test = @"
data
data
data
Part Started Trigger
data # of Unknown Lines with No Errors # i do not want this trigger to end trigger because there is no error between them
Part Complete End Trigger
data
data
Part Started Trigger
data # of Unknown Lines with Error  # i want this trigger to end trigger because it contains an error
Part Complete End Trigger
data
data
data
"@

$getData = $false

$results = foreach ($line in ($test -split [environment]::NewLine)) {

    if ($Line -like 'Part Started -*') {
        #Start flag to gather data
        $GetData = $True
    }

    if ($GetData -eq $True) {
        #Return the line
        $Line
    }
    
    if ($Line -like 'Part Complete*') {
        #We have all of the data, break loop
        $GetData = $False
       
    }
}

$results

To do what I am after (from my very much still learning opinion) is to compound the foreach to run what this would get per trigger set that looks at a regex to determine if it contains an error and if it does send it to an array if it does not move to the next trigger set.

PJ,

Let me make sure I understand your problem. Given the example data below you would want to return the array of all of segment 2 but NOT segment 1 because somewhere in segment 1 it contains an error. Is that correct?

 

$test = @"
data
data
data
Part Started Trigger
data segment 1
data error segment 1
data segment 1
Part Complete End Trigger
data
data
Part Started Trigger
data segment 2
data segment 2
Part Complete End Trigger
data
data
data
"@

Yes but backwards. I want the segment with the error to return to an array that compiles all found segments containing errors. If the segment does not contain an error i do not want it.

So I modified the example to this and I think I’m close but my guess is the regex area is multiplying data and making it not finish the script?

$SawLog = Get-Content "C:\temp\Complete.log"
$Final = @()
$FullSelect = @()

#this part is to greatly reduce lines needing parsed as each log file is over 10k lines and 
#is used to strip info down to the parts I want between triggers
$FullSelect += $SawLog -match "Part( Started -| Complete)|PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger"

$GetData = $False

$Results = foreach ($line in ($FullSelect -split [environment]::NewLine)) {
    
    if ($line -match 'Part Started -') {
        #Start flag to gather data
        $GetData = $True
    }

    if ($GetData -eq $True) {
        #Return the line
        $line
    }
    
    if ($line -match 'Part Complete') {
        #We have all of the data, break loop
        foreach ($line in ($Results -split [environment]::NewLine)) { 
        If ($line -match "PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger") {
        $Final += $Results
        } Else {
        Continue}}
        $GetData = $False

    }
}

$Final

OK. With that the logic it is a little more complicated. Here is what I came up with. Someone may have a more eliquent solution, but this works. BTW, [environment]::NewLine -ne “`n” on my system, so I can’t use it in my example for the -split operator. Maybe someone can explain why…

 

$test = @"
data
data
data
Part Started Trigger
data segment 1
data error segment 1
data segment 1
Part Complete End Trigger
data
data
Part Started Trigger
data segment 2
data segment 2
Part Complete End Trigger
data
data
data
"@

$getData = $false


$data = $test -split '\n'
$results = foreach ($line in $data) {

    if ($line -like 'Part Started *') {
        $GetData = $True
        $trap = $false
        $temp = @()
        continue #to not include the trigger line
    }

    if ($GetData -eq $True) {
        if ($line -like '*error*') {
            $trap = $True
        }
        else { 
            $temp += $line 
        }
    }
    
    if ($Line -like 'Part Complete *') {
        $GetData = $False
        if ($trap){
            $temp[0..(($temp.Count) - 2)] # if you don't want to include trigger line otherwise just $temp
        }                    
    }
}

$results

As I am looking at my code again, it may read better to use an else if on line 38 to omit the trigger line rather than the clunky code on line 46.

I do want to show trigger lines so thats no worry. Working on trying what you posted will reply shortly.

So Mike this returns the same as

$SawLog = Get-Content "C:\temp\Complete.log"

$Results = @()
$Results += $SawLog -match "Part( Started -| Complete)|PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger"

$Results

I need to see the trigger lines as they contain data in reference to what part the error happened on but I do not want to see them if there is no error between start and complete

I think I got it, anyone see any issues with the following?..

$SawLog = Get-Content "C:\temp\Complete.log"
$ProcessArray = @()

$FullSelect += $SawLog -match "Part( Started -| Complete)|PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger"

$GetData = $False

$Results = foreach ($line in ($FullSelect -split [environment]::NewLine)) {
    
    if ($line -match 'Part Started -') {
        #Start flag to gather data
        $PartStart = $Line
        $GetData = $True
        $Trap = $False
    }

    if ($GetData -eq $True) {
        #Return the line
        if ($line -match "PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger") {
            $trap = $True 
            $ProcessArray += $PartStart
            $ProcessArray += $line 
            } Else {
            continue}
    }
    
    if ($line -match 'Part Complete') {
        #We have all of the data, break loop
        $PartComplete = $line
        $GetData = $False
        if ($trap) {
        $ProcessArray += $PartComplete
        $ProcessArray
        }
        $trap = $False
        

    }
}

$ProcessArray

crap not quite that is missing part completes in the output

OK Now I think its working…

$SawLog = Get-Content "C:\temp\Complete.log"
$ProcessArray = @()

$FullSelect += $SawLog -match "Part( Started -| Complete)|PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger"

$GetData = $False

$Results = foreach ($line in ($FullSelect -split [environment]::NewLine)) {
    
    if ($line -match 'Part Started -') {
        #Start flag to gather data
        $PartStart = $Line
        $GetData = $True
        $Trap = $False
    }

    if ($line -match "Part Complete") {
        $PartComplete = $line
    }

    if ($GetData -eq $True) {
        #Return the line
        if ($line -match "PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger") {
            $trap = $True 
            $ProcessArray += $PartStart
            $ProcessArray += $line 
            $ProcessArray += $PartComplete 
            } Else {
            Continue}
    }

    if ($line -match 'Part Complete') {
        #We have all of the data, break loop
        $GetData = $False
        if ($trap) {
        $ProcessArray
        }
    }
}

$ProcessArray

Ok its got quirks and I’m not sure how to deal with them. The script prior to this post works but I get a part started and part complete for each error. I would like it to be part started, all errors on that part, then part complete. My output looks like this and looks like the error is to the prior part started…

2018-08-14 10:02:35,322 [SYS.Com.BoardStatus[Part Started - Board ID: 29 Part Seq: 8 Part XML ID: 0 Dsc: Job: 4402-J Remainder ID: 5
2018-08-14 10:00:26,976 [SYS.Com.Machine[Machine Status has been updated from ActiveCutting to WaitingForPrinterTrigger.
2018-08-14 10:02:51,445 [SYS.Com.BoardStatus[Part Complete - Board ID: 29 Part Seq: 8 Part XML ID: 0 Dsc: Job: 4402-J Remainder ID: 5

2018-08-14 10:02:35,322 [SYS.Com.BoardStatus[Part Started - Board ID: 29 Part Seq: 8 Part XML ID: 0 Dsc: Job: 4402-J Remainder ID: 5
2018-08-14 10:00:36,259 [SYS.Com.Machine[Machine Status has been updated from WaitingForPrinterTrigger to ReadyHomed.
2018-08-14 10:02:51,445 [SYS.Com.BoardStatus[Part Complete - Board ID: 29 Part Seq: 8 Part XML ID: 0 Dsc: Job: 4402-J Remainder ID: 5

2018-08-14 10:02:35,322 [SYS.Com.BoardStatus[Part Started - Board ID: 29 Part Seq: 8 Part XML ID: 0 Dsc: Job: 4402-J Remainder ID: 5
2018-08-14 10:01:00,821 [SYS.Com.Machine[Machine Status has been updated from OneBoardActive to WaitingForPrinterTrigger.
2018-08-14 10:02:51,445 [SYS.Com.BoardStatus[Part Complete - Board ID: 29 Part Seq: 8 Part XML ID: 0 Dsc: Job: 4402-J Remainder ID: 5

2018-08-14 10:02:35,322 [SYS.Com.BoardStatus[Part Started - Board ID: 29 Part Seq: 8 Part XML ID: 0 Dsc: Job: 4402-J Remainder ID: 5
2018-08-14 10:01:02,604 [SYS.Com.Machine[Machine Status has been updated from WaitingForPrinterTrigger to OneBoardActive.
2018-08-14 10:02:51,445 [SYS.Com.BoardStatus[Part Complete - Board ID: 29 Part Seq: 8 Part XML ID: 0 Dsc: Job: 4402-J Remainder ID: 5

Can I do loop inside of the foreach? I cant seem to get it working but something like bellow to get a part started, all errors on that part, then part complete. Also last post disregard time stamp order issue, that was me messing with it to work at this modification.

$SawLog = Get-Content "C:\temp\Complete.log"
$ProcessArray = @()


$FullSelect += $SawLog -match "Part( Started -| Complete)|PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger"

$GetData = $False

$Results = foreach ($line in ($FullSelect -split [environment]::NewLine)) {
    
    if ($line -match 'Part Started -') {
        #Start flag to gather data
        $PartStart = $Line
        $GetData = $True
        $Trap = $False
    }

    if ($line -match "Part Complete") {
        $PartComplete = $line
    }

    if ($GetData -eq $True) {
        #Return the line
        if ($line -match "PRT(0004|0005|0009)|ENG(0029|0037)|OPR0078|WaitingForPrinterTrigger") {
            $trap = $True 
            Do{$ProcessArray += $line} until ($line -match 'Part Complete') #< this is there a way to repeat this until it sees part complete?
            $ProcessArray += $PartComplete 
            $ProcessArray += "`n"
            } Else {
            Continue}
    }

    if ($line -match 'Part Complete') {
        #We have all of the data, break loop
        $GetData = $False
        if ($trap) {
        $ProcessArray
        }
    }
}

$ProcessArray