Powershell String operation

Hi,

I’m in a tricky situation to analyze a text output. My requirement is search a string and get a lines between them. For example, the following lines in a text file

Filename1.log
Data1
Data2
Data3
Filename2.log
Data4
Data5
Data6
Data7
Filename3.log
Data8
Data9
Data10

it goes several files as the output varies

output I expect
Filename1.log has 3 lines
Filename2.log has 4 lines
Filename3.log has 3 lines

Please let me know if anyone can provide help, couldn’t get through surfing

Charles,
welcome to the forums.

What have you tried so far? Please show your code and explain what is not working as expected or where you’ve got stuck. Include error messages if there are some.

Hi Olaf, thanks for getting back

I’ve got some knowledge in Powershell. I actually tried various method but something with regex gives little bit But it can give result for two matches matches and not sure a loop possible here, but I think we can achieve through StreamReader readline and filter by particular string?, I dont know how to do and not much help from internet

$File = Get-Content .\log.txt
$pat = "(?:[\w]\:)(\\[a-z_A-Z\-\s0-9\.]+)+\.log"
$repeat = Select-String .\logger.txt -pattern "(?:[\w]\:)(\\[a-z_A-Z\-\s0-9\.]+)+\.log" 
for ($i=0; $i -lt $repeat.Count;  $i++) {
        $patt = "$pat(.*?)$pat"
        $d = [regex]::Match($File,$patt).groups[2].value
        $f = Select-String -InputObject $d -pattern "sc-status=200"
        Write-host "The Number of Errors in the File $($e.matches[$i]) : $(($f -split('Sc-status=200')).Count-1)"
}

I hoped that would help me understand what you’re actually trying to do but it didn’t actually. :wink: :smiley:
Could you please describe in simple terms what you actually want to do?

I understood that you try to count the occurences of a particular pattern in some files. Is that right? :thinking:

Yes, need a occurrences of particular pattern but all the content in one file. so I need explore a file to pull the details.

Actually I’ve a output of generated by a system which has multiple log reading with specific codes having a log file name at the top. I need to find out - How many code for specific files which is in the output file? so I’m going through no of lines under each header (filename) and collect it

Sorry, I did not understand you last post.

If it is what I assume it is and I assume we have file names in the format you posted in your initial post and you’re looking for patterns equal to those in your initial post then something like this should be enough:

Select-String -Path D:\sample\FileName*.log -Pattern 'Data\d' -AllMatches | 
    Group-Object -Property Path |
        Select-Object -Property Name, Count

This code looks for the pattern ‘Data\d’ - so “Data” followed by a single digit - inside files with names in the format "FileName.log"*.

Thanks but this actually gives the number of occurrences of the pattern but not line numbers between the patterns.

Thanks for the post

I did not get until now that you’re looking for this information, sorry, I’m not a native English speaker. :wink: That implies that you always have a maximum of 2 occurences of the pattern per file, right?

If that’s the case this could work:

Select-String -Path D:\sample\empty\FileName*.log -Pattern 'Data\d' -AllMatches | 
    Group-Object -Property Path |
        ForEach-Object {
            [PSCustomObject]@{
                Path = $_.group[0].Path
                LinesBetweenMatches = $_.group[1].LineNumber - $_.group[0].LineNumber
            }
        }

Thanks. This is something a start for me. I’ll need to work on a loop on the group and get lines between the groups

Thanks

If only I would understand what you mean. The groups represent the different files in this case. You know that, don’t you? I thought you need the lines between the occurences of the pattern per file!?

Simply to explain
That implies that you always have a maximum of 2 occurrences of the pattern per file, right?

  • not just 2 occurrences, multiple occurrences :slight_smile:

All the text is in 1 file only, example log.txt contains the following text
Pattern 1
Data1
Data2
Data3
Pattern 2
Data4
Data5
Data6
Data7
Pattern 3
Data8
Data9
Data10

it goes like several lines

output I expect
Pattern 1 has 3 lines
Pattern 2 has 4 lines
Pattern 3 has 3 lines

Your grouping concept is correct. Actually Pattern = somefilename.log not a file itself

‘’’
LinesBetweenMatches = _.group[1].LineNumber - _.group[0].LineNumber

‘’’

The above you showed the difference between only 2 but I need it for multiple times of patterns (again same file)

The following is almost working but the last number giving a weird number:-(

$b = Select-String -Path .\logger.txt -Pattern 'Data\d -AllMatches | 
    Group-Object -Property Path

 $c =   for ($i=0; $i -lt $b.Count; $i++) {
            
            [PSCustomObject]@{
                Path = $b.group[$i].line
                NoOfErrors = $b.group[$i+1].LineNumber - $b.group[$i].LineNumber -1
                
            }
        }
        
 $c | Format-List 
  

Thank you so much Olaf. My last post code worked now with some conditional loops. It’s a multiple occurrence of patterns and need to find out the lines underneath them

It works well now :+1:

Great. I’m glad it helped. You may post the final version of the code or you could correct / complete the last code you posted. That could help others in the future looking for the same or a similar issue.

Thanks in advance.

Please check below my Final version. Once again thanks for your assistance :slight_smile:

$filePath = Get-Content .\logger.txt
$filePattern = Select-String -Path .\logger.txt '(?:[\w]\:)(\\[a-z_A-Z\-\s0-9\.]+)+\.log' -AllMatches | 
    Group-Object -Property Path

# First we need to check for last Pattern which doesn't have next pattern
$lineCount = if($filePattern) {

           	[PSCustomObject]@{
               		"File Name" = $filePattern.group[-1].line
	                "No of Errors" = ($filePath.Length) - $filePattern.group[-1].LineNumber
                                    
            	} # This gives output for last pattern, otherwise we get a weird number as there may be loop issue

            }
# For the rest of Pattern First to Last-1
	    
         	 for ($i=0; $i -lt $filePattern.Count-1; $i++) {
            
                	[PSCustomObject]@{
		                "File Name" = $filePattern.group[$i].line
                		"No of Errors" = $filePattern.group[$i+1].LineNumber - $filePattern.group[$i].LineNumber -1
            		}
       	  }
    
$lineCount

I’ve been following along today and wishing I was at my computer. This was fun, here’s what I came up with.

My thought process included what if there are extra blank lines between the matches that shouldn’t be counted as an error. With that in mind, I updated the original sample data some.

$file = New-TemporaryFile

@'
Filename1.log
Data1
Data2
Data3
Data4
Filename2.log
Data5
Data6

Data7
Filename3.log

Data8
Data9
Data10
Data11
Data12
'@|Set-Content $file -Encoding UTF8

$alllines = Get-Content -Path $file

$lines = Select-String -Path $file -Pattern '.*\.log' -AllMatches

for($i = 0; $i -lt $lines.count; $i++){

    $start = $lines[$i].linenumber

    if($lines[($i + 1)]){
        $end = $lines[($i + 1)].LineNumber - 2
    }
    else{
        $end = $alllines.Count
    }

    $errors = $alllines[$start..$end] | Where-Object{$_}

    [PSCustomObject]@{
        "File Name"    = $lines[$i].Line
        "No of Errors" = $errors.Count
        Errors         = $errors -join ', '
    }
}

Output

File Name     No of Errors Errors                              
---------     ------------ ------                              
Filename1.log            4 Data1, Data2, Data3, Data4          
Filename2.log            3 Data5, Data6, Data7                 
Filename3.log            5 Data8, Data9, Data10, Data11, Data12

I also added the “error” lines themselves in addition to the count.

2 Likes