How to handle and process irregular logs with powershell?

How to handle and process irregular logs with powershell?

hi, guys,
my line manager has given me a raw irregular syslog file that contains the following logs like below:

1.May 13 14:33:00 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[54363]): Could not find uid associated with service: 0: Undefined error: 0 501

2.May 13 15:00:11 BBAOMACBOOKAIR2 Microsoft Teams Helper[54445]: [HockeySDK] -[BITCrashManager invokeDelayedProcessing]/874 [HockeySDK] WARNING: Another exception handler was added. If this invokes any kind exit() after processing the exception, which causes any subsequent error handler not to be invoked, these crashes will NOT be reported to HockeyApp!

3.May 13 15:00:01 BBAOMACBOOKAIR2 Teams[54434]: [HockeySDK] -[BITCrashManager invokeDelayedProcessing]/874 [HockeySDK] WARNING: Another exception handler was added. If this invokes any kind exit() after processing the exception, which causes any subsequent error handler not to be invoked, these crashes will NOT be reported to HockeyApp!

4.May 13 22:53:41 BBAOMACBOOKAIR2 Finder[908]: libcoreservices: __create_or_fix_relative_directory: 1002:
__dirhelper_create_relative_with_error: error Operation not permitted

my line manager asked me to write a script(shell and powershell) that could automatically analyse the raw irregular syslog file, filter out the logs with the keyword “error”,and make further process to the specified logs.

The required script must gather following information from each log with the keyword “error”, and then convert those information in to Json

    1.deviceName
2.processId
3.processName
4.error description
5.timeWindow (ex:0100-0200,0300-0400)
6.numberOfOccurrence for same error based on hours

I have done the job with Linux Shell. However, using powershell truely beyond my ability because I hava learned powershell for only a few days. So ,I wish some master could help me with that.

The following is part of the logs in the raw log file.

May 13 00:01:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12513]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:01:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12513]): Service exited with abnormal code: 78
May 13 00:02:12 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.xpc.launchd.domain.pid.mdmclient.12523): Failed to bootstrap path: path = /usr/libexec/mdmclient, error = 108: Invalid path
May 13 00:04:20 BBAOMACBOOKAIR2 syslogd[113]: ASL Sender Statistics
May 13 00:05:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12535]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:05:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12535]): Service exited with abnormal code: 78
May 13 00:09:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12536]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:09:58 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12536]): Service exited with abnormal code: 78
May 13 00:17:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12555]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:17:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12555]): Service exited with abnormal code: 78
May 13 00:17:59 BBAOMACBOOKAIR2 syslogd[113]: ASL Sender Statistics
May 13 00:19:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12556]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:19:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12556]): Service exited with abnormal code: 78
May 13 00:21:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12560]): Could not find uid associated with service: 0: Undefined error: 0 501
May 13 00:21:59 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.mdworker.bundles[12560]): Service exited with abnormal code: 78
May 13 00:22:18 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.xpc.launchd.domain.user.914945058): Service "com.apple.xpc.launchd.unmanaged.loginwindow.594" tried to register for endpoint "com.apple.tsm.uiserver" already registered by owner: com.apple.TextInputMenuAgent
May 13 00:22:49 --- last message repeated 1 time ---
May 13 00:23:50 BBAOMACBOOKAIR2 timed[158]: settimeofday({0x5ebacd96,0x52ddf}) == 0
May 13 00:28:05 BBAOMACBOOKAIR2 syslogd[113]: ASL Sender Statistics
May 13 00:28:07 BBAOMACBOOKAIR2 com.apple.xpc.launchd[1] (com.apple.ScreenSaver.Computer-Name[12564]): Service exited due to SIGKILL | sent by Computer Name[12564]
May 13 00:28:17 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:17 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after xpc connection was activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:18 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:18 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after xpc connection was activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:19 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:19 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing target queue hierarchy after xpc connection was activated; set a breakpoint on _dispatch_bug_deprecated to debug
May 13 00:28:20 BBAOMACBOOKAIR2 VTDecoderXPCService[960]: DEPRECATED USE in libdispatch client: Changing the target of a source after it has been activated; set a breakpoint on _dispatch_bug_deprecated to debug

NormanYue
Welcome to the forum. :wave:t4:

What kind of help do you need? What do you have so far? We are happy to help with code you wrote but we cannot deliver ready to use code or solutions on request. We expect you to make an own attempt to solve your problem at first.

To get you started you may (re-)read the help topics for the following cmdlets including the examples:

Just throwing this out there … you say you have done this with Linux/Bash? Why not just use Bash on Windows? Or commonly know as WSL (Windows Services for Linux). You may be able to use the bash script you already have. If you do need to use PS, it looks like your main trigger for parsing indifferent log entries could be the reference designator for each log type as in [1], [113], [158], [960] etc …

1 Like