Being new to PS, you really need to get up to speed first before doing major stuff like this, as well as how to deal with PS and O365/Azure, which are separate module installs and use.
Vis the Microsoft Virtual Academy or search YouTube for beginning PS training, as well as the many resources on Amazon and the no-cost eBooks on this site and others.
As for your use case. There are modules/scripts from the MS PowerShellGallery.com specifically for this sort of effort.
Manage your O365 Licenses
Helps automate common license management tasks by providing a simpler interface on existing PowerShell Cmdlets. Supports Adding SKUs, Replacing SKUs, Disabling Plans, and Updating Existing SKU Plans. Also generates a comprehensive license report.
Office 365 License Reporting and Management Tool -Assign Remove Licenses in Bulk
This is a simple windows forms GUI based utility to manage Office 365 license. You can assign/unassign/update user(s) license. You can also manage licenses for multiple office365 tenants.
Office 365: Manage User License Lifecycle with PowerShell
Set-O365UserLicense A PowerShell function for managing licenses for Office 365 throughout their lifecycle. You can use this to automate licensing new users as well as maintaining the correct licenses for users as they change roles or as your licensing needs change.