Quickly retrieving data from Office 365 mailboxes 

Often we need to perform searches over many mailboxes to find just those few which match certain criteria.

One that I recently had was to find users which have a forward on their mailbox (set on the mailbox attribute) in Office 365.

One option I had was I could run:

get-mailbox -resultsize unlimited | where ($_.ForwardingSmtpAddress -like '*')

This will work.  However, if you’ve got a lot of users (e.g. over 100,000 users) it can be very slow.  But why?

As Office 365 is a shared platform, Microsoft throttle PowerShell commands.  They throttle PowerShell in a number of ways.  In this instance, by the amount of data that is sent back to your local PowerShell session.

But if I’ve only got 5 matching users out of thousands, surely that can’t be much data.. why is it throttled?

This is where it becomes important to know where the work for the PowerShell command is being done and when the data is being transferred.  As you are connecting to Office 365, some parts of the PowerShell command execute on remote server (Office 365 in our case), some is executed locally where you lauched PowerShell from and they pass data between them.

In the above example, the get-mailbox -resultsize unlimited retrieves that data from every mailbox.  This is done on Office 365 remote server and then it is all sent back to your local PowerShell session.  Once it arrives at your local PowerShell, it will execute the where ($_.ForwardingSmtpAddress -like '*') on the data.  The data is being transferred between the remote Office 365 and local server contains all the mailbox data, and of course due to the size of the data it gets throttled and slows down the command, taking longer to get the results.

We can speed this up!

The where executes locally, so we really want this to be executed on the remote server, decreasing the amount of data transferred back to your PowerShell session.  So how do we do that?  We can filter the results of the get-mailbox command using the -filter parameter and the command will only return what matches the filter and it performs it all on the remote server.

So we could change this example

get-mailbox -resultsize unlimited | where ($_.ForwardingSmtpAddress -like '*')

to

get-mailbox -resultsize unlimited -filter "ForwardingSmtpAddress -like '*'"

Prove it..

Unfortunately due to way these commands work, we can’t use -resultsize parameter to prove that these commands are faster. If we did use the -resultssize parameter e.g. -resultsize 100, the first example the command would only search the first 100 mailboxes while the updated version above would search mailboxes and return only first 100 forward results.

So in order to show the results, I’ll run this on a tenancy which contains over 300,000 mailboxes.  The original PowerShell command took over 2.5 hours while the new command took just over 10 minutes.

PS_Forwarding

As you can see, the larger the number of mailboxes being searched the more benefit using -filter will give you.

If you have a small amount of users in your tenancy, then you probably won’t see very much difference in using the -filter parameter, however if you’re company expands or your scripts are used on larger Office 365 tenancies, using -filter may help your scripts get quicker results.

AD Connect Filtering

If you’ve installed Azure AD Connect to sync objects from your local Active Directory to Office 365, you may have seen that you can use filtering to stop objects being sync.  Yeah Yeah I hear you say, you can filter objects by the OU they’re in.. Yes you can, but you can also filter by attributes on objects, which as you can imagine can be very handy.

Check out the below link for some Microsoft doco on how to do this.

https://docs.microsoft.com/en-us/azure/active-directory/connect/active-directory-aadconnectsync-configure-filtering#attribute-based-filtering

The filtering examples in the above link can be used to filter in/out users from being sync’d to Office 365 Azure AD from your local AD. It uses the Extension Attributes (on the user objects) to perform the filtering. Once you AD Connect has been setup to do this filtering, the below PowerShell examples can be used to populate the relevant ExtensionAttribute with values which will be filtered to stop users being synced to Office 365.

The Microsoft Example uses ExtensionAttribute15 being set to ‘NoSync’. In my examples, I’ve used ExtensionAttribute8 and setting to ‘DoNotSync’ as that’s how was the ExtensionAttribute and value selected in our Azure AD Connect.

To find any users within your AD which have ExtensionAttribute8 set to ‘DoNotSync’

Import-Module ActiveDirectory
$users = Get-ADUser -Filter "ExtensionAttribute8 -eq 'DoNotSync'" -properties ExtensionAttribute8

 

To find any users within your AD which have ExtensionAttribute8 set to 'DoNotSync' within a specific OU

$users = Get-ADUser -Filter "ExtensionAttribute8 -eq 'DoNotSync'" -properties ExtensionAttribute8 -SearchBase "OU=UserAccounts,DC=FABRIKAM,DC=COM"

 

Or a specific user

get-aduser -Identity username -Properties ExtensionAttribute8

To Add 'DoNotSync' in ExtensionAttribute8 to a specific user

set-aduser -Identity username -Replace @{ExtensionAttribute8='DoNotSync'}

 

Office 365 License assignment using AD Groups

If you’ve searched for Office 365 licensing using AD groups, you usually find PowerShell using AD groups to provision Office 365 licenses, but that’s not what this is.  It’s also not using Azure Automation to run PowerShell scripts within Azure to perform licensing.  This is Azure AD functionality (in Preview) to perform Office 365 licensing based on AD group membership.

It’s been announced only recently, and if you’re currently performing licensing of users in Office 365 with a PowerShell scripts, you should definitely have a look at this functionality.

Azure AD Group Licensing using E3

Of course, this functionality is inside Azure Active Directory (currently in preview in the new Azure Portal). In order to assign licences via groups, you need to have an Azure AD licence. The Microsoft documentation states only an Azure AD Basic licence is required.  You can check that out information here.

If you just want to have a look at the group licensing functionality and don’t want to buy a license, you can sign up for a trial for the Basic and the Premium licences inside your Office 365 tenancy for no cost.  You could use a trial for either of these to give this a go.

Azure AD Basic License ScreenshotAzure AD Premium P1 License Screenshot

If you don’t have Azure, you can sign up with a free trial. There is currently an offer to sign up and get $200 credit for 30 days.  Check it out here.

If you’re signing up for the Azure with a different account or have signed up to Office 365 and azure with different credentials, check out here to find out a how to connect them together. To assign the licences in Office 365 through Azure AD Preview, you need to be able to access the Azure AD that your Office 365 tenancy uses from within the Azure portal.

Once you’ve made it into Azure, it’s actually pretty straight forward. Microsoft have put a guide together on how to do this.  I didn’t think it was easy to find, so here is a link to it – link.

If you’re interested in the full process to apply the functionality to groups, I took a screenshot for the entire process in the Azure AD Preview GUI.

Azure License Application Screenshot

Of course, groups are used to assign the licences. It requires a security group and works with groups synced from an on premise directory and Azure Cloud only groups, both direct and dynamic groups. However, you should be aware that currently nested groups are not supported. If you want to use them regardless, be aware that licenses will only be applied to those users with direct membership to the group and not those in the nested groups.

I was pleasantly surprised as the licences re-evaluated quite quickly when the group was updated, both adding and removing licences according the group membership.

 

One of things you should be careful of with the groups is changes to the license configuration. Every time this is changed, it removes licenses from every user that is in that group and after that it reapplies the licenses to he new configuration.  If you’ve got a lot of users in the group, it could potentially cause an outage to users and risk data loss.

To work around this, I would lean towards two methods.

1. Create a new group with the wanted license configurations.  Add all the users from the old group into the new one.  Make sure the new licenses apply and remove the users from the old group.

2. Create multiple groups, with each group having a specific license applies to it. (E.g. A group which only applies the E3 Exchange Online part of the license).  For each part of the license you want to assign, create a new group – although you’ll have to watch out for license dependencies. (E.g. OneDrive requires SharePoint.)

Migration is quite interesting, as you can have the same license on the same user both assigned directly from PowerShell and inherited from groups. Both will exist and you can get conflicts between them.  The good thing is that if there is a conflict you can easily get results of the license application in Azure to find any conflicts.

The recommended migration path is to leave your PowerShell script in place while group licensing is configured to provide exactly the same licenses for your users that the PowerShell script applies.  Once the same licences have been applied by group, disable the licensing PowerShell Script.  Then you need to start to remove the licenses assigned with PowerShell. Of course this is recommended to be performed in batches.

If you’re used to checking if users are licensed, both the Office 365 admin portal and the existing PowerShell cmdlets cannot see the group license application – they can only see if there is a license applied.  There are some PowerShell scripts available to provide information on the number of licences assigned and what type, but it’s still limited.  You can check those out here.  Once Azure AD is out of preview, hopefully we’ll get visibility in Office 365 and PowerShell.