Phase out Legacy Authentication - Gain insights
This is part three of the six-part series on “Phase out Legacy Authentication”.
- Enable Modern Authentication
- Create prerequisites
- Gain insights
- The first 90%
- The next 9%
In the first part, Modern Authentication was enabled and in the second part, the prerequisites were created to create detailed reports and to disable Legacy Authentication for individual users.
Conditional Access policy logic
To better understand how it is possible to block individual users, let’s take a look at the Conditional Access Policies created and the login associated with them.
Phase 1 - Implicit Allow
In phase 1, we only want to block users who do not use Legacy Authentication (LA). This way, we prevent users from possibly using an app that does not support Modern Authentication (MA) during our journey.
- Bob and Alice both use Exchange Online.
- Bob uses the native iOS mail app with Exchange ActiveSync Client (LA) and Microsoft Apps for business (MA).
- Alice uses the Outlook mobile app (MA) and Microsoft Apps for business (MA).
Since Alice only uses Modern Authentication, she can be included in the group “CAPolicy-Include-Block-Legacy-Authentication”. If she now tries to access Exchange with a legacy auth client, this attempt will be blocked by the conditional access policy “Temporary Policy: Block legacy authentication Rollout”.
Phase 2 - Explicit Allow
In Phase 2, Legacy Authentication (LA) is disabled for all users and only explicitly defined users can continue to use it.
- Janine and Kevin both use Exchange Online
- Janine wants to set up a mail app with Exchange ActiveSync (LA) on her new Android smartphone.
- Kevin has an existing Android device and uses a mail app with Exchange ActiveSync (LA)
Kevin was added to the “CAPolicy-Exclude-Block-Legacy-Authentication” group and can therefore continue to use his mail app.
Janine is blocked from logging in and has to switch to a mail app that supports Modern Authentication.
Several tools are available to identify users for the first phase. Used together, these tools offer incredible flexibility and detailed insights into the use of Microsoft 365.
The Azure AD Workbooks provided by Microsoft are a great place to start. Each workbook is designed for a specific analysis, so we will use some of the workbooks over the next while.
The workbooks are based on the Azure Monitor Workbooks service. Using this, data can be prepared in graphical form very easily and it is thus made easier for the user to extract information from the data volumes.
A big advantage of the workbooks is that they can be used interactively and data can be easily reduced using filters and drill-down options.
The source for this data is the Log Analytics Workspace created in the last chapter and the SignIn data collected there.
Each workbook offers several filters that can be used to limit the displayed data. Always available is the time period, the other parameters vary depending on the selected workbook.
Sign-ins using Legacy Auth
The workbook “Sign-ins using Legacy Auth” provides insight into the use of legacy authentication and the apps and protocols affected by it.
Conditional Access Insights and Reporting
This workbook can be used to closely monitor the impact of a conditional access policy. For example, the data collected for the conditional access policy “Common Policy: Block legacy authentication” can be used to estimate what impact a shutdown will have on users.
This workbook also provides a more detailed insight into the distribution of the end devices used, the apps, sign-in risk and the location of the users, which is determined on the basis of the IP address.
In this case, filtering on the correct conditional access policy and time period is very important to get meaningful results.
The workbooks just discussed are a great graphical and interactive tool and are, as mentioned, based on the recorded SignIn data in the Log Analytics Workspace. However, you can also conveniently query the actual raw data yourself using Kusto queries.
The existing workbooks offer a good start. Via the blue Log Analytics icon the underlying query opens and can be further modified.
Under certain circumstances, the time range specified in the query is not taken into account. Here it is important to select the “Set in Query” option as Time range.
Identify legacy protocols in use
The legacy protocols used are the starting point for deactivation. Depending on which service still uses legacy authentication, different countermeasures must be taken.
Exchange is one of the bigger troublemakers in this regard, as Mail is a very old service with many old protocols. Therefore, it is not surprising if a large part of the protocols are related to Exchange.
SigninLogs // set TimeSpan to 7 days | where TimeGenerated > ago(7d) // Only query successful logins | extend errorCode = toint(Status.errorCode) | where errorCode == 0 // Check if ClientAppUsed is considered Legacy Auth - Empty is not considered Legacy Auth | extend ClientAppUsed = iff(isempty(ClientAppUsed) == true, "Unknown", ClientAppUsed) | extend isLegacyAuth = case(ClientAppUsed contains "Browser", "No", ClientAppUsed contains "Mobile Apps and Desktop clients", "No", ClientAppUsed contains "Exchange ActiveSync", "Yes", ClientAppUsed contains "Unknown", "Unknown", "Yes") | where isLegacyAuth == "Yes" | summarize Count=count() by ClientAppUsed, AppDisplayName
The query uses all data in the SigninLogs table created in the last 7 days as a data basis.
In the next step, only the logins that have successfuly logged into the system are analyzed. Whether a botfarm in Buxtehude could not log in successfuly is irrelevant to us in this context.
Now the application that was used for the login is examined. If there is no information about it, the value for
ClientAppUsed is set to “Unknown”.
Based on the client app, it is now defined if the signin is a legacy signin. Browsers, Microsoft mobile apps and desktop apps (>= 2013) are considered Modern Auth, unknown apps are marked as ‘Unknown’ and the rest is Legacy Authentication.
Only LA SignIns are returned and aggregated based on the client app and the online service used.
Identify applications used
To better judge which applications are using these outdated protocols, the
UserAgent can be evaluated.
SigninLogs // set TimeSpan to 7 days | where TimeGenerated > ago(7d) // Only query successful logins | extend errorCode = toint(Status.errorCode) | where errorCode == 0 // Check if ClientAppUsed is considered Legacy Auth - Empty is not considered Legacy Auth | extend ClientAppUsed = iff(isempty(ClientAppUsed) == true, "Unknown", ClientAppUsed) | extend isLegacyAuth = case(ClientAppUsed contains "Browser", "No", ClientAppUsed contains "Mobile Apps and Desktop clients", "No", ClientAppUsed contains "Exchange ActiveSync", "Yes", ClientAppUsed contains "Unknown", "Unknown", "Yes") | where isLegacyAuth == "Yes" | summarize Count=count() by UserAgent, AppDisplayName | sort by Count
The first 10 lines of the query are identical to the first query.
The decisive factor is the grouping based on the UserAgent and the online service used in line 11.. Line 12 sorts the result in descending order by the number of logins.
A UserAgent that starts with
Microsoft Office/14.0 indicates Office 2010 installations that are still active.
End of support for Office 2010 was on 13.10.2020.
Identify users without legacy authentication signins
So that in the next article Legacy Authentication can really be disabled for (hopefully 😉) 90% of all users, we need the information which users have not logged in using Legacy Authentication in the last 7 days.
AADNonInteractiveUserSignInLogsnicht berücksichtigt, which could lead to problems. The new query includes this data as well.
The first 10 lines are almost identical to the previous queries, but in line 1 the code is connected with the command let named
distinctSigninLogs and can be used more easily in the subsequent code.
This makes the code leaner, since the same expression does not have to occur multiple times written out.
In line 10 the distinct operator is used to get each entry only once. So for each
UserPrincipalName there is a maximum of three entries.
Lines 11 - 21 queries the same data from schema
AADNonInteractiveUserSignInLogs to include non interactive sign-ins, such as service accounts, as well.
After that, in line 22 - 23 the result from
distinctAADNonInteractiveUserSignInLogs is merged together in
In line 24-26 only those usernames are stored as
LegacyAuthSigninLogs for which
isLegacyAuth equals the value “Yes”.
All this was just preparation for the actual query. This query uses all unique SignIns from the “table”
distinctSigninLogs and only returns user names that do not appear in the “table”
LegacyAuthSigninLogs. Therefore only users who have not used legacy authentication.
In the next article we will disable legacy authentication for the users we identified as not critical.