Microsoft Entra Conditional Access: a safe rollout that does not lock out the administrators

A Conditional Access policy can stop a password-only sign-in, but it can also stop the people who would repair a bad policy. The safe rollout is therefore a controlled change: define the recovery path first, observe the policy in report-only mode, pilot it with representative users, and only then enforce it.

This guide is for a Microsoft Entra tenant using Conditional Access. It assumes a Conditional Access Administrator and an agreed change window. It does not assume that every workload, legacy client or service identity can be treated as an interactive user.

Define the recovery boundary before writing a policy

Create and document at least two cloud-only emergency-access accounts in the .onmicrosoft.com domain. They must not depend on on-premises synchronisation or federation. Microsoft recommends permanently active Global Administrator assignments for these accounts, not eligible assignments that may require an unavailable approver.

Put the accounts in a dedicated group such as EmergencyAccess. Exclude that group from Conditional Access policies that block or restrict sign-in. This is a narrowly defined exception, not a general administrator bypass. Report-only policies do not block sign-in, so they do not need the exclusion.

Use phishing-resistant credentials that do not share the same failure mode as normal administrator authentication. A FIDO2 security key stored in a separate secured location is a common choice. Do not attach the accounts to one employee's phone or day-to-day workstation.

Done criterion: two authorised administrators can identify the recovery accounts, their custody location and the audit owner without revealing credentials in a ticket or chat.

Map people, applications and non-human identities

Start with an inventory, not the All users checkbox. Identify:

• privileged administrators;
• standard employees;
• guests;
• service accounts still using interactive sign-in;
• applications using service principals or managed identities;
• break-glass accounts;
• pilot users on Windows, macOS, iOS and Android.

A user-targeted policy does not automatically govern service-principal sign-ins. Workload identities have separate Conditional Access capability and must be assessed separately. Treat a script account that signs in interactively as a migration problem: replace it with a managed identity or service principal where possible, then restrict it deliberately.

Legacy protocols require special care. If a workload still depends on an older client protocol, a blanket block may look like an authentication outage. Find these dependencies from sign-in logs before enforcement; do not discover them from an executive's missing mail on Monday morning.

Build a policy that has one job

Create the narrow report-only pilot

1. Go to Microsoft Entra admin center → Entra ID → Conditional Access → Policies → New policy. In Name, enter the approved name, for example CA-Pilot-Admins-Require-Phishing-Resistant-MFA. The policy editor should show sections for Assignments, Access controls, Session, and Enable policy. Confirm the name is a pilot name, not a production-wide name. If the wrong policy editor opens, select Cancel; no policy exists until Create is selected.
2. Open Assignments → Users or workload identities. Under Include, choose Select users and groups, then select the approved pilot group and Select. Under Exclude, choose Users and groups, select the dedicated emergency-access group (or the two approved emergency accounts), then Select and Done. The summary should show the pilot included and only the documented emergency objects excluded. Validate object names and IDs against the change record, not display-name similarity. To reverse an unsaved selection, reopen the assignment, remove the object with its X, select Done, or use Cancel for the entire policy.
3. Open Target resources → Resources (formerly cloud apps). Under Include, choose Select resources, select the approved resource, then Select. Do not choose All resources for this first pilot. The editor summary must name the chosen resource. Reopen the blade and remove that resource before saving if it is wrong.
4. Open Access controls → Grant. Choose Grant access, tick Require authentication strength, select the approved strength from the list, then select Select. Confirm the grant summary names the strength. Do not add device, terms-of-use, or session controls in this first test unless they are explicitly in the approved scenario; mixing controls makes the result ambiguous. Reopen Grant, clear the control, and select Select to undo it before creation.
5. At the bottom, set Enable policy to Report-only, then select Create. Expected result: the Policies list contains the new policy with state Report-only; it does not enforce a block or prompt. Validate by opening the policy and checking the scope, exclusions, grant control and state. If the review discovers an error, open the policy, correct the bounded field, set Report-only, and select Save; do not promote it to On to “test” it.

Use a name that describes scope and outcome, for example CA-Pilot-Admins-Require-Phishing-Resistant-MFA. A first policy should be deliberately narrow:

1. Users: a pilot group of administrative users; exclude EmergencyAccess.
2. Resources: the management surfaces or cloud apps needed for the pilot.
3. Conditions: add only conditions whose effect you can explain and test.
4. Grant: require the chosen authentication strength. Avoid combining controls until the pilot proves the simpler policy.
5. Session: leave session controls out of the first test unless session behaviour is the object of the test.
6. State: set Report-only.

Do not use an exclusion for a whole department merely because one legacy application is awkward. Record the exact application, owner, protocol, expiry date and compensating control. An exception without an owner is a permanent policy hole wearing a temporary label.

Read report-only results correctly

The What If tool in the Entra admin centre is useful before enforcement when its inputs match a real test persona. Choose the specific user, cloud application, device platform and network context that the pilot will actually use. It should show the intended policy as applying, and should show the emergency account as excluded when evaluated separately. Treat this as a policy-path simulation. It does not replace a real sign-in because it cannot validate a FIDO key, browser session, device registration, application token flow or alert delivery.

A suspicious What If result is one that includes a policy nobody expected, fails to show the new policy for the intended persona, or relies on a guessed device or location. Correct the test input or policy scope before progressing. Do not tune the policy to satisfy a simulation whose conditions are not representative. The portal is a map, not the road.

Review report-only evidence

For a real pilot sign-in, go to Microsoft Entra admin center → Entra ID → Monitoring & health → Sign-in logs. Filter by the pilot user and the test time, select the sign-in row, then select the Conditional Access tab. Expected result: the policy is listed with a report-only result such as report-only success, failure, not applied, or not enabled. Capture the correlation ID, resource, client app, device and policy result. A result of not applied needs scope investigation; it is not proof that the control works. The least-privileged reader needs access to both sign-in logs and policies (Microsoft documents Security Reader as the least built-in role for both); a missing tab can therefore be a permissions symptom. Close the details pane or clear filters to reverse the investigation; this action changes no tenant setting.

Before enforcement, use Entra ID → Conditional Access → Policies → [pilot policy] → Policy impact when available, and review the sign-in samples. Policy impact is a decision aid, not a replacement for an actual sign-in. If the policy page instead exposes What If, choose the named pilot user, the same target resource and representative device/network values, then run the simulation. Save the prediction with the real log evidence; clear or change inputs to rerun without modifying policy state.

Report-only records the policy's evaluated result but does not enforce it. Review the Conditional Access and sign-in logs for the pilot users after realistic tests: a managed device, an unmanaged device, a supported mobile platform, and an expected failure.

A successful report-only evaluation proves only that Entra could evaluate the rule for that event. It does not prove that the intended authentication method is registered, that a critical application works after enforcement, or that every client path has been observed.

Use the What If tool before changing scope. It is useful for hypothesis testing, but it is not a substitute for a real sign-in. Test a named user, target resource, device platform and network context that match the proposed change.

Pilot enforcement with rollback ready

Controlled enforcement and rollback

Only after documented report-only evidence, open Entra ID → Conditional Access → Policies → [pilot policy], change Enable policy from Report-only to On, and select Save during the agreed window. Expected result: the list state changes to On and subsequent matching sign-ins evaluate the grant. Immediately perform the same pilot sign-in and inspect its Conditional Access details. Validation is a matching enforced result plus successful access via the intended authentication strength; a green portal state alone proves nothing about the client path.

For a confirmed outage, use a still-working authorised administrator or the documented emergency path. Go to Entra ID → Conditional Access → Policies → [exact policy], set Enable policy to Report-only (preferred evidence-preserving rollback) or Off if the approved runbook requires it, then select Save. Re-run the affected sign-in and verify in Sign-in logs → [event] → Conditional Access that the enforced policy no longer blocks it. Do not delete the policy during recovery: deletion removes configuration context and makes later diagnosis harder. Record the time, correlation ID and changed state, then repair scope or prerequisites and repeat report-only testing.

Before changing the policy state to On, define:

• the pilot duration and named pilot owner;
• the sign-in events that count as success;
• the business applications and admin portals to test;
• the support contact and escalation path;
• a rollback operator who can disable the policy;
• the rollback trigger, such as a confirmed loss of access to a required service.

During the pilot, use normal administrators for every test. Do not use an emergency account merely to make the pilot appear healthy. If a pilot user is blocked unexpectedly, capture the correlation ID, policy name, resource and authentication method; then disable or narrow the policy according to the pre-agreed rollback rule.

A simple Microsoft Graph PowerShell inventory can help identify the accounts that need ownership review. Run it from an approved admin workstation with permissions appropriate to read directory roles; it is read-only.

Connect-MgGraph -Scopes "RoleManagement.Read.Directory","User.Read.All"
Get-MgDirectoryRole | ForEach-Object {
  Get-MgDirectoryRoleMember -DirectoryRoleId $_.Id |
    Select-Object @{n="Role";e={$_.AdditionalProperties.displayName}},Id
}

A successful result is a list that can be matched to your privileged-account register. An empty or incomplete list may mean insufficient Graph permissions or that the role has not been activated in the directory; it does not prove that no privileged accounts exist.

Expand by risk, not by calendar

After a stable admin pilot, expand to the next group with the same evidence: sign-in samples, application owner confirmation and rollback readiness. Common next controls include blocking legacy authentication, requiring MFA for privileged users and requiring compliant devices for selected management surfaces. Each change should be a separate policy or a separately testable revision.

Authentication strength and device-compliance policies can interact with registration, guests and external authentication methods. Check licensing and feature prerequisites before promising a control. P2 capabilities such as risk-based policies need their own design; do not quietly present them as available in every tenant.

Validate the emergency path without normalising its use

At least every 90 days, carry out an authorised emergency-access test. Use a designated secure workstation, verify that the intended account can sign in, confirm its policy exclusion in the sign-in record, check alerts, and document the outcome. Sign out immediately. A break-glass account that is never tested is a paper control.

Evidence-led policy design

A Conditional Access rollout becomes dangerous when its design is inferred from a policy blade after the fact. Keep a short change record alongside the policy: the named owner, intended population, protected resource, controls being required, exclusions, pilot group, start and review date, and a reversal decision. This is not paperwork for its own sake. It lets a second administrator distinguish a deliberate exception from an old experiment that simply survived long enough to look official.

Build the first policy around one outcome. For example, require a phishing-resistant authentication strength for a small group of active administrators when they reach the Entra admin centre. Do not also add every location, device-filter and session setting that looks sensible. Each additional condition changes the set of sign-ins being evaluated and makes a failed test ambiguous. A narrow policy can be widened later; a wide policy usually gets narrowed only after it has found an inconvenient meeting.

Before testing, use What If with a real pilot account and the actual cloud application. Record the predicted result, then perform the real sign-in. The useful comparison is not “the portal opened”; it is whether the sign-in log shows the policy as report-only success, report-only failure, not applied, or not enabled. A not-applied result may be correct scope, a missing group membership, or a condition that was not present. It is evidence to investigate, not permission to switch the policy on.

A controlled pilot with readable evidence

Choose pilot users who represent the paths that matter: an administrator on a managed Windows device, one on a managed mobile device if mobile administration is allowed, and an administrator outside the office network if the policy considers location. Tell them what is changing, how to obtain support, and what must not be improvised during the window. The pilot group should not be a collection of whoever happened to answer a message first.

For each test, capture the UTC time, user, application, client type, device state, network context, authentication method, correlation ID and policy outcome. The sign-in record is the primary evidence; screenshots without the underlying event are convenient but fragile. A successful test should demonstrate the intended method was actually required. If a familiar browser session continues without a prompt, test again in a private session or after a controlled token refresh. Existing tokens and session controls can make an unchanged policy look more successful than it is.

A practical decision point follows each test. If report-only says the policy would succeed and the intended method works, retain the event and continue to the next representative path. If it would fail, identify whether the cause is missing registration, unsupported client behaviour, device posture, licensing, or an incorrect assumption about the resource. Fix the named cause or exclude only the documented workload temporarily. Do not convert an unexplained failure into a broad bypass; that is merely a future incident with better typography.

Moving from observation to enforcement

Set enforcement during an agreed window and keep the rollback operator separate from the person changing the policy where staffing permits. The operator needs a normal administrative route and must know the policy name, tenant, change window and stop rule. The stop rule should be concrete: two independent pilot users lose a required administrative path; a documented critical application is blocked; or the emergency account test fails. “Somebody is unhappy” is a signal to investigate, not a usable control boundary.

After enabling, repeat the same tests rather than inventing a new success condition. Review sign-in logs immediately and again after enough normal activity has occurred to expose less common clients. Check that excluded emergency accounts remain excluded only where necessary, and that ordinary administrators did not become accidental exceptions. If the policy uses authentication strengths, verify that the registration experience and allowed methods match the strength. A policy can be syntactically correct while requiring a credential that its pilot population cannot enrol.

Expand by risk and evidence. Separate policies for legacy authentication, privileged MFA, selected device compliance and risky sign-ins make review and rollback clearer. Workload identities are not ordinary users; inventory service principals and managed identities before applying workload identity controls. A human service account using an interactive protocol is a migration debt with a login screen, not proof that all policies must accommodate it.

Failure paths, containment and rollback

Consider a common failure: a device-compliance requirement is added to the same pilot policy as phishing-resistant MFA. A pilot administrator on a freshly rebuilt laptop is blocked. The sign-in log shows both controls, but only one is the actual failure. Disable or revert the last bounded change, preserve the correlation ID and export the relevant sign-in evidence. Then retest authentication strength and compliance independently. Changing three controls at once produces a very efficient mystery.

Another failure is the account that signs in but cannot do useful recovery work because its Global Administrator role is eligible rather than active, or because the credential depends on the same phone estate that has failed. Contain this by using cloud-only, permanently assigned emergency accounts with separately held credentials, and by testing them on a designated workstation. If an emergency account appears in an unexpected Conditional Access result, stop routine changes, confirm the group exclusion and restore the documented policy state before further testing.

Rollback should be a normal, rehearsed administrative action: disable the specific policy or return it to report-only, confirm that affected sign-ins recover, record the time and reason, and notify the named owner. Do not delete the policy during the incident; deletion destroys the configuration and evidence you need to understand it. Once service is stable, open a bounded remediation: fix the scope, prerequisite or authentication registration, then repeat report-only evaluation before another pilot.

Operating the policy after the pilot

Conditional Access has a lifecycle after the first successful pilot. Assign a policy owner and a separate business owner for every protected application. The security owner decides whether the control is correctly expressed; the application owner confirms that supported client paths still exist. Review the policy at a defined interval and after material changes such as a new federation provider, a device-management migration, a licensing change, an acquisition, or a new administrator role. The review should begin with the policy export and recent sign-in evidence, not with a reassuring green status icon. To carry this out, open Microsoft Entra admin center → Entra ID → Monitoring & health → Sign-in logs, use Add filters with the exact UPN and time window, open the event and select Conditional Access; the expected result is a correlation ID and policy outcome to compare with the test record. Clear filters or closing the details pane safely reverts only the view and changes nothing.

Use naming that survives staff turnover. A name such as CA-PRIV-AdminCentre-PhishResistant-Pilot communicates target, resource, control and state. The description should contain the change reference, owner, review date and exception expiry. This lets an incident responder answer a basic question quickly: is the observed block intended, and who may change it? A policy called “New CA policy final v2” is a small administrative disaster preserved in Unicode.

Scope, licensing and identity types

Confirm prerequisites before making a control mandatory. Authentication strengths, device compliance, risk conditions and workload identity policies do not all have the same licensing or implementation requirements. Document what the tenant is licensed to use and verify the setting exists in the production tenant, rather than treating a documentation screenshot as a feature entitlement. A failed creation attempt is not a rollout plan. To carry this out, open Microsoft Entra admin center → Entra ID → Conditional Access → Policies → [exact policy name] and compare the relevant Assignments, Grant or Enable policy section with the change record; the expected result is the documented scope and state. Remove an incorrect unsaved selection with X, then choose Done or Cancel; the production rollback is Report-only followed by Save, then sign-in-log validation.

Keep user, workload and service scenarios distinct. A managed identity does not perform an interactive MFA ceremony; a service principal is governed through workload identity capability; a legacy script account that uses a human password needs an owner and a migration plan. For each non-human dependency, record the application, credential type, owner, authentication protocol, last observed use and intended replacement. This inventory is also the evidence needed to challenge an exception when its expiry arrives.

Guest access needs its own representative test. The guest’s home tenant, cross-tenant settings, device signals and authentication methods may differ from those of a member account. Test the exact resource and collaboration path that is in scope. A successful employee test does not demonstrate that an external user can meet the same condition, nor does a guest failure prove that the policy is wrong. It proves that the guest path needs analysis. To carry this out, open Microsoft Entra admin center → Entra ID → Users → All users, search for the approved UPN, select only the matching Object ID and check Overview; if the role must be confirmed, continue to Entra ID → Roles & admins → Global Administrator → Assignments. The expected result is an object matching the register and an active assignment; clearing Search or closing the pane safely reverts the view, but stop the test if they do not match.

Reading sign-in evidence without guessing

Start from the sign-in event and work outward. Record the user principal, application, resource tenant, client app, IP and location, device identifier, authentication requirement, Conditional Access status and correlation ID. Compare the event to the planned scenario. If the policy is not applied, inspect user and group scope, resource assignment, client type and conditions. If it is report-only failure, read the result detail before changing the policy. The log is a witness, not a fortune cookie.

Token lifetime complicates testing. A user can appear unaffected because an existing session has not re-evaluated the policy. Use a private session, a freshly acquired token, or a controlled sign-out where appropriate, then note which path was used. Conversely, repeated prompts may be caused by session controls or application behaviour rather than an authentication-strength failure. Separate the claim “the policy evaluated” from “the user experienced a prompt” and from “the application remained usable”.

Location is another frequent source of false certainty. Named locations depend on correctly maintained public egress addresses and may not represent a roaming user, split-tunnel VPN, mobile carrier or IPv6 path. Test from the intended network and compare the actual sign-in location with the configured condition. Do not widen a trusted location because a single test reports an unexpected country; first identify the egress path. Geography has embarrassed more access policies than malice has. To carry this out, open Microsoft Entra admin center → Entra ID → Monitoring & health → Sign-in logs, use Add filters with the exact UPN and time window, open the event and select Conditional Access; the expected result is a correlation ID and policy outcome to compare with the test record. Clear filters or closing the details pane safely reverts only the view and changes nothing.

Change control that supports recovery

Before every material edit, capture the previous policy configuration and the reason for the new revision. Use a short maintenance window, an observer where possible, and a communication route that does not depend on the account being tested. The change owner should be able to state the expected log result before saving. If they cannot, the policy is not ready for enforcement. To carry this out, open Microsoft Entra admin center → Entra ID → Monitoring & health → Sign-in logs, use Add filters with the exact UPN and time window, open the event and select Conditional Access; the expected result is a correlation ID and policy outcome to compare with the test record. Clear filters or closing the details pane safely reverts only the view and changes nothing.

Keep a small test matrix rather than a sprawling checklist: intended pilot success, expected blocked path, emergency-account exclusion, supported mobile or desktop path, and application-owner confirmation. Each row needs an actual event and a decision. If an event is missing, label it untested. “No complaints” is operationally useful background noise, but it is not validation.

When rolling back, reverse the smallest recent change. Capture the policy state, disable or return that policy to report-only, verify a fresh affected sign-in, and leave unrelated controls intact. A broad emergency exclusion can restore access quickly but also creates a gap that is difficult to notice later. If it is ever used, give it a named owner, expiry and follow-up test before the window closes.

Handling difficult client paths

Older mail clients, automation tools and network appliances may not expose the modern authentication and device signals a policy expects. Identify them with sign-in logs and application owners before a broad block. Decide whether to retire, modernise, isolate or temporarily exempt each path. An exception is acceptable only when it has a compensating control, owner and expiry. Otherwise it is not a transition; it is the design.

For device-compliance controls, test enrolment and recovery paths as well as the healthy managed device. A newly issued laptop, a wiped phone, a travelling administrator and a device with stale management status can all produce different evidence. Do not combine this discovery with a high-risk administrator lockout experiment. Test the device prerequisite separately, then apply the Conditional Access requirement to a deliberately narrow group.

Audit, monitoring and human response

Alert on emergency-account sign-ins and significant policy changes, but make alerts actionable. Include account, tenant, source context, correlation ID, policy name and a route to the sign-in event. Exercise the notification path during the pilot. An alert that arrives in an unmonitored mailbox has achieved the traditional security outcome of existing magnificently on paper.

Retain change records and selected sign-in evidence according to the organisation’s retention policy. During an incident, this allows responders to tell whether a new block began with a policy change, an authentication-method problem or an unrelated service condition. Avoid placing credentials, recovery codes or unnecessary personal data in the record. Evidence should make recovery easier, not create another secret store.

A controlled pilot with evidence

Consider a 240-person engineering firm introducing phishing-resistant authentication for a small privileged group. The change owner defines a narrow hypothesis: pilot administrators will meet the selected authentication strength in the Entra, Azure and Microsoft 365 admin portals; an ordinary employee outside the group remains unaffected; the two cloud-only emergency accounts remain excluded. The policy does not claim to solve device compliance or service-principal security. That restraint is useful: a policy with one job can be tested, whereas a policy with six jobs is usually discovered during an outage.

The pilot starts in report-only mode. A normal administrator, a help-desk administrator and a travelling finance approver perform one relevant portal action from approved managed devices. A colleague watches sign-in logs and the service desk has the exact pilot wording. Testing someone outside the group matters too. Silence is not proof that an assignment boundary is correct.

Read the sign-in record, not the green page

In the Entra admin centre, open Identity > Monitoring & health > Sign-in logs, filter on the pilot account and agreed time window, and open the Conditional Access tab. A healthy in-scope test identifies the intended application and shows the named policy as report-only success, or success after approved enforcement. Record the correlation ID, timestamp, application and result in restricted change evidence. A green page is comforting but cannot explain an overlapping policy.

For a repeatable read-only check, an authorised operator with audit-log permission can use Microsoft Graph PowerShell. It changes no policy and signs nobody out:

Connect-MgGraph -Scopes 'AuditLog.Read.All'
Get-MgAuditLogSignIn -Filter "userPrincipalName eq '<PILOT_ADMIN_UPN>' and createdDateTime ge 2026-10-18T18:00:00Z" -Top 20 |
  Select-Object CreatedDateTime, AppDisplayName, Status, ConditionalAccessStatus, CorrelationId

The expected row has the intended application, a successful status and a Conditional Access status compatible with the policy mode. A failure is not automatically a broken rule: inspect the failed grant, client type, device state and authentication detail. A success proves only that this sign-in completed under recorded conditions, not that every browser, network or future edit is safe.

Inspect the policy object too, rather than inferring its assignments from one log event:

Get-MgIdentityConditionalAccessPolicy -All |
  Where-Object DisplayName -eq '<POLICY_DISPLAY_NAME>' |
  Select-Object Id, DisplayName, State, Conditions, GrantControls
~~~ To carry this out, open **Microsoft Entra admin center → Entra ID → Conditional Access → Policies → [exact policy name]** and compare the relevant **Assignments**, **Grant** or **Enable policy** section with the change record; the expected result is the documented scope and state. Remove an incorrect unsaved selection with **X**, then choose **Done** or **Cancel**; the production rollback is **Report-only** followed by **Save**, then sign-in-log validation.

During observation, expect enabledForReportingButNotEnforced; enforcement should appear only after approval. A broad all-users assignment without the emergency exclusion, unexpected every-cloud-app scope, or a state different from the change record is suspicious. Stop expansion and compare the live object with the approved export. Do not repair it from memory.

## When evidence says stop

If the travelling approver is repeatedly prompted, logs may point to a token cache, browser profile, overlapping device condition or location control while the authentication-strength rule is behaving correctly. Capture the correlation ID first. Test once from the approved managed device and browser. If only the unmanaged route fails, it is evidence for a separate client decision, not a reason to widen an exclusion.

If a pilot administrator is blocked, do not add a broad bypass or delete several policies. Verify the separately held emergency account can reach Entra from the approved workstation and that its sign-in alerts the nominated recipient. Use it only to set the affected new policy back to report-only or disabled according to the rollback decision. Preserve failed correlation IDs and capture policy conditions before changing state. Expand only when every in-scope result is explainable, emergency exclusions are documented, support has a short path, and every incompatibility has an owner and expiry.

## A decision record that makes rollback possible

Before enforcing the pilot, write a short decision record that names the policy ID, target group, excluded emergency group, test personas, expected application list and the exact state to which rollback will return. Keep the export or a structured capture of the live policy with the record. This is not paperwork for its own sake. Conditional Access changes are easy to describe incorrectly after the fact, particularly where an all-users target overlaps a role assignment or another policy contributes the visible prompt.

The reviewer should ask three questions for every observed sign-in. Which policy evaluated? Which control produced the result? What condition made this test representative? If the answer to any is uncertain, retain report-only mode. A report-only result can demonstrate that a condition would apply; it cannot demonstrate that a user has completed the eventual grant control. An enforced pilot sign-in can demonstrate the latter, but it still needs a known rollback route.

A safe evidence set includes the policy's display name and ID, its state at test time, a correlation ID for each representative test, a short statement of the expected and observed outcome, and any exception with a named owner. Do not include passwords, recovery codes, token values or detailed personal location data. The point is to make a future decision reproducible, not to turn the change ticket into an identity-data archive.
## Exceptions that do not become a second architecture

Some pilot members will expose awkward paths: a legacy administrative script, a shared kiosk, a privileged support tool that launches an embedded browser, or an outsourced support arrangement. The correct response is to identify the path and choose a bounded treatment. It may mean moving the task to a supported workstation, using a modern authentication flow, reducing the role, restricting access by network, or setting a dated exception while an owner remediates the application. It does not mean excluding every administrator because one old client is inconvenient.

For each exception, define the residual risk and expiry in the change record. At expiry, either remove the exception, renew it with an evidence-based reason, or replace the affected process. A review that merely asks whether the exception is still needed is too gentle; ask which control would fail if it remained forever. This forces a real decision rather than a familiar administrative shrug. To carry this out, open **Microsoft Entra admin center → Entra ID → Conditional Access → Policies → [exact policy name]** and compare the relevant **Assignments**, **Grant** or **Enable policy** section with the change record; the expected result is the documented scope and state. Remove an incorrect unsaved selection with **X**, then choose **Done** or **Cancel**; the production rollback is **Report-only** followed by **Save**, then sign-in-log validation.

## Ending the change safely

After the pilot is enforced, repeat one successful representative sign-in and one boundary test outside the group. Confirm the emergency account has not been used as the ordinary test subject and that its own monitoring route remains ready. Announce the final scope, support contact and exception deadlines to affected operators. Then schedule the next narrow policy change rather than folding another control into the same rollout. Identity systems do not resent optimism; they simply preserve it in audit logs. To carry this out, open **Microsoft Entra admin center → Entra ID → Monitoring & health → Sign-in logs**, use **Add filters** with the exact UPN and time window, open the event and select **Conditional Access**; the expected result is a correlation ID and policy outcome to compare with the test record. **Clear filters** or closing the details pane safely reverts only the view and changes nothing.

## The final go or no-go

Before widening the assignment, have the change owner and reviewer state the outcome plainly. Go only when the intended pilot evidence is explainable, the emergency path is independent and monitored, and every exception has a bounded owner and expiry. Pause when a sign-in cannot be explained, an alert is missing, or rollback has not been rehearsed. This is not reluctance; it is the difference between a staged control and a tenant-wide experiment.

If the answer is go, add the next smallest representative group and repeat the same evidence collection. If the answer is pause, retain report-only mode, preserve the correlation IDs and repair the specific cause. Both outcomes are useful. The only unhelpful outcome is enforcing a policy whose behaviour nobody can describe.

## Operational traceability in every later action

In later reviews, expansions and recovery actions, never make the reader guess where a control lives. To inspect an existing policy, open **Microsoft Entra admin center → Entra ID → Conditional Access → Policies → [exact policy name]**. Compare **Users**, **Target resources**, **Conditions**, **Grant** and **Enable policy** with the change record; the expected result is only the approved pilot group, the documented emergency group in exclusions and the intended state. Before saving, revisit every section and remove an incorrect object with **X/Remove**; if the correction is uncertain, choose **Cancel**, not **Save**. This is a safe rollback because nothing is applied before Save.

To filter or review evidence, open **Microsoft Entra admin center → Entra ID → Monitoring & health → Sign-in logs**, select **Add filters → User**, enter the pilot user UPN, then **Date** and the test interval; open the exact row and the **Conditional Access** tab. The expected result is the policy name, report-only or enforced outcome, resource and correlation ID. Compare those fields with the test record; closing the details pane or using **Clear filters** reverses only the view, not the configuration. For an export, choose **Download → Download (CSV)** from the same log list, retain it in the approved evidence location and confirm that it contains only the intended time window; delete the local copy in accordance with organisational retention rules, not the Entra audit event.

To test predicted scope, open **Entra ID → Conditional Access → What If**. Under **User**, choose the named pilot user; under **Cloud app**, choose the same resource; then set only the real device-platform and network context and select **What If**. The expected result is the new policy with the predicted report-only or enforced result; validate it against a real sign-in-log event. To repeat safely, change or clear the inputs; the tool does not alter the policy. If a real test blocks a required path, the authorised operator opens **Policies → [policy] → Enable policy**, selects **Report-only** (the preferred rollback) or the approved **Off**, selects **Save**, then checks that precise sign-in-log event again. Do not delete the policy: deletion removes diagnostic context.

The CLI alternative is read-only: from a managed administrator workstation with the appropriate Microsoft Graph read permissions, retrieve the policy and sign-in events by policy ID and correlation ID. The expected result is the same scope and state displayed in the portal. Do not use a script to silently rewrite the policy during a pilot; any writing API step belongs only in the same approved change window and with the rollback operator.

## References

- [Microsoft: Emergency access accounts](https://learn.microsoft.com/en-us/entra/identity/role-based-access-control/security-emergency-access)
- [Microsoft: Conditional Access report-only mode](https://learn.microsoft.com/en-us/entra/identity/conditional-access/concept-conditional-access-report-only)
- [Microsoft: What If tool](https://learn.microsoft.com/en-us/entra/identity/conditional-access/what-if-tool)
- [Microsoft: Sign-in logs](https://learn.microsoft.com/en-us/entra/identity/monitoring-health/concept-sign-ins)
- [Microsoft: Authentication strengths](https://learn.microsoft.com/en-us/entra/identity/authentication/concept-authentication-strengths)