When running the Hybrid Configuration Wizard (HCW), at the point where the Hybrid Agent installs, it installs and registers fine, but then fails the verification step.
The HCW logs (found in %AppData%\Microsoft\Exchange Hybrid Configuration) show the following error:2019.10.21 13:09:24.552 *ERROR* 10349 [Client=UX, Page=HybridConnectorInstall, Thread=20] The connection to the server '3e6c87b2-dcc8-4e4d-a51e-63dc1ae42acf.resource.mailboxmigration.his.msappproxy.net' could not be completed., The call to 'https://3e6c87b2-dcc8-4e4d-a51e-63dc1ae42acf.resource.mailboxmigration.his.msappproxy.net/EWS/mrsproxy.svc' failed. Error details: The HTTP request was forbidden with client authentication scheme 'Negotiate'. --> The remote server returned an error: (403) Forbidden.., The HTTP request was forbidden with client authentication scheme 'Negotiate'., The remote server returned an error: (403) Forbidden
And the Windows application log might show the following:Log Name: Application
Source: MsExchange BackEndRehydration
Date:
Event ID: 3002
Task Category: Requests
Level: Error
Keywords: Classic
User: N/A
Computer: <Computer Name>
Description:
Protocol /OAB failed to process request from identity NT AUTHORITY\SYSTEM. Exception: Microsoft.Exchange.Security.Authentication.BackendRehydrationException: Rehydration failed. Reason: Source server 'NT AUTHORITY\SYSTEM' does not have token serialization permission.
at Microsoft.Exchange.Security.Authentication.BackendRehydrationModule. TryGetCommonAccessToken(HttpContext httpContext, Stopwatch stopwatch, CommonAccessToken& token)
at Microsoft.Exchange.Security.Authentication.BackendRehydrationModule. ProcessRequest(HttpContext httpContext)
at Microsoft.Exchange.Security.Authentication.BackendRehydrationModule. OnAuthenticateRequest(Object source, EventArgs args).
It points to an issue with EWS so check your EWS URLs are set correctly:
Get-WebServicesVirtualDirectory | fl InternalUrl, ExternalURL
It also seems serialization permissions are missing. Running the following commands in the Exchange Mgmt Shell to add the required permissions:
Get-ClientAccessServer | Add-AdPermission -AccessRights ExtendedRight -ExtendedRights "ms-exch-epi-token-serialization" -User "<domain>\Exchange Servers"
Get-ClientAccessServer | Add-ADPermission -AccessRights ExtendedRight -ExtendedRights "ms-Exch-EPI-Impersonation" -User "<domain>\Exchange Servers"
Get-MailboxServer | Add-AdPermission -AccessRights ExtendedRight -ExtendedRights "ms-exch-epi-token-serialization" -User "<domain>\Exchange Servers"
Get-MailboxServer | Add-AdPermission -AccessRights ExtendedRight -ExtendedRights "ms-Exch-EPI-Impersonation" -User "<domain>\Exchange Servers"
Lastly run an iisreset in an administrative cmd box and rerun the wizard. The agent should now verify correctly.
With thanks to David Robinson of Kick ICT who provided me with this information
Thank you! Worked for me.
This worked, thank you
Thank you ! Worked for me too, just ran the commands and iisreset
really worked perfectly
What should be the EMS command – Should try this or is it something different. My Exchange Server is EX2019.
Get-ClientAccessServer | Add-AdPermission -AccessRights ExtendedRight -ExtendedRights “ms-exch-epi-token-serialization” -Administrator “ramlan\ex2019”
Hi Ram,
your EMS command would look like this:
Get-ClientAccessServer | Add-AdPermission -AccessRights ExtendedRight -ExtendedRights “ms-exch-epi-token-serialization” -User “ramlan\Exchange Servers”
Provided your Active Directory domain is ‘RAMLAN’
Regards,
Misja
I was able to get permission details for above against User Administrator and it is FALSE FALSE (Deny & Inherited). So do you think, I should be fine with Hybrid Configuration run?
Thanks for the info. I was able to run the command and the result is FALSE FALSE (Deny and Inherited) against -User Exchange Servers
Will do iis reset and try hybrid.
Ram
Hi Ram,
the user is “%domain%\Exchange Servers” (in your case “ramlan\Exchange servers”). Mind the quotes as there is a space in ‘Exchange Servers’
Regards,
Misja
+1 for the working solution. Thanks!
+1 work for me as well.. Thanks..!
Check IIS request filtering, if its installed
Allow svc extensions
Or else, youre in trouble
Hi Magnus,
thanks for your response. Can you elaborate on your comment please? I tried to research it myself, but I can’t find any relationship between my blog and your comment.
Cheers
worked for me as well
Exchange 2013 onprem to Exchange online 365
Thanks, I just setup 3 hybrids with this error!
Thanks so much, worked like a charme on Exchange Server 2019 CU10 Jan22SU
thanks, its works.
Thanks that worked perfectly!!
Still working in 2022. Thanks for taking the time to write this up.
Thanks, given solution resolved the issue.
Thanks for this! Sorted straight away
In My case user/ or group ‘domainname\exchangeservername’ was not found.
Hi, the user is “%domain%\Exchange Servers”, not “%domain%\exchangeservername”. “Exchange Servers” is not a placeholder but the actual name of the AD object.
Thank you very much! These steps worked for me as well.
I have been trying to figure out this error for 6 months. I even have a ticket in with Microsoft and they could not help. Thank you for this!