IDX10503 Exception with Secret Issued from Partner Center for Store Listed Application (SharePoint Provider hosted addin) - Client Secret is no longer base64 string

[RafalUrbanski]

Target SharePoint environment

SharePoint Online

What SharePoint development model, framework, SDK or API is this about?

SharePoint Add-ins

Developer environment

Windows

What browser(s) / client(s) have you tested

• 💥 Internet Explorer
• 💥 Microsoft Edge
• 💥 Google Chrome
• 💥 FireFox
• 💥 Safari
• mobile (iOS/iPadOS)
• mobile (Android)
• not applicable
• other (enter in the "Additional environment details" area below)

Additional environment details

No response

Describe the bug / error

I have a Store listed SharePoint Add-in that was created few years ago, and need to update the secret. Yesterday in the morning we created a new client secret and removed old ones from the Partner Portal. New client secret that we get from the Partner Portal is no longer a base64 string and when we replaced ClientSecret in our web.config, we got the following error:
System.FormatException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.
In order to avoid this issue, we encoded the client secret to base64 string and now we get the following error:
Microsoft.IdentityModel.Tokens.SecurityTokenInvalidSignatureException: 'IDX10503: Signature validation failed. Keys tried: '[PII is hidden]'.

It's already been more than 24 hours since we created new Client Secret, so we believe it should already propagate correctly.
Is the basic encoding to base64 a wrong choice, or what could be a reason for our application not to validate a token properly?
I have also checked the new client secret using Connect-PnPOnline via powershell and without any additional encoding the secret works.

Thanks in advance - as this is about our live application, I would appreciate any help.

Steps to reproduce

1. Locate a SharePoint Store listed, provider hosted add-in, created several years ago.
2. Create a new Secret in Partner Center
3. Update the application with the new secret.

Expected behavior

The expected behaviour is for the JWT token to pass validation correctly.

[RafalUrbanski]

We have also noticed that related App Registration in Portal Azure is missing valid Certificate:

Old certificates were added automatically when we created new Client Secret in Partners Portal.

At the same time, new client secret created in Partners Portal was automatically added in our App Registration in portal azure.

Any ideas? Should we manually create a new certificate?

[jansenbe]

@RafalUrbanski : Can you please open a ticket with Partner Center for this? Did you associate your Add-In with a tenant?

[RafalUrbanski]

@jansenbe We opened tickets everywhere we could, also with Partner Center, hoping anybody had a same issue. Partner Center need 1-2 business to validate, we are still waiting for a response.

We associated Microsoft Entra Tenant with Microsoft Partner Center (acording to this article https://learn.microsoft.com/en-us/partner-center/account-settings/multi-tenant-account#add-a-microsoft-entra-tenant-to-your-account )

[RafalUrbanski]

We managed to fix the issue by ourselves. As this may be helpful for some of you and there is little chance of getting timely support, I will post the solution here.

This fix applies to SharePoint Add-ins that were registered on the Seller Dashboard. In the past, we created a new Client Secret every year using Microsoft Partner Center. This year, we did the same, but the new Client Secret didn't work, and our live application broke.

The first thing we noticed was that the new Client Secret returned by the Microsoft Partner Center was no longer a base64 encoded string. Since the TokenHelper.cs class in the SharePoint Add-in was expecting a base64 encoded string, replacing the Client Secret with the new value caused the following error:
"System.FormatException: The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters."

To overcome the issue, we encoded the new Client Secret using the following PowerShell command:
[System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($clientSecret))
Doing so fixed the first issue, but we encountered another one. The SharePoint Add-in was not able to authenticate with SharePoint Online. The error message was one of the following (based on the version of the TokenHelper.cs class):

• "Microsoft.IdentityModel.Tokens.SecurityTokenInvalidSignatureException: 'IDX10503: Signature validation failed. Keys tried: '[PII is hidden]'"
• "Invalid JWT token. Could not resolve issuer token."

At first, we thought that the new Client Secret needed some time to propagate through the Microsoft infrastructure, but after 24 hours, the issue was still present.

To keep the story short - in order to create a new Client Secret, we have to use a PowerShell script and update the related AD Application in the Azure Portal -> App Registrations. You should find the related AD Application by the Client ID of the SharePoint Add-in.

# If you haven't installed the related Graph module yet, you can install it using Install-Module -Name Microsoft.Graph
Connect-MgGraph -Scopes "Application.ReadWrite.All,Directory.ReadWrite.All"

$appreg = Get-MgApplication -ApplicationId 'Application Object ID' # Object ID, not the client ID
$params = @{
    PasswordCredential = @{
        DisplayName = "NewSecret" # Replace with a friendly name.
    }
}
$result = Add-MgApplicationPassword -ApplicationId $appreg.Id -BodyParameter $params
$base64Secret = [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($result.SecretText))
$appreg = Get-MgApplication -ApplicationId 'Application Object ID'
$existingKeyCredentials = $appreg.KeyCredentials
$dtStart = [System.DateTime]::Now
$dtEnd = $dtStart.AddYears(1)
$keyCredentials = @(
    @{
        Type = "Symmetric"
        Usage = "Verify"
        Key = [System.Text.Encoding]::ASCII.GetBytes($result.SecretText)
        StartDateTime = $dtStart
        EndDateTime = $dtEnd
    },
    @{
        Type = "Symmetric"
        Usage = "Sign"
        Key = [System.Text.Encoding]::ASCII.GetBytes($result.SecretText)
        StartDateTime = $dtStart
        EndDateTime = $dtEnd
    }
) + $existingKeyCredentials # Combine with existing
Update-MgApplication -ApplicationId $appreg.Id -KeyCredentials $keyCredentials # Update keys
$result.SecretText # Print secret text
$base64Secret # Print base64 secret
$result.EndDateTime # Print the end date.

Your new Client Secret will be printed at the end of the script. You should save it and replace the ClientSecret in the web.config of your application. The new client secret should start working almost immediately. However, we noticed that for users who were already logged in, the issue was still present until their token expired. It may take a few hours for the issue to resolve itself for all users.

[jansenbe]

@RafalUrbanski : great to hear you got the issue resolved and thanks for sharing the write-up, sharing is caring 💖

[westleyMS]

@jansenbe it looks like appregnew is creating new apps without base64 encoded keys for SP add-in tokenhelper.cs code. This breaks the add-in app when creating a new app principal for it. Can this be fixed, or do we just need to run the script to reset the keys and secrets to a base64 argument we specify?

[RafalUrbanski]

@westleyMS I tried to use appregnew.aspx this week, however I ended creating correct secret using the script available in this this page - seems that appregnew.aspx is broken and only part of it that creates app registration works, part related with client secret is broken

[SchauDK]

@RafalUrbanski First, thank you for the workaround. You should know that Microsoft Support is officially referring to it.
I can confirm that it initially worked, but now we're suddenly having issues with our existing secrets (generated using your method). Can you also confirm this?
We've tried to create new secrets, They also doesn't work.

[RafalUrbanski]

@SchauDK So far our newly generated secrets work for both the App Source app and the locally developed SharePoint Add-in. Is there a chance that you have generated more than one Client Secret for you application and currently more than one client secret is valid?
Edit:
I would suggest testing Client Secrets using powershell (both base64 encoded and not encoded - powershell supports both types), it's easy way to verify if secrets are working and can be very helpful

[SchauDK]

@RafalUrbanski Multiple secrets is not an issue. We've always had multiple for a smooth rollover when a secret is expiring.
The secrets are working when testing directly against the App Registration, but it is failing when validating from a SharePoint context token.

[RafalUrbanski]

@SchauDK From our experience, SharePoint context can only be created for one of the existing client secrets (if you have multiple valid secrets existing at the same time) - I guess it is the first valid client secret on the list, others won't work until that one expires.

Regarding the validation of secrets, I have always done it this way (with the base64 encoded secret to be sure that it is correct):
'Connect-PnPOnline -Url "SiteUrl" -ClientId -ClientSecret
Get-PnPWeb'
In case the information was returned, I treated the secret as working.
Maybe something has been updated within the App Registration that caused the Client Secret to stop working?