In my Asp .net 2.1 ADFS implementation, GetExternalLoginInfoAsync in my account controller always returns null

Question

When a user goes through the ADFS flow and I get the callback to my /account/externalLogin in my account controller. The line: var info = await _signInManager.GetExternalLoginInfoAsync(); always sets info to null.

This is the case on both our internal adfs test instance and our client's test instance. (This project is to add SSO support to our client)

The Code below is from my account controller and gets called after the user goes through the adfs flow. The if statement info == null is always true and we redirect them to the help page as a testing location.

    [HttpGet]
    [AllowAnonymous]
    public async Task<IActionResult> ExternalLogin(string returnUrl = null, string page = null, string handler = null, string remoteError = null)
    {
        // Catches the first get request from someone returning from external login
        Console.WriteLine("Login event!!!");

        if (remoteError != null)
        {
            ViewBag.ErrorMessage = $"Error from external provider: {remoteError}";
            Console.WriteLine($"Error from external provider: { remoteError}");
            return RedirectToAction("Help", "Home");
        }
        var info = await _signInManager.GetExternalLoginInfoAsync();
        if (info == null)
        {
            Console.WriteLine("no info, retrying");
            return RedirectToAction("Help", "Home");
        }

The relevant section from my startup.cs file is:

services.AddAuthentication()
        .AddWsFederation(options =>
        {
            // MetadataAddress represents the Active Directory instance used to authenticate users.
            options.MetadataAddress = "https://adfsqa.client.com/FederationMetadata/2007-06/FederationMetadata.xml";
            //options.MetadataAddress = "https://ourtestadfsinstance.westus2.cloudapp.azure.com/FederationMetadata/2007-06/FederationMetadata.xml";

            // Wtrealm is the app's identifier in the Active Directory instance.
            // For ADFS, use the relying party's identifier, its WS-Federation Passive protocol URL:
            options.Wtrealm = "https://rc.dm1.tech";

        });

My implementation is very simple and close the stock example for implementing ADFS documented here. Implementation details were also taken from this sample from the ASP .net core git repo sample project for ExternalClaims here.

I searched for this issue extensively and spend several days trying to solve it at this point. The closest public post of this issue appears to be this one but most of the other are around rolling out Oauth2 from steam, Instagram or any other 3rd party.

Any clues, hints, or even thoughts would be greatly appreciated.