How to remove custom login page for client application?

Question

I have created custom login page for my site. I have enabled file open behavior to client application.

So now when I open any word document in library, the prompt in word application displays my custom login page. When user login, it redirect user to site in that prompt. User need to then manually close this pop-up. Then the file is opened.

I want to achieve same functionality as in SharePoint default login where user enters credentials and the popup close. How can we do this in custom login page?

If this is not possible then is there a way to set SharePoint default login page for client application like word?

Here is my code sample:

protected void btnFBALogin_Click(object sender, EventArgs e)
    {
        try
        {
            SignIn(txtUserName.Text, txtPwd.Text);
        }
        catch (Exception ex)
        {
            throw ex;
        }
    }

    public void SignIn(string user, string pass)
    {
        try
        {
            //Call the _authorizer to check the credentials
            if (_authorizer.Authorize(user, pass, true))
            {
                //Checks if the pasword is expired, usually a rediriect or message if so
                if (_authorizer.IsExpired) { }

                var identity = (IClaimsIdentity)HttpContext.Current.Session["Claims"];             

                SPIisSettings iisSettings = SPContext.Current.Site.WebApplication.IisSettings[SPUrlZone.Default];
                SPFormsAuthenticationProvider formsClaimsAuthenticationProvider = iisSettings.FormsClaimsAuthenticationProvider;
                SecurityToken token = (SecurityToken)HttpContext.Current.Session["SessionToken"];

                if (null != token)
                {
                    //, SPSessionTokenWriteType.WriteSessionCookie
                    EstablishSessionWithToken(token);
                    Response.Redirect(SPContext.Current.Web.Url + "/Pages/Home.aspx");                        
                }
            }

        }
        catch (Exception e)
        {
            _failMessage.InnerText = e.Message;
            txtPwd.Text = string.Empty;
            throw e;
        }
    }

    // Microsoft.SharePoint.IdentityModel.Pages.IdentityModelSignInPageBase
    private void EstablishSessionWithToken(SecurityToken securityToken)
    {
        if (securityToken == null)
        {
            throw new ArgumentNullException("securityToken");
        }
        SPFederationAuthenticationModule fam = SPFederationAuthenticationModule.Current;
        if (fam == null)
        {
            throw new InvalidOperationException();
        }

        fam.SetPrincipalAndWriteSessionToken(securityToken);
    }

Answer 1

If you have a custom page, you probably handle the redirection (after the logon is successfull) "manually". How do you do this?
If you actually redirect without taking in account the context (was the request issued by a client application or a browser, etc.), the behavior you see is expected.
The option I generally take is to inherit my custom login page from FormsSignInPage (or IdentityModelSignInPageBase) so I can leverage default behaviors like correct redirection after logon.
You can also have a look (through ILSpy or like) at their IdentityModelSignInPageBase.RedirectToSuccessUrl to see how they handle redirection under specific conditions; or you could also call directly to this method (meaning, again, inheriting from IdentityModelSignInPageBase).