Skip to content

getToken() replays a failed token refresh to later callers while a valid token is cached #3234

Description

@GUMBOKIM

[READ] Step 1: Are you in the right place?

Yes — a bug in the core token cache in this repository (src/app/firebase-app.ts), affecting every
product that authenticates through AuthorizedHttpClient or AuthorizedHttp2Client (not a Firestore
issue).

[REQUIRED] Step 2: Describe your environment

  • Operating System version: macOS 27.0 (Darwin 27.0.0)
  • Firebase SDK version: firebase-admin@14.2.0 (regression introduced in 12.3.1)
  • Firebase Product: core / app — the shared token cache, so everything above
    AuthorizedHttpClient / AuthorizedHttp2Client is affected
  • Node.js version: v24.16.0
  • NPM version: 11.13.0

[REQUIRED] Step 3: Describe the problem

Summary

A single failed token refresh is replayed to every later caller for as long as ~55 minutes, while a
valid token sits in the cache. The credential is never retried in that window.

This is a regression from #2648, first released in 12.3.1.

Steps to reproduce:

No credentials and no network access needed.

  1. npm install firebase-admin@14.2.0
  2. Run the script under Relevant Code below.
  3. Observe that every getToken() after the failed forced refresh rejects, even though
    getCachedToken() reports a token with an hour of life left.

Relevant Code:

const { initializeApp } = require('firebase-admin/app');

let calls = 0;
const app = initializeApp({
  projectId: 'demo-project',
  credential: {
    getAccessToken: () => {
      calls++;
      return calls === 1
        ? Promise.resolve({ access_token: 'good-token', expires_in: 3600 })
        : Promise.reject(new Error('503 from the token endpoint'));
    },
  },
});

(async () => {
  console.log('first getToken():', (await app.INTERNAL.getToken()).accessToken);

  // A forced refresh, as RTDB issues after a token revocation, with the rejection swallowed.
  await app.INTERNAL.getToken(true).catch(() => {});

  const cached = app.INTERNAL.getCachedToken();
  console.log('cached token still valid for', cached.expirationTime - Date.now(), 'ms');

  for (let i = 0; i < 3; i++) {
    try {
      console.log(`later getToken() #${i + 1}:`, (await app.INTERNAL.getToken()).accessToken);
    } catch (e) {
      console.log(`later getToken() #${i + 1}: REJECTED ${e.code} | credential calls = ${calls}`);
    }
  }
})();

Expected behavior:

A failed refresh should not be replayed to later callers while a usable token is cached.

Actual behavior:

first getToken(): good-token
cached token still valid for 3600000 ms
later getToken() #1: REJECTED app/invalid-credential | credential calls = 2
later getToken() #2: REJECTED app/invalid-credential | credential calls = 2
later getToken() #3: REJECTED app/invalid-credential | credential calls = 2

credential calls = 2 on every line: the credential is asked once more and then never again for the
rest of the token's hour.

Root cause:

getToken() caches the in-flight refresh promise and hands it back to every caller until the cached
token is close to expiry (src/app/firebase-app.ts:52-57, :124-127):

public getToken(forceRefresh = false): Promise<FirebaseAccessToken> {
  if (forceRefresh || this.shouldRefresh()) {
    this.promiseToCachedToken_ = this.refreshToken();
  }
  return this.promiseToCachedToken_
}

private shouldRefresh(): boolean {
  return (!this.cachedToken_ || (this.cachedToken_.expirationTime - Date.now()) <= TOKEN_EXPIRY_THRESHOLD_MILLIS)
    && !this.isRefreshing;
}

A rejected promise is cached exactly like a resolved one. Before #2648 that could not happen: the
non-refresh path returned the cached token directly.

if (forceRefresh || this.shouldRefresh()) {
  return this.refreshToken();
}

return Promise.resolve(this.cachedToken_);

#2648 replaced that last line with return this.promiseToCachedToken_ so concurrent callers would
share one refresh, which is the right goal. They now also share one that failed.

How a rejection gets there in normal operation

Two things have to coincide, and an app using the Realtime Database gets the first for free.

A forced refresh at an arbitrary token age, and a failure at the token endpoint while it is in
flight.

In the bundled RTDB client (@firebase/database-compat 2.1.6 here, dist/index.standalone.js,
loaded at src/database/database.ts:129), onAuthRevoked_ sets forceTokenRefresh_ = true, and the next
establishConnection_ reads that flag into a local and passes it to
authTokenProvider_.getToken(forceRefresh), which forwards through to INTERNAL.getToken(true).
Nothing on that path consults token age, so a revocation makes the SDK force a refresh while the
cached token may still have most of its hour left. A transient invalid_token from the server is
enough to trigger one, and the revocation handler's own comment allows for exactly that:

// We'll wait a couple times before logging the warning / increasing the
// retry period since oauth tokens will report as "invalid" if they're
// just expired. Plus there may be transient issues that resolve themselves.

If the token endpoint then fails during that refresh, the rejection is what gets memoized. From
there RTDB cannot recover on its own: establishConnection_ clears forceTokenRefresh_ before
calling getToken, and its catch never restores it, so every later reconnect asks with
forceRefresh = false and receives the cached rejection.

This SDK also forces refreshes itself, at src/database/database.ts:166-176, which schedules
getToken(/*forceRefresh=*/ true) five minutes before expiry and swallows the rejection. That one
does not produce the long replay, because at the five-minute mark the cached token is inside the
refresh threshold anyway.

Impact:

Every caller of AuthorizedHttpClient.getToken() (src/utils/api-request.ts:1131) and
AuthorizedHttp2Client.getToken() (:1167) is affected, including the IAMSigner instances used
for Auth and App Check token signing under non-service-account credentials
(src/utils/crypto-signer.ts:209).

One momentary blip at the token endpoint therefore becomes a multi-minute, potentially ~55-minute,
failure of every Firebase service that authenticates through those clients, reported as a misleading
app/invalid-credential, with a valid token available the whole time.

Suggested fix

Stop handing promiseToCachedToken_ to later callers when it holds a rejection and cachedToken_ is
still usable. That restores the pre-#2648 read path while keeping the de-duplication #2648 wanted.

I have a fix ready with tests and will attach it as a PR.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions