Slow queries if fetched after other queries

Question

Slow queries if fetched after other queries

Closed this issue 2 months ago · 6 comments

Byloth commented 2 months ago

Operating System

Microsoft Windows 11 Pro 24H2

Environment (if applicable)

Google Chrome 139

Firebase SDK Version

11.10.0

Firebase SDK Product(s)

Firestore

Project Tooling

I'm working on WSL2 with: Ubuntu 24.04.3 LTS (GNU/Linux 6.6.87.2-microsoft-standard-WSL2 x86_64).
I'm using: Vue.js app with Vite.js

Detailed Problem Description

I have a dashboard application which shows some statistics collected & saved on Firestore.

There's a switch that allows the user to switch between a daily-basis stats to a hourly-basis one.
The application keeps track of what was the last used configuration so - when the page loads - it automatically uses the last selected config.

When it first load the page (e.g. after an Ctrl+ R), it takes a couple hundreds of milliseconds (like 100 - 200ms) to fetch the query. No matter what the last enabled configuration was.
When the application is already running (e.g. when the user changes the config), the same query might takes even seconds (like 2.500ms to 7.500ms). During this time the page also freezes and stops to be responsive.

These data are stored in 2 different collections: dailyStats & hourlyStats.

Here are the logs I collected about this:

Ctrl + R
First load: 📑dailyStats.log
When the config changes: 📑dailyStats-to-hourlyStats.log
Ctrl + R
First load: 📑hourlyStats.log
When the config changes: 📑hourlyStats-to-dailyStats.log

Steps and code to reproduce issue

The code I used to achieve these logs looks something like this:

const _benchmark = async (callback: Function) =>
{
    const start = performance.now();
    const result = await callback();
    const end = performance.now();

    console.log(`Benchmark: ${(end - start).toFixed(3)}ms`);

    return result;
};

const app = initializeApp({ ... });
const firestore = getFirestore(app);

setLogLevel("debug");

async function getHourly()
{
    const reference = collection(firestore, "hourlyStats");

    let snapshot: QuerySnapshot;
    await _benchmark(async () => { snapshot = await getDocs(query(reference, orderBy("date", "asc"))); });

    return snapshot.docs.map((doc) => doc.data());
}
async function getDaily(start: Date, end: Date)
{
    const reference = collection(firestore, "dailyStats");
    const whereCondition = and(where("date", ">=", start), where("date", "<", end));
    const orderByCondition = orderBy("date", "asc");

    let snapshot: QuerySnapshot;
    await _benchmark(async () => { snapshot = await getDocs(query(reference, whereCondition, orderByCondition)); });

    return snapshot.docs.map((doc) => doc.data());
}

await getHourly();
await getDaily(new Date("2025-08-28"), new Date("2025-09-04"));

Right now, I'm unable to share more than this...
It is part of an enterprise project.

I would say... Let's start from here:

Is it something already known?
Did I miss an existing Issue about this?
Is it something intentional?
How is it possible that no one has ever experienced / reported something like this?
Is it something related to some sort of caching?
Is it possible to disable it and avoid this problem?

I will prepare and put something together if necessary.

Regarding the last question above...
I've been struggling with this issue since April 2024 (it isn't somethin new), but now some customers are starting to complain...

Honestly, at the time, I was hoping that with some time and some new release of the Firebase SDK this problem would be solved, sooner or later... But now our development team have to face it and solve it.

Answer 1 · 2025-09-04T15:23:59.000Z

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

Answer 2 · 2025-09-04T23:25:29.000Z

@Byloth, thanks for reporting this. I appreciate you including debug logs in the initial post, it helps us get to a solution faster.

In your logs I can see the described behavior and indeed the delay does seem to be introduced when the SDK is in client side cache logic. We can dig deeper on a root cause.

In the meantime, I'm wondering if there is a quick fix or workaround we can apply in your app. Maybe you would be willing to try one or more of the following?

Clear cache of first query documents between configuration switch. The cache primarily acts to enable a seamless experience regardless of internet connectivity (fast, slow, none). The cache isn't always adding performance benefit when the app is online. It introduces extra stuff to do, like reconciling documents in the local cache with docs sent from the server. So, one thing to try is clearing your cache between the configuration switch in your application. From your logs I can see you are using memory local cache. Memory local cache has two garbage collection modes: Eager and LRU. If using Eager, documents will be cleared from the cache as soon as they are no longer associated with an active query. So, try this: ensure eager garbage collection is enabled and unsubscribe any snapshot listeners between mode change (if you have snapshot listeners). The code might look similar to this:

let firestore = initializeFirestore(app, {
  localCache: memoryLocalCache({
    garbageCollector: memoryEagerGarbageCollector()
  }
})

// If only using getDocs
await getDocs(query1);
// mode switch: no additional action. getDocs should not keep docs in an eager garbage collected cache
await getDocs(query2);

// If using onSnapshot
let unsubscribe = onSnapshot(query1, ...);
// mode switch: unsubscribe listeners. docs will be evicted from the cache after unsubscribe at next garbage collection
unsubscribe();
// set up new listener
unsubscribe = onSnapshot(query2, ...);

Set up a client side index if you need to keep docs in cache. You can use setIndexConfiguration to set up a specific client side index for these two queries. You should be able to use the same index configurations used in the backend indexes. This API is marked deprecated, that's because we are trying to push people to use auto index creation. But that will not be helpful for diagnosing in your case. If this API does help you, we would like to hear that feedback.
Try switching to persistent local cache. In the logs, the delay appears to be introduced when execution is in code for memory local cache. I'm not sure if persistent local cache will improve things for you, but it might be worth a quick try. Use persistentLocalCache() instead of memoryLocalCache() when initialzing Firestore.
Not really a workaround, but I'm curious if you ever tried an older version of the SDK (prior to April 2024), to see if the behavior still existed there?

Answer 3 · 2025-09-08T17:53:26.000Z

Hi, @MarkDuckworth.
Nice to meet you.

Here's some logs for the attempts you asked me to do.

Using Eager Garbage Collector:

It didn't work.

Edited code:

const _benchmark = async (callback: Function) =>
{
    const start = performance.now();
    const result = await callback();
    const end = performance.now();

    console.log(`Benchmark: ${(end - start).toFixed(3)}ms`);

    return result;
};

const app = initializeApp({ ... });
const firestore = initializeFirestore(app, {
    localCache: memoryLocalCache({ garbageCollector: memoryEagerGarbageCollector() }
});

setLogLevel("debug");

async function getHourly()
{
    const reference = collection(firestore, "hourlyStats");

    let snapshot: QuerySnapshot;
    await _benchmark(async () => { snapshot = await getDocs(query(reference, orderBy("date", "asc"))); });

    return snapshot.docs.map((doc) => doc.data());
}
async function getDaily(start: Date, end: Date)
{
    const reference = collection(firestore, "dailyStats");
    const whereCondition = and(where("date", ">=", start), where("date", "<", end));
    const orderByCondition = orderBy("date", "asc");

    let snapshot: QuerySnapshot;
    await _benchmark(async () => { snapshot = await getDocs(query(reference, whereCondition, orderByCondition)); });

    return snapshot.docs.map((doc) => doc.data());
}

await getDaily(new Date("2025-09-01"), new Date("2025-09-08"));
await getHourly();

Here's the log: 📑dailyStats-to-hourlyStats-eager_gc.log

Using explicit indexes:

It didn't work.

Edited code:

const _benchmark = async (callback: Function) =>
{
    const start = performance.now();
    const result = await callback();
    const end = performance.now();

    console.log(`Benchmark: ${(end - start).toFixed(3)}ms`);

    return result;
};

const app = initializeApp({ ... });
const firestore = getFirestore(app);

await setIndexConfiguration(firestore, {
    indexes: [
        { collectionGroup: "hourlyStats", fields: [{ fieldPath: "date", order: "ASCENDING" }] },
        { collectionGroup: "dailyStats", fields: [{ fieldPath: "date", order: "ASCENDING" }] }
    ]
});

setLogLevel("debug");

async function getHourly()
{
    const reference = collection(firestore, "hourlyStats");

    let snapshot: QuerySnapshot;
    await _benchmark(async () => { snapshot = await getDocs(query(reference, orderBy("date", "asc"))); });

    return snapshot.docs.map((doc) => doc.data());
}
async function getDaily(start: Date, end: Date)
{
    const reference = collection(firestore, "dailyStats");
    const whereCondition = and(where("date", ">=", start), where("date", "<", end));
    const orderByCondition = orderBy("date", "asc");

    let snapshot: QuerySnapshot;
    await _benchmark(async () => { snapshot = await getDocs(query(reference, whereCondition, orderByCondition)); });

    return snapshot.docs.map((doc) => doc.data());
}

await getDaily(new Date("2025-09-01"), new Date("2025-09-08"));
await getHourly();

Here's the log: 📑dailyStats-to-hourlyStats-indexes.log

Using persistent local cache

It worked! 🎉

Edited code:

const _benchmark = async (callback: Function) =>
{
    const start = performance.now();
    const result = await callback();
    const end = performance.now();

    console.log(`Benchmark: ${(end - start).toFixed(3)}ms`);

    return result;
};

const app = initializeApp({ ... });
const firestore = initializeFirestore(app, { localCache: persistentLocalCache({ }) });

setLogLevel("debug");

async function getHourly()
{
    const reference = collection(firestore, "hourlyStats");

    let snapshot: QuerySnapshot;
    await _benchmark(async () => { snapshot = await getDocs(query(reference, orderBy("date", "asc"))); });

    return snapshot.docs.map((doc) => doc.data());
}
async function getDaily(start: Date, end: Date)
{
    const reference = collection(firestore, "dailyStats");
    const whereCondition = and(where("date", ">=", start), where("date", "<", end));
    const orderByCondition = orderBy("date", "asc");

    let snapshot: QuerySnapshot;
    await _benchmark(async () => { snapshot = await getDocs(query(reference, whereCondition, orderByCondition)); });

    return snapshot.docs.map((doc) => doc.data());
}

await getDaily(new Date("2025-09-01"), new Date("2025-09-08"));
await getHourly();

Here's the log: 📑dailyStats-to-hourlyStats-persistent_lc.log

Old Firebase SDK versions

Honestly, I didn't try.
I mentioned that we've been seeing this problem since 2024, when we introduced the ability for users to switch between dailyStats and hourlyStats records. The problem likely existed before then.

Hoping this helps.

Answer 4 · 2025-09-09T19:49:11.000Z

@Byloth, I attempted to reproduce the behavior yesterday, but I did not see any delays for the second query. This was using my own simple app and own data, based on what you shared.

Maybe your application is resource constrained due to some application or framework specific code? Such that the second Firestore query and the rest of the application are competing for compute resources?

Things you could look into:

Attempt to reproduce the issue with a Minimal Reproducible Example. Eliminate other parts of the code down to just a plain js script that makes the two requests to your DB. Attempt to reproduce, and add back elements of your project (vue, then project specific code...) and see when the hang starts.
Use Chrome's performance profiling to determine what part of your application, framework, or dependencies are the bottleneck.
See if there are any changes in your application around April 2024 that could explain the change in behavior.

Answer 5 · 2025-09-16T18:00:40.000Z

Hey @Byloth. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

Answer 6 · 2025-09-23T18:00:43.000Z

Since there haven't been any recent updates here, I am going to close this issue.

@Byloth if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.