Omnistac/zedux

Complex Params with Singleton Atom

Opened this issue · 12 comments

decoursin commented

@bowheart Thank you very much this wonderful library. At least my first impressions are that it's amazing. I'm in the process of learning it, but I think I have a pretty good overview.

I had a question, I was hoping you would help me with it.

I noticed that the atom state factory "creates a different atom instance for every 'unique' set of params you pass" and unlike React, "Zedux doesn't compare object references. Internally, Zedux turns all params into a single string."

Imagine, however, if I pass a huge list where each item in the list is a deeply nested object with many, many properties, and I want to use this as the initial state of the injected atom's store. This doesn't seem to bode well however with how the atom state factory works.

It only needs to be a singleton store/atom.

How would you recommend that I solve this? I have considered the following solutions, but IDK exactly:

  • Setting complexParams: true when creating the ecosystem. The example you gave was just for a function as argument, but perhaps this would also work for arrays/objects?
  • Creating an atom where the valueOrFactory is the store. But there are all sorts of problems that come with this.
  • Perhaps somehow creating the store outside the atom, and then passing the store into the atom state factory as its parameter? But according to your docs the args of the atom state factory must be serializable, and I would imagine that stores are not serializable?

Theoretically speaking, all I think I need for this case is a sort of singleton atom with a predefined store. That store needs to change, but that atom should never be recreated if the original params to it somehow change.

I can try to give more detail, or perhaps you understand where I'm coming from? It seems like this would be a sort of common use case, but I'm not sure there exists a great solution for it yet?

ChrisBoon commented
  • ❤️2
decoursin commented

Hi @ChrisBoon that's an interesting suggestion, but that approach is definitely a little hacky and would be confusing to people reading the code.

If the complexParams on the ecosystem were later changed to true, that would cause the singleton atom to no longer be singleton if invoked in multiple places, or if someone forgot and change it to true it would break my app. And that would force lock-in to always having complexParams: false.

I really think there should be a way to pass in other data to the atom state factory that is not included in the "hashing". Is there some reason that hasn't been done yet, or perhaps just oversight?

bowheart commented

Hey @decoursin great question. And that's a cool suggestion @ChrisBoon. I'm gonna play with that actually.

Firstly, if you're running into this frequently, you're probably underutilizing Zedux. In an ideal setup, that big nested list would already be controlled by Zedux - e.g. it would be fetched/created in another atom rather than in a React component. The new atom could then simply access the list via an injector or atom getter. This setup also allows the new atom to react to changes in the list rather than taking a static snapshot on init.

With that in mind, there are exceptions, especially when using 3rd party tools that rely on React hooks. So I'll assume that's the case here.

complexParams isn't what you're looking for. Since the atom is for sure a singleton, you want it to have no params.

The Approaches

There are a few ways to go about it. They all basically come back to one idea:

Get that data in the ecosystem somehow.

Use another atom

const externallyPopulatedAtom = atom('externallyPopulated', null)

const bigListAtom = ion('bigList', ({ get }) => {
  const initialState = get(externallyPopulatedAtom)
  const store = injectStore(initialState)
  ...
})

function SomeTopLevelComponent() {
  const ecosystem = useEcosystem()

  useBigListFetcher(
    bigList => ecosystem.getInstance(externallyPopulatedAtom).setState(bigList)
  )
  ...
}

Set state after init

const bigListAtom = atom('bigList', () => {
  const isInitializedRef = injectRef(false)
  const store = injectStore([])

  return api(store).setExports({
    init: initialState => {
      store.setState(initialState)
      isInitializedRef.current = true
    }
  })
})

function SomeTopLevelComponent() {
  const bigListInstance = useAtomInstance(bigListAtom)

  useBigListFetcher(bigList => bigListInstance.exports.init(bigList))
  ...
}

Note that this can usually be even simpler. I'll put that example towards the end.

Use ecosystem context

Ecosystems have an optional .context property that can hold anything. I don't really recommend this approach since it isn't type safe, but it can be a useful escape hatch sometimes.

// With an ecosystem created like this:
const ecosystem = createEcosystem({
  context: { bigList: null },
  id: 'root',
})

// you can mutate and read from `ecosystem.context` at will
const bigListAtom = ion('bigList', ({ ecosystem }) => {
  const store = injectStore(ecosystem.context.bigList)
  ...
})

function SomeTopLevelComponent() {
  useBigListFetcher(bigList => {
    ecosystem.context.bigList = bigList
  })
}

This could also work with your store suggestion - create a store outside the atom with any state, put the store on the context, and grab it in bigListAtom.

If you do find this approach useful, let me know. I'm considering potentially deprecating ecosystem context and removing it in the future.

Note on Timing and Errors

None of these examples included a way to ensure that normal consumers of bigListAtom only use it after the list has been populated. There are several ways to go about it. Some examples:

  • Suspend in a top-level component until the list is populated.
  • Attach a promise directly to bigListAtom to suspend all components that use it.
  • Throw an error in bigListAtom if any consumers try using it before the list is populated to alert yourself of new timing problems that need manual adjusting.
  • Allow bigList to be empty and handle that default in all consumers.
  • Force consumers to access bigListAtom through an atom selector.

We pretty much always use the last two options.

Handle empty value

This is the most common way to deal with timing issues - make consumers handle an empty default. They'll reactively update when the data does come in anyway. This is usually very easy with arrays since consumers can handle an empty array exactly the same as a non-empty array.

So here's a full example of the most common way to initiate an atom with complex external data:

const bigListAtom = atom('bigList', [])

function SomeTopLevelComponent() {
  const bigListInstance = useAtomInstance(bigListAtom)

  useBigListFetcher(bigList => bigListInstance.setState(bigList))
}

function SomeConsumerComponent() {
  const bigList = useAtomValue(bigListAtom)

  // works whether the list has been populated yet or not:
  return bigList.map(item => <SomeItemComponent key={item.id} item={item} />)
}

Usually this is all you need.

Access via a selector

Consider this a postscript. This is an advanced pattern and may be overkill for you. If your question is already answered, ignore this. That said, we've started using this pattern in a few places and really like it. The gist:

  • Force all consumers to access the atom via an atom selector
  • Since selectors are plain functions, you can call them directly with ecosystem._evaluationStack.atomGetters as the first arg. Do that to initialize the atom.
  • All other consumers use select/similar without passing the initial state
// don't export the atom:
const bigListAtom = atom('bigList', ...)

// only export this atom selector:
export const getBigListInstance = ({ ecosystem, getInstance }: AtomGetters, initialState?: BigList) => {
  if (!initialState) {
    // make sure the instance already exists so the `getInstance` call doesn't create it
    if (!ecosystem.find(bigListAtom)) throw new Error("can't access bigList before init")

    return getInstance(bigListAtom)
  }

  // If it hasn't been initialized, initialize it
  const instance = getInstance(bigListAtom)
  instance.exports.init(initialState)
  return instance
}

// then initialize the instance like so:
getBigListInstance(ecosystem._evaluationStack.atomGetters, bigList)

// and all other consumers access it without passing the arg. E.g. in React components:
useAtomSelector(getBigListInstance) // to get the instance
useAtomValue(useAtomSelector(getBigListInstance)) // to subscribe to the value
decoursin commented

@bowheart thank you very much for your very wonderful detailed description. It introduces some new ideas for me, and confirms others I wasn't totally certain about.

Firstly, if you're running into this frequently, you're probably underutilizing Zedux. In an ideal setup, that big nested list would already be controlled by Zedux - e.g. it would be fetched/created in another atom rather than in a React component. The new atom could then simply access the list via an injector or atom getter. This setup also allows the new atom to react to changes in the list rather than taking a static snapshot on init.

I'm actually using SSR with a very popular react meta framework. I'm fetching the data first on the server side, then passing it down through the client side as props. This is a common pattern that I'll be repeating throughout my app. For a lot of reasons, it's better to request the data immediately on the server side, rather than waiting for the client to request it inside an atom.

I realize that Zedux supports SSR itself to a certain extent. Therefore, theoretically, I would maybe be able to instantiate the atom on the server side, then pass it down to the client somehow using Zedux' SSR capabilities by dehydrating and then rehydrating the ecosystem, but I don't think that's a very good/clean solution especial the ease and convenience of just passing the data down to the client as props.

Your two suggestions "Use another atom" and "Set state after init" are nice, but they come with a ton of other problems which you outlined fabulously yourself. Actually, your "Use ecosystem context" suggestion would maybe be the best approach, but it doesn't seem that stable according to you (ha) and for other reasons like issues with type safety like you suggested and probably more difficulty debugging, etc.

I'm so far in love with this library. But is there any functional reason why you don't want to add someway for users to pass additional data to the atom that isn't part of the hashing? I realize it would maybe require a large refactor or something, but it seems to me like it would be something feasible and have many uses cases.

bowheart commented

@decoursin Ah I was approaching this from a completely different angle. With SSR, it sounds like Zedux's hydration is what you want then. You don't have to pair it with dehydration on the server - it sounds like your framework already does that.

Exactly how that looks depends on your setup. Without knowing that this is kind of a shot in the dark, but I know that in Next 15, you can call ecosystem.hydrate with retroactive: false directly in a client component function body. Just specify the atom key(s) you want to hydrate directly:

'use client'

const bigListAtom = atom('bigList', [])

function MyClientComponent({ bigList }) {
  const ecosystem = useEcosystem()

  // hydrating right here may be fine depending on setup:
  ecosystem.hydrate(
    { [bigListAtom.key]: bigList },
    { retroactive: false }
  )

  // now access bigListAtom
  const listFromAtom = useAtomValue(bigListAtom)

  return <div>Hydrated {listFromAtom.length} values</div>
}

Actually, your "Use ecosystem context" suggestion would maybe be the best approach

It's just that it's unnecessary. If you're mutating ecosystem context, you have access to the ecosystem and can be updating the state of an atom instead:

ecosystem.context.bigList = bigList // not type-safe, not trackable, not reactive
ecosystem.getInstance(bigListAtom).setState(bigList) // type-safe, trackable, reactive
decoursin commented

I've tried to setup the SSR using dehydration/hydration, and it gets really complex fast. It's definitely not the direction that would make sense to go.

I would imagine you haven't done this before, because if you have, you know how complex it gets.

For starters, moving my http request into an atom results in a Query Atom. Query Atom cause additional complexities.

Query Atoms don't return pure data, they return PromiseState, which isn't actually a promise. PromiseState complies with the React's Suspense, but you can't await on it or perform other promise-like operations.

Now imagine you have a Query Atom on the server that you want to await on (because other data/queries/computation depend on it before you respond to the client request). How can you do it? As far as I can see, you can't (because you can't await on a PromiseState). If you want to, everything has to become part of the Zedux Atom universe which is no-go from the start. At this point Zedux would no longer be a library, but a "framework" where everything has to be done in the framework otherwise it's not possible.

Query Atoms return PromiseState, PromiseState complies with Suspense, but Server Components don't work with Suspense. Suspense is fundamentally meant to provide React with the capabilities of running concurrently inside Browsers - it's not meant for Server Components.

Besides the problems causes by Query Atoms, you have all the hydration/dehydration code that becomes quite a mess. Hydrating on the server, dehydrating on the client, what should be "retroactive" what shouldn't. Performance penalties of hydration? Are my atoms serializable? Debugging issues. etc.

I'm not trying to bash Zedux. Once again, I find Zedux to be amazing. But I personally think Zedux is a paradigm meant for client code / Client Components, it's not meant for server code / Server Components. The entire reactivity paradigm with "molecules" or "atoms" is a frontend/client paradigm, not a server paradigm. Once you start using Atoms on the server, half of the stuff doesn't even work anymore (at least for react it's so) for good reason, like useEffect, useMemo (etc) none of these things work or make sense on the server and likewise I'm assuming injectEffect, injectMemo (etc) don't work or make sense either.

Furthermore, once you have a codebase mixing server-side atoms with client-side atoms things start to become really complex and hard to grasp. Spaghetti code. It's like mixing Christmas in the Winter and Christmas in the Summer - it doesn't make sense / it's not consistent.

All of this in comparison to just running my http request inline in TS then passing the data down to the clients as props. This is obviously far far easier and more maintainable. The other solutions that you offered are definitely better in my opinion than trying to do SSR with atoms.

bowheart commented

TL;DR

Only use Zedux in client components. If there's a demand for using Zedux in RSCs, it's possible but we need better docs.

Full Answer

imagine you have a Query Atom on the server that you want to await on (because other data/queries/computation depend on it before you respond to the client request). How can you do it?

@decoursin Query atoms set a .promise on the atom instance you can await in an RSC like so:

'use server'

export async function MyServerComponent() {
  const queryInstance = serverEcosystem.getInstance(myQueryAtom)
  await queryInstance.promise

  return <MyClientComponent data={queryInstance.getState().data} />
}

But before we get too caught up in that approach, we should come back to this:

Zedux on the client

Zedux is a paradigm meant for client code / Client Components, it's not meant for server code / Server Components

Yes! Exactly this. The mess you're seeing is due to how frameworks (especially nowadays with RSCs) work. It isn't specific to Zedux - it would be the same with any 3rd party state manager. This complexity is why most people using frameworks avoid 3rd party state managers completely.

That may be my advice for you too. If Zedux isn't solving a problem or making things easier for you, you probably don't need it. That said, I do use Zedux in Next. Let me reiterate this:

You don't have to pair it with dehydration on the server - it sounds like your framework already does that.

Contrary to what I said before when I was assuming a client-only setup (which is where Zedux and all 3rd party state managers shine), it now sounds to me like you're trying to fight the framework by overutilizing Zedux. Your framework already has RSC support. I'd use that, not query atoms in Zedux.

I believe frameworks could be much more powerful if they integrated more deeply with 3rd party state managers. Unfortunately they don't. That means lots of manual work if you want to use Zedux (or any similar lib) on both the server and the client.

The fix is simple in theory, though will take a while to get used to (which is just RSCs in a nutshell and again isn't Zedux's fault, it's the same for any 3rd party state manager): Only use 3rd party state managers on the client.

I really only know Next. I'll try to make some time to experiment with Remix and Waku. But in Next at least, the server counterpart to my hydration example is simply:

'use server'

export async function MyServerComponent() {
  const bigList = await fetchBigList() // no Zedux involved

  return <MyClientComponent bigList={bigList} />
}

If you don't want to worry about retroactive: false, you can create a custom hook like this that covers all cases:

const useAtomHydration = <T extends AnyAtomTemplate>(
  atomTemplate: T,
  val: AtomStateType<T>
) => {
  const ecosystem = useEcosystem()
  const existingAtom = ecosystem.find(atomTemplate)

  // hydrate immediately if the atom doesn't exist yet (ideal)
  if (!existingAtom) {
    ecosystem.hydrate({ [atomTemplate.key]: val })
  }

  // defer to a useEffect to set ("hydrate") the existing atom's state safely
  useEffect(() => {
    if (existingAtom) existingAtom.setState(val)
  }, [val]) // don't add `existingAtom` - it isn't reactive (and shouldn't be)
}

Then use like so:

function MyClientComponent({ bigList }) {
  useAtomHydration(bigListAtom, bigList)
}

Zedux on the Server

While I'd avoid using Zedux on the server in general, its advanced cache management does make it a suitable replacement for Next's unstable_cache and React's cache function. Atoms can also be flagged as either global or temporal/route-specific and their lifecycles can be controlled per-RSC.

This requires quite a bit of setup. Depending on the framework/compiler, there may be manual work you'd have to remember to do per-RSC too. But at least in Next when not using Turbopack, TypeScript's using keyword is amazing to prevent manual cleanup. I'm not sure we have enough documentation for me to encourage it, but it's possible.

If people are interested in going down this route, then we should improve the docs.

Some of your confusion is probably from the SSR guide in the docs. That's a low-level guide for doing SSR from scratch with no framework. It's probably misleading. We should bury that guide more and replace that with a guide for people using frameworks since that's way more common.

Additionally, since frameworks are so common in general, we should probably add notes throughout many doc pages of APIs that are only designed for use on the client.

decoursin commented

Hi @bowheart thank you again for your detailed description. I see that I made a couple mistakes, but I also see that we agree on a lot of what I said - so that's good:)

I will have to play around with your useAtomHydration function, which I'll definitely get to later this week. Haha but my first impression still is that it's still more complex than if it were possible to simply pass in the initial state as an argument into the atom state factory function.

I really appreciate all the time you've given me on this.

decoursin commented

I've looked into all the options, and for the problem at hand I'll actually be using @ChrisBoon's solution. Thank you very much Chris! @bowheart thanks again for the wonderful library. I hope to contribute back in the future!

  • 🚀1
decoursin commented

Hey @bowheart, I've been playing around with the hydration option more for populating atom's state for the first time. It would seem like it could maybe be a good solution, but I've seen to come across a couple problems.

First, let me clarify how I'm hydrating. I have a top client component which receives props sent from a nextjs server component. The top client component looks something like this:

export default function TopClientComponent({bigList}) {
  const ecosystem = useMemo(() => createEcosystem({id: 'account-page-root'}), [])
  ecosystem.reset() // always reset the ecosystem to avoid nextjs hydration errors (is there a better way?)
  ecosystem.hydrate({ [someAtom.key]: bigList })
}

Where my someAtom might look like this:

export const someAtom = atom('some', () => {
  const store = injectStore<Set<string>>(hydration => {
    return createStore(hydration)
  }, { hydrate: true })
  return api(store)
}, {
  hydrate: (args) => {
     // modify args then return new args
    return Set<string>(args)
  }
});

Now this was working fine actually, and I was even thinking about adopting this full-on and using this instead of the function as argument approach.

However, this hydration stuff doesn't seem to work at all when used in combination with composed stores (ie. store.use(..)).

First of all, store.use(..) is pretty clumsy, which it seems you acknowledge (which is good). However, hydration only works with injectStore, and yet the function response of injectStore must be values of KnownHierarchyDescriptor in order to satisfy the requirements of store.use. Furthermore, the return value of the hydrate function doesn't even make it into injectStore when the output of that injectStore is used in a store.use(..). It seems like hydration and composing stores just doesn't fundamentally work. Can you confirm this?

I have a lot of export functions in my biglist atom, and I would like these export functions to use values from stores located/created in other atoms. Therefore, I'm confused what would be the best approach for this. To solve this, I've gone down the path of or am rather currently attempting to use injectAtomInstance(..).store in order to be able to use the values of external stores inside my export functions, something like this:

export const someAtom = atom('some', () => {
  const external = injectAtomInstance(external1Atom).store
  const store = injectStore<Set<string>>(hydration => {
    return createStore(hydration)
  }, { hydrate: true })

  const awesomeExportFN = () => {
     const { someExternalValue } = external.getState()
     // use someExternalValue ...
  }

  return api(store).exports({ awesomeExportFN })
}, {
  hydrate: (args) => {
     // modify args then return new args
    return Set<string>(args)
  }
});

It would seem like there are different possible solutions for this, but I'm trying to figure out the best one or rather understand the trade-offs between them. Other possible solutions to this problem would entail:

  • passing the someExternalValue as an argument to awesomeExportFN, which would get annoying once I have tons of these passed-in external values because I would also have to pass them to all the react components that will invoke that function and any other function that wants to use those external values.
  • using const { get } = injectAtomGetters() at the top of the atom, and then using the const { someExternalValue } = get(external1Atom) inside every external function that wants to use those external values. This could work but I thought perhaps there could be a better way with composing store?

Thank you very much for any insight. I see that you're working on signals which would seem to replace stores, and that seems very exciting.

bowheart commented

Hey @decoursin. I am indeed working on the signals implementation and I see so many reasons why I wish it was done already 😄

// always reset the ecosystem to avoid nextjs hydration errors (is there a better way?)

There might be. I'd need more context of when you're running into hydration errors (on load, on redirect, when streaming, something else?) and how often the ecosystem gets reset and when. Reset might be fine, depending. Just understand that that's destroying all atoms and letting them be recreated from the leaf nodes (e.g. nodes used directly in React components). If it's possible to be more granular - only resetting specific atoms - that's usually better, but also may be much harder.

It seems like hydration and composing stores just doesn't fundamentally work. Can you confirm this?

There are several quirks with hydrating composed stores as you saw in the signals doc. Fundamentally though, I'd say it works. Could you provide an example of it not working? Dropping this code in the first sandbox of the quick start doc works:

const childAtom1 = atom('child1', () => 1)
const childAtom2 = atom('child2', () => ({ b: 2 }))

const parentAtom = atom('parent', () => {
  const childInstance1 = injectAtomInstance(childAtom1)
  const childInstance2 = injectAtomInstance(childAtom2)

  const store = injectStore(
    hydration =>
      createStore(
        { child1: childInstance1.store, child2: childInstance2.store },
        hydration
      ),
    { hydrate: true }
  )

  // ensure child store references stay up-to-date
  store.use({ child1: childInstance1.store, child2: childInstance2.store })

  return store
})


function Greeting() {
  const ecosystem = useEcosystem()

  useMemo(() => {
    ecosystem.hydrate({ [parentAtom.key]: { child1: 11, child2: { b: 22 } } })
  }, [])

  const [val, setVal] = useAtomState(parentAtom)

  return (
    <div>
      child1: {val.child1} child2.b: {val.child2.b}{' '}
      <button onClick={() => setVal(state => ({ ...state, child1: state.child1 + 1 }))}>
        increment a
      </button>
    </div>
  )
}

Can you show me how your approach differs from this? The one thing I see, though it may just be due to you writing pseudo-code here, is that the initial state should be the second argument to createStore. That's confusing, I know, since initial state is the first argument to injectStore.

I have a lot of export functions in my biglist atom, and I would like these export functions to use values from stores located/created in other atoms. Therefore, I'm confused what would be the best approach for this.

Your last bullet is how we do this; use either an atom getter or the composed store inside the exported function.

const external = injectAtomInstance(external1Atom).store

const store = injectStore(hydration => createStore({ external }, hydration), { hydrate: true })

store.use({ external })

const awesomeExportFN = () => {
  get(external1Atom) // good - atom getters and the atom template reference are stable
  store.getState().external // good - `injectStore` returns a stable store reference
  external.getState() // bad - injected atom instances aren't technically stable references. `external` and its store could change if force-destroyed, so they shouldn't be used in atom exports
}
decoursin commented

Hey @decoursin. I am indeed working on the signals implementation and I see so many reasons why I wish it was done already 😄

haha

There might be. I'd need more context of when you're running into hydration errors (on load, on redirect, when streaming, something else?) and how often the ecosystem gets reset and when. Reset might be fine, depending. Just understand that that's destroying all atoms and letting them be recreated from the leaf nodes (e.g. nodes used directly in React components). If it's possible to be more granular - only resetting specific atoms - that's usually better, but also may be much harder.

Well it seems the nextjs server has a copy of all the atoms sitting in memory. When some of the atoms on the client change, these changes are not being propagated back to the nextjs server, the changes remain on the client, so there's then a misalignment when the client reloads the page. However, the truth is is that I'm also quite new to nextjs, I'm more of a backend developer to be honest, and I'm starting to see the need to use suspense or use-effect to resolve the hydration errors. I was trying to get around using use-effect (or suspense) because that will slow down the time to First Meaningful Paint. But anyways this is my problem ;) not yours haha but thanks for your input - very helpful.

There are several quirks with hydrating composed stores as you saw in the signals doc.

Yeah, I didn't completely read that through but I see them now.

Can you show me how your approach differs from this? The one thing I see, though it may just be due to you writing pseudo-code here, is that the initial state should be the second argument to createStore. That's confusing, I know, since initial state is the first argument to injectStore.

Yeah, that was just a mistake in my pseudo-code, but definitely a mistake I've made a lot too 😄😄

Your last bullet is how we do this; use either an atom getter or the composed store inside the exported function.

Perfect, that's really what I needed to hear. Thank you. I'm doing the following now. I'm have this getExternal function at the top of my atom, and I'll invoke that in my export functions:

export const someAtom = atom('some', () => {
  const { get } = injectAtomGetters()

  const getExternal = () => {
    return {privacies: get(privacyFilterAtom), search: get(searchFilterAtom), ... others}
  }

  const exportFN = () => {
     const {privacies, search} = getExternal()
     ...
  }

  ...
})

There are several quirks with hydrating composed stores as you saw in the signals doc. Fundamentally though, I'd say it works. Could you provide an example of it not working? Dropping this code in the first sandbox of the quick start doc works:

Yeah, that does seem to work, that's cool, that far was working for me too actually. What wasn't working for me was using that in tandem with the hydrate function in the AtomConfig being the 3rd parameter to the Atom State Factory Function. That hydrate function didn't seem to have any effect on changing the initial hydration. But I really don't think we need to spend any more time on this issue of trying to hydrate with composed stores, I (or we) have a great solution with this injecting atom getters like I have with getExternal, and the future is in Signals anyways!! 🚀🚀🚀🤩