Computed property best practices

[noahw3]

One of the problems that I've had a hard time finding a solution for with Valtio is computed or derived state based on existing state. In short, I'm looking for something similar to MST's views. The primary requirements are:

• It should only be recomputed when the upstream data used in the view is changed.
• Access should be O(1) unless it needs to be recomputed (ie there isn't an expensive "do I need to recompute" function)
• The computed value itself can be snapshotted and used as if it was normal state on the proxy

I've played around with valtio getters, with proxy-memoize, with other memoization libraries, tried to make listen with subscriptions and I'm struggling to find something that is both correct and performant. I imagine I'm not the first person with these needs who's tried to use Valtio, so I'm curious if I'm missing something obvious or if there is any way to achieve this goal.

I tweaked the introductory example on the Valtio website to show the pattern that I'm attempting to accomplish. The base state is made up of an array of Todos. I'd like to create a mapping of id -> Todo that's computed from the array so that I can do efficient lookups for any given Todo given the id. In this example I'm using the prescribed getter-with-proxy-memoize pattern. However, there are two primary flaws with this approach:

1. It has to snapshot the state every time it does a lookup. If this is a large, nested data model, this isn't free
2. Any time any property in one of the Todos changes, it forces the map to be regenerated. I only want the map to be regenerated if a reference changes (item is added or deleted), and again if there is significant state (or nested state) then that map is going to be regenerated constantly. In the example I added a "count" property that isn't used or rendered anywhere, and when it changes the recomputation still occurs.

[dai-shi]

Thanks for opening up the discussion.
Computeds are pretty difficult in valtio, and we basically don't support it natively.

Your requirements sounds reasonable, but tough. I think Valtio getters is clean and close. But, you would always need to use snapshots to read the property. And the second requirement isn't fully fulfilled.

https://github.com/valtiojs/derive-valtio is another option, but it fulfill less requirements as of now. I think we need to overhaul it.

I'm interested in any solutions outside the core, as it's one of the missing features sometimes requested.

[noahw3]

Yeah I looked at derive-valtio a bit. The theory is much closer to what I'm looking for, but the fact that it recomputes if any property on the base proxy changes is a dealbreaker.

[dai-shi]

I'm actually fine to change or overhaul derive-valtio. I think we can implement it with a better design.

[marbemac]

I'm looking for solutions to this use case as well - @noahw3 did you end up finding a good option?

@dai-shi - is this pattern easier to achieve with Jotai or Zustand (thus far I have only used Valtio)?

[dai-shi]

For React use case, I believe object getters work for most case. Have you tried it before?

valtio/readme.md

Lines 300 to 315 in e21edb3

	#### Computed properties
	You can define computed properties with object getters.
	```js
	const state = proxy({
	count: 1,
	get doubled() {
	return this.count * 2
	},
	})
	```
	Consider it as an advanced usage, because the behavior of `this` is sometimes confusing.
	For more information, check out [this guide](./docs/guides/computed-properties.mdx).

For vanilla use case, it's not sufficient and I'm planning to create a new third-party library, but no ETA.

Jotai is very good at deriving state as it forms data flow graph. Zustand doesn't have a capability for computeds but there are some (experimental?) third-party libraries.

[noahw3]

It took a few tries, but we actually created our own version internally that's very similar to derive-valtio but covers the use cases I listed above. I wanted to get a couple more months of real world usage and testing done before opening the conversation of contributing it back (I know @dai-shi mentioned being open to reworking derive-valtio), but if you're interested I'm happy to share what we have.

[marbemac]

@dai-shi ah no I understand the basic computed functionality via getters - I'm referring more specifically to the use case @noahw3 outlined in the original discussion post. The primary use case is lookup tables e.g. storing an array of Todo[] and want to be able to retrieve a todo by ID with o(1) perf (at least after first lookup).

The closest equivalent I've found in the ecosystem is Mobx computedFn (https://github.com/mobxjs/mobx-utils?tab=readme-ov-file#computedfn). @noahw3 if you're willing, I'd love to see it!

[dai-shi]

For vanilla use case, it's not sufficient and I'm planning to create a new third-party library, but no ETA.

FYI: #949 (reply in thread)

It will be different from computedFn, but I hope it will be something similar.

[dai-shi]

https://github.com/valtiojs/valtio-reactive

[noahw3]

Ah I misremembered, it actually is built on top of derive-valtio.

The three primary limitations of this solution:

• Cannot use computed props in other computed props
• Can't add computed props to any proxy cause once you snapshot that proxy it computes all properties. So we've ended up passing around an object that wraps the proxy as one property and the computed properties as another property.
• Snapshots need to be taken of the individual computed properties rather than from the entire computed object, otherwise it'll recompute all computed properties.

Definitely happy to receive feedback or have holes poked in it.

import { unstable_deriveSubscriptions as deriveSubscriptions } from "derive-valtio";
import { getVersion, proxy } from "valtio";

type ComputedGet = <T extends object>(proxyObject: T) => T;

type Subscription = Parameters<typeof deriveSubscriptions.add>[0];

/**
 * computed
 *
 * This creates computed properties and attaches them
 * to a new proxy object. Computed properties are ONLY computed when accessed
 * and are not re-computed unless the dependencies change.
 *
 * @example
 * import { proxy } from 'valtio'
 *
 * const state = proxy({
 *   count: 1,
 * })
 *
 * const computedState = computed({
 *   doubled: (get) => get(state).count * 2,
 * })
 * // Only called when accessed
 * computedState.doubled
 */
export function computed<U extends object>(
  derivedFns: {
    [K in keyof U]: (get: ComputedGet) => U[K];
  },
  options: { sync?: boolean } = { sync: true },
) {
  // All of this is from and copied from derive-valtio:
  // https://github.com/valtiojs/derive-valtio/blob/9abb907cdbc26fa668e55ffc9a863b87cbe6d0f3/src/derive.ts
  // The changes that have been made are:
  //  - Lazy compute the getters and clear the cache when dependencies change
  //  - Cache the response of the computed function
  //  - remove the cache entry when dependencies change
  const proxyObject = proxy({}) as U;
  const notifyInSync = !!options?.sync;
  const derivedKeys = Object.keys(derivedFns);
  const _cache = new Map<string, unknown>();
  derivedKeys.forEach((key) => {
    if (Object.getOwnPropertyDescriptor(proxyObject, key)) {
      throw new Error("object property already defined");
    }
    const fn = derivedFns[key as keyof U];
    interface DependencyEntry {
      v: number; // "v"ersion
      s?: Subscription; // "s"ubscription
    }
    let lastDependencies: Map<object, DependencyEntry> | null = null;
    const evaluate = () => {
      if (lastDependencies) {
        if (
          // no dependencies are changed
          Array.from(lastDependencies).every(
            ([p, entry]) => getVersion(p) === entry.v,
          )
        ) {
          return _cache.get(key);
        }
      }

      const dependencies = new Map<object, DependencyEntry>();

      const value = fn(<P extends object>(p: P) => {
        dependencies.set(p, { v: getVersion(p) as number });
        return p;
      });
      const subscribeToDependencies = () => {
        dependencies.forEach((entry, p) => {
          const lastSubscription = lastDependencies?.get(p)?.s;
          if (lastSubscription) {
            entry.s = lastSubscription;
          } else {
            const subscription = {
              s: p, // sourceObject
              d: proxyObject, // derivedObject
              k: key, // derived key
              c: () => _cache.delete(key), // callback
              n: notifyInSync,
              i: derivedKeys, // ignoringKeys
            };
            deriveSubscriptions.add(subscription);
            entry.s = subscription;
          }
        });
        lastDependencies?.forEach((entry, p) => {
          if (!dependencies.has(p) && entry.s) {
            deriveSubscriptions.remove(entry.s);
          }
        });
        lastDependencies = dependencies;
      };

      if (value instanceof Promise) {
        value
          .then((v) => {
            _cache.set(key, v);
            return v;
          })
          .finally(subscribeToDependencies);
      } else {
        _cache.set(key, value);
        subscribeToDependencies();
      }

      return _cache.get(key);
    };

    Object.defineProperty(proxyObject, key, {
      get: () => evaluate(),
      set: () => {
        throw new Error("Cannot set computed properties!");
      },
    });
  });

  return proxyObject as U;
}

Tests

import { proxy } from "valtio";
import { computed } from "./valtio";

it("Only computes when accessed", async () => {
  const proxyState = proxy({ count: 1 });

  const doubleMock = vi.fn((get) => get(proxyState).count * 2);
  const tripleMock = vi.fn((get) => get(proxyState).count * 3);

  const state = computed({
    double: doubleMock,
    triple: tripleMock,
  });

  expect(doubleMock).not.toHaveBeenCalled();
  expect(tripleMock).not.toHaveBeenCalled();
  expect(state.double).toBe(2);
  expect(state.triple).toBe(3);
  expect(doubleMock).toHaveBeenCalled();
  expect(tripleMock).toHaveBeenCalled();
  expect(state.double).toBe(2);
  expect(state.triple).toBe(3);
  expect(doubleMock).toHaveBeenCalledTimes(1);
  expect(tripleMock).toHaveBeenCalledTimes(1);

  ++proxyState.count;

  expect(doubleMock).toHaveBeenCalledTimes(1);
  expect(tripleMock).toHaveBeenCalledTimes(1);
  expect(proxyState.count).toBe(2);
  expect(state.double).toBe(4);
  expect(state.triple).toBe(6);
  expect(doubleMock).toHaveBeenCalledTimes(2);
  expect(tripleMock).toHaveBeenCalledTimes(2);
});

it("Only computes when accessed while adding properties to an existing proxy", async () => {
  const proxyState = proxy({ count: 1 });
  const mock = vi.fn((get) => get(proxyState).count * 2);

  const state = computed({
    double: mock,
  });

  expect(mock).not.toHaveBeenCalled();
  expect(state.double).toBe(2);
  expect(mock).toHaveBeenCalled();
  expect(state.double).toBe(2);
  expect(mock).toHaveBeenCalledTimes(1);

  ++proxyState.count;
  expect(mock).toHaveBeenCalledTimes(1);
  expect(proxyState.count).toBe(2);
  expect(state.double).toBe(4);
  expect(mock).toHaveBeenCalledTimes(2);
});

it("Works for nested objects", async () => {
  const proxyState = proxy({
    count: 1,
    other: {
      count: 1,
    },
  });
  const mock = vi.fn(
    (get) => get(proxyState).count * 2 + get(proxyState).other.count * 3,
  );

  const state = computed({
    computed: mock,
  });

  // Computed properties are lazy
  // They don't get called until they are accessed
  expect(mock).not.toHaveBeenCalled();
  // Accessing the computed property for the first time
  expect(state.computed).toBe(5);
  // The computed function has been called
  expect(mock).toHaveBeenCalledTimes(1);
  // Accessing the computed property again gives the same result as before
  expect(state.computed).toBe(5);
  // The computed function has not been called again
  expect(mock).toHaveBeenCalledTimes(1);

  ++proxyState.other.count;

  // Don't recompute when underlying data changed, only when called again
  expect(mock).toHaveBeenCalledTimes(1);
  // Value has been updated.
  expect(proxyState.other.count).toBe(2);
  // New computed value should be executed here
  expect(state.computed).toBe(8);
  // Computed method gets called again
  expect(mock).toHaveBeenCalledTimes(2);
});

Usage:

const _createTodosMap = (
  store: Store,
) =>
  computed({
    todosMap: (get): Record<string, Todo> =>
      Object.fromEntries(get(store).todos.map((todo) => [todo.id, todo])),
  });

export const getStore = () => {
  const store = proxy<Store>({
    filter: "all",
    todos: [],
  });
  const map = _createTodosMap(store);

  return {
    store,
    map,
  };
}

export const useTodo = (id: string) => {
  const store = useStoreContext();
  const { todosMap } = useSnapshot(store.map);
  return todosMap[id];
};