Dataloader.KV runs once, never again

[alex-knowles]

Summary

• Version: Dataloader 1.0.6
• Expected behavior: You can call Dataloader.run/1, then Dataloader.load/4 with new items and then Dataloder.run/1 again to fetch results for the new items
• Observed behavior: When Dataloader.KV is the source, the second call to Dataloader.run/1 does not fetch results for new items

Details

It's possible that the :load_function I'm passing to Dataloader.KV.new/1 is flawed.

However, it should be easy to reproduce this behavior with a proven :load_function of your own.

Try this at home

With Dataloader.KV as the source:

iex(1)> loader = Dataloader.new() |> Dataloader.add_source(Data, kv_source)
iex(2)> loader = Dataloader.load(loader, Data, kv_batch, kv_item_1)
iex(3)> Dataloader.pending_batches?(loader)
true
iex(4)> loader = Dataloader.run(loader)
iex(5)> Dataloader.pending_batches?(loader)
false
iex(6)> loader = Dataloader.load(loader, Data, kv_batch, kv_item_2)
iex(7)> Dataloader.pending_batches?(loader)
false

On the other hand, if I use Dataloader.Ecto, I get a different result:

iex(1)> loader = Dataloader.new() |> Dataloader.add_source(EctoData, ecto_source)
iex(2)> loader = Dataloader.load(loader, EctoData, ecto_batch, ecto_item_1)
iex(3)> Dataloader.pending_batches?(loader)
true
iex(4)> loader = Dataloader.run(loader)
iex(5)> Dataloader.pending_batches?(loader)
false
iex(6)> loader = Dataloader.load(loader, EctoData, ecto_batch, ecto_item_2)
iex(7)> Dataloader.pending_batches?(loader)
true

Why this matters

When you run a GraphQL query like:

query StarWarsHuman($humanID:ID!){
  human(id: $humanID) {
    name
    friends {
      name
      friends {
        name
      }
    }
  }
}

You get back:

{
  "data": {
    "human": {
      "name": "Luke Skywalker",
      "friends": [
        {
          "name": "Han Solo",
          "friends": null
        },
        {
          "name": "Leia Organa",
          "friends": null
        },
        {
          "name": "C-3PO",
          "friends": null
        },
        {
          "name": "R2-D2",
          "friends": null
        }
      ]
    }
  }
}

You can get traverse one depth level of a graph, but no further 🙁

[alex-knowles]

I wrote a test to illustrate. In commit ef5c978:

dataloader/test/dataloader/kv_test.exs

Lines 63 to 76 in ef5c978

	test "loading something not in the cache does change the loader", %{loader: loader} do
	round1_loader =
	loader
	|> Dataloader.load(Test, :users, "ben")
	|> Dataloader.run()
	round2_loader =
	round1_loader
	|> Dataloader.load(Test, :users, "bruce")
	|> Dataloader.run()
	refute round2_loader == round1_loader
	refute loader == round1_loader
	end

The test currently fails:

  1) test loading something not in the cache does change the loader (Dataloader.KVTest)
     test/dataloader/kv_test.exs:63
     Refute with == failed, both sides are exactly equal
     code: refute round2_loader == round1_loader
     left: %Dataloader{
             options: [get_policy: :raise_on_error],
             sources: %{
               Test => %Dataloader.KV{
                 batches: %{},
                 load_function: #Function<0.66469107/2 in Dataloader.KVTest.__ex_unit_setup_0/1>,
                 opts: [max_concurrency: 16, timeout: 30000],
                 results: %{
                   users: %{"ben" => [id: "ben", username: "Ben Wilson"]}
                 }
               }
             }
           }
     stacktrace:
       test/dataloader/kv_test.exs:74: (test)

Ben is in the results from round 1, but Bruce didn't make it in round 2.

[benwilson512]

Fixed by PR, thank you!