deliver broadcast messages asynchronously #41

prog-supdex · 2023-09-21T15:40:10Z

Deliver messages, which are called by method trigger, asynchronously

It`s a not final PR

palkan · 2023-09-25T02:06:22Z

graphql-anycable.gemspec

@@ -30,11 +30,13 @@ Gem::Specification.new do |spec|
  spec.add_dependency "anyway_config", ">= 1.3", "< 3"
  spec.add_dependency "graphql",       ">= 1.11", "< 3"
  spec.add_dependency "redis",         ">= 4.2.0"
+  spec.add_dependency "activejob",     ">= 5.2.0"


It shouldn't be a dependency; the gem is Rails-free, we only provide ActiveJob integration if ActiveJob is available in the runtime.

I added it to spec.add_development_dependency "activejob", ">= 6.0.0"

because otherwise, I got an error during running specs

palkan · 2023-09-25T02:08:14Z

lib/graphql-anycable.rb

@@ -6,6 +6,7 @@
 require_relative "graphql/anycable/cleaner"
 require_relative "graphql/anycable/config"
 require_relative "graphql/anycable/railtie" if defined?(Rails)
+require_relative "graphql/adapters/base_job"


Move into graphql/anycable/railtie and load only if ActiveJob is present (using ActiveSupport.on_load(:active_job) { ... } hook).

yeah, I done it, but I had an error in this case

initializer 'graphql_anycable.load_trigger_job' do ActiveSupport.on_load(:active_job) do require "graphql/adapters/trigger_job" end end

in that case, my application (rails 6, with configurated active_job) does not load it, because my application does not call any ActiveJob jobs in the code

I checked and saw, that ActiveSupport.on_load(:active_job) loads only when my rails application calls any jobs

but, maybe I misunderstood something.

Currently, I just commented ActiveSupport.on_load(:active_job) and I think, if we are in Railtie, we can require TriggerJob in initialize block

Got it.

I'm looking at how ActionMailer or ActiveStorage does this, and it seems that we can just add require "active_job/railtie" and load our job class after that.

thanks for that!

The trouble was with my test application

palkan · 2023-09-25T02:09:34Z

lib/graphql/adapters/base_job.rb

+    class BaseJob < ActiveJob::Base
+      DEFAULT_QUEUE_NAME = :default
+
+      queue_as { GraphQL::AnyCable.config.async_broadcasting["queue"] || DEFAULT_QUEUE_NAME }


The default name must be define in the configuration, no need to use a fallback here:

Suggested change

queue_as { GraphQL::AnyCable.config.async_broadcasting["queue"] || DEFAULT_QUEUE_NAME }

queue_as do

AnyCable.config.active_job_queue_name

end

Or, we go with adapter instances, we can specify the queue right when calling #perform_later:

TriggerJob.set(queue: queue).perform_later(...)

Thanks!
Done

palkan · 2023-09-25T02:10:39Z

lib/graphql/adapters/base_job.rb

+      private
+
+      def schema_parse(serialized_payload)
+        payload = JSON.parse(serialized_payload)


Maybe, we can use symbolize_names here and avoid #transform_keys?

Suggested change

payload = JSON.parse(serialized_payload)

payload = JSON.parse(serialized_payload, symbolize_names: true)

Of course, yeah. We can just put symbolized hash to ActiveJob arguments, and it will work without JSON.parse

palkan · 2023-09-25T02:13:59Z

lib/graphql/anycable/config.rb

+      attr_config use_async_broadcasting: true
+      attr_config async_broadcasting: { queue: "broadcasting", class: "GraphQL::Adapters::BaseJob" }


This is not how adapter pattern is implemented. The API must be as follows:

GraphQL::AnyCable.delivery_method = :inline GraphQL::AnyCable.delivery_method = :active_job # or with options GraphQL::AnyCable.delivery_method = :active_job, queue: "custom_queue", job_class: "MyJob"

And then, in the #trigger method we simply delegate the execution to the configured adapter instance. Smth like:

def trigger(...) AnyCable.delivery_adapter.trigger(...) end

I did it, but without "="

It works like that

GraphQL::AnyCable.delivery_method("active_job", queue: "custom_queue", job_class: "MyJob")

But, if we need like setter, we will able to call it only like that (adds { and } for options)

GraphQL::AnyCable.delivery_method = :active_job, {queue: "custom_queue", job_class: "MyJob"}

Because the setter, as I know, cannot accept keyword arguments

GraphQL::AnyCable.delivery_method = "inline", queue: "queue", job_class: "Job"

SyntaxError ((irb):7: syntax error, unexpected tLABEL) ...ble.delivery_method = 1, queue: 123, job_class: "Abc" ... ^~~~~~

if we need like setter, we will able to call it only like that

Yes, and that's how it should be done. That's a common pattern used by Rails and many libraries, We shouldn't reinvent it.

So, please, implement it using the original proposed interface.

P.S. See, for example: https://github.com/palkan/active_delivery/blob/master/lib/abstract_notifier/async_adapters.rb

mm, yea, I earlier said, why I did not do that

I fixed it, but it will work only with

GraphQL::AnyCable.delivery_method = :active_job, {queue: "custom_queue", job_class: "MyJob"}

so, we need to wrap by { and } last arguments

palkan · 2023-09-25T02:16:34Z

lib/graphql/subscriptions/anycable_subscriptions.rb

+        end
+
+        executor_class_job.perform_later(
+          serialized_arguments,


We should not serialize arguments; it's the responsibility of Active Job; we may assume that arguments are GlobalID identifiable, so we can pass them as is.

yea, currently I do not serialize them, but I still collected them (arguments)

palkan · 2023-09-25T02:17:06Z

lib/graphql/subscriptions/anycable_subscriptions.rb

+
+        return Adapters::BaseJob unless custom_class
+
+        Object.const_get(config.async_broadcasting["class"])


The job class is inferred in Rails context where we can use #safe_constantize

Thanks!
But for job_class, I put #constantize instead of #safe_constantize, because I think that #constantize will be useful for cases when the user puts anything wrong to param job_class.
And constantize will raise an error and we will see it immediately

But, if I am wrong, I will fix it

palkan · 2023-09-25T02:17:46Z

lib/graphql/subscriptions/anycable_subscriptions.rb

+        Object.const_get(config.async_broadcasting["class"])
+      end
+
+      def perform(event, object)


Where do we use this method?

It was pre-alfa version of the code, sorry :)

palkan · 2023-09-25T02:21:10Z

spec/adapters/base_job_spec.rb

+require "active_job"
+
+RSpec.describe GraphQL::Adapters::BaseJob, type: :job do
+  ActiveJob::Base.queue_adapter = :inline


Never define configuration outside of before and after hooks, and always restore the value after each test. Otherwise it's easier to introduce unpredictable global state and flakiness.

Thank you for the advice!

prog-supdex · 2023-09-27T17:51:31Z

lib/graphql/adapters/delivery_adapter.rb

+        @executor_method = executor_object.class::EXECUTOR_METHOD_NAME
+      end
+
+      def trigger(...)


I use here "...", but It requires "ruby-2.7".

I checked ci-cd, and I saw minimal ruby versions - 2.7, but if we look at the documentation (in anycable-rails, because it is a dependency), I see the requirement for ruby-version 2.6

@palkan please tell me, do we need to consider ruby 2.6 and avoid using "..." ?
If yes, I will fix it

palkan · 2023-09-27T16:17:56Z

lib/graphql-anycable.rb

@@ -6,6 +6,7 @@
 require_relative "graphql/anycable/cleaner"
 require_relative "graphql/anycable/config"
 require_relative "graphql/anycable/railtie" if defined?(Rails)
+require_relative "graphql/adapters/base_job"


Got it.

I'm looking at how ActionMailer or ActiveStorage does this, and it seems that we can just add require "active_job/railtie" and load our job class after that.

palkan · 2023-09-28T00:40:47Z

lib/graphql/adapters/delivery_adapter.rb

@@ -3,31 +3,19 @@
 module GraphQL
  module Adapters
    class DeliveryAdapter


The goal of adapter patter is to provide extensibility. Moving all different implementations into a single file and calling it an "adapter" doesn't make it an adapter.

Initially, I didn`t understand what was needed, but I now got it.

Now we can easily add any other adapter with his logic and add it in config.delivery_method

Thanks for the lesson

palkan · 2023-09-28T01:17:38Z

lib/graphql/anycable/config.rb

+      attr_config use_async_broadcasting: true
+      attr_config async_broadcasting: { queue: "broadcasting", class: "GraphQL::Adapters::BaseJob" }


if we need like setter, we will able to call it only like that

Yes, and that's how it should be done. That's a common pattern used by Rails and many libraries, We shouldn't reinvent it.

So, please, implement it using the original proposed interface.

P.S. See, for example: https://github.com/palkan/active_delivery/blob/master/lib/abstract_notifier/async_adapters.rb

prog-supdex · 2023-10-04T17:34:16Z

@palkan hi!

I sum up the changes

Currently, we have two adapters, "Inline" and "ActiveJob" in two files

The specific adapter is chosen by DeliveryAdapter.lookup and settings in the configuration (delivery_method)

Also, we have a custom serializer AnyCableSubscriptionSerializer which helps do not think about an instance of class AnyCableSubscriptions in active jobs, for example here

But for example here I used the arguments forwarding shorthand, which exists in ruby 2.7 (I see that ci-cd checks on ruby 2.7 and upper), but this gem has a dependency anycable-rails, which requires minimum 2.6 ruby

Maybe here I need not to use the arguments forwarding shorthand

prog-supdex marked this pull request as draft September 21, 2023 15:40

prog-supdex force-pushed the async_triggers branch from d686343 to f08db8e Compare September 21, 2023 15:46

palkan requested changes Sep 25, 2023

View reviewed changes

palkan reviewed Sep 25, 2023

View reviewed changes

prog-supdex marked this pull request as ready for review September 27, 2023 07:59

prog-supdex force-pushed the async_triggers branch 3 times, most recently from 79c4a14 to 1dee021 Compare September 27, 2023 13:09

prog-supdex commented Sep 27, 2023

View reviewed changes

palkan requested changes Sep 28, 2023

View reviewed changes

prog-supdex force-pushed the async_triggers branch 4 times, most recently from b9a692a to ebc7e13 Compare October 4, 2023 17:32

add an ability to deliver messages asynchronously via adapters

70dfe1c

prog-supdex force-pushed the async_triggers branch from ebc7e13 to 70dfe1c Compare October 5, 2023 20:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

deliver broadcast messages asynchronously #41

deliver broadcast messages asynchronously #41

prog-supdex commented Sep 21, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023

palkan Sep 27, 2023

prog-supdex Sep 28, 2023

palkan Sep 25, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023 •

edited

Loading

palkan Sep 28, 2023

prog-supdex Sep 28, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023

palkan Sep 25, 2023

prog-supdex Sep 26, 2023

prog-supdex Sep 27, 2023 •

edited

Loading

prog-supdex Oct 4, 2023

palkan Sep 27, 2023

palkan Sep 28, 2023

prog-supdex Sep 28, 2023

palkan Sep 28, 2023

prog-supdex commented Oct 4, 2023

-      queue_as { GraphQL::AnyCable.config.async_broadcasting["queue"] || DEFAULT_QUEUE_NAME }
+      queue_as do
+        AnyCable.config.active_job_queue_name
+      end

	payload = JSON.parse(serialized_payload)
	payload = JSON.parse(serialized_payload, symbolize_names: true)

		attr_config use_async_broadcasting: true
		attr_config async_broadcasting: { queue: "broadcasting", class: "GraphQL::Adapters::BaseJob" }


		return Adapters::BaseJob unless custom_class

		Object.const_get(config.async_broadcasting["class"])

deliver broadcast messages asynchronously #41

Are you sure you want to change the base?

deliver broadcast messages asynchronously #41

Conversation

prog-supdex commented Sep 21, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prog-supdex Sep 26, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prog-supdex Sep 27, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prog-supdex commented Oct 4, 2023

prog-supdex Sep 26, 2023 •

edited

Loading

prog-supdex Sep 27, 2023 •

edited

Loading