Performance regression in 2.4: `Schema#types` is much slower to access repeatedly

[myronmarston]

Describe the bug

I'm working on upgrading the GraphQL gem in ElasticGraph to 2.4.x. I got a warning directing me to use GraphQL::Schema::Visibility, but when I do so, my test suite takes an order-of-magnitude longer.

I dug in and believe I've found the culprit: GraphQL::Schema#types seems to be very slow on a large schema when using GraphQL::Schema::Visibility, compared to performance on 2.3.x.

Versions

graphql version: 2.4.2
rails (or other framework): N/A

GraphQL schema

https://github.com/block/elasticgraph/blob/5e8d264ad925951d98acf30c3fa319f1a55287f2/config/schema/artifacts/schema.graphql

GraphQL query

N/A

Steps to reproduce

Put this into a script:

require "bundler/inline"

gemfile do
  source "https://rubygems.org"

  gem "graphql", ENV.fetch("GRAPHQL_VERSION")
end

require "benchmark"
require "graphql"
require "open-uri"

n = ENV.fetch("N").to_i
schema_uri = ::URI.parse("https://raw.githubusercontent.com/block/elasticgraph/5e8d264ad925951d98acf30c3fa319f1a55287f2/config/schema/artifacts/schema.graphql")
schema_string = schema_uri.read

puts "GraphQL gem #{GraphQL::VERSION}--accessing `GraphQL::Schema#types` #{n} times"

Benchmark.bm(20) do |bm|
  bm.report("Not using Visibility") do
    schema = ::GraphQL::Schema.from_definition(schema_string)
    n.times { schema.types }
  end

  if ::GraphQL::VERSION >= "2.4"
    bm.report("Using Visibility") do
      schema = ::GraphQL::Schema.from_definition(schema_string, using: {::GraphQL::Schema::Visibility => {}})
      n.times { schema.types }
    end
  end
end

Run it to perform a benchmark:

$ GRAPHQL_VERSION=2.3.20 N=1000 ruby graphql_benchmark.rb
GraphQL gem 2.3.20--accessing `GraphQL::Schema#types` 1000 times
                           user     system      total        real
Not using Visibility   0.120087   0.003674   0.123761 (  0.123763)
$ GRAPHQL_VERSION=2.4.2 N=1000 ruby graphql_benchmark.rb
GraphQL gem 2.4.2--accessing `GraphQL::Schema#types` 1000 times
                           user     system      total        real
Not using Visibility   0.119798   0.004384   0.124182 (  0.124186)
Using Visibility       3.894733   0.015724   3.910457 (  3.910599)

Expected behavior

I expect GraphQL::Schema#types to perform as it did on earlier versions regardless of whether or not GraphQL::Schema::Visibility is used.

Actual behavior

GraphQL::Schema#types appears to be an order-of-magnitude slower.

Additional context

I think I can work around this by just caching a copy of what GraphQL::Schema#types returns and accessing that repeatedly rather than GraphQL::Schema#types but I still thought this was worth surfacing since it's such a drastic performance regression. Could GraphQL::Schema#types memoize it internally so callers don't have to work around this issue?

[rmosolgo]

Hey, thanks for the detailed report. I think you're spot on -- Schema.types used to use a Schema-level cache of types, but it doesn't at the moment. Caching the result from .types would accomplish the same thing, but GraphQL-Ruby should deliver the same performance for any reasonable use case ... .types included, for sure.

I recently beefed up the preload: true behavior and I think I can hook into that to continue providing a cached Schema.types result while still supporting lazy loading in development. I'll take a look soon and follow up here!