Add an index to versioned_resources check_order column

[christophermancini]

Add an index to versioned_resources check_order column for improved ORDER BY performance. We have a repository with a very large number of versioned resources, queries on the table suffer from the lack of an index on the check_order column during the ORDER BY process as you can see below.

atc=> explain select v.id, v.enabled, v.type, v.version, v.metadata, v.modified_time, v.check_order from versioned_resources v, resources r where v.resource_id = r.id and r.name ='name' and r.pipeline_id = 00 ORDER BY check_order DESC LIMIT 1;
                                                                QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=12067.66..12067.66 rows=1 width=77)
   ->  Sort  (cost=12067.66..12074.34 rows=2672 width=77)
         Sort Key: v.check_order
         ->  Nested Loop  (cost=0.70..12054.30 rows=2672 width=77)
               ->  Index Scan using unique_pipeline_id_name on resources r  (cost=0.28..8.29 rows=1 width=4)
                     Index Cond: ((pipeline_id = 36) AND (name = 'chef'::text))
               ->  Index Scan using versioned_resources_resource_id_idx on versioned_resources v  (cost=0.43..11994.71 rows=5129 width=81)
                     Index Cond: (resource_id = r.id)
(8 rows)

CREATE INDEX versioned_resources_check_order ON versioned_resources (check_order DESC);
atc=> explain select v.id, v.enabled, v.type, v.version, v.metadata, v.modified_time, v.check_order from versioned_resources v, resources r where v.resource_id = r.id and r.name ='chef' and r.pipeline_id = 36 ORDER BY check_order DESC LIMIT 1;
                                                              QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=0.70..963.77 rows=1 width=78)
   ->  Nested Loop  (cost=0.70..2572367.34 rows=2671 width=78)
         Join Filter: (v.resource_id = r.id)
         ->  Index Scan using versioned_resources_check_order on versioned_resources v  (cost=0.43..2551203.72 rows=1410355 width=82)
         ->  Materialize  (cost=0.28..8.30 rows=1 width=4)
               ->  Index Scan using unique_pipeline_id_name on resources r  (cost=0.28..8.29 rows=1 width=4)
                     Index Cond: ((pipeline_id = 36) AND (name = 'chef'::text))
(7 rows)

[vito]

This needs to be added to the migrations.go file as well. Good find though!

[vito]

Is this still good to go? I vaguely remember some concerns mentioned in Slack. :)

[vito]

This needs to be re-done now that we've completely revamped our migrations stack.

See:

• https://github.com/concourse/atc/tree/master/db/migration/migrations
• https://github.com/concourse/atc/blob/master/scripts/create-migration
• https://github.com/concourse/atc/blob/master/scripts/generate-bindata