Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Extract out data store metrics for re-usability #138

Merged

Conversation

iaroslav-ciupin
Copy link
Contributor

@iaroslav-ciupin iaroslav-ciupin commented Aug 12, 2022

TL;DR

Adjust DataStore to allow initialising it multiple times during a single application run.

Type

  • Bug Fix
  • Feature
  • Plugin

Are all requirements met?

  • Code completed
  • Smoke tested
  • Unit tests added
  • Code documentation added
  • Any pending items have an associated Issue

Complete description

How did you fix the bug, make the feature etc. Link to any design docs etc

Tracking Issue

Remove the 'fixes' keyword if there will be multiple PRs to fix the linked issue

fixes https://github.com/flyteorg/flyte/issues/

Follow-up issue

NA
OR
https://github.com/flyteorg/flyte/issues/

@welcome
Copy link

welcome bot commented Aug 12, 2022

Thank you for opening this pull request! 🙌

These tips will help get your PR across the finish line:

  • Most of the repos have a PR template; if not, fill it out to the best of your knowledge.
  • Sign off your commits (Reference: DCO Guide).

Copy link
Contributor

@katrogan katrogan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, just a few comments

storage/copy_impl.go Show resolved Hide resolved
storage/protobuf_store.go Outdated Show resolved Hide resolved
// NewDataStore creates a new Data Store with the supplied config.
func NewDataStore(cfg *Config, metricsScope promutils.Scope) (s *DataStore, err error) {
func NewDataStore(cfg *Config, metrics *DataStoreMetrics) (s *DataStore, err error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's keep the constructor as is, but also add a method func (d *DataStore) RefreshConfig(cfg *Config) error that can re-use code defined in this constructor (this shared code can be pulled out to a separate private func for re-use)

@codecov
Copy link

codecov bot commented Aug 13, 2022

Codecov Report

Merging #138 (239a484) into master (c9998d3) will increase coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #138      +/-   ##
==========================================
+ Coverage   68.75%   68.78%   +0.03%     
==========================================
  Files          69       69              
  Lines        3405     3418      +13     
==========================================
+ Hits         2341     2351      +10     
- Misses        905      908       +3     
  Partials      159      159              
Flag Coverage Δ
unittests 67.61% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
storage/storage.go 100.00% <ø> (ø)
storage/cached_rawstore.go 81.81% <100.00%> (-0.33%) ⬇️
storage/copy_impl.go 80.00% <100.00%> (ø)
storage/mem_store.go 79.31% <100.00%> (ø)
storage/protobuf_store.go 78.26% <100.00%> (+0.48%) ⬆️
storage/rawstores.go 93.61% <100.00%> (+10.75%) ⬆️
storage/stow_store.go 78.01% <100.00%> (+0.09%) ⬆️
cache/auto_refresh.go 72.72% <0.00%> (-5.79%) ⬇️
sets/generic_set.go 100.00% <0.00%> (+1.08%) ⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@iaroslav-ciupin iaroslav-ciupin marked this pull request as ready for review August 13, 2022 07:56
@iaroslav-ciupin iaroslav-ciupin force-pushed the data-store-refactor-metrics branch from 3ee4bf3 to 0a019f9 Compare August 13, 2022 08:14
@kumare3
Copy link
Contributor

kumare3 commented Aug 13, 2022

Why do we need to do this? It seems it can already be reinitialized - pass a different scope

Signed-off-by: iaroslav-ciupin <[email protected]>
@iaroslav-ciupin iaroslav-ciupin force-pushed the data-store-refactor-metrics branch from 9cbf247 to b9fb011 Compare August 15, 2022 17:41
Copy link
Contributor

@katrogan katrogan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, just one suggestion

storage/rawstores.go Outdated Show resolved Hide resolved
Signed-off-by: iaroslav-ciupin <[email protected]>
Signed-off-by: iaroslav-ciupin <[email protected]>
@iaroslav-ciupin iaroslav-ciupin merged commit bcfa537 into flyteorg:master Aug 16, 2022
@welcome
Copy link

welcome bot commented Aug 16, 2022

Congrats on merging your first pull request! 🎉

@iaroslav-ciupin iaroslav-ciupin deleted the data-store-refactor-metrics branch August 16, 2022 16:55
eapolinario pushed a commit that referenced this pull request Sep 6, 2023
* Extract out data store metrics for re-usability and add RefreshConfig method to DataStore
* Add some tests to improve coverage
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants