Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Database Model and Schema #253

Merged
merged 9 commits into from
Oct 25, 2021
Merged

Refactor Database Model and Schema #253

merged 9 commits into from
Oct 25, 2021

Conversation

amorenoz
Copy link
Collaborator

Right now, the way we use the dbModel and the Schema is unclean and error prone.

First, we have to hold pointers to them all around the code. Also, we use the dbModel to build Model instances but use the schema to populate their fields which can lead to errors. As an example, if you take the current main branch, build the ovsdb-server and try to use cmd/stress on it, the client will crash with:

panic: FieldByColumn: column dpdk_initialized not found in orm info

In this case, although validation passes (having partial Models is OK), the cache does not have the dbModel handy when it's filling up the fields.

Therefore, IMHO, it's clear that dbModel and schema are two sides of the same coin. The fact that server.go had to define a struct that holds both and is called DatabaseModel is a good proof of this:

type DatabaseModel struct {
Model *model.DBModel
Schema *ovsdb.DatabaseSchema
}

In addition, model.Info structures are created all over the code (e.g: api) that then calls Mapper functions that create model.Infostructures all over again.

Try to improve all this by performing the following refactorings:

Rename current DbModel to ClientDBModel and introduce an internal DatabaseModel type

After the rename, a DatabaseModel type is introduced that combines both central objects that are core for the library and that supports 2-step initialization to allow the schema to be provided afterwards (on [re-]Connect()). Validation is performed on the second step (without modifying the validation logic)

mapper.Mapper to use mapper.Info

Unify the Mapper into a more compact API by having mapper.Mapper functions accept a mapper.Info reference.

Use the DatabaseModel to optimize the mapper.Infocreation

Now that we have unified all the model handling inside a single struct (DatabaseModel), use it to create mapper.Info types. Since the mapper.Info is nothing but a metadata object attached to a Model pointer, this allows for an optimization: when we validate the schema and the ClientDBModel we can create all the metadata objects and cache them. Creating a new model.Info object is now faster as the Metadata object has already been created and stored .

Reviewer notes: Although it provides little functionality change, this PR has a lot of modifications. I've tried to split the PR in commits that are easy to review

MonitorCookies are marshalled as a json object, not an array. Fix it so
that benchmark test works

Signed-off-by: Adrian Moreno <[email protected]>
Clearly DBModel does not hold the full database model. Instead, only
the combination of model.DBModel, mapper.Mapper and ovsdb.Schema is a
useful database model as is used internally by libovsdb.

The fact that server.go had to defined a struct called DatabaseModel
with model.DBModel and ovsdb.Schema and dymanically create Mapper
objects from them is a proof of this.

In order to prepare for a DBModel refactoring, rename it to
ClientDBModel as it's the client's view of the DatabaseModel.

This patch does not contain functional changes

Signed-off-by: Adrian Moreno <[email protected]>
Replace the one that server.go had to define. For now, it's just a
drop-in replacement of the previous type.

Signed-off-by: Adrian Moreno <[email protected]>
It is common to first create a DatabaseModel only based on the
ClientDBModel, and then add / remove the schema to it when, e.g:
when the client (re) connects.

Validation is performed when the schema is added.

Since the schema and mapper can be dynamically added and purged, protect
them with a RWMutex

Signed-off-by: Adrian Moreno <[email protected]>
@dave-tucker
Copy link
Collaborator

@amorenoz need to fix lint errors 😉

mapper/info.go Outdated Show resolved Hide resolved
mapper/mapper.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@dave-tucker dave-tucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of nits but overall this looks good to me. Great work @amorenoz

For the cache, it's simply replacing three fields with one
For the client, use the 2-step DatabaseModel initialization

Signed-off-by: Adrian Moreno <[email protected]>
Now that client, cache and server uses the DatabaseModel as central
point of model creation and introspection, we can hide the
dbModelClient and move its public functions to the DatabaseModel

Signed-off-by: Adrian Moreno <[email protected]>
All around the codebase we're creating mapper.Info structures and then
calling Mapper functions that create Info structures again.

Given that mapper.Info already defines all the metadata that Mapper
needs to do the native-to-ovs transations, it makes sense to use Info
structures as input to all functions.

That simplifies the code inside the mapper module. Also, I'd expect some
performance improvement since we were creating multiple Info structs
unnecessarily in the host path.

It's true that, for now, it makes it sligthly more cumbersone to call
mapper functions, since the Info struct has to be created first and it
now requires an additional argument (the table name). However, this
can be improved later on by having the database model build the
Info structs for us.

Signed-off-by: Adrian Moreno <[email protected]>
The core mapper API uses mapper.Info sctructs which can be created just
by inspecting a TableSchema.

However, having the DatabaseModel now centralizing accesses to the
mapper API and containing both the Model types and the Schema, we can
pre-create the mapper.Info.Metadata sctructs and cache them so we create
Info sctructs more efficiently

Signed-off-by: Adrian Moreno <[email protected]>
@coveralls
Copy link

Pull Request Test Coverage Report for Build 1380776992

  • 308 of 378 (81.48%) changed or added relevant lines in 15 files are covered.
  • 9 unchanged lines in 3 files lost coverage.
  • Overall coverage increased (+0.3%) to 73.133%

Changes Missing Coverage Covered Lines Changed/Added Lines %
client/api.go 12 14 85.71%
client/monitor.go 0 2 0.0%
server/database.go 5 7 71.43%
server/server.go 6 8 75.0%
model/client.go 46 50 92.0%
client/client.go 26 31 83.87%
mapper/mapper.go 31 36 86.11%
server/transact.go 34 39 87.18%
client/condition.go 18 26 69.23%
cache/cache.go 33 44 75.0%
Files with Coverage Reduction New Missed Lines %
server/transact.go 1 67.1%
cache/cache.go 2 70.58%
client/client.go 6 67.67%
Totals Coverage Status
Change from base Build 1364201130: 0.3%
Covered Lines: 4113
Relevant Lines: 5624

💛 - Coveralls

@dave-tucker dave-tucker merged commit fd05ad5 into main Oct 25, 2021
@dave-tucker dave-tucker added the fix label Nov 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants