Decorate model functions to indicate their units

[denised]

It would be nice if there was a convenient way to know what the units of a model function are. There are two common ways this information is provided:

• Part of the function name, something like co2_mmt_....
• Part of the docstring

In both techniques it is either difficult to standardize this process, or results in a kind of templatization that makes the code (or comments) hard to read.
The model functions already have an attribute that gives their name. We could add an additional attribute of units. This could be automatically used by any UI layer on top, lowering the chance of mismatch.

[denised]

Another alternative would be to create types for units, and add typing information. This would have an even additional benefit of being applicable to arguments as well as results.
Not sure if this would provide the same level of support for a UI however.

[gerald-scharitzer]

Do you have a list of non-SI units that are frequently used in Drawdown? So far I have seen the following, but I am not sure that I interpreted them correctly.

• mass
  • mmt: metric megaton = 1e9 kg
  • Gt: gigatonne = 1e12 kg
  • Mt: megatonne = 1e9 kg
  • kt: kilotonne = 1e6 kg
• scalar
  • ppm: parts per million = 1e-6
  • ppb: parts per billion = 1e-9

Do you also have samples of functions where they are used?

[denised]

Hi Gerald --- Thank you for taking a look at this! I understand the code a bit better now than when I originally wrote this ticket, and I now realize that many places where there is a fixed unit, it is in the name of the function already --- e.g. co2_mmt_reduced, for million metric tonnes. In other places where it would be useful to know, it tends to be solution-specific, such as the adoption metric (which might be million-passenger-miles for one solution and terra-watt-hours for something else) --- and these are usually indicated by funit or iunit (functional or implementation adoption units, respectively) A module that isn't as clear as it could be is unitadoption.py, where many functions don't mention units. I was originally thinking of adding a piece of metadata --- either to the function itself, or to the pandas DataFrame it returns. But given that a lot of the code already has the units in the name, it would probably be less disruptive to look for places where functions could be renamed to be more clear about what they are returning --- and I'd definitely look at unitadoption.py to get a sense of that. . -- Denise Draper ***@***.***

…

On Sat, Nov 20, 2021, at 3:53 PM, Gerald Scharitzer wrote: Do you have a list of non-SI units that are frequently used in Drawdown? So far I have seen the following, but I am not sure that I interpreted them correctly. * mass * mmt: metric megaton = 1e9 kg * Gt: gigatonne = 1e12 kg * Mt: megatonne = 1e9 kg * kt: kilotonne = 1e6 kg * scalar * ppm: parts per million = 1e-6 * ppb: parts per billion = 1e-9 Do you also have samples of functions where they are used? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#361 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAMTXVKZKOC35GKCMMBRDUNAYG5ANCNFSM5A663S6Q>.

[gerald-scharitzer]

I understand that solution modules specify their units in the dictionary named units as in solution/airplanes/__init__.py.

Also, several model functions utilize a naming convention, where the second qualifier (first_second_*) specifies the unit as in co2eq_mmt_reduced, where mmt specifies million metric tonnes.

I recommend to stick to your original idea of decorating functions or data frames with their units, because this can be done independent of naming conventions. This will help us to keep stable interfaces once we reach version 1.0.0.

Function names and docstrings are unstructured and would have to be parsed based on some to-be-defined grammar.

Attributes on functions or data frames can be more structured and machine-readable to also serve UIs for labeling table columns and chart axes.

I will look at unitadoption.py from that perspective.

[denised]

I have to admit I do like the idea :-)

…

-- Denise Draper ***@***.***

On Sun, Nov 21, 2021, at 7:44 AM, Gerald Scharitzer wrote: I understand that solution modules specify their units in the dictionary named `units` as in `solution/airplanes/__init__.py` <https://github.com/ProjectDrawdown/solutions/blob/b7890c0dbd544799c6e1d102aee25f2eb2daa811/solution/airplanes/__init__.py#L31>. Several model functions utilize a naming convention, where the second qualifier (`first_second_*`) specifies the unit as in `co2eq_mmt_reduced`, where `mmt` specifies million metric tonnes. I recommend to stick to your original idea of decorating functions or data frames with their units, because this can be done independent of naming conventions. This will help us to keep stable interfaces once we reach version 1.0.0. Function names and docstrings are unstructured and would have to be parsed based on some to-be-defined grammar. Attributes on functions or data frames can be more structured and machine-readable to also serve UIs for labeling table columns and chart axes. I will look at `unitadoption.py` from that perspective. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#361 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKAMTRHWOG7EFQ5D7JQT5DUNEHXRANCNFSM5A663S6Q>.

[gerald-scharitzer]

Pandas does not provide direct support for units. There is an open issue for this since 2015 and to me it does not look like it is going to be resolved anytime soon.

Since not only data frames can have units (more specifically the data series therein), but also functions, attributes, and variables, I do not want to inject a units attribute into all of those or subclass them.

I have something less invasive in mind and will try some things. This may take a while.

[denised]

@gerald-scharitzer I look forward to seeing what you come up with.

[gerald-scharitzer]

So far I am mapping data frame columns (series) to units. Next, I will use those to decorate table columns and chart axes when rendering them for display, to get some practical use cases for user interfaces.

[gerald-scharitzer]

Labeling chart axes with units works, but labeling table headers does not work yet. I will improve the units API and then come back to the table visualization.

[havarsak]

Just to throw in my 2c: this would be really useful!
Especially if the more advanced thoughts are accounted for (keep track of units into functions and graphs, or in multiplication or division). But just being able to specify the unit of a dataframe column is useful, to remember what it was.
Just being able to add tags or information to columns would also be highly useful, to keep track of what's stored in them.

[gerald-scharitzer]

I am back from the end-of-year madness and working on it right now. More specific, I am working on the prefix API, such that for example "Mt" is megatonnes and thus 10e6 tonnes and 10e9 kilograms.

After that I will have an API for dimensions, units, and prefixes. That should be enough for a first release to integrate into Drawdown.

[gerald-scharitzer]

I have a reasonably stable API now. Next, I will test this in the context of the solutions.

[gerald-scharitzer]

It also works in this context. Next, I will test it with some interesting use cases.

[gerald-scharitzer]

I found a bug in my handling of weak references and will fix that now.

[gerald-scharitzer]

Hi @denised, my proposal to map functions and user-defined objects to units looks like this. Please let me know whether this is going into the right direction for you, or you want something else.

[denised]

@gerald-scharitzer I like this very much. Please proceed!

[gerald-scharitzer]

So far I only decorated one function, but did not use the unit anywhere, except the Jupyter notebook that demonstrates potential use cases. Do you have any actual use cases in this repository on your mind, where you think this would be useful?

[denised]

(I thought I had answered earlier -- must not have hit send?)

The Jupyter notebook scenario is the biggest one right now. And I anticipate (but have not designed) some tools that generate interesting charts and use this functionality to auto-populate legends.

...But there are a lot of functions that could be annotated, and it would make sense to focus on an initial segment. Two that pop to mind are CO2e and Hectares (either one would be a good starting place)

[gerald-scharitzer]

No problem, meanwhile I improved my test coverage.
In this context I would document in the units notebook, that if a function is not mapped to a unit, then this can be done easily with the @map_to_unit decorator.
This way we enable the developers to use the feature when the demand or use case arises.
At the same time we always have the option to launch a campaign to decorate the functions of specific modules or packages.
Or do you want to keep this issue open until all model functions are decorated?

[denised]

@gerald-scharitzer I think your suggestion is good. We'll put the stake in the ground, and then add more built in functionality as we have time to invest in it.

[gerald-scharitzer]

Then I will release version 1.0 of units-of-measure and create the pull request for the branch units of the solutions.