Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unit scaling to Console formatter #29

Merged
merged 10 commits into from
Sep 20, 2016
Merged

Add unit scaling to Console formatter #29

merged 10 commits into from
Sep 20, 2016

Conversation

wasnotrice
Copy link
Collaborator

@wasnotrice wasnotrice commented Sep 19, 2016

Integrates unit scaling into the Console formatter (solution for #27)

Features:

  • always scales results
  • always uses "short" format, e.g. M for Million
  • adds a space between the value and the unit label to match previous output, e.g. 10 μs
  • chooses the "best" unit for a column, and uses that unit for all values
  • maintains hard-coded float precision
  • adds the ability to scale a value to a specified unit
  • adds an improved failure message for column width testing:

image

Limitations:

  • has no option for disabling scaling
  • has no option for using a "long" label

@@ -5,8 +5,7 @@ defmodule Benchee.Formatters.Console do
"""

alias Benchee.Statistics

import Benchee.Unit, only: [float_precision: 1]
alias Benchee.Unit.{Count, Duration}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

# TODO: Simplify when dropping support for 1.2
# For compatibility with Elixir 1.2. In 1.3, the following group-reduce-map
# can b replaced by a single call to `group_by/3`
# Enum.group_by(fn({measurement, _}) -> measurement end, fn({_, value}) -> value end)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please open a ticket for this? Like Drop Elixir 1.2 support and then we could have a check list what to change? Oh and mention that we'd probably do this with an Elixir 1.4 release :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added #30

# Produces a map like
# %{run_time: [12345, 15431, 13222], ips: [1, 2, 3]}
collected_values = jobs
|> Enum.flat_map(fn({_name, job}) -> Map.to_list(job) end)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm code formatting wise I'm usually a fan (but not yet decided on which one) of:

mine = value
       |> pipe_stuff

# or

mine =
  value
  |> pipe_stuff

makes the start of the pipe more visible I think and more distinguished. What do you think?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree. went with option #2 since the pipes are long

# For compatibility with Elixir 1.2. In 1.3, the following group-reduce-map
# can b replaced by a single call to `group_by/3`
# Enum.group_by(fn({measurement, _}) -> measurement end, fn({_, value}) -> value end)
|> Enum.group_by(fn({measurement, _unit}) -> measurement end)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the second parameter here really the unit? Isn't it the value? e.g. shouldn't this be something like {"average", 45454}.

If I'm correct, then measurement also seems not totally spot on to me. Maybe "statistic"/"statistic_name"/"stat" - but apprently measurement is exactly the correct word. Weird I always thought of measurement as something that I measure directly e.g. a run time and not a derived computed value but Wikipedia calls it like this:

The median is a commonly used measure of the properties of a data set in statistics and probability theory.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used "stat_name" since Statistic is already a concept

# Enum.group_by(fn({measurement, _}) -> measurement end, fn({_, value}) -> value end)
|> Enum.group_by(fn({measurement, _unit}) -> measurement end)
|> Enum.reduce(%{}, fn({measurement, occurrences}, acc) ->
Map.put(acc, measurement, Enum.map(occurrences, fn({_meas, value}) -> value end))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_meas ? _measurement ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stat_name

end

defp run_time_out(average, unit) do
Duration.format(Duration.scale(average, unit))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😍 how that nitty gritty format code now just lives somewhere else more focussed and contained :)

{1.234, :millisecond}

iex(8)> Benchee.Unit.Duration.scale(11_234_567_890.123)
iex> Benchee.Unit.Duration.scale(11_234_567_890.123)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 good catch

{12.345, :millisecond}

iex(5)> Benchee.Unit.Duration.scale(12345, :minute)
{2.0575e-4, :minute}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no (3). (4) etc. :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️

assert result =~ "13000"
assert result =~ "14000"
assert result =~ "140.00"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm why is the unit of the median not checked? Should be there? :)

A separator that should appear between a value and a unit label. For example,
a space: `5.67 M` or an empty string: `5.67M`
"""
@callback separator :: String.t
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The separator always seems to be " " atm. What's the rationale behind making it dependent on the unit?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw. I'm open to changing it back to be directly attached to the number... for some reason I thought a space was more common.. but they also do it inconsistently e.g. i/s has a space while M/K don't. I also feel like M/K don't need one but Million would need to have one. Weird, huh? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, it's always " " right now. Mostly for the reasons that you are thinking about, too.

Initally, I was imagining that counts would be formatted like in du: 100K, with no space. But then I noticed that times were already formatted with a space. And I also thought about the long unit labels, and how you might want 100K but you would probably never want 100Thousand. So it seemed like something that

  • might come down to personal preference of the user
  • might need to be different for time vs count units

So, the idea is to lay the groundwork for allowing users to configure the separator as they desire (would come in #28). See Common.format/3 (currently not exposed on the Unit modules)

Copy link
Member

@PragTob PragTob left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! You're on a roll! :)

Annotations are mostly minor and about styling/naming and other minor stuff. Looks great overall - can't wait to have it on master :)

@PragTob PragTob merged commit 00cf50c into master Sep 20, 2016
@PragTob PragTob deleted the console_scale_units branch September 20, 2016 15:58
@PragTob
Copy link
Member

PragTob commented Sep 20, 2016

Thanks a lot 🎉

I'll write the Changelog once I get the time :) Or you can alsso take a stab at it if you're faster than me :)

@wasnotrice
Copy link
Collaborator Author

Awesome, thanks! I updated the CHANGELOG in 91ecf2b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants