Voltage-Compensated Dummy Load Power Measurement

[RubenKelevra]

Current State

• Currently, we measure the power consumption of the dummy load once.
• Users are instructed to wait for a 2-hour preheating period before taking measurements. This was a cautious, estimated guideline.
• The system assumes that the dummy load's power consumption remains constant throughout.

Problem Statement

• This assumption is flawed for resistive loads, as their power consumption varies with grid voltage.
• While two measurements (before and after the main measurement) could help account for slow drift, it does not fully address how voltage is regulated in the power grid.

Proposed Solution

This PR introduces a method to adjust dummy load power consumption measurements based on grid voltage.

Technical Explanation

Resistive loads exhibit varying power consumption depending on grid voltage. In Germany, the allowed voltage range is 230 V ± 10%, meaning a 4800 Ω resistor's power consumption can range between:

• 8.93 W @ 207 V
• 13.34 W @ 253 V

Although voltage fluctuations are typically minor in residential areas over short periods, long-term measurements (spanning several hours) are affected by slow voltage drifts. These drifts can be suddenly corrected by transformers, introducing a "sawtooth" noise in the readings.

To mitigate this, the following approach is proposed:

1. Resistive Measurement: Instead of storing the power consumption of the dummy load, we will record its resistance.
2. Voltage Compensation: On each device measurement, the grid voltage will also be recorded. The dummy load's power consumption will then be calculated dynamically based on this voltage.
3. Long-Term Measurements:
   Rather than relying on a single measurement from the power meter, we now collect voltage and power consumption data over a 10-minute period. Using this data, we calculate the resistor's value and average it for improved accuracy.
4. Preheat Detection:
   To automate the determination of resistor stability, we leverage the same 10-minute data collection. During this period, we capture 20 consecutive 30-second averages and assess the stability by comparing the average drift of the first and second sets of ten measurements.
   • If both sets drift in the same direction, we assume the resistor is still heating and has not yet stabilized.
   • This eliminates the need for users to manually assess stabilization, ensuring consistent and reliable measurements.

Advantages

• Provides more accurate dummy load compensation.
• Reduces noise in long-term power measurements caused by grid voltage fluctuations.
• Avoids user errors in determining if the resistor is already sufficiently heated.

Limitations

• Hardware Compatibility:

  • Measurements with a dummy load will not work with equipment that cannot record voltage (e.g., Shelly Gen1 devices).
  • Devices and integrations (e.g., Tuya, Tasmota, Kasa, MyStrom) are unsupported due to a lack of test hardware.

• Home Assistant (HA) Entity Naming:

  • This implementation assumes HA entity names follow the pattern:
    • pattern_name_power
    • pattern_name_voltage.

• OCR Exclusion:

  • Optical Character Recognition is not supported due to inherent complexities.

Accuracy of Measurements

To validate the accuracy of this approach, I conducted three test runs in my setup. In each instance, the script successfully determined whether the resistors were sufficiently preheated.

The measured resistances and corresponding power consumptions were as follows:

• Run 1: 4785.41 Ω → 11.0544 W @ 230 V
• Run 2: 4782.00 Ω → 11.0623 W @ 230 V
• Run 3: 4787.99 Ω → 11.0485 W @ 230 V

The variation across these runs is minimal, with the power differences staying well below the 0.1 W resolution of Gen3 Shelly sensors.

Conclusion

This PR addresses inaccuracies in resistive load power measurement by compensating for grid voltage fluctuations. Although this approach introduces some limitations, it enhances measurement reliability for supported hardware.

---

Due to size and complexity, I consider this a draft atm.

[RubenKelevra]

@bramstroker I just checked with the suggested method of using an incandescent light bulb as dummy load. Granted, I don't have one, but I had one of these regular shaped light bulbs with a Halogen Bulb inside.

Waiting one minute to let it heat up, and the script will be happy on it's first try and determines, that the dummy load has sufficiently heated. So they work fine as dummy load.

---

Measuring a dummy load with the script and save the resistance after it has stabilized. Then abort and wait two hours and let it measure, but do not attach a light will lead to ±0.02 W noise on the standard settings (SAMPLE_COUNT=2), with 4 lights. So the noise with one light is ±0.08 W (worst actual measurement was 0.068 W). That's much better than I had expected.

[bramstroker]

@RubenKelevra thanks. Will review in a few days.

[bramstroker]

Nice job! just did a first quick glance. here are my first comments. Will do a more throrough review later on

[RubenKelevra]

Probably missed a couple of things now, and no testing yet. Will have another look at this tomorrow or at the weekend

[RubenKelevra]

Testing complete. Should work now for HA and Shelly V2/3 devices. It reads the new .env var successfully, does auto detect and if this fails asks the user to select the voltagemeter.

[bramstroker]

@RubenKelevra before merging, ruff linting has to be fixed, see failed action.
Also tests are failing. Let me know if you need help with that.

[RubenKelevra]

@RubenKelevra before merging, ruff linting has to be fixed, see failed action. Also tests are failing. Let me know if you need help with that.

Yeah ruff fails for the fixmes, and makes 2-3 suggestions which IMHO reduce readability. You really want those changes to be made?

Tests I'll look later.

[bramstroker]

See my suggestions how to both simplify the code and make ruff happy.
For the fix me you can add ignores # noqa: FIX001, that's the annotation you can use to ignore ruff errors.
Also see https://docs.astral.sh/ruff/linter/#error-suppression

[RubenKelevra]

@bramstroker you need to take a look at the test. Looks to me like the max retries is not read from the config when running the test which is why the default of 5 is not set, and it's None. Which is why it before 7ae0c80 just ran into RecursionError. The limit of 100 now works, but the logger message can't be printed, because the config option is None, which is not a number.

And yet, the measurement is failing for some reason, but I think the "None" config option needs to be fixed as well. :)

https://github.com/bramstroker/homeassistant-powercalc/actions/runs/12484551223/job/34842108258?pr=2810

[RubenKelevra]

@bramstroker after adding a workaround for the self.config.max_retries is None in the test, it looks like test just encounters the new check for 0.00 W measurements I've added. So the test needs to be redesigned, I guess?

https://github.com/bramstroker/homeassistant-powercalc/actions/runs/12485085692/job/34843389826?pr=2810

[bramstroker]

Did have a quick look but the test is failing because it enters dummy load mode.

There is something not right with order of the questions, or the handling of these events.
It's difficult to test these console anyway, so that's why I only have one test currently testing this.
And all other tests are setting up mocked config, so the CLI is not prompting anything.

Will have a further look later.

[bramstroker]

btw I just notice there is several code added now specifically for powermeters, and even more specifically for the hass powermeter only.

There is specific extension point for this designed. See line 248.
Powermeter protocol has get_questions method where you can extend the questions with more questions in hass.py.

[bramstroker]

this one still needs resolving.

[bramstroker]

I have the test fixed. Could you replace the test method with this?

def test_wizard(mock_config_factory) -> None:  # noqa: ANN001
    """Test the CLI wizard, can we actually select certain options"""
    mock_config = mock_config_factory(
        set_question_defaults=False,
    )
    measure = _create_measure_instance(
        mock_config,
        console_events=event_factory(
            # MEASURE_TYPE
            key.ENTER,
            # GENERATE_MODEL_JSON
            "y",
            # DUMMY_LOAD
            "n",
            # MODEL_NAME
            "a", key.ENTER,
            # MEASURE_DEVICE
            "a", key.ENTER,
            # COLOR_MODE
            key.DOWN, key.DOWN, key.ENTER,
            # GZIP
            key.ENTER,
            # MULTIPLE_LIGHTS
            "n",
        ),
    )

    with patch("builtins.input", return_value=""):
        measure.start()

    assert os.path.exists(os.path.join(PROJECT_DIR, "export/dummy/brightness.csv.gz"))
    assert os.path.exists(os.path.join(PROJECT_DIR, "export/dummy/model.json"))

[bramstroker]

Also replace line 242 of light.py from time.sleep(2) to time.sleep(self.config.sleep_time), that will make the full test also run instant.
Now it's blocking for 2 seconds.
I can also globally patch time.sleep probably, but this is a cleaner solution imo.

[bramstroker]

@RubenKelevra Did you see my comments? Are you able to do final rework, than we can merge.

[RubenKelevra]

@bramstroker yes, just haven't had time yet. Maybe next year 😜

Guten Rutsch :)

[bramstroker]

No worries, take your time.

Gelukkig nieuwjaar ;-)

[RubenKelevra]

I have the test fixed. Could you replace the test method with this?

Sure: dde0a80

[RubenKelevra]

Also replace line 242 of light.py from time.sleep(2) to time.sleep(self.config.sleep_time), that will make the full test also run instant. Now it's blocking for 2 seconds. I can also globally patch time.sleep probably, but this is a cleaner solution imo.

Sure: f708bb3