integration tests failing because one of hardware-observer's metrics are down

[ca-scribner]

Bug Description

During CI runs (locally and in gh), we're seeing the following:

  File "/home/runner/work/grafana-agent-operator/grafana-agent-operator/tests/integration/test_juju_info_cos_agent.py", line 106, in test_metrics
    assert target["state"] == "up"
AssertionError: assert 'down' == 'up'

which is caused because one of hardware-observer's monitored metrics is down:

 {
    "status": "success",
    "data":
    [
        {
            "instance": "243a344db344241f404868d04272fc76",
            "target_group": "hwo_1_default",
            "endpoint": "http://localhost:10200/metrics",
            "state": "down",
            "labels":
            {
                "instance": "localhost:10200",
                "job": "hwo_1_default",
                "juju_application": "hwo",
                "juju_model": "test-juju-info-cos-agent-ylut",
                "juju_model_uuid": "671c70ef-a32d-449e-8a83-42f51a436e76",
                "juju_unit": "hwo/0"
            },
            "discovered_labels":
            {
                "__address__": "localhost:10200",
                "__metrics_path__": "/metrics",
                "__scheme__": "http",
                "__scrape_interval__": "1m",
                "__scrape_timeout__": "10s",
                "job": "hwo_1_default",
                "juju_application": "hwo",
                "juju_model": "test-juju-info-cos-agent-ylut",
                "juju_model_uuid": "671c70ef-a32d-449e-8a83-42f51a436e76",
                "juju_unit": "hwo/0"
            },
            "last_scrape": "2024-08-07T13:26:52.715197421Z",
            "scrape_duration_ms": 0,
            "scrape_error": "Get \"http://localhost:10200/metrics\": dial tcp 127.0.0.1:10200: connect: connection refused"
        },
        ...(truncated)
    ]
}

Previous successful CI runs have used hardware-observer successfully with rev70 of the charm - this appears to be new behaviour. An issue is filed in hardware-observer to see if this is a bug.

To Reproduce

tox -e integration on the current charm

Environment

Relevant log output

-

Additional context

Full logs: https://github.com/canonical/grafana-agent-operator/actions/runs/10287798773/job/28471764690