Unspecified errors with certain stations

[ks905383]

Reading data from some stations results in a KeyError: "None of ['timestamp'] are in the columns" in one of the parsers:

from ndbc_api import NdbcApi

api = NdbcApi()

# Check that station has historical data to download
[k for k in api.available_historical(station_id = '44025')['Standard meteorological data']]

# Get data
api.get_data(station_ids=['44025'],
             mode='stdmet',
             start_time = '2022-01-01',
             end_time='2023-01-01')

Results in the following error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
[~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/api/parsers/_base.py](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/api/parsers/_base.py) in ?(cls, responses, use_timestamp)
     32             df = df.set_index('timestamp').sort_index()
     33         except KeyError as e:
---> 34             raise ParserException from e
     35         return df

[~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/pandas/core/frame.py](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/pandas/core/frame.py) in ?(self, keys, drop, append, inplace, verify_integrity)
   6120 
   6121         if missing:
-> 6122             raise KeyError(f"None of {missing} are in the columns")
   6123 

KeyError: "None of ['timestamp'] are in the columns"

The above exception was the direct cause of the following exception:

ParserException                           Traceback (most recent call last)
Cell In[125], line 1
----> 1 api.get_data(station_ids=['44025'],
      2              mode='stdmet',
      3              start_time = '2022-01-01',
      4              end_time='2023-01-01')

File [~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/ndbc_api.py:466](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/ndbc_api.py#line=465), in NdbcApi.get_data(self, station_id, mode, start_time, end_time, use_timestamp, as_df, cols, station_ids, modes)
    464 for future in futures:
    465     try:
--> 466         data = future.result()
    467         accumulated_data.append(data)
    468     except (RequestException, ResponseException,
    469             HandlerException) as e:

File [~/opt/anaconda3/envs/ndbc/lib/python3.13/concurrent/futures/_base.py:456](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/concurrent/futures/_base.py#line=455), in Future.result(self, timeout)
    454     raise CancelledError()
    455 elif self._state == FINISHED:
--> 456     return self.__get_result()
    457 else:
    458     raise TimeoutError()

File [~/opt/anaconda3/envs/ndbc/lib/python3.13/concurrent/futures/_base.py:401](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/concurrent/futures/_base.py#line=400), in Future.__get_result(self)
    399 if self._exception:
    400     try:
--> 401         raise self._exception
    402     finally:
    403         # Break a reference cycle with the exception in self._exception
    404         self = None

File [~/opt/anaconda3/envs/ndbc/lib/python3.13/concurrent/futures/thread.py:58](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/concurrent/futures/thread.py#line=57), in _WorkItem.run(self)
     55     return
     57 try:
---> 58     result = self.fn(*self.args, **self.kwargs)
     59 except BaseException as exc:
     60     self.future.set_exception(exc)

File [~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/ndbc_api.py:614](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/ndbc_api.py#line=613), in NdbcApi._handle_get_data(self, mode, station_id, start_time, end_time, use_timestamp, as_df, cols)
    611     raise RequestException(
    612         'Please supply a supported mode from `get_modes()`.')
    613 try:
--> 614     data = data_api_call(
    615         self._handler,
    616         station_id,
    617         start_time,
    618         end_time,
    619         use_timestamp,
    620     )
    621 except (ResponseException, ValueError, TypeError, KeyError) as e:
    622     raise ResponseException(
    623         f'Failed to handle API call.\nRaised from {e}') from e

File [~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/api/handlers/data.py:146](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/api/handlers/data.py#line=145), in DataHandler.stdmet(cls, handler, station_id, start_time, end_time, use_timestamp)
    144 except Exception as e:
    145     raise ResponseException('Failed to execute requests.') from e
--> 146 return StdmetParser.df_from_responses(responses=resps,
    147                                       use_timestamp=use_timestamp)

File [~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/api/parsers/stdmet.py:17](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/api/parsers/stdmet.py#line=16), in StdmetParser.df_from_responses(cls, responses, use_timestamp)
     13 @classmethod
     14 def df_from_responses(cls, responses: List[dict],
     15                       use_timestamp: bool) -> pd.DataFrame:
     16     return super(StdmetParser,
---> 17                  cls).df_from_responses(responses, use_timestamp)

File [~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/api/parsers/_base.py:34](http://localhost:8889/~/opt/anaconda3/envs/ndbc/lib/python3.13/site-packages/ndbc_api/api/parsers/_base.py#line=33), in BaseParser.df_from_responses(cls, responses, use_timestamp)
     32     df = df.set_index('timestamp').sort_index()
     33 except KeyError as e:
---> 34     raise ParserException from e
     35 return df

ParserException: NDBC API: unspecified error

This is related to JOSS review openjournals/joss-reviews#7406.

[ks905383]

Apologies, like one of the other issues I've opened, this actually seems to have been fixed with the latest version.

I had python==3.13.0 in the environment I was testing this in, so pip and conda installed an older version instead.