Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/smhi class to match 0.2.0 #115

Merged
merged 71 commits into from
May 22, 2024
Merged
Changes from 2 commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
4961bbe
Initial commit, fixing base functionality
docNord Apr 6, 2024
c4fa30a
Restructure get_data
docNord Apr 6, 2024
d96bc59
Fix gps station find and parallells
docNord Apr 6, 2024
16add6a
remove parameters() property
docNord Apr 6, 2024
cfc4590
small edit
docNord Apr 7, 2024
043c823
Fix find_stations_from_gps
docNord Apr 7, 2024
f5e6e2c
Fix data interpolation
docNord Apr 7, 2024
1b304f9
Merge branch 'main' into feature/smhi-class-to-match-0.2.0
docNord Apr 7, 2024
fd6a992
Mock unit test
docNord Apr 7, 2024
57c3e05
Less state mutating, one interface get_data
mgcth Apr 7, 2024
c4dfcca
City get
mgcth Apr 7, 2024
0e5c889
refactor interpolate
mgcth Apr 7, 2024
cd8fe26
Refactor
mgcth Apr 7, 2024
92a43fa
Refactor interpolate
mgcth Apr 8, 2024
7d11797
Fix refactoring bug
mgcth Apr 8, 2024
3c4737a
Simplify interpolate
mgcth Apr 8, 2024
982e950
Merge branch 'main' into feature/smhi-class-to-match-0.2.0
mgcth Apr 9, 2024
7b7fceb
attempt at mocking __new__
docNord Apr 11, 2024
c7d9fe0
Merge branch 'feature/smhi-class-to-match-0.2.0' of https://github.co…
docNord Apr 11, 2024
e611d6d
Move code
mgcth Apr 10, 2024
d991de8
Change MetobsLinksModel to MetobsLinks
docNord Apr 16, 2024
a6e757c
fix lint
docNord Apr 17, 2024
6ccca8b
Make _find_stations_from_gps always return list
docNord Apr 17, 2024
2040698
Change interpolation arg description
docNord Apr 18, 2024
136dc52
Fix init test
docNord Apr 18, 2024
0849c89
Build get_stations and get_stations_from_title tests
docNord Apr 18, 2024
bb2eff6
Attempted test of get_data (faulty patch)
docNord Apr 18, 2024
5dc2f91
Fixed get_data test
docNord Apr 18, 2024
feba25e
Trying to mock Stations object
docNord Apr 19, 2024
7a9cf8b
Still not able to properly mock Stations
docNord Apr 21, 2024
3e07f72
fix test_find_stations_from_gps
docNord Apr 23, 2024
354fb31
test _find_stations_by_city
docNord Apr 23, 2024
38d24e1
test _interpolate
docNord Apr 24, 2024
c210bff
test _iterate_over_time
docNord Apr 24, 2024
7cd1bb6
test _find_missing_data
docNord Apr 24, 2024
e1c2faa
fix lint
docNord Apr 24, 2024
ffbf7c9
fix lint but with ruff
docNord Apr 24, 2024
ca348b0
Modify Metobs __init__() to stabilize integration when 'corrected-arc…
docNord Apr 25, 2024
f1eb0ca
add another sort to be safe
docNord Apr 26, 2024
2acc50c
try a typing shortcut
docNord Apr 26, 2024
58cf6d1
fix typing again
docNord Apr 26, 2024
f1fd265
Set ordering to latest available by default
docNord Apr 27, 2024
5172cf1
try fixing type issue by introducing lambda
docNord Apr 27, 2024
0fca406
Another typing bypass attempt
docNord Apr 27, 2024
924e79e
fix spelling error
docNord Apr 27, 2024
657915a
Update tests/unit/test_unit_smhi.py
docNord Apr 27, 2024
226fa31
fix doc strings
docNord Apr 27, 2024
a9142f3
Generalize find_stations_from_gps test through mock object classes
docNord May 4, 2024
a44a234
clean up test_iterate_over_time with a fixture
docNord May 4, 2024
7c436f2
Clean up Metobs Data init function
docNord May 4, 2024
9d85250
Fix lint
docNord May 4, 2024
9d42f22
Attempt aligning MetObs unit test with suggested structure
docNord May 4, 2024
b455417
add assertions to the SMHI unit testing per instructions
docNord May 4, 2024
e8651d4
Fix and simplify smhi
mgcth May 5, 2024
fdc1ef0
Update metobs data test for period=None
mgcth May 5, 2024
4f6f323
Fix period order in metobs
mgcth May 5, 2024
faae852
Merge branch 'main' into feature/smhi-class-to-match-0.2.0
mgcth May 5, 2024
ad2e1da
Add delay after test avoiding API limit, if any
mgcth May 5, 2024
6eb6116
typo in metobs, starting to build integration test
docNord May 5, 2024
52f64ad
Initial integration test status
docNord May 10, 2024
a3739af
Change to full integration test mode
docNord May 18, 2024
6c9310e
fix lint
docNord May 18, 2024
02b604a
Reformat with ruff
docNord May 18, 2024
3bd8a4b
Update documentation with SMHI class example
docNord May 18, 2024
2cea0b2
fix lint
docNord May 18, 2024
c674225
Rewrote sentence
docNord May 20, 2024
49f77af
space after comma
docNord May 20, 2024
63e1c8c
toolbox to package
docNord May 20, 2024
0073731
spaces in integration test
docNord May 20, 2024
5cf7d2e
Add plotly figure
docNord May 22, 2024
f0a5485
This will only work as long as we serve docs on ...se/ifk-smhi/
mgcth May 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 31 additions & 31 deletions src/smhi/smhi.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from geopy import distance
from geopy.geocoders import Nominatim
from smhi.constants import TYPE_MAP
from smhi.metobs import Metobs
from smhi.metobs import Data, Parameters, Periods, Stations, Versions


class SMHI:
Expand All @@ -21,8 +21,7 @@ def __init__(self, type: str = "json", version: str = "1.0") -> None:
version: API version
"""
self.type = TYPE_MAP[type]
self.client = Metobs(type)
self.client.get_parameters()
self.parameters = Parameters(Versions())
mgcth marked this conversation as resolved.
Show resolved Hide resolved

@property
def parameters(self):
Expand All @@ -31,7 +30,7 @@ def parameters(self):
Returns:
parameters
"""
return self.client.parameters.data
return self.parameters.data

def get_stations(self, parameter: Optional[int] = None):
"""Get stations from parameter.
Expand All @@ -42,12 +41,12 @@ def get_stations(self, parameter: Optional[int] = None):
Returns:
stations
"""
if self.client.parameters is None:
if self.parameters is None:
logging.info("No parameters available.")
return None

self.client.get_stations(parameter)
return self.client.stations.data
self.stations = Stations(self.parameters, parameter)
return self.stations.data

def get_stations_from_title(self, title: Optional[str] = None):
"""Get stations from title.
Expand All @@ -58,12 +57,12 @@ def get_stations_from_title(self, title: Optional[str] = None):
Returns:
stations
"""
if self.client.stations is None:
if self.stations is None:
logging.info("No stations available.")
return None

self.client.get_stations(None, title)
return self.client.stations.data
self.stations = Stations(self.parameters, title)
return self.stations.data

def find_stations_from_gps(
self, parameter: int, latitude: float, longitude: float, dist: float = 0
Expand Down Expand Up @@ -144,19 +143,19 @@ def get_data(
station: station id
period: period to get
"""
data, header = self.client.get_data_from_selection(
parameter=parameter, station=station, period=period
)
self.stations = Stations(Parameters(Versions()), parameter)
mgcth marked this conversation as resolved.
Show resolved Hide resolved
self.periods = Periods(self.stations, station)
data = Data(self.periods)
if interpolate > 0:
# Find the station latitude and longitude information from Metobs
stat = next(
item for item in self.client.stations.stations if item["id"] == station
)
latitude = stat["latitude"]
longitude = stat["longitude"]

holes_to_fill = data[
data.index.to_series().diff() > data.index.to_series().diff().median()
# should be replaced by a self.periods.position[0].latitude
stat = next(item for item in self.stations.station if item.id == station)
latitude = stat.latitude
longitude = stat.longitude

holes_to_fill = data.df[
data.df.index.to_series().diff()
> data.df.index.to_series().diff().median()
docNord marked this conversation as resolved.
Show resolved Hide resolved
]
# Find stations within a given radius - set in "interpolate".
self.find_stations_from_gps(
Expand All @@ -168,24 +167,25 @@ def get_data(

# Iterate over nearby stations, starting with the closest
for nearby_station in self.nearby_stations[1:]:
tmpdata, _ = self.get_data(parameter, nearby_station[0])
tmpdata = Data(Periods(self.stations, station))
for time, _ in holes_to_fill.iterrows():
earliertime = data[data.index < time].index.max()
earliertime = data.df[data.df.index < time].index.max()

if (
len(
tmpdata[
(tmpdata.index > earliertime) & (tmpdata.index < time)
tmpdata.df[
(tmpdata.df.index > earliertime)
& (tmpdata.df.index < time)
]
)
> 0
):
data = pd.concat([data, tmpdata], axis=0, join="outer")
data.df = pd.concat([data.df, tmpdata.df], axis=0, join="outer")

# Re-check how many holes remain
holes_to_fill = data[
data.index.to_series().diff()
> data.index.to_series().diff().median()
holes_to_fill = data.df[
data.df.index.to_series().diff()
> data.df.index.to_series().diff().median()
docNord marked this conversation as resolved.
Show resolved Hide resolved
]
data = data.sort_index()
return data, header
data.df = data.df.sort_index()
return data
Loading