Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Record processing steps into history attribute with context manager #4914

Open
cwerner opened this issue Feb 16, 2021 · 4 comments
Open

Record processing steps into history attribute with context manager #4914

cwerner opened this issue Feb 16, 2021 · 4 comments
Labels
topic-metadata Relating to the handling of metadata (i.e. attrs and encoding)

Comments

@cwerner
Copy link

cwerner commented Feb 16, 2021

I often want to record an entry into history of my netcdf file/ xarray. While one can always add it manually, i.e.

ds.attrs["history"] = ds.attrs["history"] + "\n" + "message"

I was wondering if there's a better way... In a first attempt I tried using a context manager for this. Not sure if there are other approaches? Would that be something useful for xarray core? What are other people using for this?

Demo:

import datetime
import xarray as xr

class XrHistory():

    def __init__(self, array, message, timestamp=True):
        self._array = array
        self._message = message
        self._timestamp = timestamp

    def __enter__(self):
        if 'history' not in self._array.attrs:
            self._array.attrs['history'] = ""
        
        if self._message != self._array.attrs['history'].split('\n')[-1]:
            ts = f"{datetime.datetime.now().strftime('%a %b %d %H:%M:%S %Y')}: " if self._timestamp else ""
            
            self._array.attrs['history'] += f"\n{ts}{self._message}"
            self._message = None
        return self._array
    
    def __exit__(self, exc_type,exc_value, exc_traceback):
        pass

# ds is any xarray dataset...

with XrHistory(ds, "normalise data") as ds:
    ds["array_one"] = (ds.array_one - ds.array_one.mean(dim='time')) / ds.array_one.std(dim='time')

with XrHistory(ds, "subset data") as ds:
    ds = ds.sel(x=slice(10, 20), y=slice(10,20))

# ...
@dcherian
Copy link
Contributor

You can help test out #4896 !

@dcherian dcherian added the topic-metadata Relating to the handling of metadata (i.e. attrs and encoding) label Apr 19, 2021
@sjvrijn
Copy link
Contributor

sjvrijn commented May 23, 2022

Now #4896 has been merged, can this issue be closed?

@dcherian
Copy link
Contributor

#5668 is the next thing needed afaict

@keewis
Copy link
Collaborator

keewis commented May 23, 2022

yes, that's true, and I'm sorry to have stalled that one (it is not forgotten, though, I was planning to have a look at it sometime soon).

#4896 added the hook, while #5668 will populate the context object.

We still need to figure out how to chain merge strategies, though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-metadata Relating to the handling of metadata (i.e. attrs and encoding)
Projects
None yet
Development

No branches or pull requests

4 participants