add a dispatch table version number to netcdf_meta.h and libnetcdf.settings

[edhartnett]

This is part of #1056

Should there be some version number in the dispatch table?

How would that work? Let's say we call the current dispatch table (after merging outstanding PRs) version 1.

Then, some package, like PIO, uses the dispatch layer and everyone is happy.

Then, we decide to change the dispatch table - we add a function. So we bump the version.

How does the version number help? PIO will still have to change its struct to include the new function. So there will be no way to use the version to keep using the old dispatch table, without the new funciton.

So what is the point of a version?

[WardF]

I'm still getting caught up, but just wanted to add my +1 to this. I'm not opposed to it, but I'd be interested in hearing more. Or, I'll chime in when I'm caught up and my questions are answered.

Back to esupport!

[DennisHeimbigner]

Yes, there should be. THis may have been one of the changes
I proposed in that issue some time back about what changes
are needed in the dispatch table.

[edhartnett]

@DennisHeimbigner yes it is from #1056. Some of the items in that list have been done, some have been discussed and discarded. This is one of the unresolved items.

But it's not clear to me how this should work. As a library that is using UDF, should I inquire what is the version of the dispatch layer that the library supports? And should PIO be responsible for always providing the correct dispatch table for the library? I think this would work well. In this case, perhaps the dispatch version should be part of netcdf_meta.h? Then, as PIO is building, it can see what version of the dispatch table is needed. If there two possible versions, the PIO library can have both ready, and use whichever one the local netCDF library wants.

Or do you have something else in mind? Your idea was to put the version in the dispatch table - does that mean the library would handle multiple versions of the dispatch table? Not sure that's worth doing. It seems reasonable to require PIO to meet the needs of the library, rather than the library meeting the needs of PIO in a more generalized way.

[edhartnett]

So sum up, I propose we add dispatch_version to netcdf_meta.h, and set it to 1.

When and if we change the dispatch table, we can increment that number. I would like to work through all the items in #1056 and see if we can get all desired changes done in one version, instead of spreading them out. Let's get the table to where we want it, and then try to leave it alone.

[DennisHeimbigner]

We could allow new fields to be added to the end of the
table, but not allow any of the existing ones to be deleted.
Then we say that user defined code must be prepared to get
a new, current version, table, with any fields it would not
recognize filled with some fake function that just fails (or always succeeds).

[edhartnett]

Yes, I agree it would be best to add not change. This is similar to our usual approach to the API, so that's fine.

But I think one round of changing is OK, because right now, no one is using the dispatch table. So we can make changes without impacting anyone.

However, I would like to release PIO with netCDF integration, before the Autumn, and then start using that in NOAA models. So at that point, we must be more careful what we do. ;-)

Do we agree I should add a version number to the netcdf_meta.h file? And then packages like PIO can use that to detect a version number for the dispatch table.

Sound OK?

[DennisHeimbigner]

It occurs to me that there is an alternate solution to the public dispatch
table problem. Instead of using the current one, we replace it with one
whose entries exactly mirror the API as defined in netcdf.h.
This would mean that it would change only when netcdf.h changes,
which is something that users might have to deal with in any case.
The one caveat is that the new dispatch table would need to include
the API entries in e.g. netcdf_par.h, netcdf_mem.h, and netcdf_filter.h.
As already mentioned, we never delete an entry, but could add entries.

For the libdispatch code, our current occurrences of netcdf.h
API calls would change from e.g.

int
nc_inq_var(int ncid, int varid, char *name, nc_type *xtypep,
int *ndimsp, int *dimidsp, int nattsp)
{
NC ncp;
int stat = NC_check_id(ncid, &ncp);
if(stat != NC_NOERR) return stat;
return ncp->dispatch->inq_var_all(ncid, varid, name, xtypep, ndimsp,
dimidsp, nattsp, NULL, NULL, NULL,
NULL, NULL, NULL, NULL, NULL, NULL,
NULL,NULL,NULL);
}

to the simpler and direct

int
nc_inq_var(int ncid, int varid, char *name, nc_type *xtypep,
int *ndimsp, int *dimidsp, int nattsp)
{
NC ncp;
int stat = NC_check_id(parent_ncid,&ncp);
if(stat != NC_NOERR) return stat;
return ncp->dispatch->inq_var(ncid,varid,name,xtypep,ndimsp,dimidsp,nattsp);
}


For the dispatchers, this means that there implementations
would need to include code something like this:

int
NC3_inq_var(int ncid, int varid, char *name, nc_type *xtypep,
int *ndimsp, int *dimidsp, int nattsp)
{
NC ncp;
int stat = NC_check_id(ncid, &ncp);
if(stat != NC_NOERR) return stat;
return NC3_inq_var_all(ncid, varid, name, xtypep, ndimsp,
dimidsp, nattsp, NULL, NULL, NULL,
NULL, NULL, NULL, NULL, NULL, NULL,
NULL,NULL,NULL);
}


The change would be pretty mechanical to implement.

[edhartnett]

Let us consider these functions:

EXTERNL int
nc_inq_var(int ncid, int varid, char *name, nc_type *xtypep,
           int *ndimsp, int *dimidsp, int *nattsp);

EXTERNL int
nc_inq_varname(int ncid, int varid, char *name);

EXTERNL int
nc_inq_vartype(int ncid, int varid, nc_type *xtypep);

EXTERNL int
nc_inq_varndims(int ncid, int varid, int *ndimsp);

EXTERNL int
nc_inq_vardimid(int ncid, int varid, int *dimidsp);

EXTERNL int
nc_inq_varnatts(int ncid, int varid, int *nattsp);

Do we want all of them in the dispatch table? Only the first is needed, and the rest can be written in terms of the first.

[DennisHeimbigner]

That is of course the reason I built the original dispatch table the
way it was. But redundancy would be the price we pay.

[edhartnett]

OK, of course we can also have NCDEFAULT versions of the extra functions, so in practice, the functions would not have to be duplicated in every layer.

[edhartnett]

Also what about the base_pe functions? Are we going to put them back in the dispatch table?

I would like to get rid of the nc_delete_pe() function (in the dispatch table).

[DennisHeimbigner]

I guess so to be consistent.

[edhartnett]

I guess I think your fist instinct was correct: the dispatch table should try to concentrate some functions. That is, I don't think it should have every function. I don't think it should have all the redundant inq functions, and I don't think it should have the pe functions.

Also, currently the dispatch table takes the any-type version of the var/var1/vara/vars/varm functions. If we exactly matched the API we would have to put all those in the dispatch table too. That would explode the size of the table.

How about we keep all those extra functions out, but try to move the ones that we have to be as close to the public signature as possible?

[edhartnett]

To summarize this discussion: we should add dispatch_version to netcdf_meta.h, and set it to 1.

I will put a PR up with this...