Skip to content

Commit

Permalink
Revert H5Dchunk_iter changes in hdf5_1_12_1 (#733)
Browse files Browse the repository at this point in the history 
* Hdf5 1 12 doxygen version correction (#658)

* OESS-98 fix tools test for plugins

* sync fork

* Merge of changes from dev

* Move problem option to bottom of the list until fixed

* HDFFV-11106 - fix parsing optional args

* HDFFV-11106 add note

* grammer fix

* Whitespace after clang formatting

* Undo format version 11 changes

* Update check to working version

* Merge workflow and minor changes from develop

* Update supported platforms

* PR#3 merge from develop

* Merge gcc 10 diagnostics option from develop

* Merge #318 OSX changes from develop

* Merge serval small changes from dev

* fix typo

* Minor non-space formatting changes

* GH #386 copyright corrections for java folder

* revert because logic requires false return

* Merges from develop

* Remove case statement for H5I_EVENTSET

* Correct call with versioning

* Remove tabs

* Double underscore change

* Merges from develop

Remove more underscores from header guards

* Merge #380 from develop

* Correct date entry

* Split format source and commit changes on repo push

* remove pre-split setting

* Change windows TS to use older VS.

* HDFFV-11212 JNI export util and Javadoc

* Suggested review changes

* Another change found

* Committing clang-format changes

* Some Javadoc warning fixes

* Committing clang-format changes

* Updated javadoc fixes

* HDFFV-11228/9 merges from develop

* remove obsolete debug comment

* Fix conflict

* HDFFV-11229 merge changes from develop

* HDFFV-11229 merge second compare from develop

* HDFFV-11229 fix reference file

* HDFFV-11229 update autotools test script for two ref files

* HDFFV-11229 merge dev changes for long double display in tools

* Committing clang-format changes

* Update with changes from develop

* Add "option" command for clang options

* Rework CMake add_custom to use the BYPRODUCTS argument

Update pkgconfig scripts for parallel builds.
Fix install COPYING file reference.
Remove unused round defines.
Change CMake default setting of BUILD_CPP to off.

* Whitespace changes

* Rework CMake add_custom to use the BYPRODUCTS argument

* Revert CMake configure checks for round defines

* With VS 2015 minimum strdup is supported

* Doxygen comments merged from develop

* doxygen build updates

* Correct version string for map functions

* TRILAB-227 and tools debug merge from develop

* TRILAB-227 Enable test

* Quote subset args

* Use MATCHES in compiler id compare, merge from dev

* Revert test enable

* Add file to list

* doxygen version errors

* if blocks needed for build modes

* Update list of test platforms

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Attempt format uniformity in RELEASE.txt.

* BUG: DataSet assignment operator is missing (#503)

* BUG: DataSet assignment operator is missing

* Some compilers complain if the copy constructor is given explicitly
  but the assignment operator is implicitly set to default.

* Explicitly defining the assignment operator allows us to properly
  handle reference counters for shared resources.

* BUG: DataSet assignment operator is missing.

* Mimicking code of H5DataType::operator() as suggested by @bmribler.

* Added test

Description:
    Added test for DataSet::operator= that Leengit added
Platform tested:
    Linux/64 (jelly)

* Removed Author field.

* Commit clang format changes.

* Entry for Leengit's github PR #503

* Removed lines left by mistake

Co-authored-by: Binh-Minh Ribler <[email protected]>
Co-authored-by: Larry Knox <[email protected]>

* Add H5Dchunk_iter method for iterating over chunks (#6)

* Add H5Dchunk_iter method for iterating over chunks

This method iterates over all chunks in dataset, calling a user-supplied
callback with the chunk information and optional user supplied data.

The iterator is stopped when ITER_STOP is returned by the user-supplied
callback or the iterator is exhausted.

Existing methods to get chunk_info performs an iteration each time, so
to get many or all chunks causes SUM(i) for i = 0 -> N operations for N
chunks, as opposed to N operations when using this iterator for this use case.

* H5Dchunk_iter: test iterating all chunks, some chunks and failing iteration.

* H5D: move H5Dchunk_iter private methods to specific

* trace: add H5D_chunk_iter_op_t and trace H5D.c

* chunks-iter: document chunk_iter

* chunk-iter: chunk add FUNC_ENTER/FUNC_LEAVE macros

* Committing clang-format changes

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* Set initial doxygen version for H5Dchunk_iter().

* Revert "Set initial doxygen version for H5Dchunk_iter()."

This reverts commit d739e46.

* Revert "Add H5Dchunk_iter method for iterating over chunks (#6)"

This reverts commit 53e1139.

* Add flushrefresh errors with check-passthrough-vol to known problems in
RELEASE.txt.

* Minor RELEASE.txt updates.

* Hdf5 1 12 Remove version from pkgcfg file names (#672)

* OESS-98 fix tools test for plugins

* sync fork

* Merge of changes from dev

* Move problem option to bottom of the list until fixed

* HDFFV-11106 - fix parsing optional args

* HDFFV-11106 add note

* grammer fix

* Whitespace after clang formatting

* Undo format version 11 changes

* Update check to working version

* Merge workflow and minor changes from develop

* Update supported platforms

* PR#3 merge from develop

* Merge gcc 10 diagnostics option from develop

* Merge #318 OSX changes from develop

* Merge serval small changes from dev

* fix typo

* Minor non-space formatting changes

* GH #386 copyright corrections for java folder

* revert because logic requires false return

* Merges from develop

#358 patches from vtk
#361 fix header guard spelling

* Remove case statement for H5I_EVENTSET

* Correct call with versioning

* Remove tabs

* Double underscore change

* Merges from develop

#340 clang -Wformat-security warnings
#360 Fixed uninitialized warnings
Remove more underscores from header guards

* Merge #380 from develop

* Correct date entry

* Split format source and commit changes on repo push

* remove pre-split setting

* Change windows TS to use older VS.

* HDFFV-11212 JNI export util and Javadoc

* Suggested review changes

* Another change found

* Committing clang-format changes

* Some Javadoc warning fixes

* Committing clang-format changes

* Updated javadoc fixes

* HDFFV-11228/9 merges from develop

* remove obsolete debug comment

* Fix conflict

* HDFFV-11229 merge changes from develop

* HDFFV-11229 merge second compare from develop

* HDFFV-11229 fix reference file

* HDFFV-11229 update autotools test script for two ref files

* HDFFV-11229 merge dev changes for long double display in tools

* Committing clang-format changes

* Update with changes from develop

* Add "option" command for clang options

* Rework CMake add_custom to use the BYPRODUCTS argument

Update pkgconfig scripts for parallel builds.
Fix install COPYING file reference.
Remove unused round defines.
Change CMake default setting of BUILD_CPP to off.

* Whitespace changes

* Rework CMake add_custom to use the BYPRODUCTS argument

* Revert CMake configure checks for round defines

* With VS 2015 minimum strdup is supported

* Doxygen comments merged from develop

* doxygen build updates

* Correct version string for map functions

* TRILAB-227 and tools debug merge from develop

* TRILAB-227 Enable test

* Quote subset args

* Use MATCHES in compiler id compare, merge from dev

* Revert test enable

* Add file to list

* doxygen version errors

* if blocks needed for build modes

* Update list of test platforms

* Issue #669 remove version from pkgcfg filename

* remove version from h5cc script

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>

* More minor updates.

* Update configure disablhing memory alloc sanity check for release.
Last 50 lines of
/mnt/scr1/SnapTest/snapshots-hdf5-branchtest4/log/moohan_0525Tue_1021
Last 50 lines of
/mnt/scr1/SnapTest/snapshots-hdf5-branchtest4/log/moohan_0525Tue

* Revert "Add H5Dchunk_iter method for iterating over chunks.

Co-authored-by: Allen Byrne <[email protected]>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Lee Newberg <[email protected]>
Co-authored-by: Binh-Minh Ribler <[email protected]>
Co-authored-by: Gaute Hope <[email protected]>
  • Loading branch information
@lrknox @byrnHDF
@github-actions @Leengit @bmribler @gauteh
6 people authored Jun 7, 2021
1 parent 6ad3f5e commit 10abe9a
Show file tree
Hide file tree
Showing 8 changed files with 19 additions and 343 deletions.
1 change: 0 additions & 1 deletion bin/trace
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,6 @@ $Source = "";
"H5D_vds_view_t" => "Dv",
"H5FD_mpio_xfer_t" => "Dt",
"H5FD_splitter_vfd_config_t" => "Dr",
"H5D_chunk_iter_op_t" => "x",
"herr_t" => "e",
"H5E_direction_t" => "Ed",
"H5E_error_t" => "Ee",
Expand Down
65 changes: 0 additions & 65 deletions src/H5D.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1126,68 +1126,3 @@ H5Dget_chunk_info_by_coord(hid_t dset_id, const hsize_t *offset, unsigned *filte
done:
FUNC_LEAVE_API(ret_value)
} /* end H5Dget_chunk_info_by_coord() */

/*-------------------------------------------------------------------------
* Function: H5Dchunk_iter
*
* Purpose: Iterates over all chunks in dataset with given callback and user data.
*
* Parameters:
* hid_t dset_id; IN: Chunked dataset ID
* H5D_chunk_iter_op_t cb IN: User callback function, called for every chunk.
* void *op_data IN/OUT: Optional user data passed on to user callback.
*
* Callback information:
* H5D_chunk_iter_op_t is defined as:
*
* typedef int (*H5D_chunk_iter_op_t)(
* const hsize_t *offset,
* uint32_t filter_mask,
* haddr_t addr,
* uint32_t nbytes,
* void *op_data);
*
* H5D_chunk_iter_op_t parameters:
* hsize_t *offset; IN/OUT: Array of starting logical coordinates of chunk.
* uint32_t filter_mask; IN: Filter mask of chunk.
* haddr_t addr; IN: Offset in file of chunk data.
* uint32_t nbytes; IN: Size in number of bytes of chunk data in file.
* void *op_data; IN/OUT: Pointer to any user-defined data
* associated with the operation.
*
* The return values from an operator are:
* Zero (H5_ITER_CONT) causes the iterator to continue, returning zero when all
* elements have been processed.
* Positive (H5_ITER_STOP) causes the iterator to immediately return that positive
* value, indicating short-circuit success.
* Negative (H5_ITER_ERROR) causes the iterator to immediately return that value,
* indicating failure.
*
* Return: Non-negative on success, negative on failure
*
* Programmer: Gaute Hope
* August 2020
*
*-------------------------------------------------------------------------
*/
herr_t
H5Dchunk_iter(hid_t dset_id, H5D_chunk_iter_op_t cb, void *op_data)
{
H5VL_object_t *vol_obj = NULL; /* Dataset for this operation */
herr_t ret_value = SUCCEED;

FUNC_ENTER_API(FAIL)
H5TRACE3("e", "ix*x", dset_id, cb, op_data);

/* Check arguments */
if (NULL == (vol_obj = (H5VL_object_t *)H5I_object_verify(dset_id, H5I_DATASET)))
HGOTO_ERROR(H5E_ARGS, H5E_BADTYPE, FAIL, "invalid dataset identifier")

/* Call private function to get the chunk info given the chunk's index */
if (H5VL_dataset_specific(vol_obj, H5VL_DATASET_CHUNK_ITER, H5P_DATASET_XFER_DEFAULT, H5_REQUEST_NULL, cb,
op_data) < 0)
HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "Can't iterate over chunks")

done:
FUNC_LEAVE_API(ret_value)
} /* end H5Dchunk_iter() */
98 changes: 0 additions & 98 deletions src/H5Dchunk.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -245,11 +245,6 @@ typedef struct H5D_chunk_coll_info_t {
} H5D_chunk_coll_info_t;
#endif /* H5_HAVE_PARALLEL */

typedef struct H5D_chunk_iter_cb_data_t {
H5D_chunk_iter_op_t cb; /* User defined callback */
void * op_data; /* User data for user defined callback */
} H5D_chunk_iter_cb_data_t;

/********************/
/* Local Prototypes */
/********************/
Expand All @@ -274,7 +269,6 @@ static herr_t H5D__chunk_dest(H5D_t *dset);
static int H5D__get_num_chunks_cb(const H5D_chunk_rec_t *chunk_rec, void *_udata);
static int H5D__get_chunk_info_cb(const H5D_chunk_rec_t *chunk_rec, void *_udata);
static int H5D__get_chunk_info_by_coord_cb(const H5D_chunk_rec_t *chunk_rec, void *_udata);
static int H5D__chunk_iter_cb(const H5D_chunk_rec_t *chunk_rec, void *udata);

/* "Nonexistent" layout operation callback */
static ssize_t H5D__nonexistent_readvv(const H5D_io_info_t *io_info, size_t chunk_max_nseq,
Expand Down Expand Up @@ -7457,95 +7451,3 @@ H5D__get_chunk_info_by_coord(const H5D_t *dset, const hsize_t *offset, unsigned
done:
FUNC_LEAVE_NOAPI_TAG(ret_value)
} /* end H5D__get_chunk_info_by_coord() */

/*-------------------------------------------------------------------------
* Function: H5D__chunk_iter
*
* Purpose: Iterate over all the chunks in the dataset with given callbak.
*
* Return: Success: Non-negative
* Failure: Negative
*
* Programmer: Gaute Hope
* August 2020
*
*-------------------------------------------------------------------------
*/
herr_t
H5D__chunk_iter(const H5D_t *dset, H5D_chunk_iter_op_t cb, void *op_data)
{
const H5O_layout_t *layout = NULL; /* Dataset layout */
const H5D_rdcc_t * rdcc = NULL; /* Raw data chunk cache */
H5D_rdcc_ent_t * ent; /* Cache entry index */
H5D_chk_idx_info_t idx_info; /* Chunked index info */
herr_t ret_value = SUCCEED; /* Return value */

FUNC_ENTER_PACKAGE_TAG(dset->oloc.addr)

/* Check args */
HDassert(dset);
HDassert(dset->shared);

/* Get dataset layout and raw data chunk cache */
layout = &(dset->shared->layout);
rdcc = &(dset->shared->cache.chunk);
HDassert(layout);
HDassert(rdcc);
HDassert(H5D_CHUNKED == layout->type);

/* Search for cached chunks that haven't been written out */
for (ent = rdcc->head; ent; ent = ent->next)
/* Flush the chunk out to disk, to make certain the size is correct later */
if (H5D__chunk_flush_entry(dset, ent, FALSE) < 0)
HGOTO_ERROR(H5E_IO, H5E_WRITEERROR, FAIL, "cannot flush indexed storage buffer")

/* Compose chunked index info struct */
idx_info.f = dset->oloc.file;
idx_info.pline = &dset->shared->dcpl_cache.pline;
idx_info.layout = &dset->shared->layout.u.chunk;
idx_info.storage = &dset->shared->layout.storage.u.chunk;

/* If the dataset is not written, return without errors */
if (H5F_addr_defined(idx_info.storage->idx_addr)) {
H5D_chunk_iter_cb_data_t data;
data.cb = cb;
data.op_data = op_data;

/* Iterate over the allocated chunks calling the iterator callback */
if ((dset->shared->layout.storage.u.chunk.ops->iterate)(&idx_info, H5D__chunk_iter_cb, &data) < 0)
HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "unable to iterate over chunks.")
} /* end if H5F_addr_defined */

done:
FUNC_LEAVE_NOAPI_TAG(ret_value)
} /* end H5D__chunk_iter() */

/*-------------------------------------------------------------------------
* Function: H5D__chunk_iter_cb
*
* Purpose: Call the user-defined function with the chunk data. The iterator continues if
* the user-defined function returns H5_ITER_CONT, and stops if H5_ITER_STOP is
* returned.
*
* Return: Success: H5_ITER_CONT or H5_ITER_STOP
* Failure: Negative (H5_ITER_ERROR)
*
* Programmer: Gaute Hope
* August 2020
*
*-------------------------------------------------------------------------
*/
static int
H5D__chunk_iter_cb(const H5D_chunk_rec_t *chunk_rec, void *udata)
{
int ret_value = 0;

FUNC_ENTER_STATIC_NOERR

const H5D_chunk_iter_cb_data_t *data = (H5D_chunk_iter_cb_data_t *)udata;

ret_value = (data->cb)(chunk_rec->scaled, chunk_rec->filter_mask, chunk_rec->chunk_addr,
chunk_rec->nbytes, data->op_data);

FUNC_LEAVE_NOAPI(ret_value)
} /* end H5D__chunk_iter_cb */
1 change: 0 additions & 1 deletion src/H5Dpkg.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -565,7 +565,6 @@ H5_DLL herr_t H5D__get_chunk_info(const H5D_t *dset, const H5S_t *space, hsize_
unsigned *filter_mask, haddr_t *offset, hsize_t *size);
H5_DLL herr_t H5D__get_chunk_info_by_coord(const H5D_t *dset, const hsize_t *coord, unsigned *filter_mask,
haddr_t *addr, hsize_t *size);
H5_DLL herr_t H5D__chunk_iter(const H5D_t *dset, H5D_chunk_iter_op_t cb, void *op_data);
H5_DLL haddr_t H5D__get_offset(const H5D_t *dset);
H5_DLL herr_t H5D__vlen_get_buf_size(H5D_t *dset, hid_t type_id, hid_t space_id, hsize_t *size);
H5_DLL herr_t H5D__vlen_get_buf_size_gen(H5VL_object_t *vol_obj, hid_t type_id, hid_t space_id,
Expand Down
55 changes: 0 additions & 55 deletions src/H5Dpublic.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -160,14 +160,6 @@ typedef herr_t (*H5D_scatter_func_t)(const void **src_buf /*out*/, size_t *src_b
typedef herr_t (*H5D_gather_func_t)(const void *dst_buf, size_t dst_buf_bytes_used, void *op_data);
//! <!-- [H5D_gather_func_t_snip] -->

//! <!-- [H5D_chunk_iter_op_t_snip] -->
/**
* Define the operator function pointer for H5Dchunk_iter()
*/
//! <!-- [H5D_chunk_iter_op_t_snip] -->
typedef int (*H5D_chunk_iter_op_t)(const hsize_t *offset, uint32_t filter_mask, haddr_t addr, uint32_t nbytes,
void *op_data);

/********************/
/* Public Variables */
/********************/
Expand Down Expand Up @@ -609,53 +601,6 @@ H5_DLL herr_t H5Dget_num_chunks(hid_t dset_id, hid_t fspace_id, hsize_t *nchunks
H5_DLL herr_t H5Dget_chunk_info_by_coord(hid_t dset_id, const hsize_t *offset, unsigned *filter_mask,
haddr_t *addr, hsize_t *size);

/**
* --------------------------------------------------------------------------
* \ingroup H5D
*
* \brief Iterate over all chunks
*
* \dset_id
* \param[in] cb User callback function, called for every chunk.
* \param[in] op_data User-defined pointer to data required by op
*
* \return \herr_t
*
* \details H5Dget_chunk_iter iterates over all chunks in the dataset, calling the
* user supplied callback with the details of the chunk and the supplied
* \p op_data.
*
* Callback information:
* H5D_chunk_iter_op_t is defined as:
*
* typedef int (*H5D_chunk_iter_op_t)(
* const hsize_t *offset,
* uint32_t filter_mask,
* haddr_t addr,
* uint32_t nbytes,
* void *op_data);
*
* H5D_chunk_iter_op_t parameters:
* hsize_t *offset; IN/OUT: Array of starting logical coordinates of chunk.
* uint32_t filter_mask; IN: Filter mask of chunk.
* haddr_t addr; IN: Offset in file of chunk data.
* uint32_t nbytes; IN: Size in number of bytes of chunk data in file.
* void *op_data; IN/OUT: Pointer to any user-defined data
* associated with the operation.
*
* The return values from an operator are:
* Zero (H5_ITER_CONT) causes the iterator to continue, returning zero when all
* elements have been processed.
* Positive (H5_ITER_STOP) causes the iterator to immediately return that positive
* value, indicating short-circuit success.
* Negative (H5_ITER_ERROR) causes the iterator to immediately return that value,
* indicating failure.
*
* \since 1.12.1
*
*/
H5_DLL herr_t H5Dchunk_iter(hid_t dset_id, H5D_chunk_iter_op_t cb, void *op_data);

/**
* --------------------------------------------------------------------------
* \ingroup H5D
Expand Down
4 changes: 1 addition & 3 deletions src/H5VLconnector.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,7 @@ typedef enum H5VL_dataset_get_t {
typedef enum H5VL_dataset_specific_t {
H5VL_DATASET_SET_EXTENT, /* H5Dset_extent */
H5VL_DATASET_FLUSH, /* H5Dflush */
H5VL_DATASET_REFRESH, /* H5Drefresh */
H5VL_DATASET_WAIT, /* H5Dwait */
H5VL_DATASET_CHUNK_ITER /* H5Dchunk_iter */
H5VL_DATASET_REFRESH /* H5Drefresh */
} H5VL_dataset_specific_t;

/* Typedef for VOL connector dataset optional VOL operations */
Expand Down
25 changes: 0 additions & 25 deletions src/H5VLnative_dataset.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -346,31 +346,6 @@ H5VL__native_dataset_specific(void *obj, H5VL_dataset_specific_t specific_type,
break;
}

case H5VL_DATASET_WAIT: { /* H5Dwait */
/* The native VOL connector doesn't support asynchronous
* operations, so this is a no-op.
*/
break;
}

case H5VL_DATASET_CHUNK_ITER: { /* H5Dchunk_iter */
H5D_chunk_iter_op_t cb = HDva_arg(arguments, H5D_chunk_iter_op_t);
void * op_data = HDva_arg(arguments, void *);

HDassert(dset->shared);

/* Make sure the dataset is chunked */
if (H5D_CHUNKED != dset->shared->layout.type) {
HGOTO_ERROR(H5E_ARGS, H5E_BADTYPE, FAIL, "not a chunked dataset")
}

/* Call private function */
if (H5D__chunk_iter(dset, cb, op_data) < 0)
HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "can't iterate over chunks")

break;
}

default:
HGOTO_ERROR(H5E_VOL, H5E_UNSUPPORTED, FAIL, "invalid specific operation")
} /* end switch */
Expand Down
Loading

0 comments on commit 10abe9a

Please sign in to comment.