h5py_utils: HDF5 I/O utilities#
This module provides utility methods on top of h5py, mainly to handle parallel writing and reading.
- retry(**kw)[source]#
Decorator for a method that needs to be executed until it not longer fails on HDF5 IO. Mainly used for reading an HDF5 file that is being written.
- Parameters:
**kw – see silx.utils.retry
- retry_contextmanager(**kw)[source]#
Decorator to make a context manager from a method that needs to be entered until it not longer fails on HDF5 IO. Mainly used for reading an HDF5 file that is being written.
- Parameters:
**kw – see silx.utils.retry_contextmanager
- retry_in_subprocess(**kw)[source]#
Same as retry but it also retries segmentation faults.
On Window you cannot use this decorator with the “@” syntax:
def _method(*args, **kw): ... method = retry_in_subprocess()(_method)
- Parameters:
**kw – see silx.utils.retry_in_subprocess
- group_has_end_time(h5item)[source]#
Returns True when the HDF5 item is a Group with an “end_time” dataset. A reader can use this as an indication that the Group has been fully written (at least if the writer supports this).
- Parameters:
h5item (Union[h5py.Group,h5py.Dataset])
- Returns bool:
- open_item(filename, name, retry_invalid=False, validate=None, **open_options)[source]#
Yield an HDF5 dataset or group (retry until it can be instantiated).
- Parameters:
filename (str)
retry_invalid (bool) – retry when item is missing or not valid
validate (callable or None)
**open_options – see File.__init__
- Yields Dataset, Group or None:
- top_level_names(filename, include_only=<function group_has_end_time>, **open_options)#
Return all valid top-level HDF5 names.
- Parameters:
filename (str)
include_only (callable or None)
**open_options – see File.__init__
- Returns list(str):
- safe_top_level_names(filename, include_only=<function group_has_end_time>, **open_options)#
Return all valid top-level HDF5 names.
- Parameters:
filename (str)
include_only (callable or None)
**open_options – see File.__init__
- Returns list(str):
- class Hdf5FileLockingManager[source]#
Manage HDF5 file locking in the current process through the HDF5_USE_FILE_LOCKING environment variable.
- class File(filename, mode=None, locking=None, enable_file_locking=None, swmr=None, libver=None, **kwargs)[source]#
Takes care of HDF5 file locking and SWMR mode without the need to handle those explicitely.
When file locking is managed through the HDF5_USE_FILE_LOCKING environment variable, you cannot open different files simultaneously with different modes.