h5py_utils: HDF5 I/O utilities

This module provides utility methods on top of h5py, mainly to handle parallel writing and reading.

retry(**kw)[source]

Decorator for a method that needs to be executed until it not longer fails on HDF5 IO. Mainly used for reading an HDF5 file that is being written.

Parameters:**kw – see silx.utils.retry
retry_contextmanager(**kw)[source]

Decorator to make a context manager from a method that needs to be entered until it not longer fails on HDF5 IO. Mainly used for reading an HDF5 file that is being written.

Parameters:**kw – see silx.utils.retry_contextmanager
retry_in_subprocess(**kw)[source]

Same as retry but it also retries segmentation faults.

On Window you cannot use this decorator with the “@” syntax:

def _method(*args, **kw):
    ...

method = retry_in_subprocess()(_method)
Parameters:**kw – see silx.utils.retry_in_subprocess
group_has_end_time(h5item)[source]

Returns True when the HDF5 item is a Group with an “end_time” dataset. A reader can use this as an indication that the Group has been fully written (at least if the writer supports this).

Parameters:h5item (Union[h5py.Group,h5py.Dataset]) –
Returns bool:
open_item(filename, name, retry_invalid=False, validate=None, **open_options)[source]

Yield an HDF5 dataset or group (retry until it can be instantiated).

Parameters:
  • filename (str) –
  • retry_invalid (bool) – retry when item is missing or not valid
  • or None validate (callable) –
  • **open_options – see File.__init__
Yields Dataset, Group or None:
 
top_level_names(filename, include_only=<function group_has_end_time>, **open_options)

Return all valid top-level HDF5 names.

Parameters:
  • filename (str) –
  • or None include_only (callable) –
  • **open_options – see File.__init__
Returns list(str):
 
safe_top_level_names(filename, include_only=<function group_has_end_time>, **open_options)

Return all valid top-level HDF5 names.

Parameters:
  • filename (str) –
  • or None include_only (callable) –
  • **open_options – see File.__init__
Returns list(str):
 
class Hdf5FileLockingManager[source]

Manage HDF5 file locking in the current process through the HDF5_USE_FILE_LOCKING environment variable.

class File(filename, mode=None, locking=None, enable_file_locking=None, swmr=None, libver=None, **kwargs)[source]

Takes care of HDF5 file locking and SWMR mode without the need to handle those explicitely.

When file locking is managed through the HDF5_USE_FILE_LOCKING environment variable, you cannot open different files simultaneously with different modes.

close()[source]

Close the file. All open objects become invalid