h5py_utils: HDF5 I/O utilities#

This module provides utility methods on top of h5py, mainly to handle parallel writing and reading.

retry(**kw)[source]#

Decorator for a method that needs to be executed until it not longer fails on HDF5 IO. Mainly used for reading an HDF5 file that is being written.

Parameters:

**kw – see silx.utils.retry

retry_contextmanager(**kw)[source]#

Decorator to make a context manager from a method that needs to be entered until it not longer fails on HDF5 IO. Mainly used for reading an HDF5 file that is being written.

Parameters:

**kw – see silx.utils.retry_contextmanager

retry_in_subprocess(**kw)[source]#

Same as retry but it also retries segmentation faults.

On Window you cannot use this decorator with the “@” syntax:

def _method(*args, **kw):
    ...

method = retry_in_subprocess()(_method)
Parameters:

**kw – see silx.utils.retry_in_subprocess

group_has_end_time(h5item)[source]#

Returns True when the HDF5 item is a Group with an “end_time” dataset. A reader can use this as an indication that the Group has been fully written (at least if the writer supports this).

Parameters:

h5item (Union[h5py.Group,h5py.Dataset]) –

Returns bool:

open_item(filename, name, retry_invalid=False, validate=None, **open_options)[source]#

Yield an HDF5 dataset or group (retry until it can be instantiated).

Parameters:
  • filename (str) –

  • retry_invalid (bool) – retry when item is missing or not valid

  • validate (callable or None) –

  • **open_options – see File.__init__

Yields Dataset, Group or None:

top_level_names(filename, include_only=<function group_has_end_time>, **open_options)#

Return all valid top-level HDF5 names.

Parameters:
  • filename (str) –

  • include_only (callable or None) –

  • **open_options – see File.__init__

Returns list(str):

safe_top_level_names(filename, include_only=<function group_has_end_time>, **open_options)#

Return all valid top-level HDF5 names.

Parameters:
  • filename (str) –

  • include_only (callable or None) –

  • **open_options – see File.__init__

Returns list(str):

class Hdf5FileLockingManager[source]#

Manage HDF5 file locking in the current process through the HDF5_USE_FILE_LOCKING environment variable.

class File(filename, mode=None, locking=None, enable_file_locking=None, swmr=None, libver=None, **kwargs)[source]#

Takes care of HDF5 file locking and SWMR mode without the need to handle those explicitely.

When file locking is managed through the HDF5_USE_FILE_LOCKING environment variable, you cannot open different files simultaneously with different modes.

close()[source]#

Close the file. All open objects become invalid