{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "skip" } }, "source": [ "# Abstract\n", "\n", "[**hdf5plugin**](https://github.com/silx-kit/hdf5plugin) is a *Python* package (1) providing a set of [**HDF5**](https://portal.hdfgroup.org/display/HDF5/) compression filters (namely: blosc, bitshuffle, lz4, FCIDECOMP, ZFP, Zstandard) and (2) enabling their use from the *Python* programming language with [**h5py**](https://docs.h5py.org/) a thin, pythonic wrapper around `libHDF5`.\n", "\n", "This presentation illustrates how to use **hdf5plugin** for reading and writing compressed datasets from *Python* and gives an overview of the different HDF5 compression filters it provides.\n", "\n", "It also illustrates how the provided compression filters can be enabled to read compressed datasets from other (non-*Python*) application.\n", "\n", "Finally, it discusses how **hdf5plugin** manages to distribute the HDF5 plugins for reuse with different `libHDF5`.\n", "\n", "License: [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "# Notebook requirements\n", "# A recent version of Pillow is required!\n", "%pip install numpy h5py hdf5plugin h5glance rise jupyterlab matplotlib ipympl Pillow" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "%matplotlib inline\n", "\n", "# Creates data.h5 used for demos\n", "import urllib.request\n", "\n", "import numpy\n", "from matplotlib import pyplot as plt\n", "from PIL import Image\n", "\n", "import h5py\n", "import hdf5plugin\n", "\n", "url = \"https://www.eiroforum.org/wp-content/uploads/pg-esrf-01.jpg\"\n", "filename = urllib.request.urlretrieve(url)[0]\n", "image = numpy.array(Image.open(filename).convert(\"L\"))\n", "plt.imshow(image)\n", "\n", "h5file = h5py.File(\"data.h5\", mode=\"w\")\n", "h5file[\"copyright\"] = \"P.Ginter/ESRF\"\n", "h5file.attrs[\"url\"] = url\n", "h5file.create_dataset(\"/data\", data=image)\n", "h5file.create_dataset(\n", " \"/compressed_data_bitshuffle_lz4\", data=image, **hdf5plugin.Bitshuffle()\n", ")\n", "h5file.close()\n", "\n", "# Restart kernel once the file is created!" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "skip" } }, "outputs": [], "source": [ "%matplotlib inline\n", "from matplotlib import pyplot as plt" ] }, { "cell_type": "markdown", "metadata": { "nbsphinx": "hidden", "slideshow": { "slide_type": "slide" } }, "source": [ "" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# hdf5plugin\n", "\n", "[**hdf5plugin**](https://github.com/silx-kit/hdf5plugin) packages a set of [HDF5](https://portal.hdfgroup.org/display/HDF5/) compression filters (namely: blosc, bitshuffle, lz4, FCIDECOMP, ZFP, Zstandard) and makes them usable from the Python programming language through [h5py](https://docs.h5py.org/).\n", "\n", "[**h5py**](https://docs.h5py.org/) is a thin, pythonic wrapper around [HDF5](https://portal.hdfgroup.org/display/HDF5/).\n", "\n", " \n", "\n", " \n", "\n", "Presenter: Thomas VINCENT\n", "\n", "European HDF Users Group Summer 2021, July 7-8, 2021" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "scrolled": false, "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/html": [ "