Change-log of versions

2023.03

  • Rework all 2D integrator to propagate variance, expose standard deviation and standard error of the mean.

  • Support all version of python between 3.7 and 3.11

  • Support Cython 3.0b1 (with performance degradation) in addition of cython 0.29.31+

2023.02 27/02/2023

  • Core:
    • sigma_clip_ng is not the default sigma_clip

    • expose documentation for sigma_clip_legacy in addition to sigma_clip_ng

    • Worker can now use sigma_clip_legacy in addition to sigma_clip_ng, integrate1d or integrate2d

    • New calibrant: Vanadinite (used in high-presure, often as single crystal)

  • GUI:
    • Fix labels on axes for some rarely used units

    • refactor IntegrationMethodDialog and share the IntegrationMethodTabs with pyFAI-calib2

  • Test:
    • silent some noisy test (OpenCL on windows)

2023.01 16/01/2023

  • Developer and packager tools:
    • Switch build system from numpy.distutils to meson-python

    • Keep the former setup.py for compatibility reasons: it will be removed in a future release

    • Drop Python 3.6 (default parameters in namedtuple feature used)

    • Require silx 1.1 (for OpenCL), scipy and matplotlib

  • GUI side:
    • several minor improvements in pyFAI-calib2

    • Fixed calibration in jupyter-lab

  • Core improvements:
    • Refactoring of the Geometry class

    • Geometry pseudo-inversion optimization

    • Improved support from Medipix-based Lambda-detectors

    • New detectors from Dectris (Pilatus 900k and Eiger 250k)

    • Support Nexus format in output: NXmonpd and NXcansas

    • Single-threaded CSC sparse matrix multiplication engine

  • Improved uncertainty propagation:
    • Refactor error model management (uses enum)

    • Hybrid error model (azimuthal for sigma-clipping but reports Poissonian noise)

    • Export peakfinder data to the CXI format (used by CrystFEL)

  • Improvement in the doc:
    • Update installation instructions

    • Multi-threaded integration tutorial

    • GPU implementation tutorial

  • Facts and figures:
    • 400+ commits, 100 PR

    +with the contribution of:

    Clemens Prescher, Elena Pascal, Jérôme Kieffer, Malte Storm, Marco Cammarata, Michael Hudson-Doyle, Picca Frédéric-Emmanuel, Rodrigo Telles, Thomas A Caswell, Tommaso Vinci, Valentin Valls, Wout de Nolf.

0.21.3 24/03/2022

  • Support hexagonal pixels in splitbbox and nosplit mode (collaboration with EuXFEL)

  • Support extra dtype in OpenCL (contribution from Desy)

  • Fix version of setuptools (enables to build pyFAI -> python 3.12)

  • Support some of the Lambda detectors from XSpectrum (collaboration with Xmas)

  • Restore back compatibility in histoBBox1d and histoBBox2d (contribution from NSLS-II)

  • Fix bug in OpenCL distortion correction (collaboration with Soleil)

  • Several other bugs fixed

0.21.2 03/02/2022

Fix documentation of sparsify-Bragg and peakfinder CLI tools

0.21.1 02/02/2022

Bug-fix release with: - issue in propagated error not reported with certain engines - diff_map tool can now perform 2D integration - issue with dummy/dela_dummy in Worker - fix improper reset of the integrator due to cabling issue inside AzimuthalIntegrator - Remove deprecation warning about scipy - Use silx for median-filter which is faster than scipy - skip 3 tests which are known to be fragile on 32-bit platforms (spotted by debian on i386 and armel)

0.21.0 20/01/2022

  • One year of development: 523 commits, close to 100 pull-requests. +53000 lines of code and 20000 suppressed.

  • Sigma-clipping allows separation of Bragg/amorphous signal:
    • Implementation in Python, Cython and OpenCL with poissonian and azimuthal error-model

    • Sparsification, compression of single crystal data
      • Application to peak-picking and quality assessement of SSX data

      • Analysis of grid-scan to find single crystal

      • Single pass variance propagation in azimuthal bin

      • Integration of the Jungfrau detector (ID29)

  • 2D integration:
    • New integrators with error propagation by default

    • Full pixel splitting in addition to BBox and no splitting

  • Refactor of all LUT and CSR to share the same code base, makes tests more robust.

  • Calibration of experimental setup using Jupyter notebooks
    • User interaction with plots in matplotlib (thanks Phil Hans)

    • Factorize code between pyFAI-calib and jupyter calibration

    • Tutorial as notebook and video recording

  • Parallax correction for thick detector (still experimental, thanks to Vadim)

  • Improved tutorial on detector geometry calibration (based on Kabsch alignment)

  • Better performances on HPC nodes by limiting simple OpenMP to fewer cores

  • Many improvement in test, typos fixed, doc …

  • Deprecate all legacy integrators since the new version sees its matrix complete.

  • Supports Python 3.6 … 3.10 under Windows, MacOS and Linux.

0.20.0 22/01/2021

  • One year of development, about 500 commits & 370 files modified.

  • Generalization of the new generation 1D integrators (better error propagation)

  • Sigma clipping and sparsification of single crystal data (OpenCL only)

  • Fix issue introduced with the scipy 1.15 (constrained calibration broken)

  • Improved distortion correction (also on GPU, …)

  • Major re-work of the documentation (thanks Thomas Kluyver and Loic Huder)

  • Improve the calibration of Pilatus and Eiger detectors based on a grid of holes.

  • New cylindrical detector from Rigaku

  • Drop deprecated OpenCL integrator

  • Support all Eiger2 detectors (thanks Clemens Weninger and Marie Ruat)

  • CI: move to Gihub workflow and gitlab-runners (bob) for building (thanks Thomas Vincent).

  • Build for debian 10 and 11 (also ubuntu 20.04), drop debian 9

  • Remove Python2 related code

  • Lower Numpy ABI dependency as much as possible (remains _distortion’s C++ code)

  • Drop tests for Python 3.5, checked on 3.9 as well.

0.19.0 31/01/2020

  • Minor revison with only 150 commits, mainly bug-fixes

  • Improvement on the GUI with many small bug-fixes

  • Support newer h5py (mode mandatory, [()], …)

  • Build for debian 10 and 11 (also ubuntu 20.04)

  • Drop tests for Python 2 and 3.4, checked on 3.8

  • Improved compatibility with ImageD11

  • Use hdf5plugin to provide hdf5 io-filters in apps

  • Rework diffraction mapping tools to use a worker

  • New generation azimuthal integrator using CSR algorithm implemented in Python, Cython and OpenCL.

  • Sigma-clipping implemented in OpenCL

  • Publication on new generation integration, the GUI for calibration and the goniometers accepted in J. Synch. Rad. DOI: 10.1107/S1600577520000776

  • A big thank to Florian from Germany, Bertrand from Xenocs, Alex from Soleil and Jon from ESRF for their contributions.

0.18.0 15/05/2019

  • Last release with Valentin as he finishes his contract soon

  • almost 800 commits, 60 PR since the last release: this is a huge release !

  • Major rework on all GUIs, mainly pyFAI-calib2 and pyFAI-integrate.

  • Possibility to integrate image stacks (i.e. from HDF5), …

  • Rework the method to specify the algorithm, pixel splitting and implementation with sensible fall-backs. Also available via the different GUIs

  • 3D visualization of detectors and experimental setup, useful for non flat detectors.

  • integrate1d_ng is available with histogramming without pixel splitting in Python, Cython and OpenCL. Now, propagates the variance properly !

  • IO sub-packages with associated refactoring for ponifile, json, …

  • Improved management of OpenMP: simplify the code for histograms.

  • Improved geometry description and tutorial for writing exchange with other software (ImageD11, thanks to Carsten Detlefs).

  • More reliable simple ellipse fitting with tests and doc.

  • Better POCL integration (debugged on cuda, x87, Power9, …)

  • Rely on silx mechanics for the build, test, download, GUI, opencl …

  • Many new tutorials, now available on binder-hub: new calibrants, Pilatus calibration, …

  • Fix many issues reported in third-party software (Dioptas, …)

  • Drop support of debian8, Python 2.7 and Python 3.4 on all platforms. It is likely most functionalities still work but without guaranty.

0.17.0 19/12/2018

  • Only 200 commits in a couple of month, this is a small release

  • Fix major bugs in pyFAI-calib2 (double validator, initial guess, ring position)

  • Constrains have been added to the geometry fitting of pyFAI-calib2

  • New pyFAI-integrate graphical application

  • Much better support for user defined detector (HDF5)

  • Start the rewrite of all integrators to allow proper error propagation (2D done)

  • Factorize the preprocessing steps for many integrators

  • Remove tons of code which has been deprecated for years in AzimuthalIntegrator

  • Featuring contribution from Soleil and Berkeley

  • Stop supporting Python2.7 on Windows (there won’t be wheels!)

  • All scripts are now using Python 3.x (x>=4)

  • This is the last release supporting Python 2.7, 3.4 hence debian 8

0.16.0: 26/10/2018

  • Almost 800 commits since 0.15 !

  • Huge improvements on the graphical application for calibration

  • New detector definition (with manufacturers)

  • Improved tests: the GUI is now tested

  • Preparation for changing all rebinning engines (see variance tutorials)

  • Azimuthal integrators (and most other objects) are now serializable with Pickle

  • New distortion correction using the SparseBuilder C++ code

  • New PONI-file format (detector definition changed)

  • Isocontour is now provided by silx

  • Peak-picker clean up (better peak selection near gaps)

  • new Goniometer refinement with enhanced rotation using Euler angles

  • Updated documentation: new cookbooks and tutorials about: - The use of the calibration application (cookbook) - log-scale integration of SAXS data (notebook) - Variance propagation (notebook)

0.15.0: 01/02/2018

  • 150 commits since last revision

  • New tutorials on image inpainting, sensor thickness correction, …

  • Improve scripts

  • Improve the new calibration GUI (pyFAI-calib2)

  • Use scipy physical constants instead of hard-coded values

  • Improved detector serialization and binning assessement

  • Update documentation (images, better rendering of notebook & tutorials)

  • Converge project with silx and fabio

  • Remove generated rst- and C-files from repository

  • This is the last version supporting python2.7

0.14.2: 14/09/2017

  • Fix seg-fault with manylinux1 wheels, in fastcrc module (thanks Thomas)

  • Fix Qt4-Qt5 compatibility (thanks Vadim)

  • Easier to understand geometry transformation (thanks Jon)

  • Lower memory consumption, better cache management

  • Unified debian packaging working for 6->9

  • New detector: Mythen & CirPad (thanks Fred)

  • Clean up debug code which avoid to use pyFAI-calib2

  • pyFAI-calib2 now expect fabio >= 0.5

  • Fix issue with metadata saving in 1d

  • Fix performance regression with pyopencl >2015.2 (Thanks Andreas)

  • pyFAI saxs and waxs scripts guess now the binning of the detector (thanks Fred).

0.14.1: 25/07/2017

  • Fixes Debian 7 and 8 packages

0.14.0: 20/07/2017

  • Graphical user interface for calibration (pyFAI-calib2)

  • Goniometer calibration tools and multi-geometry enhancements

  • Integration scripts and averaging scripts are now able to normalize the data from monitors found in the header.

  • Propagate metadata information as part of the integrated data

  • Common pre-processing factorization on Python, Cython and OpenCL

  • Test clean up and acceleration (avoid tests on too large images)

  • Many new tutorials http://pyfai.readthedocs.io/en/latest/usage/tutorial/index.html

  • New averaging / integration methods: - Azimuthal median filtering - Azimuthal trimmed mean - sigma-clipping on azimuthal angle - Radial averaging

  • Diffraction image inpainting to fill-up the gaps with plausible values.

  • This release correspond to 572 commits

  • Change of license: now all pyFAI is MIT license.

0.13.0: 01/12/2016

  • Global improvement of tests, packaging, code quality, documentation and project tools

  • Scripts
    • Add support for multiframe formats on pyFAI-average

    • Add support for monitoring correction from header file (on pyFAI-average)

    • Add progressbar in the shell (on pyFAI-average and pyFAI-integrate)

    • Script drawMask_pymca is renamed into pyFAI-drawmask

    • Rework of the drawmask GUI using silx

    • pyFAI-drawmask do not have anymore hard dependency on PyMCA

    • pyFAI-integrate can now be used without qt dependency (–no-gui)

    • Fix the script to support both Python 2 and 3 (pyFAI-calib, pyFAI-benchmark)

    • Fix selection of units on diff-map (the user selection was not propagated)

  • For users
    • More source code in MIT license

    • Update name and specification for cameras

    • Add cameras: Eiger500k, RaspberryPi5M, RaspberryPi8M

    • Fix Xpad S540 flat detector geometry

    • Fix definition of CeO2 calibrant

    • Add mask and flat on multi-geometry

    • Fix solid angle of the multi-geometry

    • Fix geometry processing for custom output space

    • Fix normalization factor and variance

    • Add support for Qt5

    • Add support for Debian 9 packaging

  • For developers
    • Create common preprocessing for distortion correction

    • Create common image preprocessing using Cython (NaN filter, flatfield, dark, polarisation)

    • Refactoring of units module. It allows to register custom units.

    • Worker can now use Writer

    • Worker polarization argument is renamed into polarization_factor

    • Remove the dependency from python-fftw3, use numpy instead

    • Remove QtWebKit dependency

    • Fix un-correction of images using sparse matrix from scipy

0.12.0: 06/06/2016

  • Continuous integration on linux, windows using Python 2.7 and 3.4+

  • Drop support of Python 2.6, 3.2, 3.3 and debian6 packaging

  • New radial output units: Reciprocal spacing squared and log(q) ID02

  • GPU accelerate version of ai.separate (Bragg & amorphous) ID13

  • Quantile filtering in pyFAI-average ID02

  • New graphical application for diffraction imaging ID21

  • Migrate to a common structure with silx (reorganize tests, benchmarks, …)

  • Extensions (binary sub-modules) have all been moved to ext directory

  • Many improvements multigeometry integrators

  • Compatibility with the copy module (copy.deepcopy) for azimuthal integrator ID02

  • Distortion correction works also for non-contiguous detectors

  • Update documentation and provide advanced tutorials:
    • Introduction to pyFAI using the jupyter notebook

    • detector calibration ID15, BM02

    • Correction of detector distortion, examples of pixel detectors.

    • calibrant calculation ID30

    • error handling ID02, BM29

  • pyFAI-integrate can now be used with or without GUI

  • Many new detectors (ADSC, Pilatus CdTe, Apex II, Pixium):
    • support for non-flat/curved detectors (Aarhus)

    • non-contiguous detectors (WOS Xpad)

  • Include tests and benchmarking tools as part of the library

  • Better testing.

0.11.0: 07/2015

  • All calibrant from NIST are now available, + Nickel, Aluminum, … with bibliographic references

  • The Cell class helps defining new calibrants.

  • OpenCL Bitonic sort (to be integrated into Bragg/Amorphous separation)

  • Calib is available from the Python interface (procedural API), not only from the shell script.

  • Many new options in calib for reset/assign/delete/validate/validate2/chiplot.
    • reset: set the detector, orthogonal, centered and at 10cm

    • assign: checks the assignment of groups of points to rings

    • delete: remove a group of peaks

    • validate: autocorrelation of images: error on the center

    • validate2: autocorrelation of patterns at 180° apart: error on the center function of chi

    • chiplot: assesses the quality of control points of one/multiple rings.

  • Fix the regression of the initial guess in calib (Thanks Jon Wright)

  • New peak picking algorithm named “watershed” and based on inverse watershed for ridge recognition

  • start factorizing cython regridding engines (work ongoing)

  • Add “–poni” option for pyFAI-calib (Thanks Vadim Dyakin)

  • Improved “guess_binning”, especially for Perkin Elmer flat panel detectors.

  • Support for non planar detectors like Curved Imaging plate developped at Aarhus

  • Support for Multi-geometry experiments (tested)

  • Speed improvement for detector initialization

  • better isotropy in peak picking (add penalization term)

  • enhanced documentation on http://pyfai.readthedocs.org

0.10.3: 03/2015

  • Image segmentation based on inverse watershed (only for recalib, not for calib)

  • Python3 compatibility

  • include testimages into distribution

0.10.2: 11/2014

  • Update documentation

  • Packaging for debian 8

0.10.1: 10/2014

  • Fix issue in peak-picking

  • Improve doc & manpages

  • Compatibility with PyMca5

0.10.0: 10/2014

  • Correct Caglioti’s formula

  • Update tests and OpenCL -> works with Beignet and pocl open source drivers

  • Compatibility with MacOSX and windows

0.9.4: 06/2014

  • include spec of Maxwell GPU

  • fix issues with intel OpenCL icd v4.4

  • introduce shape & max_shape in detectors

  • work on marchingsquares/sorted controurplot for calibration

  • Enforce the use the Qt4Agg for Matplotlib and other GUI stuff.

  • Update shape of detector in case of binning

  • unified distortion class: merge OpenCL & OpenMP implementation #108

  • Benchmarks for distortion

  • Raise the level to warning when inverting the mask

  • set of new ImXpad detectors Related issue #111

  • Fix issue with recalib within MX-calibrate

  • saving detector description in Nexus files issue #110

  • Update some calibrants: gold

  • about to make peak-picking more user-friendly

  • test for bragg separation

  • work on PEP8 compliance

  • Do not re-cythonize: makes debian package generation able to benefit from ccache

  • conversion to SPD (rotation is missing)

  • pixelwise worker

  • correct both LUT & OCL for memory error

  • replace os.linsep with “n” when file file opened in text mode (not binary)

  • rework the Extension part to be explicit instead of “black magic” :)

  • implement Kahan summation in Cython (default still use Doubles: faster)

  • Preprocessing kernel containing all cast to float kernels #120

  • update setup for no-openmp option related to issue #127

  • Add read-out mode for mar345 as “guess_binning” method for detector. Also for MAR and Rayonix #125

  • tool to benchmark HDF5 writing

  • try to be compatible with both PySide and PyQt4 … the uic stuff is untested and probably buggy #130

  • Deactivate the automatic saturation correction by default. now it is opt-in #131

0.9.3: 02/2014

  • Better control for peak-picking (Contribution from Gero Flucke, Desy)

  • Precise Rayonix detectors description thanks to Michael Blum

  • Start integrating blob-detection algorithm for peak-picking: #70

  • Switch fron OptParse to ArgPrse: #83

  • Provide some calibrant by default: #91

  • Description of Mar345 detector + mask#92

  • Auto-registration of detectors: #97

  • Recalib and check-calib can be called from calib: #99

  • Fake diffraction image from calibrant: #101

  • Implementation of the CSR matrix representation to replace LUT

  • Tight pixel splitting: #43

  • Update documentation

0.9.2: (01/2014)

  • Fix memory leak in Cython part of the look-up table generation

  • Benchmarks with memory profiling

0.9: 10/2013

  • Add detector S140 from ImXpad, Titan from Agilent, Rayonix

  • Fix issues: 61, 62, 68, 76, 81, 82, 85, 86, 87

  • Enhancement in LImA plugins (better structure)

  • IO module with Ascii/EDF/HDF5 writers

  • Switch some GUI to pyQtGraph in addition to Qt

  • Correction for solid-angle formula

0.8: 10/2012

  • Detector object is member of the geometry

  • Binning of the detector, propagation to the spline if needed

  • Detector object know about their masks.

  • Automatic mask for some detectors like Pilatus or XPad

  • Implementation of sub-pixel position correction for Pilatus detectors

  • LUT implementation in 1D & 2D (fully tested) both with OpenMP and with OpenCL

  • Switch from C++/Cython OpenCL framework to PyOpenCL

  • Port opencl code to both Windows 32/64 bits and MacOSX

  • Add polarization corrections

  • Use fast-CRC checksum on x86 using SSE4 (when available) to track array change on GPU buffers

  • Support for flat 7*8 modules Xpad detectors.

  • Benchmark with live graphics (still a memory issue with python2.6)

  • Fat source distribution (python setup.py sdist –with-test-images) for debian

  • Enhanced tests, especially for Saxs and OpenCL

  • Recalibration tool for refining automatically parameters

  • Enhancement of peak picking (much faster, recoded in pure Cython)

  • Easy calibration for pixel detector (reconstruction of inter-module space)

  • Error-bar generation using Poisson law

  • Unified programming interface for all integration methods in 2theta, q or radius unit

  • Graphical interface for azimuthal integration (pyFAI-integrate)

  • Lots of test to prevent non regression

  • Tool for merging images using various method (mean, median) and with outlayer rejection

  • LImA plugin which can perform azimuthal integration live during the acquisition

  • Distortion correction is available alone and as LImA plugin

  • Recalibration can refine the wavelength in addition to 6 other parameters

  • Calibration always done vs calibrant’s ring number, lots of new calibrant are available

  • Selection by hand of single peaks for calibration

  • New detectors: Dexela and Perkin-Elmer flat panel

  • Automatic refinement of multiple images at various geometries (for MX)

  • Many improvements requested by ID11 and ID13

0.7.2: 08/2012

  • Add diff_tomo script

  • Geometry calculation optimized in (parallel) cython

0.7: 07/2012

Implementation of look-up table based integration and OpenCL version of it

0.6: 07/2012

  • OpenCL flavor works well on GPU in double precision with device selection

0.5: 06/2012

  • Include OpenCL version of azimuthal integration (based on histograms)

0.4: 06/2012

  • Global clean up of the code regarding options from command line and better design

  • Correct the orientation of the azimuthal angle chi

  • Rename scripts in pyFAI-calib, pyFAI-saxs and pyFAI-waxs

0.3: 11/2011

  • Azimuthal integration splits pixels like fit2d

0.2: 07/2011

  • Azimuthal integration using cython histogramming is working

0.1: 05/2011

  • Geometry is OK