Change-log of versions
2024.2 01/02/2024
Include grazing-incidence capabilities + tutorial (thanks Edgar)
Fix segmentation-fault in calib2 application #2047 (thanks Thomas & Edgar)
Use the dynamic mask to mask-out dead pixels in Eiger images in calib2
Extend Poisson+azimuthal uncertainties to all Cython integrators (+ non regression tests)
Support for Python 3.7-3.12 (requires silx v2 for 3.12)
2024.1 18/01/2024
Possibility to define the detector orientation: + It is the position of the origin of the detector at any of the 4 corners of the image + Uses the EXIF nomenclature where pyFAI’s (default) orientation is tagged 3 + Offers compatibility with calibration made by Dioptas (where orientation=2 since images are flipped) + Expose the feature in the calib2 GUI for custom detectors. + Tutorial on the usage + New sub-version of the PoniFile API (2.1) for this feature
Expose the number of corners of a detector pixel (feature unused for now)
Refactor pyFAI-benchmark tool with better looking results (Thanks Edgar)
Possibility to integrate in 2D with any second dimension unit: + No more limited to the azimuthal angle chi + Several new units have been added + Offers the qx/qy integration as example
Support for Dectris Pilatus4 detector both with Si and CdTe sensors (thanks to Max Burian)
Support XRDML format (compatibility with MAUD software)
Multigeometry gains a reset() method to free some memory with optimized garbage collection
Support pathlib when reading-PONI files
Change in the build system: + Drop of setup.py the build system based on distutils/numpy.distutils/setuptools + Enforce the use of meson-python + Move the sources of the code into src directory + Support for Python 3.7-3.12 (requires silx v2 for 3.12) + Provide debian packages for debian12 but ubuntu 20.04 is too old and lack meson-python
2023.09 08/09/2023
Bug fix release over 2023.08
Fix calibrant selection in calib2 (Thanks Valentin)
Centralized control of Random Number Generator for tests, to make non-regression test more consistent for CI.
Correct behaviour of dynamic mask (was causing systematic crashes)
Correct spelling of several messages.
Compatibility with matplotlib 3.7
New application icon and launcher under linux
Updated notebook (addresses API changes)
2023.08 30/08/2023 BUGGY, DO NOT USE !!!
Minor release, mostly bug-fixes. 87 commits.
fix mal-formed headers in 1D ASCII export
Debian packaging without setup.py (at all)
Fix cache invalidation issue for polarization in geometry
More compact (+scrollable) calibrant widget in Calib2 (Contribution from Valentin Valls)
Compatibility with PyQt6 and PySide6 bacckends (Contribution from malte-storm)
Comatibility with elder Matplotlib (<3.2)
Fix reproducibility issue with no-split sparse matrix multiplication (all 3)
Lower memory consumption of hexagonal detector instanciation.
Enforce the usage of the detector_factory. Detector(**config) stops working.
2023.05 16/05/2023
New detector: Jungfrau 8M used at SwissFEL
new method in Geometry to guess the number of bin (and avoid oversampling)
new method in AzimuthalIntegrator to guess the polarization factor
CSC-integrators (1D + 2D) implemented in pure python using scipy.sparse
Update documentation
Minor correction (remove debugging)
Packaging for debian12 using meson-python
2023.03 22/03/2023
Rework all 2D integrator to propagate variance, expose standard deviation and standard error of the mean.
Support all version of python between 3.7 and 3.11
Support Cython 3.0b1 (with performance degradation) in addition of cython 0.29.31+
2023.02 27/02/2023
- Core:
sigma_clip_ng is not the default sigma_clip
expose documentation for sigma_clip_legacy in addition to sigma_clip_ng
Worker can now use sigma_clip_legacy in addition to sigma_clip_ng, integrate1d or integrate2d
New calibrant: Vanadinite (used in high-presure, often as single crystal)
- GUI:
Fix labels on axes for some rarely used units
refactor IntegrationMethodDialog and share the IntegrationMethodTabs with pyFAI-calib2
- Test:
silent some noisy test (OpenCL on windows)
2023.01 16/01/2023
- Developer and packager tools:
Switch build system from
numpy.distutils
tomeson-python
Keep the former
setup.py
for compatibility reasons: it will be removed in a future releaseDrop Python 3.6 (default parameters in namedtuple feature used)
Require silx 1.1 (for OpenCL), scipy and matplotlib
- GUI side:
several minor improvements in pyFAI-calib2
Fixed calibration in jupyter-lab
- Core improvements:
Refactoring of the Geometry class
Geometry pseudo-inversion optimization
Improved support from Medipix-based Lambda-detectors
New detectors from Dectris (Pilatus 900k and Eiger 250k)
Support Nexus format in output: NXmonpd and NXcansas
Single-threaded CSC sparse matrix multiplication engine
- Improved uncertainty propagation:
Refactor error model management (uses enum)
Hybrid error model (azimuthal for sigma-clipping but reports Poissonian noise)
Export peakfinder data to the CXI format (used by CrystFEL)
- Improvement in the doc:
Update installation instructions
Multi-threaded integration tutorial
GPU implementation tutorial
- Facts and figures:
400+ commits, 100 PR
- +with the contribution of:
Clemens Prescher, Elena Pascal, Jérôme Kieffer, Malte Storm, Marco Cammarata, Michael Hudson-Doyle, Picca Frédéric-Emmanuel, Rodrigo Telles, Thomas A Caswell, Tommaso Vinci, Valentin Valls, Wout de Nolf.
0.21.3 24/03/2022
Support hexagonal pixels in splitbbox and nosplit mode (collaboration with EuXFEL)
Support extra dtype in OpenCL (contribution from Desy)
Fix version of setuptools (enables to build pyFAI -> python 3.12)
Support some of the Lambda detectors from XSpectrum (collaboration with Xmas)
Restore back compatibility in histoBBox1d and histoBBox2d (contribution from NSLS-II)
Fix bug in OpenCL distortion correction (collaboration with Soleil)
Several other bugs fixed
0.21.2 03/02/2022
Fix documentation of sparsify-Bragg and peakfinder CLI tools
0.21.1 02/02/2022
Bug-fix release with: - issue in propagated error not reported with certain engines - diff_map tool can now perform 2D integration - issue with dummy/dela_dummy in Worker - fix improper reset of the integrator due to cabling issue inside AzimuthalIntegrator - Remove deprecation warning about scipy - Use silx for median-filter which is faster than scipy - skip 3 tests which are known to be fragile on 32-bit platforms (spotted by debian on i386 and armel)
0.21.0 20/01/2022
One year of development: 523 commits, close to 100 pull-requests. +53000 lines of code and 20000 suppressed.
- Sigma-clipping allows separation of Bragg/amorphous signal:
Implementation in Python, Cython and OpenCL with poissonian and azimuthal error-model
- Sparsification, compression of single crystal data
Application to peak-picking and quality assessement of SSX data
Analysis of grid-scan to find single crystal
Single pass variance propagation in azimuthal bin
Integration of the Jungfrau detector (ID29)
- 2D integration:
New integrators with error propagation by default
Full pixel splitting in addition to BBox and no splitting
Refactor of all LUT and CSR to share the same code base, makes tests more robust.
- Calibration of experimental setup using Jupyter notebooks
User interaction with plots in matplotlib (thanks Phil Hans)
Factorize code between pyFAI-calib and jupyter calibration
Tutorial as notebook and video recording
Parallax correction for thick detector (still experimental, thanks to Vadim)
Improved tutorial on detector geometry calibration (based on Kabsch alignment)
Better performances on HPC nodes by limiting simple OpenMP to fewer cores
Many improvement in test, typos fixed, doc …
Deprecate all legacy integrators since the new version sees its matrix complete.
Supports Python 3.6 … 3.10 under Windows, MacOS and Linux.
0.20.0 22/01/2021
One year of development, about 500 commits & 370 files modified.
Generalization of the new generation 1D integrators (better error propagation)
Sigma clipping and sparsification of single crystal data (OpenCL only)
Fix issue introduced with the scipy 1.15 (constrained calibration broken)
Improved distortion correction (also on GPU, …)
Major re-work of the documentation (thanks Thomas Kluyver and Loic Huder)
Improve the calibration of Pilatus and Eiger detectors based on a grid of holes.
New cylindrical detector from Rigaku
Drop deprecated OpenCL integrator
Support all Eiger2 detectors (thanks Clemens Weninger and Marie Ruat)
CI: move to Gihub workflow and gitlab-runners (bob) for building (thanks Thomas Vincent).
Build for debian 10 and 11 (also ubuntu 20.04), drop debian 9
Remove Python2 related code
Lower Numpy ABI dependency as much as possible (remains _distortion’s C++ code)
Drop tests for Python 3.5, checked on 3.9 as well.
0.19.0 31/01/2020
Minor revison with only 150 commits, mainly bug-fixes
Improvement on the GUI with many small bug-fixes
Support newer h5py (mode mandatory, [()], …)
Build for debian 10 and 11 (also ubuntu 20.04)
Drop tests for Python 2 and 3.4, checked on 3.8
Improved compatibility with ImageD11
Use hdf5plugin to provide hdf5 io-filters in apps
Rework diffraction mapping tools to use a worker
New generation azimuthal integrator using CSR algorithm implemented in Python, Cython and OpenCL.
Sigma-clipping implemented in OpenCL
Publication on new generation integration, the GUI for calibration and the goniometers accepted in J. Synch. Rad. DOI: 10.1107/S1600577520000776
A big thank to Florian from Germany, Bertrand from Xenocs, Alex from Soleil and Jon from ESRF for their contributions.
0.18.0 15/05/2019
Last release with Valentin as he finishes his contract soon
almost 800 commits, 60 PR since the last release: this is a huge release !
Major rework on all GUIs, mainly pyFAI-calib2 and pyFAI-integrate.
Possibility to integrate image stacks (i.e. from HDF5), …
Rework the method to specify the algorithm, pixel splitting and implementation with sensible fall-backs. Also available via the different GUIs
3D visualization of detectors and experimental setup, useful for non flat detectors.
integrate1d_ng is available with histogramming without pixel splitting in Python, Cython and OpenCL. Now, propagates the variance properly !
IO sub-packages with associated refactoring for ponifile, json, …
Improved management of OpenMP: simplify the code for histograms.
Improved geometry description and tutorial for writing exchange with other software (ImageD11, thanks to Carsten Detlefs).
More reliable simple ellipse fitting with tests and doc.
Better POCL integration (debugged on cuda, x87, Power9, …)
Rely on silx mechanics for the build, test, download, GUI, opencl …
Many new tutorials, now available on binder-hub: new calibrants, Pilatus calibration, …
Fix many issues reported in third-party software (Dioptas, …)
Drop support of debian8, Python 2.7 and Python 3.4 on all platforms. It is likely most functionalities still work but without guaranty.
0.17.0 19/12/2018
Only 200 commits in a couple of month, this is a small release
Fix major bugs in pyFAI-calib2 (double validator, initial guess, ring position)
Constrains have been added to the geometry fitting of pyFAI-calib2
New pyFAI-integrate graphical application
Much better support for user defined detector (HDF5)
Start the rewrite of all integrators to allow proper error propagation (2D done)
Factorize the preprocessing steps for many integrators
Remove tons of code which has been deprecated for years in AzimuthalIntegrator
Featuring contribution from Soleil and Berkeley
Stop supporting Python2.7 on Windows (there won’t be wheels!)
All scripts are now using Python 3.x (x>=4)
This is the last release supporting Python 2.7, 3.4 hence debian 8
0.16.0: 26/10/2018
Almost 800 commits since 0.15 !
Huge improvements on the graphical application for calibration
New detector definition (with manufacturers)
Improved tests: the GUI is now tested
Preparation for changing all rebinning engines (see variance tutorials)
Azimuthal integrators (and most other objects) are now serializable with Pickle
New distortion correction using the SparseBuilder C++ code
New PONI-file format (detector definition changed)
Isocontour is now provided by silx
Peak-picker clean up (better peak selection near gaps)
new Goniometer refinement with enhanced rotation using Euler angles
Updated documentation: new cookbooks and tutorials about: - The use of the calibration application (cookbook) - log-scale integration of SAXS data (notebook) - Variance propagation (notebook)
0.15.0: 01/02/2018
150 commits since last revision
New tutorials on image inpainting, sensor thickness correction, …
Improve scripts
Improve the new calibration GUI (pyFAI-calib2)
Use scipy physical constants instead of hard-coded values
Improved detector serialization and binning assessement
Update documentation (images, better rendering of notebook & tutorials)
Converge project with silx and fabio
Remove generated rst- and C-files from repository
This is the last version supporting python2.7
0.14.2: 14/09/2017
Fix seg-fault with manylinux1 wheels, in fastcrc module (thanks Thomas)
Fix Qt4-Qt5 compatibility (thanks Vadim)
Easier to understand geometry transformation (thanks Jon)
Lower memory consumption, better cache management
Unified debian packaging working for 6->9
New detector: Mythen & CirPad (thanks Fred)
Clean up debug code which avoid to use pyFAI-calib2
pyFAI-calib2 now expect fabio >= 0.5
Fix issue with metadata saving in 1d
Fix performance regression with pyopencl >2015.2 (Thanks Andreas)
pyFAI saxs and waxs scripts guess now the binning of the detector (thanks Fred).
0.14.1: 25/07/2017
Fixes Debian 7 and 8 packages
0.14.0: 20/07/2017
Graphical user interface for calibration (pyFAI-calib2)
Goniometer calibration tools and multi-geometry enhancements
Integration scripts and averaging scripts are now able to normalize the data from monitors found in the header.
Propagate metadata information as part of the integrated data
Common pre-processing factorization on Python, Cython and OpenCL
Test clean up and acceleration (avoid tests on too large images)
Many new tutorials http://pyfai.readthedocs.io/en/latest/usage/tutorial/index.html
New averaging / integration methods: - Azimuthal median filtering - Azimuthal trimmed mean - sigma-clipping on azimuthal angle - Radial averaging
Diffraction image inpainting to fill-up the gaps with plausible values.
This release correspond to 572 commits
Change of license: now all pyFAI is MIT license.
0.13.0: 01/12/2016
Global improvement of tests, packaging, code quality, documentation and project tools
- Scripts
Add support for multiframe formats on pyFAI-average
Add support for monitoring correction from header file (on pyFAI-average)
Add progressbar in the shell (on pyFAI-average and pyFAI-integrate)
Script drawMask_pymca is renamed into pyFAI-drawmask
Rework of the drawmask GUI using silx
pyFAI-drawmask do not have anymore hard dependency on PyMCA
pyFAI-integrate can now be used without qt dependency (–no-gui)
Fix the script to support both Python 2 and 3 (pyFAI-calib, pyFAI-benchmark)
Fix selection of units on diff-map (the user selection was not propagated)
- For users
More source code in MIT license
Update name and specification for cameras
Add cameras: Eiger500k, RaspberryPi5M, RaspberryPi8M
Fix Xpad S540 flat detector geometry
Fix definition of CeO2 calibrant
Add mask and flat on multi-geometry
Fix solid angle of the multi-geometry
Fix geometry processing for custom output space
Fix normalization factor and variance
Add support for Qt5
Add support for Debian 9 packaging
- For developers
Create common preprocessing for distortion correction
Create common image preprocessing using Cython (NaN filter, flatfield, dark, polarisation)
Refactoring of units module. It allows to register custom units.
Worker can now use Writer
Worker polarization argument is renamed into polarization_factor
Remove the dependency from python-fftw3, use numpy instead
Remove QtWebKit dependency
Fix un-correction of images using sparse matrix from scipy
0.12.0: 06/06/2016
Continuous integration on linux, windows using Python 2.7 and 3.4+
Drop support of Python 2.6, 3.2, 3.3 and debian6 packaging
New radial output units: Reciprocal spacing squared and log(q) ID02
GPU accelerate version of ai.separate (Bragg & amorphous) ID13
Quantile filtering in pyFAI-average ID02
New graphical application for diffraction imaging ID21
Migrate to a common structure with silx (reorganize tests, benchmarks, …)
Extensions (binary sub-modules) have all been moved to ext directory
Many improvements multigeometry integrators
Compatibility with the copy module (copy.deepcopy) for azimuthal integrator ID02
Distortion correction works also for non-contiguous detectors
- Update documentation and provide advanced tutorials:
Introduction to pyFAI using the jupyter notebook
detector calibration ID15, BM02
Correction of detector distortion, examples of pixel detectors.
calibrant calculation ID30
error handling ID02, BM29
pyFAI-integrate can now be used with or without GUI
- Many new detectors (ADSC, Pilatus CdTe, Apex II, Pixium):
support for non-flat/curved detectors (Aarhus)
non-contiguous detectors (WOS Xpad)
Include tests and benchmarking tools as part of the library
Better testing.
0.11.0: 07/2015
All calibrant from NIST are now available, + Nickel, Aluminum, … with bibliographic references
The Cell class helps defining new calibrants.
OpenCL Bitonic sort (to be integrated into Bragg/Amorphous separation)
Calib is available from the Python interface (procedural API), not only from the shell script.
- Many new options in calib for reset/assign/delete/validate/validate2/chiplot.
reset: set the detector, orthogonal, centered and at 10cm
assign: checks the assignment of groups of points to rings
delete: remove a group of peaks
validate: autocorrelation of images: error on the center
validate2: autocorrelation of patterns at 180° apart: error on the center function of chi
chiplot: assesses the quality of control points of one/multiple rings.
Fix the regression of the initial guess in calib (Thanks Jon Wright)
New peak picking algorithm named “watershed” and based on inverse watershed for ridge recognition
start factorizing cython regridding engines (work ongoing)
Add “–poni” option for pyFAI-calib (Thanks Vadim Dyakin)
Improved “guess_binning”, especially for Perkin Elmer flat panel detectors.
Support for non planar detectors like Curved Imaging plate developped at Aarhus
Support for Multi-geometry experiments (tested)
Speed improvement for detector initialization
better isotropy in peak picking (add penalization term)
enhanced documentation on http://pyfai.readthedocs.org
0.10.3: 03/2015
Image segmentation based on inverse watershed (only for recalib, not for calib)
Python3 compatibility
include testimages into distribution
0.10.2: 11/2014
Update documentation
Packaging for debian 8
0.10.1: 10/2014
Fix issue in peak-picking
Improve doc & manpages
Compatibility with PyMca5
0.10.0: 10/2014
Correct Caglioti’s formula
Update tests and OpenCL -> works with Beignet and pocl open source drivers
Compatibility with MacOSX and windows
0.9.4: 06/2014
include spec of Maxwell GPU
fix issues with intel OpenCL icd v4.4
introduce shape & max_shape in detectors
work on marchingsquares/sorted controurplot for calibration
Enforce the use the Qt4Agg for Matplotlib and other GUI stuff.
Update shape of detector in case of binning
unified distortion class: merge OpenCL & OpenMP implementation #108
Benchmarks for distortion
Raise the level to warning when inverting the mask
set of new ImXpad detectors Related issue #111
Fix issue with recalib within MX-calibrate
saving detector description in Nexus files issue #110
Update some calibrants: gold
about to make peak-picking more user-friendly
test for bragg separation
work on PEP8 compliance
Do not re-cythonize: makes debian package generation able to benefit from ccache
conversion to SPD (rotation is missing)
pixelwise worker
correct both LUT & OCL for memory error
replace os.linsep with “n” when file file opened in text mode (not binary)
rework the Extension part to be explicit instead of “black magic” :)
implement Kahan summation in Cython (default still use Doubles: faster)
Preprocessing kernel containing all cast to float kernels #120
update setup for no-openmp option related to issue #127
Add read-out mode for mar345 as “guess_binning” method for detector. Also for MAR and Rayonix #125
tool to benchmark HDF5 writing
try to be compatible with both PySide and PyQt4 … the uic stuff is untested and probably buggy #130
Deactivate the automatic saturation correction by default. now it is opt-in #131
0.9.3: 02/2014
Better control for peak-picking (Contribution from Gero Flucke, Desy)
Precise Rayonix detectors description thanks to Michael Blum
Start integrating blob-detection algorithm for peak-picking: #70
Switch fron OptParse to ArgPrse: #83
Provide some calibrant by default: #91
Description of Mar345 detector + mask#92
Auto-registration of detectors: #97
Recalib and check-calib can be called from calib: #99
Fake diffraction image from calibrant: #101
Implementation of the CSR matrix representation to replace LUT
Tight pixel splitting: #43
Update documentation
0.9.2: (01/2014)
Fix memory leak in Cython part of the look-up table generation
Benchmarks with memory profiling
0.9: 10/2013
Add detector S140 from ImXpad, Titan from Agilent, Rayonix
Fix issues: 61, 62, 68, 76, 81, 82, 85, 86, 87
Enhancement in LImA plugins (better structure)
IO module with Ascii/EDF/HDF5 writers
Switch some GUI to pyQtGraph in addition to Qt
Correction for solid-angle formula
0.8: 10/2012
Detector object is member of the geometry
Binning of the detector, propagation to the spline if needed
Detector object know about their masks.
Automatic mask for some detectors like Pilatus or XPad
Implementation of sub-pixel position correction for Pilatus detectors
LUT implementation in 1D & 2D (fully tested) both with OpenMP and with OpenCL
Switch from C++/Cython OpenCL framework to PyOpenCL
Port opencl code to both Windows 32/64 bits and MacOSX
Add polarization corrections
Use fast-CRC checksum on x86 using SSE4 (when available) to track array change on GPU buffers
Support for flat 7*8 modules Xpad detectors.
Benchmark with live graphics (still a memory issue with python2.6)
Fat source distribution (python setup.py sdist –with-test-images) for debian
Enhanced tests, especially for Saxs and OpenCL
Recalibration tool for refining automatically parameters
Enhancement of peak picking (much faster, recoded in pure Cython)
Easy calibration for pixel detector (reconstruction of inter-module space)
Error-bar generation using Poisson law
Unified programming interface for all integration methods in 2theta, q or radius unit
Graphical interface for azimuthal integration (pyFAI-integrate)
Lots of test to prevent non regression
Tool for merging images using various method (mean, median) and with outlayer rejection
LImA plugin which can perform azimuthal integration live during the acquisition
Distortion correction is available alone and as LImA plugin
Recalibration can refine the wavelength in addition to 6 other parameters
Calibration always done vs calibrant’s ring number, lots of new calibrant are available
Selection by hand of single peaks for calibration
New detectors: Dexela and Perkin-Elmer flat panel
Automatic refinement of multiple images at various geometries (for MX)
Many improvements requested by ID11 and ID13
0.7.2: 08/2012
Add diff_tomo script
Geometry calculation optimized in (parallel) cython
0.7: 07/2012
Implementation of look-up table based integration and OpenCL version of it
0.6: 07/2012
OpenCL flavor works well on GPU in double precision with device selection
0.5: 06/2012
Include OpenCL version of azimuthal integration (based on histograms)
0.4: 06/2012
Global clean up of the code regarding options from command line and better design
Correct the orientation of the azimuthal angle chi
Rename scripts in pyFAI-calib, pyFAI-saxs and pyFAI-waxs
0.3: 11/2011
Azimuthal integration splits pixels like fit2d
0.2: 07/2011
Azimuthal integration using cython histogramming is working
0.1: 05/2011
Geometry is OK