pyFAI.utils package
pyFAI.utils.bayes module
Bayesian evaluation of background for 1D powder diffraction pattern.
Code according to Sivia and David, J. Appl. Cryst. (2001). 34, 318-324
Version: 0.1 2012/03/28
Version: 0.2 2016/10/07: OOP implementation
- class pyFAI.utils.bayes.BayesianBackground
Bases:
object
This class estimates the background of a powder diffraction pattern
http://journals.iucr.org/j/issues/2001/03/00/he0278/he0278.pdf
The log likelihood is described in correspond to eq7 of the paper:
\[z = y / sigma^2\]if z<0 a quadratic behaviour is expected
if z>>1 it is likely a bragg peak so the penalty should be small: log(z).
The spline is used to have a quadratic behaviour near 0 and the log one near the junction
The threshold is taken at 8 as erf is 1 above 6: The points 6, 7 and 8 are used in the spline to ensure a continuous junction with the logarithmic continuation.
- PREFACTOR = 1
- __init__()
- background_image(img, sigma=None, mask=None, npt=10, k=3)
- classmethod bayes_llk(z)
Calculate actually the log-likelihood from a set of weighted error
Re implementation of the following code even slightly faster:
(y<=0)*5*y**2 + (y>0)*(y<8)*pyFAI.utils.bayes.background.spline(y) + (y>=8)*(s1+log(abs(y)+1*(y<8)))
- Parameters:
z (float[:]) – weighted error
- Returns:
log likelihood
- Return type:
float[:]
- classmethod bayes_llk_large(z)
used to calculate the log-likelihood of large positive values: logarithmic
- classmethod bayes_llk_negative(z)
used to calculate the log-likelihood of negative values: quadratic
- classmethod bayes_llk_small(z)
used to calculate the log-likelihood of small positive values: fitted with spline
- classmethod classinit()
- classmethod func2d_min(values, d0_sparse, d1_sparse, d0_pos, d1_pos, y_obs, w_obs, valid, k)
Function to optimize
- Parameters:
values – values of the background on spline knots
d0_sparse – positions along slowest axis of the spline knots
d1_pos – positions along fastest axis of the spline knots
d0_pos – positions along slowest axis (all coordinates)
d1_pos – positions along fastest axis (all coordinates)
y_obs – intensities actually measured
w_obs – weights of the experimental points
valid – coordinated of valid pixels
k – order of the spline, usually 3
- Returns:
sum of the log-likelihood to be minimized
- classmethod func_min(y0, x_obs, y_obs, w_obs, x0, k)
Function to optimize
- Parameters:
y0 – values of the background
x_obs – experimental values
y_obs – experimental values
w_obs – weights of the experimental points
x0 – position of evaluation of the spline
k – order of the spline, usually 3
- Returns:
sum of the log-likelihood to be minimized
- s1 = 2.5435828321944816
- spline = <scipy.interpolate._fitpack2.InterpolatedUnivariateSpline object>
- classmethod test_bayes_llk()
Test plot of log(likelihood) Similar to as figure 3 of Sivia and David, J. Appl. Cryst. (2001). 34, 318-324
pyFAI.utils.decorators module
Bunch of useful decorators
- pyFAI.utils.decorators.deprecated(func=None, reason=None, replacement=None, since_version=None, only_once=False, skip_backtrace_count=1, deprecated_since=None)
Decorator that deprecates the use of a function
- Parameters:
reason (str) – Reason for deprecating this function (e.g. “feature no longer provided”,
replacement (str) – Name of replacement function (if the reason for deprecating was to rename the function)
since_version (str) – First pyFAI version for which the function was deprecated (e.g. “0.5.0”).
only_once (bool) – If true, the deprecation warning will only be generated one time. Default is true.
skip_backtrace_count (int) – Amount of last backtrace to ignore when logging the backtrace
deprecated_since (Union[int,str]) – If provided, log it as warning since a version of the library, else log it as debug
- pyFAI.utils.decorators.deprecated_warning(type_, name, reason=None, replacement=None, since_version=None, only_once=True, skip_backtrace_count=0, deprecated_since=None)
Function to log a deprecation warning
- Parameters:
type (str) – Nature of the object to be deprecated: “Module”, “Function”, “Class” …
name – Object name.
reason (str) – Reason for deprecating this function (e.g. “feature no longer provided”,
replacement (str) – Name of replacement function (if the reason for deprecating was to rename the function)
since_version (str) – First pyFAI version for which the function was deprecated (e.g. “0.5.0”).
only_once (bool) – If true, the deprecation warning will only be generated one time for each different call locations. Default is true.
skip_backtrace_count (int) – Amount of last backtrace to ignore when logging the backtrace
deprecated_since (Union[int,str]) – If provided, log the deprecation as warning since a version of the library, else log it as debug.
- pyFAI.utils.decorators.timeit(func)
pyFAI.utils.ellipse module
This modules contains a function to fit without refinement an ellipse on a set of points ….
- class pyFAI.utils.ellipse.Ellipse(center_1, center_2, angle, half_long_axis, half_short_axis)
Bases:
NamedTuple
- angle: float
Angle in radian
- center_1: float
Center position in the slow axis (pty)
- center_2: float
Center position in the fast axis (ptx)
- half_long_axis: float
Alias for field number 3
- half_short_axis: float
Alias for field number 4
- pyFAI.utils.ellipse.fit_ellipse(pty, ptx, _allow_delta=True)
Fit an ellipse
Math from https://mathworld.wolfram.com/Ellipse.html #15
inspired from http://nicky.vanforeest.com/misc/fitEllipse/fitEllipse.html
- Parameters:
pty – point coordinates in the slow dimension (y)
ptx – point coordinates in the fast dimension (x)
- Raises:
ValueError – If the ellipse can’t be fitted
pyFAI.utils.header_utils module
This modules contains helper function relative to image header.
- exception pyFAI.utils.header_utils.MonitorNotFound
Bases:
Exception
Raised when monitor information in not found or is not valid.
- pyFAI.utils.header_utils.get_monitor_value(image, monitor_key)
Return the monitor value from an image using an header key.
- Parameters:
image (fabio.fabioimage.FabioImage) – Image containing the header
monitor_key (str) – Key containing the monitor
- Returns:
returns the monitor else raise an exception
- Return type:
float
- Raises:
MonitorNotFound – when the expected monitor is not found on the header
pyFAI.utils.logging_utils module
This modules contains helper function relative to logging system.
- class pyFAI.utils.logging_utils.PrePostEmitStreamHandler(handler)
Bases:
Handler
Handler to allow to hook a function before and after the emit function.
The main logging feature is delegated to a sub handler.
- __init__(handler)
Initializes the instance - basically setting the formatter to None and the filter list to empty.
- emit(record)
Call pre_emit function then delegate the emit to the sub handler.
- post_emit()
- pre_emit()
- pyFAI.utils.logging_utils.prepost_emit_callback(logger, pre_callback, post_callback)
Context manager to add pre/post emit callback to a logger
- pyFAI.utils.logging_utils.set_prepost_emit_callback(logger, pre_callback, post_callback)
Patch the logging system to have a working progress bar without glitch. pyFAI define a default handler then we have to rework it
- Returns:
The new handler
pyFAI.utils.mathutil module
Utilities, mainly for image treatment
- class pyFAI.utils.mathutil.LongestRunOfHeads
Bases:
object
Implements the “longest run of heads” by Mark F. Schilling The College Mathematics Journal, Vol. 21, No. 3, (1990), pp. 196-207
See: http://www.maa.org/sites/default/files/pdf/upload_library/22/Polya/07468342.di020742.02p0021g.pdf
- A(n, c)
Calculate A(number_of_toss, length_of_longest_run)
- Parameters:
n – number of coin toss in the experiment, an integer
c – length of the longest run of
- Returns:
The A parameter used in the formula
- B(n, c)
Calculate B(number_of_toss, length_of_longest_run) to have either a run of Heads either a run of Tails
- Parameters:
n – number of coin toss in the experiment, an integer
c – length of the longest run of
- Returns:
The B parameter used in the formula
- __init__()
We store already calculated values for (n,c)
- probaHeadOrTail(n, c)
Calculate the probability of a longest run of head or tails to occur
- Parameters:
n – number of coin toss in the experiment, an integer
c – length of the longest run of heads or tails, an integer
- Returns:
The probablility of having c subsequent heads or tails in a n toss of fair coin
- probaLongerRun(n, c)
Calculate the probability for the longest run of heads or tails to exceed the observed length
- Parameters:
n – number of coin toss in the experiment, an integer
c – length of thee observed run of heads or tails, an integer
- Returns:
The probablility of having more than c subsequent heads or tails in a n toss of fair coin
- pyFAI.utils.mathutil.binning(input_img, binsize, norm=True)
- Parameters:
input_img – input ndarray
binsize – int or 2-tuple representing the size of the binning
norm – if False, do average instead of sum
- Returns:
binned input ndarray
- pyFAI.utils.mathutil.center_of_mass(img)
Calculate the center of mass of of the array. Like scipy.ndimage.measurements.center_of_mass :param img: 2-D array :return: 2-tuple of float with the center of mass
- pyFAI.utils.mathutil.chi_square(obt, ref)
Compute \(\sqrt{\sum \frac{4\cdot(obt-ref)^2}{(obt + ref)^2}}\).
This is done for symmetry reason between obt and ref
- Parameters:
obt (3-tuple of array of the same size containing position, intensity, variance) – obtained data
obt – reference data
- Returns:
Chi² value, lineary interpolated
- pyFAI.utils.mathutil.cormap(ref, obt)
Calculate the probabily of two array to be the same based on the CorMap algorithm This is a simplifed implementation
- pyFAI.utils.mathutil.deg2rad(dd, disc=1)
Convert degrees to radian in the range [-π->π[ or [0->2π[
- Parameters:
dd – angle in degrees
- Returns:
angle in radians in the selected range
- pyFAI.utils.mathutil.dog(s1, s2, shape=None)
2D difference of gaussian typically 1 to 10 parameters
- pyFAI.utils.mathutil.dog_filter(input_img, sigma1, sigma2, mode='reflect', cval=0.0)
2-dimensional Difference of Gaussian filter implemented with FFT
- Parameters:
input_img (array-like) – input_img array to filter
sigma (scalar or sequence of scalars) – standard deviation for Gaussian kernel. The standard deviations of the Gaussian filter are given for each axis as a sequence, or as a single number, in which case it is equal for all axes.
mode – {‘reflect’,’constant’,’nearest’,’mirror’, ‘wrap’}, optional The
mode
parameter determines how the array borders are handled, wherecval
is the value when mode is equal to ‘constant’. Default is ‘reflect’cval – scalar, optional Value to fill past edges of input if
mode
is ‘constant’. Default is 0.0
- pyFAI.utils.mathutil.expand(input_img, sigma, mode='constant', cval=0.0)
Expand array a with its reflection on boundaries
- Parameters:
a – 2D array
sigma – float or 2-tuple of floats.
mode – “constant”, “nearest”, “reflect” or “mirror”
cval – filling value used for constant, 0.0 by default
Nota: sigma is the half-width of the kernel. For gaussian convolution it is adviced that it is 4*sigma_of_gaussian
- pyFAI.utils.mathutil.expand2d(vect, size2, vertical=True)
This expands a vector to a 2d-array.
The result is the same as:
if vertical: numpy.outer(numpy.ones(size2), vect) else: numpy.outer(vect, numpy.ones(size2))
This is a ninja optimization: replace *1 with a memcopy, saves 50% of time at the ms level.
- Parameters:
vect – 1d vector
size2 – size of the expanded dimension
vertical – if False the vector is expanded to the first dimension. If True, it is expanded to the second dimension.
- pyFAI.utils.mathutil.gaussian(M, std)
Return a Gaussian window of length M with standard-deviation std.
This differs from the scipy.signal.gaussian implementation as: - The default for sym=False (needed for gaussian filtering without shift) - This implementation is normalized
- Parameters:
M – length of the windows (int)
std – standatd deviation sigma
The FWHM is 2*numpy.sqrt(2 * numpy.pi)*std
- pyFAI.utils.mathutil.gaussian_filter(input_img, sigma, mode='reflect', cval=0.0, use_scipy=True)
2-dimensional Gaussian filter implemented with FFT
- Parameters:
input_img (array-like) – input array to filter
sigma (scalar or sequence of scalars) – standard deviation for Gaussian kernel. The standard deviations of the Gaussian filter are given for each axis as a sequence, or as a single number, in which case it is equal for all axes.
mode – {‘reflect’,’constant’,’nearest’,’mirror’, ‘wrap’}, optional The
mode
parameter determines how the array borders are handled, wherecval
is the value when mode is equal to ‘constant’. Default is ‘reflect’cval – scalar, optional Value to fill past edges of input if
mode
is ‘constant’. Default is 0.0
- pyFAI.utils.mathutil.interp_filter(ary, out=None)
Interpolate missing values (nan or infinite) in a 1D array
- Parameters:
ary – 1D array
out – destination array (use ary to avoid allocation)
- Returns:
1D array
- pyFAI.utils.mathutil.is_far_from_group(pt, lst_pts, d2)
Tells if a point is far from a group of points, distance greater than d2 (distance squared)
- Parameters:
pt – point of interest
lst_pts – list of points
d2 – minimum distance squarred
- Returns:
True If the point is far from all others.
- pyFAI.utils.mathutil.maximum_position(img)
Same as scipy.ndimage.measurements.maximum_position: Find the position of the maximum of the values of the array.
- Parameters:
img – 2-D image
- Returns:
2-tuple of int with the position of the maximum
- pyFAI.utils.mathutil.measure_offset(img1, img2, method='numpy', withLog=False, withCorr=False)
Measure the actual offset between 2 images :param img1: ndarray, first image :param img2: ndarray, second image, same shape as img1 :param withLog: shall we return logs as well ? boolean :return: tuple of floats with the offsets
- pyFAI.utils.mathutil.relabel(label, data, blured, max_size=None)
Relabel limits the number of region in the label array. They are ranked relatively to their max(I0)-max(blur(I0))
- Parameters:
label – a label array coming out of
scipy.ndimage.measurement.label
data – an array containing the raw data
blured – an array containing the blurred data
max_size – the max number of label wanted
- Returns:
array like label
- pyFAI.utils.mathutil.round_fft(N)
This function returns the integer >=N for which size the Fourier analysis is faster (fron the FFT point of view)
Credit: Alessandro Mirone, ESRF, 2012
- Parameters:
N – interger on which one would like to do a Fourier transform
- Returns:
integer with a better choice
- pyFAI.utils.mathutil.roundfft(*args, **kwargs)
- pyFAI.utils.mathutil.rwp(obt, ref, scale=1.0)
Compute \(\sqrt{\sum \frac{4\cdot(obt-ref)^2}{(obt + ref)^2}}\).
This is done for symmetry reason between obt and ref
- Parameters:
obt (2-list of array of the same size) – obtained data
obt – reference data
scale – scale obt intensity
- Returns:
Rwp value, lineary interpolated
- pyFAI.utils.mathutil.shift(input_img, shift_val)
Shift an array like scipy.ndimage.interpolation.shift(input_img, shift_val, mode=”wrap”, order=0) but faster :param input_img: 2d numpy array :param shift_val: 2-tuple of integers :return: shifted image
- pyFAI.utils.mathutil.shiftFFT(*args, **kwargs)
- pyFAI.utils.mathutil.shift_fft(input_img, shift_val, method='fft')
Do shift using FFTs
Shift an array like scipy.ndimage.interpolation.shift(input, shift, mode=”wrap”, order=”infinity”) but faster :param input_img: 2d numpy array :param shift_val: 2-tuple of float :return: shifted image
- pyFAI.utils.mathutil.unBinning(*args, **kwargs)
- pyFAI.utils.mathutil.unbinning(binnedArray, binsize, norm=True)
- Parameters:
binnedArray – input ndarray
binsize – 2-tuple representing the size of the binning
norm – if True (default) decrease the intensity by binning factor. If False, it is non-conservative
- Returns:
unBinned input ndarray
pyFAI.utils.orderedset module
pyFAI.utils.shell module
Module containing utilities around shell command line.
- class pyFAI.utils.shell.ProgressBar(title, max_value, bar_width)
Bases:
object
Progress bar in shell mode
- __init__(title, max_value, bar_width)
Create a progress bar using a title, a maximum value and a graphical size.
The display is done with stdout using carriage return to to hide the previous progress. It is not possible to use stdout for something else whill a progress bar is in use.
The result looks like:
Title [■■■■■■ ] 50% Message
- Parameters:
title (str) – Title displayed before the progress bar
max_value (float) – The maximum value of the progress bar
bar_width (int) – Size of the progressbar in the screen
- clear()
Remove the progress bar from the display and move the cursor at the beginning of the line using carriage return.
- display()
Display the progress bar to stdout
- update(value, message='', max_value=None)
Update the progrss bar with the progress bar’s current value.
Set the progress bar’s current value, compute the percentage of progress and update the screen with. Carriage return is used first and then the content of the progress bar. The cursor is at the begining of the line.
- Parameters:
value (float) – progress bar’s current value
message (str) – message displayed after the progress bar
max_value (float) – If not none, update the maximum value of the progress bar
pyFAI.utils.stringutil module
Module containing enhanced string formatters.
- class pyFAI.utils.stringutil.SafeFormatter
Bases:
Formatter
Like default formater but unmatched keys are still present into the result string
- get_field(field_name, args, kwargs)
- pyFAI.utils.stringutil.latex_to_unicode(string)
Returns a unicode representation from latex strings used by pyFAI.
Note
The latex string could be removed from the pyFAI core.
- Parameters:
string (str) – A latex string to convert
- Return type:
str
- pyFAI.utils.stringutil.safe_format(format_string, arguments)
Like default str.format but unmatching patterns will be still present into the result string.
- Parameters:
str (format_string) – Format string as defined in the default formatter.
tuple (arguments dict or) – Arbitrary set of positional and keyword arguments.
- Return type:
str
- pyFAI.utils.stringutil.to_bool(string)
Returns a safe boolean from a string.
- Raises:
ValueError – If the string do not contains a boolean information.
- pyFAI.utils.stringutil.to_ordinal(number)
Returns a string from an ordinal value with it’s suffix.
- Parameters:
number (int) – A number refering to a position
- Return type:
str
- pyFAI.utils.stringutil.to_scientific_unicode(value, digits=3)
Convert a float value into a string using scientific notation and superscript unicode character.
This avoid using HTML in some case, when Qt widget does not support it.
- Parameters:
value (float) – Value to convert to displayable string
digits (int) – Number of digits expected (3 means 1.000).
Module contents
Module with miscelaneous tools
- class pyFAI.utils.FixedParameters
Bases:
set
Like a set, made for FixedParameters in geometry refinement
- add_or_discard(key, value=True)
Add a value to a set if value, else discard it :param key: element to added or discared from set :type value: boolean. If None do nothing ! :return: None
- pyFAI.utils.calc_checksum(ary, safe=True)
Calculate the checksum by default (or returns its buffer location if unsafe)
- pyFAI.utils.convert_CamelCase(name)
convert a function name in CamelCase into camel_case
- pyFAI.utils.expand_args(args)
Takes an argv and expand it (under Windows, cmd does not convert
*.tif
into a list of files. Keeps only valid files (thanks to glob)- Parameters:
args – list of files or wilcards
- Returns:
list of actual args
- pyFAI.utils.float_(val)
Convert anything to a float … or None if not applicable
- pyFAI.utils.fully_qualified_name(obj)
Return the fully qualified name of an object
- pyFAI.utils.get_calibration_dir()
get the full path of a calibration directory
- Returns:
the full path of the calibrant file
- pyFAI.utils.get_cl_file(resource)
get the full path of a openCL resource file
The resource name can be prefixed by the name of a resource directory. For example “silx:foo.png” identify the resource “foo.png” from the resource directory “silx”. See also
silx.resources.register_resource_directory()
.- Parameters:
resource (str) – Resource name. File name contained if the opencl directory of the resources.
- Returns:
the full path of the openCL source file
- pyFAI.utils.get_ui_file(filename)
get the full path of a user-interface file
- Returns:
the full path of the ui
- pyFAI.utils.int_(val)
Convert anything to an int … or None if not applicable
- class pyFAI.utils.lazy_property(fget)
Bases:
object
meant to be used for lazy evaluation of an object attribute. property should represent non-mutable data, as it replaces itself.
- __init__(fget)
- pyFAI.utils.readFloatFromKeyboard(text, dictVar)
Read float from the keyboard ….
- Parameters:
text – string to be displayed
dictVar – dict of this type: {1: [set_dist_min],3: [set_dist_min, set_dist_guess, set_dist_max]}
- pyFAI.utils.str_(val)
Convert anything to a string … but None -> “”