petitRADTRANS.retrieval.preparing
=================================

.. py:module:: petitRADTRANS.retrieval.preparing

.. autoapi-nested-parse::

   Stores useful functions for data reduction.



Functions
---------

.. autoapisummary::

   petitRADTRANS.retrieval.preparing.__init_pipeline
   petitRADTRANS.retrieval.preparing.__init_pipeline_outputs
   petitRADTRANS.retrieval.preparing.__sysrem_iteration_orders_a
   petitRADTRANS.retrieval.preparing.__sysrem_iteration_orders_c
   petitRADTRANS.retrieval.preparing.bias_pipeline_metric
   petitRADTRANS.retrieval.preparing.remove_noisy_wavelength_channels
   petitRADTRANS.retrieval.preparing.remove_telluric_lines_fit
   petitRADTRANS.retrieval.preparing.remove_telluric_lines_mean
   petitRADTRANS.retrieval.preparing.remove_throughput_fit
   petitRADTRANS.retrieval.preparing.remove_throughput_mean
   petitRADTRANS.retrieval.preparing.trim_spectrum
   petitRADTRANS.retrieval.preparing.polyfit
   petitRADTRANS.retrieval.preparing.sysrem


Module Contents
---------------

.. py:function:: __init_pipeline(spectrum, uncertainties)

.. py:function:: __init_pipeline_outputs(spectrum, reduction_matrix, uncertainties)

.. py:function:: __sysrem_iteration_orders_a(spectrum_uncertainties_squared, uncertainties_squared_inverted, a)

   SYSREM iteration for all orders at once. Starting with a.
   For the first iteration, a should be 1.
   The inputs are chosen in order to maximize speed.

   Args:
       spectrum_uncertainties_squared: spectral data to correct over the uncertainties ** 2 (..., exposure, wavelength)
       uncertainties_squared_inverted: invers of the squared uncertainties on the data (..., exposure, wavelength)
       a: 2-D matrix (..., exposures, wavelengths) containing the a-priori "airmass"

   Returns:
       The lower-rank estimation of the spectrum (systematics), and the estimated "extinction coefficients"


.. py:function:: __sysrem_iteration_orders_c(spectrum_uncertainties_squared, uncertainties_squared_inverted, c)

   SYSREM iteration for all orders at once. Starting with c.
   For the first iteration, c should be 1.
   The inputs are chosen in order to maximize speed.

   Args:
       spectrum_uncertainties_squared: spectral data to correct over the uncertainties ** 2 (..., exposure, wavelength)
       uncertainties_squared_inverted: invers of the squared uncertainties on the data (..., exposure, wavelength)
       c: 2-D matrix (..., exposures, wavelengths) containing the a-priori "extinction coefficients"

   Returns:
       The lower-rank estimation of the spectrum (systematics), and the estimated "extinction coefficients"


.. py:function:: bias_pipeline_metric(prepared_true_model, prepared_mock_observations, mock_observations_preparation_matrix=None, mock_noise=None)

.. py:function:: remove_noisy_wavelength_channels(spectrum, reduction_matrix, mean_subtract=False)

.. py:function:: remove_telluric_lines_fit(spectrum, reduction_matrix, airmass, uncertainties=None, mask_threshold=1e-16, polynomial_fit_degree=2, correct_uncertainties=True, uncertainties_as_weights=True)

   Remove telluric lines with a polynomial function.
   The telluric transmittance can be written as:
       T = exp(-airmass * optical_depth),
   hence the log of the transmittance can be written as a first order polynomial:
       log(T) ~ b * airmass + a.
   Using a 1st order polynomial might be not enough, as the atmospheric composition can change slowly over time. Using
   a second order polynomial, as in:
       log(T) ~ c * airmass ** 2 + b * airmass + a,
   might be safer.

   Args:
       spectrum: spectral data to correct
       reduction_matrix: matrix storing all the operations made to reduce the data
       airmass: airmass of the data
       uncertainties: uncertainties on the data
       mask_threshold: mask wavelengths where the Earth atmospheric transmittance estimate is below this value
       polynomial_fit_degree: degree of the polynomial fit of the Earth atmospheric transmittance
       correct_uncertainties:
       uncertainties_as_weights:

   Returns:
       Corrected spectral data, reduction matrix and uncertainties after correction


.. py:function:: remove_telluric_lines_mean(spectrum, reduction_matrix, uncertainties=None, mask_threshold=1e-16, uncertainties_as_weights=True)

   Remove the telluric lines using the weighted arithmetic mean over time.

   Args:
       spectrum: spectral data to correct
       reduction_matrix: matrix storing all the operations made to reduce the data
       uncertainties: uncertainties on the data
       mask_threshold: mask wavelengths where the Earth atmospheric transmittance estimate is below this value
       uncertainties_as_weights:

   Returns:
       Corrected spectral data, reduction matrix and uncertainties after correction


.. py:function:: remove_throughput_fit(spectrum, reduction_matrix, wavelengths, uncertainties=None, mask_threshold=1e-16, polynomial_fit_degree=2, correct_uncertainties=True, uncertainties_as_weights=True)

   Remove variable throughput with a polynomial function.

   Args:
       spectrum: spectral data to correct
       reduction_matrix: matrix storing all the operations made to reduce the data
       wavelengths: wavelengths of the data
       uncertainties: uncertainties on the data
       mask_threshold: mask wavelengths where the Earth atmospheric transmittance estimate is below this value
       polynomial_fit_degree: degree of the polynomial fit of the Earth atmospheric transmittance
       correct_uncertainties:
       uncertainties_as_weights:

   Returns:
       Corrected spectral data, reduction matrix and uncertainties after correction


.. py:function:: remove_throughput_mean(spectrum, reduction_matrix=None, uncertainties=None, uncertainties_as_weights=True)

   Correct for the variable throughput using the weighted arithmetic mean over wavelength.

   Args:
       spectrum: spectral data to correct
       reduction_matrix: matrix storing all the operations made to reduce the data
       uncertainties: uncertainties on the data

   Returns:
       Corrected spectral data, reduction matrix and uncertainties after correction


.. py:function:: trim_spectrum(spectrum, uncertainties=None, wavelengths=None, airmass=None, threshold_low=0.8, threshold_high=1.2, threshold_outlier=None, polynomial_fit_degree=2, relative_to_continnum=True, uncertainties_as_weights=True)

.. py:function:: polyfit(spectrum, uncertainties=None, wavelengths=None, airmass=None, tellurics_mask_threshold=0.1, polynomial_fit_degree=1, apply_throughput_removal=True, apply_telluric_lines_removal=True, correct_uncertainties=True, uncertainties_as_weights=False, full=False, **kwargs)

   Removes the telluric lines and variable throughput of some data.
   If airmass is None, the Earth atmospheric transmittance is assumed to be time-independent, so telluric transmittance
   will be fitted using the weighted arithmetic mean. Otherwise, telluric transmittance are fitted with a polynomial.

   Args:
       spectrum: spectral data to correct
       uncertainties: uncertainties on the data
       wavelengths: wavelengths of the data
       airmass: airmass of the data
       tellurics_mask_threshold: mask wavelengths where the atmospheric transmittance estimate is below this value
       polynomial_fit_degree: degree of the polynomial fit of the Earth atmospheric transmittance
       apply_throughput_removal: if True, apply the throughput removal correction
       apply_telluric_lines_removal: if True, apply the telluric lines removal correction
       correct_uncertainties:
       full: if True, return the reduced matrix and reduced uncertainties in addition to the reduced spectrum

   Returns:
       Reduced spectral data (and reduction matrix and uncertainties after reduction if full is True)


.. py:function:: sysrem(spectrum, uncertainties, wavelengths, n_passes=1, n_iterations_max=10, convergence_criterion=0.001, tellurics_mask_threshold=0.8, polynomial_fit_degree=1, apply_throughput_removal=True, apply_telluric_lines_removal=True, correct_uncertainties=True, uncertainties_as_weights=False, subtract=True, remove_mean=True, full=False, verbose=False, **kwargs)

   SYSREM preparing pipeline.
   SYSREM tries to find the coefficients a and c such as:
       S**2 = sum_ij ((spectrum_ij - a_j * c_i) / uncertainties)**2
   is minimized. Several iterations can be performed. This assumes that the spectrum is deformed by a combination of
   linear effects.
   The coefficients a and c can be seen as estimates for any strong (linear) systematic effect in the data, they are
   not necessarily related to the airmass and extinction coefficients.

   Source: Tamuz et al. 2005 (doi:10.1111/j.1365-2966.2004.08585.x).
   Thanks to Alejandro Sanchez-Lopez (26-09-2017) for sharing his version of the algorithm.

   Args:
       spectrum: spectral data to correct
       uncertainties: uncertainties on the data
       wavelengths: wavelengths of the data
       n_passes: number of SYSREM passes
       n_iterations_max: maximum number of SYSREM iterations
       convergence_criterion: SYSREM convergence criterion
       tellurics_mask_threshold: mask wavelengths where the atmospheric transmittance estimate is below this value
       polynomial_fit_degree: degree of the polynomial fit of the instrumental deformations
       apply_throughput_removal: if True, divide the spectrum by its mean over wavelengths
       apply_telluric_lines_removal: if True, apply the telluric lines removal correction
       correct_uncertainties:
       subtract: if True, subtract the fitted systematics to the spectrum instead of dividing them
       remove_mean:
       full: if True, return the reduced matrix and reduced uncertainties in addition to the reduced spectrum
       verbose: if True, print the convergence status at each iteration

   Returns:
       Reduced spectral data (and reduction matrix and uncertainties after reduction if full is True)