scipy.stats._stats_py:ttest_ind

Notes

Suppose we observe two independent samples, e.g. flower petal lengths, and we are considering whether the two samples were drawn from the same population (e.g. the same species of flower or two species with similar petal characteristics) or two different populations.

The t-test quantifies the difference between the arithmetic means of the two samples. The p-value quantifies the probability of observing as or more extreme values assuming the null hypothesis, that the samples are drawn from populations with the same population means, is true. A p-value larger than a chosen threshold (e.g. 5% or 1%) indicates that our observation is not so unlikely to have occurred by chance. Therefore, we do not reject the null hypothesis of equal population means. If the p-value is smaller than our threshold, then we have evidence against the null hypothesis of equal population means.

By default, the p-value is determined by comparing the t-statistic of the observed data against a theoretical t-distribution.

It is also possible to compute the test statistic using a permutation test by passing method=scipy.stats.PermutationMethod(n_resamples=permutations), where permutations is the desired number of "permutations" to use in forming the null distribution. When 1 < permutations < binom(n, k), where

• k is the number of observations in a,

• n is the total number of observations in a and b, and

• binom(n, k) is the binomial coefficient (n choose k),

the data are pooled (concatenated), randomly assigned to either group a or b, and the t-statistic is calculated. This process is performed repeatedly (permutations times), generating a distribution of the t-statistic under the null hypothesis, and the t-statistic of the observed data is compared to this distribution to determine the p-value. Specifically, the p-value reported is the "achieved significance level" (ASL) as defined in 4.4 of ^[3]. Note that there are other ways of estimating p-values using randomized permutation tests; for other options, see the more general permutation_test.

When permutations >= binom(n, k), an exact test is performed: the data are partitioned between the groups in each distinct way exactly once.

The permutation test can be computationally expensive and not necessarily more accurate than the analytical test, but it does not make strong assumptions about the shape of the underlying distribution.

Use of trimming is commonly referred to as the trimmed t-test. At times called Yuen's t-test, this is an extension of Welch's t-test, with the difference being the use of winsorized means in calculation of the variance and the trimmed sample size in calculation of the statistic. Trimming is recommended if the underlying distribution is long-tailed or contaminated with outliers ^[4].

The statistic is calculated as (np.mean(a) - np.mean(b))/se, where se is the standard error. Therefore, the statistic will be positive when the sample mean of a is greater than the sample mean of b and negative when the sample mean of a is less than the sample mean of b.

Beginning in SciPy 1.9, np.matrix inputs (not recommended for new code) are converted to np.ndarray before the calculation is performed. In this case, the output will be a scalar or np.ndarray of appropriate shape rather than a 2D np.matrix. Similarly, while masked elements of masked arrays are ignored, the output will be a scalar or np.ndarray rather than a masked array with mask=False.

Array API Standard Support

ttest_ind has experimental support for Python Array API Standard compatible backends in addition to NumPy. Please consider testing these features by setting an environment variable SCIPY_ARRAY_API=1 and providing CuPy, PyTorch, JAX, or Dask arrays as array arguments. The following combinations of backend and device (or other capability) are supported.

====================  ====================  ====================
Library               CPU                   GPU
====================  ====================  ====================
NumPy                 ✅                     n/a                 
CuPy                  n/a                   ✅                   
PyTorch               ✅                     ⛔                   
JAX                   ✅                     ✅                   
Dask                  ✅                     n/a                 
====================  ====================  ====================

See dev-arrayapi for more information.