{ "__type": "IngestedDoc", "__tag": 4010, "_content": { "Notes": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If this function is applied to tests with a discrete statistics such as any rank test or contingency-table test, it will yield systematically wrong results, e.g. Fisher's method will systematically overestimate the p-value " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "1" }, { "__type": "Text", "__tag": 4046, "value": ". This problem becomes less severe for large sample sizes when the discrete distributions become approximately continuous." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The differences between the methods can be best illustrated by their statistics and what aspects of a combination of p-values they emphasise when considering significance " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "2" }, { "__type": "Text", "__tag": 4046, "value": ". For example, methods emphasising large p-values are more sensitive to strong false and true negatives; conversely methods focussing on small p-values are sensitive to positives." } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The statistics of Fisher's method (also known as Fisher's combined probability test) " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "3" }, { "__type": "Text", "__tag": 4046, "value": " is " }, { "__type": "InlineMath", "__tag": 4057, "value": "-2\\sum_i \\log(p_i)" }, { "__type": "Text", "__tag": 4046, "value": ", which is equivalent (as a test statistics) to the product of individual p-values: " }, { "__type": "InlineMath", "__tag": 4057, "value": "\\prod_i p_i" }, { "__type": "Text", "__tag": 4046, "value": ". Under the null hypothesis, this statistics follows a " }, { "__type": "InlineMath", "__tag": 4057, "value": "\\chi^2" }, { "__type": "Text", "__tag": 4046, "value": " distribution. This method emphasises small p-values." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Pearson's method uses " }, { "__type": "InlineMath", "__tag": 4057, "value": "-2\\sum_i\\log(1-p_i)" }, { "__type": "Text", "__tag": 4046, "value": ", which is equivalent to " }, { "__type": "InlineMath", "__tag": 4057, "value": "\\prod_i \\frac{1}{1-p_i}" }, { "__type": "Text", "__tag": 4046, "value": " " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "2" }, { "__type": "Text", "__tag": 4046, "value": ". It thus emphasises large p-values." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Mudholkar and George compromise between Fisher's and Pearson's method by averaging their statistics " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "4" }, { "__type": "Text", "__tag": 4046, "value": ". Their method emphasises extreme p-values, both close to 1 and 0." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Stouffer's method " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "5" }, { "__type": "Text", "__tag": 4046, "value": " uses Z-scores and the statistic: " }, { "__type": "InlineMath", "__tag": 4057, "value": "\\sum_i \\Phi^{-1} (p_i)" }, { "__type": "Text", "__tag": 4046, "value": ", where " }, { "__type": "InlineMath", "__tag": 4057, "value": "\\Phi" }, { "__type": "Text", "__tag": 4046, "value": " is the CDF of the standard normal distribution. The advantage of this method is that it is straightforward to introduce weights, which can make Stouffer's method more powerful than Fisher's method when the p-values are from studies of different size " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "6" }, { "__type": "Text", "__tag": 4046, "value": " " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "7" }, { "__type": "Text", "__tag": 4046, "value": "." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Tippett's method uses the smallest p-value as a statistic. (Mind that this minimum is not the combined p-value.)" } ] } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Fisher's method may be extended to combine p-values from dependent tests " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "8" }, { "__type": "Text", "__tag": 4046, "value": ". Extensions such as Brown's method and Kost's method are not currently implemented." } ] }, { "__type": "Admonition", "__tag": 4056, "kind": "versionadded", "base_type": "neutral", "children": [ { "__type": "AdmonitionTitle", "__tag": 4055, "children": [ { "__type": "Text", "__tag": 4046, "value": "versionadded 0.15.0" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Beginning in SciPy 1.9, " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.matrix" }, { "__type": "Text", "__tag": 4046, "value": " inputs (not recommended for new code) are converted to " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.ndarray" }, { "__type": "Text", "__tag": 4046, "value": " before the calculation is performed. In this case, the output will be a scalar or " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.ndarray" }, { "__type": "Text", "__tag": 4046, "value": " of appropriate shape rather than a 2D " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.matrix" }, { "__type": "Text", "__tag": 4046, "value": ". Similarly, while masked elements of masked arrays are ignored, the output will be a scalar or " }, { "__type": "InlineCode", "__tag": 4051, "value": "np.ndarray" }, { "__type": "Text", "__tag": 4046, "value": " rather than a masked array with " }, { "__type": "InlineCode", "__tag": 4051, "value": "mask=False" }, { "__type": "Text", "__tag": 4046, "value": "." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "Array API Standard Support" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "CrossRef", "__tag": 4002, "value": "combine_pvalues", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "module", "path": "scipy.stats._stats_py:combine_pvalues" }, "kind": "module" }, { "__type": "Text", "__tag": 4046, "value": " has experimental support for Python Array API Standard compatible backends in addition to NumPy. Please consider testing these features by setting an environment variable " }, { "__type": "InlineCode", "__tag": 4051, "value": "SCIPY_ARRAY_API=1" }, { "__type": "Text", "__tag": 4046, "value": " and providing CuPy, PyTorch, JAX, or Dask arrays as array arguments. The following combinations of backend and device (or other capability) are supported." } ] }, { "__type": "Code", "__tag": 4050, "value": "==================== ==================== ====================\nLibrary CPU GPU\n==================== ==================== ====================\nNumPy ✅ n/a \nCuPy n/a ✅ \nPyTorch ✅ ⛔ \nJAX ⚠️ no JIT ⚠️ no JIT \nDask ⚠️ computes graph n/a \n==================== ==================== ====================", "execution_status": null }, { "__type": "Blockquote", "__tag": 4059, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "See " }, { "__type": "InlineRole", "__tag": 4003, "value": "dev-arrayapi", "domain": null, "role": "ref", "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " for more information." } ] } ] } ], "title": [], "level": 0, "target": null }, "Warns": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Raises": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Yields": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Methods": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Returns": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Parameters", "__tag": 4026, "children": [ { "__type": "DocParam", "__tag": 4016, "name": "res", "annotation": "SignificanceResult", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "An object containing attributes:" } ] }, { "__type": "DefList", "__tag": 4033, "children": [ { "__type": "DefListItem", "__tag": 4037, "dt": { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "statistic" } ] }, "dd": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "statistic" } ] } ] }, { "__type": "DefListItem", "__tag": 4037, "dt": { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "pvalue" } ] }, "dd": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "pvalue" } ] } ] } ] } ] } ] } ], "title": [], "level": 0, "target": null }, "Summary": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Combine p-values from independent tests that bear upon the same hypothesis." } ] } ], "title": [], "level": 0, "target": null }, "Receives": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Warnings": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Attributes": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Parameters": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Parameters", "__tag": 4026, "children": [ { "__type": "DocParam", "__tag": 4016, "name": "pvalues", "annotation": "array_like", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Array of p-values assumed to come from independent tests based on continuous distributions." } ] } ] }, { "__type": "DocParam", "__tag": 4016, "name": "method", "annotation": "{'fisher', 'pearson', 'tippett', 'stouffer', 'mudholkar_george'}", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Name of method to use to combine p-values." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The available methods are (see Notes for details):" } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "'fisher': Fisher's method (Fisher's combined probability test)" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "'pearson': Pearson's method" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "'mudholkar_george': Mudholkar's and George's method" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "'tippett': Tippett's method" } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "'stouffer': Stouffer's Z-score method" } ] } ] } ] } ] }, { "__type": "DocParam", "__tag": 4016, "name": "weights", "annotation": "array_like, optional", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Optional array of weights used only for Stouffer's Z-score method. Ignored by other methods." } ] } ] }, { "__type": "DocParam", "__tag": 4016, "name": "axis", "annotation": "int or None, default: 0", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If an int, the axis of the input along which to compute the statistic. The statistic of each axis-slice (e.g. row) of the input will appear in a corresponding element of the output. If " }, { "__type": "InlineCode", "__tag": 4051, "value": "None" }, { "__type": "Text", "__tag": 4046, "value": ", the input will be raveled before computing the statistic." } ] } ] }, { "__type": "DocParam", "__tag": 4016, "name": "nan_policy", "annotation": "{'propagate', 'omit', 'raise'}", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Defines how to handle input NaNs." } ] }, { "__type": "BulletList", "__tag": 4053, "ordered": false, "start": 1, "children": [ { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "propagate" }, { "__type": "Text", "__tag": 4046, "value": ": if a NaN is present in the axis slice (e.g. row) along which the statistic is computed, the corresponding entry of the output will be NaN." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "omit" }, { "__type": "Text", "__tag": 4046, "value": ": NaNs will be omitted when performing the calculation. If insufficient data remains in the axis slice along which the statistic is computed, the corresponding entry of the output will be NaN." } ] } ] }, { "__type": "ListItem", "__tag": 4054, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "InlineCode", "__tag": 4051, "value": "raise" }, { "__type": "Text", "__tag": 4046, "value": ": if a NaN is present, a " }, { "__type": "InlineCode", "__tag": 4051, "value": "ValueError" }, { "__type": "Text", "__tag": 4046, "value": " will be raised." } ] } ] } ] } ] }, { "__type": "DocParam", "__tag": 4016, "name": "keepdims", "annotation": "bool, default: False", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If this is set to True, the axes which are reduced are left in the result as dimensions with size one. With this option, the result will broadcast correctly against the input array." } ] } ] } ] } ], "title": [], "level": 0, "target": null }, "Extended Summary": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "These methods are intended only for combining p-values from hypothesis tests based upon continuous distributions." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Each method assumes that under the null hypothesis, the p-values are sampled independently and uniformly from the interval [0, 1]. A test statistic (different for each method) is computed and a combined p-value is calculated based upon the distribution of this test statistic under the null hypothesis." } ] } ], "title": [], "level": 0, "target": null }, "Other Parameters": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null } }, "_ordered_sections": [ "Summary", "Extended Summary", "Parameters", "Attributes", "Methods", "Returns", "Yields", "Receives", "Other Parameters", "Raises", "Warns", "Warnings", "Notes" ], "item_file": "/scipy/stats/_stats_py.py", "item_line": 8741, "item_type": "function", "aliases": [ "scipy.stats.combine_pvalues" ], "example_section_data": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Text", "__tag": 4046, "value": "Suppose we wish to combine p-values from four independent tests\nof the same null hypothesis using Fisher's method (default).\n\n" }, { "__type": "Code", "__tag": 4050, "value": "from scipy.stats import combine_pvalues\npvalues = [0.1, 0.05, 0.02, 0.3]\n", "execution_status": "success" }, { "__type": "Code", "__tag": 4050, "value": "combine_pvalues(pvalues)\n", "execution_status": "failure" }, { "__type": "Text", "__tag": 4046, "value": "\nWhen the individual p-values carry different weights, consider Stouffer's\nmethod.\n\n" }, { "__type": "Code", "__tag": 4050, "value": "weights = [1, 2, 3, 4]\nres = combine_pvalues(pvalues, method='stouffer', weights=weights)\n", "execution_status": "success" }, { "__type": "Code", "__tag": 4050, "value": "res.pvalue\n", "execution_status": "failure" } ], "title": [], "level": 0, "target": null }, "see_also": [], "signature": { "__type": "SignatureNode", "__tag": 4029, "kind": "function", "parameters": [ { "__type": "SigParam", "__tag": 4030, "name": "pvalues", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "POSITIONAL_OR_KEYWORD", "default": { "__type": "Empty", "__tag": 4031 } }, { "__type": "SigParam", "__tag": 4030, "name": "method", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "POSITIONAL_OR_KEYWORD", "default": "fisher" }, { "__type": "SigParam", "__tag": 4030, "name": "weights", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "POSITIONAL_OR_KEYWORD", "default": "None" }, { "__type": "SigParam", "__tag": 4030, "name": "axis", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "KEYWORD_ONLY", "default": "0" }, { "__type": "SigParam", "__tag": 4030, "name": "nan_policy", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "KEYWORD_ONLY", "default": "propagate" }, { "__type": "SigParam", "__tag": 4030, "name": "keepdims", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "KEYWORD_ONLY", "default": "False" } ], "return_annotation": { "__type": "Empty", "__tag": 4031 }, "target_name": "combine_pvalues" }, "references": [ ".. [1] Kincaid, W. M., \"The Combination of Tests Based on Discrete", " Distributions.\" Journal of the American Statistical Association 57,", " no. 297 (1962), 10-19.", ".. [2] Heard, N. and Rubin-Delanchey, P. \"Choosing between methods of", " combining p-values.\" Biometrika 105.1 (2018): 239-246.", ".. [3] https://en.wikipedia.org/wiki/Fisher%27s_method", ".. [4] George, E. O., and G. S. Mudholkar. \"On the convolution of logistic", " random variables.\" Metrika 30.1 (1983): 1-13.", ".. [5] https://en.wikipedia.org/wiki/Fisher%27s_method#Relation_to_Stouffer.27s_Z-score_method", ".. [6] Whitlock, M. C. \"Combining probability from independent tests: the", " weighted Z-method is superior to Fisher's approach.\" Journal of", " Evolutionary Biology 18, no. 5 (2005): 1368-1373.", ".. [7] Zaykin, Dmitri V. \"Optimally weighted Z-test is a powerful method", " for combining probabilities in meta-analysis.\" Journal of", " Evolutionary Biology 24, no. 8 (2011): 1836-1841.", ".. [8] https://en.wikipedia.org/wiki/Extensions_of_Fisher%27s_method" ], "qa": "scipy.stats._stats_py:combine_pvalues", "arbitrary": [], "local_refs": [ "axis", "keepdims", "method", "nan_policy", "pvalues", "res", "weights" ] }