{ "__type": "IngestedDoc", "__tag": 4010, "_content": { "Notes": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "In multiple hypothesis testing, false discovery control procedures tend to offer higher power than familywise error rate control procedures (e.g. Bonferroni correction " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "1" }, { "__type": "Text", "__tag": 4046, "value": ")." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "If the p-values correspond with independent tests (or tests with \"positive regression dependencies\" " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "2" }, { "__type": "Text", "__tag": 4046, "value": "), rejecting null hypotheses corresponding with Benjamini-Hochberg-adjusted p-values below " }, { "__type": "InlineMath", "__tag": 4057, "value": "q" }, { "__type": "Text", "__tag": 4046, "value": " controls the false discovery rate at a level less than or equal to " }, { "__type": "InlineMath", "__tag": 4057, "value": "q m_0 / m" }, { "__type": "Text", "__tag": 4046, "value": ", where " }, { "__type": "InlineMath", "__tag": 4057, "value": "m_0" }, { "__type": "Text", "__tag": 4046, "value": " is the number of true null hypotheses and " }, { "__type": "InlineMath", "__tag": 4057, "value": "m" }, { "__type": "Text", "__tag": 4046, "value": " is the total number of null hypotheses tested. The same is true even for dependent tests when the p-values are adjusted accorded to the more conservative Benjaminini-Yekutieli procedure." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The adjusted p-values produced by this function are comparable to those produced by the R function " }, { "__type": "InlineCode", "__tag": 4051, "value": "p.adjust" }, { "__type": "Text", "__tag": 4046, "value": " and the statsmodels function " }, { "__type": "InlineRole", "__tag": 4003, "value": "statsmodels.stats.multitest.multipletests", "domain": null, "role": null, "inventory": null }, { "__type": "Text", "__tag": 4046, "value": ". Please consider the latter for more advanced methods of multiple comparison correction." } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Strong", "__tag": 4048, "children": [ { "__type": "Text", "__tag": 4046, "value": "Array API Standard Support" } ] } ] }, { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "CrossRef", "__tag": 4002, "value": "false_discovery_control", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "module", "path": "scipy.stats._morestats:false_discovery_control" }, "kind": "module" }, { "__type": "Text", "__tag": 4046, "value": " has experimental support for Python Array API Standard compatible backends in addition to NumPy. Please consider testing these features by setting an environment variable " }, { "__type": "InlineCode", "__tag": 4051, "value": "SCIPY_ARRAY_API=1" }, { "__type": "Text", "__tag": 4046, "value": " and providing CuPy, PyTorch, JAX, or Dask arrays as array arguments. The following combinations of backend and device (or other capability) are supported." } ] }, { "__type": "Code", "__tag": 4050, "value": "==================== ==================== ====================\nLibrary CPU GPU\n==================== ==================== ====================\nNumPy ✅ n/a \nCuPy n/a ✅ \nPyTorch ✅ ✅ \nJAX ⚠️ no JIT ⚠️ no JIT \nDask ⛔ n/a \n==================== ==================== ====================", "execution_status": null }, { "__type": "Blockquote", "__tag": 4059, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "See " }, { "__type": "InlineRole", "__tag": 4003, "value": "dev-arrayapi", "domain": null, "role": "ref", "inventory": null }, { "__type": "Text", "__tag": 4046, "value": " for more information." } ] } ] } ], "title": [], "level": 0, "target": null }, "Warns": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Raises": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Yields": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Methods": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Returns": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Parameters", "__tag": 4026, "children": [ { "__type": "DocParam", "__tag": 4016, "name": "ps_adusted", "annotation": "array_like", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The adjusted p-values. If the null hypothesis is rejected where these fall below a specified level, the false discovery rate is controlled at that level." } ] } ] } ] } ], "title": [], "level": 0, "target": null }, "Summary": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "Adjust p-values to control the false discovery rate." } ] } ], "title": [], "level": 0, "target": null }, "Receives": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Warnings": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Attributes": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null }, "Parameters": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Parameters", "__tag": 4026, "children": [ { "__type": "DocParam", "__tag": 4016, "name": "ps", "annotation": "1D array_like", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The p-values to adjust. Elements must be real numbers between 0 and 1." } ] } ] }, { "__type": "DocParam", "__tag": 4016, "name": "axis", "annotation": "int", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The axis along which to perform the adjustment. The adjustment is performed independently along each axis-slice. If " }, { "__type": "ParamRef", "__tag": 4071, "name": "axis" }, { "__type": "Text", "__tag": 4046, "value": " is None, " }, { "__type": "ParamRef", "__tag": 4071, "name": "ps" }, { "__type": "Text", "__tag": 4046, "value": " is raveled before performing the adjustment." } ] } ] }, { "__type": "DocParam", "__tag": 4016, "name": "method", "annotation": "{'bh', 'by'}", "desc": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The false discovery rate control procedure to apply: " }, { "__type": "InlineCode", "__tag": 4051, "value": "'bh'" }, { "__type": "Text", "__tag": 4046, "value": " is for Benjamini-Hochberg " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "1" }, { "__type": "Text", "__tag": 4046, "value": " (Eq. 1), " }, { "__type": "InlineCode", "__tag": 4051, "value": "'by'" }, { "__type": "Text", "__tag": 4046, "value": " is for Benjaminini-Yekutieli " }, { "__type": "FootnoteReference", "__tag": 4066, "label": "2" }, { "__type": "Text", "__tag": 4046, "value": " (Theorem 1.3). The latter is more conservative, but it is guaranteed to control the FDR even when the p-values are not from independent tests." } ] } ] } ] } ], "title": [], "level": 0, "target": null }, "Extended Summary": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Paragraph", "__tag": 4045, "children": [ { "__type": "Text", "__tag": 4046, "value": "The false discovery rate (FDR) is the expected proportion of rejected null hypotheses that are actually true. If the null hypothesis is rejected when the " }, { "__type": "Emphasis", "__tag": 4047, "children": [ { "__type": "Text", "__tag": 4046, "value": "adjusted" } ] }, { "__type": "Text", "__tag": 4046, "value": " p-value falls below a specified level, the false discovery rate is controlled at that level." } ] } ], "title": [], "level": 0, "target": null }, "Other Parameters": { "__type": "Section", "__tag": 4015, "children": [], "title": [], "level": 0, "target": null } }, "_ordered_sections": [ "Summary", "Extended Summary", "Parameters", "Attributes", "Methods", "Returns", "Yields", "Receives", "Other Parameters", "Raises", "Warns", "Warnings", "Notes" ], "item_file": "/scipy/stats/_morestats.py", "item_line": 4637, "item_type": "function", "aliases": [ "scipy.stats.false_discovery_control" ], "example_section_data": { "__type": "Section", "__tag": 4015, "children": [ { "__type": "Text", "__tag": 4046, "value": "We follow the example from [1]_.\n\n Thrombolysis with recombinant tissue-type plasminogen activator (rt-PA)\n and anisoylated plasminogen streptokinase activator (APSAC) in\n myocardial infarction has been proved to reduce mortality. [4]_\n investigated the effects of a new front-loaded administration of rt-PA\n versus those obtained with a standard regimen of APSAC, in a randomized\n multicentre trial in 421 patients with acute myocardial infarction.\n\nThere were four families of hypotheses tested in the study, the last of\nwhich was \"cardiac and other events after the start of thrombolitic\ntreatment\". FDR control may be desired in this family of hypotheses\nbecause it would not be appropriate to conclude that the front-loaded\ntreatment is better if it is merely equivalent to the previous treatment.\n\nThe p-values corresponding with the 15 hypotheses in this family were\n\n" }, { "__type": "Code", "__tag": 4050, "value": "ps = [0.0001, 0.0004, 0.0019, 0.0095, 0.0201, 0.0278, 0.0298, 0.0344,\n 0.0459, 0.3240, 0.4262, 0.5719, 0.6528, 0.7590, 1.000]\n", "execution_status": "success" }, { "__type": "Text", "__tag": 4046, "value": "\nIf the chosen significance level is 0.05, we may be tempted to reject the\nnull hypotheses for the tests corresponding with the first nine p-values,\nas the first nine p-values fall below the chosen significance level.\nHowever, this would ignore the problem of \"multiplicity\": if we fail to\ncorrect for the fact that multiple comparisons are being performed, we\nare more likely to incorrectly reject true null hypotheses.\n\nOne approach to the multiplicity problem is to control the family-wise\nerror rate (FWER), that is, the rate at which the null hypothesis is\nrejected when it is actually true. A common procedure of this kind is the\nBonferroni correction [1]_. We begin by multiplying the p-values by the\nnumber of hypotheses tested.\n\n" }, { "__type": "Code", "__tag": 4050, "value": "import numpy as np\n", "execution_status": "success" }, { "__type": "Code", "__tag": 4050, "value": "np.array(ps) * len(ps)\n", "execution_status": "failure" }, { "__type": "Text", "__tag": 4046, "value": "\nTo control the FWER at 5%, we reject only the hypotheses corresponding\nwith adjusted p-values less than 0.05. In this case, only the hypotheses\ncorresponding with the first three p-values can be rejected. According to\n[1]_, these three hypotheses concerned \"allergic reaction\" and \"two\ndifferent aspects of bleeding.\"\n\nAn alternative approach is to control the false discovery rate: the\nexpected fraction of rejected null hypotheses that are actually true. The\nadvantage of this approach is that it typically affords greater power: an\nincreased rate of rejecting the null hypothesis when it is indeed false. To\ncontrol the false discovery rate at 5%, we apply the Benjamini-Hochberg\np-value adjustment.\n\n" }, { "__type": "Code", "__tag": 4050, "value": "from scipy import stats\n", "execution_status": "success" }, { "__type": "Code", "__tag": 4050, "value": "stats.false_discovery_control(ps)\n", "execution_status": "failure" }, { "__type": "Text", "__tag": 4046, "value": "\nNow, the first *four* adjusted p-values fall below 0.05, so we would reject\nthe null hypotheses corresponding with these *four* p-values. Rejection\nof the fourth null hypothesis was particularly important to the original\nstudy as it led to the conclusion that the new treatment had a\n\"substantially lower in-hospital mortality rate.\"\n\nFor simplicity of exposition, the p-values in the example above were given in\nsorted order, but this is not required; `false_discovery_control` returns\nadjusted p-values in order corresponding with the input `ps`.\n\n" }, { "__type": "Code", "__tag": 4050, "value": "stats.false_discovery_control([0.5, 0.6, 0.1, 0.001])\n", "execution_status": "success" } ], "title": [], "level": 0, "target": null }, "see_also": [ { "__type": "SeeAlsoItem", "__tag": 4028, "name": { "__type": "CrossRef", "__tag": 4002, "value": "combine_pvalues", "reference": { "__type": "LocalRef", "__tag": 4022, "kind": "module", "path": "scipy.stats._stats_py:combine_pvalues" }, "kind": "module" }, "descriptions": [], "type": "func" }, { "__type": "SeeAlsoItem", "__tag": 4028, "name": { "__type": "CrossRef", "__tag": 4002, "value": "statsmodels.stats.multitest.multipletests", "reference": { "__type": "RefInfo", "__tag": 4000, "module": "current-module", "version": "current-version", "kind": "to-resolve", "path": "statsmodels.stats.multitest.multipletests" }, "kind": "module" }, "descriptions": [], "type": "func" } ], "signature": { "__type": "SignatureNode", "__tag": 4029, "kind": "function", "parameters": [ { "__type": "SigParam", "__tag": 4030, "name": "ps", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "POSITIONAL_OR_KEYWORD", "default": { "__type": "Empty", "__tag": 4031 } }, { "__type": "SigParam", "__tag": 4030, "name": "axis", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "KEYWORD_ONLY", "default": "0" }, { "__type": "SigParam", "__tag": 4030, "name": "method", "annotation": { "__type": "Empty", "__tag": 4031 }, "kind": "KEYWORD_ONLY", "default": "bh" } ], "return_annotation": { "__type": "Empty", "__tag": 4031 }, "target_name": "false_discovery_control" }, "references": [ ".. [1] Benjamini, Yoav, and Yosef Hochberg. \"Controlling the false", " discovery rate: a practical and powerful approach to multiple", " testing.\" Journal of the Royal statistical society: series B", " (Methodological) 57.1 (1995): 289-300.", "", ".. [2] Benjamini, Yoav, and Daniel Yekutieli. \"The control of the false", " discovery rate in multiple testing under dependency.\" Annals of", " statistics (2001): 1165-1188.", "", ".. [3] TileStats. FDR - Benjamini-Hochberg explained - Youtube.", " https://www.youtube.com/watch?v=rZKa4tW2NKs.", "", ".. [4] Neuhaus, Karl-Ludwig, et al. \"Improved thrombolysis in acute", " myocardial infarction with front-loaded administration of alteplase:", " results of the rt-PA-APSAC patency study (TAPS).\" Journal of the", " American College of Cardiology 19.5 (1992): 885-891." ], "qa": "scipy.stats._morestats:false_discovery_control", "arbitrary": [], "local_refs": [ "axis", "method", "ps", "ps_adusted" ] }