NSI Research: Effect Size Before the Headline

A threshold is not the result’s full meaning

A proposed indicator can acquire an impressive headline when a statistical test crosses a chosen threshold. That headline still leaves the reader asking what changed, by how much and whether the difference matters for the decision being studied. The American Statistical Association cautions that a p-value does not measure an effect’s size or importance. It also warns against basing conclusions only on whether a result crosses a particular threshold.^[1] Those principles apply to proposed NSI research just as they apply to other quantitative claims. They do not establish that the Nervous System Indicator itself is valid.

Name the outcome before comparing its magnitude

Consider an entirely hypothetical evaluation of a decision worksheet: one group completes it in a median of nine minutes and another in ten. “One minute shorter” is interpretable; “better performance” is not yet precise enough. Does the study concern completion time, rule violations, self-reported pressure or net trading results? Each outcome needs its own units and interpretation. Faster completion would not by itself demonstrate more accurate decisions. These invented numbers illustrate reporting, not findings about NSI. A Prospective NSI Observation Plan explains why the question and observation procedure should be specified before results are seen.

Request the uncertainty alongside the estimate

A clear research summary should show the estimated difference, its uncertainty and the assumptions behind the analysis. Also ask how many observations were available, what was missing and which comparisons were examined. If an estimate is imprecise, the appropriate response is to preserve that uncertainty rather than select the most favourable interpretation. Avoid translating a p-value into a probability that the framework is true; the ASA explicitly rejects that reading.^[1] What Does the Simplest Comparator Show? adds another necessary question: compared with what? A difference from a weak comparison may leave the practical question unresolved.

Define what would justify a next step

Before a future study, write the difference that would be meaningful for its stated purpose and explain why. For a record-keeping tool, usability might matter; for a trading claim, transaction costs and downside outcomes would require attention too. An interesting exploratory result can justify a better study without justifying a sales promise. The NSI Test-Set Boundary discusses keeping evaluation separate from development. NSI.GOLD and The Gold Nervous System Indicator are routes into the author’s proposed framework. They should be read as that framework and its questions, not as proof of a validated biological measurement, clinical assessment or reliable profit signal.