Under the Star Gate program and later under the auspices of the Laboratories for Fundamental Research (LFR) there were two approaches for the analysis of psi data. The overriding principle for both is that the analyst (a.k.a., judge) must be blind to the correct answer. Another important notion is that any human response including one mediated by psi can NEVER be considered as random. The only part of a psi session that is and must be random is the target stimulus.
At the outset it must be stated that psi perceptions are not multimedia extravaganzas. They are more like seeing through squinted eyes.
Rank-Order: Here a given response is to be matched to a set of randomly selected target stimuli one of which was the intended stimulus. In most cases under Star Gate and LFR there were four decoy stimuli and one intended stimulus for a total of five images. In other techniques such as the Ganzfeld method, typically the total number of stimuli are four. Here is an example of a rank-order analysis task.
The drawing on the left is the percipient’s complete response followed by five images one of which is the intended target. Even though the response may not match any of the photos, the analyst is required to pick the best match, followed by the second best, and so on. This is not as easy as it may sound. For example, the CROSS in the response could match the fronds of the palm tries in the first photo or the X in the windmills. WAVE in the response easily matches the first photo; SEA WALL might easily match the 4th photo. For sure this photo is not a see all, but conceptually it is a wall of rock which is not seen but water nonetheless. The GAP in the response best matches looking down on the city photo.
The analyst for the session chose the following for the best to worst match to the response: Windmills, Beach, Waterfall, Desert and City—the correct answer was the Windmills, so the analyst was correct. Keep in mind, however, that there is a 20% chance the analyst got it right by chance alone, and in that case the percipient would have had no psi ability at all. We note that a rank-order analysis depends strongly upon how different the target stimuli are from one another including the intended stimulus. For example, in this case if all photos contained an X-like feature, the analyst would have had a much more difficult task.
Rating System: In a rating system some metric is used to determine the degree to which the response matches each photo independent of the others in the judging set. Under the Star Gate program and later under LFR the metric that was used was called the Figure of Merit (FoM). The FoM is defined as the product of the Accuracy times the Reliability. The Accuracy is defined as the fraction of the single target stimuli (i.e., intended or decoy) that was correct in the response. The Reliability is defined as the fraction of the response that was correct. Fuzzy set mathematics is used to define each of these terms and the details can be found in the paper Anomalous Cognition: Two Protocols for Data Collection and Analyses in the Library.