The Witztum, Rips and Rosenberg method

The paper

"Equidistant Letter Sequences in the Book of Genesis" appeared in Statistical Science in August 1994. Its authors were Doron Witztum, Eliyahu Rips and Yoav Rosenberg. Rips taught mathematics at the Hebrew University. People shorten the paper to WRR, and so do we.

The editor, Robert Kass, printed it with an unusual note saying the referees were baffled and that the paper was offered as a challenging puzzle. The abstract gave a result significant at the level of 0.00002. The authors credited the starting idea to Rabbi Weissmandl, who had noticed patterns of equally spaced letters decades earlier, and they set out to test whether such patterns could be explained by chance.

Their opening picture is a good one. Imagine a text in a language you cannot read, and a dictionary that lets you pick out a few words. You cannot follow the text. But if related words such as "hammer" and "anvil" tend to stand near each other, you have a reason to think the text is not meaningless. WRR asked the same question of letter sequences. They did not look for messages. They looked for a tendency of related pairs to sit close together.

Step 1: choose the word pairs

The pairs came from the lives of rabbis. WRR used Margalioth's Encyclopedia of Great Men in Israel. Their first list took every person whose entry filled at least three columns and gave a date of birth or death. That made 34 people. To avoid any appearance of fitting the test to the data, they then took a second list of everyone with an entry of 1.5 to 3 columns. That made 32. Only the second list was formally tested in the paper.

Each person has several names: full names, titles, acronyms. The appellations in the paper were listed by Professor S. Z. Havlin of Bar-Ilan University. The paper says he worked from a computer search of the Responsa database. Each person also has dates, written as day and month with no year. WRR wrote each date in three forms. For the 19th of Tishri those are י"ט תשרי, בי"ט תשרי and י"ט בתשרי. The 15th and 16th of a month were written both the common way and the older way.

Every name was paired with every date form for the same person. Words had to be 5 to 8 letters long, for technical reasons explained below. The second list produced 298 word pairs. A little over half of them gave a defined distance. Which names and which date forms went in is the main point of dispute, and the critique page covers it.

Step 2: decide which sequences count

Short words have far too many sequences to measure, so WRR limited the skip. For each word w they chose a bound D(w), the skip at which the expected number of sequences reaches 10. Skips of 1 are left out, since that is the plain text.

E(w, D) = p(w) × (D − 1) × (2L − (k − 1)(D + 2))
D(w) = the smallest D with E(w, D) ≥ 10

The paper says this bound was only there to cut the amount of computing. Bear that in mind. McKay and his co-authors later found the result is sharply at its best very near 10.

WRR also favored sequences that are "noteworthy", meaning the skip is the smallest over a large part of the text. The domain of minimality of a sequence e, written T(e), is the longest stretch of Genesis that contains e and holds no other sequence of the same word with a smaller skip. When two sequences are compared, the weight is the share of the text where both are minimal at once.

Step 3: measure the distance between two sequences

Write Genesis around a cylinder, in one long spiral with h letters per turn. A sequence becomes a straight line on the cylinder. For two sequences e and e′ on a cylinder of row length h, WRR take three lengths. The first, f, is the step between neighboring letters of e. The second, f′, is the same for e′. The third, l, is the shortest distance between a letter of e and a letter of e′.

δ(e, e′) = f² + f′² + l²
μ(e, e′) = 1 / δ(e, e′)

δ is small when both sequences look like short tight strokes and sit close together. Which row lengths should we try? The paper uses the ones where a sequence looks most compact: the nearest whole number to |d|/i for i from 1 to 10, for both skips. Ten is, in the paper's words, an arbitrarily selected moderate number. Adding up gives σ, the overall compactness of the pair.

σ(e, e′) = sum for i = 1..10 of μ on h(i)  +  sum for i = 1..10 of μ on h′(i)
ω(e, e′) = length of T(e) ∩ T(e′) / length of the text
Ω(w, w′) = sum over all e of w, e′ of w′ of ω(e, e′) × σ(e, e′)

Ω is a raw closeness score for two words. Larger means closer. But it cannot be compared across pairs yet, because word length and letter makeup change the score.

Step 4: correct for each word

This is the clever part. WRR ask how Ω compares with near misses of the same pair. Take a triple of small shifts (x, y, z), each from −2 to 2. There are 125 triples. A perturbed sequence keeps equal spacing until the last three letters, which are shifted by x, x+y and x+y+z. Compute Ω the same way for every triple where both words actually have perturbed sequences.

M = the triples (x, y, z) where Ω is defined, m = the size of M
v = how many of them have Ω(x, y, z) ≥ Ω(0, 0, 0)
c(w, w′) = v / m

So c is a rank. If the true pair beats almost every near miss, c is small. If the near misses do just as well, c is near 1. The value is defined only when the unshifted pair exists and at least 10 triples work. The perturbation needs a word of at least five letters. Words over eight letters have too few perturbed sequences to meet the rule. That gives the 5 to 8 letter limit.

Step 5: sum up a whole list

One pair says little. WRR combine all of them. Let N be the number of pairs with a defined c, let k be how many have c at or below 0.2, and let X be the product of all the c values.

P₁ = sum for j = k..N of C(N, j) (1/5)^j (4/5)^(N−j)
P₂ = X × (1 + (−ln X) + (−ln X)²/2! + ... + (−ln X)^(N−1)/(N−1)!)

P₃ and P₄ are the same two statistics after removing every name that begins with "Rabbi". The reason is simple. Several people are called Rabbi Avraham, so such names can match another man's dates.

one should think of it simply as an ordinal index

Witztum, Rips and Rosenberg, on P₂, 1994

Please take that sentence seriously. If the c values were independent and evenly spread, P₁ and P₂ would be true probabilities. They are not independent and they are not evenly spread, and the authors say they make no such assumption. These numbers rank one list against another. They are not the chance of anything.

Step 6: the permutation test

The test itself is easy to state. Keep the names. Scramble which person gets which dates. Rabbi 1's names now meet Rabbi 17's dates, and so on, and all the P values are computed again. If nothing connects each man to his own dates, the true pairing should land anywhere in the pack.

WRR ran 999,999 random scrambles plus the true pairing, one million in all. The random seed was built from numbers supplied by three scientists just before the run. The rank of the true pairing, divided by a million, gives ρ. Since four statistics were tried, the paper multiplies the smallest ρ by 4, the Bonferroni correction. They give the reason plainly: pick the best of four and you must pay for having had four chances.

The results

The smallest ρ for Genesis was 4 in a million. Four times that is 0.000016, or 1 in 62,500. That is the famous number. You will also see it rounded to 1 in 60,000. In every control text the result was flat. The paper's conclusion is that the proximity of related sequences in Genesis is not due to chance.

Later computation changed the figure a little. With 200 million scrambles, McKay and his co-authors put the ranks for P₂ and P₄ at about 1.9 and 0.68 in a million. They agree the number is that small. They disagree about what produced it.

These figures are for the Koren edition of Genesis. On the Leningrad text we search, which differs from Koren by 22 letters in Genesis, the published best rank is 0.0013 for the same list. McKay and his co-authors give that figure in their Table 4. The reader who wants to test sequences of this kind should expect edition to matter.

Using this measure in our lab

The proximity panel in the laboratory computes c(w, w′) for a pair you choose, using WRR's definitions on the Leningrad text. A full c needs millions of distance measurements, so the browser works within a bounded skip range. When a pair is too common to rank at full reach, the panel tells you and offers a smaller range. It never rounds its way past the problem.

Two cautions go with every number there. First, c was designed for words of 5 to 8 letters and for lists, not single pairs. One low c is not a finding. Second, if you try many pairs and keep the best, the trial counter will say so. WRR's own test needed 298 pairs and a million scrambles, and it is still argued over.

What came next

In 1999 the same journal printed a long reply by Brendan McKay, Dror Bar-Natan, Maya Bar-Hillel and Gil Kalai. They accepted that the number was small and argued that it came from the choices made in building the list. They showed a list with a similar score in War and Peace, and their own replications found nothing. The critique page sets out their case and the answers to it, so that you can weigh the two.