The communities experiment and the committee

The claim

In the mid-1990s Harold Gans took the 66 rabbis of WRR's two lists and replaced their dates with the names of the communities they were linked to, drawing on data prepared by Zvi Inbal. He reported that names stood close to those communities in Genesis, later giving a rank of about 6 in a million. He withdrew the preprint in 1998 and reissued the result afterward.

This idea had appeal. Community names are fairly objective compared with appellations, and the test used the same method as the rabbis experiment. It looked like an independent replication.

A first check

Barry Simon designed a stricter version: every city named in each rabbi's entry in the encyclopedia, with no spelling changes and no prefixes. MBBK report his results. The least permutation ranks were 0.133 and 0.324 for the two lists from Margalioth, and 0.324 and 0.052 from Encyclopedia Hebraica. Keeping closer to Gans's own rules gave 0.550 and 0.117. None of that is a signal.

The committee

In 1996 a committee was set up at the Hebrew University. The chair was Robert Aumann, the game theorist and later Nobel laureate, who had handled the rabbis paper at a national academy years before. The members were Hillel Furstenberg, Eliyahu Rips, Isaak Lapides and Dror Bar-Natan. So it held a co-author of the original paper and one of its sharpest critics.

Instead of using an encyclopedia, they had outside experts, chosen by Furstenberg, gather the data. They ran two experiments. The "fresh" one was a new design, apart from the theme. The "replicative" one followed Gans's procedure as closely as possible.

These sit in the middle of the pack, where chance would put them. McKay did the computing, and Witztum confirmed the numbers.

The disagreement

The committee's report appeared in 2004. Aumann and Furstenberg wrote that both tests failed to confirm the existence of the code. A companion paper gives four separate analyses. Aumann and Furstenberg support the report, while Lapides and Witztum do not. Their main objection is to the data: errors by the consultants and prefixes left out.

Gans, Inbal and Bombach then published a version in 2006 with community names derived by an algorithm. They report ranks of 4 and 2 in a million. They attribute the committee's outcome to human consultants' errors. In 2022 Witztum measured the committee's own fresh list with shuffled verses in place of the permutation test, and reports a strong result: 205 of 50,000, and as few as 3 of 50,000 with prefixes added.

Aumann has written that the research he supervised failed to confirm the codes, that it did not establish their non-existence, and that he regards them as improbable.

Why this entry matters

A committee that includes both sides is rare in any field, and a clean disagreement like this one is worth studying. Notice what is agreed: the numbers are correct. Notice what is contested: whether the data were right, and whether the test is the right test.

This is a general pattern in the subject. Tests with data gathered outside the proponents' group have failed. Proponents reply that the data were flawed, then produce a new list or test that succeeds. That reply may be right. It is also the move a mistaken theory would need to make. The way to separate the two is a test whose list, rules and judging are all fixed in advance by people who agree on them, with the result accepted either way. The committee tried to do that, and the dissent shows how hard it is.

If you want to work in this field, that is the standard to aim for. Our Lock and search button is a small version of it for one person.