How Torah Codex measures

What a result is

Write out a text with no spaces. Pick a starting letter and a number. Read every letter that many places apart. If the letters you read spell a word, you have an equidistant letter sequence, or ELS. The number is the skip. Torah codes is the common name for these sequences in the Hebrew text.

A sequence is fully described by three things: where it starts, its skip, and its length. A negative skip reads backward through the text. That is all a result is. It is a position and a distance, and anyone with the same text can check it by hand.

Torah Codex measures. It finds the sequences, shows where every letter stands, and prints what chance alone would produce beside every count. It does not say what any sequence means.

The text we search

We search the Westminster Leningrad Codex, as published by the Open Scriptures Hebrew Bible project. It is a digital transcription of the Leningrad Codex, the oldest complete manuscript of the Hebrew Bible. We search the 24 books of Tanakh in the traditional order and nothing else.

• The Torah has 304,850 letters in this text. The whole Tanakh has 1,197,042.
• We read the ketiv, the letters as written. Where the tradition reads a word differently from how it is written, the written letters are what we count. There are 1,254 such places in the Tanakh, and a verse card shows you the reading.
• Final letters count as their regular forms: ך as כ, ם as מ, ן as נ, ף as פ, ץ as צ. Your search term is folded the same way.
• Spaces, vowel points, cantillation marks and punctuation are not letters. They are removed before counting.

You will often see 304,805 given as the number of letters in the Torah. That is the count of the Koren edition, the one Witztum, Rips and Rosenberg used in 1994. Our text has 45 more. Editions differ here and there in full and defective spellings, where a vowel letter such as ו or י is written or left out. Genesis has 78,069 letters in our text and 78,064 in Koren. McKay and his co-authors counted 22 letter differences between the two in Genesis alone.

We do not call either count the correct one. The Talmud itself says, in Kiddushin 30a, that later generations were no longer expert in full and defective spellings. What matters for you is this: one letter added or removed inside a sequence's span moves everything after it, so the sequence is gone. Every result on this site is a statement about this one text.

The chance figure

Every count stands beside an expected count: how many times the same term would appear if the letters of the same text were in no meaningful order. Here is the arithmetic, using the most famous example. In Genesis, how many times do we expect תורה, forward, at skip 50?

1. Count the places it could start. A four-letter word at skip 50 spans 150 letters past its first letter, so it fits at 78,069 minus 150 places: 77,919.
2. Take each letter's share of Genesis. ת is 5.318% of the letters, ו is 10.820%, ר is 6.139% and ה is 8.048%.
3. Multiply the four shares. That gives 0.00002843, the chance that any one placement spells the word.
4. Multiply by the number of places: 77,919 × 0.00002843 = 2.215.

So chance alone makes about 2.2 sequences. The real count is 3. From the expected count we also get a probability, using the Poisson distribution: if chance alone expects 2.215, the odds of seeing 3 or more are about 0.38. That is an ordinary result. It happens more than one time in three.

p(term) = p(letter 1) × p(letter 2) × ... × p(letter k)
places  = sum over d from a to b of (N − (k − 1) × d)
E       = p(term) × places × directions
P(X ≥ n) = the Poisson tail with mean E

Letter shares come from the scope you searched, so a search of Leviticus uses Leviticus's letters. A palindrome counts once, since forward and backward are the same sequence.

This figure has a known limit. It treats letters as independent, and Hebrew text is not. Letters next to each other depend on each other, and that matters most at skips 1 to 3. So we never leave the formula alone. It stands beside shuffled texts and, where you ask, another book.

Why short words are everywhere

The number of possible sequences in a long text is enormous. In the Torah there are about 3 billion places a four-letter sequence could stand at skips of 2 to 5,000, forward and backward. Most four-letter words made of common letters turn up tens of thousands of times. Longer words fall off fast, because each extra letter multiplies the chance by a small share.

Read that table slowly. The counts sit close to chance almost everywhere. That is the normal state of affairs, and it is the first thing a new reader should see. Finding a word tells you very little. The interesting question is always how the count compares with chance, and how many things you tried before you got here.

Shuffled texts

The formula is one baseline. A shuffled text is a second one, and it needs no theory. We take the same letters, mix them with a numbered seed, and run the same search again. Do that many times and you can see what the search does on text with no order in it. Genesis has 3 forward sequences of תורה at skip 50. In 1,000 letter shuffles (seeds 1 to 1,000), 404 had 3 or more.

Letter shuffles are only one kind of control. Each kind keeps something and destroys something, and a fair test usually needs more than one. The controls page goes through all of them.

The trial counter

Here is the mistake almost everyone makes, including experts. You search a word. Nothing stands out. You try another spelling, a different skip range, a different book. Then you get a good-looking result and you report that one. The figure beside it was worked out for a single try, but you made twenty.

Statisticians call this the look-elsewhere effect. Ten yes-or-no choices make 1,024 possible searches. If you tried them all, a result that looks like one in a thousand is what you should expect to find.

So the laboratory keeps count for you. Each new term, skip range, place searched and matrix width is logged in your session, and the count sits beside every figure. It says it plainly: "You have tried 4 spellings and 3 skip ranges. The best of 12 looks about 12× better than one try." It is a fact about your session. It is never a scolding.

Locking a search first

The fairest test is one where you commit before you look. Pre-register lets you do that. You fix the terms, spellings, scope, skip range and measure, and lock them. The laboratory runs exactly that search, once, and the share link and export carry a badge that says so.

Persi Diaconis, then a statistician at Harvard, asked the authors of the rabbis experiment for much the same thing: a fresh sample and a test set in advance. A locked search cannot be improved after the fact, and that is what makes its number worth something.

Minimal skip and the domain of minimality

A short word has thousands of sequences, so which ones count? Witztum, Rips and Rosenberg gave weight to sequences with the smallest skip over a stretch of text. A sequence's domain of minimality is the longest stretch of text that contains it and holds no other sequence of the same word with a smaller skip. When you highlight a minimal sequence, the laboratory shows that stretch and how often shuffled texts match it.

Every word that appears at all has a smallest skip somewhere, so "minimal" alone says nothing. The size of the domain is the figure to look at.

Pairs and sets of words

Two or more terms bring a new question: are they closer together than they should be? There is no formula for that, so the answer has to come from simulation. The laboratory offers two published families of measure. The WRR proximity c(w,w′) is explained on its own page. The Haralick table measures take the smallest rectangle that holds all your terms in a matrix and score its size.

Each score is ranked among control runs and shown as a count, such as "shorter in 610 of 1,000 shuffled texts." The control family is always named. A rank from 1,000 runs cannot be finer than about 1 in 1,000, and the laboratory says so.

What we leave out

• No "odds of 1 in N" for a word you chose. Multiplying letter chances that way answers the wrong question, because it ignores all the words and skips you could have picked.
• No single-number code score. Heuristic scores used in older software mix several things and hide the comparison. We show the count and the baseline.
• No AI interpretation of a matrix. Nothing here interprets.
• No current events, elections or living people in our gallery.

Every result can be repeated. A share link holds the whole search, and each export carries a verification line with the stream version and a short hash of the search. If you think we got a number wrong, you have all you need to show it.