A Spectre is Haunting Unicode
A Spectre is Haunting Unicode
This post is part of collections on Code and Computers and Japanese Language Technology.
In 1978 Japan's Ministry of Economy, Trade and<br>Industry<br>established the encoding that would later be known as JIS X 0208, which still<br>serves as an important reference for all Japanese encodings. However, after the<br>JIS standard was released people noticed something strange - several of the<br>added characters had no obvious sources, and nobody could tell what they meant<br>or how they should be pronounced. Nobody was sure where they came from. These<br>are what came to be known as the ghost characters<br>(幽霊文字).
Be careful what you write. via the NDL
For a long time the ghost characters remained an unexplained and mostly<br>forgotten curiosity, but in 1997 an investigation was launched to discover<br>where they had come from. While all characters in the JIS standard were<br>supposed to have a record of their sources, even when it existed it<br>wasn't very specific, typically just listing the document it was sourced from.
You'd think that listing the source would make tracking down the origins of the<br>characters easy, but it's important to clarify what counts as a "source" - one<br>of the more common sources for the ghost characters was the "Overview of<br>National Administrative Districts" (国土行政区画総覧), a comprehensive list of<br>place names in Japan. You might, as I initially did, imagine this to be a kind<br>of atlas, an oversize book with at most a few hundred pages. It turns out the<br>latest edition is a seven volume set<br>with each volume having roughly nine hundred pages. Imagine tracking down a<br>single character without a page reference.
Despite the difficulty, the investigation into the ghost characters was<br>successful in discovering their origins - mostly. By interviewing the<br>catalogers involved in the creation of the standard, the investigators<br>established that some characters were inadvertently invented as mistakes in the<br>cataloging process. For example, 妛 was an error introduced while trying to<br>record "山 over 女". "山 over 女" occurs in the name of a particular place and<br>was thus suitable for inclusion in the JIS standard, but because they couldn't<br>print it as one character yet, 山 and 女 were printed separately, cut out, and<br>pasted onto a sheet of paper, and then copied. When reading the copy, the line<br>where the two little pieces of paper met looked like a stroke and was added to<br>the character by mistake. The original character<br>(𡚴) was not added to JIS or<br>Unicode until much later and doesn't display on most sites for me.
The core ghost characters: 妛挧暃椦槞蟐袮閠駲墸壥彁
In the end only one character had neither a clear source nor any historical<br>precedent: 彁. The most likely explanation is that it was created as a<br>misreading of the 彊 character, but no specific incident was uncovered.
Following the general adoption of the JIS standards these characters all made<br>their way into Unicode, which has its own separate set of ghost<br>characters<br>introduced during CJK unification.
To sum up - in 1978 a series of small mistakes created some characters out of<br>nothing. The errors went undiscovered just long enough to be set in stone, and<br>now these ghosts are, at least in potential, a part of every computer on the<br>planet, lurking in the dark corners of character tables.
At this rate they'll presumably be with humanity forever. Ψ
References / related links:
幽霊文字 ‐ 通信用語の基礎知識 - the most thorough online source, with citations from the 1997 investigation.
大正十二年の幽霊文字 - ことばマガジン：朝日新聞デジタル - an example of 彁 mistakenly used in a digitized Taisho newspaper due to a faded printing of 彊.
Nico Nico Douga's Wiki treats each of them as the name of a youkai.
天书 or A Book from the Sky, a hand-printed book by Xu Bing using only made-up Chinese characters.
2018-07-29T14:03:09+09:00