ACIP To Tibetan Converters

This document describes the ACIP->Tibetan converters built atop Jskad.  These converters were initially written by David Chandler, a volunteer with the Tibetan and Himalayan Digital Library, in the latter half of 2003.  They built upon the work of Tony Duff, Edward Garrett, and Than Garson, and they would not be possible without the assistance of David Chapman, Robert Chilton, and Andrés Montano Pellegrini.  (Please correct, and forgive, any omissions from these lists.)

These converters accept Asian Classics Input Project (ACIP) transliteration of Tibetan (using ACIP's Tibetan Input Code), a Roman transliteration scheme.  ACIP has many Buddhist texts available in ACIP transliteration, which alone makes ACIP transliteration (or just ACIP for short) important.

The converters here accept a text file of ACIP and output either a Unicode UTF-8-encoded text file or a Rich Text Format (RTF) file of Tibetan Machine Web (TMW).  The latter is ready to use onscreen and to make beautiful hardcopy today; the former will be understood by software for a long time to come.

The converters are meant to produce perfect results even for imperfect input.  To give you an idea of the thought and care that went into these converters, consider the following partial list of features:

The ACIP->Unicode and ACIP->TMW converters are equally good.  There are some differences between the two, though.  The TMW font has only a fixed set of glyphs, whereas Unicode can encode arbitrary Tibetan glyphs.  Thus, the hypothetical ACIP {GAI}, which parses as {G+AI} due to prefix rules, will give an error in an ACIP->TMW conversion because no glyph exists for this stack.  The ACIP->Unicode conversion will succeed, having generated correct Unicode.  This is the only difference between the two conversions.

The converters are actively maintained; your feedback is valued.

Note that there are also TMW->ACIP converters available; this document does not cover them.

In what follows, you will learn how to use the converters, including all the features listed above, and you'll find a list of known bugs and places where there is room for improvement.

Using the Converters

This section briefly describes how the converters are best used.

The GUI and command-line interfaces are both sufficient; the GUI interface is your best bet if you've not used the converters before.  To learn how to invoke these interfaces, read these instructions.

First, review the known bugs and be sure you can live with them.

Now perform a trial conversion of your document with warnings disabled.  You will first ensure that no outright errors appear in the input.  If any do, make a copy of the input, edit the input, and feed it through again.  Feel free to try this out as soon as you're comfortable; the error messages themselves are sometimes self-explanatory.

Once all errors have been corrected, do a conversion with warning level 'Some'.  If any warnings mark real problems, correct those problems.

If you have the patience, now do a conversion with warning level 'Most' and correct further problems.  If any warnings mark real problems, correct those problems.

The 'All' warning level is pedantic; you might find it useful if you're writing software that is to produce ACIP transliteration that is easily read by machines.  If you find any useful warnings at this level, report it as a bug -- such warnings should be 'Most' or 'Some' level.

For best results, produce color-coded output.  Scan the output for non-native tsheg bars and ensure that they match the original document (the one from which the ACIP transliteration was produced).  Color-coding is useful because, for example, {ZHIGN} is probably a typo for {ZHING}; {ZHIGN} will appear colored, whereas {ZHING} is not colored.

Note that the ACIP {%} gives a warning every time.  Use the Unicode escape {\u0F35} if you want to avoid this warning, but note well that Unicode escapes are not part of the ACIP standard.  Thus, other tools that work with ACIP transliteration will likely not understand {\u0F35}.

To save time, you may use the tsheg-bar substitution mechanism when appropriate.

Even if your desired end result is Unicode output, an ACIP->TMW conversion is sometimes useful.  One benefit is that errors will appear for any ACIP tsheg bar that refers to a consonant stack not included in TMW.  These stacks should be scrutinized, because TMW contains over 500 of the most common consonant stacks.

Finally, check a few folios by hand against the original document to be sure that you're satisfied with the conversion.

Diagnostics: Warnings and Errors

These converters are designed such that the output is just what you yourself would create by hand.  Whenever there is doubt about what output is desired, a warning or error is issued.  This means that a helpful warning or error message will appear in the output, and that you will be told at the end of the conversion that one or more warnings or errors have indeed occurred.  You can then search your output document for the text [#ERROR or [#WARNING.

There are four warning levels: 'None', 'Some', 'Most', and 'All'.  Choose 'None' if you don't want any warnings to appear in your output and be brought to your attention at the end of conversion.  Choose 'Some' if you want to see the most important warnings, 'Most' if you want some real confidence in your output, and 'All' if you've absolutely got to know that the output is right.

Errors will always appear; you cannot disable them.

The following are some (but not all) error and warning messages, accompanied by further explication:

When warning or error messages refer to a 'Lexical error', that is an error that occurs when breaking an input text up into tsheg bars.  To fully understand all warning and error messages, a thorough understanding of that process and of the interpretation of ACIP tsheg bars is required.

Coloration

For ACIP->TMW conversions (not ACIP->Unicode), color-coding of tsheg bars is an option.  The command-line converters accept a flag --colors yes|no; the conversion GUI in Jskad has a checkbox for color-coding.

Warnings and errors appear in red; tsheg bars that would parse differently if other prefix rules were used appear in yellow; non-native tsheg bars appear in green.

Tsheg-bar Statistics

The ACIP->Tibetan converters provide a simple-minded accounting mechanism with which one can determine which tsheg bars appear in a conversion or how many times each tsheg bar appears.  This mechanism is for power users only at this point; its user interface leaves much to be desired.  If you wish to produce frequency information, and if you are not familiar with some sort of scripting (via Excel macros, Unix shell scripts, etc.), then the output produced will likely be useless to you.

To support the calculation of frequency statistics, that is, how many times each tsheg bar appears, the converter can output all tsheg bars to the Java error console (i.e., System.err).  Each will appear on the console as many times as it appears in the input.  To activate this functionality, set the system property org.thdl.tib.text.ttt.OutputAllTshegBars to true, and be prepared for voluminous output.  Massaging this output into a friendly tabular format is quite possible but not described here; contact the developers for help.

To support the generation of syllabaries, the converter can output each tsheg bar encountered to the Java error console (i.e., System.err).  Each will appear on the console only once, no matter how many times it appears in the input.  To activate this functionality, set the system property org.thdl.tib.text.ttt.OutputUniqueTshegBars to true, and be prepared for voluminous output.

If desired, each tsheg bar output can be prefixed with a string of your choice by setting the system property org.thdl.tib.text.ttt.PrefixForOutputTshegBars to that string.  This is useful if the converter is producing other output on the console and you want to separate that output from the statistics.

Tsheg-bar Substitution

The ACIP->Tibetan converters provide a mechanism for automatically correcting common transliteration typos.  For example, if your document contains 100 occurrences of {KAsh} that all in fact intend {K+sh}, then you can specify just once the rule {KAsh}->{K+sh}, and all 100 occurrences will be treated correctly.  This mechanism is not very easy to use, but it is completely customizable; you can specify any number of rules.  You can only perform such substitutions at the tsheg bar level, though.  This means, for example, that you cannot specify the rule {GONG SA}->{^GONG SA}; you can only specify {GONG}->{^GONG}, which would affect {GONG LA} just as it would affect {GONG SA}.

To perform substitutions, set the system property org.thdl.tib.text.ttt.ReplacementMap to be a comma-delimited list of x=>y pairs.  For example, if you think BLKU, which parses as B+L+KU, should parse as B-L+KU, and you want KAsh to be parsed as K+sh because the input operators mistyped it, then set org.thdl.tib.text.ttt.ReplacementMap to BLKU=>B-L+KU,KAsh=>K+sh.  Note that this will not cause {B+L+KU} to become {B-L+KU} -- we are doing the replacement during lexical analysis of the input file, not during parsing.  And it will cause {SBLKU} to become {SB-L+KU}, which is parsed as {S+B-L+KU}, probably not what you wanted.  If you fear such things, you can see if they happen by setting the system property org.thdl.tib.text.ttt.VerboseReplacementMap to true, which will cause an informational message to be printed on the Java console every time a replacement is made.

Furthermore, you can use the regular expression notations ^ and $ to denote the beginning and end of the tsheg bar, respectively.  For example, ^BLKU$=>B-L+KU is a useful rule.  Note that full regular expressions are not supported -- the tool just borrows a bit of the notation.  The rule ^BLKU=>B-L+KU means that {BLKUM} and {BLKU} will both be replaced, but {SBLKU} and {SBLKUM} will not be.  The caret, ^, means that we only match if BLKU is at the beginning.  The dollar sign, $, means that we only match if the pattern is at the end.  The rule BLKU$=>B-L+KU will cause {SBLKU} to be replaced, but not {BLKUM}.  Note that performance is far better for ^FOO$ than for ^FOO, FOO$, or FOO alone.

Only one substitution is made per tsheg bar.  ^FOO$-style mappings will be tried first, then ^FOO-style, then FOO$-style, and finally FOO-style.

An example of a useful substitution is o$=>\u0F35.  This is useful because the converters interpret the ACIP {o} as U+0F37 by default, but you might prefer U+0F35 in your output.

Note that you cannot literally replace {FOO} with {BAR} using this mechanism -- because {F} is not an ACIP character, the lex will not get far enough to use this substitution mechanism.  This is not considered a design flaw -- serious errors require user intervention.  Sophisticated users can use something akin to perl, sed, or awk scripts to preprocess the input.

Note also that you cannot use the rule ONYA=>O&, although it would be nice if you could.  Technically, {&} is considered to be punctuation (i.e., that which divides tsheg bars) and is not understood inside a tsheg bar.

Note that this mechanism is also useful for fixing problems in the converter itself rather than in the input.

Unicode Character Escapes

The ACIP->Tibetan converters support some non-standard extensions to the ACIP Tibetan Input Code Standard.  One of those is Unicode character escape sequences.  This extension makes it possible to represent characters that the ACIP standard does not address, and to represent one character, U+0F84, that ACIP does address with the transliteration {\} but that is misused in practice so often to refer to U+0F3C that the ACIP->Tibetan converters always produce an error upon seeing {\}.

Outside of comments, {\uKLMN} is interpreted as referring to the Unicode character with ordinal KLMN, where each of K, L, M, and N are case-insensitive hexadecimal digits.  For example, the ACIP {KA KHA GA NGA } is exactly equivalent to {\u0F40\u0f0B\u0F41\u0F0B\u0F42\u0F0B\u0F44\u0f0b}.  Unicode escapes produce the obvious Unicode in an ACIP->Unicode conversion, and they produce the correct TMW glyph in an ACIP->TMW conversion.  There are limits, though, when converting to TMW; multiple escapes in sequence are not handled correctly.  It would take a Unicode to TMW converter to produce the correct glyphs for {\u0F42\u0F92\u0FB7\u0F7C}.  The escapes for vowels and other characters that are mapped to multiple TMW glyphs are also not handled perfectly.  Best practice is to use escapes only when necessary in an ACIP->TMW conversion.

The Unicode character represented need not be a Tibetan one; for example, {\u0040} produces the at sign, @.

The latest Extended Wylie Transliteration Scheme standard has assigned private-use area (PUA) Unicode codepoints to some TMW glyphs.  ACIP documents that have a Unicode escape in the range U+F021 to U+F0FF, inclusive, are interpreted as intending these TMW glyphs.  ACIP->Unicode produces an error for such an escape because it is font-dependent and not standard.  Other tools will likely not understand such Unicode, so the converter will not produce it.  If you want it in the output, it is there in the error message.

Note well the known bug with regard to whitespace in transliteration that follows a Unicode escape.  In large part, this bug affects characters that can be transliterated by other, simpler, standard means.

If you do want to disable the use of Unicode escapes, set the system property thdl.tib.text.disallow.unicode.character.escapes.in.acip to true.

Breaking a Text Up Into tsheg bars

The ACIP->Tibetan converters all take ACIP transliteration as input.  The first step in conversion is to break up the input into manageable pieces.  (This is known as lexical analysis in the context of programming languages, and you may see the term in diagnostic messages though a linguist who studies human language like Tibetan might balk at the term.)  The correct pieces in this case are tsheg bars (in ACIP, {TSEG BAR}), punctuation, comments, whitespace, folio markers, formatting codes, etc.  In this section, the intracacies of how the converter does that will be laid bare.  With luck, this will help you understand why the converter treated one space character (i.e, ' ', U+0020) as a tsheg and another as Tibetan whitespace.

The Tibetan term tsheg bar refers to "the stuff between the dots".  In the ACIP {BKRA SHIS [# Notice that this comment is embedded in the Tibetan greeting pronounced 'tashi delay']BDE LEGS,}, there are four tsheg bars, 'BKRA', 'SHIS', 'BDE', and 'LEGS'.  In this case 'BDE' is literally "between the dots"; i.e., it is sandwiched by two U+0F0B characters (because comments are in a sense invisible).  One of the "dots" that touches 'LEGS' does not look like a dot -- it is a shad, U+0F0D.  The lexical analyzer also finds one comment, which will appear in a Latin typeface in the output, and it finds four pieces of punctuation -- three tshegs and a shad.

The converter will not allow an illegal character into a tsheg bar.  For example, {jA} is an error and causes an error message to appear in the output.

Now that the basic operation is clear from the above example, let's cover the fine points of how standard ACIP is handled.  We'll also cover some non-standard constructs that appear commonly in actual ACIP Release V texts.

The first construct that deserves explanation is the line break.  By the ACIP standard, line breaks in the input do not become line breaks in the output unless there are two line breaks in the input.  For example, the ACIP snippet below has only one line break in the output although three line breaks appear in the input: 

BKRA SHIS 
BDE LEGS,

THUGS RJE CHE ... and so on ...

One fine point is that the converter does not require a space before a line break. If {SHIS} appears before a line break, the converter inserts a space so that it's treated just like {SHIS } is treated.  This oddity is needed to convert real ACIP documents.

Another fine point is that ACIP's {^} character "eats" a following space or a newline.  This is so that {^ GONG SA } is treated identically to {^GONG SA }.

Comments appear in a Latin typeface always.  Comments are not allowed just anywhere -- a comment cannot occur within a single tsheg bar, for example, and it cannot appear between a tsheg bar and the tsheg that follows it.  That is, {BD[#COMMENT]E} is not like {BDE}, and {BDE[#COMMENT] LEGS} is not like {BDE LEGS} (though {BDE [#COMMENT]LEGS} is).

Corrections are interpreted as Tibetan, not English, by default, but there is a built-in list of corrections that should appear in the output in a Latin typeface.  (Actually, any correction that starts with a certain string will appear in a Latin typeface.)  The full list is the following: 

"LINE"                // from KD0001I1.ACT
"DATA"                // from KL0009I2.INC
"BLANK"               // from KL0009I2.INC
"NOTE"                // from R0001F.ACM
"alternate"           // from R0018F.ACE
"02101-02150 missing" // from R1003A3.INC
"51501-51550 missing" // from R1003A52.ACT
"BRTAGS ETC"          // from S0002N.ACT
"TSAN, ETC"           // from S0015N.ACT
"SNYOMS, THROUGHOUT"  // from S0016N.ACT
"KYIS ETC"            // from S0019N.ACT
"MISSING"             // from S0455M.ACT
"this"                // from S6850I1B.ALT
"THIS"                // from S0057M.ACT

Somewhat related is the converter's treatment of a few oddball comments.  The oddity is that these comments use the syntax {[COMMENT]} rather than the standard syntax {[#COMMENT]}.  The converter will treat the following as comments: 

From S5274I.ACT:
"[FIRST]"
From S5274I.ACT:
"[SECOND]"
From S0216M.ACT:
"[Additional verses added by Khen Rinpoche here are]"
From S0216M.ACT:
"[ADDENDUM: The text of]"
From S0216M.ACT:
"[END OF ADDENDUM]"
From S0216M.ACT:
"[Some of the verses added here by Khen Rinpoche include:]"
From S0216M.ACT (note the typo):
"[Note that, in the second verse, the {YUL LJONG} was orignally {GANG LJONG},
and is now recited this way since the ceremony is not only taking place in Tibet.]"
From S6954E1.ACT:
"[text missing]"
From TD3817I.INC:
"[INCOMPLETE]"
From S0935m.act:
"[MISSING PAGE]"
From S0975I.INC:
"[MISSING FOLIO]"
From S0839D1I.INC:
"[UNCLEAR LINE]"
From SE6260A.INC:
"[THE FOLLOWING TEXT HAS INCOMPLETE SECTIONS, WHICH ARE ON ORDER]"
From SE6260A.INC:
"[@DATA INCOMPLETE HERE]"
From SE6260A.INC:
"[@DATA MISSING HERE]"
From TD4035I.INC:
"[LINE APPARENTLY MISSING THIS PAGE]"
From TD4226I2.INC:
"[DATA INCOMPLETE HERE]"
To be consistent with the above:
"[DATA MISSING HERE]"
From S0018N.ACT:
"[FOLLOWING SECTION WAS NOT AVAILABLE WHEN THIS EDITION WAS
PRINTED, AND IS SUPPLIED FROM ANOTHER, PROBABLY THE ORIGINAL:]"
From S0018N.ACT:
"[THESE PAGE NUMBERS RESERVED IN THIS EDITION FOR PAGES
MISSING FROM ORIGINAL ON WHICH IT WAS BASED]"
From S0018N.ACT:
"[PAGE NUMBERS RESERVED FROM THIS EDITION FOR MISSING
SECTION SUPPLIED BY PRECEDING]"
From S0057M.ACT:
"[SW: OK]"
From S0057M.ACT:
"[m:ok]"
From S0057M.ACT:
"[A FIRST ONE
MISSING HERE?]"
From S0195A1.INC:
"[THE INITIAL PART OF THIS TEXT WAS INPUT BY THE SERA MEY LIBRARY IN
TIBETAN FONT AND NEEDS TO BE REDONE BY DOUBLE INPUT]"

The converter also supports several non-standard folio markers.  A review of ACIP Release V texts determined that the following types of folio markers can appear: 

@001
@001A
@001B
@01A.3
@012A.3
@[07B]
@00007B
@00007
@B00007
@[00007A]

Similarly, to support real ACIP Release V texts, the converter treats {[DD1]}, {[DD2]}, {[ DD ]}, and {[DDD]} just like {[DD]} (which is specified in the ACIP standard).  It treats {[ BP ]} and {[BLANK PAGE]} just like {[BP]}, also.

The lists above were created by a most fallible process of reviewing a large number of ACIP Release V texts.  Your suggestions for additions to these lists are highly valued; please contact the developers.

The converters will insert a tsheg in some places where no ACIP { } appears; this happens after {PA} and {DANG,} below: 

GA PA

GA PHA 

DAM,
LHAG

GA CA,

GA

Note that a space appears after {PHA}, and a comma appears after {CA}, but {PA} has nothing between it and a line break.  The converters are smart enough to insert a tsheg regardless.

Also missing from the above ACIP, but inserted automatically by the converters, is Tibetan whitespace; the converter sees {DAM, LHAG} instead of {DAM,LHAG} above.

If such automatic corrections are not desired, try using a Unicode escape before the line break instead of {PA} or {,}.

The converters also treat {NGA,} as a typo for {NGA ,} (actually, {NGA\u0F0C,} since one wouldn't want a line break to occur after the tsheg and cause a shad to begin a line; see the section on formatting Tibetan texts in the Tibetan! 5.1 documentation) because Tibetan typesetting requires that NGA not appear directly before a shad.  (Perhaps {NGA,} would look too much like {KA}.)

The converters embody the rule that a shad does not appear after GA or KA unless a shabs kyu vowel is on the GA or KA.  For example, the space in {MA ,HA} is a tsheg, and the space in {KU ,HA} is a tsheg, but the space in {GA ,HA} is Tibetan whitespace.

If you find that the converters put a tsheg where it does not belong, miss a tsheg, or put whitespace where it does belong, please contact the developers.

Though the ACIP standard does not mention it, it appears that some ACIP Release V texts use a period (i.e., {.}) to indicate a non-breaking tsheg (i.e., U+0F0C).  Search for {NGO.,}, {....,DAM}, etc.  Unless {,}, {.}, or a letter (i.e., a through z) follows the {.}, it is only grudingly interpreted as a non-breaking tsheg -- a warning is generated, too.  FIXME: Is this right?  Allow for treating {.} as an outright error.

Note that the treatment of the very last line in an input text is circumspect.

Parsing tsheg bars: Greedy Stacking and Nativeness

This section is a technical reference sufficiently detailed so that you can fully understand the inner workings of the converter as it decides which Unicode or TMW to use for a given tsheg bar.  The problem of breaking up a text into tsheg bars is a separate issue; this section describes what happens to a tsheg bar after it's been chipped away from the text.

The ACIP->Tibetan converters have a notion of nativeness.  Each tsheg bar is either native Tibetan or non-native.  For example, in Buddhist texts written in Tibetan, Sanskrit mantras often appear in Tibetan characters.  This "Tibetanized Sanskrit" is non-native.  The tsheg bars that make up this mantra (and here, take "tsheg bar" somewhat literally to mean the characters delimited by punctuation and whitespace) are some native and some non-native in the converter's eyes.  For example, the tsheg bar {MA } appears in some mantras, and is thus in fact non-native.  The converter, however, treats {MA } as native in all contexts.  Thus, "native" is a technical term with a slightly different meaning than usual.

The idea of nativeness is important because it affects how the converter treats a tsheg bar.  In ACIP transliteration, the rule is that consonants stack up until punctuation, whitespace, or a vowel appears.  For example, {RDZYA} is equivalent to {R+DZ+YA}.  ({DZA} always means the letter {DZA} itself, never {D+ZA}.)  But this greedy stacking does not apply to {SOGS}, which is equivalent to {SOG-S}, not {SOG+S}.  Why not?  Because {SOGS} is a native tsheg bar where GA is the suffix and SA is the postsuffix.  Similarly, {GNAD} is {G-NAD}, not {G+NAD}.  Why?  Because GA is a prefix in this native Tibetan tsheg bar.

In this section, we will illustrate the inner workings of this aspect of the converter.  You will be able to determine which snippets of transliteration the converter considers to be native tsheg bars, where greedy stacking does not apply except for the root stack, and which snippets are non-native, and thus wholly subject to greedy stacking.

Anatomy of a Native tsheg bar

First, the lexical analyzer ensures that only the Tibetan and Sanskrit consonants, the vowels {A}, {I}, {U}, {E}, {O}, {OO}, {EE}, {i}, {'A}, {'I}, {'U}, {'E}, {'O}, {'OO}, {'EE}, and {'i}, and the adornments {m} and {:} are allowed in a tsheg bar.

As far as the converter is concerned, a native tsheg bar consists of an optional prefix, a native root stack, an optional suffix, an optional postsuffix (also known as a secondary suffix) that may only be present if a suffix is present, and zero or more appendages (my term, created because I don't know what a grammarian calls such a thing).  An appendage is one of the following stack sequences:

A tsheg bar is non-native if it has a non-native root stack or if it contains the {:} character.  Any vowel is allowed on a native root stack, even {'EEm}, {i}, or the like.

The rule about native root stacks is important, for example, in determining that {KTYAMS} is {K+T+YAM+SA} instead of {K+T+YAMASA} (because K+T+YA is not a native stack).  Another example is {GNVA}, which is treated like {G+N+VA}, not {G-N+VA}, even though {GNA} is treated like {G-NA} because NA can take a GA prefix.  The complete list of native stacks is the following:

(Some would argue that LVA is notably absent.  It is seen in ACIP Buddhist texts in {AELVA}, {LVAm}, {LVU}, {LVUN}, {LVAR}, {LVE}, {LVANG}, and {LVA}.  Greedy stacking affects none of these tsheg bars' parsing, however.)

Not all characters can be prefixes and the like.  Only the five prefixes (GA, DA, BA, MA, 'A), ten suffixes (GA, NGA, DA, NA, BA, MA, 'A, RA, LA, SA), and two postsuffixes (DA, SA) every Tibetan student knows are allowed, and they cannot appear with vowels.  (In {LE'U}, {'} is not a suffix -- it is part of an appendage.)  In fact, certain prefixes may only appear with certain root stacks.  The reason that these prefix rules matter is that they govern how tsheg bars are parsed.  For example, {GNA} is parsed like {G-NA}, because NA takes a GA prefix.  But {GPA} is parsed like {G+PA}, because PA does not take a GA prefix.

Prefix rules are a topic of some controversy; different grammars give different lists of prefix rules.  For a converter, it is important that the converter's knowledge of prefix rules matches the knowledge of the person who typed in the ACIP transliteration, not that the converter agrees with a grammarian.  For example, if the input technician thought that PA could take a GA prefix, then the converter will produce {G+PA} when {G-PA} was intended.  For this reason, the converter can produce a warning every time a prefix rule prohibited the treatment of one of the five prefixes as a prefix.  For example, {GPA} produces this warning.  However, {GNA} produces no warning, because the converter assumes that it is unlikely that an input technician would enter {GNA} upon seeing {G+NA}.  Part of the reason for this assumption is that the Asian Classics Input Project Entry Operator Transcription Chart as of Spring, 1993, explicitly enumerates the following cases for special treatment by input operators:

Regardless, for best results, you should ensure that the input technician's knowledge of prefix rules matches the converter's knowledge.  The following are the legal combinations of prefix and root stack in the converter's eyes:

In the above list, the presence of wa-zur (ACIP {V}) does not disallow a prefix-root combination; nor does the presence of any vowel, even {'EEm}.  The presence of {:} does disallow prefix-root combinations; e.g., {GN'EEm} is {G-N'EEm}, but {GNA:} is {G+NA:}.  ({GNVA} is parsed as {G+N+VA} not because NVA cannot take a GA prefix, but because NVA is not a native stack.)

The converter will allow any suffix to go with any native root or prefix-root combination; it will allow any postsuffix to follow any suffix.  It will allow any appendage on any native tsheg bar.

For example, {SOGS}, {BSOGS}, {BS'EEmGS}, {LE'U'I'O} and {BSKYABS-'UR-'UNG-'O} are all native tsheg bars in the converter's eyes.  Note the need for disambiguation: {PAM-'AM} is a native tsheg bar, but {PAM'AM}, which parses as the three stacks {PA}, {M'A}, and {MA}, is not.  (In practice, appendages rarely occur after prefixes.  {BUR-'ANG} appears at least once in ACIP files and {DGA'-'AM} appears at least twice, but these may be typos.  The converter does allow it, though.  It thinks {BIR'U} and {WAN'U} (which also occur, but only very rarely) are both non-native, though, and thus treats {'} as U+0F71 (subscribed) and not U+0F60 (full form) in each case.)

Note a fine point.  When turning a tsheg bar into Tibetan, the ACIP->Tibetan converters assume that subjoined YA and RA consonants are not fixed-form -- not U+0FBB and U+0FBC -- but rather are the usual subjoined forms U+0FB1 and U+0FB2.  The only exceptions are the stacks R+Y, Y+Y, and n+d+Y, which are known to have fixed-form subjoined YA, and the stacks n+d+R+Y (where RA but not YA is full-form) and K+sh+R, which are known to have fixed-form subjoined RA.  Wa-zur, U+0FAD, is never confused with full-form subjoined WA, U+0FBA, though, because ACIP represents the former with {V} and the latter with {W}.  Furthermore, the converter never generates U+0F6A, the fixed-form RA (rango); U+0F62 is always produced.  (Note that U+0F62 is often displayed as a fixed-form RA itself, as in {RNYA}.)

So far, we have spoken about consonants and vowels.  In fact, it is not trivial to determine when something is a consonant and when it is a vowel.  {A} can represent U+0F68, the Tibetan letter, or the implicit vowel.  {'} can represent U+0F71, the subscribed a-chung, or U+0F60, the full-sized consonant a-chung.  The converter treats {TAA} as {T+AA}, not {TA-AA}, but treats {TAAA} like {TA-AA}, not {T+AA-A}.  It treats {PA'AM} like {PA-'A-M}, not {P+A'A-M}.  In short, it first tries out treating {'} and {A} like vowels, but will backtrack if that leads to a clearly invalid tsheg bar.

Finally, a string of numbers can be a tsheg bar also.  It is illegal for numbers and consonants to appear together within one tsheg bar, however.

The above is the complete understanding of the converter's algorithms for parsing tsheg bars.  You the native Tibetan speaker may know that {BSKYABS-'UR-'UNG-'O} is not allowed and thus think that {B+S+K+YAB+S-'UR-'UNG-'O} should be the result, but the converter has no such knowledge, and thinks this is a native tsheg bar equivalent to {B-S+K+YAB-S-'UR-'UNG-'O}.

System Properties

The tsheg-bar substitution mechanism is customizable via system properties.  Java developers likely know what these are, but few users do.  This section will perhaps get a determined person started, but if you have trouble, contact the developers so that we can improve this documentation or create a better user interface.

For the tool to respect the value of a system property, you must invoke the tool from the command line as follows:

java "-Dorg.thdl.tib.text.ttt.ReplacementMap=KAsh=>K+sh,ONYA=>[#ERROR-ONYA-IS-O&]" -Dorg.thdl.tib.text.ttt.VerboseReplacementMap=true -jar Jskad.jar

Known Bugs

This section presents areas where the current tool's behavior is wrong.  Before doing serious work with the converter, familiarize yourself with this section and develop a plan to work around the bugs or to ensure that your documents will not trigger the bugs.  At the same time, if any of these bugs affects you, contact the developers so that we can fix them.  The squeaky wheel surely gets the grease; these bugs may never be fixed if there are no complaints.

The following are all known bugs:

Room for Improvement

This section presents areas where the current tool could be improved.  None of the current behavior described here is incontrovertibly flawed (i.e., there are no bugs described here, see known bugs for that); current behavior is technically correct.  However, the current behavior is not, in everyone's eyes, perfect.

The following are the current areas in which the tool could be better:

License

Both the ACIP->Tibetan converters and this document are released under the THDL Open Community License Version 1.0.

Please e-mail us your comments about this page.

The THDL Tools project is generously hosted by: SourceForge Logo