Commit Graph

27 Commits

Author SHA1 Message Date
dchandler 5788416629 A reverter that converts Unicode to computer-friendly (but not, yet,
human-friendly) EWTS is here in alpha mode.  It probably doesn't deal
well with non-Tibetan.
2005-08-01 05:54:20 +00:00
dchandler 00afd75362 The scaffolding for a Unicode->EWTS reverter. No guts yet. 2005-07-17 03:32:57 +00:00
dchandler dc18165992 Added a class for performing EWTS->Unicode conversions during XSLT
transformations.  I haven't actually used it with Xalan XSLT yet, but
it ought to work if TibetanHTML did (which it must have at one point).

I do have a unit test, but an end-to-end test with Xalan is what we
need.
2005-07-13 07:25:18 +00:00
dchandler 0b3a636f63 Tremendously better EWTS->Unicode and EWTS->TMW conversion, though still not tested end-to-end and without perfect unit tests. See EWTSTest.RUN_FAILING_TESTS, for example, to find imperfection. 2005-07-06 02:19:38 +00:00
dchandler 7198f23361 I really hesitate to commit this because I'm not sure what it brings to the
table exactly and I fear that it makes the ACIP->Tibetan converter code
a lot uglier.  The TODO(DLC)[EWTS->Tibetan] comments littered throughout
are part of the ugliness; they point to the ugliness.  If each were addressed,
cleanliness could perhaps be achieved.

I've largely forgotten exactly what this change does, but it attempts to
improve EWTS->Tibetan conversion.  The lexer is probably really, really
primitive.  I concentrate here on converting a single tsheg bar rather than
a whole document.

Eclipse was used during part of my journey here and some imports were
reorganized merely because I could.  :)

(Eclipse was needed when the usual ant build failed to run a new test
EWTSTest.  And I wanted its debugger.)

Next steps: end-to-end EWTS tests should bring many problems to light.  Fix
those.  Triage all the TODO comments.

I don't know that I'll ever really trust the implementation.  The tests are
valuable, though.  A clean implementation of EWTS->Tibetan in Jython
might hold enough interest for me; I'd like to learn Python.
2005-06-20 06:18:00 +00:00
dchandler 37bf9a736d I did this stuff back in August. It's all in support of EWTS->Tibetan
conversion.  The tag 'TODO(DLC)[EWTS->Tibetan]' exists all over the
place.  EWTS->Tibetan isn't here yet; lexing isn't here yet; this is
mainly a refactoring so that the ACIP->Tibetan code can be reused to
do EWTS->Tibetan.

I'm committing this because tests pass (it shouldn't be breaking
anything), because I want a checkpoint, and because the laptop this
sandbox was on isn't my preferred development environment.
2005-02-21 01:16:10 +00:00
dchandler 8dcb623382 TMW->EWTS:
Fixed part of bug 998476 and part of an undocumented bug.  Discovered a
new bug, "aM" should be generated but only "M" is.

The undocumented bug was that laMA was generated when lAM should have been.

The part of bug 998476 that was fixed: laM, laH, etc. are now generated.

This does nothing about paN etc.

Some refactoring here; this is not a minimal diff.

Added tests of TMW->EWTS that use ACIP to get the TMW in place
because EWTS->TMW is a faulty keyboard at present.
2005-02-07 03:17:40 +00:00
dchandler 00961b633f Added a test case for bug 998476. No fix, just a test case verifying the bug. 2005-02-05 18:47:17 +00:00
dchandler 3115f22484 Now you may run a single JUnit test via a command-line like 'ant
check-one -Dsoletest=org.thdl.tib.text.tshegbar.UnicodeUtilsTest'.
This saves you the time of running the slow tests.
2004-12-13 02:29:05 +00:00
dchandler f490978227 Now using Jskad/build.properties file, which I keep looking like this:
halt.after.trouble=no
2004-01-17 16:43:47 +00:00
dchandler fcb0c57792 I'm committing an old edit that may or may not help headless tests --
they seemed fine anyway, but what's the harm?
2003-12-13 21:50:23 +00:00
dchandler 31b3020d07 Added a test case that runs almost all the tsheg bars from all
non-reference, publicly available ACIP files (hundreds of megabytes of
them) through the converter.  The frequencies of these tsheg bars in
in the file, too.
2003-10-26 06:02:48 +00:00
dchandler e21d3774a9 Added an unfinished ACIP->Tibetan converter. Once it works properly
for ACIP, it'll easily be made to work as a perfect EWTS
Wylie->Tibetan converter.  It has an extensive suite of tests for the
existing functionality.
2003-08-10 19:30:07 +00:00
dchandler f95240890c Don't halt for failure or error on DuffPaneTest; it requires a headed display. 2003-07-19 14:46:58 +00:00
dchandler 1c29566aee I'm now using the Unix diff built in to Apache Jakarta Commons JRCS
(which I found on suigeneris.org, not apache.org) in order to bulletproof the
Tibetan Converter tests.  They used to fail due to nondeterminism in the
Java RTF writer; they should no longer fail.

I've also changed it so that the Tibetan Converter tests run in headless
mode, which means that they'll run on the nightly builds server.
2003-07-14 12:26:26 +00:00
dchandler 689c1910aa To deal with java.swing.text.rtf bugs regarding hexadecimal escape
sequences, I've created RTFFixerInputStream.  It turns illegal hexadecimal
escapes into Unicode escapes.
2003-06-29 02:30:08 +00:00
dchandler 56d5ac7210 TM->TMW and TMW->TM conversion in RTF is now supported. I've
noticed that formatting is mostly OK but sometimes gets bungled slightly.
I tried everything I could think of, and now I'm passing the buck to Java's
RTF support.

TMW_RTF_TO_THDL_WYLIE (now misnamed) support TMW->TM
conversion (but not TM->TMW).  There is an automated test case for a
TMW->TM conversion.

I have full confidence in this conversion.  Even the smallest glitch in the core
functionality (not formatting) would surprise me.
2003-05-31 23:18:17 +00:00
dchandler ec7fec695f Added some automated JUnit tests for TMW_RTF_TO_THDL_WYLIE. 2003-05-18 17:17:52 +00:00
dchandler efa8fc1f25 DuffPane now has the start of a unit test suite. Invoke it via 'ant
clean check'.  Right now there are tests to ensure that typing certain
sequences of keys in the Extended Wylie keyboard gives the expected
Extended Wylie back when "Tools/Convert Tibetan to Wylie" is invoked.

The syntactically illegal d.wa now converts to Tibetan and then back
to d.wa (not dwa, as it did); likewise with the illegal g.wa.  wa
doesn't take any prefixes, but I prefer clean end-to-end
behavior. (jeskd doesn't go end-to-end, though.)

Note that you cannot successfully run the DuffPane tests on a Linux
box unless your DISPLAY variable is set correctly.  Thus, my nightly
builds will fail with an Error (as opposed to a Failure).
2003-04-14 05:22:27 +00:00
dchandler 08d2a5d702 Added a test for org.thdl.tib.text.tshegbar.UnicodeCodepointToThdlWylie. 2003-03-22 04:55:17 +00:00
dchandler 16cbfb6033 Moved ad-hoc test.java test cases to UnicodeGraphemeClusterTest.java,
a JUnit test which can be run via 'ant check'.  Removed test.java and
its build process.
2003-03-22 03:55:39 +00:00
dchandler 395eca7bb1 Moved ad-hoc test.java test cases to LegalTshegBarTest.java, a JUnit
test which can be run via 'ant check'.
2003-03-22 03:46:32 +00:00
dchandler 879b477902 Made some ad-hoc tests in test.java into JUnit tests, run by 'ant
check'.

NORM_NFD was replaced with NORM_NFKD in three cases in testMostlyNFKD.
2003-03-22 03:24:56 +00:00
dchandler bb19c4f6d1 You may no longer put xml-apis.jar, xercesImpl.jar, or xalan.jar in
Jskad/extensions.  If you're lazy, you can move them to
extensions/drop-ins, but the correct thing to do is to move x*.jar to
$ANT_HOME/lib [next to vamp.jar, if you're already set up for Java Web
Start builds].  This is a side effect of improving the nightly builds.

Nightly builds now feature an HTML summary of the JUnit test results,
a datestamp, and full API docs in two flavors.

If you use a patched vamp.jar [e-mail me] that can run when an X11
display is not available (there is not an analogous problem for
Windows servers, I suspect), all you have to do to put up a nightly
builds site is to set up CVS access so that no password is requires
using SSH public-key crypto (sf.net documents how to do so well) and
then use the following daily cron job on your Unix box:

#! /bin/sh
renice +19 -p $$ >/dev/null 2>&1
su -l joe-user /bin/sh -c /var/www/thdl/nightly/doTheBuild.sh

where joe-user is an unprivileged user who has installed Ant properly
(see the updated BuildSystems.html on the developer's site off of
thdltools.sf.net) and set himself up a Jskad sandbox with a Fonts
sandbox underneath it in, e.g., /var/www/thdl/nightly/Jskad.  Here's
doTheBuild.sh:

#! /bin/sh
JSKAD=/var/www/thdl/nightly/Jskad
DEST=/var/www/thdl/nightly/builds
HISTORY=/var/www/thdl/nightly/history
DATE=`date`
if test ! -d $DEST; then \
     echo "$DEST does not exist ($DATE)." >> history; exit 1; fi
(cd $JSKAD && ant dc-nightly-build \
 && rm -fr ${DEST}/* \
 && cp dist/nightlyBuild.zip $DEST \
 && cd $DEST \
     && unzip nightlyBuild.zip)
if test $? != 0; then echo "NIGHTLY BUILDS FAILED on $DATE" >> $HISTORY; exit 2; fi
DDDATE=`date`
echo "Success on start=$DATE end=$DDDATE" >> $HISTORY
exit 0
2003-02-10 04:22:38 +00:00
dchandler 56de29cf4f jskad-all-in-one-dist, and thus jskad-jws, was botched.
nightly-build now cleans only ${bin} so that ${dist}'s files stay
around.

We now unwar all JAR files.
2003-02-03 08:21:37 +00:00
dchandler 380f749d1b Changed project name. 2003-02-01 06:43:13 +00:00
dchandler a31fed2ea7 We're now set up to support automated unit tests. Besides the fact
that I like unit tests, my Unicode conversion work is going to have to
be thorougly tested for reasons I will outline in
http://thdltools.sf.net/BuildSystems.html later today.

Added the freely licensed JUnit 3.8.1 binary to the repository, along
with some README files.  Added a new supporting buildfile,
junitbuild.xml.  'ant clean check' is now good to go (though it uses
the text UI for JUnit, and some developers may want the Swing GUI)

Also, I cleaned up build.xml a bit, including adding all buildfiles
(but not junit.jar and things like that) to the source distribution
('ant src-dist') rather than just build.xml.
2003-01-12 20:24:01 +00:00