+ David Chandler has put together a build system for Jskad, Savant, + QuillDriver, the translation tool, and for the fonts. This document + describes them, and should be updated when we put together build + systems for the two diacritics packages and for Wylie Word. +
++ What is a build system, you ask? A build system is a set of + clear, repeatable procedures that software developers go by to + compile, run, test, create documentation, and cut releases. Our + build systems are fully automated, which provides a good basis for + collaborative development. +
+ +Fonts
+ module
+ www
+ module -- how to publish on this very web site, that is
+ Diacritics
+ module
+ WylieWord
+ module
+ Jskad
+ module
+
+
+ + This document uses the following terms: +
+ +'cvs checkout'. It's called a
+ sandbox because you can safely play in it; your work will not
+ affect others until you commit your changes to the repository.
+ (Though if you really wanted to, you could create a CVS branch and
+ then even your commits (to that branch) wouldn't affect the work
+ of others on the trunk, or main branch.)
+ thdltools.sourceforge.net. Windows users should
+ either get these tools with Cygwin or should install PuTTY; the
+ latter is probably easier.
+ + Ant is built from the ground up as a full-featured, extensible + build system for Java programs. If you use Ant, your + classpath-related headaches disappear (after clearing initial + hurdles), and you can be sure that the build will work the same on + Linux and the Mac as it does on Windows XP, because Ant itself is + a Java program. +
+ ++ Although Ant is for Java programs, it has enough features to make + it a viable choice of build system for non-Java stuff like our + Tibetan Machine Web fonts. +
+ ++ To install Apache's Jakarta Ant (or "Apache Ant" or simply "Ant"), + see the Apache Ant + website. Their instructions for installation are pretty good, but + you may have trouble if you're on Windows and you don't know how to + set environment variables (try Control + Panel->System->Advanced->Environment Variables) and the like. +
+ +
+ To test your installation, use cvs to checkout the
+ Fonts module. It has a simpler Ant build file
+ (Fonts/build.xml) than the Jskad module
+ has, so it's better for testing. Change directory to
+ Fonts and run the following command:
+
+ ant clean dist
+
+
++ If you have any trouble, see Ant's FAQs and then e-mail David + Chandler. +
+ ++ Now that you have a good installation of Ant itself, you will want + to install the Vamp Ant + task provided by the Venus + Application Publisher. You need this to cut Java Web + Start releases of Jskad, Savant, QuillDriver, and the + translation tool. +
+ +
+ Installing Vamp's Ant task is simple enough, and is in fact simpler
+ than Vamp's web pages of October 1, 2002 imply. That is, if you
+ have installed Ant 1.5.1 or later, then you can simply drop
+ vamp.jar into Ant's lib directory and
+ forget about changing your classpath.
+
+ If for some reason you don't want to use Vamp, you'll need to edit
+ Jskad/build.xml and delete the two associated
+ taskdefs so that Ant doesn't complain.
+
+ If you're working on Windows, it's probably easiest to follow
+ SourceForge's instructions on using WinCVS and PuTTY. The
+ instructions there aren't up to date for the latest version of
+ WinCVS (1.3b8 as of this writing). Instead of telling WinCVS about
+ plink.exe in the non-existent Ports tab of
+ the Admin->Preferences menu, you must click the
+ Settings button next to the server type ("ssh") and
+ click the 'If ssh is not in the path' checkbox, and
+ enter plink.exe's full path there.
+
+ Also note that + this page is very helpful; it's about how to run WinCVS. +
+ +
+ If you're using WinCVS 1.3b8, note that it has a bug -- it truncates
+ the output of 'cvs diff' (as explained in
+ this bug report). For that reason, you may wish to use what
+ WinCVS calls an "external diff" program, which is easy to install
+ and configure. Try using
+ ExamDiff.
+
+ If you're on Linux or Unix, SF.net's site docs are excellent. If + you're on a Mac, and SF.net's docs don't do the trick, you're in + trouble. +
+ + + +Fonts module
+ The Fonts CVS module comes with an Apache Ant build
+ system. We made this choice because the Jskad CVS
+ module already has an Apache Ant build system, and because Ant is
+ more than powerful enough for our needs. For this module, our only
+ task is to cut releases. There is nothing to compile here; the only
+ task that a developer will face is updating the associated
+ documentation and then cutting a new release, ready to go up on
+ SourceForge's File Release System (FRS).
+
+ To cut releases for one or both of the Tibetan Machine Web and + Tibetan Machine, do the following: +
+ +Fonts module. If you have
+ already checked it out, run 'cvs -f update -d' (in
+ WinCVS, hold down the shift key while selecting "Update" and
+ then be sure the "Create missing directories that exist in the
+ repository" box is checked).
+ Fonts and run the following
+ command:
+
+ ant
+
+ If you have any problems, be sure you're using the correct
+ version of Ant and then read Ant's FAQs and documentation. Or
+ you can e-mail David Chandler. If it turns out that the problem
+ is in the Ant build file Fonts/build.xml, make the
+ required changes and be sure to commit them to the repository.
+ Now run the following command to rebuild the releases:
+
+ ant clean dist
+
+ 'cvs -f rtag tag-name
+ Fonts' or (and this is preferred) you can run 'cvs
+ -f tag -c tag-name'. Good tag names are like the
+ following:
+ Fonts_1_0 (referring to the 1.0 release of the
+ Fonts module)
+ September_23_2002_release (referring to the
+ release of the Fonts module made on September 23, 2002
+ www module+ Our SourceForge.net + website is entirely under CVS control. This means that we can + freely make changes, knowing that they are never lost. It also + means that downloading our + nightly CVS repository tarball is all that is necessary to save + a backup copy of our SF.net web site. +
+ ++ To change an existing web page or to add a new one, you must do the + following: +
+ +www module. If you've
+ already done that, then please use cvs to update (using 'cvs
+ update -d' if you want to create missing directories that
+ exist in the repository) now.
+ 'cvs add'. Change
+ any files you wish. Now validate your HTML using W3C's validator. Once it passes
+ with no errors, commit your changes.
+ thdltools.sourceforge.net:/home/groups/t/th/thdltools/htdocs.
+ To do so, use PuTTY or another ssh to log in to
+ thdltools.sourceforge.net. Now change directory
+ (cd) to
+ /home/groups/t/th/thdltools/htdocs and run 'cvs
+ -f update -d -P'. You'll see your changes being made to
+ the "real" website. Surf to the website and see
+ for yourself.
+ www
+module
+ Not all of the material up at our SF.net website is in the
+ www CVS module, but most of it is. Currently, the only
+ exception is the
+ Javadoc API documentation for Jskad, Savant, QuillDriver, and
+ the translation tool.
+
+ Here is how to update these API docs: +
+ +http://thdltools.sourceforge.net/api lives at
+ /home/groups/t/th/thdltools/htdocs/api. To update
+ it, first commit any changes you've made to the repository.
+ (This, combined with the date stamp that Ant puts into our
+ Javadoc docs, will give other developers enough information to
+ recover the revision of the source files that you used. But
+ feel free to 'cvs tag' the revision explicitly if
+ you think it's important.)
+ 'ant private-javadocs-dist' from the
+ Jskad directory, which creates
+ Jskad/dist/docs/private-javadocs-DSTAMP.zip.
+ Examine the output for warnings; resolve them if necessary.
+ scp that file over with the command
+
+ scp dist/docs/private-javadocs-WHATEVER.zip
+ yourUserId@thdltools.sourceforge.net/home/groups/t/th/thdltools/
+
+ or use PuTTY's scp to do something equivalent.
+ thdltools.sourceforge.net with
+ 'ssh -l yourUserId thdltools.sourceforge.net' or
+ with PuTTY.
+ cd) to
+ /home/groups/t/th/thdltools/htdocs/.
+ 'cvs -f update -d -P' --
+ just in case some developer made changes to the web pages but
+ forgot to update (don't worry about setting the CVSROOT--cvs is
+ smart enough to determine everything (but your password)
+ automatically).
+ 'rm -fr api' (from the
+ htdocs/ directory).
+ 'mkdir api'.
+ 'cd api'.
+ 'unzip
+ ../../private-javadocs-WHATEVER.zip'.
+ http://thdltools.sourceforge.net/api/ looks
+ up to date (see the date at the very bottom of the page, for
+ starters). Also, verify that links to J2SE Javadoc
+ documentation are functioning -- click on a link to
+ java.lang.Object and be sure that it works. If
+ everything is fine, delete
+ ../../private-javadocs-WHATEVER.zip and log out.
+ + That's all! +
+ + + +Diacritics module+ We don't yet have source code in this module. Check back here once + we do. +
+ + + +WylieWord module+ We don't yet have source code in this module. Check back here once + we do. +
+ + + +Jskad module
+ The following sections describe how to use the Ant build system we
+ have for the Jskad module.
+
+ The first thing to note is that the Jskad module is
+ misnamed. We have all our Java code inside this module, including
+ Jskad, Savant, QuillDriver, and the translation tool.
+
Jskad
+module+ You'll have to first get your computer set up to use CVS. See above. +
+ +
+ The next thing you'll need to do is get a CVS sandbox -- a Jskad directory on
+ your local computer. Use 'cvs checkout Jskad' or
+ WinCVS's "Checkout module" command to do so.
+
+ Now go get the JARs you need. Be sure to note which versions you + get, because you'll need to mention that in the release notes of any + releases you cut. The following are needed: +
+xalan.jar, xercesImpl.jar, and
+ xml-apis.jar to Jskad/extensions/.
+ jdom.jar from
+ JDOM.org.
+ thdl.media.player
+ option) the Java(TM)
+ Media Framework. After it is installed, either change your
+ CLASSPATH to include its jmf.jar, or avoid the
+ classpath hassle by copying jmf.jar to
+ Jskad/extensions/.
+ thdl.media.player
+ option) QuickTime for
+ Java. Installing it requires installing QuickTime, selecting
+ a "Custom" install, and checking the box for QuickTime for Java.
+ After it is installed, either change your CLASSPATH to include its
+ QTJava.zip, or avoid the classpath hassle by copying
+ QTJava.zip to Jskad/extensions/drop-ins.
+
+ If you need to add something else to the classpath, be it a .jar
+ file, a .zip file, or a directory, you have several options. You
+ can set your CLASSPATH environment variable, or you can call Ant
+ using 'ant -Dadditional.class.path=c:\my.jar', or you
+ can drop JAR or ZIP files into
+ Jskad/extensions/drop-ins.
+
+ While you're hacking, use our API docs on the + web, or build your own. +
+ + + +Jskad
+module+ This section describes how our build system interacts with your + daily work hacking on Jskad, Savant, QuillDriver, and the + translation tool. +
+ + +
+ Compiling any of the tools is a simple matter. First, ensure that
+ you are in the Jskad/ directory of your sandbox.
+
+ Now run 'ant program-compile' to create a
+ directory tree Jskad/bin/program/ that contains
+ all the .class files and needed resources (.ini files, .xsl files,
+ et cetera).
+
+ Or you may run 'ant program-dist' to create a
+ JAR file Jskad/dist/lib/program-vanilla.jar that
+ contains all the .class files and needed resources, but does not
+ contain any third-party JARs.
+
+ You may also run 'ant program-jws' to create a
+ Java Web Start release. See below to
+ learn more about this.
+
+ Gotcha: We currently have the build system set up so that it
+ builds a program, say Jskad, given merely the name of the main
+ class, say org.thdl.tib.input.Jskad. This means that
+ our releases include only classes that we're actually using. But it
+ also means that your changes to any but the main class will not
+ cause recompilation. You must use 'ant clean' before
+ recompiling. In addition, you should clean after any cvs update.
+
+ The alternative to specifying just the main class is specifying
+ every needed class. If you do this, Ant often rebuilds the correct
+ classes. But you may have to modify build.xml when you
+ add a new file, and your releases may include vestigial classes.
+
+ Running the tools is easy for a developer. Just run 'ant
+ program-run', where program is one of
+ jskad, tt, savant, or
+ qd. This deals with classpaths and all of that, and it
+ compiles beforehand if needed. Note the gotcha above if you're
+ rerunning after changing the source code -- you'll have to use
+ 'ant clean program-run'. If you modify a
+ resource, such as Jskad/source/options.txt, there are
+ no gotchas -- Ant is smart enough to handle this.
+
+ While you're hacking, use our API docs on the
+ web, or build your own Javadoc API docs using 'ant
+ public-javadocs' (to see only protected and public members)
+ or 'ant private-javadocs' (to see all members).
+ They'll be waiting for you in
+ Jskad/docs/{public,private}-javadocs/. If you want to
+ update the API docs on our website, follow these instructions.
+
+ Nowadays, any good Java IDE is aware of Ant and will let you use Ant + for the things it is good at while using the IDE for the things it + is good at. Ask David Chandler how he got Eclipse up and running; + he did it without too much trouble. +
+ + + +
+ We will soon use JUnit to provide
+ automated unit tests. Running them will be as simple as making Ant
+ aware of JUnit and then running 'ant check'.
+
+ To make testing easy, you should use interfaces rather than + hard-wiring in calls to classes that are difficult to instantiate. + For example, if we were to make use of a database class in our + DuffPane class, doing so directly would mean that the database + software would have to be properly installed and up and running in + order to test DuffPane individually. If you create an interface and + use the database only through that interface, then creating a mock object that pretends to + be that database but returns canned answers is a simple matter. +
+ + +Jskad/build.xml
+ Should you need to modify the Ant build file
+ Jskad/build.xml, feel free to ask David Chandler to do
+ it for you. Or just look at the contents of that file and find
+ something similar to what you're doing. There are a few comments in
+ the file, and all commonly used functionality is factored out into
+ targets that are then summoned via antcall tasks.
+
+ Java Web + Start makes it possible (in theory, anyway) to install a complex + application like Savant with one mouse click of a link in your + favorite web browser and one more click of a "Yes, install Savant" + button in a dialog box (necessary because our applications require + security privileges to read and write from disk etc.). Upon the + second run of the program, Java Web Start asks the user if they want + to create shortcuts on their computer. Also, any time the program + is invoked (from a shortcut or a web link) and a network connection + is found, new versions of the software will be downloaded + automatically, keeping the user up to date. +
+ ++ In addition, the Venus Application + Publisher provides a mechanism for creating CD-ROM-based + installers that load our Java Web Start code into the user's Web + Start cache directory. Then running the software will check for + updates as described above. (I don't know if we'll ever use this, + and it's kind of ugly, but it's here.) +
+ ++ For the above scenario to play out, the user must have installed a + recent (1.2 or later) Java runtime environment. That's it from the + user's point of view. +
+ + ++ The web server that delivers the content must associate the MIME + type application/x-java-jnlp-file with the "jnlp" extension and must + associate the MIME type application/x-java-archive-diff with the + "jardiff" extension. In addition, if you want to be able to serve + up a new release with the least difficulty (simply dropping a .war + file (a Java Web ARchive) into the appropriate directory on the + server), the server must run a J2EE application server such as + Apache's Tomcat. +
+ + + ++ For the reasons stated above, Java Web Start (JWS) is our + distribution mechanism of choice. But cutting a Java Web Start + release is not the easiest thing you've ever done. For one thing, + because our applications require access to the local disk, they must + ask for security privileges. JWS warns users not to grant such + privileges unless they know the software provider is trustworthy. + Trust depends in turn on identity, and identity is proven by public + key cryptography. So to cut a JWS release, you have to sign your + JARs and rejar and sign any third-party JARs. (If signing JARs is + new to you, the J2SE documentation on the subject is okay, but + hopefully reading below will tell you all you need to know.) +
+ ++ First, be sure you've installed Vamp's Ant task as described above. +
+ ++ Second, create the release notes and the changelog. These should be + two different text files. Somewhere in one or both, you should + mention the versions of the third-party JARs and APIs that we have + packaged with our release. Most importantly, put in a link to + the web server on which you're going to put the web start + release! Commit these text files to the repository (adding them + first if necessary). +
+ +
+ Third, commit any changes you've made to the repository. After
+ being sure that your Jskad directory is a mirror of the
+ one in the repository, use cvs to tag this revision that you wish to
+ release with a name that another developer, years down the road,
+ will recognize as referring to this release. To do this, you can
+ run 'cvs -f rtag tag-name Jskad' or (and this is
+ preferred) you can run 'cvs -f tag -c tag-name'.
+ Good tag names are like the following:
+
thdl_tools_1_0 (referring to the 1.0 release of the
+ Jskad module)
+ September_23_2002_release (referring to the
+ release of all programs in Fonts module made on September 23, 2002
+ + As you may have inferred, we release all four programs when we + release any one of them. This is only a half-truth, though--we + release the unified source distribution any time we release a binary + distribution, but we don't necessarily put out all four binary + distributions in unison. +
+ +
+ Now run 'ant -Dkeystore=foo -Dkeystore.password=foo
+ -Ddgkey.password=foo -Dkey.alias=foo program-jws',
+ where program is one of jskad, tt,
+ savant, or qd. This cleans first, so your
+ changes will be respected. This then creates a WAR file
+ Jskad/dist/java-web-start/program-JWS.war. This
+ also extracts the signed JAR from inside that WAR, which is handy if
+ you're deploying to a web server that isn't running a J2EE
+ application server. More on that below.
+
+ The argument to -Dkeystore is a URL like
+ file:///c:/foo/keystore. This should be a keystore
+ that you've created with keytool. We haven't yet paid the money to
+ get a real X.509 key, so create your own. Users will see a nasty
+ warning saying that we may not be who we say we are, but so be it.
+ Each key in your key store has an associated alias, such as your
+ name. Specify it as the argument to -Dkey.alias. The
+ other two arguments are the password for the keystore as a whole
+ (-Dkeystore.password) and the password for the specific
+ key whose alias you're giving.
+
+ Inspect the output for warnings. Ignore the warning about
+ "[vampwar]
+ org.apache.velocity.runtime.exception.ReferenceException: reference
+ : template = html/test.html [line 37,column 14] : $jnlp-url is not a
+ valid reference." but fix any others.
+
+ If you wish to deploy to a J2EE application server, such as Apache's
+ Tomcat, now copy the .war file to the server. You're
+ done with the binary part of the release.
+
+ If you're deploying to a plain-vanilla web server, first ensure that
+ its MIME type associations are correct (see above). Now ignore the .war
+ file and copy the appropriate .jnlp file(s) from
+ Jskad/dist/ to the webserver, and copy the appropriate
+ JAR(s) from Jskad/dist/java-web-start/ to the
+ webserver. Those two files are all you need, but you must edit the
+ .jnlp file appropriately for your webserver.
+
+ Now put up a source release on SourceForge's File Release System
+ (FRS). After ensuring that you've made NOT EVEN ONE CHANGE to your
+ sandbox, and after updating it, and after verifying that this
+ revision is cvs-tagged appropriately, use 'ant
+ src-dist' to create
+ Jskad/source/THDL-Tools-src-todays-date.zip.
+ Now put this zip file up on the FRS with the appropriate release
+ notes and changelog (which should have been committed and tagged in
+ the repository).
+
+ You're done! Except, of course, for the gotcha: At present,
+ only the translation tool and Jskad have working Java Web Start
+ releases; Savant's and QuillDriver's are on the way. Savant and
+ QuillDriver, like Jskad, requires all-permissions security. That
+ means that xalan.jar, xercesImpl.jar, and
+ xml-apis.jar need to be jar-signed by the same
+ certificate as Savant. We don't yet do this automatically , but here's how to do this manually:
+
+ Now send an e-mail to + thdltools-announce@lists.sourceforge.net telling users where the + JWS binary release is, where the source code distribution is, and + why they should upgrade or try it out. +
+ + + +