From 1ef4b5cff8277c3ec3aea3d5de3bf34e876b634f Mon Sep 17 00:00:00 2001 From: dchandler Date: Sun, 20 Oct 2002 22:40:18 +0000 Subject: [PATCH] Added documentation on our build systems -- how to compile, run, cut releases, etc. --- htdocs/BuildSystems.html | 966 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 966 insertions(+) create mode 100644 htdocs/BuildSystems.html diff --git a/htdocs/BuildSystems.html b/htdocs/BuildSystems.html new file mode 100644 index 0000000..cb885a5 --- /dev/null +++ b/htdocs/BuildSystems.html @@ -0,0 +1,966 @@ + + + + + + + + + + THDL Tools Build Systems + + + +

THDL Tools Build Systems

+ +

+ David Chandler has put together a build system for Jskad, Savant, + QuillDriver, the translation tool, and for the fonts. This document + describes them, and should be updated when we put together build + systems for the two diacritics packages and for Wylie Word. +

+

+ What is a build system, you ask? A build system is a set of + clear, repeatable procedures that software developers go by to + compile, run, test, create documentation, and cut releases. Our + build systems are fully automated, which provides a good basis for + collaborative development. +

+ +

Table of Contents

+ + + + + +

Glossary

+ +

+ This document uses the following terms: +

+ + + + + +

Why we chose Apache Ant

+ +

+ Ant is built from the ground up as a full-featured, extensible + build system for Java programs. If you use Ant, your + classpath-related headaches disappear (after clearing initial + hurdles), and you can be sure that the build will work the same on + Linux and the Mac as it does on Windows XP, because Ant itself is + a Java program. +

+ +

+ Although Ant is for Java programs, it has enough features to make + it a viable choice of build system for non-Java stuff like our + Tibetan Machine Web fonts. +

+ +

Installing Apache Ant

+ +

+ To install Apache's Jakarta Ant (or "Apache Ant" or simply "Ant"), + see the Apache Ant + website. Their instructions for installation are pretty good, but + you may have trouble if you're on Windows and you don't know how to + set environment variables (try Control + Panel->System->Advanced->Environment Variables) and the like. +

+ +

+ To test your installation, use cvs to checkout the + Fonts module. It has a simpler Ant build file + (Fonts/build.xml) than the Jskad module + has, so it's better for testing. Change directory to + Fonts and run the following command: +

+
+ ant clean dist +
+ +

+ If you have any trouble, see Ant's FAQs and then e-mail David + Chandler. +

+ +

+ Now that you have a good installation of Ant itself, you will want + to install the Vamp Ant + task provided by the Venus + Application Publisher. You need this to cut Java Web + Start releases of Jskad, Savant, QuillDriver, and the + translation tool. +

+ +

+ Installing Vamp's Ant task is simple enough, and is in fact simpler + than Vamp's web pages of October 1, 2002 imply. That is, if you + have installed Ant 1.5.1 or later, then you can simply drop + vamp.jar into Ant's lib directory and + forget about changing your classpath. +

+ +

+ If for some reason you don't want to use Vamp, you'll need to edit + Jskad/build.xml and delete the two associated + taskdefs so that Ant doesn't complain. +

+ + + +

Setting things up to use CVS

+ +

+ If you're working on Windows, it's probably easiest to follow + SourceForge's instructions on using WinCVS and PuTTY. The + instructions there aren't up to date for the latest version of + WinCVS (1.3b8 as of this writing). Instead of telling WinCVS about + plink.exe in the non-existent Ports tab of + the Admin->Preferences menu, you must click the + Settings button next to the server type ("ssh") and + click the 'If ssh is not in the path' checkbox, and + enter plink.exe's full path there. +

+ +

+ Also note that + this page is very helpful; it's about how to run WinCVS. +

+ +

+ If you're using WinCVS 1.3b8, note that it has a bug -- it truncates + the output of 'cvs diff' (as explained in + this bug report). For that reason, you may wish to use what + WinCVS calls an "external diff" program, which is easy to install + and configure. Try using + ExamDiff. +

+ +

+ If you're on Linux or Unix, SF.net's site docs are excellent. If + you're on a Mac, and SF.net's docs don't do the trick, you're in + trouble. +

+ + + +

Building the Fonts module

+ +

+ The Fonts CVS module comes with an Apache Ant build + system. We made this choice because the Jskad CVS + module already has an Apache Ant build system, and because Ant is + more than powerful enough for our needs. For this module, our only + task is to cut releases. There is nothing to compile here; the only + task that a developer will face is updating the associated + documentation and then cutting a new release, ready to go up on + SourceForge's File Release System (FRS). +

+ +

+ To cut releases for one or both of the Tibetan Machine Web and + Tibetan Machine, do the following: +

+ +
    +
  1. + use cvs to checkout the Fonts module. If you have + already checked it out, run 'cvs -f update -d' (in + WinCVS, hold down the shift key while selecting "Update" and + then be sure the "Create missing directories that exist in the + repository" box is checked). +
    2. +
  2. + make whatever changes you need to +
    3. +
  3. + be certain the the release notes (a text or html file, I + believe) are under CVS control. Perhaps you'll reuse an + existing document, or perhaps you'll have to add a file to the + repository. Go to + our project's download page and see how we've done release + notes in the past. +
    4. +
  4. + commit those changes to the repository +
    5. +
  5. + install Apache Ant +
    6. +
  6. + change directory to Fonts and run the following + command: +
    + ant +
    + If you have any problems, be sure you're using the correct + version of Ant and then read Ant's FAQs and documentation. Or + you can e-mail David Chandler. If it turns out that the problem + is in the Ant build file Fonts/build.xml, make the + required changes and be sure to commit them to the repository. + Now run the following command to rebuild the releases: +
    + ant clean dist +
    +
    7. +
  7. + after being sure that your Fonts directory is a mirror of the + one in the repository, use cvs to tag this revision that you + wish to release with a name that another developer, years down + the road, will recognize as referring to this release. To do + this, you can run 'cvs -f rtag tag-name + Fonts' or (and this is preferred) you can run 'cvs + -f tag -c tag-name'. Good tag names are like the + following: +
      +
    • + Fonts_1_0 (referring to the 1.0 release of the + Fonts module) +
      • +
    • + September_23_2002_release (referring to the + release of the Fonts module made on September 23, 2002 +
      • +
    + Note that tagging for a release of just the Tibetan Machine Web + fonts and not the Tibetan Machine fonts is no different + than tagging for a simulaneous release of both. +
    8. +
  8. + Now that you've tagged this release, it will be possible for any + other developer to recreate the same release. This is the heart + of a good build system -- clear, repeatable procedures that + aren't painful to follow. +
    9. +
+ + + +

Building the www module

+ +

+ Our SourceForge.net + website is entirely under CVS control. This means that we can + freely make changes, knowing that they are never lost. It also + means that downloading our + nightly CVS repository tarball is all that is necessary to save + a backup copy of our SF.net web site. +

+ +

+ To change an existing web page or to add a new one, you must do the + following: +

+ + + +

Updating the parts of the website not in the www +module

+ +

+ Not all of the material up at our SF.net website is in the + www CVS module, but most of it is. Currently, the only + exception is the + Javadoc API documentation for Jskad, Savant, QuillDriver, and + the translation tool. +

+ +

+ Here is how to update these API docs: +

+ + + +

+ That's all! +

+ + + +

Building the Diacritics module

+ +

+ We don't yet have source code in this module. Check back here once + we do. +

+ + + +

Building the WylieWord module

+ +

+ We don't yet have source code in this module. Check back here once + we do. +

+ + + +

Building the Jskad module

+ +

+ The following sections describe how to use the Ant build system we + have for the Jskad module. +

+ +

+ The first thing to note is that the Jskad module is + misnamed. We have all our Java code inside this module, including + Jskad, Savant, QuillDriver, and the translation tool. +

+ + + +

Setting things up for your first build of the Jskad +module

+ +

+ You'll have to first get your computer set up to use CVS. See above. +

+ +

+ The next thing you'll need to do is get a CVS sandbox -- a Jskad directory on + your local computer. Use 'cvs checkout Jskad' or + WinCVS's "Checkout module" command to do so. +

+ +

+ Now go get the JARs you need. Be sure to note which versions you + get, because you'll need to mention that in the release notes of any + releases you cut. The following are needed: +

+ + +

+ If you need to add something else to the classpath, be it a .jar + file, a .zip file, or a directory, you have several options. You + can set your CLASSPATH environment variable, or you can call Ant + using 'ant -Dadditional.class.path=c:\my.jar', or you + can drop JAR or ZIP files into + Jskad/extensions/drop-ins. +

+ +

+ While you're hacking, use our API docs on the + web, or build your own. +

+ + + +

Day-to-day development activities with the Jskad +module

+ +

+ This section describes how our build system interacts with your + daily work hacking on Jskad, Savant, QuillDriver, and the + translation tool. +

+ + +
Compiling
+ +

+ Compiling any of the tools is a simple matter. First, ensure that + you are in the Jskad/ directory of your sandbox. +

+ +

+ Now run 'ant program-compile' to create a + directory tree Jskad/bin/program/ that contains + all the .class files and needed resources (.ini files, .xsl files, + et cetera). +

+ +

+ Or you may run 'ant program-dist' to create a + JAR file Jskad/dist/lib/program-vanilla.jar that + contains all the .class files and needed resources, but does not + contain any third-party JARs. +

+ +

+ You may also run 'ant program-jws' to create a + Java Web Start release. See below to + learn more about this. +

+ +

+ Gotcha: We currently have the build system set up so that it + builds a program, say Jskad, given merely the name of the main + class, say org.thdl.tib.input.Jskad. This means that + our releases include only classes that we're actually using. But it + also means that your changes to any but the main class will not + cause recompilation. You must use 'ant clean' before + recompiling. In addition, you should clean after any cvs update. +

+ +

+ The alternative to specifying just the main class is specifying + every needed class. If you do this, Ant often rebuilds the correct + classes. But you may have to modify build.xml when you + add a new file, and your releases may include vestigial classes. +

+ +
Running
+ +

+ Running the tools is easy for a developer. Just run 'ant + program-run', where program is one of + jskad, tt, savant, or + qd. This deals with classpaths and all of that, and it + compiles beforehand if needed. Note the gotcha above if you're + rerunning after changing the source code -- you'll have to use + 'ant clean program-run'. If you modify a + resource, such as Jskad/source/options.txt, there are + no gotchas -- Ant is smart enough to handle this. +

+ + +
Generating API Documentation
+ +

+ While you're hacking, use our API docs on the + web, or build your own Javadoc API docs using 'ant + public-javadocs' (to see only protected and public members) + or 'ant private-javadocs' (to see all members). + They'll be waiting for you in + Jskad/docs/{public,private}-javadocs/. If you want to + update the API docs on our website, follow these instructions. +

+ + +
Integration with IDEs such as Eclipse
+ +

+ Nowadays, any good Java IDE is aware of Ant and will let you use Ant + for the things it is good at while using the IDE for the things it + is good at. Ask David Chandler how he got Eclipse up and running; + he did it without too much trouble. +

+ + + +
Testing
+ +

+ We will soon use JUnit to provide + automated unit tests. Running them will be as simple as making Ant + aware of JUnit and then running 'ant check'. +

+ +

+ To make testing easy, you should use interfaces rather than + hard-wiring in calls to classes that are difficult to instantiate. + For example, if we were to make use of a database class in our + DuffPane class, doing so directly would mean that the database + software would have to be properly installed and up and running in + order to test DuffPane individually. If you create an interface and + use the database only through that interface, then creating a mock object that pretends to + be that database but returns canned answers is a simple matter. +

+ + +
Modifying Jskad/build.xml
+ +

+ Should you need to modify the Ant build file + Jskad/build.xml, feel free to ask David Chandler to do + it for you. Or just look at the contents of that file and find + something similar to what you're doing. There are a few comments in + the file, and all commonly used functionality is factored out into + targets that are then summoned via antcall tasks. +

+ + + +

Why we chose Java Web Start and how it works

+ +

+ Java Web + Start makes it possible (in theory, anyway) to install a complex + application like Savant with one mouse click of a link in your + favorite web browser and one more click of a "Yes, install Savant" + button in a dialog box (necessary because our applications require + security privileges to read and write from disk etc.). Upon the + second run of the program, Java Web Start asks the user if they want + to create shortcuts on their computer. Also, any time the program + is invoked (from a shortcut or a web link) and a network connection + is found, new versions of the software will be downloaded + automatically, keeping the user up to date. +

+ +

+ In addition, the Venus Application + Publisher provides a mechanism for creating CD-ROM-based + installers that load our Java Web Start code into the user's Web + Start cache directory. Then running the software will check for + updates as described above. (I don't know if we'll ever use this, + and it's kind of ugly, but it's here.) +

+ +

+ For the above scenario to play out, the user must have installed a + recent (1.2 or later) Java runtime environment. That's it from the + user's point of view. +

+ + +

+ The web server that delivers the content must associate the MIME + type application/x-java-jnlp-file with the "jnlp" extension and must + associate the MIME type application/x-java-archive-diff with the + "jardiff" extension. In addition, if you want to be able to serve + up a new release with the least difficulty (simply dropping a .war + file (a Java Web ARchive) into the appropriate directory on the + server), the server must run a J2EE application server such as + Apache's Tomcat. +

+ + + +

Cutting a Java Web Start release

+ +

+ For the reasons stated above, Java Web Start (JWS) is our + distribution mechanism of choice. But cutting a Java Web Start + release is not the easiest thing you've ever done. For one thing, + because our applications require access to the local disk, they must + ask for security privileges. JWS warns users not to grant such + privileges unless they know the software provider is trustworthy. + Trust depends in turn on identity, and identity is proven by public + key cryptography. So to cut a JWS release, you have to sign your + JARs and rejar and sign any third-party JARs. (If signing JARs is + new to you, the J2SE documentation on the subject is okay, but + hopefully reading below will tell you all you need to know.) +

+ +

+ First, be sure you've installed Vamp's Ant task as described above. +

+ +

+ Second, create the release notes and the changelog. These should be + two different text files. Somewhere in one or both, you should + mention the versions of the third-party JARs and APIs that we have + packaged with our release. Most importantly, put in a link to + the web server on which you're going to put the web start + release! Commit these text files to the repository (adding them + first if necessary). +

+ +

+ Third, commit any changes you've made to the repository. After + being sure that your Jskad directory is a mirror of the + one in the repository, use cvs to tag this revision that you wish to + release with a name that another developer, years down the road, + will recognize as referring to this release. To do this, you can + run 'cvs -f rtag tag-name Jskad' or (and this is + preferred) you can run 'cvs -f tag -c tag-name'. + Good tag names are like the following: +

+ + +

+ As you may have inferred, we release all four programs when we + release any one of them. This is only a half-truth, though--we + release the unified source distribution any time we release a binary + distribution, but we don't necessarily put out all four binary + distributions in unison. +

+ +

+ Now run 'ant -Dkeystore=foo -Dkeystore.password=foo + -Ddgkey.password=foo -Dkey.alias=foo program-jws', + where program is one of jskad, tt, + savant, or qd. This cleans first, so your + changes will be respected. This then creates a WAR file + Jskad/dist/java-web-start/program-JWS.war. This + also extracts the signed JAR from inside that WAR, which is handy if + you're deploying to a web server that isn't running a J2EE + application server. More on that below. +

+ +

+ The argument to -Dkeystore is a URL like + file:///c:/foo/keystore. This should be a keystore + that you've created with keytool. We haven't yet paid the money to + get a real X.509 key, so create your own. Users will see a nasty + warning saying that we may not be who we say we are, but so be it. + Each key in your key store has an associated alias, such as your + name. Specify it as the argument to -Dkey.alias. The + other two arguments are the password for the keystore as a whole + (-Dkeystore.password) and the password for the specific + key whose alias you're giving. +

+ +

+ Inspect the output for warnings. Ignore the warning about + "[vampwar] + org.apache.velocity.runtime.exception.ReferenceException: reference + : template = html/test.html [line 37,column 14] : $jnlp-url is not a + valid reference." but fix any others. +

+ +

+ If you wish to deploy to a J2EE application server, such as Apache's + Tomcat, now copy the .war file to the server. You're + done with the binary part of the release. +

+ +

+ If you're deploying to a plain-vanilla web server, first ensure that + its MIME type associations are correct (see above). Now ignore the .war + file and copy the appropriate .jnlp file(s) from + Jskad/dist/ to the webserver, and copy the appropriate + JAR(s) from Jskad/dist/java-web-start/ to the + webserver. Those two files are all you need, but you must edit the + .jnlp file appropriately for your webserver. +

+ +

+ Now put up a source release on SourceForge's File Release System + (FRS). After ensuring that you've made NOT EVEN ONE CHANGE to your + sandbox, and after updating it, and after verifying that this + revision is cvs-tagged appropriately, use 'ant + src-dist' to create + Jskad/source/THDL-Tools-src-todays-date.zip. + Now put this zip file up on the FRS with the appropriate release + notes and changelog (which should have been committed and tagged in + the repository). +

+ +

+ You're done! Except, of course, for the gotcha: At present, + only the translation tool and Jskad have working Java Web Start + releases; Savant's and QuillDriver's are on the way. Savant and + QuillDriver, like Jskad, requires all-permissions security. That + means that xalan.jar, xercesImpl.jar, and + xml-apis.jar need to be jar-signed by the same + certificate as Savant. We don't yet do this automatically , but here's how to do this manually: +

+ +
    +
  1. unjar these jars.
    2. +
  2. delete their META-INF directory, which i think contains signature info.
    3. +
  3. rejar, then sign with appropriate certificate.
    4. +
+ +

+ Now send an e-mail to + thdltools-announce@lists.sourceforge.net telling users where the + JWS binary release is, where the source code distribution is, and + why they should upgrade or try it out. +

+ + + +
+ + + Please + + + e-mail us + + your comments about this page. + + +
+ +The + + + THDL Tools + +project is generously hosted by: + + + + SourceForge Logo + + + + + +