Author:   Scott Sanner
Content:  Instructions, code, and test files for running a java HMM-based 
          text extraction system.
Includes: GNU GPL HTML Parser (com.quiotix.html.parser).
=========================================================================


Compilation and Execution:
--------------------------

To compile, use the following command from the directory containing this 
README file:

* javac ./Extractor/*.java

To run the extractor, use:

* java Extractor.TemplateExtractor <trainset> <testset> \ 
       <template> {comp|nocomp}

where <trainset> and <testset> are files containing lists of training and
testing files and {comp|nocomp} is a flag to indicate whether to perform
statistical comparison of the testset at the end (only works if the test
set is properly marked up with the correct template keywords).

Alternately, one can use the provided script 'te' to save some typing, 
i.e. the following command can be entered either of two ways:

* java Extractor.TemplateExtractor trainfiles1.txt testfiles1.txt \
       title1.template comp

or equivalently:

* ./te trainfiles1 testfiles1 title1 comp


Provided Test Files:
--------------------

Following is a brief description of the training and testing files that
can be provided on the command line:

* trainfiles1.txt - A set of 80 cached CS course web pages from the WebKb 
project which I have hand-labeled with title, instructor, time, and
location markers.  Used for training in the above experiments.  
(Note: ./train/* is the actual location of the marked up data.)
 
* testfiles1.txt - A separate set of 20 cached CS course web pages from the 
WebKb project which I have hand-labeled with title, instructor, time, and 
location markers.  Used for testing in the above experiments.  (Note: 
Extractor/train/* is the actual location of the marked up data.)
 
* webaddr1.txt - A list of 20 web pages of CS courses currently being
offered by Stanford, CMU, U. Texas, and Wisconsin.  Although these pages 
obviously are not marked with the appropriate tags, one can verify from
the template output that the correct course title is identified in 17 of 
the 19 web pages.

Following are a list of template files that can be provided on the command
line:

* title1.template - The optimal HMM template for extracting course titles.
* inst1.template  - The optimal HMM template for extracting course instructors.
* time1.template  - The optimal HMM template for extracting course times.
* loc1.template   - The optimal HMM template for extracting course locations.