org.annotation.wordfreak.annotator
Class SimpleTokenAnnotator

java.lang.Object
  org.annotation.wordfreak.annotator.Annotator
      org.annotation.wordfreak.annotator.DocumentProcessor
          org.annotation.wordfreak.annotator.ParagraphProcessor
              org.annotation.wordfreak.annotator.SentenceProcessor
                  org.annotation.wordfreak.annotator.TokenAnnotator
                      org.annotation.wordfreak.annotator.SimpleTokenAnnotator

All Implemented Interfaces:

java.awt.event.ActionListener, AnnotatedFileListener, java.util.EventListener, Plugin

public class SimpleTokenAnnotator

extends TokenAnnotator

Field Summary

Fields inherited from class org.annotation.wordfreak.annotator.SentenceProcessor

sentenceTypes

Fields inherited from class org.annotation.wordfreak.annotator.Annotator

annotationFilter, dataDirectory, DEFAULT_ANNOTATOR_NAME, files, guiListener, listeners, loaded, progress, trainingFilter

Constructor Summary

SimpleTokenAnnotator(java.lang.String type)

Method Summary

static java.lang.String[]	getNames()
double[]	getTokProbs() Returns a confidence associated with each token returned in the most recent call to tokenize.
protected void	initTraining() Initializes annotator for training.
boolean	loadAnnotator() Loads the annotator.
Span[]	tokenize(java.lang.String s) Returns character offsets which are the tokens of the text parametter.
protected void	train() Trains a model based on the tokens provided in previous calls to trainWithTokens.
protected void	trainWithTokens(Span[] spans, java.lang.String name) Uses the tokens provided to construct events for traiing the current tokenizer model.

Methods inherited from class org.annotation.wordfreak.annotator.TokenAnnotator

processSentence, training, training

Methods inherited from class org.annotation.wordfreak.annotator.SentenceProcessor

processParagraph

Methods inherited from class org.annotation.wordfreak.annotator.ParagraphProcessor

processDocument

Methods inherited from class org.annotation.wordfreak.annotator.DocumentProcessor

annotating

Methods inherited from class org.annotation.wordfreak.annotator.Annotator

actionPerformed, addAnnotatorListener, annotate, annotatedFile, closeAnnotatedFile, done, getDataDirectory, hideWaitDialog, loaded, removeAnnotatorListener, setAnnotationFilter, setDataDirectory, setGuiListener, setProgress, setTrainingFilter, showWaitDialog, sortedOutcomes, supportsTraining, train, updateProgress

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleTokenAnnotator

public SimpleTokenAnnotator(java.lang.String type)

Method Detail

loadAnnotator

public boolean loadAnnotator()

Description copied from class: Annotator

Loads the annotator.

Specified by:

loadAnnotator in class Annotator

train

protected void train()

Description copied from class: TokenAnnotator

Trains a model based on the tokens provided in previous calls to trainWithTokens.

Specified by:

train in class TokenAnnotator

trainWithTokens

protected void trainWithTokens(Span[] spans,
                               java.lang.String name)

Description copied from class: TokenAnnotator

Uses the tokens provided to construct events for traiing the current tokenizer model.

Specified by:

trainWithTokens in class TokenAnnotator

Parameters:

spans - character offsets into text which are tokens to be used for training.

name - string into which offsets specified in tokens refer to.

initTraining

protected void initTraining()

Description copied from class: TokenAnnotator

Initializes annotator for training.

Specified by:

initTraining in class TokenAnnotator

getNames

public static java.lang.String[] getNames()

getTokProbs

public double[] getTokProbs()

Description copied from class: TokenAnnotator

Returns a confidence associated with each token returned in the most recent call to tokenize.

Specified by:

getTokProbs in class TokenAnnotator

Returns:

array of confidences associated with each token returned in the most recent call to tokenize.

tokenize

public Span[] tokenize(java.lang.String s)

Description copied from class: TokenAnnotator

Returns character offsets which are the tokens of the text parametter.

Specified by:

tokenize in class TokenAnnotator

Parameters:

s - the string to be tokenized. Typically a sentence.

Returns:

character offsets in to which are the tokens