FileDocCategorySizeDatePackage
WhitespaceTokenizer.javaAPI DocApache Lucene 2.1.01356Wed Feb 14 10:46:38 GMT 2007org.apache.lucene.analysis

WhitespaceTokenizer

public class WhitespaceTokenizer extends CharTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

Fields Summary
Constructors Summary
public WhitespaceTokenizer(Reader in)
Construct a new WhitespaceTokenizer.

    super(in);
Methods Summary
protected booleanisTokenChar(char c)
Collects only characters which do not satisfy {@link Character#isWhitespace(char)}.

    return !Character.isWhitespace(c);