FileDocCategorySizeDatePackage
WhitespaceTokenizer.javaAPI DocApache Lucene 1.4.31168Tue Mar 30 00:48:00 BST 2004org.apache.lucene.analysis

WhitespaceTokenizer

public class WhitespaceTokenizer extends CharTokenizer
A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

Fields Summary
Constructors Summary
public WhitespaceTokenizer(Reader in)
Construct a new WhitespaceTokenizer.

    super(in);
Methods Summary
protected booleanisTokenChar(char c)
Collects only characters which do not satisfy {@link Character#isWhitespace(char)}.

    return !Character.isWhitespace(c);