The BreakIterator class implements methods for finding
the location of boundaries in text. Instances of BreakIterator
maintain a current position and scan over text
returning the index of characters where boundaries occur.
Internally, BreakIterator scans text using a
CharacterIterator, and is thus able to scan text held
by any object implementing that protocol. A StringCharacterIterator
is used to scan String objects passed to setText.
You use the factory methods provided by this class to create
instances of various types of break iterators. In particular,
use getWordIterator, getLineIterator,
getSentenceIterator, and getCharacterIterator
to create BreakIterators that perform
word, line, sentence, and character boundary analysis respectively.
A single BreakIterator can work only on one unit
(word, line, sentence, and so on). You must use a different iterator
for each unit boundary analysis you wish to perform.
Line boundary analysis determines where a text string can be
broken when line-wrapping. The mechanism correctly handles
punctuation and hyphenated words.
Sentence boundary analysis allows selection with correct interpretation
of periods within numbers and abbreviations, and trailing punctuation
marks such as quotation marks and parentheses.
Word boundary analysis is used by search and replace functions, as
well as within text editing applications that allow the user to
select words with a double click. Word selection provides correct
interpretation of punctuation marks within and following
words. Characters that are not part of a word, such as symbols
or punctuation marks, have word-breaks on both sides.
Character boundary analysis allows users to interact with characters
as they expect to, for example, when moving the cursor through a text
string. Character boundary analysis provides correct navigation of
through character strings, regardless of how the character is stored.
For example, an accented character might be stored as a base character
and a diacritical mark. What users consider to be a character can
differ between languages.
BreakIterator is intended for use with natural
languages only. Do not use this class to tokenize a programming language.
Examples:
Creating and using text boundaries
public static void main(String args[]) {
if (args.length == 1) {
String stringToExamine = args[0];
//print each word in order
BreakIterator boundary = BreakIterator.getWordInstance();
boundary.setText(stringToExamine);
printEachForward(boundary, stringToExamine);
//print each sentence in reverse order
boundary = BreakIterator.getSentenceInstance(Locale.US);
boundary.setText(stringToExamine);
printEachBackward(boundary, stringToExamine);
printFirst(boundary, stringToExamine);
printLast(boundary, stringToExamine);
}
}
Print each element in order
public static void printEachForward(BreakIterator boundary, String source) {
int start = boundary.first();
for (int end = boundary.next();
end != BreakIterator.DONE;
start = end, end = boundary.next()) {
System.out.println(source.substring(start,end));
}
}
Print each element in reverse order
public static void printEachBackward(BreakIterator boundary, String source) {
int end = boundary.last();
for (int start = boundary.previous();
start != BreakIterator.DONE;
end = start, start = boundary.previous()) {
System.out.println(source.substring(start,end));
}
}
Print first element
public static void printFirst(BreakIterator boundary, String source) {
int start = boundary.first();
int end = boundary.next();
System.out.println(source.substring(start,end));
}
Print last element
public static void printLast(BreakIterator boundary, String source) {
int end = boundary.last();
int start = boundary.previous();
System.out.println(source.substring(start,end));
}
Print the element at a specified position
public static void printAt(BreakIterator boundary, int pos, String source) {
int end = boundary.following(pos);
int start = boundary.previous();
System.out.println(source.substring(start,end));
}
Find the next word
public static int nextWordStartAfter(int pos, String text) {
BreakIterator wb = BreakIterator.getWordInstance();
wb.setText(text);
int last = wb.following(pos);
int current = wb.next();
while (current != BreakIterator.DONE) {
for (int p = last; p < current; p++) {
if (Character.isLetter(text.codePointAt(p))
return last;
}
last = current;
current = wb.next();
}
return BreakIterator.DONE;
}
(The iterator returned by BreakIterator.getWordInstance() is unique in that
the break positions it returns don't represent both the start and end of the
thing being iterated over. That is, a sentence-break iterator returns breaks
that each represent the end of one sentence and the beginning of the next.
With the word-break iterator, the characters between two boundaries might be a
word, or they might be the punctuation or whitespace between two words. The
above code uses a simple heuristic to determine which boundary is the beginning
of a word: If the characters between this boundary and the next boundary
include at least one letter (this can be an alphabetical letter, a CJK ideograph,
a Hangul syllable, a Kana character, etc.), then the text between this boundary
and the next is a word; otherwise, it's the material between words.)
|
| Methods Summary |
|---|
| public java.lang.Object | clone()Create a copy of this iterator
try {
return super.clone();
}
catch (CloneNotSupportedException e) {
throw new InternalError();
}
|
| private static java.text.BreakIterator | createBreakInstance(java.util.Locale where, int type, java.lang.String dataName, java.lang.String dictionaryName)
ResourceBundle bundle = getBundle(
"sun.text.resources.BreakIteratorInfo", where);
String[] classNames = bundle.getStringArray("BreakIteratorClasses");
String dataFile = bundle.getString(dataName);
try {
if (classNames[type].equals("RuleBasedBreakIterator")) {
return new RuleBasedBreakIterator(dataFile);
}
else if (classNames[type].equals("DictionaryBasedBreakIterator")) {
String dictionaryFile = bundle.getString(dictionaryName);
return new DictionaryBasedBreakIterator(dataFile, dictionaryFile);
}
else {
throw new IllegalArgumentException("Invalid break iterator class \"" +
classNames[type] + "\"");
}
}
catch (Exception e) {
throw new InternalError(e.toString());
}
|
| public abstract int | current()Return character index of the text boundary that was most recently
returned by next(), previous(), first(), or last()
|
| public abstract int | first()Return the first boundary. The iterator's current position is set
to the first boundary.
|
| public abstract int | following(int offset)Return the first boundary following the specified offset.
The value returned is always greater than the offset or
the value BreakIterator.DONE
|
| public static synchronized java.util.Locale[] | getAvailableLocales()Returns an array of all locales for which the
get*Instance methods of this class can return
localized instances.
The array returned must contain at least a Locale
instance equal to {@link java.util.Locale#US Locale.US}.
//FIX ME - this is a known bug. It should return
//all locales.
return LocaleData.getAvailableLocales("NumberPatterns");
|
| private static java.text.BreakIterator | getBreakInstance(java.util.Locale where, int type, java.lang.String dataName, java.lang.String dictionaryName)
if (iterCache[type] != null) {
BreakIteratorCache cache = (BreakIteratorCache) iterCache[type].get();
if (cache != null) {
if (cache.getLocale().equals(where)) {
return cache.createBreakInstance();
}
}
}
BreakIterator result = createBreakInstance(where,
type,
dataName,
dictionaryName);
BreakIteratorCache cache = new BreakIteratorCache(where, result);
iterCache[type] = new SoftReference(cache);
return result;
|
| private static java.util.ResourceBundle | getBundle(java.lang.String baseName, java.util.Locale locale)
return (ResourceBundle) AccessController.doPrivileged(new PrivilegedAction() {
public Object run() {
return ResourceBundle.getBundle(baseName, locale);
}
});
|
| public static java.text.BreakIterator | getCharacterInstance()Create BreakIterator for character-breaks using default locale
Returns an instance of a BreakIterator implementing character breaks.
Character breaks are boundaries of combining character sequences.
return getCharacterInstance(Locale.getDefault());
|
| public static java.text.BreakIterator | getCharacterInstance(java.util.Locale where)Create BreakIterator for character-breaks using specified locale
Returns an instance of a BreakIterator implementing character breaks.
Character breaks are boundaries of combining character sequences.
return getBreakInstance(where,
CHARACTER_INDEX,
"CharacterData",
"CharacterDictionary");
|
| protected static int | getInt(byte[] buf, int offset)
int num = buf[offset]&0xFF;
for (int i = 1; i < 4; i++) {
num = num<<8 | (buf[offset+i]&0xFF);
}
return num;
|
| public static java.text.BreakIterator | getLineInstance()Create BreakIterator for line-breaks using default locale.
Returns an instance of a BreakIterator implementing line breaks. Line
breaks are logically possible line breaks, actual line breaks are
usually determined based on display width.
LineBreak is useful for word wrapping text.
return getLineInstance(Locale.getDefault());
|
| public static java.text.BreakIterator | getLineInstance(java.util.Locale where)Create BreakIterator for line-breaks using specified locale.
Returns an instance of a BreakIterator implementing line breaks. Line
breaks are logically possible line breaks, actual line breaks are
usually determined based on display width.
LineBreak is useful for word wrapping text.
return getBreakInstance(where,
LINE_INDEX,
"LineData",
"LineDictionary");
|
| protected static long | getLong(byte[] buf, int offset)
long num = buf[offset]&0xFF;
for (int i = 1; i < 8; i++) {
num = num<<8 | (buf[offset+i]&0xFF);
}
return num;
|
| public static java.text.BreakIterator | getSentenceInstance()Create BreakIterator for sentence-breaks using default locale
Returns an instance of a BreakIterator implementing sentence breaks.
return getSentenceInstance(Locale.getDefault());
|
| public static java.text.BreakIterator | getSentenceInstance(java.util.Locale where)Create BreakIterator for sentence-breaks using specified locale
Returns an instance of a BreakIterator implementing sentence breaks.
return getBreakInstance(where,
SENTENCE_INDEX,
"SentenceData",
"SentenceDictionary");
|
| protected static short | getShort(byte[] buf, int offset)
short num = (short)(buf[offset]&0xFF);
num = (short)(num<<8 | (buf[offset+1]&0xFF));
return num;
|
| public abstract java.text.CharacterIterator | getText()Get the text being scanned
|
| public static java.text.BreakIterator | getWordInstance()Create BreakIterator for word-breaks using default locale.
Returns an instance of a BreakIterator implementing word breaks.
WordBreak is usefull for word selection (ex. double click)
return getWordInstance(Locale.getDefault());
|
| public static java.text.BreakIterator | getWordInstance(java.util.Locale where)Create BreakIterator for word-breaks using specified locale.
Returns an instance of a BreakIterator implementing word breaks.
WordBreak is usefull for word selection (ex. double click)
return getBreakInstance(where,
WORD_INDEX,
"WordData",
"WordDictionary");
|
| public boolean | isBoundary(int offset)Return true if the specified position is a boundary position.
// NOTE: This implementation probably is wrong for most situations
// because it fails to take into account the possibility that a
// CharacterIterator passed to setText() may not have a begin offset
// of 0. But since the abstract BreakIterator doesn't have that
// knowledge, it assumes the begin offset is 0. If you subclass
// BreakIterator, copy the SimpleTextBoundary implementation of this
// function into your subclass. [This should have been abstract at
// this level, but it's too late to fix that now.]
if (offset == 0)
return true;
else
return following(offset - 1) == offset;
|
| public abstract int | last()Return the last boundary. The iterator's current position is set
to the last boundary.
|
| public abstract int | next(int n)Return the nth boundary from the current boundary
|
| public abstract int | next()Return the boundary following the current boundary.
|
| public int | preceding(int offset)Return the last boundary preceding the specfied offset.
The value returned is always less than the offset or the value
BreakIterator.DONE.
// NOTE: This implementation is here solely because we can't add new
// abstract methods to an existing class. There is almost ALWAYS a
// better, faster way to do this.
int pos = following(offset);
while (pos >= offset && pos != DONE)
pos = previous();
return pos;
|
| public abstract int | previous()Return the boundary preceding the current boundary.
|
| public void | setText(java.lang.String newText)Set a new text string to be scanned. The current scan
position is reset to first().
setText(new StringCharacterIterator(newText));
|
| public abstract void | setText(java.text.CharacterIterator newText)Set a new text for scanning. The current scan
position is reset to first().
|
