Package org.egothor.stemmer

Class StemmerDictionaryParser

java.lang.Object

org.egothor.stemmer.StemmerDictionaryParser

public final class StemmerDictionaryParser
extends Object

Parser of line-oriented stemmer dictionary files.

Each non-empty logical line uses a tab-separated values layout. The first column is interpreted as the canonical stem, and every following tab-separated column on the same line is interpreted as a variant belonging to that stem.

Input line case normalization is controlled by CaseProcessingMode. Leading and trailing whitespace around each column is ignored.

The parser supports line remarks and trailing remarks. The remark markers # and // terminate the logical content of the line, and the remainder of that line is ignored.

Dictionary items containing any Unicode whitespace character are currently not supported. Such items are ignored and reported through a single warning-level log entry per physical line together with the source line number, the normalized stem column, and the list of ignored items from that line.

This class is intentionally stateless and allocation-light so it can be used both by runtime loading and by offline compilation tooling.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	StemmerDictionaryParser.EntryHandler	Callback receiving one parsed dictionary line.
static final record	StemmerDictionaryParser.ParseStatistics	Immutable parsing statistics.

Method Summary

Modifier and Type	Method	Description
static StemmerDictionaryParser.ParseStatistics	parse(Reader reader, String sourceDescription, CaseProcessingMode caseProcessingMode, StemmerDictionaryParser.EntryHandler entryHandler)	Parses a dictionary from a reader.
static StemmerDictionaryParser.ParseStatistics	parse(Reader reader, String sourceDescription, StemmerDictionaryParser.EntryHandler entryHandler)	Parses a dictionary from a reader.
static StemmerDictionaryParser.ParseStatistics	parse(String fileName, CaseProcessingMode caseProcessingMode, StemmerDictionaryParser.EntryHandler entryHandler)	Parses a dictionary file from a path string.
static StemmerDictionaryParser.ParseStatistics	parse(String fileName, StemmerDictionaryParser.EntryHandler entryHandler)	Parses a dictionary file from a path string.
static StemmerDictionaryParser.ParseStatistics	parse(Path path, CaseProcessingMode caseProcessingMode, StemmerDictionaryParser.EntryHandler entryHandler)	Parses a dictionary file from a filesystem path.
static StemmerDictionaryParser.ParseStatistics	parse(Path path, StemmerDictionaryParser.EntryHandler entryHandler)	Parses a dictionary file from a filesystem path.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

parse

public static StemmerDictionaryParser.ParseStatistics parse(Path path,
 StemmerDictionaryParser.EntryHandler entryHandler)
                                                     throws IOException

Parses a dictionary file from a filesystem path.

Parameters:

path - dictionary file path

entryHandler - handler receiving parsed entries

Returns:

parsing statistics

Throws:

NullPointerException - if any argument is null

IOException - if reading fails

parse

public static StemmerDictionaryParser.ParseStatistics parse(Path path,
 CaseProcessingMode caseProcessingMode,
 StemmerDictionaryParser.EntryHandler entryHandler)
                                                     throws IOException

Parses a dictionary file from a filesystem path.

Parameters:

path - dictionary file path

caseProcessingMode - case processing mode

entryHandler - handler receiving parsed entries

Returns:

parsing statistics

Throws:

NullPointerException - if any argument is null

IOException - if reading fails

parse

public static StemmerDictionaryParser.ParseStatistics parse(String fileName,
 StemmerDictionaryParser.EntryHandler entryHandler)
                                                     throws IOException

Parses a dictionary file from a path string.

Parameters:

fileName - dictionary file name or path string

entryHandler - handler receiving parsed entries

Returns:

parsing statistics

Throws:

NullPointerException - if any argument is null

IOException - if reading fails

parse

public static StemmerDictionaryParser.ParseStatistics parse(String fileName,
 CaseProcessingMode caseProcessingMode,
 StemmerDictionaryParser.EntryHandler entryHandler)
                                                     throws IOException

Parses a dictionary file from a path string.

Parameters:

fileName - dictionary file name or path string

caseProcessingMode - case processing mode

entryHandler - handler receiving parsed entries

Returns:

parsing statistics

Throws:

NullPointerException - if any argument is null

IOException - if reading fails

parse

public static StemmerDictionaryParser.ParseStatistics parse(Reader reader,
 String sourceDescription,
 StemmerDictionaryParser.EntryHandler entryHandler)
                                                     throws IOException

Parses a dictionary from a reader.

Parameters:

reader - source reader

sourceDescription - logical source description for diagnostics

entryHandler - handler receiving parsed entries

Returns:

parsing statistics

Throws:

NullPointerException - if any argument is null

IOException - if reading or handler processing fails

parse

public static StemmerDictionaryParser.ParseStatistics parse(Reader reader,
 String sourceDescription,
 CaseProcessingMode caseProcessingMode,
 StemmerDictionaryParser.EntryHandler entryHandler)
                                                     throws IOException

Parses a dictionary from a reader.

Parameters:

reader - source reader

sourceDescription - logical source description for diagnostics

caseProcessingMode - case processing mode

entryHandler - handler receiving parsed entries

Returns:

parsing statistics

Throws:

NullPointerException - if any argument is null

IOException - if reading or handler processing fails