Greek stemmer benchmarks ★★★★★

This page reports dictionary corpus, exact-root agreement, and runtime evidence for the independently available el-gr-default Greek model.

Dictionary size: ★★★★★. The exact count is 76,869 distinct usable word forms after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all benchmarked dictionaries in five nearly equal groups; they do not measure linguistic quality or benchmark accuracy.

The language metadata declares left-to-right writing.

All speed values are environment-specific and were measured on the hardware and JVM listed in the benchmark overview. The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java 4.4.0 snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language.

How to read this page

Start with the corpus and patch-command distribution, then compare exact-root agreement with runtime. Dictionary-size stars are contextual metadata, not an accuracy result.

Dictionary corpus

Model ID	Model version	Language	Dictionary rows	Distinct usable forms	Complete quality tokens	Already-root tokens	Changed tokens	Timing workload	JMH timing tokens
el-gr-default	1.0.0	EL_GR	11,793	76,869	80,320	11,793	68,527	changed tokens	68,527

Radixor patch-command distribution

Radixor stores the preferred transformation for each normalized dictionary word form as a compiled patch command. This distribution shows which runtime command class is selected by the trained trie for the complete default-model dictionary. The total number of preferred patch commands analyzed for this language is 80,320.

Command class	Meaning	Word forms	Share
AppendCharacterCommand	Appends one character to the end of the word form.	4,597	5.723%
BackwardCompoundCommand	Applies a multi-step backward patch made from skip, delete, insert, and replace operations.	53,371	66.448%
DeleteSuffixCommand	Deletes one or more trailing characters from the word form.	4,957	6.172%
PreserveCommand	Returns the word form unchanged because it already matches the preferred root.	11,934	14.858%
ReplaceLastCharacterCommand	Replaces the final character of the word form.	5,461	6.799%

Accuracy

Accuracy uses the complete dictionary and reports exact agreement with the dictionary root for each identified model and candidate.

Stemmer	All exact	Changed exact	Root preserved	Note
Radixor	95.703%	95.199%	98.635%	Exact model-ID benchmark; measured in this snapshot.
Official Snowball direct (Java)	0.010%	0.004%	0.042%	Official Snowball 3.1.0 generated Java stemmer; measured in this snapshot.
Lucene SnowballFilter	0.010%	0.004%	0.042%	Lucene integration of the matching Snowball algorithm; measured in this snapshot.

Speed

Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread.

The canonical timing workload uses changed tokens when available; a root-only dictionary uses its complete root-preservation corpus. Smaller populations are repeated deterministically to the timing minimum. Relative factors use the Radixor row as the baseline.

Stemmer	Benchmark method	Score ms/op	Error ms	ns/token	Relative vs Radixor	Note
Radixor	radixor[el-gr-default]	8.294	1.657	121.0	1.000	Canonical model timing workload; measured in this snapshot.
Official Snowball direct (Java)	snowballDirect[GREEK]	50.747	1.968	740.5	6.119	Official generated Java stemmer; measured in this snapshot.
Lucene SnowballFilter	luceneSnowballFilter[GREEK]	56.205	2.401	820.2	6.777	Lucene TokenStream integration; measured in this snapshot.

Interpretation notes

• The star tier reflects only relative distinct-form count among all benchmarked dictionaries.
• Values shown above come from the identified canonical benchmark snapshot.
• Runtime and exact-root agreement describe different properties and must be interpreted together.

Dictionary-family generalization conclusion

This is the language-specific conclusion from the independent radixor-generalization-v1 baseline experiment. It is intentionally separate from the wider edit-cost protocol below; values from the two frozen snapshots are not substituted for one another.

Evidence

Model el-gr-default version 1.0.0 is evaluated over five predeclared nested splits. Unseen metrics remove withheld occurrences whose normalized surface also appeared in training. Parentheses show the observed split minimum–maximum.

Training rows	Median unseen occurrences	Unseen all exact	Unseen changed exact	Unseen root preserved
10%	70,918	60.208% (59.027–61.709)	54.741% (53.440–56.626)	90.957% (90.152–92.617)
20%	62,654	62.992% (61.332–63.589)	57.807% (55.922–58.522)	92.323% (91.691–92.428)
30%	54,643	64.663% (64.102–65.469)	59.800% (58.994–60.719)	92.403% (92.169–93.188)
40%	46,615	65.678% (65.368–66.758)	60.970% (60.594–62.151)	92.560% (92.037–93.088)
50%	38,644	67.003% (66.400–67.555)	62.382% (61.713–62.966)	93.192% (92.651–93.592)
60%	30,453	68.280% (67.787–68.827)	63.740% (63.142–64.446)	93.424% (93.403–93.801)
70%	22,824	69.379% (68.405–69.738)	64.998% (63.943–65.390)	93.759% (93.211–94.026)
80%	15,114	70.762% (69.441–71.219)	66.576% (65.106–67.130)	94.125% (93.313–94.386)
90%	7,513	71.881% (70.673–72.434)	67.694% (66.472–68.281)	94.512% (93.170–95.848)

Generalization conclusion

• Median exactness on genuinely unseen changed forms moves from 54.741% at 10% training knowledge to 67.694% at 90%, a measured +12.953 percentage-point change.
• Unseen all-form exactness moves from 60.208% at 10% training knowledge to 71.881% at 90%, a measured +11.673 percentage-point change.
• Preservation of unseen already-root forms moves from 90.957% at 10% training knowledge to 94.512% at 90%, a measured +3.555 percentage-point change.
• The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora.

The complete ten-level table and split ranges remain in the complete generalization appendix; raw counters and provenance are in active machine-readable snapshot. The frozen methodology defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim.

Stemming quality

Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language EL_GR using the complete validated stemming-quality result matrix and the canonical linguistic-quality methodology, including its overlapping-group relation, dictionary modes, output policies, metrics, and ranking rules.

Radixor's model was trained from the same lexical resource that defines this benchmark's dictionary-reference relation. The result therefore measures same-resource agreement, not independent external linguistic validity. Third-party adapters receive the same evaluated forms but were not trained by this benchmark.

The tables retain both canonical dictionary-processing modes and every applicable output policy without redefining them on each language page. Download the complete machine-readable result snapshot.

Evaluation scope and key findings

The default model is el-gr-default, loaded from classpath resource org/egothor/stemmer/models/el-gr-default/stemmer.gz. The following findings compare only deterministic PRIMARY_OUTPUT rows over identical included groups; candidate policies are reported separately as capability analyses.

• ALL_WORDS: Radixor ranks first by balanced accuracy at 0.978296 among 4 deterministic stemmers. The runner-up is SNOWBALL GREEK DIRECT at 0.871412, a difference of 0.106884. This rank does not imply leadership in throughput or every secondary metric.
• LOWERCASE_GROUPS_ONLY: Radixor ranks first by balanced accuracy at 0.978296 among 4 deterministic stemmers. The runner-up is SNOWBALL GREEK DIRECT at 0.871412, a difference of 0.106884. This rank does not imply leadership in throughput or every secondary metric.

ALL_WORDS

This mode contains 6 result rows, 4 evaluated stemmers, and 3 output policies. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Where at least two deterministic rows are available, PRIMARY_OUTPUT and ALL_CANDIDATES rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. ANY_CANDIDATE has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.

PRIMARY_OUTPUT ranking

Rank	Stemmer	Balanced accuracy	Over-stemming (OI)	Under-stemming (UI)
1	Radixor	0.978296	0.000000%	4.340735%
2	SNOWBALL GREEK DIRECT	0.871412	0.002225%	25.715374%
3	SNOWBALL GREEK LUCENE FILTER	0.871404	0.002225%	25.716917%
4	GREEK LUCENE GREEK STEM FILTER	0.859295	0.001746%	28.139270%

Classification metrics

Rank	Stemmer	Output policy	Precision	Recall	Specificity	Balanced accuracy	Pairwise accuracy	Error rate
1	Radixor	PRIMARY_OUTPUT	1.000000	0.956593	1.000000	0.978296	0.999994	0.000006
2	SNOWBALL GREEK DIRECT	PRIMARY_OUTPUT	0.814589	0.742846	0.999978	0.871412	0.999944	0.000056
3	SNOWBALL GREEK LUCENE FILTER	PRIMARY_OUTPUT	0.814593	0.742831	0.999978	0.871404	0.999944	0.000056
4	GREEK LUCENE GREEK STEM FILTER	PRIMARY_OUTPUT	0.844112	0.718607	0.999983	0.859295	0.999946	0.000054

Pair-relation metrics

Rank	Stemmer	Output policy	F0.5	F1	F2	Jaccard	Fowlkes–Mallows	MCC
1	Radixor	PRIMARY_OUTPUT	0.991006	0.977815	0.964970	0.956593	0.978056	0.978053
2	SNOWBALL GREEK DIRECT	PRIMARY_OUTPUT	0.799153	0.777065	0.756166	0.635410	0.777891	0.777863
3	SNOWBALL GREEK LUCENE FILTER	PRIMARY_OUTPUT	0.799152	0.777059	0.756154	0.635401	0.777885	0.777857
4	GREEK LUCENE GREEK STEM FILTER	PRIMARY_OUTPUT	0.815623	0.776320	0.740631	0.634414	0.778836	0.778809

Raw pair counts

Rank	Stemmer	Output policy	TP	FP	FN	TN	Over error / possible	Under error / possible
1	Radixor	PRIMARY_OUTPUT	371840	0	16873	2953994433	0 / 2953994433	16873 / 388713
2	SNOWBALL GREEK DIRECT	PRIMARY_OUTPUT	288754	65724	99959	2953928709	65724 / 2953994433	99959 / 388713
3	SNOWBALL GREEK LUCENE FILTER	PRIMARY_OUTPUT	288748	65721	99965	2953928712	65721 / 2953994433	99965 / 388713
4	GREEK LUCENE GREEK STEM FILTER	PRIMARY_OUTPUT	279332	51586	109381	2953942847	51586 / 2953994433	109381 / 388713

ANY_CANDIDATE oracle bounds

These results are measured, not missing. ANY_CANDIDATE answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically not applicable, rather than unknown.

Stemmer	Optimistic over-stemming (OI)	Optimistic under-stemming (UI)
Radixor	0.000000%	0.000000%

Oracle-bound pair counts

Stemmer	Unavoidable over errors / gold-negative pairs	Unrepairable under errors / gold-related pairs
Radixor	0 / 2953994433	0 / 388713

ALL_CANDIDATES result (no same-language comparator)

Rank	Stemmer	Balanced accuracy	Over-stemming (OI)	Under-stemming (UI)
n/a	Radixor	1.000000	0.000000%	0.000000%

Classification metrics

Rank	Stemmer	Output policy	Precision	Recall	Specificity	Balanced accuracy	Pairwise accuracy	Error rate
n/a	Radixor	ALL_CANDIDATES	1.000000	1.000000	1.000000	1.000000	1.000000	0.000000

Pair-relation metrics

Rank	Stemmer	Output policy	F0.5	F1	F2	Jaccard	Fowlkes–Mallows	MCC
n/a	Radixor	ALL_CANDIDATES	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000

Raw pair counts

Rank	Stemmer	Output policy	TP	FP	FN	TN	Over error / possible	Under error / possible
n/a	Radixor	ALL_CANDIDATES	388713	0	0	2953994433	0 / 2953994433	0 / 388713

Multi-output analysis

Alternative candidates are capability analyses, not replacements for the deterministic comparison.

Stemmer	Under pairs repaired	Best-case over pairs avoided	All-candidate collisions added	Multi-candidate forms	Multi-candidate share	Maximum candidates	Total candidate assignments
Radixor	16873	0	0	2338	3.041538%	827	80320

LOWERCASE_GROUPS_ONLY

This mode contains 6 result rows, 4 evaluated stemmers, and 3 output policies. Applied-row and form counts are shown per row because adapters share the language corpus but policy rows remain independently auditable. Where at least two deterministic rows are available, PRIMARY_OUTPUT and ALL_CANDIDATES rankings are ordered by unrounded balanced accuracy, followed by MCC, F1, over-stemming rate, over-stemming count, under-stemming rate, and stemmer. ANY_CANDIDATE has no single rank metric and is listed alphabetically. Balanced accuracy is a navigation metric, not a universally authoritative quality score.

PRIMARY_OUTPUT ranking

Rank	Stemmer	Balanced accuracy	Over-stemming (OI)	Under-stemming (UI)
1	Radixor	0.978296	0.000000%	4.340735%
2	SNOWBALL GREEK DIRECT	0.871412	0.002225%	25.715374%
3	SNOWBALL GREEK LUCENE FILTER	0.871404	0.002225%	25.716917%
4	GREEK LUCENE GREEK STEM FILTER	0.859295	0.001746%	28.139270%

Classification metrics

Rank	Stemmer	Output policy	Precision	Recall	Specificity	Balanced accuracy	Pairwise accuracy	Error rate
1	Radixor	PRIMARY_OUTPUT	1.000000	0.956593	1.000000	0.978296	0.999994	0.000006
2	SNOWBALL GREEK DIRECT	PRIMARY_OUTPUT	0.814589	0.742846	0.999978	0.871412	0.999944	0.000056
3	SNOWBALL GREEK LUCENE FILTER	PRIMARY_OUTPUT	0.814593	0.742831	0.999978	0.871404	0.999944	0.000056
4	GREEK LUCENE GREEK STEM FILTER	PRIMARY_OUTPUT	0.844112	0.718607	0.999983	0.859295	0.999946	0.000054

Pair-relation metrics

Rank	Stemmer	Output policy	F0.5	F1	F2	Jaccard	Fowlkes–Mallows	MCC
1	Radixor	PRIMARY_OUTPUT	0.991006	0.977815	0.964970	0.956593	0.978056	0.978053
2	SNOWBALL GREEK DIRECT	PRIMARY_OUTPUT	0.799153	0.777065	0.756166	0.635410	0.777891	0.777863
3	SNOWBALL GREEK LUCENE FILTER	PRIMARY_OUTPUT	0.799152	0.777059	0.756154	0.635401	0.777885	0.777857
4	GREEK LUCENE GREEK STEM FILTER	PRIMARY_OUTPUT	0.815623	0.776320	0.740631	0.634414	0.778836	0.778809

Raw pair counts

Rank	Stemmer	Output policy	TP	FP	FN	TN	Over error / possible	Under error / possible
1	Radixor	PRIMARY_OUTPUT	371840	0	16873	2953994433	0 / 2953994433	16873 / 388713
2	SNOWBALL GREEK DIRECT	PRIMARY_OUTPUT	288754	65724	99959	2953928709	65724 / 2953994433	99959 / 388713
3	SNOWBALL GREEK LUCENE FILTER	PRIMARY_OUTPUT	288748	65721	99965	2953928712	65721 / 2953994433	99965 / 388713
4	GREEK LUCENE GREEK STEM FILTER	PRIMARY_OUTPUT	279332	51586	109381	2953942847	51586 / 2953994433	109381 / 388713

ANY_CANDIDATE oracle bounds

These results are measured, not missing. ANY_CANDIDATE answers two separate optimistic questions for each pair: a gold-related pair avoids under-stemming when the candidate sets intersect, while a gold-negative pair avoids over-stemming when some non-colliding candidate selection exists. The oracle may choose a different candidate for the same word in different pairs. Consequently, these decisions do not form one globally realizable predicted relation or one TP/FP/FN/TN confusion matrix. Balanced accuracy, F-scores, Jaccard, Fowlkes–Mallows, and MCC are therefore mathematically not applicable, rather than unknown.

Stemmer	Optimistic over-stemming (OI)	Optimistic under-stemming (UI)
Radixor	0.000000%	0.000000%

Oracle-bound pair counts

Stemmer	Unavoidable over errors / gold-negative pairs	Unrepairable under errors / gold-related pairs
Radixor	0 / 2953994433	0 / 388713

ALL_CANDIDATES result (no same-language comparator)

Rank	Stemmer	Balanced accuracy	Over-stemming (OI)	Under-stemming (UI)
n/a	Radixor	1.000000	0.000000%	0.000000%

Classification metrics

Rank	Stemmer	Output policy	Precision	Recall	Specificity	Balanced accuracy	Pairwise accuracy	Error rate
n/a	Radixor	ALL_CANDIDATES	1.000000	1.000000	1.000000	1.000000	1.000000	0.000000

Pair-relation metrics

Rank	Stemmer	Output policy	F0.5	F1	F2	Jaccard	Fowlkes–Mallows	MCC
n/a	Radixor	ALL_CANDIDATES	1.000000	1.000000	1.000000	1.000000	1.000000	1.000000

Raw pair counts

Rank	Stemmer	Output policy	TP	FP	FN	TN	Over error / possible	Under error / possible
n/a	Radixor	ALL_CANDIDATES	388713	0	0	2953994433	0 / 2953994433	0 / 388713

Multi-output analysis

Alternative candidates are capability analyses, not replacements for the deterministic comparison.

Stemmer	Under pairs repaired	Best-case over pairs avoided	All-candidate collisions added	Multi-candidate forms	Multi-candidate share	Maximum candidates	Total candidate assignments
Radixor	16873	0	0	2338	3.041538%	827	80320

Method and interpretation boundaries

The linguistic-quality methodology defines the overlapping gold relation, both dictionary-processing modes, all three output policies, confusion counts, formulas, undefined denominators, aggregation, ranking, and inapplicable partition metrics. The candidate-policy reference explains why ANY_CANDIDATE is an oracle-assisted capability bound rather than deterministic runtime behavior.

This page preserves the language-specific raw counts, metrics, candidate distributions, comparison availability, and caveats. The machine-readable CSV remains authoritative for every field.

Provenance

• Authoritative source: docs/benchmarks/data/stemming-quality-2026-09-13.csv
• Source SHA-256: d41e00160cda44758e806c37d210f2d9b90b1ebad22eebe5ece89a40f4ed9ab0
• Evaluation command: ./gradlew stemmingQuality --no-daemon
• Dictionary language: EL_GR
• Processing modes: ALL_WORDS, LOWERCASE_GROUPS_ONLY
• Stemmer versions and transitive artifacts: resolved by the repository's JMH Gradle configuration and gradle.lockfile
• Model ID, version, and SHA-256: recorded in every CSV row
• Run date, core source state, JDK, operating system, and hardware: recorded on the benchmark environment page