diff --git a/README.md b/README.md index eac685d..9bb3b56 100644 --- a/README.md +++ b/README.md @@ -98,7 +98,13 @@ final FrequencyTrie polish = `Language.PL_PL` selects the documented default `pl-pl-unimorph`. The optional `pl-pl-polimorf` model requires its own runtime artifact and explicit selection; adding it does not change the default. See [Model Selection and Loading](docs/model-selection-and-loading.md) for complete executable examples and [Stemmer Models](docs/stemmer-models.md) for artifact concepts. -`radixor-models-standard` is a POM-only runtime aggregate: it brings the 20 default model JARs transitively but publishes no empty aggregate JAR. `radixor-models-bom` is the separate POM-only Maven dependency BOM for version management; importing it alone adds no model. The root CycloneDX SBOM report is unrelated to that dependency BOM. +`radixor-models-standard` is a POM-only runtime aggregate: it brings the +original 20 standard model JARs transitively but publishes no empty aggregate +JAR. The other 123 language defaults and optional PoliMorf remain individually +installable and are deliberately outside that aggregate. +`radixor-models-bom` is the separate POM-only Maven dependency BOM for version +management across all 144 user-facing models; importing it alone adds no model. +The root CycloneDX SBOM report is unrelated to that dependency BOM. ```java final FrequencyTrie polimorf = @@ -141,16 +147,16 @@ Radixor performance is best read together with stemming quality. The English dic | Used rows | Actual row ratio | All exact | Changed exact | Root preserved | Speed ms/op | Error ms | ns/token | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 100.000% | 97.668% | 98.110% | 97.552% | 20.425 | 3.636 | 97.9 | -| 90% | 90.000% | 97.239% | 95.821% | 97.612% | 17.779 | 1.827 | 85.3 | -| 80% | 80.000% | 96.827% | 93.673% | 97.656% | 15.343 | 1.321 | 73.6 | -| 70% | 70.000% | 96.392% | 91.430% | 97.695% | 16.444 | 2.027 | 78.9 | -| 60% | 60.000% | 95.935% | 89.244% | 97.693% | 14.330 | 1.350 | 68.7 | -| 50% | 50.000% | 95.453% | 86.979% | 97.678% | 14.953 | 2.625 | 71.7 | -| 40% | 40.000% | 94.939% | 84.667% | 97.638% | 12.919 | 1.155 | 62.0 | -| 30% | 30.000% | 94.398% | 82.443% | 97.538% | 12.166 | 1.305 | 58.3 | -| 20% | 20.000% | 93.821% | 80.174% | 97.406% | 11.549 | 1.535 | 55.4 | -| 10% | 10.000% | 93.057% | 77.327% | 97.190% | 14.360 | 3.524 | 68.9 | +| 100% | 100.000% | 97.668% | 98.110% | 97.552% | 25.107 | 4.358 | 120.4 | +| 90% | 90.000% | 97.239% | 95.821% | 97.612% | 24.021 | 1.865 | 115.2 | +| 80% | 80.000% | 96.827% | 93.673% | 97.656% | 23.593 | 2.656 | 113.1 | +| 70% | 70.000% | 96.392% | 91.430% | 97.695% | 18.965 | 1.559 | 90.9 | +| 60% | 60.000% | 95.935% | 89.244% | 97.693% | 18.588 | 1.040 | 89.1 | +| 50% | 50.000% | 95.453% | 86.979% | 97.678% | 18.479 | 2.304 | 88.6 | +| 40% | 40.000% | 94.939% | 84.667% | 97.638% | 17.345 | 1.504 | 83.2 | +| 30% | 30.000% | 94.398% | 82.443% | 97.538% | 17.164 | 1.252 | 82.3 | +| 20% | 20.000% | 93.821% | 80.174% | 97.406% | 14.912 | 1.512 | 71.5 | +| 10% | 10.000% | 93.057% | 77.327% | 97.190% | 14.937 | 2.320 | 71.6 | Column meanings: @@ -163,23 +169,23 @@ Column meanings: - `Error ms` is the JMH score error converted to milliseconds. - `ns/token` is average nanoseconds per changed token in that operation. -The contracted trie result is materially stronger than the older uncontracted profile: full English coverage reaches 97.668% all-token exactness and 98.110% changed-token exactness at 97.9 ns/token, while even a 10% deterministic dictionary slice remains at 93.057% all-token exactness and 77.327% changed-token exactness at 68.9 ns/token. This is why Radixor benchmark results are documented with both speed and quality instead of a single Porter speed badge. +The contracted trie result is materially stronger than the older uncontracted profile: full English coverage reaches 97.668% all-token exactness and 98.110% changed-token exactness at 120.4 ns/token, while even a 10% deterministic dictionary slice remains at 93.057% all-token exactness and 77.327% changed-token exactness at 71.6 ns/token. This is why Radixor benchmark results are documented with both speed and quality instead of a single Porter speed badge. The English curve evaluates the complete dictionary, so it intentionally mixes trained and withheld rows. The separate -[20-language dictionary-family generalization report](docs/benchmarks/generalization.md) +[143-language dictionary-family generalization report](docs/benchmarks/generalization.md) isolates held-out rows, removes surface forms duplicated in training, and reports -five frozen splits at every 10% coverage step. It contains 1,000 raw scenarios +five frozen splits at every 10% coverage step. It contains 8,300 raw scenarios with exact model provenance and makes clear where transfer is strong—and where a small resource does not support a broad generalization claim. The complementary [edit-cost sensitivity experiment](docs/benchmarks/edit-cost-sensitivity.md) expands 16,700 physically measured exact command classes into a validated 234,000-observation -logical matrix. It finds that suitable relative edit costs and their structural effect are -language-dependent. Each [language benchmark page](docs/benchmarks/languages/index.md) therefore -publishes its own 10%–90% knowledge curve, command-equivalence evidence, selected-cost effect, -factor associations, and bounded conclusion; exploratory non-baseline settings are not presented -as production defaults without external validation. +logical matrix over the historical 20-language cohort. It finds that suitable relative edit costs +and their structural effect are language-dependent. Those 20 language pages retain their own +10%–90% knowledge curve, command-equivalence evidence, selected-cost effect, factor associations, +and bounded conclusion; no edit-cost result is extrapolated to the new dictionaries, and +exploratory non-baseline settings are not presented as production defaults without external validation. For benchmark scope, workload design, environment, commands, report locations, and interpretation guidance, see [Benchmarking](docs/benchmarking.md). @@ -390,9 +396,10 @@ Egothor showed that stemming could be both algorithmic and compact. Stempel prov The established `org.egothor:radixor` artifact remains the algorithmic core and contains no language-model data. From version 4 onward, applications explicitly add individual `org.egothor:radixor-model-` runtime artifacts or the optional metadata-only `org.egothor:radixor-models-standard` aggregate. Polish defaults to `pl-pl-unimorph`; `pl-pl-polimorf` is opt-in. See [Stemmer Models](docs/stemmer-models.md) and [Migration and Backward Compatibility](docs/migration-and-backward-compatibility.md). -Radixor Java software remains licensed under BSD-3-Clause. UniMorph-derived model data is -distributed under CC BY-SA 3.0, with upstream attribution, the canonical license URI, Radixor -transformations, and Leo Galambos's limited contribution notice carried by each model artifact. +Radixor Java software remains licensed under BSD-3-Clause. Each UniMorph-derived model artifact +declares its audited file-applicable license: primarily CC BY-SA 3.0, with CC BY-SA 4.0, CC BY +4.0, and LGPLLR exceptions where required upstream. Upstream attribution, the canonical license +URI, Radixor transformations, and Leo Galambos's limited contribution notice travel with it. PoliMorf model data retains its separate BSD-2-Clause license. There is no project-wide CC license directory because the root artifact contains no model data. diff --git a/build-logic/src/main/groovy/org/egothor/radixor/PrepareModelResourcesTask.groovy b/build-logic/src/main/groovy/org/egothor/radixor/PrepareModelResourcesTask.groovy index 0fb661d..e13b5d3 100644 --- a/build-logic/src/main/groovy/org/egothor/radixor/PrepareModelResourcesTask.groovy +++ b/build-logic/src/main/groovy/org/egothor/radixor/PrepareModelResourcesTask.groovy @@ -56,7 +56,6 @@ abstract class PrepareModelResourcesTask extends DefaultTask { @InputFile @PathSensitive(PathSensitivity.RELATIVE) abstract RegularFileProperty getVersionFile() @Optional @InputFile @PathSensitive(PathSensitivity.RELATIVE) abstract RegularFileProperty getLicenseFile() @Optional @InputFile @PathSensitive(PathSensitivity.RELATIVE) abstract RegularFileProperty getNoticeFile() - @Input abstract Property getShareAlike() @Input abstract MapProperty getDescriptorValues() @OutputDirectory abstract DirectoryProperty getGeneratedDirectory() @@ -80,12 +79,13 @@ abstract class PrepareModelResourcesTask extends DefaultTask { Files.createDirectories(index.parent) Files.writeString(index, "META-INF/radixor/models/${id}.properties\n") - if (shareAlike.get()) { + if (noticeFile.isPresent()) { final Path notice = generated.resolve("META-INF/NOTICE/${id}-data.txt") Files.createDirectories(notice.parent) Files.copy(noticeFile.get().asFile.toPath(), notice, StandardCopyOption.REPLACE_EXISTING) - } else { - final Path license = generated.resolve('META-INF/LICENSES/PoliMorf-BSD-2-Clause.txt') + } + if (licenseFile.isPresent()) { + final Path license = generated.resolve("META-INF/LICENSES/${licenseFile.get().asFile.name}") Files.createDirectories(license.parent) Files.copy(licenseFile.get().asFile.toPath(), license, StandardCopyOption.REPLACE_EXISTING) } @@ -102,7 +102,7 @@ model.default=${value['model.default']} model.format=radixor-dictionary-tsv-gzip model.formatVersion=1 model.sha256=${checksum} -model.rightToLeft=${['FA_IR', 'HE_IL', 'YI'].contains(value['model.language'])} +model.rightToLeft=${value['model.rightToLeft']} model.caseProcessing=LOWERCASE_WITH_LOCALE_ROOT model.diacriticProcessing=AS_IS model.storeOriginal=true diff --git a/build-logic/src/main/groovy/org/egothor/radixor/RadixorModelExtension.groovy b/build-logic/src/main/groovy/org/egothor/radixor/RadixorModelExtension.groovy index 12e1e2f..2480ebd 100644 --- a/build-logic/src/main/groovy/org/egothor/radixor/RadixorModelExtension.groovy +++ b/build-logic/src/main/groovy/org/egothor/radixor/RadixorModelExtension.groovy @@ -50,6 +50,12 @@ abstract class RadixorModelExtension { /** Whether this is the documented default for its language. */ abstract Property getDefaultModel() + /** Whether source identity and output bytes are governed by the pinned UniMorph manifest. */ + abstract Property getManifestManaged() + + /** Whether the model language is conventionally written right-to-left. */ + abstract Property getRightToLeft() + /** Source dictionary name. */ abstract Property getSourceName() @@ -96,6 +102,8 @@ abstract class RadixorModelExtension { @Inject RadixorModelExtension(final ObjectFactory objects) { defaultModel.convention(false) + manifestManaged.convention(false) + rightToLeft.convention(false) sourceVersion.convention('unavailable') sourceLicense.convention('LicenseRef-Radixor-Stemmer-Data') licenseFileName.convention('LICENSE-stemmer-data.txt') diff --git a/build-logic/src/main/groovy/org/egothor/radixor/RadixorModelPlugin.groovy b/build-logic/src/main/groovy/org/egothor/radixor/RadixorModelPlugin.groovy index e586894..ad5a2fe 100644 --- a/build-logic/src/main/groovy/org/egothor/radixor/RadixorModelPlugin.groovy +++ b/build-logic/src/main/groovy/org/egothor/radixor/RadixorModelPlugin.groovy @@ -80,7 +80,8 @@ final class RadixorModelPlugin implements Plugin { versionFile = project.layout.projectDirectory.file('model-version.txt') modelId = model.modelId moduleName = project.name - shareAlike = model.sourceLicense.map { String license -> license == 'CC-BY-SA-3.0' } + shareAlike = model.sourceLicense.map { String license -> license != 'BSD-2-Clause' } + manifestManaged = model.manifestManaged metadata.put('source.project', model.sourceProject) metadata.put('source.repository', model.sourceRepository) metadata.put('source.dataset', model.sourceDataset) @@ -99,12 +100,12 @@ final class RadixorModelPlugin implements Plugin { dependsOn(validate) dictionaryFile = project.layout.projectDirectory.file('src/modelInput/stemmer.gz') versionFile = project.layout.projectDirectory.file('model-version.txt') - shareAlike = model.sourceLicense.map { String license -> license == 'CC-BY-SA-3.0' } generatedDirectory = project.layout.buildDirectory.dir('generated/modelResources') descriptorValues.put('model.id', model.modelId) descriptorValues.put('model.language', model.language) descriptorValues.put('model.displayName', model.displayName) descriptorValues.put('model.default', model.defaultModel.map(String::valueOf)) + descriptorValues.put('model.rightToLeft', model.rightToLeft.map(String::valueOf)) descriptorValues.put('source.name', model.sourceName) descriptorValues.put('source.version', model.sourceVersion) descriptorValues.put('source.project', model.sourceProject) @@ -119,19 +120,25 @@ final class RadixorModelPlugin implements Plugin { descriptorValues.put('transformations.summary', model.transformationsSummary) } project.afterEvaluate { - final boolean shareAlike = model.sourceLicense.get() == 'CC-BY-SA-3.0' + final boolean shareAlike = model.sourceLicense.get() != 'BSD-2-Clause' if (shareAlike) { final def notice = project.layout.projectDirectory.file("src/modelInput/${model.noticeFileName.get()}") validate.configure { noticeFile = notice } prepare.configure { noticeFile = notice } - } else { + } + if (!shareAlike || model.sourceLicense.get() == 'LGPLLR') { final def license = project.layout.projectDirectory.file("src/modelInput/${model.licenseFileName.get()}") validate.configure { licenseFile = license } prepare.configure { licenseFile = license } } } project.tasks.named('processResources', Copy).configure { dependsOn(prepare); duplicatesStrategy = DuplicatesStrategy.FAIL } - project.tasks.named('sourcesJar', Jar).configure { dependsOn(prepare); exclude('**/stemmer.gz') } + project.tasks.named('sourcesJar', Jar).configure { + dependsOn(prepare) + if (model.sourceLicense.get() != 'LGPLLR') { + exclude('**/stemmer.gz') + } + } project.tasks.named('javadocJar', Jar).configure { exclude('**/stemmer.gz') } project.tasks.named('jar', Jar).configure { archiveBaseName.set("radixor-model-${project.name}") @@ -155,9 +162,12 @@ final class RadixorModelPlugin implements Plugin { final File archive = project.tasks.named('jar', Jar).get().archiveFile.get().asFile final List names = [] final String resource = "org/egothor/stemmer/models/${model.modelId.get()}/stemmer.gz" - final boolean shareAlike = model.sourceLicense.get() == 'CC-BY-SA-3.0' - final String licenseResource = 'META-INF/LICENSES/PoliMorf-BSD-2-Clause.txt' - final File sourceLicense = shareAlike ? null : project.file("src/modelInput/${model.licenseFileName.get()}") + final boolean shareAlike = model.sourceLicense.get() != 'BSD-2-Clause' + final boolean requiresFullLicense = !shareAlike || model.sourceLicense.get() == 'LGPLLR' + final String licenseResource = requiresFullLicense + ? "META-INF/LICENSES/${model.licenseFileName.get()}" : null + final File sourceLicense = requiresFullLicense + ? project.file("src/modelInput/${model.licenseFileName.get()}") : null final File sourceNotice = shareAlike ? project.file("src/modelInput/${model.noticeFileName.get()}") : null final String noticeResource = "META-INF/NOTICE/${model.modelId.get()}-data.txt" @@ -170,7 +180,7 @@ final class RadixorModelPlugin implements Plugin { if (entry != null) { packagedChecksum = sha256(zip.getInputStream(entry).bytes) } - final def licenseEntry = zip.getEntry(licenseResource) + final def licenseEntry = licenseResource == null ? null : zip.getEntry(licenseResource) if (licenseEntry != null) { packagedLicenseChecksum = sha256(zip.getInputStream(licenseEntry).bytes) } @@ -187,7 +197,10 @@ final class RadixorModelPlugin implements Plugin { } if (shareAlike) { requireMatchingChecksum('notice', noticeResource, sha256(sourceNotice), packagedNoticeChecksum) - validateUniMorphJarContents(names) + if (requiresFullLicense) { + requireMatchingChecksum('license', licenseResource, sha256(sourceLicense), packagedLicenseChecksum) + } + validateUniMorphJarContents(names, licenseResource) } else { requireMatchingChecksum('license', licenseResource, sha256(sourceLicense), packagedLicenseChecksum) validatePoliMorfJarContents(names) @@ -198,8 +211,21 @@ final class RadixorModelPlugin implements Plugin { [project.tasks.named('sourcesJar', Jar).get(), project.tasks.named('javadocJar', Jar).get()].each { Jar task -> final File documentationArchive = task.archiveFile.get().asFile new java.util.zip.ZipFile(documentationArchive).withCloseable { zip -> - if (zip.entries().any { entry -> entry.name.endsWith('/stemmer.gz') || entry.name == 'stemmer.gz' }) { - throw new GradleException("Documentation artifact ${documentationArchive.name} must not contain a model dictionary.") + final List documentationNames = [] + zip.entries().each { entry -> documentationNames.add(entry.name) } + final boolean containsDictionary = documentationNames.any { + String name -> name.endsWith('/stemmer.gz') || name == 'stemmer.gz' + } + final boolean sourceFormRequired = model.sourceLicense.get() == 'LGPLLR' + && task.name == 'sourcesJar' + if (containsDictionary != sourceFormRequired) { + final String requirement = sourceFormRequired + ? 'must contain the LGPLLR legible dictionary form' + : 'must not contain a model dictionary' + throw new GradleException("Documentation artifact ${documentationArchive.name} ${requirement}.") + } + if (sourceFormRequired && !documentationNames.contains(licenseResource)) { + throw new GradleException("Documentation artifact ${documentationArchive.name} must contain ${licenseResource}.") } } } @@ -221,7 +247,7 @@ final class RadixorModelPlugin implements Plugin { publication.pom { name.set("Radixor model ${project.name}") description.set(model.displayName.zip(model.sourceLicense) { String displayName, String licenseId -> - final String material = licenseId == 'CC-BY-SA-3.0' + final String material = licenseId != 'BSD-2-Clause' ? 'See the packaged model-specific notice.' : 'See the packaged model-data license.' return "${displayName}. This artifact contains Radixor-derived model data licensed under ${licenseId}; " @@ -333,7 +359,13 @@ final class RadixorModelPlugin implements Plugin { throw new GradleException("Required model metadata is missing: ${key}") } } - validateRevisionMetadata(model.sourceRevision.get(), model.sourceRevisionStatus.get()) + validateManifestRevisionMetadata(model.sourceRevision.get(), model.sourceRevisionStatus.get(), + model.manifestManaged.get()) + final Set supportedLicenses = [ + 'CC-BY-SA-3.0', 'CC-BY-SA-4.0', 'CC-BY-4.0', 'LGPLLR', 'BSD-2-Clause'] as Set + if (!supportedLicenses.contains(model.sourceLicense.get())) { + throw new GradleException("Unsupported or unknown model-data license: ${model.sourceLicense.get()}") + } } /** Accepts an exact recorded revision or the explicit legacy-import sentinel, but never an absent status. */ @@ -353,23 +385,45 @@ final class RadixorModelPlugin implements Plugin { } } + /** Rejects the legacy revision sentinel for manifest-managed imports. */ + static void validateManifestRevisionMetadata(final String revision, final String status, + final boolean manifestManaged) { + validateRevisionMetadata(revision, status) + if (manifestManaged && revision == 'not-recorded-in-legacy-import') { + throw new GradleException('Manifest-managed UniMorph models require an exact pinned source revision.') + } + } + /** Validates the model-specific attribution and ShareAlike notice. */ static void validateShareAlikeNotice(final File notice, final RadixorModelExtension model) { validateShareAlikeNoticeText(notice.getText('UTF-8'), notice.toString(), model.modelId.get(), - model.sourceRepository.get(), model.sourceLicenseUri.get(), model.sourceRevision.get(), + model.sourceRepository.get(), model.sourceLicense.get(), model.sourceLicenseUri.get(), model.sourceRevision.get(), model.sourceRevisionStatus.get()) } /** Validates required content in one UniMorph model-data notice. */ static void validateShareAlikeNoticeText(final String rawText, final String noticeName, - final String modelId, final String repository, final String licenseUri, + final String modelId, final String repository, final String licenseId, final String licenseUri, final String revision, final String revisionStatus) { final String text = rawText.replace('\r\n', '\n') + final Map> licenseText = [ + 'CC-BY-SA-3.0': ['Creative Commons Attribution-ShareAlike 3.0 Unported', + 'is distributed under Creative Commons Attribution-ShareAlike 3.0'], + 'CC-BY-SA-4.0': ['Creative Commons Attribution-ShareAlike 4.0 International', + 'is distributed under Creative Commons Attribution-ShareAlike 4.0'], + 'CC-BY-4.0': ['Creative Commons Attribution 4.0 International', + 'is distributed under Creative Commons Attribution 4.0'], + 'LGPLLR': ['Lesser General Public License For Linguistic Resources', + 'is distributed under the Lesser General Public License For Linguistic Resources']] + final List expectedLicenseText = licenseText[licenseId] + if (expectedLicenseText == null) { + throw new GradleException("Unsupported notice-based model-data license: ${licenseId}") + } final List required = [ "Model ID: ${modelId}", "Official repository: ${repository}", 'Attribution:', - 'License:\nCreative Commons Attribution-ShareAlike 3.0 Unported', + "License:\n${expectedLicenseText[0]}", "Canonical license URI: ${licenseUri}", 'Radixor modifications:', "Revision status: ${revisionStatus}", @@ -378,7 +432,7 @@ final class RadixorModelPlugin implements Plugin { 'to the extent protected by applicable law.', 'The underlying morphological data remains attributed to UniMorph and', "This derived model data, including Radixor's protectable contributions,", - 'is distributed under Creative Commons Attribution-ShareAlike 3.0', + expectedLicenseText[1], 'Neither UniMorph nor any upstream contributor endorses Radixor.'] if (revision == 'not-recorded-in-legacy-import') { required.add('The exact UniMorph commit used for the original Radixor import was not recorded.') @@ -390,15 +444,26 @@ final class RadixorModelPlugin implements Plugin { } /** Rejects generic license files and foreign notices in a UniMorph model artifact. */ - static void validateUniMorphJarContents(final List names) { - if (names.any { String name -> name.startsWith('META-INF/LICENSES/') }) { - throw new GradleException('A UniMorph model artifact must use only its model-specific notice for data licensing.') + static void validateUniMorphJarContents(final List names, final String permittedLicenseResource = null) { + final List licenseResources = names.findAll { + String name -> name.startsWith('META-INF/LICENSES/') && !name.endsWith('/') + } + if (licenseResources != (permittedLicenseResource == null ? [] : [permittedLicenseResource])) { + throw new GradleException('A UniMorph model artifact contains unexpected model-data license resources.') } if (names.count { String name -> name.startsWith('META-INF/NOTICE/') && !name.endsWith('/') } != 1) { throw new GradleException('A UniMorph model artifact must contain exactly one model-specific notice.') } } + /** Validates the canonical SPDX LGPLLR text used by the Khaling model. */ + static void validateLgpllrLicense(final File licenseFile) { + final String expected = 'e4e0f2f92769aad680aeca07f359521004dac4598d8b03def0e6fe507e871134' + if (sha256(licenseFile) != expected) { + throw new GradleException('The LGPLLR license must match the canonical SPDX text byte-for-byte.') + } + } + /** Rejects UniMorph licensing material in the separately licensed PoliMorf artifact. */ static void validatePoliMorfJarContents(final List names) { if (names.any { String name -> name.startsWith('META-INF/NOTICE/') @@ -438,16 +503,14 @@ final class RadixorModelPlugin implements Plugin { long lineNumber = 0L while ((line = reader.readLine()) != null) { lineNumber++ - final String trimmed = line.trim() - if (trimmed && !trimmed.startsWith('#') && !trimmed.startsWith('//')) { - final String[] columns = line.split('\\t', -1) - if (columns[0].isEmpty()) { - throw new GradleException("Invalid Radixor dictionary row ${lineNumber} in ${file}.") - } - if (containsUnicodeWhitespace(columns[0])) continue + final String logicalLine = stripRemark(line).trim() + if (logicalLine) { + final String[] columns = logicalLine.split('\\t', -1) + final String stem = columns[0].strip() + if (!stem || containsUnicodeWhitespace(stem)) continue long acceptedRowForms = 1L for (int index = 1; index < columns.length; index++) { - final String variant = columns[index] + final String variant = columns[index].strip() if (variant.isEmpty()) { ignoredEmptyVariants++ } else if (!containsUnicodeWhitespace(variant)) { @@ -475,6 +538,21 @@ final class RadixorModelPlugin implements Plugin { return new DictionaryValidationResult(acceptedGroups, acceptedForms, ignoredEmptyVariants) } + /** Removes the earliest production-parser comment marker from one physical line. */ + private static String stripRemark(final String line) { + final int hash = line.indexOf('#') + final int slash = line.indexOf('//') + final int remark + if (hash < 0) { + remark = slash + } else if (slash < 0) { + remark = hash + } else { + remark = Math.min(hash, slash) + } + return remark < 0 ? line : line.substring(0, remark) + } + /** Detects Unicode whitespace in one bounded dictionary field. */ private static boolean containsUnicodeWhitespace(final String value) { for (int index = 0; index < value.length(); index++) { @@ -495,7 +573,7 @@ model.default=${model.defaultModel.get()} model.format=radixor-dictionary-tsv-gzip model.formatVersion=1 model.sha256=${checksum} -model.rightToLeft=${['FA_IR', 'HE_IL', 'YI'].contains(model.language.get())} +model.rightToLeft=${model.rightToLeft.get()} model.caseProcessing=LOWERCASE_WITH_LOCALE_ROOT model.diacriticProcessing=AS_IS model.storeOriginal=true diff --git a/build-logic/src/main/groovy/org/egothor/radixor/ValidateModelInputTask.groovy b/build-logic/src/main/groovy/org/egothor/radixor/ValidateModelInputTask.groovy index c41534e..6888706 100644 --- a/build-logic/src/main/groovy/org/egothor/radixor/ValidateModelInputTask.groovy +++ b/build-logic/src/main/groovy/org/egothor/radixor/ValidateModelInputTask.groovy @@ -52,6 +52,7 @@ abstract class ValidateModelInputTask extends DefaultTask { @Input abstract Property getModelId() @Input abstract Property getModuleName() @Input abstract Property getShareAlike() + @Input abstract Property getManifestManaged() @Input abstract MapProperty getMetadata() /** Performs deterministic metadata, licensing, and streaming dictionary validation. */ @@ -60,7 +61,7 @@ abstract class ValidateModelInputTask extends DefaultTask { final File dictionary = dictionaryFile.get().asFile final String id = modelId.get() final String version = versionFile.get().asFile.getText('UTF-8').trim() - if (id != moduleName.get() || !(id ==~ /[a-z]{2}(?:-[a-z]{2})?-[a-z0-9]+(?:-[a-z0-9]+)*/)) { + if (id != moduleName.get() || !(id ==~ /[a-z]{2,3}(?:-[a-z]{2})?-[a-z0-9]+(?:-[a-z0-9]+)*/)) { throw new GradleException("Model ID '${id}' must equal module '${moduleName.get()}' and use the safe model-ID syntax.") } if (!(version ==~ /[0-9]+\.[0-9]+\.[0-9]+(?:[-+][0-9A-Za-z.-]+)?/)) { @@ -70,12 +71,16 @@ abstract class ValidateModelInputTask extends DefaultTask { values.each { String key, String value -> if (value == null || value.isBlank()) throw new GradleException("Required model metadata is missing: ${key}") } - RadixorModelPlugin.validateRevisionMetadata(values['source.revision'], values['source.revisionStatus']) + RadixorModelPlugin.validateManifestRevisionMetadata(values['source.revision'], + values['source.revisionStatus'], manifestManaged.get()) if (shareAlike.get()) { final File notice = noticeFile.get().asFile RadixorModelPlugin.validateShareAlikeNoticeText(notice.getText('UTF-8'), notice.toString(), id, - values['source.repository'], values['source.licenseUri'], values['source.revision'], + values['source.repository'], values['source.license'], values['source.licenseUri'], values['source.revision'], values['source.revisionStatus']) + if (values['source.license'] == 'LGPLLR') { + RadixorModelPlugin.validateLgpllrLicense(licenseFile.get().asFile) + } } else { final String text = licenseFile.get().asFile.getText('UTF-8') if (!text.contains('SPDX-License-Identifier: BSD-2-Clause') diff --git a/build-logic/src/test/groovy/org/egothor/radixor/RadixorModelPluginTest.groovy b/build-logic/src/test/groovy/org/egothor/radixor/RadixorModelPluginTest.groovy index c8220ce..cf468a7 100644 --- a/build-logic/src/test/groovy/org/egothor/radixor/RadixorModelPluginTest.groovy +++ b/build-logic/src/test/groovy/org/egothor/radixor/RadixorModelPluginTest.groovy @@ -32,6 +32,8 @@ package org.egothor.radixor import org.gradle.api.GradleException +import org.gradle.testkit.runner.BuildResult +import org.gradle.testkit.runner.GradleRunner import org.junit.jupiter.api.Test import org.junit.jupiter.api.io.TempDir @@ -39,6 +41,7 @@ import java.nio.charset.StandardCharsets import java.nio.file.Files import java.nio.file.Path import java.util.zip.GZIPOutputStream +import java.util.zip.ZipFile import static org.junit.jupiter.api.Assertions.assertEquals import static org.junit.jupiter.api.Assertions.assertThrows @@ -62,6 +65,17 @@ final class RadixorModelPluginTest { 'not-recorded-in-legacy-import', 'not-recorded-in-legacy-import') } + /** Preserves legacy compatibility while forbidding sentinels in manifest-managed models. */ + @Test + void rejectsUnknownRevisionForManifestManagedModel() { + assertThrows(GradleException) { + RadixorModelPlugin.validateManifestRevisionMetadata( + 'not-recorded-in-legacy-import', 'not-recorded-in-legacy-import', true) + } + RadixorModelPlugin.validateManifestRevisionMetadata( + 'not-recorded-in-legacy-import', 'not-recorded-in-legacy-import', false) + } + /** Rejects a missing revision-status declaration. */ @Test void rejectsMissingRevisionStatus() { @@ -126,6 +140,126 @@ final class RadixorModelPluginTest { } } + /** Packages CC BY-SA 4.0 as ShareAlike notice material and describes that notice in the POM. */ + @Test + void packagesCcBySaFourNoticeAndDescribesItInPom() { + final Path project = temporaryDirectory.resolve('cc4-model') + final Path modelInput = project.resolve('src/modelInput') + Files.createDirectories(modelInput) + Files.writeString(project.resolve('settings.gradle'), "rootProject.name = 'hsi-default'\n", + StandardCharsets.UTF_8) + Files.writeString(project.resolve('model-version.txt'), '1.0.0\n', StandardCharsets.UTF_8) + writeGzip(modelInput.resolve('stemmer.gz').toFile()) { BufferedWriter writer -> + writer.write('root\trooted\n') + } + Files.writeString(modelInput.resolve('NOTICE-model-data.txt'), validVersionFourNotice(), + StandardCharsets.UTF_8) + Files.writeString(project.resolve('build.gradle'), '''plugins { + id 'org.egothor.radixor.model' +} +radixorModel { + modelId = 'hsi-default' + language = 'HSI' + displayName = 'Hsilimo — UniMorph' + defaultModel = true + sourceName = 'UniMorph' + sourceVersion = 'revision' + sourceRevision = 'revision' + sourceProject = 'UniMorph' + sourceRepository = 'https://github.com/unimorph/hsi' + sourceDataset = 'UniMorph Hsilimo morphological dataset' + sourceRevisionStatus = 'recorded' + sourceLicense = 'CC-BY-SA-4.0' + sourceLicenseUri = 'https://creativecommons.org/licenses/by-sa/4.0/' + sourceAttribution = 'UniMorph contributors' + sourceVerificationDate = '2026-09-10' + transformationsSummary = 'Cleaning and deterministic model packaging' + noticeFileName = 'NOTICE-model-data.txt' +} +''', StandardCharsets.UTF_8) + + final BuildResult result = GradleRunner.create() + .withProjectDir(project.toFile()) + .withPluginClasspath() + .withArguments('verifyModelJar', 'generatePomFileForModelPublication', '--stacktrace') + .build() + + assertTrue(result.output.contains('BUILD SUCCESSFUL')) + final Path archive = project.resolve('build/libs/radixor-model-hsi-default-1.0.0.jar') + new ZipFile(archive.toFile()).withCloseable { ZipFile zip -> + assertTrue(zip.getEntry('META-INF/NOTICE/hsi-default-data.txt') != null) + assertTrue(zip.getEntry('META-INF/LICENSES/PoliMorf-BSD-2-Clause.txt') == null) + } + final String pom = Files.readString( + project.resolve('build/publications/model/pom-default.xml'), StandardCharsets.UTF_8) + assertTrue(pom.contains('See the packaged model-specific notice.')) + assertTrue(!pom.contains('See the packaged model-data license.')) + } + + /** Packages the canonical LGPLLR text and legible dictionary form in the Khaling source artifact. */ + @Test + void packagesLgpllrLicenseAndLegibleDictionarySource() { + final Path project = temporaryDirectory.resolve('lgpllr-model') + final Path modelInput = project.resolve('src/modelInput') + Files.createDirectories(modelInput) + Files.writeString(project.resolve('settings.gradle'), "rootProject.name = 'klr-default'\n", + StandardCharsets.UTF_8) + Files.writeString(project.resolve('model-version.txt'), '1.0.0\n', StandardCharsets.UTF_8) + writeGzip(modelInput.resolve('stemmer.gz').toFile()) { BufferedWriter writer -> + writer.write('root\trooted\n') + } + Files.writeString(modelInput.resolve('NOTICE-model-data.txt'), validLgpllrNotice(), + StandardCharsets.UTF_8) + final List licenseCandidates = [ + Path.of('models/klr-default/src/modelInput/LGPLLR.txt'), + Path.of('../models/klr-default/src/modelInput/LGPLLR.txt')] + final Path license = licenseCandidates.find { Path candidate -> Files.isRegularFile(candidate) } + assertTrue(license != null, 'The canonical LGPLLR license text must be checked in.') + Files.copy(license, modelInput.resolve('LGPLLR.txt')) + Files.writeString(project.resolve('build.gradle'), '''plugins { + id 'org.egothor.radixor.model' +} +radixorModel { + modelId = 'klr-default' + language = 'KLR' + displayName = 'Khaling — UniMorph' + defaultModel = true + sourceName = 'UniMorph' + sourceVersion = 'revision' + sourceRevision = 'revision' + sourceProject = 'UniMorph' + sourceRepository = 'https://github.com/unimorph/klr' + sourceDataset = 'UniMorph Khaling morphological dataset' + sourceRevisionStatus = 'recorded' + sourceLicense = 'LGPLLR' + sourceLicenseUri = 'https://spdx.org/licenses/LGPLLR.html' + sourceAttribution = 'UniMorph contributors' + sourceVerificationDate = '2026-09-11' + transformationsSummary = 'Cleaning and deterministic model packaging' + noticeFileName = 'NOTICE-model-data.txt' + licenseFileName = 'LGPLLR.txt' +} +''', StandardCharsets.UTF_8) + + final BuildResult result = GradleRunner.create() + .withProjectDir(project.toFile()) + .withPluginClasspath() + .withArguments('verifyModelJar', '--stacktrace') + .build() + + assertTrue(result.output.contains('BUILD SUCCESSFUL')) + final Path archive = project.resolve('build/libs/radixor-model-klr-default-1.0.0.jar') + new ZipFile(archive.toFile()).withCloseable { ZipFile zip -> + assertTrue(zip.getEntry('META-INF/NOTICE/klr-default-data.txt') != null) + assertTrue(zip.getEntry('META-INF/LICENSES/LGPLLR.txt') != null) + } + final Path sources = project.resolve('build/libs/klr-default-1.0.0-sources.jar') + new ZipFile(sources.toFile()).withCloseable { ZipFile zip -> + assertTrue(zip.getEntry('org/egothor/stemmer/models/klr-default/stemmer.gz') != null) + assertTrue(zip.getEntry('META-INF/LICENSES/LGPLLR.txt') != null) + } + } + /** Streams a large dictionary while retaining only aggregate counters and the current row. */ @Test void validatesLargeDictionaryWithBoundedState() { @@ -142,7 +276,7 @@ final class RadixorModelPluginTest { assertEquals(groups, result.acceptedGroupCount) assertEquals(groups * 2L, result.acceptedFormCount) - assertEquals(groups, result.ignoredEmptyVariantCount) + assertEquals(0L, result.ignoredEmptyVariantCount) } /** Rejects a source that is not a GZip stream. */ @@ -163,12 +297,15 @@ final class RadixorModelPluginTest { assertThrows(GradleException) { RadixorModelPlugin.validateDictionary(dictionary) } } - /** Rejects structurally invalid rows with an empty stem. */ + /** Trims a physical row before splitting it exactly as the production parser does. */ @Test - void rejectsInvalidRows() { + void trimsRowsBeforeSplittingFields() { final File dictionary = temporaryDirectory.resolve('invalid-row.gz').toFile() - writeGzip(dictionary) { BufferedWriter writer -> writer.write("\tvariant\n") } - assertThrows(GradleException) { RadixorModelPlugin.validateDictionary(dictionary) } + writeGzip(dictionary) { BufferedWriter writer -> writer.write("\tvariant\nvalid\tvariant\n") } + final RadixorModelPlugin.DictionaryValidationResult result = + RadixorModelPlugin.validateDictionary(dictionary) + assertEquals(2L, result.acceptedGroupCount) + assertEquals(3L, result.acceptedFormCount) } /** Preserves the production parser policy for Unicode-whitespace items. */ @@ -185,6 +322,26 @@ final class RadixorModelPluginTest { assertEquals(2L, result.acceptedFormCount) } + /** Mirrors production parsing for trimming, comments, NBSP, marks, joiners, and punctuation. */ + @Test + void matchesProductionDictionaryFieldGrammar() { + final File dictionary = temporaryDirectory.resolve('parser-conformance.gz').toFile() + writeGzip(dictionary) { BufferedWriter writer -> + writer.write(" root \t variant \n") + writer.write("bad stem\tvariant\n") + writer.write("nbsp\u00a0stem\taccepted\n") + writer.write("mark\u0301\tzwnj\u200c\tzwj\u200d\to'neil\tco-op\ta:b\ta/b\n") + writer.write("comment\tkept#discarded\n") + writer.write("slash\tkept//discarded\n") + } + + final RadixorModelPlugin.DictionaryValidationResult result = + RadixorModelPlugin.validateDictionary(dictionary) + + assertEquals(5L, result.acceptedGroupCount) + assertEquals(15L, result.acceptedFormCount) + } + /** Streams the complete maintained PoliMorf model input successfully. */ @Test void validatesFullPoliMorfInput() { @@ -210,7 +367,8 @@ final class RadixorModelPluginTest { private static void validateNotice(final String text) { RadixorModelPlugin.validateShareAlikeNoticeText(text, 'test notice', 'test-model', - 'https://github.com/unimorph/test', 'https://creativecommons.org/licenses/by-sa/3.0/', + 'https://github.com/unimorph/test', 'CC-BY-SA-3.0', + 'https://creativecommons.org/licenses/by-sa/3.0/', 'not-recorded-in-legacy-import', 'not-recorded-in-legacy-import') } @@ -231,6 +389,44 @@ The underlying morphological data remains attributed to UniMorph and This derived model data, including Radixor's protectable contributions, is distributed under Creative Commons Attribution-ShareAlike 3.0 Neither UniMorph nor any upstream contributor endorses Radixor. +''' + } + + private static String validVersionFourNotice() { + return '''Model ID: hsi-default +Official repository: https://github.com/unimorph/hsi +Attribution: UniMorph contributors +License: +Creative Commons Attribution-ShareAlike 4.0 International +Canonical license URI: https://creativecommons.org/licenses/by-sa/4.0/ +Radixor modifications: Cleaning and deterministic model packaging. +Revision status: recorded +Copyright (C) 2026, Leo Galambos. +Radixor-specific selection, verification, cleaning, normalization, +to the extent protected by applicable law. +The underlying morphological data remains attributed to UniMorph and +This derived model data, including Radixor's protectable contributions, +is distributed under Creative Commons Attribution-ShareAlike 4.0 +Neither UniMorph nor any upstream contributor endorses Radixor. +''' + } + + private static String validLgpllrNotice() { + return '''Model ID: klr-default +Official repository: https://github.com/unimorph/klr +Attribution: UniMorph contributors +License: +Lesser General Public License For Linguistic Resources +Canonical license URI: https://spdx.org/licenses/LGPLLR.html +Radixor modifications: Cleaning and deterministic model packaging. +Revision status: recorded +Copyright (C) 2026, Leo Galambos. +Radixor-specific selection, verification, cleaning, normalization, +to the extent protected by applicable law. +The underlying morphological data remains attributed to UniMorph and +This derived model data, including Radixor's protectable contributions, +is distributed under the Lesser General Public License For Linguistic Resources +Neither UniMorph nor any upstream contributor endorses Radixor. ''' } } diff --git a/build.gradle b/build.gradle index 40ac853..b5b6d92 100644 --- a/build.gradle +++ b/build.gradle @@ -135,6 +135,47 @@ def modelProjects() { } } +def quarantinedModelIds() { + Properties quarantine = new Properties() + rootProject.file('models/model-quarantine.properties').withInputStream { InputStream input -> + quarantine.load(input) + } + return quarantine.stringPropertyNames().toList().sort() +} + +tasks.register('verifyModelQuarantine') { + group = 'verification' + description = 'Verifies quarantined model projects remain outside the active topology.' + inputs.files(rootProject.file('models/model-projects.properties'), + rootProject.file('models/model-quarantine.properties')) + doLast { + Properties topology = new Properties() + rootProject.file('models/model-projects.properties').withInputStream { InputStream input -> + topology.load(input) + } + Properties quarantine = new Properties() + rootProject.file('models/model-quarantine.properties').withInputStream { InputStream input -> + quarantine.load(input) + } + Set overlap = topology.stringPropertyNames().intersect( + quarantine.stringPropertyNames()) + if (!overlap.isEmpty()) { + throw new GradleException("Quarantined model IDs remain active: ${overlap.toList().sort().join(', ')}") + } + quarantine.stringPropertyNames().each { String modelId -> + String record = quarantine.getProperty(modelId) + if (!rootProject.file("models/${modelId}").isDirectory()) { + throw new GradleException("Quarantined model directory is missing: models/${modelId}") + } + if (!(record ==~ /https:\/\/github\.com\/unimorph\/[a-z0-9-]+\|\S.*\S/)) { + throw new GradleException("Invalid quarantine evidence record for ${modelId}.") + } + } + logger.lifecycle("Verified ${quarantine.size()} quarantined model projects outside " + + "${topology.size()} active model projects.") + } +} + def defaultModelProjects() { Properties topology = new Properties() rootProject.file('models/model-projects.properties').withInputStream { InputStream input -> @@ -145,6 +186,26 @@ def defaultModelProjects() { }.sort().collect { String modelId -> project(":models:${modelId}") } } +def languageDefaultModelProjects() { + Properties topology = new Properties() + rootProject.file('models/model-projects.properties').withInputStream { InputStream input -> + topology.load(input) + } + return topology.stringPropertyNames().findAll { String modelId -> + topology.getProperty(modelId) in ['default', 'standalone'] + }.sort().collect { String modelId -> project(":models:${modelId}") } +} + +def standaloneModelProjects() { + Properties topology = new Properties() + rootProject.file('models/model-projects.properties').withInputStream { InputStream input -> + topology.load(input) + } + return topology.stringPropertyNames().findAll { String modelId -> + topology.getProperty(modelId) == 'standalone' + }.sort().collect { String modelId -> project(":models:${modelId}") } +} + def alternativeModelProjects() { Properties topology = new Properties() rootProject.file('models/alternative-model-projects.properties').withInputStream { InputStream input -> @@ -712,7 +773,7 @@ tasks.register('verifySoftwareLicenseHeaders') { tasks.register('verifyAllDefaultModels') { group = 'verification' description = 'Verifies that every language default model project is configured.' - dependsOn(defaultModelProjects().collect { Project modelProject -> + dependsOn(languageDefaultModelProjects().collect { Project modelProject -> modelProject.path + ':verifyModelDescriptor' }) } @@ -1160,7 +1221,7 @@ tasks.register('stemmingQuality', JavaExec) { tasks.register('benchmarkCorpusReport', JavaExec) { group = 'verification' - description = 'Reports corpus and preferred patch-command counts for every default model.' + description = 'Reports corpus, distinct usable forms, and patch-command counts for all user-facing models.' dependsOn(tasks.named('jmhClasses')) classpath = files(sourceSets.jmh.runtimeClasspath, configurations.stemmingQualityJmhRuntime) mainClass = 'org.egothor.stemmer.benchmark.BenchmarkCorpusReportApplication' @@ -1190,6 +1251,68 @@ tasks.register('dictionaryGeneralization', JavaExec) { maxHeapSize = '8g' } +tasks.register('writeStandaloneGeneralizationSelection') { + group = 'verification' + description = 'Writes the exact topology-derived standalone generalization selector.' + final Provider selector = layout.buildDirectory.file('reports/generalization/standalone-default-models.txt') + outputs.file(selector) + doLast { + final List modelIds = standaloneModelProjects().collect { Project modelProject -> modelProject.name } + if (modelIds.isEmpty()) { + throw new GradleException('The active topology contains no standalone default models.') + } + final File output = selector.get().asFile + output.parentFile.mkdirs() + output.setText(modelIds.join(System.lineSeparator()) + System.lineSeparator(), 'UTF-8') + } +} + +tasks.register('dictionaryGeneralizationStandalone', JavaExec) { + group = 'verification' + description = 'Evaluates only the topology-derived standalone language defaults for snapshot continuation.' + dependsOn(tasks.named('testClasses')) + dependsOn(tasks.named('writeStandaloneGeneralizationSelection')) + classpath = sourceSets.test.runtimeClasspath + mainClass = 'org.egothor.stemmer.benchmark.generalization.DictionaryGeneralizationApplication' + outputs.file(layout.buildDirectory.file('reports/generalization/dictionary-generalization-standalone-2026-09-11.csv')) + final String dictionaryGeneralizationSourceState = providers.exec { + commandLine 'git', 'status', '--porcelain' + }.standardOutput.asText.get().trim().isEmpty() ? 'clean' : 'dirty' + args layout.buildDirectory.file('reports/generalization/dictionary-generalization-standalone-2026-09-11.csv') + .get().asFile.absolutePath, + dictionaryGeneralizationReleaseVersion.get(), + providers.exec { commandLine 'git', 'rev-parse', 'HEAD' }.standardOutput.asText.get().trim(), + dictionaryGeneralizationSourceState, + layout.projectDirectory.file('src/test/java/org/egothor/stemmer/benchmark/generalization/DictionaryGeneralizationApplication.java').asFile.absolutePath, + layout.buildDirectory.file('reports/generalization/standalone-default-models.txt').get().asFile.absolutePath + maxHeapSize = '8g' +} + +tasks.register('dictionaryGeneralizationExactModel', JavaExec) { + group = 'verification' + description = 'Evaluates one exact authoritative default for targeted snapshot repair.' + dependsOn(tasks.named('testClasses')) + classpath = sourceSets.test.runtimeClasspath + mainClass = 'org.egothor.stemmer.benchmark.generalization.DictionaryGeneralizationApplication' + final String exactModel = providers.gradleProperty('dictionaryGeneralizationModel').getOrElse('') + final String dictionaryGeneralizationSourceState = providers.exec { + commandLine 'git', 'status', '--porcelain' + }.standardOutput.asText.get().trim().isEmpty() ? 'clean' : 'dirty' + args layout.buildDirectory.file('reports/generalization/dictionary-generalization-exact-model.csv') + .get().asFile.absolutePath, + dictionaryGeneralizationReleaseVersion.get(), + providers.exec { commandLine 'git', 'rev-parse', 'HEAD' }.standardOutput.asText.get().trim(), + dictionaryGeneralizationSourceState, + layout.projectDirectory.file('src/test/java/org/egothor/stemmer/benchmark/generalization/DictionaryGeneralizationApplication.java').asFile.absolutePath, + '--exact-model', exactModel + doFirst { + if (exactModel.isBlank()) { + throw new GradleException('Set -PdictionaryGeneralizationModel to one exact default model ID.') + } + } + maxHeapSize = '8g' +} + tasks.register('editCostSensitivity', JavaExec) { group = 'verification' description = 'Evaluates normalized edit-cost ratios across knowledge levels and deterministic dictionary splits.' @@ -1285,18 +1408,35 @@ tasks.register('checkFilteredModels') { tasks.register('publishDictionaryGeneralizationDocumentation', Exec) { group = 'documentation' description = 'Publishes the validated multilingual dictionary-generalization report.' - dependsOn(tasks.named('dictionaryGeneralization')) + dependsOn('mergeDictionaryGeneralizationSnapshots') commandLine 'python3', layout.projectDirectory.file('tools/update-generalization-documentation.py').asFile, - layout.buildDirectory.file('reports/generalization/dictionary-generalization.csv').get().asFile, - layout.projectDirectory.dir('docs').asFile, 'update' + layout.projectDirectory.file('docs/benchmarks/data/dictionary-generalization-2026-09-11.csv').asFile, + layout.projectDirectory.dir('docs').asFile, 'update', + '--corpus', layout.projectDirectory.file('docs/benchmarks/data/java-benchmark-corpora-2026-09-11.csv').asFile, + '--snapshot-name', 'dictionary-generalization-2026-09-11.csv' +} + +tasks.register('mergeDictionaryGeneralizationSnapshots', Exec) { + group = 'documentation' + description = 'Merges the immutable 20-model archive with the active standalone continuation.' + dependsOn(tasks.named('dictionaryGeneralizationStandalone')) + commandLine 'python3', layout.projectDirectory.file('tools/merge-generalization-snapshots.py').asFile, + '--archive', layout.projectDirectory.file('docs/benchmarks/data/dictionary-generalization.csv').asFile, + '--continuation', layout.buildDirectory.file('reports/generalization/dictionary-generalization-standalone-2026-09-11.csv').get().asFile, + '--corpus', layout.projectDirectory.file('docs/benchmarks/data/java-benchmark-corpora-2026-09-11.csv').asFile, + '--topology', layout.projectDirectory.file('models/model-projects.properties').asFile, + '--output', layout.projectDirectory.file('docs/benchmarks/data/dictionary-generalization-2026-09-11.csv').asFile, + '--manifest', layout.projectDirectory.file('docs/benchmarks/data/active-snapshots.properties').asFile } tasks.register('verifyDictionaryGeneralizationDocumentation', Exec) { group = 'verification' description = 'Verifies the rendered generalization page and checked-in raw counters.' commandLine 'python3', layout.projectDirectory.file('tools/update-generalization-documentation.py').asFile, - layout.projectDirectory.file('docs/benchmarks/data/dictionary-generalization.csv').asFile, - layout.projectDirectory.dir('docs').asFile, 'verify' + layout.projectDirectory.file('docs/benchmarks/data/dictionary-generalization-2026-09-11.csv').asFile, + layout.projectDirectory.dir('docs').asFile, 'verify', + '--corpus', layout.projectDirectory.file('docs/benchmarks/data/java-benchmark-corpora-2026-09-11.csv').asFile, + '--snapshot-name', 'dictionary-generalization-2026-09-11.csv' } tasks.register('publishEditCostSensitivityDocumentation', Exec) { @@ -1323,20 +1463,28 @@ tasks.register('verifyPublishedPerformanceSnapshotChecksums', Exec) { commandLine 'sha256sum', '--check', 'performance-snapshots.sha256' } +tasks.register('verifyActiveBenchmarkSnapshots', Exec) { + group = 'verification' + description = 'Verifies active benchmark pointers, selected checksums, and immutable archives.' + commandLine 'python3', layout.projectDirectory.file('tools/verify-active-benchmark-snapshots.py').asFile +} + tasks.register('verifyPublishedJavaBenchmarkDocumentation', Exec) { group = 'verification' - description = 'Verifies Java benchmark documentation against the checked-in 2026-08-25 reports.' + description = 'Verifies Java benchmark documentation against the active 2026-09-11 reports.' dependsOn(tasks.named('verifyPublishedPerformanceSnapshotChecksums')) commandLine 'python3', layout.projectDirectory.file('tools/update-benchmark-documentation.py').asFile, '--docs-root', layout.projectDirectory.dir('docs').asFile, '--readme', layout.projectDirectory.file('README.md').asFile, - '--corpus', layout.projectDirectory.file('docs/benchmarks/data/java-benchmark-corpora-2026-08-25.csv').asFile, - '--accuracy', layout.projectDirectory.file('docs/benchmarks/data/java-stemmer-accuracy-2026-08-25.csv').asFile, - '--speed', layout.projectDirectory.file('docs/benchmarks/data/java-stemmer-speed-2026-08-25.csv').asFile, - '--coverage-accuracy', layout.projectDirectory.file('docs/benchmarks/data/java-english-coverage-accuracy-2026-08-25.csv').asFile, - '--coverage-speed', layout.projectDirectory.file('docs/benchmarks/data/java-english-coverage-speed-2026-08-25.csv').asFile, - '--date', '2026-08-25', - '--release-version', '4.2.0-6-g84e57fb', + '--corpus', layout.projectDirectory.file('docs/benchmarks/data/java-benchmark-corpora-2026-09-11.csv').asFile, + '--accuracy', layout.projectDirectory.file('docs/benchmarks/data/java-stemmer-accuracy-2026-09-11.csv').asFile, + '--speed', layout.projectDirectory.file('docs/benchmarks/data/java-stemmer-speed-2026-09-11.csv').asFile, + '--coverage-accuracy', layout.projectDirectory.file('docs/benchmarks/data/java-english-coverage-accuracy-2026-09-10.csv').asFile, + '--coverage-speed', layout.projectDirectory.file('docs/benchmarks/data/java-english-coverage-speed-2026-09-10.csv').asFile, + '--quality', layout.projectDirectory.file('docs/benchmarks/data/stemming-quality-2026-09-11.csv').asFile, + '--snowball-catalog', layout.projectDirectory.file('docs/benchmarks/data/snowball-language-cases.csv').asFile, + '--date', '2026-09-11', + '--release-version', '4.3.0.dirty', '--mode', 'verify' } @@ -1371,12 +1519,15 @@ tasks.register('writeJmhRuntimeClasspath') { tasks.register('prepareBenchmarkModelInputs', Sync) { group = 'verification' - description = 'Prepares default model inputs for JMH and quality evaluation without changing source data.' + description = 'Prepares all registered language-default inputs for JMH and quality evaluation without changing source data.' into(layout.buildDirectory.dir('generated/benchmark-model-inputs')) - defaultModelProjects().each { Project modelProject -> - String languageDirectory = modelProject.name == 'pl-pl-unimorph' - ? 'pl_pl' - : modelProject.name.replace('-default', '').replace('-', '_') + languageDefaultModelProjects().each { Project modelProject -> + final String modelBuildText = modelProject.file('build.gradle').getText('UTF-8') + final java.util.regex.Matcher languageMatcher = modelBuildText =~ /(?m)^\s*language\s*=\s*'([A-Z_]+)'\s*$/ + if (!languageMatcher.find()) { + throw new GradleException("Model ${modelProject.path} does not declare a language.") + } + final String languageDirectory = languageMatcher.group(1).toLowerCase(java.util.Locale.ROOT) from(modelProject.file('src/modelInput/stemmer.gz')) { into(languageDirectory) } @@ -1391,7 +1542,8 @@ tasks.register('publishStemmingQualityDocumentation', JavaExec) { mainClass = 'org.egothor.stemmer.benchmark.quality.StemmingQualityDocumentationPublisher' args layout.buildDirectory.file('reports/stemming-quality/stemming-quality.csv').get().asFile.absolutePath, layout.projectDirectory.dir('docs').asFile.absolutePath, - 'update' + 'update', + 'stemming-quality-2026-09-11.csv' doFirst { if (!file("$buildDir/reports/stemming-quality/stemming-quality.csv").isFile()) { throw new GradleException('A complete stemming-quality CSV is required. Run stemmingQuality only when no validated complete report is available.') @@ -1405,12 +1557,47 @@ tasks.register('verifyStemmingQualityDocumentation', JavaExec) { dependsOn(tasks.named('testClasses')) classpath = sourceSets.test.runtimeClasspath mainClass = 'org.egothor.stemmer.benchmark.quality.StemmingQualityDocumentationPublisher' - args layout.projectDirectory.file('docs/benchmarks/data/stemming-quality.csv').asFile.absolutePath, + args layout.projectDirectory.file('docs/benchmarks/data/stemming-quality-2026-09-11.csv').asFile.absolutePath, layout.projectDirectory.dir('docs').asFile.absolutePath, + 'verify', + 'stemming-quality-2026-09-11.csv' +} + +tasks.register('publishActiveStemmingQualityDocumentation', JavaExec) { + group = 'documentation' + description = 'Publishes the checksum-bound active stemming-quality snapshot on every language page.' + dependsOn(tasks.named('testClasses')) + classpath = sourceSets.test.runtimeClasspath + mainClass = 'org.egothor.stemmer.benchmark.quality.StemmingQualityDocumentationPublisher' + args layout.projectDirectory.file('docs/benchmarks/data/stemming-quality-2026-09-11.csv').asFile.absolutePath, + layout.projectDirectory.dir('docs').asFile.absolutePath, + 'update', + 'stemming-quality-2026-09-11.csv' +} + +tasks.register('publishSnowballLanguageCatalog', JavaExec) { + group = 'documentation' + description = 'Publishes the documentation Snowball catalog from the Java benchmark authority.' + dependsOn(tasks.named('jmhClasses')) + classpath = sourceSets.jmh.runtimeClasspath + mainClass = 'org.egothor.stemmer.benchmark.SnowballLanguageCatalogApplication' + args layout.projectDirectory.file('docs/benchmarks/data/snowball-language-cases.csv').asFile.absolutePath, + 'update' +} + +tasks.register('verifySnowballLanguageCatalog', JavaExec) { + group = 'verification' + description = 'Verifies the documentation Snowball catalog against the Java benchmark authority.' + dependsOn(tasks.named('jmhClasses')) + classpath = sourceSets.jmh.runtimeClasspath + mainClass = 'org.egothor.stemmer.benchmark.SnowballLanguageCatalogApplication' + args layout.projectDirectory.file('docs/benchmarks/data/snowball-language-cases.csv').asFile.absolutePath, 'verify' } tasks.named('check') { + dependsOn(tasks.named('verifyModelQuarantine')) + dependsOn(tasks.named('verifyActiveBenchmarkSnapshots')) dependsOn(tasks.named('verifyStemmingQualityDocumentation')) dependsOn(tasks.named('verifyDictionaryGeneralizationDocumentation')) dependsOn(tasks.named('verifyEditCostSensitivityDocumentation')) diff --git a/docs/architecture.md b/docs/architecture.md index b1b6e27..541fc1f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -11,7 +11,7 @@ This document explains the structural architecture of **Radixor**: what data is | `StemmerModelRegistry` | Deterministic index/descriptor discovery and selection by model ID or language default | | `StemmerModelDescriptor` | Immutable public view of validated runtime identity, format, resource, checksum, and source URL | | Model convention plugin | Validates inputs and generates the resource namespace, descriptor, index, license, and publication | -| Standard aggregate | POM-only transitive runtime dependencies for one default per language | +| Standard aggregate | POM-only transitive runtime dependencies for the original 20 standard models | | Verification classpaths | Direct individual-model dependencies for tests, quality evaluation, and JMH, including optional PoliMorf | | Models BOM | POM-only recommended individual model versions in Maven dependency management | | Documentation staging | Reviewed `docs/`, including the checked-in generated catalog; staging independently regenerates a byte-identical catalog under `build/mkdocs-source/` | @@ -77,7 +77,15 @@ No equality relationship is implied between these values. ## Build topology and generated output -`models/model-projects.properties` is the single Gradle-readable topology list for the 21 individual model projects and their default or optional aggregate role. Per-model build scripts and generated descriptors remain authoritative for language, resource, provenance, checksum, and model-specific metadata. `settings.gradle`, root verification classpaths, the standard POM, and BOM constraints all derive membership from the topology list. +`models/model-projects.properties` is the single Gradle-readable topology list +for the 144 user-facing model projects. The `default` role selects the original +20 standard-aggregate members, `standalone` identifies 123 registered language +defaults that remain individually publishable, and `optional` identifies +PoliMorf. All three roles participate in settings, verification classpaths, +JMH, the BOM, catalog generation, and individual publication; only `default` +participates in the Java and Python standard aggregates. Per-model build +scripts and generated descriptors remain authoritative for language, writing +direction, resource, provenance, checksum, and model-specific metadata. Gradle implicitly creates the lifecycle parent `:models` because child paths are nested. It has no build script, applied project plugin, Maven coordinate, publication, or archive. The root CycloneDX plugin exposes direct-task instances to subprojects internally; every subproject instance is disabled, so only root `:cyclonedxDirectBom` can generate an SBOM. The ignored path `models/build/` is generated output, not a module, and the supported build does not write reports there. Root aggregate reports, including `verifyJmhModelClasspath`, belong under `build/reports/models/`; each individual model retains its own outputs under `models//build/`. @@ -89,8 +97,9 @@ The `org.egothor.radixor.model` convention plugin treats `src/modelInput` as imm For UniMorph models, the convention validates and packages one model-specific attribution, licensing, provenance, and contribution notice. Source and packaged notice bytes must match. The -notice identifies CC BY-SA 3.0 through its canonical URI; no project-wide CC license directory or -duplicated full legal text is used. Descriptors distinguish exact revisions from the explicit +notice identifies its audited license through the canonical URI; no project-wide model-data +license is used. The LGPLLR Khaling artifact also packages the complete canonical license text and +legible dictionary form. Descriptors distinguish exact revisions from the explicit legacy-import sentinel. UniMorph supplies morphological data; runtime patch commands and tries are constructed by Radixor. The Java software remains BSD-3-Clause, while PoliMorf data remains under its separately packaged BSD-2-Clause license. diff --git a/docs/assets/stylesheets/landing-v2.css b/docs/assets/stylesheets/landing-v2.css index 35cb9f8..4ba67b2 100644 --- a/docs/assets/stylesheets/landing-v2.css +++ b/docs/assets/stylesheets/landing-v2.css @@ -48,6 +48,32 @@ .rx2-evidence-line{margin:10px 2px 0;color:#52617a;font-size:.68rem;line-height:1.5}.rx2-evidence-line strong{color:#263751;font-weight:750}.rx2-evidence-line a{color:#1769ef;font-weight:700;text-decoration:none} +/* Dense benchmark summaries remain legible at narrow widths. */ +.rx2-highlight-stats{grid-template-columns:repeat(4,minmax(0,1fr))} +.rx2-highlight-stat{min-width:0} +.rx2-highlight-stat strong{max-width:100%;font-size:clamp(1.35rem,3.2vw,1.9rem);white-space:nowrap} + +/* Symmetric log-ratio chart: the center axis is equal runtime (1x). */ +.rx2-ratio-chart{margin-top:10px;padding:8px 10px 10px;border:1px solid #dce6f2;border-radius:11px;background:#fbfdff} +.rx2-ratio-axis,.rx2-ratio-row{display:grid;grid-template-columns:minmax(92px,1.1fr) minmax(180px,4fr) 66px;gap:9px;align-items:center} +.rx2-ratio-axis{margin-bottom:5px;color:#66748b;font-size:.55rem} +.rx2-ratio-axis>*:nth-child(2){text-align:center} +.rx2-ratio-axis>*:last-child{text-align:right} +.rx2-ratio-row{min-height:21px;font-size:.57rem} +.rx2-ratio-row>span:first-child{overflow:hidden;text-overflow:ellipsis;white-space:nowrap;color:#33415a} +.rx2-ratio-row>b{text-align:right;color:#23385f;font-size:.58rem;font-variant-numeric:tabular-nums} +.rx2-ratio-track{position:relative;height:11px;border-radius:5px;background:linear-gradient(90deg,#eef2ff 0 49.7%,#99a9c2 49.7% 50.3%,#edfbf8 50.3% 100%)} +.rx2-ratio-track i{position:absolute;left:var(--left);width:max(2px,var(--width));height:100%;border-radius:4px;background:linear-gradient(90deg,#5b45ec,#08aa91)} + +.rx2-language-icon{display:grid;width:32px;height:32px;margin:auto;place-items:center;border-radius:50%;background:#f0f5fb;font-style:normal;font-size:1.35rem;line-height:1} + +@media(max-width:650px){ + .rx2-highlight-stats{grid-template-columns:repeat(2,minmax(0,1fr))} + .rx2-highlight-stat strong{font-size:clamp(1.2rem,7vw,1.75rem)} + .rx2-ratio-chart{overflow-x:auto} + .rx2-ratio-axis,.rx2-ratio-row{grid-template-columns:92px minmax(150px,1fr) 58px;min-width:330px} +} + /* Benchmark-linked language badges */ .rx2-language { text-decoration: none; diff --git a/docs/benchmarks/data/performance-snapshots.sha256 b/docs/benchmarks/data/performance-snapshots.sha256 index 9a597dc..97a5ad3 100644 --- a/docs/benchmarks/data/performance-snapshots.sha256 +++ b/docs/benchmarks/data/performance-snapshots.sha256 @@ -3,5 +3,13 @@ c6ae08cc0d32b61e59276e5d6d4911f23029a5c783d3fe2bc293d15c00991dd9 java-benchmark 1a0fefdcc773a46a5ab2155afe4c10a3043dfd56b834119a2aeed48d3c8ac1fe java-stemmer-speed-2026-08-25.csv 7bfa6bc150bfa626607f5529a68f5f6cd662c9d2404c0bc7ebf5be6bb3e5e1b9 java-english-coverage-accuracy-2026-08-25.csv e119f63793aa405f649c21e9c855837e7c3ff7e094b54a57af01424354915fe1 java-english-coverage-speed-2026-08-25.csv +09cb331a28fbcce9d1b8c9e3e2ee85797b5ef3f1a41d06dea58d651eec0dd20e java-benchmark-corpora-2026-09-10.csv +886266fe981535115f92914b88fd1b862efe1ca18da5f409e10b3e67b951b0b4 java-stemmer-accuracy-2026-09-10.csv +165b653523cdf04ebf8430044876efbade80aaddb5893edaad943b24858547c0 java-stemmer-speed-2026-09-10.csv +1ba2e3e0473454839975e0dc89c9cdd7a9273447559df1b8a57e5a553ebee730 java-english-coverage-accuracy-2026-09-10.csv +b45062a085af4cfb520ceea9360d7401ac45fb8d22fe91ea10cfd5b0f607e799 java-english-coverage-speed-2026-09-10.csv +f6624fdc966fe0aa2476a8593b87ed77576b5e31c255737a2bc281485e104c4a java-benchmark-corpora-2026-09-11.csv +61581612523036b2cbe765c87c351909eca9aaa5c7ee4cd3389fcfd180c65456 java-stemmer-accuracy-2026-09-11.csv +4090940bdbc7bcf89ddf24b596e2639d9d0b25c84824823612470358bdbd11ca java-stemmer-speed-2026-09-11.csv f248c84075cf4b5aa917032327d33020adae68c41041e4379652b5325a9e780b python-all-languages-batch-2026-08-25.csv 43d0de0301d7a6267e08ba3c3608232eb56981cda2722c08c604f9734b62abec python-all-languages-batch-2026-08-25.json diff --git a/docs/benchmarks/generalization.md b/docs/benchmarks/generalization.md index 983ec3f..9b951e4 100644 --- a/docs/benchmarks/generalization.md +++ b/docs/benchmarks/generalization.md @@ -4,7 +4,7 @@ Radixor learns transformations from lexical families rather than storing a close word-to-root answer list. This experiment measures how those transformations transfer to dictionary families that were not used to build the Java trie. -For every one of the 20 default models, complete dictionary rows are placed in a +For every one of the 143 default models, complete dictionary rows are placed in a frozen pseudorandom order. Exact-size, nested prefixes retain 10% through 100% of the rows for training. Five predeclared splits are evaluated against the complete dictionary; the primary `Unseen` columns exclude a held-out occurrence whenever its @@ -17,31 +17,31 @@ out-of-distribution text. See the [methodology and limitations](reference/genera ## All-Language Summary -Each cell is the language-macro mean of 20 per-language split medians, so large +Each cell is the language-macro mean across languages with a defined denominator, so large dictionaries do not dominate small ones. Changed-form exactness is the most demanding measure because it excludes words whose expected root is already the input token. | Training rows | Unseen all exact | Unseen changed exact | Unseen root preserved | | ---: | ---: | ---: | ---: | | 100% | n/a | n/a | n/a | -| 90% | 74.21% | 69.07% | 94.66% | -| 80% | 73.77% | 68.54% | 94.24% | -| 70% | 73.53% | 68.20% | 93.83% | -| 60% | 72.87% | 67.55% | 93.50% | -| 50% | 72.31% | 66.80% | 93.17% | -| 40% | 71.60% | 65.97% | 92.67% | -| 30% | 70.51% | 64.72% | 92.14% | -| 20% | 69.19% | 63.13% | 91.55% | -| 10% | 66.96% | 60.49% | 90.80% | +| 90% | 52.61% | 44.54% | 95.46% | +| 80% | 51.73% | 43.77% | 95.11% | +| 70% | 51.33% | 43.33% | 94.83% | +| 60% | 50.55% | 42.55% | 94.65% | +| 50% | 50.01% | 41.99% | 94.15% | +| 40% | 49.46% | 41.40% | 93.90% | +| 30% | 48.48% | 40.32% | 93.64% | +| 20% | 47.04% | 38.80% | 93.69% | +| 10% | 44.68% | 36.22% | 93.20% | The language range is material and is therefore not replaced by the macro mean. At 10% -knowledge, median unseen changed-form exactness ranges from **10.423%** -for Hebrew to **90.053%** for -Portuguese. At 90%, it ranges from **15.493%** -for Persian to **94.412%** for -Portuguese. The largest 10%–90% change is -**+19.147 pp** for -Norwegian Nynorsk. These are descriptive within-resource results, not a causal +knowledge, median unseen changed-form exactness ranges from **0.000%** +for Akan to **94.807%** for +Bashkir. At 90%, it ranges from **0.000%** +for Akan to **95.477%** for +Crimean Tatar. The largest 10%–90% change is +**+59.536 pp** for +Cree. These are descriptive within-resource results, not a causal ranking of language, script, dictionary size, or regularity. Every language's evidence-derived endpoint conclusion is also published on its separate [language benchmark page](languages/index.md), rather than inferred from this macro mean. @@ -53,62 +53,2502 @@ split range. `Whole all exact` retains comparability with the earlier English co curve but mixes trained and held-out rows; the unseen columns are the generalization evidence. At 100% there is no held-out set, so unseen metrics are `n/a`. +
+Adyghe — ady-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,635 / 1,635 | 0 | 0 | n/a | n/a | n/a | 99.907% | +| 90% | 1,472 / 1,635 | 2,029 | 2 | 63.708% (60.402–72.893) | 61.034% (57.760–71.222) | 90.798% (89.506–94.479) | 96.273% (95.944–97.211) | +| 80% | 1,308 / 1,635 | 4,056 | 6 | 65.656% (64.362–68.064) | 63.475% (62.127–66.184) | 89.877% (89.571–92.025) | 93.067% (92.831–93.558) | +| 70% | 1,145 / 1,635 | 6,088 | 7 | 65.177% (64.769–68.561) | 62.851% (62.366–66.643) | 91.376% (90.574–92.229) | 89.502% (89.404–90.523) | +| 60% | 981 / 1,635 | 8,135 | 8 | 64.665% (62.917–66.052) | 62.266% (60.499–63.814) | 91.718% (90.015–92.649) | 85.815% (85.073–86.369) | +| 50% | 818 / 1,635 | 10,164 | 9 | 65.683% (61.154–65.872) | 63.372% (58.603–63.687) | 91.043% (90.529–93.252) | 82.800% (80.517–82.908) | +| 40% | 654 / 1,635 | 12,212 | 8 | 63.323% (60.215–63.904) | 60.819% (57.274–61.521) | 91.224% (90.900–93.878) | 77.934% (76.112–78.292) | +| 30% | 491 / 1,635 | 14,235 | 7 | 61.064% (57.064–63.036) | 58.469% (53.887–60.594) | 91.068% (89.336–93.514) | 72.724% (69.940–74.109) | +| 20% | 327 / 1,635 | 16,279 | 6 | 60.496% (57.284–63.137) | 57.881% (54.289–60.874) | 90.192% (89.059–91.584) | 68.364% (65.845–70.515) | +| 10% | 164 / 1,635 | 18,317 | 3 | 60.912% (55.294–61.045) | 58.302% (52.020–58.764) | 90.136% (87.202–92.794) | 64.843% (59.756–64.942) | + +
+
+ +
+Afrikaans — af-za-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 170,974 / 170,974 | 0 | 0 | n/a | n/a | n/a | 98.544% | +| 90% | 153,877 / 170,974 | 29,397 | 783 | 91.405% (91.373–91.546) | 82.230% (81.935–82.270) | 98.382% (98.253–98.452) | 97.722% (97.710–97.741) | +| 80% | 136,779 / 170,974 | 59,014 | 1,374 | 91.038% (90.962–91.290) | 81.473% (81.359–82.033) | 98.239% (98.177–98.301) | 96.847% (96.839–96.904) | +| 70% | 119,682 / 170,974 | 88,657 | 1,817 | 90.796% (90.649–91.051) | 81.075% (80.734–81.540) | 98.164% (98.107–98.193) | 95.976% (95.927–96.032) | +| 60% | 102,584 / 170,974 | 118,535 | 2,098 | 90.496% (90.414–90.729) | 80.504% (80.357–80.939) | 98.017% (98.008–98.091) | 95.049% (95.006–95.113) | +| 50% | 85,487 / 170,974 | 148,525 | 2,175 | 90.278% (90.139–90.418) | 80.134% (79.899–80.400) | 97.915% (97.820–97.968) | 94.119% (94.044–94.170) | +| 40% | 68,390 / 170,974 | 178,815 | 2,135 | 89.906% (89.824–90.038) | 79.704% (79.440–79.787) | 97.765% (97.634–97.786) | 93.069% (93.021–93.154) | +| 30% | 51,292 / 170,974 | 209,118 | 1,874 | 89.518% (89.369–89.604) | 78.972% (78.656–79.104) | 97.556% (97.515–97.684) | 91.974% (91.869–92.039) | +| 20% | 34,195 / 170,974 | 239,864 | 1,406 | 88.938% (88.796–89.043) | 77.870% (77.696–78.045) | 97.372% (97.239–97.417) | 90.681% (90.555–90.740) | +| 10% | 17,097 / 170,974 | 270,632 | 804 | 87.881% (87.749–88.168) | 75.953% (75.644–76.671) | 96.964% (96.926–96.972) | 88.840% (88.718–89.104) | + +
+
+ +
+Aimele — ail-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 636 / 636 | 0 | 0 | n/a | n/a | n/a | 97.702% | +| 90% | 572 / 636 | 385 | 14 | 16.883% (13.127–25.103) | 2.020% (1.747–2.752) | 95.000% (93.103–96.774) | 87.520% (83.371–91.893) | +| 80% | 509 / 636 | 616 | 23 | 19.934% (15.162–24.950) | 1.670% (1.412–2.605) | 94.309% (91.057–97.521) | 81.998% (75.263–85.669) | +| 70% | 445 / 636 | 866 | 33 | 21.479% (18.200–23.916) | 1.799% (1.434–2.253) | 94.054% (92.350–95.676) | 76.253% (71.210–79.381) | +| 60% | 382 / 636 | 1,205 | 35 | 21.245% (20.238–23.425) | 2.077% (1.680–2.519) | 95.161% (91.463–95.968) | 67.826% (65.592–73.125) | +| 50% | 318 / 636 | 1,514 | 36 | 21.400% (20.619–24.302) | 1.931% (1.207–2.579) | 93.871% (92.258–95.793) | 60.294% (58.730–68.592) | +| 40% | 254 / 636 | 1,783 | 32 | 21.287% (21.134–21.967) | 1.986% (1.618–2.284) | 94.906% (92.761–96.543) | 53.687% (51.420–55.474) | +| 30% | 191 / 636 | 2,102 | 29 | 21.509% (19.918–21.690) | 1.562% (1.480–2.719) | 94.050% (93.593–94.977) | 45.547% (42.739–47.750) | +| 20% | 127 / 636 | 2,521 | 21 | 20.447% (19.767–20.903) | 1.686% (1.415–1.809) | 94.800% (91.929–95.652) | 35.238% (33.387–38.047) | +| 10% | 64 / 636 | 2,836 | 11 | 19.746% (18.628–21.392) | 1.598% (1.024–2.079) | 93.182% (90.893–95.431) | 26.939% (24.162–31.120) | + +
+
+ +
+Akan — aka-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 79 / 79 | 0 | 0 | n/a | n/a | n/a | 99.684% | +| 90% | 71 / 79 | 192 | 0 | 4.167% (3.646–4.167) | 0.000% | 100.000% (87.500–100.000) | 89.979% (89.926–89.979) | +| 80% | 63 / 79 | 384 | 0 | 4.167% (3.906–4.233) | 0.000% | 100.000% (93.750–100.000) | 80.274% (80.222–80.591) | +| 70% | 55 / 79 | 576 | 0 | 4.167% (4.167–4.211) | 0.000% | 100.000% | 70.570% (70.570–70.886) | +| 60% | 47 / 79 | 768 | 0 | 4.167% (4.036–4.199) | 0.000% | 100.000% (96.875–100.000) | 61.129% (60.865–61.181) | +| 50% | 40 / 79 | 936 | 0 | 4.167% (4.167–4.194) | 0.000% | 100.000% | 52.690% (52.373–52.690) | +| 40% | 32 / 79 | 1,128 | 0 | 4.167% (4.100–4.167) | 0.000% | 100.000% (97.872–100.000) | 42.932% (42.669–42.985) | +| 30% | 24 / 79 | 1,320 | 0 | 4.167% (4.091–4.186) | 0.000% | 100.000% (98.182–100.000) | 33.281% (32.911–33.281) | +| 20% | 16 / 79 | 1,512 | 0 | 4.167% (4.101–4.183) | 0.000% | 100.000% (98.413–100.000) | 23.576% (23.523–23.576) | +| 10% | 8 / 79 | 1,704 | 0 | 4.167% (4.108–4.167) | 0.000% | 100.000% (98.592–100.000) | 13.871% (13.819–13.871) | + +
+
+ +
+Albanian — sq-al-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 587 / 587 | 0 | 0 | n/a | n/a | n/a | 99.552% | +| 90% | 528 / 587 | 940 | 9 | 78.064% (76.395–84.163) | 77.573% (75.406–83.859) | 89.655% (83.051–91.525) | 97.808% (97.467–97.847) | +| 80% | 470 / 587 | 1,985 | 10 | 76.717% (73.480–84.213) | 75.953% (72.512–84.056) | 88.034% (86.842–90.598) | 95.090% (93.901–96.444) | +| 70% | 411 / 587 | 2,961 | 21 | 77.069% (75.762–81.269) | 76.346% (74.809–80.997) | 86.857% (85.632–91.429) | 92.936% (92.342–94.184) | +| 60% | 352 / 587 | 4,029 | 24 | 77.039% (75.500–79.709) | 76.328% (74.854–79.250) | 87.234% (85.897–90.987) | 90.559% (90.004–91.758) | +| 50% | 294 / 587 | 5,026 | 25 | 77.370% (75.716–78.898) | 76.705% (75.125–78.182) | 86.007% (84.880–90.722) | 88.416% (87.987–89.137) | +| 40% | 235 / 587 | 6,104 | 19 | 75.168% (74.636–79.673) | 74.350% (73.866–79.255) | 86.610% (84.943–88.889) | 84.811% (84.616–87.646) | +| 30% | 176 / 587 | 7,117 | 20 | 74.795% (73.907–76.690) | 74.149% (73.099–76.025) | 85.330% (84.597–87.561) | 82.424% (81.352–83.574) | +| 20% | 117 / 587 | 8,179 | 17 | 73.560% (70.760–74.716) | 72.611% (69.902–73.998) | 88.085% (84.681–89.362) | 78.478% (76.705–79.657) | +| 10% | 59 / 587 | 9,213 | 10 | 70.692% (66.959–71.613) | 69.493% (66.162–70.796) | 85.769% (80.114–90.720) | 73.207% (70.129–74.542) | + +
+
+ +
+Alsatian — gsw-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 145 / 145 | 0 | 0 | n/a | n/a | n/a | 99.664% | +| 90% | 131 / 145 | 115 | 0 | 58.261% (47.500–62.832) | 52.475% (41.509–57.576) | 100.000% (92.857–100.000) | 95.630% (94.370–96.134) | +| 80% | 116 / 145 | 239 | 1 | 58.333% (53.975–64.035) | 52.607% (48.095–58.794) | 100.000% (93.103–100.000) | 91.261% (90.420–92.773) | +| 70% | 102 / 145 | 351 | 1 | 57.434% (48.876–60.114) | 52.000% (43.131–54.545) | 97.674% (90.698–100.000) | 87.395% (84.370–87.899) | +| 60% | 87 / 145 | 474 | 2 | 53.376% (51.483–58.656) | 47.356% (45.652–53.349) | 98.276% (93.103–98.276) | 81.092% (80.084–82.941) | +| 50% | 73 / 145 | 583 | 2 | 55.326% (50.257–58.895) | 49.216% (44.031–53.254) | 98.611% (94.444–98.611) | 77.815% (74.454–79.748) | +| 40% | 58 / 145 | 704 | 2 | 54.298% (50.427–58.635) | 47.954% (44.065–53.485) | 95.402% (90.805–98.851) | 72.857% (70.504–75.210) | +| 30% | 44 / 145 | 820 | 2 | 52.683% (46.437–55.286) | 46.314% (39.832–49.584) | 96.040% (93.069–99.010) | 67.059% (63.109–68.824) | +| 20% | 29 / 145 | 947 | 1 | 51.760% (46.845–54.136) | 45.294% (40.171–47.795) | 99.138% (93.966–100.000) | 60.840% (57.983–62.941) | +| 10% | 15 / 145 | 1,063 | 2 | 45.908% (38.140–52.004) | 38.371% (29.762–45.387) | 99.231% (94.615–100.000) | 51.429% (44.958–56.555) | + +
+
+ +
+Amharic — am-et-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 2,461 / 2,461 | 0 | 0 | n/a | n/a | n/a | 97.282% | +| 90% | 2,215 / 2,461 | 4,069 | 202 | 46.550% (39.916–47.751) | 43.331% (36.518–44.590) | 97.531% (96.680–98.354) | 92.023% (91.199–92.360) | +| 80% | 1,969 / 2,461 | 8,235 | 333 | 43.519% (38.970–45.328) | 40.003% (35.355–42.111) | 97.101% (95.868–97.938) | 86.729% (85.352–86.988) | +| 70% | 1,723 / 2,461 | 12,121 | 377 | 41.850% (39.000–44.269) | 38.493% (35.534–40.924) | 97.383% (95.879–97.514) | 80.307% (79.306–81.765) | +| 60% | 1,477 / 2,461 | 16,542 | 525 | 40.444% (38.470–42.177) | 36.906% (34.866–38.795) | 96.498% (96.300–96.798) | 74.547% (73.555–75.206) | +| 50% | 1,231 / 2,461 | 20,765 | 580 | 39.639% (38.697–40.861) | 36.084% (35.202–37.461) | 96.623% (95.738–96.880) | 68.664% (67.180–69.069) | +| 40% | 984 / 2,461 | 25,111 | 594 | 39.166% (36.597–39.775) | 35.718% (32.959–36.175) | 96.575% (94.956–96.995) | 62.103% (59.969–63.249) | +| 30% | 738 / 2,461 | 29,207 | 581 | 36.289% (32.838–37.946) | 32.766% (28.996–34.293) | 96.021% (95.047–96.096) | 54.439% (51.594–55.963) | +| 20% | 492 / 2,461 | 33,549 | 530 | 34.913% (32.261–35.773) | 31.169% (28.327–32.005) | 95.621% (95.262–96.478) | 47.254% (45.012–48.465) | +| 10% | 246 / 2,461 | 38,138 | 231 | 31.003% (28.964–31.314) | 27.034% (24.945–27.301) | 95.436% (94.024–95.654) | 37.408% (35.239–38.481) | + +
+
+ +
+Amol Azerbaijani — azg-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 258 / 258 | 0 | 0 | n/a | n/a | n/a | 99.259% | +| 90% | 232 / 258 | 917 | 27 | 2.729% (2.658–2.873) | 0.000% | 100.000% | 89.821% (89.175–89.948) | +| 80% | 206 / 258 | 1,885 | 31 | 2.748% (2.665–2.761) | 0.000% | 100.000% (98.077–100.000) | 79.843% (79.144–80.256) | +| 70% | 181 / 258 | 2,780 | 23 | 2.734% (2.664–2.777) | 0.000% | 100.000% (98.701–100.000) | 70.618% (69.908–71.189) | +| 60% | 155 / 258 | 3,742 | 23 | 2.753% (2.688–2.803) | 0.000% | 100.000% | 61.042% (60.216–61.646) | +| 50% | 129 / 258 | 4,690 | 26 | 2.751% (2.722–2.764) | 0.000% | 100.000% | 51.223% (50.630–51.584) | +| 40% | 103 / 258 | 5,626 | 19 | 2.755% (2.697–2.759) | 0.000% | 100.000% (98.710–100.000) | 41.574% (41.002–42.029) | +| 30% | 77 / 258 | 6,604 | 27 | 2.737% (2.722–2.741) | 0.000% | 100.000% (98.895–100.000) | 31.745% (31.448–31.840) | +| 20% | 52 / 258 | 7,528 | 19 | 2.729% (2.720–2.736) | 0.000% | 99.515% (99.512–100.000) | 22.243% (22.148–22.413) | +| 10% | 26 / 258 | 8,490 | 14 | 2.728% (2.689–2.738) | 0.000% | 100.000% (98.276–100.000) | 12.403% (12.266–12.467) | + +
+
+ +
+Ancient Greek — grc-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 2,426 / 2,426 | 0 | 0 | n/a | n/a | n/a | 99.266% | +| 90% | 2,183 / 2,426 | 2,639 | 23 | 27.536% (24.744–31.815) | 20.639% (17.723–25.641) | 94.606% (94.239–95.062) | 92.118% (91.806–92.367) | +| 80% | 1,941 / 2,426 | 5,280 | 61 | 27.968% (27.018–33.015) | 21.224% (20.075–26.816) | 95.445% (94.617–96.480) | 85.027% (84.745–85.858) | +| 70% | 1,698 / 2,426 | 7,921 | 77 | 30.008% (27.640–31.131) | 23.347% (20.789–24.700) | 95.455% (94.759–96.000) | 78.451% (77.781–78.835) | +| 60% | 1,456 / 2,426 | 10,608 | 87 | 29.025% (27.219–30.210) | 22.443% (20.416–23.788) | 94.720% (94.324–95.553) | 71.126% (70.517–71.536) | +| 50% | 1,213 / 2,426 | 13,278 | 87 | 28.586% (27.607–29.318) | 21.972% (20.951–22.739) | 94.967% (93.476–95.782) | 63.820% (63.557–64.189) | +| 40% | 970 / 2,426 | 15,909 | 77 | 27.728% (27.441–29.312) | 20.891% (20.812–22.704) | 95.314% (93.466–96.143) | 56.484% (56.078–57.120) | +| 30% | 728 / 2,426 | 18,561 | 60 | 27.328% (25.502–28.164) | 20.377% (18.528–21.459) | 94.864% (94.752–95.868) | 49.016% (47.654–49.562) | +| 20% | 485 / 2,426 | 21,180 | 48 | 25.386% (23.521–25.890) | 18.309% (16.276–18.991) | 95.513% (94.487–96.233) | 40.529% (38.742–40.567) | +| 10% | 243 / 2,426 | 23,821 | 33 | 22.164% (20.788–24.125) | 14.963% (13.151–17.164) | 93.764% (91.976–96.564) | 29.984% (28.769–31.955) | + +
+
+ +
+Anglo-Norman — xno-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 5 / 5 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 5 / 5 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 80% | 4 / 5 | 35 | 0 | 8.571% (2.703–100.000) | 5.882% (0.000–100.000) | 100.000% | 82.796% (80.645–100.000) | +| 70% | 4 / 5 | 35 | 0 | 8.571% (2.703–100.000) | 5.882% (0.000–100.000) | 100.000% | 82.796% (80.645–100.000) | +| 60% | 3 / 5 | 72 | 0 | 8.571% (2.469–50.000) | 5.882% (0.000–48.571) | 100.000% (50.000–100.000) | 65.591% (57.527–80.645) | +| 50% | 3 / 5 | 72 | 0 | 8.571% (2.469–50.000) | 5.882% (0.000–48.571) | 100.000% (50.000–100.000) | 65.591% (57.527–80.645) | +| 40% | 2 / 5 | 116 | 0 | 23.364% (21.552–44.860) | 21.154% (19.469–43.269) | 100.000% | 55.914% (51.075–68.280) | +| 30% | 2 / 5 | 116 | 0 | 23.364% (21.552–44.860) | 21.154% (19.469–43.269) | 100.000% | 55.914% (51.075–68.280) | +| 20% | 1 / 5 | 151 | 0 | 39.735% (0.704–39.735) | 38.095% (0.000–38.095) | 100.000% (25.000–100.000) | 51.075% (24.194–51.075) | +| 10% | 1 / 5 | 151 | 0 | 39.735% (0.704–39.735) | 38.095% (0.000–38.095) | 100.000% (25.000–100.000) | 51.075% (24.194–51.075) | + +
+
+ +
+Arabic — ar-default 1.0.1 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 12,815 / 12,815 | 0 | 0 | n/a | n/a | n/a | 98.399% | +| 90% | 11,534 / 12,815 | 44,486 | 1,336 | 14.855% (14.616–15.068) | 12.470% (12.255–12.672) | 98.164% (97.828–99.127) | 90.200% (89.946–90.503) | +| 80% | 10,252 / 12,815 | 89,671 | 2,338 | 14.884% (14.592–15.182) | 12.439% (12.177–12.775) | 98.439% (98.031–98.806) | 81.877% (81.758–82.370) | +| 70% | 8,971 / 12,815 | 134,443 | 3,017 | 14.652% (14.379–14.913) | 12.232% (11.959–12.480) | 98.596% (98.002–98.618) | 73.591% (73.377–73.871) | +| 60% | 7,689 / 12,815 | 179,895 | 3,586 | 14.457% (14.354–14.575) | 12.045% (11.929–12.162) | 98.456% (97.892–98.610) | 65.184% (65.124–65.502) | +| 50% | 6,408 / 12,815 | 225,546 | 3,714 | 14.114% (13.896–14.429) | 11.685% (11.460–12.000) | 98.274% (98.022–98.483) | 56.669% (56.546–57.051) | +| 40% | 5,126 / 12,815 | 272,168 | 3,486 | 13.736% (13.635–13.847) | 11.282% (11.201–11.417) | 98.499% (98.223–98.516) | 48.011% (47.747–48.469) | +| 30% | 3,845 / 12,815 | 318,095 | 3,060 | 13.313% (13.200–13.416) | 10.851% (10.738–10.967) | 98.538% (98.324–98.605) | 39.342% (38.968–39.697) | +| 20% | 2,563 / 12,815 | 364,452 | 2,421 | 12.828% (12.761–12.975) | 10.371% (10.296–10.509) | 98.618% (98.431–98.763) | 30.424% (30.082–30.582) | +| 10% | 1,282 / 12,815 | 412,269 | 1,410 | 11.991% (11.707–12.516) | 9.492% (9.200–10.055) | 98.800% (98.339–99.113) | 21.070% (20.598–21.276) | + +
+
+ +
+Armenian — hy-am-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 6,990 / 6,990 | 0 | 0 | n/a | n/a | n/a | 99.505% | +| 90% | 6,291 / 6,990 | 24,487 | 235 | 69.740% (68.616–72.259) | 69.080% (67.922–71.648) | 93.835% (93.383–95.152) | 96.521% (96.381–96.805) | +| 80% | 5,592 / 6,990 | 49,373 | 396 | 69.925% (68.793–71.043) | 69.267% (68.105–70.382) | 93.568% (93.328–94.654) | 93.530% (93.317–93.827) | +| 70% | 4,893 / 6,990 | 74,063 | 525 | 69.445% (69.174–70.424) | 68.763% (68.515–69.745) | 93.373% (92.537–94.431) | 90.446% (90.341–90.825) | +| 60% | 4,194 / 6,990 | 98,745 | 578 | 69.346% (68.769–70.333) | 68.695% (68.106–69.680) | 93.306% (92.451–93.456) | 87.365% (87.146–87.771) | +| 50% | 3,495 / 6,990 | 123,633 | 623 | 68.911% (68.483–69.122) | 68.226% (67.807–68.446) | 92.769% (92.473–93.117) | 84.082% (83.901–84.304) | +| 40% | 2,796 / 6,990 | 148,341 | 573 | 68.280% (67.572–68.505) | 67.598% (66.889–67.827) | 92.302% (91.640–92.490) | 80.664% (80.255–80.854) | +| 30% | 2,097 / 6,990 | 173,169 | 494 | 67.599% (66.815–68.266) | 66.900% (66.110–67.588) | 91.549% (90.862–92.046) | 77.114% (76.494–77.586) | +| 20% | 1,398 / 6,990 | 198,108 | 392 | 66.540% (65.927–68.287) | 65.815% (65.216–67.626) | 91.304% (90.282–91.762) | 73.068% (72.578–74.480) | +| 10% | 699 / 6,990 | 223,028 | 216 | 65.016% (64.402–66.160) | 64.272% (63.678–65.429) | 90.256% (89.494–91.529) | 68.418% (67.847–69.416) | + +
+
+ +
+Ashaninka — cni-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 407 / 407 | 0 | 0 | n/a | n/a | n/a | 99.664% | +| 90% | 366 / 407 | 1,019 | 1 | 9.582% (7.150–9.715) | 5.816% (3.425–5.930) | 100.000% (97.500–100.000) | 90.552% (89.920–90.907) | +| 80% | 326 / 407 | 2,009 | 6 | 9.388% (8.211–10.147) | 5.519% (4.478–6.296) | 100.000% (98.750–100.000) | 82.032% (81.183–82.417) | +| 70% | 285 / 407 | 3,016 | 21 | 9.798% (9.052–10.326) | 5.933% (5.285–6.489) | 99.174% (99.174–100.000) | 73.058% (72.643–73.670) | +| 60% | 244 / 407 | 4,031 | 22 | 9.642% (8.805–10.083) | 5.899% (5.024–6.250) | 99.383% (98.773–100.000) | 63.827% (63.323–64.893) | +| 50% | 204 / 407 | 5,010 | 21 | 9.014% (8.565–9.610) | 5.251% (4.798–5.746) | 99.505% (99.010–100.000) | 54.843% (54.072–55.929) | +| 40% | 163 / 407 | 6,048 | 8 | 9.031% (8.399–9.753) | 5.224% (4.597–5.909) | 99.590% (99.180–100.000) | 45.572% (44.851–46.688) | +| 30% | 122 / 407 | 7,103 | 8 | 8.971% (8.053–9.661) | 5.149% (4.259–5.834) | 99.649% (99.298–99.649) | 36.094% (34.949–37.457) | +| 20% | 81 / 407 | 8,134 | 7 | 8.982% (8.608–9.487) | 5.238% (4.815–5.663) | 99.693% (99.080–100.000) | 26.893% (26.419–27.900) | +| 10% | 41 / 407 | 9,122 | 2 | 8.724% (8.653–9.172) | 4.872% (4.857–5.393) | 99.726% (99.180–100.000) | 17.889% (17.603–18.393) | + +
+
+ +
+Assamese — as-in-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,466 / 1,466 | 0 | 0 | n/a | n/a | n/a | 97.206% | +| 90% | 1,319 / 1,466 | 6,589 | 327 | 69.829% (63.969–70.714) | 69.204% (63.366–70.198) | 96.183% (93.846–100.000) | 94.366% (93.693–94.436) | +| 80% | 1,173 / 1,466 | 13,249 | 759 | 68.346% (65.575–71.057) | 67.743% (64.988–70.545) | 95.817% (94.340–97.753) | 91.168% (90.844–91.824) | +| 70% | 1,026 / 1,466 | 20,148 | 888 | 67.749% (64.788–69.603) | 67.167% (64.183–69.069) | 95.309% (94.103–96.049) | 88.187% (87.500–88.532) | +| 60% | 880 / 1,466 | 26,947 | 893 | 66.512% (65.657–67.539) | 65.905% (65.067–66.965) | 95.082% (93.885–95.795) | 84.602% (84.515–85.039) | +| 50% | 733 / 1,466 | 33,854 | 944 | 66.293% (64.847–66.976) | 65.713% (64.238–66.403) | 94.614% (93.974–95.708) | 81.583% (80.610–81.770) | +| 40% | 586 / 1,466 | 40,824 | 907 | 65.769% (65.293–66.521) | 65.139% (64.673–65.913) | 94.793% (93.972–95.611) | 78.074% (77.537–78.416) | +| 30% | 440 / 1,466 | 47,838 | 910 | 65.115% (64.689–65.986) | 64.484% (64.040–65.363) | 95.110% (93.496–95.190) | 74.658% (74.261–75.198) | +| 20% | 293 / 1,466 | 55,106 | 681 | 64.005% (63.915–64.449) | 63.345% (63.273–63.816) | 94.741% (93.945–96.007) | 70.591% (70.189–70.875) | +| 10% | 147 / 1,466 | 62,369 | 382 | 61.905% (61.513–62.974) | 61.204% (60.788–62.265) | 95.336% (94.576–96.296) | 65.405% (64.991–66.293) | + +
+
+ +
+Asturian — ast-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 436 / 436 | 0 | 0 | n/a | n/a | n/a | 98.896% | +| 90% | 392 / 436 | 2,196 | 110 | 89.355% (87.773–91.678) | 89.149% (87.524–91.505) | 100.000% | 97.792% (97.691–98.076) | +| 80% | 349 / 436 | 4,327 | 114 | 89.210% (86.525–89.996) | 88.989% (86.269–89.803) | 100.000% (97.701–100.000) | 96.775% (96.285–96.825) | +| 70% | 305 / 436 | 6,479 | 114 | 88.082% (85.725–89.040) | 87.874% (85.445–88.812) | 100.000% (98.473–100.000) | 95.263% (95.066–95.694) | +| 60% | 262 / 436 | 8,585 | 114 | 87.325% (85.906–88.106) | 87.063% (85.626–87.887) | 99.422% (98.844–100.000) | 93.967% (93.376–94.187) | +| 50% | 218 / 436 | 10,893 | 116 | 86.606% (85.490–86.964) | 86.354% (85.200–86.708) | 99.078% (98.157–99.541) | 92.332% (91.796–92.877) | +| 40% | 174 / 436 | 13,032 | 128 | 85.228% (84.695–87.630) | 84.937% (84.391–87.384) | 99.615% (98.467–99.618) | 90.325% (89.867–92.020) | +| 30% | 131 / 436 | 15,205 | 198 | 85.962% (84.967–86.223) | 85.695% (84.675–85.950) | 99.340% (97.690–99.672) | 89.432% (88.621–89.689) | +| 20% | 87 / 436 | 17,403 | 106 | 82.158% (81.228–85.516) | 81.830% (80.868–85.228) | 99.425% (98.271–99.713) | 84.851% (84.572–88.351) | +| 10% | 44 / 436 | 19,563 | 17 | 83.849% (83.082–86.007) | 83.534% (82.762–85.733) | 99.233% (98.462–99.745) | 85.140% (84.723–87.352) | + +
+
+ +
+Aymara — aym-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 3,319 / 3,319 | 0 | 0 | n/a | n/a | n/a | 99.580% | +| 90% | 2,987 / 3,319 | 30,890 | 268 | 90.939% (90.434–91.492) | 90.955% (90.456–91.509) | 89.557% (88.141–90.675) | 98.703% (98.567–98.735) | +| 80% | 2,655 / 3,319 | 62,588 | 500 | 90.889% (90.131–91.661) | 90.916% (90.151–91.689) | 88.942% (88.133–89.748) | 97.765% (97.610–97.972) | +| 70% | 2,323 / 3,319 | 92,944 | 571 | 90.699% (90.397–91.066) | 90.732% (90.415–91.094) | 88.279% (87.461–88.620) | 96.848% (96.722–96.934) | +| 60% | 1,991 / 3,319 | 122,831 | 610 | 90.413% (90.137–91.038) | 90.448% (90.153–91.073) | 87.744% (86.776–88.629) | 95.866% (95.747–96.134) | +| 50% | 1,660 / 3,319 | 153,956 | 656 | 90.316% (89.707–91.066) | 90.341% (89.726–91.109) | 86.994% (85.688–87.973) | 94.871% (94.570–95.293) | +| 40% | 1,328 / 3,319 | 184,980 | 689 | 90.329% (89.458–90.748) | 90.383% (89.499–90.789) | 86.193% (85.134–87.203) | 93.932% (93.437–94.259) | +| 30% | 996 / 3,319 | 215,696 | 511 | 90.244% (89.589–90.929) | 90.292% (89.642–90.982) | 85.983% (84.720–86.951) | 92.879% (92.582–93.498) | +| 20% | 664 / 3,319 | 247,764 | 396 | 90.311% (89.061–90.407) | 90.364% (89.116–90.463) | 85.322% (83.935–86.779) | 92.083% (91.174–92.186) | +| 10% | 332 / 3,319 | 278,289 | 219 | 89.745% (89.150–90.373) | 89.796% (89.196–90.418) | 85.017% (83.036–86.137) | 90.738% (90.186–91.195) | + +
+
+ +
+Azerbaijani — az-az-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 340 / 340 | 0 | 0 | n/a | n/a | n/a | 99.970% | +| 90% | 306 / 340 | 683 | 0 | 80.137% (72.506–83.980) | 79.741% (71.703–83.721) | 82.353% (81.818–88.235) | 97.796% (97.587–98.546) | +| 80% | 272 / 340 | 1,436 | 0 | 79.817% (74.875–83.503) | 79.561% (74.425–83.274) | 85.294% (82.353–88.235) | 95.758% (95.353–96.327) | +| 70% | 238 / 340 | 2,086 | 1 | 79.703% (74.703–80.968) | 79.426% (73.985–80.645) | 85.294% (82.353–87.255) | 93.404% (92.490–94.019) | +| 60% | 204 / 340 | 2,737 | 1 | 77.766% (75.462–80.097) | 77.422% (74.834–79.622) | 86.567% (83.088–88.971) | 90.736% (90.571–91.980) | +| 50% | 170 / 340 | 3,370 | 1 | 76.795% (73.211–78.911) | 76.406% (72.477–78.466) | 87.059% (84.118–87.647) | 88.248% (86.164–89.582) | +| 40% | 136 / 340 | 4,078 | 1 | 74.129% (70.682–77.229) | 73.696% (69.828–76.734) | 87.129% (82.353–91.176) | 84.185% (81.982–86.359) | +| 30% | 102 / 340 | 4,643 | 1 | 74.275% (70.993–74.646) | 73.472% (69.996–74.134) | 90.254% (84.034–90.756) | 82.162% (79.134–82.536) | +| 20% | 68 / 340 | 5,282 | 1 | 71.379% (67.200–72.903) | 70.619% (65.970–71.929) | 86.029% (85.294–90.809) | 76.555% (74.202–78.579) | +| 10% | 34 / 340 | 5,924 | 0 | 67.201% (66.164–70.809) | 65.931% (64.925–69.903) | 88.562% (86.601–91.830) | 70.874% (70.394–73.857) | + +
+
+ +
+Bashkir — bak-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,059 / 1,059 | 0 | 0 | n/a | n/a | n/a | 99.823% | +| 90% | 953 / 1,059 | 1,077 | 1 | 95.079% (94.013–95.810) | 95.060% (93.880–96.281) | 95.283% (91.509–95.283) | 99.330% (99.209–99.460) | +| 80% | 847 / 1,059 | 2,157 | 7 | 95.178% (94.265–96.219) | 95.123% (94.204–96.272) | 95.283% (91.509–95.714) | 98.855% (98.706–99.050) | +| 70% | 741 / 1,059 | 3,263 | 10 | 95.221% (94.637–96.636) | 95.152% (94.565–96.681) | 95.283% (92.722–96.215) | 98.352% (98.250–98.799) | +| 60% | 635 / 1,059 | 4,295 | 14 | 95.074% (94.656–96.497) | 95.082% (94.615–96.554) | 95.035% (93.128–95.972) | 97.859% (97.701–98.436) | +| 50% | 530 / 1,059 | 5,393 | 8 | 95.179% (93.232–96.115) | 95.307% (93.053–96.272) | 94.697% (93.182–96.183) | 97.468% (96.444–97.980) | +| 40% | 424 / 1,059 | 6,475 | 8 | 95.232% (94.707–95.814) | 95.196% (94.646–95.984) | 95.118% (93.533–95.563) | 97.002% (96.760–97.440) | +| 30% | 318 / 1,059 | 7,506 | 3 | 94.745% (93.808–96.363) | 94.850% (93.898–96.497) | 93.919% (92.973–95.135) | 96.258% (95.606–97.431) | +| 20% | 212 / 1,059 | 8,600 | 2 | 94.695% (93.731–95.450) | 94.976% (93.679–95.559) | 94.194% (92.090–94.444) | 95.708% (94.992–96.323) | +| 10% | 106 / 1,059 | 9,669 | 1 | 94.564% (92.675–95.406) | 94.807% (92.643–95.764) | 92.130% (91.396–92.962) | 95.085% (93.418–95.848) | + +
+
+ +
+Belarusian — be-by-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,020 / 1,020 | 0 | 0 | n/a | n/a | n/a | 99.248% | +| 90% | 918 / 1,020 | 1,951 | 15 | 47.307% (46.233–48.319) | 44.700% (43.591–45.827) | 94.000% (89.216–96.000) | 93.977% (93.916–94.170) | +| 80% | 816 / 1,020 | 3,857 | 29 | 45.673% (43.494–49.619) | 42.969% (40.655–47.024) | 93.564% (89.055–97.525) | 88.808% (88.224–89.189) | +| 70% | 714 / 1,020 | 5,851 | 69 | 46.233% (43.360–48.096) | 43.642% (40.653–45.407) | 92.763% (89.735–97.360) | 83.405% (82.440–83.842) | +| 60% | 612 / 1,020 | 7,832 | 77 | 44.005% (43.644–47.319) | 41.432% (40.979–44.796) | 92.611% (90.842–94.321) | 77.149% (76.900–78.424) | +| 50% | 510 / 1,020 | 9,726 | 96 | 42.937% (40.520–44.971) | 40.215% (37.605–42.359) | 93.124% (92.141–94.280) | 71.349% (69.779–72.183) | +| 40% | 408 / 1,020 | 11,732 | 96 | 40.639% (38.066–42.009) | 37.807% (34.990–39.373) | 92.799% (90.296–95.074) | 64.025% (62.684–65.027) | +| 30% | 306 / 1,020 | 13,708 | 63 | 37.083% (36.525–37.543) | 34.021% (33.443–34.603) | 92.847% (91.268–94.803) | 55.578% (55.375–56.066) | +| 20% | 204 / 1,020 | 15,681 | 49 | 32.223% (31.519–33.429) | 29.004% (28.180–30.250) | 91.656% (90.909–92.638) | 45.586% (45.057–46.521) | +| 10% | 102 / 1,020 | 17,677 | 28 | 25.195% (22.885–26.106) | 21.488% (19.003–22.659) | 92.810% (89.107–94.111) | 32.778% (30.287–33.504) | + +
+
+ +
+Bengali — bn-bd-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 111 / 111 | 0 | 0 | n/a | n/a | n/a | 99.720% | +| 90% | 100 / 111 | 247 | 0 | 58.039% (36.025–78.261) | 56.557% (33.762–77.333) | 90.909% (90.909–100.000) | 95.972% (92.504–98.494) | +| 80% | 89 / 111 | 534 | 5 | 48.339% (35.768–60.263) | 46.538% (33.203–58.101) | 95.455% (90.909–100.000) | 89.912% (85.884–94.431) | +| 70% | 78 / 111 | 730 | 5 | 41.404% (37.500–56.516) | 38.647% (34.820–54.902) | 96.970% (93.939–100.000) | 85.394% (78.599–88.231) | +| 60% | 67 / 111 | 1,049 | 5 | 50.334% (45.455–57.261) | 48.259% (43.157–55.651) | 97.727% (95.455–100.000) | 81.471% (75.061–83.433) | +| 50% | 56 / 111 | 1,300 | 8 | 51.530% (34.463–54.996) | 49.452% (32.185–53.043) | 96.364% (96.364–98.182) | 78.459% (63.152–79.685) | +| 40% | 44 / 111 | 1,611 | 8 | 44.726% (22.373–51.928) | 42.462% (19.623–49.756) | 98.507% (97.015–98.507) | 68.476% (47.461–74.396) | +| 30% | 33 / 111 | 1,961 | 8 | 38.869% (25.565–49.720) | 36.416% (22.716–47.485) | 98.718% (97.436–98.718) | 57.548% (45.464–68.231) | +| 20% | 22 / 111 | 2,249 | 5 | 35.403% (27.069–42.139) | 32.905% (24.441–39.865) | 96.629% (92.135–100.000) | 51.979% (39.229–53.485) | +| 10% | 11 / 111 | 2,592 | 0 | 33.924% (25.077–39.100) | 31.435% (22.472–36.825) | 96.000% (87.000–100.000) | 39.755% (31.699–44.518) | + +
+
+ +
+Bininj Kun-wok — gup-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 73 / 73 | 0 | 0 | n/a | n/a | n/a | 99.730% | +| 90% | 66 / 73 | 31 | 0 | 22.581% (13.208–44.444) | 0.000% (0.000–9.091) | 100.000% (85.714–100.000) | 93.261% (87.332–97.035) | +| 80% | 58 / 73 | 83 | 0 | 18.421% (14.286–24.590) | 0.000% (0.000–1.639) | 100.000% (86.667–100.000) | 81.671% (77.089–87.332) | +| 70% | 51 / 73 | 127 | 0 | 17.969% (14.894–19.444) | 0.000% (0.000–1.163) | 95.455% (90.909–100.000) | 71.429% (67.385–76.280) | +| 60% | 44 / 73 | 162 | 0 | 17.442% (17.262–19.580) | 0.833% (0.000–1.754) | 96.552% (89.655–96.552) | 63.612% (61.456–68.733) | +| 50% | 37 / 73 | 192 | 0 | 18.325% (16.908–18.919) | 0.585% (0.000–1.342) | 97.222% (91.667–97.222) | 57.682% (53.369–59.299) | +| 40% | 29 / 73 | 231 | 1 | 19.556% (17.928–19.913) | 0.552% (0.483–1.604) | 97.727% (95.455–100.000) | 49.865% (44.205–50.943) | +| 30% | 22 / 73 | 269 | 1 | 19.847% (17.568–21.687) | 0.847% (0.408–2.525) | 100.000% (96.078–100.000) | 42.049% (33.962–47.170) | +| 20% | 15 / 73 | 303 | 1 | 19.865% (18.692–21.107) | 0.797% (0.418–2.165) | 100.000% (96.552–100.000) | 34.771% (29.380–38.275) | +| 10% | 7 / 73 | 333 | 0 | 20.000% (19.130–21.530) | 0.000% (0.000–3.833) | 100.000% (98.485–100.000) | 28.032% (24.798–29.380) | + +
+
+ +
+Braj — bra-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,246 / 1,246 | 0 | 0 | n/a | n/a | n/a | 99.656% | +| 90% | 1,121 / 1,246 | 197 | 1 | 65.641% (59.494–73.171) | 16.964% (7.500–18.947) | 94.355% (92.800–97.600) | 96.364% (94.939–97.494) | +| 80% | 997 / 1,246 | 398 | 2 | 64.500% (58.730–68.320) | 15.328% (9.896–16.556) | 93.574% (91.935–96.386) | 92.727% (90.713–94.005) | +| 70% | 872 / 1,246 | 633 | 3 | 61.538% (60.436–63.714) | 13.060% (11.583–16.667) | 94.385% (91.935–95.989) | 87.862% (87.273–89.189) | +| 60% | 748 / 1,246 | 842 | 2 | 61.447% (60.570–64.554) | 12.319% (11.628–18.611) | 93.763% (92.540–95.582) | 83.489% (83.440–86.192) | +| 50% | 623 / 1,246 | 1,031 | 2 | 61.463% (60.674–65.661) | 12.010% (10.918–16.972) | 93.730% (92.765–95.666) | 80.197% (79.214–83.440) | +| 40% | 498 / 1,246 | 1,220 | 4 | 62.479% (60.841–64.370) | 12.288% (10.474–14.110) | 94.779% (91.432–95.588) | 77.641% (75.971–78.821) | +| 30% | 374 / 1,246 | 1,426 | 3 | 62.412% (61.544–64.414) | 11.372% (9.009–14.359) | 95.183% (94.839–95.757) | 73.415% (72.875–75.332) | +| 20% | 249 / 1,246 | 1,614 | 3 | 63.139% (60.509–65.045) | 11.437% (5.352–12.898) | 95.683% (94.679–96.690) | 70.074% (67.912–72.826) | +| 10% | 125 / 1,246 | 1,846 | 1 | 61.038% (59.839–62.638) | 6.586% (5.660–14.141) | 95.183% (90.812–97.324) | 64.963% (63.145–66.634) | + +
+
+ +
+Breton — bre-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 44 / 44 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 40 / 44 | 174 | 0 | 13.793% (2.963–26.286) | 11.765% (0.000–24.561) | 100.000% | 92.268% (92.216–93.351) | +| 80% | 35 / 44 | 394 | 0 | 18.020% (2.247–28.571) | 16.104% (0.000–27.154) | 88.889% (88.889–100.000) | 83.351% (79.897–85.567) | +| 70% | 31 / 44 | 575 | 0 | 17.419% (8.958–23.986) | 15.815% (6.988–22.383) | 92.308% (92.308–100.000) | 75.258% (71.186–77.784) | +| 60% | 26 / 44 | 829 | 0 | 11.947% (7.212–17.884) | 10.256% (5.160–16.108) | 94.444% (88.889–100.000) | 62.010% (60.206–66.392) | +| 50% | 22 / 44 | 1,003 | 0 | 10.267% (6.250–19.298) | 8.476% (4.260–17.529) | 90.909% (86.364–95.455) | 54.330% (51.289–59.691) | +| 40% | 18 / 44 | 1,144 | 0 | 6.854% (5.748–20.367) | 5.036% (3.630–18.605) | 92.308% (84.615–100.000) | 45.361% (42.526–53.041) | +| 30% | 13 / 44 | 1,361 | 0 | 10.914% (4.864–17.414) | 9.130% (3.156–15.639) | 93.548% (74.194–100.000) | 35.206% (33.454–42.062) | +| 20% | 9 / 44 | 1,546 | 0 | 7.633% (7.246–18.499) | 5.745% (5.412–16.744) | 94.286% (80.000–97.143) | 26.907% (24.124–35.052) | +| 10% | 4 / 44 | 1,765 | 0 | 7.025% (4.703–11.841) | 4.986% (2.493–9.855) | 97.500% (95.000–100.000) | 15.412% (13.299–19.794) | + +
+
+ +
+Bulgarian — bg-bg-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 2,413 / 2,413 | 0 | 0 | n/a | n/a | n/a | 99.732% | +| 90% | 2,172 / 2,413 | 4,485 | 21 | 62.842% (55.624–66.042) | 61.341% (53.532–64.641) | 91.489% (90.833–94.142) | 96.096% (95.480–96.416) | +| 80% | 1,930 / 2,413 | 9,005 | 41 | 61.838% (60.104–62.754) | 60.148% (58.375–61.028) | 92.034% (91.476–93.347) | 92.260% (91.737–92.445) | +| 70% | 1,689 / 2,413 | 13,589 | 51 | 62.033% (59.657–64.736) | 60.374% (57.731–63.175) | 92.190% (90.972–93.084) | 88.232% (88.116–89.318) | +| 60% | 1,448 / 2,413 | 18,284 | 67 | 60.960% (60.091–63.742) | 59.094% (58.404–62.132) | 91.946% (90.084–93.965) | 84.528% (83.873–85.432) | +| 50% | 1,207 / 2,413 | 22,788 | 62 | 60.919% (58.660–62.113) | 59.080% (56.877–60.518) | 91.319% (90.341–93.917) | 80.289% (79.501–80.705) | +| 40% | 965 / 2,413 | 27,478 | 56 | 60.884% (59.487–61.497) | 59.175% (57.747–59.793) | 91.023% (90.235–92.169) | 76.396% (75.952–76.866) | +| 30% | 724 / 2,413 | 31,900 | 48 | 59.890% (57.711–60.748) | 58.132% (55.892–59.057) | 90.309% (89.667–91.400) | 72.011% (70.742–72.403) | +| 20% | 483 / 2,413 | 36,589 | 34 | 57.780% (54.358–58.649) | 55.927% (52.405–56.854) | 90.177% (89.195–91.178) | 66.193% (63.561–67.149) | +| 10% | 241 / 2,413 | 41,101 | 20 | 52.580% (50.622–54.376) | 50.586% (48.563–52.412) | 89.590% (87.471–90.355) | 57.661% (55.856–59.126) | + +
+
+ +
+Catalan — ca-es-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 15,176 / 15,176 | 0 | 0 | n/a | n/a | n/a | 95.955% | +| 90% | 13,658 / 15,176 | 13,444 | 981 | 80.281% (77.943–81.908) | 78.945% (76.244–80.465) | 92.780% (91.913–93.682) | 94.061% (93.820–94.320) | +| 80% | 12,141 / 15,176 | 26,286 | 1,724 | 79.707% (78.367–81.535) | 78.102% (76.515–80.354) | 92.462% (91.957–92.760) | 92.202% (92.111–92.358) | +| 70% | 10,623 / 15,176 | 38,922 | 2,238 | 79.078% (78.033–80.075) | 77.418% (76.351–78.715) | 91.822% (91.232–92.151) | 90.342% (90.158–90.551) | +| 60% | 9,106 / 15,176 | 51,320 | 2,797 | 78.592% (78.180–79.596) | 77.046% (76.614–78.158) | 91.324% (90.954–91.578) | 88.564% (88.039–88.906) | +| 50% | 7,588 / 15,176 | 65,160 | 2,723 | 78.596% (77.366–78.924) | 77.137% (75.791–77.459) | 90.746% (90.203–90.998) | 86.576% (85.991–86.916) | +| 40% | 6,070 / 15,176 | 78,802 | 2,632 | 77.563% (76.767–78.089) | 75.963% (75.151–76.617) | 89.934% (89.512–90.348) | 84.453% (83.976–84.707) | +| 30% | 4,553 / 15,176 | 92,174 | 2,318 | 76.912% (75.581–77.281) | 75.415% (73.827–75.819) | 89.040% (88.848–89.559) | 82.231% (81.131–82.607) | +| 20% | 3,035 / 15,176 | 106,349 | 1,781 | 75.570% (74.288–77.080) | 73.997% (72.516–75.654) | 88.558% (88.098–88.762) | 79.258% (78.195–80.619) | +| 10% | 1,518 / 15,176 | 121,030 | 1,030 | 73.460% (72.114–76.283) | 71.772% (70.147–74.931) | 87.789% (86.869–88.353) | 75.544% (74.342–78.068) | + +
+
+ +
+Cebuano — ceb-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 95 / 95 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 86 / 95 | 39 | 0 | 23.077% (20.930–24.324) | 0.000% | 100.000% | 93.023% (92.093–93.488) | +| 80% | 76 / 95 | 85 | 0 | 22.353% (21.111–23.171) | 0.000% | 100.000% | 84.651% (83.488–85.349) | +| 70% | 67 / 95 | 127 | 0 | 22.047% (21.875–22.400) | 0.000% | 100.000% | 76.977% (76.744–77.442) | +| 60% | 57 / 95 | 172 | 0 | 22.093% (21.469–22.093) | 0.000% | 100.000% | 68.837% (67.674–68.837) | +| 50% | 48 / 95 | 212 | 0 | 22.170% (21.461–22.816) | 0.000% | 100.000% | 61.628% (60.000–63.023) | +| 40% | 38 / 95 | 258 | 0 | 22.093% (21.839–22.441) | 0.000% | 100.000% | 53.256% (52.558–54.186) | +| 30% | 29 / 95 | 300 | 0 | 22.000% (21.523–22.297) | 0.000% | 100.000% (98.485–100.000) | 45.581% (44.419–46.512) | +| 20% | 19 / 95 | 344 | 0 | 22.093% (21.714–22.485) | 0.000% | 100.000% | 37.674% (36.279–39.070) | +| 10% | 10 / 95 | 384 | 0 | 21.990% (21.851–22.310) | 0.000% | 100.000% (98.824–100.000) | 30.465% (29.302–31.163) | + +
+
+ +
+Chichewa — ny-mw-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 227 / 227 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 204 / 227 | 322 | 0 | 7.143% (6.832–7.143) | 0.000% | 100.000% (95.652–100.000) | 90.592% (90.560–90.592) | +| 80% | 182 / 227 | 630 | 0 | 7.143% (6.984–7.143) | 0.000% | 100.000% (97.778–100.000) | 81.592% (81.561–81.592) | +| 70% | 159 / 227 | 952 | 0 | 7.143% (6.933–7.143) | 0.000% | 100.000% (97.059–100.000) | 72.184% (72.121–72.184) | +| 60% | 136 / 227 | 1,274 | 0 | 7.143% (6.986–7.143) | 0.000% | 100.000% (97.802–100.000) | 62.775% (62.712–62.775) | +| 50% | 114 / 227 | 1,582 | 0 | 7.143% (7.080–7.143) | 0.000% | 100.000% (99.115–100.000) | 53.776% (53.744–53.776) | +| 40% | 91 / 227 | 1,904 | 0 | 7.143% (7.090–7.143) | 0.000% | 100.000% (99.265–100.000) | 44.368% (44.336–44.368) | +| 30% | 68 / 227 | 2,226 | 0 | 7.143% (7.098–7.143) | 0.000% | 100.000% (99.371–100.000) | 34.959% (34.928–34.959) | +| 20% | 45 / 227 | 2,548 | 0 | 7.143% (7.104–7.143) | 0.000% | 100.000% (99.451–100.000) | 25.551% (25.519–25.551) | +| 10% | 23 / 227 | 2,856 | 0 | 7.108% (7.108–7.143) | 0.000% | 99.510% (99.510–100.000) | 16.520% (16.520–16.551) | + +
+
+ +
+Chukchi — ckt-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 191 / 191 | 0 | 0 | n/a | n/a | n/a | 99.342% | +| 90% | 172 / 191 | 27 | 0 | 69.231% (63.333–76.923) | 8.333% (0.000–14.286) | 100.000% (94.737–100.000) | 96.711% (95.724–97.368) | +| 80% | 153 / 191 | 60 | 1 | 63.333% (57.812–73.077) | 4.545% (0.000–14.286) | 97.368% (91.667–100.000) | 92.105% (90.461–94.737) | +| 70% | 134 / 191 | 89 | 1 | 61.798% (58.163–70.732) | 6.061% (3.125–8.000) | 98.214% (94.545–100.000) | 88.158% (85.855–91.447) | +| 60% | 115 / 191 | 118 | 1 | 61.864% (56.693–69.091) | 2.564% (1.887–5.882) | 96.000% (94.737–100.000) | 84.539% (81.250–88.158) | +| 50% | 96 / 191 | 152 | 1 | 60.127% (58.553–69.565) | 3.125% (1.724–6.977) | 94.737% (93.617–98.936) | 78.947% (78.618–85.526) | +| 40% | 76 / 191 | 188 | 1 | 59.574% (57.979–64.943) | 2.703% (1.351–6.667) | 95.614% (94.690–96.522) | 74.342% (73.355–79.276) | +| 30% | 57 / 191 | 215 | 1 | 60.930% (58.636–65.854) | 2.817% (1.149–9.722) | 96.269% (94.737–97.744) | 72.039% (69.737–76.316) | +| 20% | 38 / 191 | 248 | 1 | 61.290% (59.036–62.810) | 2.174% (1.020–6.667) | 96.053% (95.425–98.039) | 68.421% (66.118–70.066) | +| 10% | 19 / 191 | 275 | 0 | 61.011% (60.215–63.869) | 2.830% (0.000–10.784) | 97.076% (94.767–97.674) | 64.803% (63.487–67.434) | + +
+
+ +
+Church Slavonic — chu-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 152 / 152 | 0 | 0 | n/a | n/a | n/a | 81.379% | +| 90% | 137 / 152 | 132 | 72 | 44.203% (15.000–58.333) | 42.857% (11.628–56.667) | 70.000% (54.545–77.778) | 78.128% (76.404–78.818) | +| 80% | 122 / 152 | 266 | 144 | 45.161% (27.444–50.644) | 42.920% (23.457–47.222) | 69.565% (66.667–85.714) | 74.433% (72.463–75.714) | +| 70% | 106 / 152 | 444 | 162 | 41.892% (37.721–44.632) | 39.386% (33.547–42.202) | 71.795% (59.459–85.366) | 70.443% (68.424–71.084) | +| 60% | 91 / 152 | 659 | 162 | 41.166% (31.241–43.582) | 39.458% (27.480–40.933) | 74.074% (59.184–81.481) | 65.862% (62.266–67.389) | +| 50% | 76 / 152 | 840 | 198 | 39.930% (33.736–43.750) | 36.849% (29.797–40.587) | 79.104% (65.079–83.582) | 62.315% (57.882–63.202) | +| 40% | 61 / 152 | 1,075 | 162 | 38.605% (33.117–41.198) | 35.484% (29.618–38.323) | 75.904% (67.500–79.762) | 57.586% (52.857–58.818) | +| 30% | 46 / 152 | 1,247 | 162 | 34.612% (27.987–45.389) | 31.278% (23.565–42.472) | 79.592% (66.327–86.735) | 49.852% (46.010–56.700) | +| 20% | 30 / 152 | 1,541 | 108 | 33.899% (27.038–40.013) | 29.794% (22.222–38.257) | 81.897% (61.864–88.136) | 44.089% (39.557–49.557) | +| 10% | 15 / 152 | 1,756 | 72 | 26.640% (23.927–40.949) | 21.680% (19.071–38.760) | 83.333% (68.148–86.567) | 33.103% (30.099–45.517) | + +
+
+ +
+Classical Armenian — xcl-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 4,300 / 4,300 | 0 | 0 | n/a | n/a | n/a | 94.362% | +| 90% | 3,870 / 4,300 | 5,676 | 530 | 58.951% (56.973–64.152) | 56.614% (54.823–61.939) | 91.317% (90.000–95.143) | 90.831% (90.251–91.436) | +| 80% | 3,440 / 4,300 | 11,645 | 1,090 | 60.031% (57.825–62.927) | 57.967% (55.623–60.761) | 91.565% (91.144–93.986) | 87.255% (86.579–88.048) | +| 70% | 3,010 / 4,300 | 17,252 | 1,495 | 58.666% (57.184–59.272) | 56.257% (54.763–56.993) | 91.584% (90.761–93.926) | 83.373% (83.282–83.626) | +| 60% | 2,580 / 4,300 | 23,388 | 1,703 | 58.873% (57.300–59.035) | 56.638% (54.842–56.787) | 91.339% (89.882–92.536) | 79.647% (79.279–80.012) | +| 50% | 2,150 / 4,300 | 29,673 | 1,800 | 57.626% (56.787–58.976) | 55.152% (54.461–56.709) | 91.129% (89.892–91.269) | 75.670% (74.975–76.471) | +| 40% | 1,720 / 4,300 | 35,893 | 1,775 | 56.999% (56.075–57.373) | 54.704% (53.600–55.058) | 89.458% (88.725–90.542) | 71.695% (71.185–71.769) | +| 30% | 1,290 / 4,300 | 42,711 | 1,436 | 56.782% (53.073–58.169) | 54.478% (50.429–56.063) | 89.114% (87.743–90.095) | 67.501% (65.194–68.717) | +| 20% | 860 / 4,300 | 49,304 | 1,141 | 55.229% (54.188–56.999) | 52.893% (51.734–54.752) | 87.912% (86.991–88.376) | 62.877% (61.992–64.335) | +| 10% | 430 / 4,300 | 55,842 | 592 | 54.388% (52.421–55.202) | 52.101% (50.001–52.854) | 85.680% (85.348–88.186) | 58.187% (56.640–59.114) | + +
+
+ +
+Classical Syriac — syc-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 3,257 / 3,257 | 0 | 0 | n/a | n/a | n/a | 99.372% | +| 90% | 2,931 / 3,257 | 2,706 | 32 | 18.071% (16.174–19.930) | 6.970% (6.250–8.713) | 98.452% (97.840–98.754) | 91.402% (90.182–92.319) | +| 80% | 2,606 / 3,257 | 5,632 | 58 | 16.903% (16.178–18.162) | 6.553% (5.979–7.454) | 97.991% (97.531–98.302) | 82.482% (81.645–83.341) | +| 70% | 2,280 / 3,257 | 8,401 | 76 | 17.274% (17.188–17.984) | 7.136% (6.636–7.313) | 97.844% (97.216–98.251) | 74.328% (73.353–75.411) | +| 60% | 1,954 / 3,257 | 11,223 | 87 | 17.250% (16.964–17.541) | 6.700% (6.341–7.340) | 97.769% (96.993–98.228) | 65.950% (64.715–67.191) | +| 50% | 1,629 / 3,257 | 13,893 | 84 | 17.282% (16.724–17.405) | 6.568% (6.212–6.930) | 97.901% (97.102–98.149) | 57.936% (56.893–59.604) | +| 40% | 1,303 / 3,257 | 16,637 | 92 | 17.124% (16.990–17.465) | 6.498% (6.049–6.680) | 98.048% (97.228–98.821) | 49.782% (49.042–50.854) | +| 30% | 977 / 3,257 | 19,247 | 79 | 17.482% (16.726–17.630) | 6.526% (6.381–7.017) | 97.889% (97.451–98.067) | 42.078% (39.147–43.176) | +| 20% | 651 / 3,257 | 22,088 | 66 | 16.818% (16.312–17.279) | 6.306% (5.404–6.467) | 97.964% (96.269–98.692) | 33.560% (32.191–34.610) | +| 10% | 326 / 3,257 | 24,625 | 37 | 16.381% (16.169–17.052) | 5.476% (5.101–6.191) | 97.439% (96.689–99.215) | 25.474% (23.546–26.257) | + +
+
+ +
+Congo Swahili — swc-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 183 / 183 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 165 / 183 | 1,073 | 0 | 1.654% (1.573–1.798) | 0.000% | 100.000% (88.889–100.000) | 90.513% (89.992–91.161) | +| 80% | 146 / 183 | 2,337 | 0 | 1.540% (1.443–1.713) | 0.000% | 97.297% (91.892–100.000) | 79.309% (77.889–82.457) | +| 70% | 128 / 183 | 3,498 | 0 | 1.515% (1.483–1.570) | 0.000% | 96.364% (89.091–100.000) | 69.023% (68.303–70.740) | +| 60% | 110 / 183 | 4,614 | 0 | 1.539% (1.472–1.659) | 0.000% | 97.260% (90.411–100.000) | 59.149% (58.862–61.631) | +| 50% | 92 / 183 | 5,676 | 0 | 1.515% (1.371–1.565) | 0.000% | 92.308% (91.209–100.000) | 49.735% (45.661–52.477) | +| 40% | 73 / 183 | 6,869 | 0 | 1.511% (1.482–1.548) | 0.000% | 95.455% (90.909–98.182) | 39.178% (36.696–41.660) | +| 30% | 55 / 183 | 7,941 | 0 | 1.549% (1.516–1.575) | 0.000% | 96.094% (92.188–100.000) | 29.701% (28.082–33.378) | +| 20% | 37 / 183 | 8,959 | 0 | 1.530% (1.457–1.608) | 0.000% | 93.836% (91.781–100.000) | 20.673% (16.689–22.687) | +| 10% | 18 / 183 | 9,886 | 0 | 1.502% (1.406–1.671) | 0.000% | 89.091% (88.485–100.000) | 12.427% (7.302–13.929) | + +
+
+ +
+Copala Triqui — cpa-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 555 / 555 | 0 | 0 | n/a | n/a | n/a | 88.706% | +| 90% | 500 / 555 | 259 | 58 | 24.906% (21.933–26.606) | 5.430% (3.158–9.390) | 98.000% (97.917–100.000) | 82.621% (81.974–83.657) | +| 80% | 444 / 555 | 507 | 99 | 25.290% (25.000–26.411) | 7.226% (6.952–9.250) | 97.959% (97.030–100.000) | 76.893% (76.149–77.896) | +| 70% | 389 / 555 | 777 | 140 | 25.225% (24.399–25.732) | 7.414% (6.864–8.504) | 98.693% (96.026–99.333) | 71.036% (70.129–71.197) | +| 60% | 333 / 555 | 1,047 | 176 | 25.766% (24.045–26.249) | 8.747% (6.120–9.187) | 98.500% (97.500–99.034) | 64.887% (63.657–65.469) | +| 50% | 278 / 555 | 1,343 | 190 | 24.799% (23.973–26.585) | 8.594% (6.117–9.212) | 98.077% (97.200–99.219) | 58.220% (57.379–59.482) | +| 40% | 222 / 555 | 1,656 | 190 | 24.494% (23.913–26.145) | 7.988% (6.543–9.201) | 98.089% (96.743–99.035) | 51.618% (51.068–52.621) | +| 30% | 167 / 555 | 1,993 | 155 | 24.277% (23.691–24.887) | 7.396% (6.785–8.292) | 98.374% (97.581–98.919) | 44.757% (44.207–45.761) | +| 20% | 111 / 555 | 2,315 | 135 | 24.297% (22.637–24.622) | 7.732% (5.851–8.413) | 97.163% (96.706–98.595) | 37.961% (37.314–38.641) | +| 10% | 56 / 555 | 2,709 | 74 | 23.946% (22.148–24.859) | 7.973% (5.415–8.541) | 97.732% (95.910–99.588) | 31.100% (29.256–32.395) | + +
+
+ +
+Cornish — cor-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 8 / 8 | 0 | 0 | n/a | n/a | n/a | 93.103% | +| 90% | 7 / 8 | 11 | 0 | 9.091% (9.091–100.000) | 0.000% | 100.000% | 87.356% (87.356–93.103) | +| 80% | 6 / 8 | 21 | 0 | 12.766% (4.167–18.182) | 0.000% (0.000–8.889) | 100.000% | 82.184% (66.667–87.931) | +| 70% | 6 / 8 | 21 | 0 | 12.766% (4.167–18.182) | 0.000% (0.000–8.889) | 100.000% | 82.184% (66.667–87.931) | +| 60% | 5 / 8 | 48 | 12 | 6.250% (2.703–14.583) | 0.000% (0.000–8.889) | 100.000% (33.333–100.000) | 69.540% (66.092–82.184) | +| 50% | 4 / 8 | 73 | 12 | 6.154% (3.529–8.219) | 0.000% (0.000–5.797) | 75.000% (50.000–100.000) | 58.046% (45.977–61.494) | +| 40% | 3 / 8 | 110 | 12 | 5.405% (3.922–8.264) | 0.943% (0.000–5.172) | 80.000% (80.000–100.000) | 39.655% (29.310–41.954) | +| 30% | 2 / 8 | 132 | 12 | 7.483% (5.357–8.850) | 4.255% (0.943–4.673) | 83.333% | 22.989% (21.839–33.908) | +| 20% | 2 / 8 | 132 | 12 | 7.483% (5.357–8.850) | 4.255% (0.943–4.673) | 83.333% | 22.989% (21.839–33.908) | +| 10% | 1 / 8 | 157 | 0 | 7.006% (2.685–8.696) | 3.333% (0.000–3.817) | 85.714% (57.143–100.000) | 16.092% (9.770–27.586) | + +
+
+ +
+Cree — cre-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 32 / 32 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 29 / 32 | 144 | 0 | 93.056% (77.381–97.059) | 94.118% (76.543–97.512) | 66.667% (33.333–100.000) | 99.071% (97.491–99.442) | +| 80% | 26 / 32 | 283 | 0 | 83.772% (52.478–94.048) | 83.784% (52.522–95.062) | 83.333% (50.000–100.000) | 96.561% (84.851–99.071) | +| 70% | 22 / 32 | 375 | 0 | 89.067% (53.061–91.538) | 89.041% (52.708–92.400) | 80.000% (70.000–100.000) | 96.004% (78.625–97.955) | +| 60% | 19 / 32 | 459 | 0 | 79.303% (55.253–93.510) | 78.700% (54.691–94.172) | 84.615% (76.923–100.000) | 91.171% (78.625–97.955) | +| 50% | 16 / 32 | 603 | 0 | 62.310% (38.473–81.590) | 61.838% (37.577–81.602) | 81.250% (75.000–100.000) | 76.952% (61.803–91.822) | +| 40% | 13 / 32 | 682 | 0 | 66.499% (42.145–81.851) | 66.067% (41.379–81.768) | 84.211% (73.684–94.737) | 75.186% (56.877–90.520) | +| 30% | 10 / 32 | 766 | 0 | 65.014% (42.978–76.780) | 64.035% (42.040–76.923) | 81.818% (72.727–95.455) | 74.628% (56.227–86.059) | +| 20% | 6 / 32 | 913 | 0 | 41.521% (32.749–67.419) | 40.127% (30.891–67.098) | 92.308% (76.923–100.000) | 47.770% (42.937–75.836) | +| 10% | 3 / 32 | 997 | 0 | 35.752% (8.460–41.230) | 34.581% (5.865–39.668) | 93.103% (72.414–100.000) | 43.401% (10.502–45.818) | + +
+
+ +
+Crimean Tatar — crh-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,210 / 1,210 | 0 | 0 | n/a | n/a | n/a | 99.944% | +| 90% | 1,089 / 1,210 | 717 | 1 | 95.556% (94.693–96.111) | 95.477% (93.792–96.500) | 95.041% (94.167–99.167) | 99.500% (99.417–99.555) | +| 80% | 968 / 1,210 | 1,438 | 1 | 95.271% (94.367–96.181) | 94.987% (93.787–96.414) | 95.021% (92.562–98.340) | 99.000% (98.819–99.194) | +| 70% | 847 / 1,210 | 2,159 | 2 | 95.370% (93.948–95.926) | 95.442% (93.169–96.105) | 95.028% (93.113–97.790) | 98.555% (98.139–98.736) | +| 60% | 726 / 1,210 | 2,876 | 2 | 95.028% (94.286–95.859) | 95.029% (93.758–96.151) | 94.617% (93.375–96.894) | 97.972% (97.666–98.305) | +| 50% | 605 / 1,210 | 3,593 | 2 | 95.243% (94.766–95.606) | 95.154% (94.779–95.822) | 94.702% (94.205–96.026) | 97.569% (97.347–97.764) | +| 40% | 484 / 1,210 | 4,314 | 2 | 95.228% (94.576–95.435) | 95.210% (94.290–95.566) | 95.317% (94.069–95.994) | 97.111% (96.708–97.208) | +| 30% | 363 / 1,210 | 5,036 | 2 | 95.092% (93.467–95.955) | 95.206% (93.009–96.282) | 95.035% (93.853–95.740) | 96.527% (95.388–97.152) | +| 20% | 242 / 1,210 | 5,758 | 1 | 94.340% (92.480–95.084) | 94.864% (92.321–95.177) | 94.623% (91.736–95.346) | 95.458% (93.944–96.055) | +| 10% | 121 / 1,210 | 6,479 | 1 | 94.366% (93.781–94.628) | 94.212% (93.990–94.730) | 94.123% (90.993–95.588) | 94.930% (94.388–95.166) | + +
+
+
Czech — cs-cz-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 5,113 / 5,113 | 0 | 0 | n/a | n/a | n/a | 99.465% | -| 90% | 4,602 / 5,113 | 5,497 | 52 | 73.270% (71.970–75.506) | 69.900% (67.596–72.033) | 91.296% (89.606–92.449) | 96.872% (96.693–97.110) | -| 80% | 4,090 / 5,113 | 11,295 | 108 | 74.375% (72.961–75.341) | 70.977% (68.757–71.820) | 91.822% (89.763–92.700) | 94.512% (94.070–94.620) | -| 70% | 3,579 / 5,113 | 16,910 | 167 | 74.517% (73.186–75.890) | 70.679% (69.464–72.757) | 91.409% (90.430–92.421) | 91.850% (91.537–92.385) | -| 60% | 3,068 / 5,113 | 22,567 | 171 | 74.064% (73.458–75.665) | 70.641% (69.532–72.243) | 91.201% (89.195–91.780) | 89.311% (89.002–89.831) | -| 50% | 2,557 / 5,113 | 28,195 | 172 | 74.013% (73.648–74.299) | 70.612% (69.908–70.743) | 91.128% (89.182–91.481) | 86.632% (86.397–86.793) | -| 40% | 2,045 / 5,113 | 33,944 | 151 | 74.037% (73.092–74.837) | 70.380% (69.385–71.348) | 90.721% (89.903–91.126) | 84.092% (83.521–84.645) | -| 30% | 1,534 / 5,113 | 39,562 | 122 | 73.325% (71.387–74.104) | 69.913% (67.160–70.569) | 90.571% (89.035–90.982) | 81.062% (79.736–81.666) | -| 20% | 1,023 / 5,113 | 45,111 | 109 | 72.003% (69.047–72.412) | 68.155% (64.204–68.726) | 89.767% (89.232–91.428) | 77.461% (75.136–77.787) | -| 10% | 511 / 5,113 | 50,905 | 74 | 69.921% (66.235–70.712) | 65.439% (60.991–66.671) | 89.636% (88.686–90.665) | 72.859% (69.489–73.555) | +| 100% | 5,113 / 5,113 | 0 | 0 | n/a | n/a | n/a | 99.465% | +| 90% | 4,602 / 5,113 | 5,497 | 52 | 73.270% (71.970–75.506) | 69.900% (67.596–72.033) | 91.296% (89.606–92.449) | 96.872% (96.693–97.110) | +| 80% | 4,090 / 5,113 | 11,295 | 108 | 74.375% (72.961–75.341) | 70.977% (68.757–71.820) | 91.822% (89.763–92.700) | 94.512% (94.070–94.620) | +| 70% | 3,579 / 5,113 | 16,910 | 167 | 74.517% (73.186–75.890) | 70.679% (69.464–72.757) | 91.409% (90.430–92.421) | 91.850% (91.537–92.385) | +| 60% | 3,068 / 5,113 | 22,567 | 171 | 74.064% (73.458–75.665) | 70.641% (69.532–72.243) | 91.201% (89.195–91.780) | 89.311% (89.002–89.831) | +| 50% | 2,557 / 5,113 | 28,195 | 172 | 74.013% (73.648–74.299) | 70.612% (69.908–70.743) | 91.128% (89.182–91.481) | 86.632% (86.397–86.793) | +| 40% | 2,045 / 5,113 | 33,944 | 151 | 74.037% (73.092–74.837) | 70.380% (69.385–71.348) | 90.721% (89.903–91.126) | 84.092% (83.521–84.645) | +| 30% | 1,534 / 5,113 | 39,562 | 122 | 73.325% (71.387–74.104) | 69.913% (67.160–70.569) | 90.571% (89.035–90.982) | 81.062% (79.736–81.666) | +| 20% | 1,023 / 5,113 | 45,111 | 109 | 72.003% (69.047–72.412) | 68.155% (64.204–68.726) | 89.767% (89.232–91.428) | 77.461% (75.136–77.787) | +| 10% | 511 / 5,113 | 50,905 | 74 | 69.921% (66.235–70.712) | 65.439% (60.991–66.671) | 89.636% (88.686–90.665) | 72.859% (69.489–73.555) | + +
+
+ +
+Dakota — dak-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 473 / 473 | 0 | 0 | n/a | n/a | n/a | 99.226% | +| 90% | 426 / 473 | 326 | 10 | 24.375% (21.951–28.840) | 12.545% (9.609–17.216) | 95.745% (89.362–97.872) | 91.900% (91.602–92.466) | +| 80% | 378 / 473 | 662 | 10 | 23.845% (23.688–26.300) | 12.153% (11.888–14.286) | 94.737% (94.681–98.947) | 84.336% (84.008–84.872) | +| 70% | 331 / 473 | 992 | 13 | 24.093% (23.124–24.476) | 12.235% (11.834–12.922) | 95.070% (90.780–96.454) | 76.802% (76.712–76.891) | +| 60% | 284 / 473 | 1,325 | 13 | 23.925% (21.278–24.851) | 12.148% (9.019–13.247) | 95.745% (92.553–96.296) | 69.267% (68.195–69.446) | +| 50% | 237 / 473 | 1,657 | 15 | 22.269% (21.553–23.778) | 10.134% (9.397–12.197) | 94.894% (91.915–95.339) | 60.929% (60.572–61.525) | +| 40% | 189 / 473 | 2,000 | 13 | 21.678% (20.435–24.248) | 9.366% (8.387–12.850) | 93.310% (92.580–96.466) | 52.680% (51.846–54.527) | +| 30% | 142 / 473 | 2,333 | 14 | 21.046% (18.955–22.889) | 8.837% (6.920–10.984) | 93.636% (92.727–95.152) | 44.580% (42.942–46.010) | +| 20% | 95 / 473 | 2,676 | 10 | 18.498% (17.779–19.933) | 5.962% (5.362–7.472) | 94.974% (92.042–96.021) | 34.693% (34.127–35.825) | +| 10% | 47 / 473 | 3,021 | 4 | 19.185% (15.619–20.563) | 6.481% (2.696–7.977) | 94.836% (92.958–97.412) | 27.129% (24.032–28.350) | + +
+
+ +
+Danish — da-dk-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 4,179 / 4,179 | 0 | 0 | n/a | n/a | n/a | 99.371% | +| 90% | 3,761 / 4,179 | 3,232 | 38 | 74.192% (73.356–75.186) | 67.504% (66.790–69.750) | 92.718% (90.865–93.961) | 96.813% (96.698–96.897) | +| 80% | 3,343 / 4,179 | 6,419 | 59 | 74.015% (73.293–74.244) | 67.477% (67.434–67.862) | 92.377% (90.132–93.494) | 94.212% (94.063–94.358) | +| 70% | 2,925 / 4,179 | 9,622 | 72 | 72.901% (72.673–74.059) | 66.648% (65.699–67.713) | 91.066% (90.273–92.613) | 91.447% (91.276–91.726) | +| 60% | 2,507 / 4,179 | 12,858 | 89 | 72.406% (72.124–72.935) | 65.740% (65.579–67.061) | 90.565% (89.988–91.933) | 88.498% (88.442–88.641) | +| 50% | 2,090 / 4,179 | 16,078 | 86 | 71.433% (70.792–72.016) | 64.706% (63.871–65.934) | 90.561% (89.489–90.758) | 85.333% (85.042–85.612) | +| 40% | 1,672 / 4,179 | 19,316 | 87 | 70.449% (69.207–70.919) | 63.495% (61.895–64.640) | 90.106% (88.956–90.560) | 81.882% (81.244–82.196) | +| 30% | 1,254 / 4,179 | 22,543 | 70 | 68.316% (67.096–69.463) | 60.962% (59.342–62.755) | 89.335% (88.687–90.062) | 77.558% (76.727–78.444) | +| 20% | 836 / 4,179 | 25,755 | 61 | 65.725% (64.691–66.441) | 57.555% (56.445–58.672) | 89.232% (88.279–90.082) | 72.436% (71.615–73.034) | +| 10% | 418 / 4,179 | 29,022 | 28 | 61.104% (59.899–62.277) | 51.055% (49.995–53.218) | 88.238% (87.507–89.957) | 64.847% (63.836–65.997) | + +
+
+ +
+Dutch — nl-nl-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 4,992 / 4,992 | 0 | 0 | n/a | n/a | n/a | 99.120% | +| 90% | 4,493 / 4,992 | 3,085 | 57 | 74.468% (73.250–76.272) | 65.891% (63.190–67.730) | 94.512% (92.371–95.697) | 96.574% (96.437–96.777) | +| 80% | 3,994 / 4,992 | 6,160 | 110 | 74.369% (74.091–75.340) | 64.865% (64.550–66.383) | 94.408% (92.901–95.806) | 94.083% (94.025–94.245) | +| 70% | 3,494 / 4,992 | 9,298 | 148 | 74.137% (73.135–74.845) | 64.553% (63.215–66.338) | 94.412% (93.155–95.287) | 91.461% (91.190–91.734) | +| 60% | 2,995 / 4,992 | 12,412 | 171 | 73.828% (73.295–73.872) | 64.368% (63.661–64.668) | 94.242% (93.471–95.070) | 88.817% (88.524–88.845) | +| 50% | 2,496 / 4,992 | 15,581 | 174 | 73.408% (73.027–73.660) | 63.952% (63.070–64.202) | 94.186% (93.539–94.678) | 86.068% (85.791–86.153) | +| 40% | 1,997 / 4,992 | 18,714 | 171 | 72.501% (71.803–73.110) | 62.655% (61.524–63.525) | 93.853% (93.540–94.029) | 82.969% (82.495–83.280) | +| 30% | 1,498 / 4,992 | 21,856 | 141 | 71.097% (70.602–72.004) | 61.026% (59.937–61.731) | 93.565% (92.739–94.166) | 79.432% (79.003–79.969) | +| 20% | 998 / 4,992 | 25,065 | 109 | 69.543% (68.997–70.471) | 58.908% (57.908–60.208) | 92.781% (92.430–93.088) | 75.447% (74.855–76.168) | +| 10% | 499 / 4,992 | 28,267 | 62 | 67.039% (65.411–67.577) | 55.524% (53.812–55.891) | 92.616% (90.386–92.866) | 70.231% (68.760–70.692) | + +
+
+ +
+Eastern Chatino — cly-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 185 / 185 | 0 | 0 | n/a | n/a | n/a | 90.404% | +| 90% | 167 / 185 | 173 | 39 | 9.827% (7.692–11.047) | 1.911% (0.000–3.774) | 100.000% (87.500–100.000) | 83.474% (82.408–83.829) | +| 80% | 148 / 185 | 381 | 70 | 9.351% (8.136–10.133) | 0.890% (0.000–2.041) | 96.875% (93.939–100.000) | 75.300% (74.767–75.700) | +| 70% | 130 / 185 | 583 | 88 | 9.326% (8.233–9.982) | 0.752% (0.000–1.949) | 96.000% (95.833–100.000) | 67.437% (66.459–68.858) | +| 60% | 111 / 185 | 797 | 105 | 9.206% (8.629–9.547) | 1.068% (0.417–1.872) | 95.588% (90.000–97.183) | 59.307% (58.463–59.973) | +| 50% | 93 / 185 | 1,001 | 113 | 9.046% (8.732–9.127) | 0.849% (0.455–1.313) | 93.407% (90.805–96.386) | 52.155% (50.600–52.999) | +| 40% | 74 / 185 | 1,226 | 120 | 8.795% (8.548–9.024) | 0.797% (0.453–1.070) | 94.340% (90.566–96.078) | 43.847% (43.003–44.114) | +| 30% | 56 / 185 | 1,467 | 100 | 8.521% (8.328–8.725) | 0.689% (0.446–0.880) | 92.683% (91.129–96.748) | 35.318% (34.340–36.117) | +| 20% | 37 / 185 | 1,710 | 71 | 8.568% (8.406–8.841) | 0.805% (0.640–0.831) | 93.836% (92.308–96.454) | 26.788% (24.967–27.232) | +| 10% | 19 / 185 | 1,969 | 52 | 8.406% (8.278–8.829) | 0.769% (0.447–0.997) | 95.679% (88.415–98.160) | 17.725% (16.659–18.392) | + +
+
+ +
+Egyptian Arabic — arz-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 6,004 / 6,004 | 0 | 0 | n/a | n/a | n/a | 97.781% | +| 90% | 5,404 / 6,004 | 1,733 | 61 | 39.956% (39.161–40.739) | 10.760% (8.455–12.903) | 96.121% (95.110–96.965) | 92.177% (91.916–92.254) | +| 80% | 4,803 / 6,004 | 3,518 | 124 | 39.061% (38.780–40.392) | 10.239% (9.622–11.717) | 96.293% (95.795–96.799) | 86.181% (85.995–86.475) | +| 70% | 4,203 / 6,004 | 5,345 | 173 | 39.884% (38.874–40.123) | 11.314% (9.955–11.690) | 96.745% (95.910–96.973) | 80.462% (80.255–80.549) | +| 60% | 3,602 / 6,004 | 7,142 | 202 | 39.386% (38.818–39.796) | 10.880% (9.841–11.227) | 96.556% (96.313–97.235) | 74.520% (74.166–74.597) | +| 50% | 3,002 / 6,004 | 8,978 | 208 | 39.163% (38.873–40.027) | 10.907% (9.934–11.215) | 96.984% (96.480–96.986) | 68.666% (68.447–69.260) | +| 40% | 2,402 / 6,004 | 10,747 | 182 | 39.008% (38.441–39.397) | 10.196% (9.818–10.790) | 96.596% (96.485–97.321) | 62.762% (62.124–63.171) | +| 30% | 1,801 / 6,004 | 12,729 | 165 | 38.828% (38.659–39.011) | 10.110% (9.447–10.334) | 97.299% (96.584–97.613) | 56.509% (56.247–56.852) | +| 20% | 1,201 / 6,004 | 14,564 | 119 | 38.382% (37.388–39.069) | 9.268% (8.010–10.134) | 97.601% (96.973–98.019) | 50.365% (49.357–50.758) | +| 10% | 600 / 6,004 | 16,458 | 78 | 37.445% (36.685–38.165) | 7.926% (7.223–9.429) | 97.166% (96.036–97.683) | 43.813% (42.423–44.145) | + +
+
+ +
+English — us-uk-default 1.0.1 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 396,939 / 396,939 | 0 | 0 | n/a | n/a | n/a | 97.668% | +| 90% | 357,245 / 396,939 | 96,702 | 3,746 | 94.680% (94.609–94.847) | 78.854% (78.538–79.603) | 98.744% (98.642–98.790) | 97.241% (97.237–97.256) | +| 80% | 317,551 / 396,939 | 193,781 | 6,659 | 94.608% (94.514–94.637) | 78.795% (78.580–79.033) | 98.592% (98.577–98.658) | 96.818% (96.816–96.831) | +| 70% | 277,857 / 396,939 | 291,831 | 8,726 | 94.425% (94.290–94.438) | 78.505% (77.971–78.614) | 98.454% (98.436–98.500) | 96.386% (96.367–96.394) | +| 60% | 238,163 / 396,939 | 390,750 | 10,090 | 94.178% (94.121–94.242) | 77.976% (77.824–78.311) | 98.314% (98.268–98.319) | 95.919% (95.916–95.953) | +| 50% | 198,470 / 396,939 | 490,475 | 10,590 | 93.934% (93.888–94.011) | 77.643% (77.410–77.726) | 98.133% (98.089–98.192) | 95.434% (95.421–95.484) | +| 40% | 158,776 / 396,939 | 591,318 | 10,198 | 93.699% (93.682–93.712) | 77.308% (77.149–77.479) | 97.940% (97.912–97.948) | 94.946% (94.916–94.960) | +| 30% | 119,082 / 396,939 | 692,940 | 8,930 | 93.440% (93.409–93.459) | 76.944% (76.715–76.961) | 97.741% (97.718–97.755) | 94.413% (94.372–94.433) | +| 20% | 79,388 / 396,939 | 795,287 | 6,816 | 93.155% (93.122–93.220) | 76.608% (76.306–76.669) | 97.509% (97.464–97.541) | 93.824% (93.807–93.898) | +| 10% | 39,694 / 396,939 | 898,572 | 3,741 | 92.805% (92.757–92.843) | 76.010% (75.505–76.378) | 97.184% (97.156–97.293) | 93.157% (93.123–93.211) | + +
+
+ +
+Estonian — et-ee-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 886 / 886 | 0 | 0 | n/a | n/a | n/a | 99.683% | +| 90% | 797 / 886 | 2,498 | 14 | 72.543% (70.256–75.531) | 72.027% (69.473–74.979) | 91.954% (87.059–93.182) | 96.955% (96.697–97.232) | +| 80% | 709 / 886 | 4,941 | 34 | 73.075% (70.676–74.043) | 72.300% (69.872–73.433) | 90.805% (90.173–94.286) | 94.331% (93.865–94.576) | +| 70% | 620 / 886 | 7,416 | 41 | 73.590% (70.159–74.949) | 72.846% (69.407–74.337) | 91.154% (88.931–93.916) | 91.812% (90.796–92.246) | +| 60% | 532 / 886 | 9,874 | 39 | 71.967% (70.482–73.764) | 71.262% (69.720–73.092) | 91.143% (89.971–92.045) | 88.594% (87.987–89.289) | +| 50% | 443 / 886 | 12,395 | 40 | 69.786% (68.011–71.332) | 68.977% (67.150–70.599) | 91.344% (89.954–91.972) | 84.705% (83.825–85.452) | +| 40% | 354 / 886 | 14,894 | 43 | 68.900% (67.070–70.827) | 68.110% (66.209–70.099) | 90.512% (90.341–91.255) | 81.173% (80.064–82.290) | +| 30% | 266 / 886 | 17,352 | 34 | 67.866% (64.674–70.321) | 67.011% (63.758–69.641) | 89.739% (88.799–92.195) | 77.384% (75.159–79.168) | +| 20% | 177 / 886 | 19,880 | 24 | 65.953% (60.416–68.265) | 64.959% (59.249–67.400) | 91.773% (88.385–93.020) | 72.680% (68.252–74.556) | +| 10% | 89 / 886 | 22,377 | 18 | 59.670% (57.126–62.316) | 58.663% (55.871–61.203) | 91.195% (85.912–93.082) | 63.712% (61.398–66.051) | + +
+
+ +
+Evenki — evn-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 4,493 / 4,493 | 0 | 0 | n/a | n/a | n/a | 99.224% | +| 90% | 4,044 / 4,493 | 1,332 | 16 | 57.112% (53.841–61.939) | 40.202% (36.185–41.864) | 93.258% (91.403–94.558) | 95.042% (94.295–96.126) | +| 80% | 3,594 / 4,493 | 2,746 | 33 | 56.535% (55.070–59.610) | 39.027% (37.948–41.680) | 93.813% (91.432–95.125) | 90.802% (89.821–91.959) | +| 70% | 3,145 / 4,493 | 4,012 | 41 | 56.505% (55.558–57.731) | 38.088% (37.468–40.151) | 92.927% (91.811–93.967) | 86.503% (85.954–87.279) | +| 60% | 2,696 / 4,493 | 5,325 | 46 | 55.704% (54.591–56.376) | 37.165% (36.785–38.585) | 92.772% (91.531–93.154) | 82.065% (80.703–82.827) | +| 50% | 2,247 / 4,493 | 6,868 | 47 | 55.387% (53.368–56.451) | 36.926% (35.702–38.022) | 91.903% (91.592–92.284) | 76.975% (75.335–78.953) | +| 40% | 1,797 / 4,493 | 8,423 | 48 | 53.520% (51.145–54.525) | 35.325% (32.679–36.057) | 92.201% (90.906–92.468) | 70.882% (69.308–72.838) | +| 30% | 1,348 / 4,493 | 9,591 | 48 | 52.973% (51.922–53.645) | 33.658% (33.143–35.688) | 91.877% (90.215–92.231) | 66.203% (65.170–67.807) | +| 20% | 899 / 4,493 | 11,044 | 38 | 51.920% (49.783–52.722) | 33.092% (29.788–33.853) | 91.076% (90.070–91.678) | 60.879% (58.938–62.190) | +| 10% | 449 / 4,493 | 12,308 | 20 | 49.277% (48.384–50.374) | 29.966% (27.644–30.596) | 90.458% (88.512–91.485) | 54.390% (52.801–55.196) | + +
+
+ +
+Faroese — fo-fo-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 3,077 / 3,077 | 0 | 0 | n/a | n/a | n/a | 94.099% | +| 90% | 2,769 / 3,077 | 3,013 | 355 | 58.598% (55.022–60.940) | 55.616% (51.802–57.952) | 88.645% (86.447–91.513) | 90.373% (90.223–90.553) | +| 80% | 2,462 / 3,077 | 6,049 | 627 | 57.193% (56.722–58.952) | 53.984% (53.690–55.863) | 88.889% (87.339–91.560) | 86.473% (86.260–86.740) | +| 70% | 2,154 / 3,077 | 9,142 | 846 | 56.317% (55.675–56.778) | 53.132% (52.478–53.720) | 88.214% (87.515–90.315) | 82.564% (82.105–82.813) | +| 60% | 1,846 / 3,077 | 12,348 | 986 | 54.858% (54.513–55.960) | 51.566% (51.238–52.872) | 87.390% (86.894–88.235) | 78.370% (77.647–78.454) | +| 50% | 1,539 / 3,077 | 15,658 | 1,058 | 54.005% (53.129–54.560) | 50.720% (49.856–51.365) | 86.494% (85.976–86.750) | 73.594% (73.033–74.182) | +| 40% | 1,231 / 3,077 | 18,993 | 1,004 | 51.666% (51.130–53.360) | 48.468% (47.771–50.141) | 84.696% (83.623–85.395) | 68.170% (67.801–69.556) | +| 30% | 923 / 3,077 | 22,370 | 975 | 49.655% (48.319–51.081) | 46.296% (44.840–47.635) | 84.510% (82.821–85.068) | 62.578% (61.774–64.000) | +| 20% | 615 / 3,077 | 25,938 | 797 | 46.439% (45.686–47.277) | 42.922% (42.262–43.516) | 81.707% (79.688–84.380) | 55.732% (54.736–56.632) | +| 10% | 308 / 3,077 | 29,641 | 469 | 41.712% (39.458–42.240) | 37.711% (35.058–38.392) | 81.262% (79.618–83.070) | 46.740% (44.523–47.583) | + +
+
+ +
+Finnish — fi-fi-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 57,027 / 57,027 | 0 | 0 | n/a | n/a | n/a | 98.661% | +| 90% | 51,324 / 57,027 | 181,248 | 5,149 | 86.644% (86.280–86.823) | 86.177% (85.790–86.342) | 94.835% (94.730–95.762) | 97.363% (97.337–97.369) | +| 80% | 45,622 / 57,027 | 363,208 | 9,479 | 86.152% (86.055–86.432) | 85.686% (85.568–85.979) | 94.265% (94.183–94.726) | 95.981% (95.972–96.026) | +| 70% | 39,919 / 57,027 | 546,876 | 12,346 | 85.689% (85.586–85.804) | 85.219% (85.114–85.335) | 93.710% (93.575–93.808) | 94.531% (94.518–94.560) | +| 60% | 34,216 / 57,027 | 730,977 | 14,257 | 85.172% (85.110–85.293) | 84.708% (84.639–84.829) | 93.033% (92.895–93.427) | 92.999% (92.978–93.035) | +| 50% | 28,514 / 57,027 | 917,680 | 14,946 | 84.681% (84.520–84.723) | 84.232% (84.039–84.280) | 92.203% (92.123–92.566) | 91.382% (91.295–91.404) | +| 40% | 22,811 / 57,027 | 1,103,628 | 14,607 | 83.991% (83.783–84.165) | 83.556% (83.313–83.715) | 91.299% (91.199–91.628) | 89.582% (89.463–89.675) | +| 30% | 17,108 / 57,027 | 1,292,480 | 12,981 | 83.290% (82.866–83.366) | 82.848% (82.399–82.936) | 90.499% (90.312–90.537) | 87.658% (87.378–87.720) | +| 20% | 11,405 / 57,027 | 1,481,274 | 10,279 | 81.958% (81.525–81.970) | 81.481% (81.032–81.508) | 89.459% (89.355–89.678) | 85.129% (84.780–85.155) | +| 10% | 5,703 / 57,027 | 1,672,373 | 6,067 | 79.210% (78.547–79.594) | 78.644% (77.966–79.067) | 88.247% (87.860–88.496) | 81.064% (80.468–81.422) | + +
+
+ +
+French — fr-fr-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 59,240 / 59,240 | 0 | 0 | n/a | n/a | n/a | 94.831% | +| 90% | 53,316 / 59,240 | 41,750 | 5,161 | 83.469% (82.278–83.711) | 80.031% (78.595–80.412) | 96.664% (96.351–96.813) | 93.302% (93.201–93.351) | +| 80% | 47,392 / 59,240 | 85,315 | 9,281 | 82.721% (81.861–83.091) | 79.345% (78.141–79.693) | 96.074% (95.896–96.266) | 91.731% (91.568–91.794) | +| 70% | 41,468 / 59,240 | 130,120 | 12,345 | 82.101% (81.643–82.329) | 78.644% (78.015–78.839) | 95.431% (95.257–95.560) | 90.086% (89.942–90.135) | +| 60% | 35,544 / 59,240 | 175,326 | 14,017 | 81.224% (80.630–81.431) | 77.560% (76.779–77.867) | 94.813% (94.641–94.831) | 88.270% (88.228–88.450) | +| 50% | 29,620 / 59,240 | 222,533 | 14,808 | 80.344% (79.709–80.528) | 76.569% (75.778–76.888) | 94.006% (93.785–94.099) | 86.507% (86.336–86.609) | +| 40% | 23,696 / 59,240 | 270,536 | 14,411 | 79.255% (79.039–79.552) | 75.314% (75.088–75.702) | 93.386% (93.086–93.418) | 84.453% (84.319–84.752) | +| 30% | 17,772 / 59,240 | 318,984 | 12,728 | 78.161% (78.141–78.274) | 74.079% (73.976–74.276) | 92.537% (92.395–92.906) | 82.316% (82.244–82.378) | +| 20% | 11,848 / 59,240 | 369,337 | 9,783 | 76.769% (76.583–76.946) | 72.504% (72.339–72.772) | 91.463% (90.996–91.884) | 79.741% (79.646–79.873) | +| 10% | 5,924 / 59,240 | 420,578 | 5,742 | 74.276% (73.276–74.926) | 69.627% (68.400–70.528) | 90.013% (89.987–90.359) | 75.978% (75.101–76.554) | + +
+
+ +
+Friulian — fur-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 168 / 168 | 0 | 0 | n/a | n/a | n/a | 99.801% | +| 90% | 151 / 168 | 505 | 2 | 78.968% (61.057–90.891) | 78.234% (59.919–90.574) | 100.000% (88.235–100.000) | 97.688% (95.834–98.884) | +| 80% | 134 / 168 | 1,010 | 4 | 75.522% (71.767–84.378) | 74.691% (70.787–83.934) | 100.000% (94.118–100.000) | 94.897% (94.140–96.671) | +| 70% | 118 / 168 | 1,485 | 3 | 76.142% (73.670–78.851) | 75.452% (72.911–78.182) | 96.000% (94.000–100.000) | 92.725% (92.007–93.562) | +| 60% | 101 / 168 | 1,991 | 4 | 72.060% (70.200–73.839) | 71.258% (69.271–72.934) | 95.522% (93.939–100.000) | 88.718% (87.921–89.476) | +| 50% | 84 / 168 | 2,499 | 6 | 70.991% (70.148–74.359) | 70.140% (69.234–73.726) | 94.048% (92.771–100.000) | 85.390% (84.991–87.084) | +| 40% | 67 / 168 | 3,009 | 4 | 71.424% (69.042–75.573) | 70.651% (68.060–74.940) | 95.050% (94.000–97.980) | 82.759% (81.304–85.270) | +| 30% | 50 / 168 | 3,522 | 1 | 71.275% (67.235–75.092) | 70.437% (66.186–74.404) | 96.610% (94.915–98.305) | 79.749% (76.879–82.480) | +| 20% | 34 / 168 | 4,000 | 1 | 68.530% (67.927–71.704) | 67.493% (66.891–70.903) | 96.269% (94.776–98.507) | 74.846% (74.327–77.417) | +| 10% | 17 / 168 | 4,507 | 1 | 63.002% (60.674–69.770) | 61.880% (59.381–68.883) | 96.026% (95.364–98.013) | 66.693% (64.640–72.793) | + +
+
+ +
+Ga — gaa-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 94 / 94 | 0 | 0 | n/a | n/a | n/a | 99.787% | +| 90% | 85 / 94 | 44 | 1 | 60.000% (59.091–61.364) | 50.000% (50.000–51.429) | 100.000% | 95.957% (95.957–96.170) | +| 80% | 75 / 94 | 94 | 1 | 60.000% (58.511–60.638) | 50.000% (50.000–50.667) | 100.000% (94.444–100.000) | 91.702% (91.489–91.915) | +| 70% | 66 / 94 | 139 | 1 | 60.000% (58.993–60.432) | 50.000% (50.000–50.450) | 100.000% (96.296–100.000) | 87.872% (87.660–88.085) | +| 60% | 56 / 94 | 189 | 1 | 60.000% (59.259–60.317) | 50.000% (50.000–50.331) | 100.000% (97.297–100.000) | 83.830% (83.404–83.830) | +| 50% | 47 / 94 | 234 | 1 | 60.000% (59.402–60.256) | 50.000% (50.000–50.267) | 100.000% (97.826–100.000) | 80.000% (79.574–80.000) | +| 40% | 38 / 94 | 280 | 0 | 59.643% (57.143–60.215) | 50.000% (46.429–50.224) | 100.000% (98.182–100.000) | 75.745% (74.468–76.170) | +| 30% | 28 / 94 | 329 | 1 | 60.000% (56.231–60.182) | 50.000% (46.970–50.190) | 100.000% (92.308–100.000) | 71.915% (69.149–71.915) | +| 20% | 19 / 94 | 375 | 0 | 58.133% (57.067–59.358) | 50.000% (47.333–50.167) | 96.000% (90.667–100.000) | 66.596% (65.745–67.447) | +| 10% | 9 / 94 | 425 | 0 | 56.000% (53.176–58.491) | 50.000% (42.353–50.147) | 91.765% (80.000–100.000) | 60.213% (57.660–62.340) | + +
+
+ +
+Galolen — gal-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 486 / 486 | 0 | 0 | n/a | n/a | n/a | 97.103% | +| 90% | 437 / 486 | 2,551 | 94 | 81.665% (80.263–89.176) | 81.307% (79.911–88.961) | 100.000% (97.959–100.000) | 95.331% (95.198–96.076) | +| 80% | 389 / 486 | 5,001 | 227 | 81.528% (76.585–83.327) | 81.162% (76.142–83.020) | 100.000% (98.969–100.000) | 93.587% (92.888–93.957) | +| 70% | 340 / 486 | 7,535 | 334 | 81.102% (77.226–84.013) | 80.731% (76.790–83.695) | 100.000% (99.315–100.000) | 91.739% (90.838–92.674) | +| 60% | 292 / 486 | 10,155 | 293 | 79.048% (78.338–81.300) | 78.656% (77.914–80.946) | 99.485% (98.969–100.000) | 89.490% (89.044–90.265) | +| 50% | 243 / 486 | 12,745 | 341 | 77.875% (76.205–80.400) | 77.476% (75.790–80.027) | 99.585% (97.531–99.588) | 86.894% (86.421–88.246) | +| 40% | 194 / 486 | 15,410 | 319 | 77.051% (75.106–77.968) | 76.628% (74.635–77.541) | 98.969% (97.945–100.000) | 84.589% (83.669–85.020) | +| 30% | 146 / 486 | 18,051 | 283 | 73.281% (71.502–75.971) | 72.785% (70.964–75.510) | 99.412% (98.824–100.000) | 80.130% (79.069–82.245) | +| 20% | 97 / 486 | 20,701 | 277 | 70.301% (66.061–73.722) | 69.742% (65.425–73.231) | 99.229% (98.201–99.486) | 75.465% (71.808–77.973) | +| 10% | 49 / 486 | 23,414 | 143 | 68.424% (62.659–71.339) | 67.824% (61.966–70.845) | 99.085% (97.254–100.000) | 71.140% (66.074–73.793) | + +
+
+ +
+German — de-de-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 54,092 / 54,092 | 0 | 0 | n/a | n/a | n/a | 92.725% | +| 90% | 48,683 / 54,092 | 27,967 | 5,327 | 67.322% (66.336–67.716) | 59.108% (57.552–59.642) | 93.117% (92.600–93.345) | 89.908% (89.841–89.987) | +| 80% | 43,274 / 54,092 | 57,109 | 9,524 | 66.531% (65.981–66.706) | 58.016% (57.177–58.316) | 92.441% (91.771–92.581) | 87.066% (86.980–87.121) | +| 70% | 37,864 / 54,092 | 87,177 | 12,653 | 65.784% (65.442–66.138) | 56.948% (56.648–57.396) | 91.897% (91.743–91.984) | 84.105% (83.974–84.251) | +| 60% | 32,455 / 54,092 | 118,916 | 14,631 | 65.062% (64.946–65.321) | 56.115% (55.912–56.303) | 91.372% (91.222–91.619) | 81.048% (80.993–81.213) | +| 50% | 27,046 / 54,092 | 151,254 | 15,476 | 64.493% (64.083–64.603) | 55.371% (54.795–55.510) | 90.896% (90.661–91.008) | 78.014% (77.799–78.062) | +| 40% | 21,637 / 54,092 | 184,273 | 15,105 | 63.564% (63.219–63.981) | 54.206% (53.646–54.739) | 90.246% (89.885–90.446) | 74.680% (74.544–74.924) | +| 30% | 16,228 / 54,092 | 219,296 | 13,318 | 62.696% (62.282–63.101) | 53.083% (52.613–53.663) | 89.436% (88.996–89.500) | 71.309% (71.004–71.509) | +| 20% | 10,818 / 54,092 | 255,708 | 10,520 | 61.569% (60.724–61.785) | 51.959% (50.533–52.161) | 88.244% (88.017–88.723) | 67.504% (66.844–67.696) | +| 10% | 5,409 / 54,092 | 293,618 | 6,095 | 59.280% (58.219–59.702) | 49.432% (48.026–49.731) | 86.150% (85.836–86.733) | 62.456% (61.579–62.888) | + +
+
+ +
+Gothic — got-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 6,227 / 6,227 | 0 | 0 | n/a | n/a | n/a | 97.403% | +| 90% | 5,604 / 6,227 | 13,232 | 649 | 80.472% (79.383–82.707) | 80.207% (79.175–82.565) | 87.083% (84.937–87.526) | 95.574% (95.449–95.740) | +| 80% | 4,982 / 6,227 | 26,611 | 1,193 | 80.622% (79.648–82.183) | 80.498% (79.469–82.070) | 85.100% (83.537–86.476) | 93.779% (93.537–94.088) | +| 70% | 4,359 / 6,227 | 39,937 | 1,522 | 79.657% (79.456–80.073) | 79.510% (79.290–79.986) | 83.477% (82.269–83.808) | 91.746% (91.603–91.859) | +| 60% | 3,736 / 6,227 | 53,676 | 1,749 | 78.335% (78.092–79.303) | 78.131% (77.945–79.165) | 82.319% (81.506–83.396) | 89.338% (89.245–89.820) | +| 50% | 3,114 / 6,227 | 67,349 | 1,787 | 77.388% (76.558–78.588) | 77.219% (76.413–78.480) | 81.137% (79.912–81.979) | 86.951% (86.541–87.607) | +| 40% | 2,491 / 6,227 | 81,349 | 1,720 | 75.277% (74.626–76.815) | 75.101% (74.451–76.677) | 79.386% (78.283–80.006) | 83.696% (83.412–84.687) | +| 30% | 1,868 / 6,227 | 95,318 | 1,471 | 73.681% (71.916–74.582) | 73.521% (71.620–74.436) | 77.919% (77.080–78.837) | 80.564% (79.306–81.072) | +| 20% | 1,245 / 6,227 | 109,649 | 1,133 | 70.595% (69.373–71.060) | 70.340% (68.955–70.794) | 77.280% (76.258–78.677) | 75.746% (74.681–76.058) | +| 10% | 623 / 6,227 | 123,790 | 616 | 65.034% (64.610–66.241) | 64.563% (64.213–65.759) | 75.487% (73.189–76.679) | 68.243% (67.743–69.337) | + +
+
+ +
+Greek — el-gr-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 11,793 / 11,793 | 0 | 0 | n/a | n/a | n/a | 95.703% | +| 90% | 10,614 / 11,793 | 7,513 | 543 | 71.881% (70.673–72.434) | 67.694% (66.472–68.281) | 94.512% (93.170–95.848) | 93.126% (92.922–93.181) | +| 80% | 9,434 / 11,793 | 15,114 | 990 | 70.762% (69.441–71.219) | 66.576% (65.106–67.130) | 94.125% (93.313–94.386) | 90.334% (90.110–90.375) | +| 70% | 8,255 / 11,793 | 22,824 | 1,334 | 69.379% (68.405–69.738) | 64.998% (63.943–65.390) | 93.759% (93.211–94.026) | 87.210% (87.051–87.446) | +| 60% | 7,076 / 11,793 | 30,453 | 1,688 | 68.280% (67.787–68.827) | 63.740% (63.142–64.446) | 93.424% (93.403–93.801) | 84.066% (84.041–84.361) | +| 50% | 5,897 / 11,793 | 38,644 | 1,755 | 67.003% (66.400–67.555) | 62.382% (61.713–62.966) | 93.192% (92.651–93.592) | 80.479% (80.469–80.793) | +| 40% | 4,717 / 11,793 | 46,615 | 1,785 | 65.678% (65.368–66.758) | 60.970% (60.594–62.151) | 92.560% (92.037–93.088) | 77.065% (76.619–77.463) | +| 30% | 3,538 / 11,793 | 54,643 | 1,734 | 64.663% (64.102–65.469) | 59.800% (58.994–60.719) | 92.403% (92.169–93.188) | 73.205% (72.745–73.703) | +| 20% | 2,359 / 11,793 | 62,654 | 1,562 | 62.992% (61.332–63.589) | 57.807% (55.922–58.522) | 92.323% (91.691–92.428) | 68.677% (67.388–69.178) | +| 10% | 1,179 / 11,793 | 70,918 | 1,288 | 60.208% (59.027–61.709) | 54.741% (53.440–56.626) | 90.957% (90.152–92.617) | 63.220% (61.976–64.391) | + +
+
+ +
+Gulf Arabic — afb-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 6,345 / 6,345 | 0 | 0 | n/a | n/a | n/a | 97.307% | +| 90% | 5,711 / 6,345 | 2,809 | 145 | 39.840% (39.643–40.940) | 24.696% (22.936–25.303) | 97.907% (97.271–98.387) | 91.713% (91.143–92.128) | +| 80% | 5,076 / 6,345 | 5,733 | 252 | 40.788% (39.624–41.737) | 24.656% (23.643–25.718) | 97.994% (97.753–98.320) | 86.169% (85.499–86.540) | +| 70% | 4,442 / 6,345 | 8,691 | 323 | 39.931% (39.496–40.628) | 24.104% (23.624–24.729) | 98.134% (97.704–98.455) | 80.309% (79.659–80.757) | +| 60% | 3,807 / 6,345 | 11,554 | 393 | 40.107% (39.628–40.305) | 23.806% (23.585–24.761) | 98.044% (97.885–98.325) | 74.721% (74.062–75.308) | +| 50% | 3,173 / 6,345 | 14,560 | 419 | 39.533% (39.024–39.733) | 23.435% (22.678–23.971) | 98.056% (97.387–98.152) | 68.693% (68.275–69.240) | +| 40% | 2,538 / 6,345 | 17,455 | 404 | 39.046% (38.782–39.550) | 22.895% (22.407–23.918) | 97.827% (97.510–98.198) | 62.847% (62.382–63.095) | +| 30% | 1,904 / 6,345 | 20,720 | 366 | 38.716% (37.872–39.173) | 22.870% (21.789–23.152) | 97.508% (97.416–97.983) | 56.523% (55.770–57.272) | +| 20% | 1,269 / 6,345 | 23,738 | 280 | 37.147% (35.732–38.147) | 20.812% (19.190–22.067) | 97.547% (97.507–97.901) | 49.605% (47.927–50.295) | +| 10% | 635 / 6,345 | 26,926 | 162 | 35.662% (34.014–36.076) | 19.314% (16.972–19.461) | 97.754% (96.471–97.859) | 42.088% (40.264–42.446) | + +
+
+ +
+Haida — hai-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 36 / 36 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 32 / 36 | 614 | 0 | 77.850% (52.512–81.922) | 77.705% (52.202–81.803) | 100.000% (75.000–100.000) | 97.474% (94.559–98.162) | +| 80% | 29 / 36 | 1,076 | 0 | 57.621% (54.498–67.721) | 57.530% (54.268–67.603) | 85.714% (71.429–85.714) | 91.922% (91.012–93.556) | +| 70% | 25 / 36 | 1,690 | 0 | 65.071% (58.885–73.609) | 64.961% (58.654–73.496) | 90.909% (81.818–90.909) | 89.025% (88.227–91.718) | +| 60% | 22 / 36 | 2,152 | 0 | 62.207% (50.640–74.117) | 62.041% (50.347–73.994) | 85.714% (78.571–92.857) | 85.942% (81.374–89.656) | +| 50% | 18 / 36 | 2,767 | 0 | 64.657% (47.677–70.401) | 64.499% (47.318–70.280) | 88.889% (77.778–100.000) | 81.820% (74.280–84.791) | +| 40% | 14 / 36 | 3,261 | 0 | 61.628% (41.138–69.599) | 61.435% (40.735–69.485) | 90.909% (86.364–100.000) | 76.100% (64.661–81.578) | +| 30% | 11 / 36 | 3,722 | 0 | 58.958% (55.611–60.183) | 58.734% (55.344–59.940) | 96.000% (92.000–100.000) | 71.625% (68.561–72.479) | +| 20% | 7 / 36 | 4,310 | 0 | 58.320% (50.980–59.165) | 58.084% (50.673–58.888) | 96.552% (93.103–100.000) | 66.128% (60.520–67.317) | +| 10% | 4 / 36 | 4,770 | 0 | 52.485% (50.388–61.069) | 52.164% (50.123–60.848) | 96.875% (90.625–100.000) | 57.920% (54.930–65.515) | + +
+
+ +
+Hebrew — he-il-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 2,358 / 2,358 | 0 | 0 | n/a | n/a | n/a | 98.250% | +| 90% | 2,122 / 2,358 | 5,914 | 196 | 21.813% (21.558–23.195) | 16.362% (15.846–17.917) | 88.785% (87.416–93.636) | 90.688% (90.578–90.888) | +| 80% | 1,886 / 2,358 | 11,868 | 390 | 21.715% (19.936–22.602) | 16.441% (14.182–17.242) | 89.628% (87.585–91.818) | 83.007% (82.678–83.339) | +| 70% | 1,651 / 2,358 | 17,846 | 499 | 21.488% (20.548–21.734) | 16.059% (15.072–16.256) | 88.713% (88.166–89.569) | 75.314% (75.106–75.538) | +| 60% | 1,415 / 2,358 | 23,856 | 551 | 20.530% (20.267–21.311) | 15.043% (14.721–15.824) | 88.702% (87.417–89.456) | 67.302% (67.264–67.705) | +| 50% | 1,179 / 2,358 | 29,985 | 576 | 20.205% (19.833–20.807) | 14.533% (14.185–15.123) | 88.771% (87.642–90.375) | 59.531% (59.185–59.763) | +| 40% | 943 / 2,358 | 36,073 | 547 | 19.758% (19.628–20.304) | 14.068% (13.841–14.608) | 89.236% (88.497–90.062) | 51.381% (51.289–51.799) | +| 30% | 707 / 2,358 | 42,248 | 465 | 19.125% (18.459–19.270) | 13.345% (12.730–13.462) | 88.961% (87.481–90.603) | 43.063% (42.627–43.264) | +| 20% | 472 / 2,358 | 48,464 | 365 | 17.734% (17.573–18.304) | 11.846% (11.656–12.416) | 89.000% (88.647–89.233) | 34.046% (33.884–34.388) | +| 10% | 236 / 2,358 | 54,723 | 206 | 16.532% (15.398–18.499) | 10.423% (9.169–12.678) | 89.756% (88.191–91.158) | 24.765% (23.666–26.661) | + +
+
+ +
+Hiligaynon — hil-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 97 / 97 | 0 | 0 | n/a | n/a | n/a | 98.744% | +| 90% | 87 / 97 | 38 | 1 | 23.684% (18.421–27.027) | 0.000% | 100.000% (87.500–100.000) | 91.457% (90.452–91.960) | +| 80% | 78 / 97 | 74 | 2 | 24.658% (21.622–25.676) | 0.000% | 100.000% (94.118–100.000) | 84.925% (83.417–84.925) | +| 70% | 68 / 97 | 114 | 4 | 24.138% (22.807–25.688) | 0.000% | 100.000% (96.296–100.000) | 76.884% (76.131–78.392) | +| 60% | 58 / 97 | 156 | 3 | 23.780% (23.602–24.359) | 0.000% | 100.000% (97.368–100.000) | 69.095% (67.588–69.347) | +| 50% | 49 / 97 | 198 | 3 | 24.242% (23.737–24.599) | 0.000% | 100.000% (97.872–100.000) | 62.060% (60.804–63.568) | +| 40% | 39 / 97 | 237 | 4 | 24.348% (24.167–24.473) | 0.000% | 100.000% (98.246–100.000) | 54.271% (53.266–55.276) | +| 30% | 29 / 97 | 277 | 3 | 24.014% (23.105–24.632) | 0.000% | 98.529% (94.118–100.000) | 45.729% (45.226–47.739) | +| 20% | 19 / 97 | 318 | 4 | 23.975% (23.197–24.214) | 0.000% | 98.701% (94.872–98.718) | 38.442% (38.442–38.693) | +| 10% | 10 / 97 | 356 | 0 | 24.079% (23.315–24.438) | 0.000% | 98.837% (95.402–100.000) | 31.910% (31.407–32.412) | + +
+
+ +
+Hsilimo — hsi-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 49 / 49 | 0 | 0 | n/a | n/a | n/a | 99.371% | +| 90% | 44 / 49 | 11 | 0 | 40.000% (25.000–45.455) | 0.000% (0.000–10.526) | 100.000% (80.000–100.000) | 95.597% (88.050–95.597) | +| 80% | 39 / 49 | 30 | 0 | 33.333% (30.769–43.478) | 7.692% (5.000–11.538) | 90.000% (90.000–100.000) | 86.792% (82.390–91.195) | +| 70% | 34 / 49 | 49 | 0 | 38.000% (30.952–41.667) | 11.429% (3.704–17.647) | 93.333% (80.000–100.000) | 81.132% (76.101–86.164) | +| 60% | 29 / 49 | 60 | 0 | 37.736% (32.353–44.000) | 10.417% (9.091–13.725) | 90.000% (85.000–95.000) | 76.101% (70.440–81.761) | +| 50% | 25 / 49 | 79 | 0 | 35.443% (32.500–40.678) | 10.256% (2.857–10.909) | 91.667% (83.333–95.833) | 67.925% (66.038–77.358) | +| 40% | 20 / 49 | 90 | 1 | 34.043% (30.841–36.364) | 6.410% (2.500–9.836) | 86.207% (82.759–96.552) | 62.264% (53.459–71.698) | +| 30% | 15 / 49 | 121 | 0 | 33.058% (27.344–38.095) | 6.000% (3.333–11.494) | 88.235% (85.294–97.059) | 49.057% (41.509–66.667) | +| 20% | 10 / 49 | 132 | 0 | 32.576% (28.777–38.947) | 7.143% (1.786–12.903) | 87.179% (79.487–92.308) | 44.025% (37.736–62.893) | +| 10% | 5 / 49 | 147 | 0 | 31.034% (19.048–35.338) | 6.731% (3.883–8.911) | 84.091% (54.545–100.000) | 37.107% (25.157–45.283) | + +
+
+ +
+Hungarian — hu-hu-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 19,406 / 19,406 | 0 | 0 | n/a | n/a | n/a | 99.222% | +| 90% | 17,465 / 19,406 | 92,253 | 1,360 | 92.212% (91.551–92.510) | 92.203% (91.476–92.460) | 93.446% (92.426–93.786) | 98.479% (98.407–98.493) | +| 80% | 15,525 / 19,406 | 184,401 | 2,417 | 91.771% (91.652–92.138) | 91.756% (91.609–92.123) | 92.536% (92.147–92.726) | 97.646% (97.626–97.712) | +| 70% | 13,584 / 19,406 | 277,343 | 3,300 | 91.756% (91.390–92.053) | 91.744% (91.350–92.058) | 91.940% (90.827–92.376) | 96.861% (96.757–96.947) | +| 60% | 11,644 / 19,406 | 370,552 | 3,836 | 91.156% (91.058–91.366) | 91.187% (91.074–91.418) | 90.673% (90.092–91.129) | 95.888% (95.829–95.941) | +| 50% | 9,703 / 19,406 | 463,693 | 3,952 | 90.729% (90.537–90.756) | 90.777% (90.567–90.810) | 89.530% (89.200–89.806) | 94.834% (94.749–94.863) | +| 40% | 7,762 / 19,406 | 557,829 | 3,717 | 90.002% (89.679–90.181) | 90.053% (89.707–90.254) | 88.776% (88.318–89.016) | 93.526% (93.390–93.683) | +| 30% | 5,822 / 19,406 | 651,802 | 3,207 | 89.047% (88.909–89.354) | 89.126% (89.006–89.437) | 87.178% (86.603–87.420) | 92.006% (91.882–92.204) | +| 20% | 3,881 / 19,406 | 746,292 | 2,401 | 88.280% (87.847–88.842) | 88.393% (87.931–88.968) | 85.865% (85.589–85.920) | 90.395% (90.054–90.844) | +| 10% | 1,941 / 19,406 | 840,551 | 1,399 | 86.761% (86.100–87.668) | 86.876% (86.177–87.841) | 84.086% (83.633–85.050) | 87.963% (87.354–88.786) | + +
+
+ +
+Icelandic — is-is-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 4,772 / 4,772 | 0 | 0 | n/a | n/a | n/a | 97.769% | +| 90% | 4,295 / 4,772 | 5,139 | 218 | 64.289% (61.499–65.188) | 61.732% (58.785–62.751) | 90.414% (89.912–90.828) | 94.390% (94.016–94.422) | +| 80% | 3,818 / 4,772 | 10,259 | 385 | 62.943% (61.744–63.349) | 60.284% (59.010–60.723) | 90.220% (89.531–90.497) | 90.741% (90.399–90.796) | +| 70% | 3,340 / 4,772 | 15,482 | 476 | 61.305% (60.755–62.259) | 58.577% (57.919–59.577) | 89.462% (88.501–89.753) | 86.875% (86.499–86.984) | +| 60% | 2,863 / 4,772 | 20,771 | 546 | 60.579% (60.281–60.931) | 57.821% (57.431–58.036) | 89.411% (87.216–90.535) | 82.786% (82.637–82.899) | +| 50% | 2,386 / 4,772 | 26,119 | 574 | 59.244% (58.720–59.451) | 56.433% (55.741–56.818) | 87.967% (86.583–89.537) | 78.407% (78.042–78.546) | +| 40% | 1,909 / 4,772 | 31,507 | 568 | 57.580% (56.882–57.842) | 54.646% (53.900–55.045) | 87.442% (85.776–87.790) | 73.593% (73.016–73.771) | +| 30% | 1,432 / 4,772 | 36,883 | 514 | 55.572% (54.757–56.370) | 52.696% (51.938–53.522) | 85.505% (83.693–87.120) | 68.233% (67.465–68.602) | +| 20% | 954 / 4,772 | 42,353 | 403 | 52.643% (51.376–53.000) | 49.578% (48.304–50.022) | 83.294% (82.782–84.766) | 61.534% (60.491–61.986) | +| 10% | 477 / 4,772 | 47,839 | 234 | 47.359% (45.806–47.853) | 43.922% (42.363–44.487) | 82.097% (80.468–82.846) | 52.246% (50.880–52.885) | + +
+
+ +
+Indonesian — id-id-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 3,877 / 3,877 | 0 | 0 | n/a | n/a | n/a | 99.991% | +| 90% | 3,489 / 3,877 | 2,128 | 0 | 25.191% (24.658–26.363) | 8.960% (8.724–10.172) | 97.938% (97.680–98.969) | 92.628% (92.243–92.652) | +| 80% | 3,102 / 3,877 | 4,313 | 1 | 25.369% (24.948–26.210) | 9.489% (9.065–10.139) | 98.189% (98.065–98.450) | 84.844% (84.665–85.318) | +| 70% | 2,714 / 3,877 | 6,413 | 1 | 25.443% (25.035–25.925) | 9.374% (9.002–9.848) | 98.194% (97.762–98.709) | 77.505% (77.129–78.181) | +| 60% | 2,326 / 3,877 | 8,477 | 1 | 25.561% (25.455–25.857) | 9.399% (9.005–9.667) | 98.645% (97.742–98.904) | 70.387% (69.772–70.739) | +| 50% | 1,939 / 3,877 | 10,630 | 1 | 25.680% (25.254–25.756) | 9.500% (8.838–9.904) | 98.503% (98.246–98.916) | 62.771% (62.053–63.119) | +| 40% | 1,551 / 3,877 | 12,805 | 1 | 25.615% (25.342–25.860) | 9.478% (9.047–9.886) | 98.366% (98.108–98.710) | 55.169% (54.897–55.451) | +| 30% | 1,163 / 3,877 | 14,933 | 0 | 25.486% (25.119–25.775) | 9.417% (8.774–9.572) | 98.563% (98.047–98.674) | 47.624% (47.432–48.051) | +| 20% | 775 / 3,877 | 17,102 | 0 | 25.219% (24.846–25.535) | 8.976% (8.465–9.414) | 98.420% (97.840–98.775) | 39.868% (39.675–40.436) | +| 10% | 388 / 3,877 | 19,211 | 0 | 25.084% (24.160–25.386) | 8.790% (7.665–9.036) | 98.653% (98.424–98.825) | 32.271% (31.641–32.919) | + +
+
+ +
+Ingrian — izh-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 50 / 50 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 45 / 50 | 103 | 0 | 31.068% (4.808–38.679) | 28.571% (0.000–36.634) | 100.000% (80.000–100.000) | 93.080% (90.351–93.665) | +| 80% | 40 / 50 | 206 | 0 | 32.500% (18.537–38.350) | 29.474% (14.359–35.204) | 100.000% (90.000–100.000) | 86.647% (83.723–87.622) | +| 70% | 35 / 50 | 308 | 0 | 24.503% (18.065–32.692) | 21.603% (14.237–29.630) | 93.333% (80.000–100.000) | 77.778% (75.244–79.532) | +| 60% | 30 / 50 | 408 | 0 | 22.542% (14.976–26.225) | 18.892% (10.914–23.454) | 90.000% (80.000–95.000) | 68.811% (65.692–70.663) | +| 50% | 25 / 50 | 512 | 0 | 23.183% (15.595–26.953) | 19.421% (11.475–23.614) | 96.000% (88.000–96.000) | 61.891% (57.797–63.548) | +| 40% | 20 / 50 | 613 | 0 | 15.447% (12.235–24.188) | 11.282% (8.576–20.819) | 90.000% (76.667–96.667) | 49.318% (47.563–54.483) | +| 30% | 15 / 50 | 716 | 0 | 17.246% (14.385–19.720) | 13.158% (10.866–15.735) | 94.286% (74.286–97.143) | 42.008% (40.253–44.055) | +| 20% | 10 / 50 | 819 | 0 | 14.408% (12.317–16.361) | 10.398% (8.184–12.580) | 90.000% (80.000–95.000) | 31.676% (29.825–33.236) | +| 10% | 5 / 50 | 923 | 0 | 14.843% (8.234–16.143) | 11.060% (5.011–12.984) | 82.222% (71.111–93.333) | 23.392% (17.446–24.561) | + +
+
+ +
+Irish — ga-ie-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 7,287 / 7,287 | 0 | 0 | n/a | n/a | n/a | 97.970% | +| 90% | 6,558 / 7,287 | 2,441 | 97 | 62.515% (58.703–64.252) | 48.046% (45.339–50.654) | 95.286% (94.193–96.591) | 94.249% (93.625–94.628) | +| 80% | 5,830 / 7,287 | 4,727 | 169 | 62.021% (59.558–62.780) | 47.969% (45.253–48.409) | 95.400% (94.433–95.593) | 90.494% (89.818–91.126) | +| 70% | 5,101 / 7,287 | 7,180 | 192 | 59.959% (59.763–61.558) | 46.037% (45.003–47.023) | 94.747% (93.942–94.855) | 86.353% (85.966–87.242) | +| 60% | 4,372 / 7,287 | 9,534 | 229 | 60.036% (58.677–61.097) | 45.376% (44.646–46.897) | 93.897% (93.196–94.281) | 82.905% (81.608–83.219) | +| 50% | 3,644 / 7,287 | 12,029 | 243 | 58.873% (58.630–60.254) | 43.920% (43.837–46.118) | 93.630% (92.576–93.996) | 78.209% (77.981–78.967) | +| 40% | 2,915 / 7,287 | 14,494 | 242 | 58.366% (57.960–60.425) | 43.582% (42.943–46.513) | 93.353% (92.730–93.718) | 74.161% (73.631–75.478) | +| 30% | 2,186 / 7,287 | 16,925 | 214 | 57.339% (56.455–58.407) | 42.711% (41.117–43.849) | 92.463% (92.177–93.337) | 69.213% (68.459–70.354) | +| 20% | 1,457 / 7,287 | 19,432 | 187 | 56.294% (55.272–56.642) | 41.141% (39.771–41.389) | 92.039% (91.943–92.453) | 64.574% (63.539–64.990) | +| 10% | 729 / 7,287 | 21,938 | 112 | 52.789% (52.518–53.743) | 36.464% (35.603–37.611) | 91.687% (91.525–92.561) | 57.290% (56.866–58.370) | + +
+
+ +
+Italian — it-it-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 10,009 / 10,009 | 0 | 0 | n/a | n/a | n/a | 99.056% | +| 90% | 9,008 / 10,009 | 33,156 | 558 | 93.965% (93.205–94.474) | 93.600% (92.783–94.134) | 99.900% (99.600–100.000) | 98.473% (98.385–98.520) | +| 80% | 8,007 / 10,009 | 66,376 | 1,008 | 93.594% (93.308–93.801) | 93.192% (92.886–93.418) | 99.900% (99.750–99.950) | 97.816% (97.782–97.848) | +| 70% | 7,006 / 10,009 | 99,459 | 1,285 | 93.065% (92.894–93.176) | 92.631% (92.450–92.747) | 99.867% (99.800–99.900) | 97.072% (97.043–97.115) | +| 60% | 6,005 / 10,009 | 132,730 | 1,556 | 92.510% (92.196–92.768) | 92.037% (91.712–92.312) | 99.863% (99.750–99.900) | 96.267% (96.099–96.348) | +| 50% | 5,005 / 10,009 | 166,665 | 1,637 | 91.882% (91.717–92.283) | 91.375% (91.195–91.801) | 99.840% (99.780–99.920) | 95.296% (95.190–95.476) | +| 40% | 4,004 / 10,009 | 199,950 | 1,479 | 91.606% (91.107–91.863) | 91.077% (90.547–91.356) | 99.842% (99.783–99.883) | 94.458% (94.135–94.521) | +| 30% | 3,003 / 10,009 | 234,119 | 1,370 | 90.846% (90.583–91.131) | 90.272% (89.989–90.573) | 99.879% (99.800–99.929) | 93.137% (92.971–93.395) | +| 20% | 2,002 / 10,009 | 268,626 | 1,035 | 89.611% (89.163–90.122) | 88.964% (88.479–89.500) | 99.888% (99.725–99.963) | 91.388% (91.005–91.839) | +| 10% | 1,001 / 10,009 | 303,036 | 513 | 87.447% (87.014–87.906) | 86.665% (86.198–87.161) | 99.794% (99.556–99.917) | 88.555% (88.194–88.979) | + +
+
+ +
+Itelmen — itl-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,633 / 1,633 | 0 | 0 | n/a | n/a | n/a | 99.891% | +| 90% | 1,470 / 1,633 | 345 | 0 | 52.754% (49.322–58.095) | 14.835% (10.680–18.421) | 95.092% (93.865–98.160) | 95.441% (94.731–96.287) | +| 80% | 1,306 / 1,633 | 699 | 0 | 52.217% (50.689–54.425) | 14.500% (12.694–15.670) | 95.719% (94.495–96.024) | 90.800% (90.145–91.455) | +| 70% | 1,143 / 1,633 | 1,070 | 1 | 51.648% (49.228–53.571) | 14.925% (11.438–15.663) | 96.524% (95.297–96.933) | 85.804% (84.630–86.759) | +| 60% | 980 / 1,633 | 1,423 | 2 | 51.406% (49.557–53.127) | 14.324% (12.255–16.083) | 96.319% (94.172–96.933) | 81.054% (79.716–81.682) | +| 50% | 817 / 1,633 | 1,812 | 2 | 50.084% (49.007–50.416) | 12.755% (10.030–13.392) | 96.201% (94.969–96.687) | 74.884% (74.720–75.566) | +| 40% | 653 / 1,633 | 2,205 | 2 | 49.162% (48.934–49.794) | 11.491% (10.440–12.893) | 96.327% (95.603–97.143) | 69.397% (69.206–69.997) | +| 30% | 490 / 1,633 | 2,547 | 2 | 48.486% (48.284–50.640) | 10.965% (9.979–11.570) | 96.056% (95.534–97.025) | 64.182% (63.582–66.257) | +| 20% | 327 / 1,633 | 2,914 | 1 | 48.461% (47.577–49.429) | 9.724% (9.327–10.992) | 95.865% (95.475–97.011) | 58.832% (57.439–60.060) | +| 10% | 163 / 1,633 | 3,297 | 1 | 47.766% (47.277–49.802) | 9.390% (7.953–11.443) | 96.463% (95.034–97.211) | 53.372% (52.143–55.037) | + +
+
+ +
+Japanese — ja-jp-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 107 / 107 | 0 | 0 | n/a | n/a | n/a | 93.582% | +| 90% | 96 / 107 | 1,051 | 289 | 32.953% (16.580–35.775) | 32.358% (15.612–35.159) | 100.000% | 87.129% (86.680–87.759) | +| 80% | 86 / 107 | 2,026 | 460 | 29.268% (18.252–31.935) | 28.591% (17.410–31.256) | 100.000% | 81.686% (79.055–82.057) | +| 70% | 75 / 107 | 3,125 | 613 | 30.312% (22.755–34.724) | 29.529% (22.050–34.088) | 100.000% (93.750–100.000) | 75.224% (71.981–76.803) | +| 60% | 64 / 107 | 4,313 | 464 | 28.292% (23.997–36.642) | 27.617% (23.279–35.994) | 100.000% (95.349–100.000) | 68.953% (65.485–70.549) | +| 50% | 54 / 107 | 5,315 | 626 | 27.649% (23.698–35.551) | 26.984% (22.938–34.958) | 96.154% (94.231–100.000) | 61.767% (59.938–65.666) | +| 40% | 43 / 107 | 6,522 | 403 | 29.566% (20.132–35.524) | 28.902% (19.399–34.983) | 96.825% (95.238–98.438) | 56.461% (51.259–58.756) | +| 30% | 32 / 107 | 8,012 | 276 | 27.471% (23.357–30.781) | 26.833% (22.715–30.177) | 95.946% (93.243–98.630) | 47.886% (43.892–50.138) | +| 20% | 21 / 107 | 9,182 | 222 | 28.009% (21.292–30.423) | 27.447% (20.600–29.827) | 95.294% (88.372–100.000) | 41.615% (35.343–42.227) | +| 10% | 11 / 107 | 10,133 | 214 | 24.223% (19.027–29.252) | 23.578% (18.330–28.626) | 92.708% (92.632–96.875) | 32.203% (27.372–35.913) | + +
+
+ +
+Kabardian — kbd-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 249 / 249 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 224 / 249 | 307 | 0 | 59.283% (51.792–61.489) | 56.738% (48.227–58.719) | 88.000% (80.000–96.000) | 95.907% (95.154–96.103) | +| 80% | 199 / 249 | 612 | 0 | 57.166% (52.614–62.233) | 53.723% (49.288–59.928) | 90.000% (86.000–96.000) | 91.388% (90.504–92.469) | +| 70% | 174 / 249 | 920 | 0 | 54.148% (51.316–56.957) | 50.654% (47.790–53.846) | 92.000% (85.333–93.333) | 86.248% (85.363–87.033) | +| 60% | 149 / 249 | 1,222 | 0 | 54.010% (53.295–57.878) | 50.980% (50.221–54.844) | 88.000% (88.000–92.000) | 81.598% (81.205–83.104) | +| 50% | 125 / 249 | 1,518 | 0 | 53.057% (48.219–58.432) | 49.821% (44.325–55.667) | 90.323% (89.516–91.935) | 76.621% (74.296–79.339) | +| 40% | 100 / 249 | 1,824 | 0 | 51.371% (47.917–52.626) | 48.358% (44.119–49.792) | 89.933% (84.564–91.275) | 70.956% (68.893–71.644) | +| 30% | 75 / 249 | 2,131 | 0 | 46.457% (43.212–50.328) | 42.514% (39.348–47.293) | 86.782% (84.483–90.805) | 62.639% (60.282–65.324) | +| 20% | 50 / 249 | 2,441 | 0 | 46.308% (40.639–48.259) | 42.296% (36.128–44.871) | 91.457% (82.915–93.467) | 57.138% (52.554–58.644) | +| 10% | 25 / 249 | 2,748 | 0 | 38.014% (32.860–46.308) | 32.990% (27.456–42.059) | 93.750% (80.804–94.643) | 44.204% (39.587–51.670) | + +
+
+ +
+Kalaallisut — kl-gl-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 23 / 23 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 21 / 23 | 28 | 0 | 28.571% (7.143–75.000) | 23.077% (0.000–73.077) | 100.000% | 93.769% (91.900–97.819) | +| 80% | 18 / 23 | 70 | 0 | 26.087% (7.246–31.429) | 20.312% (0.000–26.154) | 100.000% | 84.112% (80.062–85.047) | +| 70% | 16 / 23 | 98 | 0 | 17.526% (11.224–28.571) | 11.111% (4.396–23.077) | 100.000% | 75.078% (72.897–78.193) | +| 60% | 14 / 23 | 125 | 0 | 20.800% (12.800–30.952) | 14.655% (6.034–25.641) | 100.000% | 69.159% (66.044–72.897) | +| 50% | 12 / 23 | 153 | 0 | 22.078% (14.379–29.412) | 16.084% (7.746–23.944) | 100.000% | 62.617% (59.190–66.355) | +| 40% | 9 / 23 | 195 | 0 | 23.590% (21.026–28.718) | 17.680% (14.917–23.204) | 100.000% (92.857–100.000) | 53.583% (52.025–56.698) | +| 30% | 7 / 23 | 223 | 0 | 26.457% (8.072–32.735) | 21.256% (0.966–27.536) | 93.750% (93.750–100.000) | 48.910% (36.137–53.271) | +| 20% | 5 / 23 | 251 | 0 | 21.514% (8.367–33.865) | 15.451% (1.288–29.185) | 100.000% (94.444–100.000) | 38.629% (28.349–48.287) | +| 10% | 2 / 23 | 293 | 0 | 21.502% (7.850–29.693) | 15.441% (0.735–26.103) | 100.000% (76.190–100.000) | 28.349% (15.888–35.826) | + +
+
+ +
+Kannada — kn-in-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 159 / 159 | 0 | 0 | n/a | n/a | n/a | 91.241% | +| 90% | 143 / 159 | 387 | 36 | 75.223% (71.008–86.996) | 74.537% (70.721–85.990) | 93.750% (75.000–100.000) | 89.177% (88.049–90.545) | +| 80% | 127 / 159 | 779 | 77 | 78.354% (54.367–88.331) | 78.232% (52.532–87.863) | 90.625% (81.250–96.875) | 87.137% (83.969–89.513) | +| 70% | 111 / 159 | 1,175 | 113 | 79.745% (60.090–82.735) | 79.462% (58.531–82.481) | 89.583% (85.417–95.833) | 85.529% (82.697–85.697) | +| 60% | 95 / 159 | 1,495 | 140 | 76.067% (59.623–80.262) | 75.547% (58.065–79.783) | 89.062% (87.500–93.750) | 81.689% (78.426–84.377) | +| 50% | 80 / 159 | 1,884 | 176 | 67.685% (63.587–83.071) | 67.094% (62.143–83.048) | 89.873% (82.278–92.405) | 76.770% (75.618–84.401) | +| 40% | 64 / 159 | 2,257 | 221 | 71.988% (65.007–84.571) | 71.137% (63.961–84.689) | 86.316% (82.105–92.632) | 75.114% (73.938–83.633) | +| 30% | 48 / 159 | 2,591 | 243 | 67.518% (60.948–79.644) | 66.653% (59.428–79.578) | 89.189% (81.081–96.396) | 70.050% (66.067–79.002) | +| 20% | 32 / 159 | 3,093 | 297 | 64.719% (61.571–77.494) | 63.920% (60.302–77.249) | 84.252% (81.102–91.339) | 64.747% (62.707–76.146) | +| 10% | 16 / 159 | 3,388 | 315 | 71.257% (46.724–75.516) | 70.520% (44.838–75.135) | 83.916% (76.224–89.510) | 67.819% (48.044–71.994) | + +
+
+ +
+Karelian — krl-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 20 / 20 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 18 / 20 | 57 | 0 | 63.793% (3.571–91.228) | 62.500% (0.000–90.909) | 100.000% (50.000–100.000) | 96.290% (90.459–99.117) | +| 80% | 16 / 20 | 114 | 0 | 33.913% (15.929–59.649) | 31.532% (12.844–58.182) | 100.000% (75.000–100.000) | 86.572% (83.216–91.873) | +| 70% | 14 / 20 | 171 | 0 | 40.000% (19.883–40.351) | 38.182% (16.970–38.788) | 83.333% (83.333–100.000) | 81.979% (75.795–81.979) | +| 60% | 12 / 20 | 227 | 0 | 36.444% (27.313–41.667) | 35.484% (24.658–40.367) | 87.500% (62.500–100.000) | 74.735% (70.848–76.678) | +| 50% | 10 / 20 | 283 | 0 | 34.859% (13.380–43.772) | 32.847% (10.219–43.173) | 80.000% (60.000–100.000) | 67.314% (56.537–72.085) | +| 40% | 8 / 20 | 340 | 0 | 37.941% (18.475–52.522) | 35.976% (15.502–52.000) | 83.333% (66.667–100.000) | 62.721% (50.883–71.731) | +| 30% | 6 / 20 | 397 | 0 | 35.859% (11.055–46.348) | 34.293% (7.812–45.431) | 78.571% (71.429–100.000) | 55.124% (37.456–62.367) | +| 20% | 4 / 20 | 453 | 0 | 23.620% (15.859–31.347) | 21.281% (13.242–29.977) | 87.500% (68.750–100.000) | 38.869% (32.509–45.053) | +| 10% | 2 / 20 | 510 | 0 | 16.275% (13.725–24.902) | 13.415% (11.179–22.561) | 88.889% (77.778–94.444) | 24.558% (22.261–32.332) | + +
+
+ +
+Kashubian — csb-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 37 / 37 | 0 | 0 | n/a | n/a | n/a | 98.592% | +| 90% | 33 / 37 | 38 | 0 | 45.161% (26.667–78.947) | 37.037% (15.385–76.471) | 100.000% | 94.366% (91.549–96.338) | +| 80% | 30 / 37 | 62 | 0 | 46.667% (22.414–88.406) | 39.623% (11.765–87.097) | 100.000% | 89.577% (87.042–96.338) | +| 70% | 26 / 37 | 101 | 0 | 47.525% (44.444–76.852) | 41.111% (37.500–74.227) | 100.000% | 85.352% (83.662–91.549) | +| 60% | 22 / 37 | 139 | 1 | 54.422% (43.796–70.504) | 50.000% (36.885–66.935) | 100.000% (93.333–100.000) | 80.000% (78.028–87.042) | +| 50% | 19 / 37 | 167 | 1 | 61.932% (43.373–70.659) | 58.228% (38.255–67.333) | 94.444% (88.235–100.000) | 79.718% (73.239–84.789) | +| 40% | 15 / 37 | 207 | 1 | 63.415% (42.927–72.947) | 59.783% (36.957–69.730) | 95.455% (95.238–100.000) | 78.028% (66.761–83.944) | +| 30% | 11 / 37 | 247 | 1 | 61.728% (50.206–63.563) | 57.798% (44.954–59.729) | 96.000% (92.308–100.000) | 73.521% (65.634–74.085) | +| 20% | 7 / 37 | 284 | 1 | 56.446% (45.296–61.620) | 52.140% (38.911–57.874) | 96.552% (93.333–100.000) | 63.662% (55.775–69.296) | +| 10% | 4 / 37 | 315 | 0 | 51.746% (29.936–55.414) | 47.518% (22.420–50.355) | 93.939% (87.879–100.000) | 57.183% (38.028–60.282) | + +
+
+ +
+Kazakh — kk-kz-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,744 / 1,744 | 0 | 0 | n/a | n/a | n/a | 99.870% | +| 90% | 1,570 / 1,744 | 3,543 | 10 | 89.984% (88.675–94.239) | 89.542% (88.200–94.208) | 94.828% (93.103–98.844) | 98.835% (98.755–99.314) | +| 80% | 1,395 / 1,744 | 7,139 | 15 | 90.499% (88.868–92.873) | 90.154% (88.588–92.829) | 94.813% (93.696–97.126) | 97.976% (97.530–98.483) | +| 70% | 1,221 / 1,744 | 10,678 | 25 | 90.825% (89.263–91.628) | 90.536% (89.012–91.489) | 95.183% (94.242–96.346) | 97.145% (96.561–97.400) | +| 60% | 1,046 / 1,744 | 14,436 | 21 | 90.053% (88.731–91.144) | 89.777% (88.443–90.998) | 95.396% (93.966–95.696) | 95.815% (95.205–96.365) | +| 50% | 872 / 1,744 | 17,925 | 22 | 90.309% (89.158–91.132) | 90.042% (88.871–90.963) | 95.287% (94.374–95.502) | 95.038% (94.241–95.509) | +| 40% | 698 / 1,744 | 21,450 | 21 | 89.753% (89.473–89.919) | 89.496% (89.200–89.648) | 94.823% (93.971–95.110) | 93.601% (93.499–93.884) | +| 30% | 523 / 1,744 | 24,875 | 19 | 89.918% (88.571–90.303) | 89.736% (88.231–90.098) | 94.668% (93.443–95.399) | 92.844% (91.760–93.144) | +| 20% | 349 / 1,744 | 28,152 | 22 | 89.158% (87.839–89.727) | 88.909% (87.502–89.544) | 93.978% (93.252–94.971) | 91.191% (90.056–91.769) | +| 10% | 174 / 1,744 | 31,956 | 2 | 84.509% (81.987–87.786) | 84.045% (81.378–87.437) | 93.810% (91.013–94.837) | 86.033% (83.646–88.857) | + +
+
+ +
+Khakas — kjh-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 74 / 74 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 67 / 74 | 110 | 0 | 89.091% (83.636–97.297) | 90.291% (83.495–97.115) | 85.714% (71.429–100.000) | 98.976% (98.464–99.744) | +| 80% | 59 / 74 | 236 | 0 | 88.983% (86.809–89.496) | 88.739% (86.364–89.140) | 93.333% (80.000–100.000) | 97.782% (97.355–97.867) | +| 70% | 52 / 74 | 346 | 0 | 90.517% (83.815–92.264) | 90.184% (83.642–92.049) | 95.455% (86.364–95.455) | 97.184% (95.222–97.696) | +| 60% | 44 / 74 | 473 | 0 | 90.063% (86.469–92.437) | 90.068% (86.230–92.152) | 93.333% (90.000–96.667) | 95.990% (94.539–96.928) | +| 50% | 37 / 74 | 585 | 0 | 87.521% (73.024–91.610) | 87.226% (71.560–91.590) | 91.892% (89.189–94.595) | 93.771% (86.604–95.819) | +| 40% | 30 / 74 | 696 | 0 | 88.429% (75.325–90.531) | 87.957% (73.960–90.505) | 95.455% (88.636–95.455) | 93.089% (85.410–94.369) | +| 30% | 22 / 74 | 822 | 0 | 78.710% (73.082–88.443) | 77.922% (71.521–87.953) | 96.154% (90.385–98.077) | 85.068% (81.143–91.894) | +| 20% | 15 / 74 | 934 | 0 | 72.883% (68.128–76.231) | 71.167% (65.982–75.200) | 98.305% (91.525–100.000) | 78.413% (74.573–81.058) | +| 10% | 7 / 74 | 1,061 | 0 | 67.295% (61.038–73.917) | 65.392% (58.812–72.663) | 95.522% (92.537–98.507) | 70.392% (64.761–76.365) | + +
+
+ +
+Khinalugh — klr-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 591 / 591 | 0 | 0 | n/a | n/a | n/a | 91.557% | +| 90% | 532 / 591 | 5,193 | 1,045 | 6.586% (6.293–7.570) | 5.512% (5.242–6.477) | 100.000% | 83.423% (82.881–83.851) | +| 80% | 473 / 591 | 10,373 | 1,645 | 7.299% (6.460–7.719) | 6.228% (5.448–6.656) | 100.000% (99.153–100.000) | 75.545% (74.568–76.001) | +| 70% | 414 / 591 | 16,046 | 2,000 | 7.716% (6.375–8.050) | 6.682% (5.331–7.017) | 100.000% (99.435–100.000) | 67.384% (66.358–67.695) | +| 60% | 355 / 591 | 21,089 | 2,596 | 7.225% (6.959–8.578) | 6.167% (5.900–7.543) | 100.000% (99.576–100.000) | 59.711% (58.392–60.055) | +| 50% | 296 / 591 | 27,059 | 2,482 | 7.417% (6.642–8.527) | 6.397% (5.602–7.526) | 100.000% (99.661–100.000) | 51.396% (50.589–52.169) | +| 40% | 236 / 591 | 33,181 | 2,371 | 7.505% (6.579–8.113) | 6.504% (5.548–7.125) | 100.000% (99.437–100.000) | 43.279% (42.383–43.289) | +| 30% | 177 / 591 | 39,347 | 2,013 | 7.712% (7.589–8.641) | 6.738% (6.606–7.664) | 100.000% (99.758–100.000) | 34.666% (33.449–35.629) | +| 20% | 118 / 591 | 45,965 | 1,650 | 7.750% (7.367–8.809) | 6.792% (6.422–7.861) | 100.000% (99.789–100.000) | 25.996% (23.831–26.552) | +| 10% | 59 / 591 | 52,551 | 937 | 7.120% (6.516–8.075) | 6.167% (5.560–7.133) | 100.000% (99.812–100.000) | 16.381% (14.826–17.670) | + +
+
+ +
+Kodi — kod-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 65 / 65 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 59 / 65 | 48 | 0 | 12.500% (12.500–20.000) | 0.000% (0.000–10.204) | 100.000% | 91.985% (91.603–91.985) | +| 80% | 52 / 65 | 104 | 0 | 15.315% (12.500–20.192) | 4.082% (0.000–9.184) | 100.000% | 83.015% (82.061–84.160) | +| 70% | 46 / 65 | 152 | 0 | 15.789% (13.333–18.543) | 3.759% (0.763–6.818) | 100.000% | 75.191% (74.046–76.527) | +| 60% | 39 / 65 | 213 | 0 | 14.493% (13.592–16.432) | 2.660% (1.111–4.813) | 100.000% | 66.031% (65.076–66.221) | +| 50% | 33 / 65 | 261 | 0 | 14.504% (12.644–17.625) | 2.609% (0.437–6.114) | 100.000% | 57.252% (56.489–58.969) | +| 40% | 26 / 65 | 317 | 0 | 15.773% (12.579–17.350) | 3.957% (0.358–5.755) | 100.000% | 49.046% (46.947–50.000) | +| 30% | 20 / 65 | 365 | 0 | 14.835% (12.295–17.582) | 2.821% (0.000–5.956) | 100.000% | 40.840% (38.740–42.748) | +| 20% | 13 / 65 | 420 | 0 | 17.381% (12.352–19.524) | 5.978% (0.271–8.152) | 100.000% (98.077–100.000) | 33.779% (29.580–35.496) | +| 10% | 7 / 65 | 468 | 0 | 15.139% (13.248–16.880) | 3.171% (1.463–5.854) | 96.552% (94.828–100.000) | 24.046% (22.519–25.763) | + +
+
+ +
+Kongo — kon-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 186 / 186 | 0 | 0 | n/a | n/a | n/a | 99.821% | +| 90% | 167 / 186 | 57 | 0 | 64.912% (61.404–66.667) | 47.368% (42.105–50.000) | 100.000% | 96.237% (95.878–96.416) | +| 80% | 149 / 186 | 110 | 1 | 64.865% (62.162–66.364) | 47.297% (44.595–50.000) | 100.000% (97.297–100.000) | 92.832% (92.294–93.190) | +| 70% | 130 / 186 | 168 | 0 | 64.881% (62.500–66.467) | 48.649% (44.643–50.000) | 100.000% (94.643–100.000) | 89.427% (88.530–89.785) | +| 60% | 112 / 186 | 222 | 0 | 65.315% (63.063–66.516) | 48.299% (45.270–50.000) | 100.000% (95.946–100.000) | 86.201% (85.125–86.559) | +| 50% | 93 / 186 | 279 | 0 | 65.468% (64.158–66.547) | 48.649% (46.237–50.000) | 100.000% (96.774–100.000) | 82.616% (81.900–83.154) | +| 40% | 74 / 186 | 336 | 0 | 65.774% (61.012–66.369) | 48.879% (41.518–50.000) | 100.000% (97.321–100.000) | 79.391% (76.344–79.749) | +| 30% | 56 / 186 | 390 | 0 | 64.010% (61.282–66.410) | 45.946% (41.923–50.000) | 99.231% (97.692–100.000) | 74.731% (72.760–76.523) | +| 20% | 37 / 186 | 447 | 0 | 66.219% (58.613–66.443) | 50.000% (38.255–50.000) | 99.329% (98.658–100.000) | 72.939% (66.667–73.118) | +| 10% | 19 / 186 | 501 | 0 | 66.267% (62.200–66.467) | 50.000% (43.243–50.000) | 99.401% (98.802–100.000) | 69.713% (65.950–69.892) | + +
+
+ +
+Kyrgyz — ky-kg-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 72 / 72 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 65 / 72 | 291 | 0 | 61.724% (58.419–89.078) | 60.777% (57.394–88.811) | 100.000% | 96.296% (95.963–98.932) | +| 80% | 58 / 72 | 584 | 0 | 72.260% (67.526–76.298) | 71.579% (66.725–75.709) | 100.000% | 94.595% (93.694–95.429) | +| 70% | 50 / 72 | 917 | 0 | 67.467% (55.761–72.618) | 66.890% (54.677–71.942) | 100.000% (90.909–100.000) | 90.057% (86.420–91.658) | +| 60% | 43 / 72 | 1,208 | 0 | 60.348% (58.989–69.421) | 59.372% (57.980–68.840) | 100.000% (93.103–100.000) | 84.017% (83.483–87.654) | +| 50% | 36 / 72 | 1,499 | 0 | 61.041% (56.379–67.399) | 60.219% (55.305–66.735) | 100.000% (94.444–100.000) | 80.514% (78.212–83.650) | +| 40% | 29 / 72 | 1,791 | 0 | 57.327% (49.888–65.737) | 56.275% (48.768–65.009) | 95.349% (95.349–100.000) | 74.541% (70.103–79.513) | +| 30% | 22 / 72 | 2,080 | 0 | 57.109% (43.861–64.856) | 56.152% (42.575–64.089) | 96.000% (96.000–100.000) | 70.204% (61.094–75.609) | +| 20% | 14 / 72 | 2,414 | 0 | 53.499% (47.015–60.821) | 52.440% (45.794–59.941) | 96.552% (96.552–100.000) | 62.529% (57.357–68.468) | +| 10% | 7 / 72 | 2,705 | 0 | 42.382% (31.941–45.830) | 41.038% (30.341–44.575) | 96.923% | 48.015% (38.572–51.018) | + +
+
+ +
+Ladin — lld-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 180 / 180 | 0 | 0 | n/a | n/a | n/a | 99.279% | +| 90% | 162 / 180 | 491 | 0 | 77.800% (60.887–79.388) | 76.956% (59.414–78.602) | 100.000% (94.444–100.000) | 97.033% (95.282–97.198) | +| 80% | 144 / 180 | 972 | 0 | 78.969% (66.157–80.597) | 78.373% (64.868–80.066) | 97.222% (94.286–100.000) | 95.076% (92.439–95.529) | +| 70% | 126 / 180 | 1,433 | 11 | 73.688% (70.986–76.343) | 72.820% (69.936–75.580) | 96.226% (94.444–98.148) | 91.533% (90.606–92.295) | +| 60% | 108 / 180 | 1,911 | 11 | 76.335% (73.781–76.939) | 75.571% (72.882–76.157) | 97.183% (95.833–98.611) | 89.905% (88.752–90.214) | +| 50% | 90 / 180 | 2,421 | 11 | 73.647% (69.540–76.177) | 72.727% (68.841–75.346) | 97.753% (87.778–98.889) | 86.341% (83.993–87.495) | +| 40% | 72 / 180 | 2,881 | 25 | 67.750% (67.515–75.417) | 66.655% (66.546–74.576) | 97.196% (89.815–98.131) | 80.264% (79.810–84.693) | +| 30% | 54 / 180 | 3,375 | 25 | 68.326% (64.209–74.487) | 67.251% (62.921–73.657) | 96.000% (90.476–97.600) | 77.750% (74.495–81.809) | +| 20% | 36 / 180 | 3,865 | 25 | 71.384% (57.749–74.390) | 70.361% (56.314–73.538) | 96.503% (91.667–97.902) | 76.597% (65.843–79.151) | +| 10% | 18 / 180 | 4,357 | 1 | 63.080% (61.212–76.946) | 62.091% (59.785–76.109) | 93.210% (86.420–98.758) | 66.399% (65.183–78.822) | + +
+
+ +
+Latin — la-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 46,170 / 46,170 | 0 | 0 | n/a | n/a | n/a | 97.133% | +| 90% | 41,553 / 46,170 | 47,545 | 2,582 | 78.052% (77.620–78.993) | 76.426% (76.045–77.427) | 94.736% (93.520–94.916) | 95.060% (95.027–95.163) | +| 80% | 36,936 / 46,170 | 95,630 | 4,516 | 77.449% (77.132–78.300) | 75.824% (75.533–76.728) | 94.046% (93.465–94.163) | 92.975% (92.830–93.105) | +| 70% | 32,319 / 46,170 | 143,770 | 5,859 | 77.125% (76.973–77.589) | 75.499% (75.342–75.988) | 93.403% (93.114–93.657) | 90.839% (90.691–90.941) | +| 60% | 27,702 / 46,170 | 192,657 | 6,861 | 76.929% (76.487–77.265) | 75.320% (74.870–75.700) | 92.940% (92.669–93.079) | 88.703% (88.535–88.817) | +| 50% | 23,085 / 46,170 | 242,767 | 7,221 | 76.450% (76.123–76.728) | 74.847% (74.507–75.156) | 92.520% (92.212–92.682) | 86.369% (86.304–86.552) | +| 40% | 18,468 / 46,170 | 292,467 | 6,915 | 75.970% (75.530–76.199) | 74.377% (73.904–74.592) | 92.097% (91.700–92.251) | 84.073% (83.852–84.239) | +| 30% | 13,851 / 46,170 | 344,053 | 6,094 | 75.177% (74.966–75.591) | 73.506% (73.313–73.994) | 91.405% (91.208–91.850) | 81.431% (81.316–81.762) | +| 20% | 9,234 / 46,170 | 395,630 | 4,643 | 73.983% (73.755–74.765) | 72.279% (72.003–73.162) | 90.627% (90.348–91.119) | 78.388% (78.224–78.965) | +| 10% | 4,617 / 46,170 | 447,988 | 2,618 | 72.583% (70.889–72.914) | 70.834% (68.965–71.203) | 89.804% (89.258–89.896) | 74.894% (73.426–75.227) | + +
+
+ +
+Latvian — lv-lv-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 7,238 / 7,238 | 0 | 0 | n/a | n/a | n/a | 94.098% | +| 90% | 6,514 / 7,238 | 7,202 | 850 | 79.567% (77.953–79.911) | 77.881% (76.181–78.385) | 96.183% (95.483–96.621) | 92.272% (92.198–92.332) | +| 80% | 5,790 / 7,238 | 14,695 | 1,512 | 78.588% (77.419–79.735) | 76.842% (75.548–78.150) | 96.213% (95.973–96.877) | 90.436% (90.241–90.588) | +| 70% | 5,067 / 7,238 | 22,244 | 1,986 | 77.781% (76.813–78.430) | 76.017% (74.893–76.728) | 95.775% (95.622–96.171) | 88.489% (88.253–88.615) | +| 60% | 4,343 / 7,238 | 29,791 | 2,255 | 76.641% (76.242–77.432) | 74.797% (74.289–75.627) | 95.613% (95.404–95.948) | 86.316% (86.149–86.592) | +| 50% | 3,619 / 7,238 | 37,686 | 2,349 | 75.857% (75.500–76.087) | 73.934% (73.553–74.183) | 95.248% (95.113–95.345) | 84.197% (83.988–84.319) | +| 40% | 2,895 / 7,238 | 45,887 | 2,257 | 74.784% (74.270–75.101) | 72.828% (72.268–73.132) | 94.682% (94.520–95.029) | 81.728% (81.481–82.086) | +| 30% | 2,171 / 7,238 | 54,169 | 1,991 | 73.419% (72.903–73.876) | 71.372% (70.760–71.872) | 94.029% (93.670–94.516) | 79.030% (78.728–79.433) | +| 20% | 1,448 / 7,238 | 62,679 | 1,487 | 72.310% (71.337–72.412) | 70.176% (69.052–70.300) | 93.799% (93.797–94.400) | 76.267% (75.497–76.292) | +| 10% | 724 / 7,238 | 71,294 | 855 | 70.293% (68.353–71.195) | 68.037% (65.975–69.051) | 92.818% (92.118–93.056) | 72.405% (70.742–73.318) | + +
+
+ +
+Lingala — lin-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 57 / 57 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 51 / 57 | 24 | 0 | 79.167% (70.833–83.333) | 72.222% (61.111–77.778) | 100.000% | 97.826% (96.957–98.261) | +| 80% | 46 / 57 | 44 | 0 | 79.545% (77.273–86.667) | 72.727% (69.697–82.353) | 100.000% | 96.087% (95.652–97.391) | +| 70% | 40 / 57 | 69 | 0 | 85.507% (77.941–86.957) | 80.769% (70.588–82.692) | 100.000% | 95.652% (93.478–96.087) | +| 60% | 34 / 57 | 93 | 0 | 84.043% (80.435–86.022) | 78.873% (73.913–81.429) | 100.000% | 93.478% (92.174–94.348) | +| 50% | 29 / 57 | 113 | 0 | 82.301% (77.193–86.842) | 77.647% (69.767–82.558) | 100.000% (96.429–100.000) | 91.304% (88.696–93.478) | +| 40% | 23 / 57 | 138 | 0 | 78.986% (71.014–83.212) | 72.115% (62.500–78.641) | 97.059% (97.059–100.000) | 87.391% (82.609–90.000) | +| 30% | 17 / 57 | 162 | 0 | 74.534% (70.370–75.776) | 66.942% (60.656–68.595) | 97.500% (97.500–100.000) | 82.174% (79.130–83.043) | +| 20% | 11 / 57 | 186 | 0 | 69.892% (68.280–71.505) | 60.714% (57.857–62.857) | 100.000% (97.826–100.000) | 75.652% (74.348–76.957) | +| 10% | 6 / 57 | 206 | 0 | 62.621% (59.709–66.990) | 50.323% (46.452–56.129) | 100.000% (98.039–100.000) | 66.522% (63.913–70.435) | + +
+
+ +
+Lithuanian — lt-lt-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,391 / 1,391 | 0 | 0 | n/a | n/a | n/a | 99.624% | +| 90% | 1,252 / 1,391 | 3,098 | 20 | 52.408% (48.059–60.658) | 50.780% (45.635–59.213) | 92.806% (89.928–94.245) | 94.782% (94.141–95.334) | +| 80% | 1,113 / 1,391 | 5,924 | 38 | 56.555% (52.829–58.980) | 54.788% (50.959–57.262) | 93.165% (92.446–93.885) | 90.803% (89.472–91.261) | +| 70% | 974 / 1,391 | 8,816 | 45 | 55.172% (53.005–56.545) | 53.233% (51.076–54.632) | 93.046% (92.326–94.245) | 86.251% (85.230–87.230) | +| 60% | 835 / 1,391 | 11,423 | 41 | 54.959% (53.373–55.165) | 53.023% (51.393–53.172) | 92.973% (92.266–93.705) | 81.954% (80.964–82.213) | +| 50% | 696 / 1,391 | 14,406 | 42 | 54.463% (52.847–55.810) | 52.556% (50.779–53.867) | 92.374% (91.655–93.813) | 77.147% (76.819–77.926) | +| 40% | 556 / 1,391 | 17,342 | 48 | 54.480% (50.707–55.714) | 52.544% (48.617–53.826) | 92.934% (92.695–93.533) | 72.567% (69.857–73.081) | +| 30% | 417 / 1,391 | 20,211 | 41 | 51.961% (49.300–54.333) | 49.943% (47.092–52.387) | 92.916% (92.189–93.737) | 66.787% (64.518–67.681) | +| 20% | 278 / 1,391 | 23,192 | 25 | 50.338% (47.680–52.416) | 48.199% (45.387–50.333) | 93.172% (91.105–93.980) | 60.139% (58.049–61.566) | +| 10% | 139 / 1,391 | 26,154 | 15 | 41.955% (39.757–46.002) | 39.455% (37.165–43.747) | 90.895% (90.415–91.693) | 47.596% (46.017–51.131) | + +
+
+ +
+Livonian — liv-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 201 / 201 | 0 | 0 | n/a | n/a | n/a | 99.409% | +| 90% | 181 / 201 | 273 | 0 | 67.925% (64.945–75.824) | 65.714% (63.347–73.913) | 94.737% (85.000–100.000) | 96.456% (96.108–97.116) | +| 80% | 161 / 201 | 562 | 0 | 72.152% (69.603–72.810) | 69.981% (67.904–70.866) | 92.500% (92.308–100.000) | 93.989% (93.294–94.406) | +| 70% | 141 / 201 | 842 | 5 | 70.441% (65.367–73.134) | 68.590% (63.290–71.007) | 95.000% (94.915–100.000) | 90.723% (88.603–92.078) | +| 60% | 121 / 201 | 1,128 | 6 | 66.696% (65.000–68.440) | 64.423% (62.685–66.239) | 96.250% (92.405–97.500) | 86.484% (85.302–87.526) | +| 50% | 101 / 201 | 1,435 | 6 | 66.500% (62.604–66.690) | 64.385% (60.119–64.719) | 94.000% (92.929–96.000) | 82.835% (80.646–83.391) | +| 40% | 80 / 201 | 1,729 | 11 | 62.558% (57.524–67.989) | 60.510% (54.592–65.926) | 94.215% (90.000–95.833) | 77.137% (74.531–80.056) | +| 30% | 60 / 201 | 2,013 | 16 | 58.718% (51.079–66.965) | 56.487% (48.138–64.939) | 90.071% (88.571–94.326) | 70.744% (65.566–76.338) | +| 20% | 40 / 201 | 2,295 | 11 | 57.118% (49.735–62.832) | 54.695% (46.629–60.890) | 90.062% (88.750–90.683) | 65.497% (59.868–69.805) | +| 10% | 20 / 201 | 2,588 | 11 | 42.835% (33.321–48.370) | 38.905% (29.295–45.200) | 90.556% (87.293–95.580) | 48.158% (39.576–53.405) | + +
+
+ +
+Low German — nds-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 325 / 325 | 0 | 0 | n/a | n/a | n/a | 98.035% | +| 90% | 293 / 325 | 246 | 10 | 48.770% (38.846–54.545) | 41.509% (31.004–47.964) | 96.875% (93.333–100.000) | 93.202% (91.984–93.517) | +| 80% | 260 / 325 | 501 | 20 | 49.301% (45.309–54.851) | 42.202% (37.385–48.409) | 98.438% (96.825–98.462) | 88.055% (87.466–89.077) | +| 70% | 228 / 325 | 754 | 22 | 46.997% (42.308–53.774) | 40.179% (35.258–47.214) | 94.845% (90.625–97.917) | 82.475% (81.139–84.558) | +| 60% | 195 / 325 | 1,016 | 27 | 44.389% (42.421–49.650) | 37.244% (35.738–43.070) | 93.023% (88.372–96.850) | 76.542% (75.324–78.232) | +| 50% | 163 / 325 | 1,274 | 25 | 44.861% (42.048–45.997) | 37.751% (35.106–38.744) | 93.125% (88.889–96.855) | 71.434% (69.391–71.827) | +| 40% | 130 / 325 | 1,526 | 16 | 43.559% (39.043–44.233) | 36.329% (32.077–37.698) | 88.601% (87.629–97.927) | 65.344% (62.043–65.933) | +| 30% | 98 / 325 | 1,780 | 11 | 39.446% (37.875–42.391) | 32.172% (30.344–35.085) | 90.265% (86.726–96.000) | 56.621% (55.756–59.411) | +| 20% | 65 / 325 | 2,032 | 11 | 33.560% (32.430–37.451) | 25.889% (23.785–30.095) | 89.535% (86.873–91.506) | 45.776% (45.540–49.470) | +| 10% | 33 / 325 | 2,289 | 8 | 29.623% (26.394–33.814) | 20.833% (18.603–25.876) | 90.000% (80.069–92.808) | 36.582% (33.045–40.314) | + +
+
+ +
+Lower Sorbian — dsb-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 990 / 990 | 0 | 0 | n/a | n/a | n/a | 98.713% | +| 90% | 891 / 990 | 1,191 | 35 | 46.264% (45.200–52.122) | 42.009% (41.628–48.914) | 89.691% (82.474–94.792) | 93.572% (93.424–93.809) | +| 80% | 792 / 990 | 2,394 | 58 | 47.460% (45.155–52.924) | 43.845% (41.610–49.707) | 88.083% (85.128–91.667) | 88.545% (87.947–89.406) | +| 70% | 693 / 990 | 3,603 | 71 | 46.784% (43.297–49.183) | 43.096% (39.474–45.926) | 86.644% (84.983–89.726) | 83.478% (82.035–83.806) | +| 60% | 594 / 990 | 4,825 | 83 | 44.684% (42.560–48.259) | 40.956% (38.568–44.901) | 86.957% (86.041–89.286) | 76.984% (76.845–78.460) | +| 50% | 495 / 990 | 5,987 | 78 | 45.645% (44.147–48.204) | 42.032% (40.470–44.796) | 86.558% (84.929–89.634) | 72.352% (71.786–73.532) | +| 40% | 396 / 990 | 7,214 | 78 | 44.627% (42.546–45.774) | 40.775% (38.793–42.062) | 87.075% (84.407–88.494) | 66.227% (65.349–67.334) | +| 30% | 297 / 990 | 8,432 | 62 | 43.759% (42.434–44.378) | 39.792% (38.375–40.579) | 87.391% (84.761–88.646) | 60.339% (59.651–60.815) | +| 20% | 198 / 990 | 9,687 | 47 | 42.199% (40.525–46.112) | 38.157% (36.675–42.646) | 85.298% (82.532–87.802) | 53.690% (52.329–56.609) | +| 10% | 99 / 990 | 10,941 | 35 | 40.243% (34.758–42.127) | 36.394% (29.972–38.736) | 83.802% (80.359–88.739) | 46.163% (41.259–48.024) | + +
+
+ +
+Luganda — lg-ug-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 87 / 87 | 0 | 0 | n/a | n/a | n/a | 98.462% | +| 90% | 78 / 87 | 478 | 12 | 1.883% (1.863–2.079) | 0.000% | 100.000% | 88.580% (88.475–89.528) | +| 80% | 70 / 87 | 902 | 13 | 1.885% (1.872–1.947) | 0.000% | 100.000% | 79.815% (79.688–80.426) | +| 70% | 61 / 87 | 1,382 | 25 | 1.881% (1.854–1.929) | 0.000% | 100.000% | 69.890% (69.469–70.607) | +| 60% | 52 / 87 | 1,846 | 73 | 1.896% (1.871–1.913) | 0.000% | 100.000% | 60.303% (59.777–60.641) | +| 50% | 44 / 87 | 2,284 | 73 | 1.883% (1.861–1.894) | 0.000% | 100.000% | 51.370% (51.201–51.686) | +| 40% | 35 / 87 | 2,775 | 61 | 1.874% (1.847–1.885) | 0.000% | 100.000% | 41.361% (41.256–41.445) | +| 30% | 26 / 87 | 3,298 | 24 | 1.850% (1.835–1.869) | 0.000% | 100.000% | 31.247% (31.121–31.290) | +| 20% | 17 / 87 | 3,815 | 0 | 1.835% (1.834–1.862) | 0.000% | 100.000% | 21.070% (20.965–21.091) | +| 10% | 9 / 87 | 4,251 | 0 | 1.835% (1.833–1.854) | 0.000% | 100.000% | 11.989% (11.968–12.073) | + +
+
+ +
+Macedonian — mk-mk-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 9,788 / 9,788 | 0 | 0 | n/a | n/a | n/a | 99.414% | +| 90% | 8,809 / 9,788 | 13,423 | 164 | 80.917% (79.930–82.055) | 80.215% (79.079–81.339) | 91.014% (89.597–91.605) | 97.523% (97.448–97.623) | +| 80% | 7,830 / 9,788 | 27,148 | 280 | 81.039% (80.669–81.481) | 80.369% (79.904–80.929) | 90.609% (88.743–91.266) | 95.655% (95.619–95.783) | +| 70% | 6,852 / 9,788 | 40,859 | 389 | 80.732% (80.307–81.301) | 80.051% (79.540–80.702) | 89.812% (89.129–90.549) | 93.678% (93.530–93.925) | +| 60% | 5,873 / 9,788 | 54,136 | 436 | 80.283% (79.406–80.775) | 79.588% (78.599–80.107) | 89.518% (89.353–90.080) | 91.712% (91.266–91.864) | +| 50% | 4,894 / 9,788 | 67,687 | 435 | 79.484% (79.052–79.861) | 78.686% (78.227–79.128) | 89.823% (89.387–89.959) | 89.381% (89.070–89.582) | +| 40% | 3,915 / 9,788 | 81,525 | 413 | 78.965% (77.747–79.261) | 78.230% (76.855–78.519) | 89.389% (88.553–89.843) | 87.073% (86.277–87.204) | +| 30% | 2,936 / 9,788 | 95,378 | 360 | 77.935% (76.280–78.779) | 77.056% (75.315–78.031) | 88.763% (88.263–89.409) | 84.248% (83.133–84.873) | +| 20% | 1,958 / 9,788 | 109,039 | 265 | 75.934% (74.893–77.333) | 75.052% (73.871–76.479) | 88.214% (87.347–88.560) | 80.633% (79.675–81.699) | +| 10% | 979 / 9,788 | 122,831 | 127 | 72.790% (72.183–75.657) | 71.795% (71.133–74.901) | 85.672% (85.477–87.760) | 75.465% (74.818–77.963) | + +
+
+ +
+Magahi — mag-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 938 / 938 | 0 | 0 | n/a | n/a | n/a | 99.451% | +| 90% | 844 / 938 | 142 | 1 | 71.831% (69.427–78.125) | 26.562% (22.449–34.286) | 97.849% (94.624–98.925) | 96.703% (96.154–97.527) | +| 80% | 750 / 938 | 301 | 3 | 69.204% (66.369–83.333) | 24.561% (19.608–33.962) | 97.326% (96.257–98.930) | 93.201% (91.690–96.703) | +| 70% | 657 / 938 | 450 | 4 | 69.333% (65.574–74.265) | 21.154% (18.248–22.656) | 97.857% (96.786–98.571) | 89.973% (88.049–92.239) | +| 60% | 563 / 938 | 584 | 4 | 69.692% (67.203–74.259) | 20.565% (18.957–21.687) | 97.594% (96.247–98.391) | 87.294% (85.577–89.904) | +| 50% | 469 / 938 | 713 | 4 | 70.407% (67.624–72.957) | 20.000% (16.290–22.819) | 96.154% (95.931–97.430) | 84.959% (82.555–86.951) | +| 40% | 375 / 938 | 863 | 5 | 69.177% (66.447–74.055) | 20.601% (18.991–22.837) | 95.907% (94.643–98.046) | 81.319% (78.709–85.302) | +| 30% | 281 / 938 | 1,013 | 4 | 69.200% (66.350–71.310) | 19.218% (16.573–20.168) | 95.725% (94.656–97.717) | 78.297% (75.412–80.563) | +| 20% | 188 / 938 | 1,142 | 3 | 68.860% (67.075–71.074) | 16.552% (14.504–20.370) | 96.800% (94.660–96.933) | 75.343% (73.283–77.335) | +| 10% | 94 / 938 | 1,297 | 1 | 67.896% (66.492–69.011) | 17.031% (12.373–19.604) | 95.498% (94.187–98.104) | 71.223% (69.231–72.253) | + +
+
+ +
+Malagasy — mg-mg-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 159 / 159 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 143 / 159 | 64 | 0 | 25.000% | 0.000% | 100.000% | 92.453% | +| 80% | 127 / 159 | 128 | 0 | 25.000% | 0.000% | 100.000% | 84.906% | +| 70% | 111 / 159 | 192 | 0 | 25.000% | 0.000% | 100.000% | 77.358% | +| 60% | 95 / 159 | 256 | 0 | 25.000% | 0.000% | 100.000% | 69.811% | +| 50% | 80 / 159 | 316 | 0 | 25.000% | 0.000% | 100.000% | 62.736% | +| 40% | 64 / 159 | 380 | 0 | 25.000% | 0.000% | 100.000% | 55.189% | +| 30% | 48 / 159 | 444 | 0 | 25.000% | 0.000% | 100.000% | 47.642% | +| 20% | 32 / 159 | 508 | 0 | 25.000% | 0.000% | 100.000% | 40.094% | +| 10% | 16 / 159 | 572 | 0 | 25.000% | 0.000% | 100.000% | 32.547% | + +
+
+ +
+Maltese — mt-mt-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 112 / 112 | 0 | 0 | n/a | n/a | n/a | 99.400% | +| 90% | 101 / 112 | 144 | 0 | 16.667% (8.333–21.333) | 10.072% (1.493–15.108) | 100.000% (90.909–100.000) | 91.333% (90.600–92.133) | +| 80% | 90 / 112 | 295 | 0 | 13.851% (11.702–15.333) | 7.299% (4.981–8.633) | 95.455% (95.238–100.000) | 82.533% (82.400–82.800) | +| 70% | 78 / 112 | 454 | 1 | 11.184% (10.538–14.097) | 4.662% (3.632–7.143) | 96.970% (91.176–100.000) | 72.800% (71.933–74.000) | +| 60% | 67 / 112 | 592 | 8 | 13.007% (10.697–14.480) | 6.093% (4.021–7.749) | 95.556% (95.455–100.000) | 65.067% (62.667–66.000) | +| 50% | 56 / 112 | 737 | 8 | 12.278% (11.564–13.067) | 5.579% (4.965–6.110) | 96.364% (90.909–100.000) | 56.533% (54.533–57.333) | +| 40% | 45 / 112 | 887 | 8 | 11.939% (11.387–12.059) | 5.153% (4.390–5.370) | 96.970% (92.424–97.015) | 47.000% (45.800–47.933) | +| 30% | 34 / 112 | 1,038 | 8 | 12.694% (11.542–13.572) | 5.765% (4.932–7.121) | 94.872% (92.308–97.436) | 38.600% (38.267–39.600) | +| 20% | 22 / 112 | 1,199 | 8 | 12.010% (10.461–13.756) | 5.140% (3.648–7.918) | 95.556% (86.667–100.000) | 29.400% (27.533–29.667) | +| 10% | 11 / 112 | 1,351 | 0 | 12.435% (8.754–12.702) | 5.578% (1.363–6.027) | 98.020% (88.119–100.000) | 20.733% (18.000–21.533) | + +
+
+ +
+Manx — gv-im-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 80% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 70% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 60% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 50% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 40% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 30% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 20% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 10% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | + +
+
+ +
+Maori — mi-nz-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 104 / 104 | 0 | 0 | n/a | n/a | n/a | 99.519% | +| 90% | 94 / 104 | 20 | 0 | 70.000% (65.000–75.000) | 40.000% (30.000–50.000) | 100.000% (90.000–100.000) | 96.635% (96.154–97.115) | +| 80% | 83 / 104 | 41 | 1 | 68.293% (58.537–76.190) | 40.000% (20.000–52.381) | 95.238% (95.238–100.000) | 93.269% (91.346–95.192) | +| 70% | 73 / 104 | 61 | 1 | 67.213% (62.903–75.410) | 33.333% (25.806–50.000) | 100.000% (96.774–100.000) | 89.904% (88.942–92.308) | +| 60% | 62 / 104 | 83 | 1 | 68.675% (63.095–71.084) | 36.585% (28.571–41.463) | 97.619% (97.619–100.000) | 87.019% (85.096–87.981) | +| 50% | 52 / 104 | 103 | 1 | 67.308% (66.346–72.816) | 36.538% (33.333–47.059) | 98.077% (96.154–100.000) | 83.654% (83.173–86.058) | +| 40% | 42 / 104 | 124 | 0 | 70.161% (66.935–79.675) | 41.935% (33.871–60.656) | 98.387% (96.774–100.000) | 82.212% (80.288–87.500) | +| 30% | 31 / 104 | 146 | 0 | 67.123% (64.384–77.931) | 34.247% (30.137–58.333) | 98.630% (94.521–100.000) | 76.923% (75.000–84.135) | +| 20% | 21 / 104 | 166 | 0 | 67.470% (62.048–75.904) | 39.759% (25.301–54.217) | 97.590% (95.181–98.795) | 74.038% (69.712–80.769) | +| 10% | 10 / 104 | 188 | 0 | 70.213% (57.447–75.000) | 41.489% (17.021–52.128) | 97.872% (97.872–98.936) | 73.077% (61.538–77.404) | + +
+
+ +
+Mapudungun — arn-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 26 / 26 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 23 / 26 | 63 | 0 | 46.032% (33.333–88.889) | 43.333% (30.000–88.333) | 100.000% | 93.796% (92.336–98.723) | +| 80% | 21 / 26 | 105 | 0 | 48.571% (33.962–73.333) | 46.000% (30.693–72.000) | 100.000% | 90.146% (87.226–94.891) | +| 70% | 18 / 26 | 169 | 0 | 59.172% (43.787–69.643) | 57.143% (40.994–68.125) | 100.000% | 87.409% (82.664–90.693) | +| 60% | 16 / 26 | 211 | 0 | 56.398% (51.659–66.351) | 54.229% (49.254–64.677) | 100.000% | 83.212% (81.387–87.044) | +| 50% | 13 / 26 | 275 | 0 | 58.909% (48.175–67.636) | 56.870% (45.594–66.031) | 100.000% | 79.380% (74.088–83.759) | +| 40% | 10 / 26 | 338 | 0 | 55.621% (49.704–67.160) | 53.583% (47.205–65.528) | 100.000% (93.750–100.000) | 72.628% (68.978–79.745) | +| 30% | 8 / 26 | 380 | 0 | 53.421% (32.368–62.105) | 51.105% (29.006–60.221) | 100.000% (94.444–100.000) | 67.701% (53.102–73.723) | +| 20% | 5 / 26 | 443 | 0 | 40.858% (35.892–58.916) | 37.915% (32.701–56.872) | 100.000% | 52.190% (48.175–66.788) | +| 10% | 3 / 26 | 485 | 0 | 37.320% (25.361–54.227) | 34.199% (21.645–51.948) | 100.000% | 44.526% (33.942–59.489) | + +
+
+ +
+Middle French — frm-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 603 / 603 | 0 | 0 | n/a | n/a | n/a | 99.224% | +| 90% | 543 / 603 | 2,680 | 24 | 80.852% (77.393–87.793) | 80.409% (76.870–87.519) | 100.000% | 97.364% (97.075–98.005) | +| 80% | 482 / 603 | 5,421 | 54 | 79.722% (78.325–83.290) | 79.283% (77.824–82.909) | 100.000% (99.174–100.000) | 95.160% (95.017–95.954) | +| 70% | 422 / 603 | 8,121 | 79 | 78.997% (76.000–83.088) | 78.533% (75.466–82.715) | 99.448% (99.448–100.000) | 92.952% (92.242–94.234) | +| 60% | 362 / 603 | 10,807 | 114 | 77.429% (75.058–81.635) | 76.913% (74.495–81.226) | 99.585% (99.585–100.000) | 90.368% (89.569–92.000) | +| 50% | 302 / 603 | 13,506 | 126 | 78.703% (76.302–79.765) | 78.235% (75.765–79.311) | 99.668% (99.336–99.668) | 88.709% (87.662–89.445) | +| 40% | 241 / 603 | 16,254 | 122 | 77.925% (76.130–80.706) | 77.429% (75.615–80.280) | 99.448% (97.514–99.724) | 86.318% (85.147–87.981) | +| 30% | 181 / 603 | 18,971 | 96 | 77.220% (72.026–80.620) | 76.750% (71.416–80.192) | 99.526% (97.867–99.763) | 83.748% (80.113–86.223) | +| 20% | 121 / 603 | 21,738 | 72 | 76.858% (71.690–78.493) | 76.334% (71.072–78.018) | 99.378% (98.133–100.000) | 81.277% (77.100–82.482) | +| 10% | 60 / 603 | 24,561 | 13 | 75.715% (67.286–79.763) | 75.188% (66.585–79.341) | 98.343% (98.343–100.000) | 78.117% (70.546–81.511) | + +
+
+ +
+Middle High German — gmh-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 29 / 29 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 26 / 29 | 40 | 0 | 52.500% (6.000–84.000) | 48.649% (0.000–81.818) | 100.000% | 95.052% (87.760–98.958) | +| 80% | 23 / 29 | 84 | 0 | 60.870% (29.167–75.000) | 58.140% (22.727–73.077) | 100.000% (83.333–100.000) | 90.625% (86.719–94.531) | +| 70% | 20 / 29 | 119 | 0 | 51.376% (40.146–74.545) | 47.000% (35.938–72.277) | 100.000% (88.889–100.000) | 86.198% (78.646–92.708) | +| 60% | 17 / 29 | 157 | 0 | 37.736% (16.340–60.000) | 32.653% (12.057–56.391) | 100.000% (66.667–100.000) | 74.219% (64.323–84.896) | +| 50% | 15 / 29 | 190 | 0 | 33.333% (16.959–43.455) | 27.027% (12.102–38.983) | 100.000% (71.429–100.000) | 69.271% (58.594–71.875) | +| 40% | 12 / 29 | 230 | 0 | 34.634% (20.796–51.082) | 28.723% (16.746–47.196) | 100.000% (70.588–100.000) | 63.542% (53.385–70.573) | +| 30% | 9 / 29 | 270 | 0 | 27.737% (20.641–38.148) | 24.016% (15.041–33.200) | 95.000% (75.000–100.000) | 48.438% (41.927–57.031) | +| 20% | 6 / 29 | 307 | 0 | 19.544% (10.280–24.912) | 14.437% (4.362–18.321) | 95.652% (82.609–100.000) | 35.677% (25.000–44.271) | +| 10% | 3 / 29 | 347 | 0 | 16.954% (6.761–23.631) | 10.248% (0.608–17.445) | 100.000% (84.615–100.000) | 27.344% (13.802–33.594) | + +
+
+ +
+Middle Low German — gml-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 54 / 54 | 0 | 0 | n/a | n/a | n/a | 98.268% | +| 90% | 49 / 54 | 68 | 0 | 54.717% (23.529–77.778) | 50.000% (17.460–76.119) | 100.000% | 93.543% (90.079–95.748) | +| 80% | 43 / 54 | 135 | 5 | 52.778% (38.519–79.710) | 48.872% (33.065–78.125) | 100.000% (90.909–100.000) | 87.559% (85.197–93.858) | +| 70% | 38 / 54 | 187 | 5 | 51.429% (28.877–66.495) | 47.423% (23.392–63.483) | 100.000% (87.500–100.000) | 82.205% (78.110–88.189) | +| 60% | 32 / 54 | 249 | 5 | 44.726% (38.824–60.241) | 39.535% (33.906–58.150) | 90.909% (77.273–95.455) | 77.638% (73.701–82.835) | +| 50% | 27 / 54 | 310 | 5 | 44.194% (35.938–50.489) | 40.283% (30.375–47.143) | 96.296% (85.185–96.296) | 71.181% (67.559–75.591) | +| 40% | 22 / 54 | 372 | 5 | 42.473% (26.250–48.943) | 37.941% (20.652–44.242) | 90.625% (87.097–96.875) | 64.724% (53.386–71.811) | +| 30% | 16 / 54 | 443 | 1 | 38.600% (19.136–45.455) | 33.744% (13.839–41.732) | 84.211% (81.579–92.105) | 55.591% (37.953–62.520) | +| 20% | 11 / 54 | 510 | 1 | 32.941% (19.378–35.903) | 27.778% (13.095–32.589) | 90.476% (69.767–95.349) | 45.354% (30.551–49.606) | +| 10% | 5 / 54 | 580 | 0 | 27.692% (9.866–30.690) | 22.761% (3.097–25.612) | 81.633% (75.510–85.714) | 33.386% (15.118–36.693) | + +
+
+ +
+Mongolian — mn-mn-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 2,140 / 2,140 | 0 | 0 | n/a | n/a | n/a | 97.942% | +| 90% | 1,926 / 2,140 | 1,714 | 56 | 76.453% (72.769–80.955) | 73.677% (69.631–78.721) | 95.755% (93.839–96.682) | 95.691% (95.311–96.146) | +| 80% | 1,712 / 2,140 | 3,409 | 118 | 75.666% (74.572–77.183) | 72.905% (71.589–74.456) | 95.735% (94.258–96.690) | 93.287% (93.009–93.583) | +| 70% | 1,498 / 2,140 | 5,132 | 173 | 74.603% (73.765–76.228) | 71.662% (70.738–73.587) | 95.404% (93.968–95.741) | 90.678% (90.388–91.156) | +| 60% | 1,284 / 2,140 | 6,853 | 203 | 73.499% (73.147–74.946) | 70.647% (70.032–72.323) | 94.104% (93.832–95.171) | 87.853% (87.654–88.439) | +| 50% | 1,070 / 2,140 | 8,623 | 185 | 72.898% (71.936–73.941) | 70.038% (68.929–71.161) | 93.585% (93.302–94.821) | 85.159% (84.698–85.528) | +| 40% | 856 / 2,140 | 10,368 | 176 | 72.162% (71.020–73.814) | 69.196% (67.738–71.116) | 93.087% (92.543–94.362) | 82.118% (81.430–83.175) | +| 30% | 642 / 2,140 | 12,165 | 150 | 70.675% (69.396–72.765) | 67.631% (65.971–69.943) | 92.632% (92.550–93.830) | 78.793% (77.633–80.299) | +| 20% | 428 / 2,140 | 13,953 | 142 | 69.689% (66.518–70.499) | 66.477% (62.702–67.386) | 92.907% (92.443–93.783) | 75.405% (72.540–75.678) | +| 10% | 214 / 2,140 | 15,767 | 54 | 65.937% (64.789–68.812) | 62.041% (60.864–65.609) | 93.080% (91.150–94.072) | 69.130% (67.805–71.631) | + +
+
+ +
+Murrinh-Patha — mwf-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 29 / 29 | 0 | 0 | n/a | n/a | n/a | 82.222% | +| 90% | 26 / 29 | 55 | 20 | 3.636% (1.724–5.128) | 0.000% (0.000–2.222) | 100.000% (66.667–100.000) | 74.861% (73.889–77.083) | +| 80% | 23 / 29 | 116 | 37 | 3.922% (0.833–4.902) | 0.000% (0.000–1.020) | 100.000% (33.333–100.000) | 67.639% (65.694–69.028) | +| 70% | 20 / 29 | 165 | 55 | 3.553% (2.484–4.575) | 0.000% (0.000–0.680) | 85.714% (66.667–100.000) | 60.417% (56.667–62.500) | +| 60% | 17 / 29 | 228 | 69 | 3.774% (2.881–4.386) | 0.000% (0.000–0.490) | 87.500% (75.000–90.909) | 54.583% (52.222–55.833) | +| 50% | 15 / 29 | 271 | 79 | 3.610% (2.951–4.833) | 0.375% (0.000–0.781) | 90.000% (84.615–90.909) | 50.139% (46.250–52.500) | +| 40% | 12 / 29 | 343 | 77 | 3.499% (3.306–4.545) | 0.000% (0.000–0.633) | 92.308% (92.308–100.000) | 40.139% (39.722–42.917) | +| 30% | 9 / 29 | 419 | 68 | 3.890% (3.341–4.423) | 0.000% (0.000–0.513) | 94.118% (93.333–100.000) | 32.639% (31.667–35.000) | +| 20% | 6 / 29 | 519 | 61 | 3.861% (3.238–3.940) | 0.000% (0.000–0.195) | 100.000% (94.444–100.000) | 21.806% (20.000–23.194) | +| 10% | 3 / 29 | 629 | 31 | 3.975% (3.698–4.114) | 0.000% | 100.000% (95.833–100.000) | 11.806% (11.389–14.861) | + +
+
+ +
+Navajo — nav-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 627 / 627 | 0 | 0 | n/a | n/a | n/a | 97.082% | +| 90% | 564 / 627 | 1,152 | 48 | 5.690% (5.628–6.813) | 0.310% (0.236–0.456) | 96.825% (96.774–100.000) | 87.537% (87.467–89.638) | +| 80% | 502 / 627 | 2,137 | 101 | 6.036% (5.573–6.495) | 0.440% (0.186–0.510) | 97.521% (95.200–99.194) | 79.469% (78.300–81.332) | +| 70% | 439 / 627 | 3,251 | 140 | 5.875% (5.652–6.360) | 0.384% (0.341–0.464) | 97.849% (94.681–98.370) | 70.469% (68.958–72.587) | +| 60% | 376 / 627 | 4,382 | 155 | 5.728% (5.527–6.041) | 0.367% (0.218–0.416) | 97.561% (95.600–99.190) | 61.426% (60.125–61.839) | +| 50% | 314 / 627 | 5,560 | 157 | 5.713% (5.586–5.735) | 0.265% (0.228–0.381) | 97.106% (95.820–98.387) | 51.617% (51.389–52.496) | +| 40% | 251 / 627 | 6,745 | 176 | 5.663% (5.579–5.713) | 0.235% (0.173–0.378) | 96.533% (96.247–99.198) | 42.283% (41.528–42.784) | +| 30% | 188 / 627 | 7,954 | 145 | 5.577% (5.529–5.746) | 0.213% (0.174–0.267) | 98.165% (97.025–99.315) | 32.967% (32.000–33.143) | +| 20% | 125 / 627 | 9,161 | 108 | 5.502% (5.392–5.672) | 0.127% (0.069–0.186) | 98.802% (97.809–99.800) | 22.737% (22.517–23.774) | +| 10% | 63 / 627 | 10,276 | 54 | 5.465% (5.378–5.563) | 0.041% (0.010–0.062) | 99.468% (98.404–99.823) | 13.939% (13.377–15.038) | + +
+
+ +
+Neapolitan — nap-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 40 / 40 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 36 / 40 | 150 | 0 | 69.677% (46.452–87.838) | 68.874% (45.033–87.500) | 100.000% | 96.860% (94.456–98.798) | +| 80% | 32 / 40 | 302 | 0 | 72.277% (65.563–90.541) | 71.525% (64.626–90.278) | 100.000% | 94.389% (93.053–98.130) | +| 70% | 28 / 40 | 450 | 0 | 72.406% (69.111–85.809) | 71.655% (68.265–85.421) | 100.000% | 91.650% (90.715–95.725) | +| 60% | 24 / 40 | 601 | 0 | 73.579% (65.455–81.940) | 72.852% (64.516–81.443) | 100.000% | 89.446% (86.039–92.786) | +| 50% | 20 / 40 | 753 | 0 | 72.112% (61.355–76.944) | 71.351% (60.300–76.309) | 100.000% (95.000–100.000) | 85.972% (80.561–88.510) | +| 40% | 16 / 40 | 901 | 0 | 71.207% (65.338–75.973) | 70.421% (64.391–75.429) | 100.000% (95.833–100.000) | 82.632% (79.092–85.571) | +| 30% | 12 / 40 | 1,051 | 0 | 69.933% (66.063–76.543) | 69.110% (65.132–76.000) | 100.000% (96.429–100.000) | 78.891% (76.219–83.500) | +| 20% | 8 / 40 | 1,201 | 0 | 73.439% (56.797–75.187) | 72.797% (55.613–74.679) | 96.875% (93.750–100.000) | 78.691% (65.397–80.094) | +| 10% | 4 / 40 | 1,349 | 0 | 72.128% (55.902–74.351) | 71.516% (54.691–73.724) | 97.222% (94.444–100.000) | 74.883% (60.321–76.887) | + +
+
+ +
+North Frisian — frr-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 51 / 51 | 0 | 0 | n/a | n/a | n/a | 95.538% | +| 90% | 46 / 51 | 37 | 3 | 29.730% (21.739–53.571) | 25.000% (14.286–47.826) | 80.000% (66.667–100.000) | 88.714% (86.089–92.126) | +| 80% | 41 / 51 | 71 | 7 | 31.818% (22.667–56.923) | 22.222% (13.636–50.909) | 88.889% (62.500–100.000) | 83.727% (80.315–88.976) | +| 70% | 36 / 51 | 111 | 10 | 30.357% (29.245–48.571) | 23.232% (17.582–41.111) | 92.857% (84.615–100.000) | 76.378% (74.803–82.415) | +| 60% | 31 / 51 | 142 | 10 | 32.394% (30.263–39.437) | 23.387% (21.053–31.707) | 94.737% (77.778–100.000) | 70.866% (68.766–72.966) | +| 50% | 26 / 51 | 186 | 12 | 32.447% (26.064–40.909) | 24.324% (15.854–32.895) | 91.667% (86.364–100.000) | 65.092% (60.105–69.029) | +| 40% | 20 / 51 | 229 | 9 | 33.188% (26.316–38.565) | 24.020% (15.736–31.088) | 93.548% (86.667–100.000) | 56.693% (52.756–61.155) | +| 30% | 15 / 51 | 264 | 8 | 33.333% (27.138–37.879) | 24.336% (19.313–28.384) | 91.429% (77.778–100.000) | 51.706% (46.719–54.331) | +| 20% | 10 / 51 | 300 | 6 | 29.934% (28.620–35.333) | 20.833% (18.359–25.769) | 92.500% (87.805–97.500) | 42.520% (41.732–47.769) | +| 10% | 5 / 51 | 340 | 3 | 23.599% (20.235–27.059) | 15.593% (8.475–18.056) | 91.304% (77.273–95.652) | 30.971% (27.822–34.383) | + +
+
+ +
+Northern Sami — sme-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 2,103 / 2,103 | 0 | 0 | n/a | n/a | n/a | 98.924% | +| 90% | 1,893 / 2,103 | 5,366 | 96 | 62.095% (56.824–63.831) | 60.992% (55.424–62.783) | 88.177% (86.275–91.584) | 95.248% (94.832–95.562) | +| 80% | 1,682 / 2,103 | 10,878 | 206 | 60.772% (59.449–62.776) | 59.723% (58.278–61.760) | 87.864% (86.861–88.725) | 91.212% (91.135–91.822) | +| 70% | 1,472 / 2,103 | 16,554 | 240 | 59.450% (59.165–61.858) | 58.241% (58.093–60.868) | 87.379% (86.795–89.756) | 87.190% (86.708–87.508) | +| 60% | 1,262 / 2,103 | 22,029 | 296 | 58.137% (56.674–60.620) | 56.985% (55.400–59.630) | 88.043% (86.061–88.429) | 82.496% (82.102–83.281) | +| 50% | 1,052 / 2,103 | 27,761 | 290 | 57.337% (56.084–58.632) | 56.116% (54.859–57.504) | 87.838% (86.847–88.406) | 77.949% (77.318–78.309) | +| 40% | 841 / 2,103 | 33,181 | 254 | 55.050% (53.662–55.707) | 53.884% (52.362–54.500) | 86.597% (84.603–86.960) | 72.506% (71.347–72.835) | +| 30% | 631 / 2,103 | 38,585 | 255 | 52.142% (50.575–52.662) | 50.800% (49.298–51.367) | 85.332% (82.841–86.342) | 65.539% (65.048–66.515) | +| 20% | 421 / 2,103 | 43,995 | 213 | 48.326% (46.637–49.281) | 46.944% (45.145–47.929) | 83.962% (80.419–84.661) | 58.299% (56.852–58.462) | +| 10% | 210 / 2,103 | 49,599 | 117 | 38.261% (37.210–41.489) | 36.509% (35.569–40.044) | 78.674% (73.181–82.160) | 44.500% (43.291–47.088) | + +
+
+ +
+Norwegian Bokmål — nb-no-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 17,929 / 17,929 | 0 | 0 | n/a | n/a | n/a | 96.852% | +| 90% | 16,136 / 17,929 | 8,682 | 459 | 79.616% (79.365–80.300) | 70.546% (70.229–71.560) | 95.510% (95.252–96.117) | 95.003% (94.958–95.075) | +| 80% | 14,343 / 17,929 | 17,357 | 841 | 79.282% (78.391–79.483) | 69.935% (68.919–70.124) | 95.430% (94.759–95.606) | 93.090% (92.952–93.176) | +| 70% | 12,550 / 17,929 | 26,235 | 1,056 | 78.616% (78.247–79.117) | 69.080% (68.975–69.623) | 95.083% (94.075–95.571) | 91.051% (91.022–91.248) | +| 60% | 10,757 / 17,929 | 35,053 | 1,222 | 77.888% (77.692–78.230) | 68.373% (67.934–68.693) | 94.544% (94.086–94.654) | 88.955% (88.860–89.131) | +| 50% | 8,965 / 17,929 | 44,080 | 1,264 | 77.213% (76.853–77.697) | 67.372% (67.269–68.071) | 94.168% (93.324–94.348) | 86.679% (86.578–86.965) | +| 40% | 7,172 / 17,929 | 53,261 | 1,266 | 76.064% (75.979–76.622) | 66.133% (65.907–66.950) | 93.222% (92.859–93.532) | 84.095% (83.980–84.407) | +| 30% | 5,379 / 17,929 | 62,496 | 1,103 | 74.899% (74.255–75.323) | 64.655% (63.730–65.508) | 92.387% (92.185–92.553) | 81.227% (80.765–81.536) | +| 20% | 3,586 / 17,929 | 71,829 | 816 | 72.851% (72.564–73.423) | 62.099% (61.353–62.757) | 91.669% (91.337–91.848) | 77.536% (77.244–77.940) | +| 10% | 1,793 / 17,929 | 81,276 | 458 | 69.855% (69.368–70.692) | 58.598% (57.048–59.496) | 89.971% (89.224–90.585) | 72.437% (72.066–73.258) | + +
+
+ +
+Norwegian Nynorsk — nn-no-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 4,688 / 4,688 | 0 | 0 | n/a | n/a | n/a | 93.089% | +| 90% | 4,219 / 4,688 | 1,714 | 230 | 78.608% (77.071–80.395) | 70.338% (68.194–72.339) | 95.153% (94.473–97.054) | 91.278% (91.145–91.385) | +| 80% | 3,750 / 4,688 | 3,536 | 415 | 77.555% (74.632–78.831) | 68.830% (64.861–70.289) | 95.308% (94.801–95.734) | 89.242% (88.866–89.563) | +| 70% | 3,282 / 4,688 | 5,324 | 569 | 76.278% (73.779–76.665) | 67.482% (64.436–68.148) | 94.306% (93.322–94.907) | 87.054% (86.449–87.247) | +| 60% | 2,813 / 4,688 | 7,179 | 679 | 74.537% (72.582–75.521) | 65.383% (63.050–67.121) | 93.672% (92.678–95.267) | 84.601% (83.970–84.998) | +| 50% | 2,344 / 4,688 | 9,118 | 713 | 72.900% (71.989–74.007) | 62.808% (62.003–65.179) | 93.120% (92.681–94.000) | 81.986% (81.456–82.423) | +| 40% | 1,875 / 4,688 | 11,136 | 714 | 70.773% (70.483–71.887) | 60.799% (60.313–62.437) | 92.181% (91.901–92.710) | 78.826% (78.688–79.390) | +| 30% | 1,406 / 4,688 | 13,143 | 632 | 68.697% (68.490–69.926) | 58.151% (57.820–59.696) | 91.536% (91.227–91.890) | 75.436% (75.045–75.894) | +| 20% | 938 / 4,688 | 15,283 | 459 | 66.030% (65.878–67.801) | 55.037% (54.754–57.088) | 90.572% (89.668–91.189) | 71.121% (70.882–72.220) | +| 10% | 469 / 4,688 | 17,401 | 298 | 63.195% (60.378–64.426) | 51.191% (46.485–52.220) | 90.470% (89.673–91.396) | 66.017% (63.259–66.984) | + +
+
+ +
+Old English — ang-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 7,146 / 7,146 | 0 | 0 | n/a | n/a | n/a | 96.417% | +| 90% | 6,431 / 7,146 | 6,338 | 397 | 71.726% (71.108–72.675) | 69.192% (68.500–70.111) | 93.165% (92.411–94.239) | 93.823% (93.705–93.982) | +| 80% | 5,717 / 7,146 | 12,643 | 736 | 70.859% (70.075–71.520) | 68.234% (67.428–68.905) | 93.121% (92.697–93.426) | 91.026% (90.922–91.244) | +| 70% | 5,002 / 7,146 | 19,195 | 968 | 69.947% (69.610–70.711) | 67.258% (66.829–68.201) | 92.401% (91.925–92.794) | 88.200% (88.192–88.356) | +| 60% | 4,288 / 7,146 | 25,758 | 1,187 | 69.249% (68.853–69.679) | 66.496% (66.006–67.028) | 92.057% (91.584–92.640) | 85.211% (85.170–85.412) | +| 50% | 3,573 / 7,146 | 32,512 | 1,214 | 68.137% (67.024–68.767) | 65.406% (64.127–66.110) | 91.149% (90.475–91.327) | 81.902% (81.562–82.403) | +| 40% | 2,858 / 7,146 | 39,286 | 1,174 | 66.297% (65.430–67.131) | 63.492% (62.471–64.417) | 90.213% (89.488–90.647) | 77.996% (77.671–78.721) | +| 30% | 2,144 / 7,146 | 46,184 | 1,026 | 63.972% (63.769–65.295) | 61.002% (60.870–62.615) | 88.711% (87.932–89.145) | 73.499% (73.431–74.477) | +| 20% | 1,429 / 7,146 | 53,259 | 789 | 61.886% (60.677–62.167) | 58.922% (57.636–59.197) | 86.910% (86.072–87.259) | 68.685% (67.556–68.833) | +| 10% | 715 / 7,146 | 60,396 | 490 | 56.959% (55.121–57.888) | 53.692% (51.571–54.728) | 84.802% (83.819–85.219) | 60.821% (59.058–61.727) | + +
+
+ +
+Old French — fro-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,699 / 1,699 | 0 | 0 | n/a | n/a | n/a | 89.552% | +| 90% | 1,529 / 1,699 | 8,939 | 1,567 | 62.647% (61.241–69.134) | 61.934% (60.538–68.588) | 98.802% (97.024–99.412) | 86.459% (86.272–86.858) | +| 80% | 1,359 / 1,699 | 17,509 | 3,630 | 60.379% (59.208–64.509) | 59.639% (58.441–63.859) | 98.516% (97.619–99.410) | 83.326% (82.968–83.734) | +| 70% | 1,189 / 1,699 | 26,962 | 4,699 | 59.828% (56.031–63.092) | 59.118% (55.187–62.421) | 98.232% (96.844–98.622) | 79.891% (79.344–80.759) | +| 60% | 1,019 / 1,699 | 36,496 | 5,497 | 58.144% (56.789–60.305) | 57.415% (56.015–59.601) | 98.375% (97.194–98.817) | 76.563% (75.720–76.942) | +| 50% | 850 / 1,699 | 46,933 | 5,640 | 56.933% (55.066–58.912) | 56.175% (54.285–58.190) | 98.109% (97.041–99.052) | 73.089% (71.720–73.472) | +| 40% | 680 / 1,699 | 57,657 | 5,275 | 55.738% (54.333–57.653) | 54.971% (53.561–56.913) | 98.223% (97.635–98.912) | 69.036% (67.836–69.346) | +| 30% | 510 / 1,699 | 68,813 | 4,752 | 54.061% (51.720–56.253) | 53.283% (50.899–55.532) | 97.980% (96.967–98.730) | 63.876% (62.378–65.923) | +| 20% | 340 / 1,699 | 80,446 | 3,828 | 50.496% (48.800–53.073) | 49.689% (47.945–52.332) | 98.007% (96.174–98.817) | 57.647% (56.616–60.584) | +| 10% | 170 / 1,699 | 92,661 | 2,100 | 43.123% (41.897–48.241) | 42.195% (40.956–47.457) | 98.103% (95.095–98.168) | 47.642% (46.457–52.655) | + +
+
+ +
+Old High German — goh-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 482 / 482 | 0 | 0 | n/a | n/a | n/a | 99.377% | +| 90% | 434 / 482 | 548 | 8 | 43.163% (40.855–53.467) | 38.998% (36.499–49.400) | 91.667% (89.362–95.833) | 93.868% (92.849–94.566) | +| 80% | 386 / 482 | 1,161 | 20 | 48.079% (43.997–52.810) | 44.353% (39.875–49.467) | 91.667% (86.170–92.708) | 88.283% (86.528–90.189) | +| 70% | 337 / 482 | 1,658 | 13 | 47.587% (44.229–51.365) | 43.536% (40.100–47.311) | 90.278% (85.915–93.007) | 83.094% (81.264–84.642) | +| 60% | 289 / 482 | 2,220 | 15 | 45.991% (43.664–47.752) | 42.033% (39.426–44.179) | 89.062% (84.127–90.576) | 77.321% (75.774–78.774) | +| 50% | 241 / 482 | 2,758 | 10 | 46.240% (44.337–47.186) | 42.241% (40.243–43.062) | 88.235% (86.555–89.167) | 72.019% (70.264–73.057) | +| 40% | 193 / 482 | 3,264 | 11 | 46.067% (42.737–47.243) | 41.818% (38.396–43.265) | 88.153% (87.500–88.502) | 67.302% (63.906–68.094) | +| 30% | 145 / 482 | 3,773 | 9 | 44.719% (41.114–45.826) | 40.588% (36.563–41.739) | 87.240% (86.228–89.286) | 60.208% (58.094–62.906) | +| 20% | 96 / 482 | 4,265 | 8 | 43.312% (38.656–44.003) | 39.203% (33.865–39.589) | 87.013% (84.715–87.728) | 54.189% (50.434–55.679) | +| 10% | 48 / 482 | 4,764 | 3 | 39.216% (32.200–41.498) | 34.440% (26.628–37.096) | 85.714% (82.949–87.788) | 45.170% (38.962–47.094) | + +
+
+ +
+Old Irish — sga-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 49 / 49 | 0 | 0 | n/a | n/a | n/a | 99.786% | +| 90% | 44 / 49 | 89 | 0 | 6.173% (3.356–11.236) | 1.316% (0.000–7.143) | 80.000% | 91.346% (84.402–93.697) | +| 80% | 39 / 49 | 192 | 1 | 6.618% (5.189–8.854) | 1.554% (0.794–4.396) | 80.000% (80.000–90.000) | 81.090% (78.312–86.218) | +| 70% | 34 / 49 | 260 | 1 | 8.032% (6.050–9.554) | 2.991% (0.562–5.686) | 86.667% (80.000–93.333) | 74.359% (69.444–80.556) | +| 60% | 29 / 49 | 330 | 1 | 6.536% (6.140–7.309) | 1.423% (1.049–1.835) | 95.000% (85.000–100.000) | 66.987% (54.060–69.979) | +| 50% | 25 / 49 | 434 | 1 | 6.250% (5.069–6.849) | 1.336% (0.732–1.760) | 83.333% (79.167–95.833) | 56.090% (44.551–63.462) | +| 40% | 20 / 49 | 520 | 1 | 6.318% (4.944–8.269) | 1.590% (0.836–3.666) | 86.207% (79.310–89.655) | 48.825% (36.218–53.953) | +| 30% | 15 / 49 | 587 | 1 | 6.228% (5.095–8.589) | 1.670% (0.306–4.531) | 82.353% (70.588–97.059) | 41.987% (30.235–43.697) | +| 20% | 10 / 49 | 736 | 0 | 7.473% (5.263–8.431) | 3.443% (0.395–4.098) | 87.179% (69.231–100.000) | 27.244% (19.231–35.043) | +| 10% | 5 / 49 | 819 | 0 | 6.716% (4.612–8.036) | 1.935% (0.355–3.919) | 86.364% (77.273–90.909) | 18.269% (9.402–22.863) | + +
+
+ +
+Old Norse — non-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 2,666 / 2,666 | 0 | 0 | n/a | n/a | n/a | 92.899% | +| 90% | 2,399 / 2,666 | 4,348 | 637 | 46.596% (44.961–49.988) | 44.439% (42.738–47.839) | 89.450% (88.679–93.665) | 88.273% (88.019–88.698) | +| 80% | 2,133 / 2,666 | 8,876 | 1,179 | 45.910% (45.779–46.884) | 43.900% (43.617–44.722) | 87.991% (85.550–90.092) | 83.431% (83.248–83.893) | +| 70% | 1,866 / 2,666 | 13,327 | 1,498 | 44.786% (44.147–46.037) | 42.590% (41.780–43.909) | 87.068% (85.843–88.872) | 78.570% (78.273–79.195) | +| 60% | 1,600 / 2,666 | 18,126 | 1,734 | 43.529% (42.618–45.647) | 41.354% (40.298–43.612) | 85.998% (83.721–87.459) | 73.250% (73.057–74.345) | +| 50% | 1,333 / 2,666 | 23,125 | 1,846 | 42.483% (41.700–43.802) | 40.146% (39.393–41.689) | 84.820% (82.549–86.364) | 68.187% (67.085–68.802) | +| 40% | 1,066 / 2,666 | 28,118 | 1,815 | 41.538% (40.030–42.327) | 39.194% (37.653–40.129) | 84.397% (82.867–85.875) | 61.628% (61.346–62.850) | +| 30% | 800 / 2,666 | 33,516 | 1,598 | 40.345% (38.141–40.542) | 38.013% (35.756–38.168) | 82.751% (81.680–84.687) | 55.839% (55.158–56.627) | +| 20% | 533 / 2,666 | 38,752 | 1,220 | 38.020% (35.648–38.940) | 35.608% (33.167–36.554) | 82.289% (80.188–83.094) | 48.623% (47.589–50.274) | +| 10% | 267 / 2,666 | 44,071 | 703 | 34.624% (33.189–35.549) | 32.147% (30.466–33.161) | 80.128% (77.687–81.657) | 40.389% (39.702–41.976) | + +
+
+ +
+Old Saxon — osx-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 863 / 863 | 0 | 0 | n/a | n/a | n/a | 98.470% | +| 90% | 777 / 863 | 1,168 | 33 | 53.448% (46.839–53.853) | 50.220% (43.487–50.693) | 94.118% (91.139–97.590) | 93.930% (93.715–94.201) | +| 80% | 690 / 863 | 2,407 | 56 | 54.343% (47.071–56.514) | 51.569% (43.605–53.827) | 92.442% (91.765–95.152) | 89.405% (88.329–90.265) | +| 70% | 604 / 863 | 3,616 | 80 | 52.431% (49.034–55.515) | 49.493% (45.697–52.512) | 93.333% (90.980–94.902) | 84.601% (83.852–85.820) | +| 60% | 518 / 863 | 4,808 | 92 | 48.605% (45.812–53.078) | 45.007% (42.048–50.067) | 93.491% (92.647–96.481) | 78.985% (77.448–80.738) | +| 50% | 432 / 863 | 6,086 | 97 | 48.011% (45.939–52.136) | 44.683% (42.393–49.134) | 92.453% (91.962–94.588) | 73.927% (72.023–75.663) | +| 40% | 345 / 863 | 7,379 | 89 | 44.655% (42.126–46.890) | 41.008% (38.397–43.485) | 93.177% (90.607–94.347) | 66.566% (64.415–67.737) | +| 30% | 259 / 863 | 8,715 | 84 | 40.138% (38.413–41.793) | 36.122% (34.265–38.163) | 94.649% (90.970–95.826) | 57.628% (56.003–58.854) | +| 20% | 173 / 863 | 10,026 | 64 | 35.174% (31.506–36.436) | 30.773% (26.789–32.407) | 93.851% (90.497–96.070) | 47.758% (44.436–49.502) | +| 10% | 86 / 863 | 11,355 | 33 | 26.441% (23.697–30.817) | 21.522% (18.468–26.220) | 93.264% (91.495–95.484) | 33.124% (30.383–37.617) | + +
+
+ +
+Oodham — ood-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 368 / 368 | 0 | 0 | n/a | n/a | n/a | 98.387% | +| 90% | 331 / 368 | 109 | 1 | 34.862% (31.746–41.748) | 9.722% (8.989–12.500) | 91.667% (83.784–94.595) | 92.360% (91.087–93.294) | +| 80% | 294 / 368 | 231 | 4 | 35.169% (32.035–38.288) | 8.025% (6.329–12.865) | 87.838% (86.486–94.595) | 85.569% (84.975–86.757) | +| 70% | 258 / 368 | 338 | 6 | 36.152% (34.320–38.663) | 9.009% (6.987–11.966) | 91.743% (88.991–95.455) | 80.475% (79.711–80.900) | +| 60% | 221 / 368 | 454 | 9 | 35.759% (33.333–36.585) | 8.418% (7.792–11.148) | 89.726% (89.116–96.575) | 74.024% (71.986–74.448) | +| 50% | 184 / 368 | 576 | 8 | 36.458% (35.195–38.143) | 8.906% (7.084–13.810) | 93.478% (90.164–95.628) | 67.572% (66.129–69.015) | +| 40% | 147 / 368 | 705 | 8 | 36.018% (34.468–37.912) | 7.930% (6.834–13.976) | 93.665% (91.364–95.909) | 60.781% (59.508–63.328) | +| 30% | 110 / 368 | 811 | 6 | 36.059% (33.539–38.509) | 10.182% (7.052–12.409) | 91.440% (89.494–94.163) | 53.480% (52.971–56.876) | +| 20% | 74 / 368 | 936 | 4 | 35.543% (34.342–37.074) | 10.280% (8.466–11.493) | 92.177% (89.796–95.222) | 48.132% (46.095–51.019) | +| 10% | 37 / 368 | 1,047 | 3 | 36.381% (32.625–36.682) | 8.616% (5.816–11.549) | 92.727% (89.728–95.770) | 42.530% (39.898–43.803) | + +
+
+ +
+Pashto — ps-af-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 381 / 381 | 0 | 0 | n/a | n/a | n/a | 96.116% | +| 90% | 343 / 381 | 266 | 11 | 23.030% (21.053–34.300) | 12.915% (10.044–21.176) | 94.595% (89.189–97.368) | 89.262% (87.826–91.678) | +| 80% | 305 / 381 | 531 | 32 | 25.697% (22.242–27.637) | 13.828% (12.134–15.824) | 91.892% (86.667–95.946) | 83.453% (81.886–84.922) | +| 70% | 267 / 381 | 770 | 44 | 24.120% (22.934–28.344) | 12.960% (12.500–17.290) | 91.892% (91.071–93.805) | 77.121% (74.543–78.655) | +| 60% | 229 / 381 | 1,018 | 50 | 25.540% (23.182–27.473) | 13.924% (12.159–16.334) | 91.391% (90.667–96.667) | 71.377% (68.603–72.454) | +| 50% | 191 / 381 | 1,389 | 60 | 23.055% (21.855–25.714) | 11.750% (11.084–14.019) | 91.579% (89.362–95.213) | 61.097% (59.824–65.796) | +| 40% | 152 / 381 | 1,737 | 76 | 21.416% (20.849–25.967) | 11.074% (10.473–14.341) | 92.105% (89.381–93.833) | 51.860% (50.065–59.530) | +| 30% | 114 / 381 | 1,984 | 76 | 21.573% (19.838–25.892) | 10.529% (9.205–14.276) | 93.208% (91.353–93.585) | 45.855% (42.004–53.819) | +| 20% | 76 / 381 | 2,324 | 80 | 20.654% (19.205–21.550) | 10.045% (8.245–10.884) | 92.409% (90.789–94.408) | 36.521% (34.040–41.090) | +| 10% | 38 / 381 | 2,645 | 88 | 21.190% (18.601–31.194) | 10.807% (7.425–22.136) | 92.690% (88.889–94.737) | 31.070% (26.664–37.565) | + +
+
+ +
+Persian — fa-ir-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 69 / 69 | 0 | 0 | n/a | n/a | n/a | 95.836% | +| 90% | 62 / 69 | 360 | 29 | 18.699% (15.833–19.469) | 15.493% (13.006–16.000) | 100.000% (85.714–100.000) | 87.878% (87.745–88.594) | +| 80% | 55 / 69 | 743 | 29 | 15.979% (13.625–19.473) | 12.725% (10.400–16.031) | 100.000% (92.857–100.000) | 79.204% (79.072–81.247) | +| 70% | 48 / 69 | 1,054 | 87 | 17.998% (15.808–19.631) | 14.544% (12.546–16.211) | 100.000% (95.238–100.000) | 73.077% (71.645–73.899) | +| 60% | 41 / 69 | 1,474 | 87 | 17.843% (16.847–18.371) | 14.739% (13.572–15.175) | 100.000% (96.429–100.000) | 65.305% (63.740–65.517) | +| 50% | 35 / 69 | 1,792 | 88 | 17.796% (17.243–18.567) | 14.596% (13.979–15.398) | 100.000% (97.059–100.000) | 58.090% (57.560–59.019) | +| 40% | 28 / 69 | 2,185 | 87 | 18.078% (16.720–19.128) | 14.979% (13.479–15.918) | 100.000% (97.561–100.000) | 50.186% (49.310–51.114) | +| 30% | 21 / 69 | 2,604 | 58 | 17.696% (16.285–18.267) | 14.648% (13.104–15.153) | 100.000% (97.917–100.000) | 41.857% (40.981–42.706) | +| 20% | 14 / 69 | 2,978 | 30 | 18.105% (16.353–18.304) | 15.009% (13.145–15.164) | 100.000% (98.182–100.000) | 34.058% (33.130–34.536) | +| 10% | 7 / 69 | 3,367 | 29 | 17.987% (16.428–17.998) | 14.842% (13.213–14.887) | 100.000% | 25.995% (24.987–26.180) | + +
+
+ +
+Polish — pl-pl-unimorph 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 9,990 / 9,990 | 0 | 0 | n/a | n/a | n/a | 98.837% | +| 90% | 8,991 / 9,990 | 12,963 | 296 | 73.070% (71.833–74.408) | 69.972% (68.686–71.530) | 90.985% (89.948–91.264) | 96.179% (96.039–96.269) | +| 80% | 7,992 / 9,990 | 25,947 | 560 | 73.298% (71.542–73.560) | 70.344% (68.301–70.589) | 90.225% (89.430–90.594) | 93.619% (93.242–93.666) | +| 70% | 6,993 / 9,990 | 39,130 | 726 | 72.217% (71.567–72.865) | 69.231% (68.375–70.016) | 89.428% (88.990–89.729) | 90.660% (90.510–90.869) | +| 60% | 5,994 / 9,990 | 52,228 | 831 | 71.858% (71.219–72.488) | 68.868% (68.033–69.513) | 89.261% (88.574–89.356) | 87.814% (87.665–88.121) | +| 50% | 4,995 / 9,990 | 65,434 | 856 | 71.160% (70.975–71.928) | 68.066% (67.805–69.051) | 88.517% (88.330–89.078) | 84.847% (84.707–85.211) | +| 40% | 3,996 / 9,990 | 78,368 | 857 | 70.344% (69.790–71.025) | 67.165% (66.449–67.971) | 88.216% (88.039–88.588) | 81.587% (81.365–81.994) | +| 30% | 2,997 / 9,990 | 91,501 | 747 | 69.088% (68.299–70.063) | 65.677% (64.723–66.909) | 88.256% (87.960–88.753) | 78.020% (77.407–78.451) | +| 20% | 1,998 / 9,990 | 105,119 | 565 | 68.340% (67.352–68.619) | 64.876% (63.830–65.097) | 87.827% (87.200–88.611) | 74.371% (73.606–74.469) | +| 10% | 999 / 9,990 | 118,711 | 291 | 65.763% (65.395–67.001) | 61.996% (61.861–63.283) | 86.633% (85.310–87.965) | 69.126% (68.742–70.110) | + +
+
+ +
+Portuguese — pt-pt-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 4,001 / 4,001 | 0 | 0 | n/a | n/a | n/a | 99.815% | +| 90% | 3,601 / 4,001 | 21,519 | 66 | 94.620% (93.771–95.072) | 94.412% (93.530–94.882) | 100.000% (99.749–100.000) | 99.279% (99.194–99.326) | +| 80% | 3,201 / 4,001 | 43,021 | 113 | 94.273% (93.721–94.673) | 94.056% (93.478–94.472) | 100.000% (99.875–100.000) | 98.675% (98.576–98.762) | +| 70% | 2,801 / 4,001 | 64,495 | 146 | 93.982% (93.731–94.081) | 93.752% (93.489–93.852) | 100.000% (99.917–100.000) | 98.035% (97.952–98.049) | +| 60% | 2,401 / 4,001 | 85,970 | 216 | 93.987% (93.386–94.179) | 93.758% (93.130–93.957) | 99.938% (99.937–100.000) | 97.430% (97.204–97.509) | +| 50% | 2,001 / 4,001 | 107,492 | 243 | 93.601% (93.084–94.280) | 93.358% (92.822–94.061) | 99.900% (99.800–100.000) | 96.646% (96.404–97.001) | +| 40% | 1,600 / 4,001 | 129,092 | 242 | 93.154% (92.758–94.021) | 92.889% (92.480–93.795) | 99.958% (99.875–100.000) | 95.791% (95.526–96.281) | +| 30% | 1,200 / 4,001 | 150,645 | 201 | 92.584% (91.628–93.106) | 92.297% (91.306–92.854) | 99.893% (99.643–100.000) | 94.717% (94.035–95.078) | +| 20% | 800 / 4,001 | 172,258 | 154 | 91.775% (90.674–92.113) | 91.460% (90.315–91.809) | 99.938% (99.750–100.000) | 93.379% (92.481–93.616) | +| 10% | 400 / 4,001 | 193,881 | 56 | 90.423% (88.898–91.005) | 90.053% (88.476–90.666) | 99.833% (99.778–100.000) | 91.313% (89.988–91.878) | + +
+
+ +
+Quechua — que-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,003 / 1,003 | 0 | 0 | n/a | n/a | n/a | 98.439% | +| 90% | 903 / 1,003 | 14,159 | 418 | 87.614% (87.075–89.562) | 87.601% (87.049–89.541) | 90.244% (87.952–92.771) | 97.283% (97.249–97.538) | +| 80% | 802 / 1,003 | 29,049 | 731 | 88.048% (87.025–88.824) | 88.043% (87.005–88.829) | 90.270% (88.068–90.909) | 96.179% (95.964–96.469) | +| 70% | 702 / 1,003 | 44,103 | 969 | 87.543% (86.009–87.942) | 87.527% (85.992–87.940) | 88.760% (87.591–90.262) | 94.939% (94.595–95.065) | +| 60% | 602 / 1,003 | 58,894 | 1,202 | 87.450% (85.650–87.905) | 87.437% (85.628–87.895) | 89.532% (87.805–89.694) | 93.609% (93.174–93.850) | +| 50% | 502 / 1,003 | 73,322 | 1,194 | 86.869% (84.692–87.841) | 86.860% (84.666–87.839) | 88.428% (88.110–89.760) | 92.251% (91.467–92.953) | +| 40% | 401 / 1,003 | 89,682 | 1,101 | 85.822% (82.961–86.710) | 85.807% (82.930–86.707) | 88.142% (87.211–88.947) | 90.452% (88.877–91.030) | +| 30% | 301 / 1,003 | 104,007 | 992 | 86.333% (78.919–88.228) | 86.346% (78.859–88.236) | 87.988% (84.234–88.774) | 89.665% (84.398–91.179) | +| 20% | 201 / 1,003 | 119,570 | 751 | 83.805% (76.610–88.329) | 83.786% (76.516–88.349) | 85.219% (82.368–90.909) | 86.626% (80.757–90.082) | +| 10% | 100 / 1,003 | 135,287 | 379 | 78.591% (75.365–82.879) | 78.545% (75.321–82.877) | 85.459% (81.910–87.838) | 80.407% (77.818–84.307) | + +
+
+ +
+Romanian — ro-ro-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 4,387 / 4,387 | 0 | 0 | n/a | n/a | n/a | 99.175% | +| 90% | 3,948 / 4,387 | 4,742 | 70 | 74.659% (73.030–76.354) | 73.593% (71.508–75.100) | 88.235% (85.176–89.202) | 96.784% (96.488–96.925) | +| 80% | 3,510 / 4,387 | 9,657 | 127 | 74.645% (73.922–76.412) | 73.446% (72.512–75.392) | 87.044% (85.936–88.318) | 94.223% (94.123–94.517) | +| 70% | 3,071 / 4,387 | 14,679 | 170 | 74.310% (73.479–75.682) | 73.032% (72.291–74.557) | 87.412% (85.927–87.549) | 91.538% (91.303–91.923) | +| 60% | 2,632 / 4,387 | 19,466 | 195 | 73.800% (72.871–74.215) | 72.529% (71.488–72.936) | 87.002% (86.480–87.507) | 88.877% (88.685–89.015) | +| 50% | 2,194 / 4,387 | 24,311 | 207 | 72.983% (72.075–73.477) | 71.728% (70.710–72.145) | 86.034% (84.669–87.216) | 85.887% (85.552–86.142) | +| 40% | 1,755 / 4,387 | 29,095 | 194 | 71.893% (70.956–72.457) | 70.609% (69.717–71.169) | 85.218% (83.617–85.808) | 82.526% (82.260–82.938) | +| 30% | 1,316 / 4,387 | 34,201 | 176 | 70.530% (69.863–71.305) | 69.211% (68.443–69.923) | 84.309% (83.894–85.278) | 79.021% (78.448–79.720) | +| 20% | 877 / 4,387 | 39,061 | 139 | 68.598% (67.720–70.052) | 66.948% (66.061–68.629) | 84.587% (83.133–85.443) | 74.678% (73.710–75.830) | +| 10% | 439 / 4,387 | 44,011 | 84 | 65.973% (65.003–67.866) | 64.222% (63.190–66.537) | 83.189% (81.399–83.825) | 69.223% (68.270–71.010) | + +
+
+ +
+Russian — ru-ru-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 37,410 / 37,410 | 0 | 0 | n/a | n/a | n/a | 98.807% | +| 90% | 33,669 / 37,410 | 79,477 | 1,784 | 82.007% (81.842–82.574) | 80.744% (80.515–81.372) | 94.506% (94.455–94.967) | 97.047% (97.012–97.088) | +| 80% | 29,928 / 37,410 | 158,745 | 3,165 | 81.665% (81.300–82.243) | 80.343% (79.956–81.034) | 94.359% (93.988–94.456) | 95.273% (95.172–95.333) | +| 70% | 26,187 / 37,410 | 238,018 | 4,273 | 81.575% (81.414–81.952) | 80.296% (80.115–80.758) | 94.006% (93.750–94.173) | 93.468% (93.397–93.575) | +| 60% | 22,446 / 37,410 | 318,625 | 4,709 | 81.182% (80.809–81.611) | 79.915% (79.506–80.371) | 93.632% (93.525–93.734) | 91.517% (91.394–91.717) | +| 50% | 18,705 / 37,410 | 398,725 | 5,007 | 80.854% (80.304–81.219) | 79.578% (78.976–79.972) | 93.316% (93.219–93.447) | 89.614% (89.371–89.752) | +| 40% | 14,964 / 37,410 | 479,920 | 4,763 | 80.312% (79.666–80.706) | 79.019% (78.331–79.445) | 93.010% (92.649–93.088) | 87.513% (87.184–87.695) | +| 30% | 11,223 / 37,410 | 560,108 | 4,217 | 79.739% (79.241–79.963) | 78.423% (77.874–78.684) | 92.571% (92.488–92.673) | 85.305% (84.992–85.415) | +| 20% | 7,482 / 37,410 | 640,968 | 3,381 | 78.648% (78.391–79.002) | 77.254% (76.977–77.673) | 91.951% (91.925–92.274) | 82.544% (82.404–82.868) | +| 10% | 3,741 / 37,410 | 723,165 | 1,916 | 76.473% (76.268–77.001) | 74.948% (74.735–75.545) | 91.268% (90.883–91.995) | 78.669% (78.459–79.115) | + +
+
+ +
+Seneca — see-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 139 / 139 | 0 | 0 | n/a | n/a | n/a | 99.756% | +| 90% | 125 / 139 | 537 | 1 | 2.602% (2.403–2.612) | 0.000% | 100.000% | 89.946% (89.852–89.964) | +| 80% | 111 / 139 | 1,072 | 1 | 2.519% (2.505–2.634) | 0.000% | 100.000% (96.429–100.000) | 80.154% (80.060–80.341) | +| 70% | 97 / 139 | 1,604 | 11 | 2.556% (2.413–2.628) | 0.000% | 100.000% (92.857–100.000) | 70.437% (70.174–70.568) | +| 60% | 83 / 139 | 2,148 | 11 | 2.568% (2.466–2.607) | 0.000% | 100.000% (94.643–100.000) | 60.514% (60.364–60.758) | +| 50% | 70 / 139 | 2,650 | 11 | 2.585% (2.487–2.604) | 0.000% | 100.000% (95.652–100.000) | 51.341% (51.229–51.754) | +| 40% | 56 / 139 | 3,180 | 11 | 2.583% (2.513–2.618) | 0.000% | 100.000% (96.386–100.000) | 41.643% (41.268–41.887) | +| 30% | 42 / 139 | 3,718 | 11 | 2.588% (2.526–2.615) | 0.000% | 100.000% (96.907–100.000) | 31.814% (31.589–32.002) | +| 20% | 28 / 139 | 4,245 | 11 | 2.594% (2.526–2.600) | 0.000% | 100.000% (97.297–100.000) | 22.191% (21.797–22.435) | +| 10% | 14 / 139 | 4,789 | 1 | 2.608% (2.593–2.611) | 0.000% | 100.000% (99.200–100.000) | 12.418% (12.324–12.549) | + +
+
+ +
+Serbo-Croatian — hbs-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 24,385 / 24,385 | 0 | 0 | n/a | n/a | n/a | 98.061% | +| 90% | 21,947 / 24,385 | 26,814 | 932 | 64.884% (64.413–65.313) | 61.807% (61.429–62.316) | 95.286% (94.607–95.719) | 94.672% (94.586–94.780) | +| 80% | 19,508 / 24,385 | 53,548 | 1,684 | 64.092% (63.870–64.678) | 61.086% (60.771–61.689) | 94.741% (94.595–95.279) | 91.179% (91.162–91.366) | +| 70% | 17,070 / 24,385 | 80,657 | 2,232 | 63.722% (62.794–64.345) | 60.710% (59.603–61.356) | 94.584% (94.288–95.180) | 87.676% (87.453–87.940) | +| 60% | 14,631 / 24,385 | 108,349 | 2,570 | 63.351% (62.478–63.666) | 60.291% (59.297–60.632) | 94.594% (94.200–95.051) | 84.068% (83.723–84.309) | +| 50% | 12,193 / 24,385 | 135,615 | 2,778 | 62.642% (62.355–63.020) | 59.558% (59.269–59.959) | 94.199% (93.989–94.334) | 80.220% (80.098–80.582) | +| 40% | 9,754 / 24,385 | 163,679 | 2,657 | 61.776% (61.212–62.010) | 58.682% (58.066–58.880) | 94.064% (93.545–94.107) | 76.106% (75.884–76.389) | +| 30% | 7,316 / 24,385 | 192,170 | 2,289 | 60.705% (60.324–61.019) | 57.558% (57.074–57.877) | 93.449% (93.345–93.927) | 71.716% (71.578–72.056) | +| 20% | 4,877 / 24,385 | 220,464 | 1,800 | 59.329% (59.153–59.632) | 56.068% (55.892–56.408) | 93.044% (92.817–93.173) | 67.046% (66.947–67.197) | +| 10% | 2,439 / 24,385 | 248,808 | 1,084 | 57.218% (56.568–58.465) | 53.844% (53.138–55.190) | 92.409% (92.058–92.723) | 61.263% (60.766–62.413) | + +
+
+ +
+Shipibo-Conibo — shp-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 1,999 / 1,999 | 0 | 0 | n/a | n/a | n/a | 96.952% | +| 90% | 1,799 / 1,999 | 757 | 41 | 84.655% (83.884–85.411) | 81.206% (79.965–82.526) | 95.722% (93.717–97.895) | 95.547% (95.459–95.597) | +| 80% | 1,599 / 1,999 | 1,526 | 70 | 84.112% (83.083–84.712) | 80.617% (80.225–81.195) | 94.947% (91.799–95.789) | 94.029% (93.778–94.117) | +| 70% | 1,399 / 1,999 | 2,297 | 94 | 83.777% (83.123–84.491) | 80.264% (79.191–81.312) | 94.170% (91.930–95.288) | 92.423% (92.198–92.737) | +| 60% | 1,199 / 1,999 | 3,084 | 106 | 82.821% (82.754–83.722) | 79.699% (78.784–80.242) | 93.995% (92.298–94.961) | 90.655% (90.580–91.182) | +| 50% | 1,000 / 1,999 | 3,858 | 125 | 83.044% (82.271–83.589) | 79.736% (78.549–80.201) | 93.458% (92.340–93.821) | 89.200% (89.024–89.551) | +| 40% | 800 / 1,999 | 4,649 | 138 | 82.709% (82.111–82.970) | 79.503% (78.469–79.769) | 92.255% (90.987–93.035) | 87.506% (87.293–87.908) | +| 30% | 600 / 1,999 | 5,451 | 129 | 81.968% (81.563–83.087) | 78.568% (78.042–80.336) | 91.860% (91.256–92.347) | 85.775% (85.462–86.327) | +| 20% | 400 / 1,999 | 6,276 | 112 | 81.886% (80.003–83.270) | 78.450% (76.312–81.085) | 91.001% (89.784–92.094) | 84.295% (82.526–85.261) | +| 10% | 200 / 1,999 | 7,089 | 89 | 81.553% (79.770–82.254) | 78.067% (76.244–79.762) | 90.711% (87.787–91.895) | 82.514% (80.682–82.928) | + +
+
+ +
+Shona — sn-zw-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 86 / 86 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 77 / 86 | 279 | 0 | 3.226% (3.226–3.557) | 0.000% | 100.000% | 89.773% (89.773–90.758) | +| 80% | 69 / 86 | 527 | 0 | 3.226% (3.226–3.393) | 0.000% | 100.000% | 80.682% (80.682–81.667) | +| 70% | 60 / 86 | 806 | 0 | 3.226% (3.226–3.333) | 0.000% | 100.000% | 70.455% (70.455–71.439) | +| 60% | 52 / 86 | 1,054 | 0 | 3.226% (3.226–3.307) | 0.000% | 100.000% | 61.364% (61.364–62.348) | +| 50% | 43 / 86 | 1,307 | 0 | 3.290% (3.226–3.290) | 0.000% | 100.000% | 52.121% (51.136–52.121) | +| 40% | 34 / 86 | 1,586 | 0 | 3.279% (3.226–3.279) | 0.000% | 100.000% | 41.894% (40.909–41.894) | +| 30% | 26 / 86 | 1,834 | 0 | 3.272% (3.226–3.272) | 0.000% | 100.000% | 32.803% (31.818–32.803) | +| 20% | 17 / 86 | 2,113 | 0 | 3.265% (3.226–3.265) | 0.000% | 100.000% | 22.576% (21.591–22.576) | +| 10% | 9 / 86 | 2,361 | 0 | 3.261% (3.226–3.261) | 0.000% | 100.000% | 13.485% (12.500–13.485) |
-Danish — da-dk-default 1.0.0 +Sotho, Southern — st-za-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 4,179 / 4,179 | 0 | 0 | n/a | n/a | n/a | 99.371% | -| 90% | 3,761 / 4,179 | 3,232 | 38 | 74.192% (73.356–75.186) | 67.504% (66.790–69.750) | 92.718% (90.865–93.961) | 96.813% (96.698–96.897) | -| 80% | 3,343 / 4,179 | 6,419 | 59 | 74.015% (73.293–74.244) | 67.477% (67.434–67.862) | 92.377% (90.132–93.494) | 94.212% (94.063–94.358) | -| 70% | 2,925 / 4,179 | 9,622 | 72 | 72.901% (72.673–74.059) | 66.648% (65.699–67.713) | 91.066% (90.273–92.613) | 91.447% (91.276–91.726) | -| 60% | 2,507 / 4,179 | 12,858 | 89 | 72.406% (72.124–72.935) | 65.740% (65.579–67.061) | 90.565% (89.988–91.933) | 88.498% (88.442–88.641) | -| 50% | 2,090 / 4,179 | 16,078 | 86 | 71.433% (70.792–72.016) | 64.706% (63.871–65.934) | 90.561% (89.489–90.758) | 85.333% (85.042–85.612) | -| 40% | 1,672 / 4,179 | 19,316 | 87 | 70.449% (69.207–70.919) | 63.495% (61.895–64.640) | 90.106% (88.956–90.560) | 81.882% (81.244–82.196) | -| 30% | 1,254 / 4,179 | 22,543 | 70 | 68.316% (67.096–69.463) | 60.962% (59.342–62.755) | 89.335% (88.687–90.062) | 77.558% (76.727–78.444) | -| 20% | 836 / 4,179 | 25,755 | 61 | 65.725% (64.691–66.441) | 57.555% (56.445–58.672) | 89.232% (88.279–90.082) | 72.436% (71.615–73.034) | -| 10% | 418 / 4,179 | 29,022 | 28 | 61.104% (59.899–62.277) | 51.055% (49.995–53.218) | 88.238% (87.507–89.957) | 64.847% (63.836–65.997) | +| 100% | 26 / 26 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 23 / 26 | 48 | 0 | 12.500% | 6.667% | 100.000% | 89.904% | +| 80% | 21 / 26 | 80 | 0 | 12.500% | 6.667% | 100.000% | 83.173% | +| 70% | 18 / 26 | 128 | 0 | 12.500% | 6.667% | 100.000% | 73.077% | +| 60% | 16 / 26 | 160 | 0 | 12.500% | 6.667% | 100.000% | 66.346% | +| 50% | 13 / 26 | 208 | 0 | 12.500% | 6.667% | 100.000% | 56.250% | +| 40% | 10 / 26 | 256 | 0 | 12.500% | 6.667% | 100.000% | 46.154% | +| 30% | 8 / 26 | 288 | 0 | 12.500% | 6.667% | 100.000% | 39.423% | +| 20% | 5 / 26 | 336 | 0 | 12.500% | 6.667% | 100.000% | 29.327% | +| 10% | 3 / 26 | 368 | 0 | 12.500% | 6.667% | 100.000% | 22.596% |
-German — de-de-default 1.0.0 +Southern Kurdish — sdh-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 54,092 / 54,092 | 0 | 0 | n/a | n/a | n/a | 92.725% | -| 90% | 48,683 / 54,092 | 27,967 | 5,327 | 67.322% (66.336–67.716) | 59.108% (57.552–59.642) | 93.117% (92.600–93.345) | 89.908% (89.841–89.987) | -| 80% | 43,274 / 54,092 | 57,109 | 9,524 | 66.531% (65.981–66.706) | 58.016% (57.177–58.316) | 92.441% (91.771–92.581) | 87.066% (86.980–87.121) | -| 70% | 37,864 / 54,092 | 87,177 | 12,653 | 65.784% (65.442–66.138) | 56.948% (56.648–57.396) | 91.897% (91.743–91.984) | 84.105% (83.974–84.251) | -| 60% | 32,455 / 54,092 | 118,916 | 14,631 | 65.062% (64.946–65.321) | 56.115% (55.912–56.303) | 91.372% (91.222–91.619) | 81.048% (80.993–81.213) | -| 50% | 27,046 / 54,092 | 151,254 | 15,476 | 64.493% (64.083–64.603) | 55.371% (54.795–55.510) | 90.896% (90.661–91.008) | 78.014% (77.799–78.062) | -| 40% | 21,637 / 54,092 | 184,273 | 15,105 | 63.564% (63.219–63.981) | 54.206% (53.646–54.739) | 90.246% (89.885–90.446) | 74.680% (74.544–74.924) | -| 30% | 16,228 / 54,092 | 219,296 | 13,318 | 62.696% (62.282–63.101) | 53.083% (52.613–53.663) | 89.436% (88.996–89.500) | 71.309% (71.004–71.509) | -| 20% | 10,818 / 54,092 | 255,708 | 10,520 | 61.569% (60.724–61.785) | 51.959% (50.533–52.161) | 88.244% (88.017–88.723) | 67.504% (66.844–67.696) | -| 10% | 5,409 / 54,092 | 293,618 | 6,095 | 59.280% (58.219–59.702) | 49.432% (48.026–49.731) | 86.150% (85.836–86.733) | 62.456% (61.579–62.888) | +| 100% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 80% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 70% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 60% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 50% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 40% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 30% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 20% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 10% | 1 / 1 | 0 | 0 | n/a | n/a | n/a | 100.000% |
@@ -134,336 +2574,354 @@ evidence. At 100% there is no held-out set, so unseen metrics are `n/a`.
-Persian — fa-ir-default 1.0.0 +Swedish — sv-se-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 69 / 69 | 0 | 0 | n/a | n/a | n/a | 95.836% | -| 90% | 62 / 69 | 360 | 29 | 18.699% (15.833–19.469) | 15.493% (13.006–16.000) | 100.000% (85.714–100.000) | 87.878% (87.745–88.594) | -| 80% | 55 / 69 | 743 | 29 | 15.979% (13.625–19.473) | 12.725% (10.400–16.031) | 100.000% (92.857–100.000) | 79.204% (79.072–81.247) | -| 70% | 48 / 69 | 1,054 | 87 | 17.998% (15.808–19.631) | 14.544% (12.546–16.211) | 100.000% (95.238–100.000) | 73.077% (71.645–73.899) | -| 60% | 41 / 69 | 1,474 | 87 | 17.843% (16.847–18.371) | 14.739% (13.572–15.175) | 100.000% (96.429–100.000) | 65.305% (63.740–65.517) | -| 50% | 35 / 69 | 1,792 | 88 | 17.796% (17.243–18.567) | 14.596% (13.979–15.398) | 100.000% (97.059–100.000) | 58.090% (57.560–59.019) | -| 40% | 28 / 69 | 2,185 | 87 | 18.078% (16.720–19.128) | 14.979% (13.479–15.918) | 100.000% (97.561–100.000) | 50.186% (49.310–51.114) | -| 30% | 21 / 69 | 2,604 | 58 | 17.696% (16.285–18.267) | 14.648% (13.104–15.153) | 100.000% (97.917–100.000) | 41.857% (40.981–42.706) | -| 20% | 14 / 69 | 2,978 | 30 | 18.105% (16.353–18.304) | 15.009% (13.145–15.164) | 100.000% (98.182–100.000) | 34.058% (33.130–34.536) | -| 10% | 7 / 69 | 3,367 | 29 | 17.987% (16.428–17.998) | 14.842% (13.213–14.887) | 100.000% | 25.995% (24.987–26.180) | +| 100% | 12,371 / 12,371 | 0 | 0 | n/a | n/a | n/a | 96.713% | +| 90% | 11,134 / 12,371 | 10,386 | 675 | 82.188% (81.351–82.660) | 78.458% (77.818–79.022) | 94.926% (93.807–95.482) | 95.085% (95.025–95.156) | +| 80% | 9,897 / 12,371 | 20,899 | 1,144 | 81.612% (81.194–82.294) | 78.137% (77.386–78.776) | 94.577% (93.139–94.972) | 93.421% (93.317–93.570) | +| 70% | 8,660 / 12,371 | 31,613 | 1,500 | 81.378% (80.737–81.694) | 77.755% (77.158–78.370) | 93.450% (93.333–94.124) | 91.702% (91.610–91.808) | +| 60% | 7,423 / 12,371 | 42,503 | 1,726 | 80.584% (80.441–80.710) | 77.113% (76.939–77.166) | 93.114% (92.742–93.192) | 89.845% (89.812–89.898) | +| 50% | 6,186 / 12,371 | 53,431 | 1,736 | 79.553% (79.081–79.924) | 75.813% (75.376–76.278) | 92.204% (92.067–92.705) | 87.777% (87.484–87.889) | +| 40% | 4,948 / 12,371 | 64,540 | 1,677 | 78.689% (78.113–78.970) | 74.894% (74.347–75.197) | 91.778% (91.278–92.147) | 85.502% (85.283–85.636) | +| 30% | 3,711 / 12,371 | 75,916 | 1,458 | 77.249% (76.663–77.381) | 73.248% (72.517–73.345) | 91.335% (90.486–91.670) | 82.749% (82.497–82.863) | +| 20% | 2,474 / 12,371 | 87,316 | 1,095 | 75.205% (74.488–75.890) | 70.861% (70.211–72.027) | 89.668% (89.346–90.344) | 79.323% (78.735–79.852) | +| 10% | 1,237 / 12,371 | 98,793 | 636 | 72.010% (70.983–72.505) | 67.214% (66.144–68.191) | 87.789% (86.974–88.659) | 74.396% (73.426–74.785) |
-Finnish — fi-fi-default 1.0.0 +Tagalog — tl-ph-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 57,027 / 57,027 | 0 | 0 | n/a | n/a | n/a | 98.661% | -| 90% | 51,324 / 57,027 | 181,248 | 5,149 | 86.644% (86.280–86.823) | 86.177% (85.790–86.342) | 94.835% (94.730–95.762) | 97.363% (97.337–97.369) | -| 80% | 45,622 / 57,027 | 363,208 | 9,479 | 86.152% (86.055–86.432) | 85.686% (85.568–85.979) | 94.265% (94.183–94.726) | 95.981% (95.972–96.026) | -| 70% | 39,919 / 57,027 | 546,876 | 12,346 | 85.689% (85.586–85.804) | 85.219% (85.114–85.335) | 93.710% (93.575–93.808) | 94.531% (94.518–94.560) | -| 60% | 34,216 / 57,027 | 730,977 | 14,257 | 85.172% (85.110–85.293) | 84.708% (84.639–84.829) | 93.033% (92.895–93.427) | 92.999% (92.978–93.035) | -| 50% | 28,514 / 57,027 | 917,680 | 14,946 | 84.681% (84.520–84.723) | 84.232% (84.039–84.280) | 92.203% (92.123–92.566) | 91.382% (91.295–91.404) | -| 40% | 22,811 / 57,027 | 1,103,628 | 14,607 | 83.991% (83.783–84.165) | 83.556% (83.313–83.715) | 91.299% (91.199–91.628) | 89.582% (89.463–89.675) | -| 30% | 17,108 / 57,027 | 1,292,480 | 12,981 | 83.290% (82.866–83.366) | 82.848% (82.399–82.936) | 90.499% (90.312–90.537) | 87.658% (87.378–87.720) | -| 20% | 11,405 / 57,027 | 1,481,274 | 10,279 | 81.958% (81.525–81.970) | 81.481% (81.032–81.508) | 89.459% (89.355–89.678) | 85.129% (84.780–85.155) | -| 10% | 5,703 / 57,027 | 1,672,373 | 6,067 | 79.210% (78.547–79.594) | 78.644% (77.966–79.067) | 88.247% (87.860–88.496) | 81.064% (80.468–81.422) | +| 100% | 342 / 342 | 0 | 0 | n/a | n/a | n/a | 99.682% | +| 90% | 308 / 342 | 247 | 1 | 16.194% (14.340–17.121) | 2.817% (1.531–4.484) | 100.000% | 91.461% (90.786–92.017) | +| 80% | 274 / 342 | 501 | 3 | 15.496% (14.532–16.331) | 1.782% (1.683–3.037) | 100.000% | 82.963% (82.049–83.439) | +| 70% | 239 / 342 | 774 | 3 | 14.987% (14.861–15.668) | 1.937% (1.902–2.171) | 100.000% | 73.670% (72.955–75.099) | +| 60% | 205 / 342 | 1,021 | 2 | 15.452% (14.908–15.529) | 1.984% (1.810–2.466) | 100.000% | 65.330% (64.893–66.322) | +| 50% | 171 / 342 | 1,265 | 3 | 15.361% (14.571–15.494) | 2.204% (1.648–2.285) | 100.000% | 57.029% (55.560–57.506) | +| 40% | 137 / 342 | 1,519 | 2 | 15.076% (14.516–15.251) | 1.783% (1.487–1.828) | 100.000% | 48.610% (47.180–49.404) | +| 30% | 103 / 342 | 1,777 | 3 | 14.836% (14.270–15.006) | 1.607% (0.973–1.703) | 100.000% | 39.793% (39.118–40.905) | +| 20% | 68 / 342 | 2,013 | 2 | 14.655% (14.328–15.037) | 1.208% (0.914–1.817) | 100.000% | 31.652% (31.096–31.970) | +| 10% | 34 / 342 | 2,275 | 1 | 14.485% (13.846–15.040) | 1.210% (0.356–1.780) | 100.000% (99.675–100.000) | 22.796% (22.121–23.193) |
-French — fr-fr-default 1.0.0 +Temoaya Otomi — ote-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 59,240 / 59,240 | 0 | 0 | n/a | n/a | n/a | 94.831% | -| 90% | 53,316 / 59,240 | 41,750 | 5,161 | 83.469% (82.278–83.711) | 80.031% (78.595–80.412) | 96.664% (96.351–96.813) | 93.302% (93.201–93.351) | -| 80% | 47,392 / 59,240 | 85,315 | 9,281 | 82.721% (81.861–83.091) | 79.345% (78.141–79.693) | 96.074% (95.896–96.266) | 91.731% (91.568–91.794) | -| 70% | 41,468 / 59,240 | 130,120 | 12,345 | 82.101% (81.643–82.329) | 78.644% (78.015–78.839) | 95.431% (95.257–95.560) | 90.086% (89.942–90.135) | -| 60% | 35,544 / 59,240 | 175,326 | 14,017 | 81.224% (80.630–81.431) | 77.560% (76.779–77.867) | 94.813% (94.641–94.831) | 88.270% (88.228–88.450) | -| 50% | 29,620 / 59,240 | 222,533 | 14,808 | 80.344% (79.709–80.528) | 76.569% (75.778–76.888) | 94.006% (93.785–94.099) | 86.507% (86.336–86.609) | -| 40% | 23,696 / 59,240 | 270,536 | 14,411 | 79.255% (79.039–79.552) | 75.314% (75.088–75.702) | 93.386% (93.086–93.418) | 84.453% (84.319–84.752) | -| 30% | 17,772 / 59,240 | 318,984 | 12,728 | 78.161% (78.141–78.274) | 74.079% (73.976–74.276) | 92.537% (92.395–92.906) | 82.316% (82.244–82.378) | -| 20% | 11,848 / 59,240 | 369,337 | 9,783 | 76.769% (76.583–76.946) | 72.504% (72.339–72.772) | 91.463% (90.996–91.884) | 79.741% (79.646–79.873) | -| 10% | 5,924 / 59,240 | 420,578 | 5,742 | 74.276% (73.276–74.926) | 69.627% (68.400–70.528) | 90.013% (89.987–90.359) | 75.978% (75.101–76.554) | +| 100% | 2,026 / 2,026 | 0 | 0 | n/a | n/a | n/a | 95.875% | +| 90% | 1,823 / 2,026 | 329 | 26 | 61.515% (60.182–63.665) | 7.463% (5.344–14.179) | 96.907% (95.000–98.469) | 92.396% (92.200–92.649) | +| 80% | 1,621 / 2,026 | 668 | 49 | 60.546% (58.555–62.267) | 8.519% (7.609–10.887) | 96.134% (94.444–97.201) | 88.805% (88.187–89.198) | +| 70% | 1,418 / 2,026 | 1,014 | 63 | 59.724% (59.002–60.926) | 8.451% (7.294–11.083) | 96.441% (94.128–97.293) | 84.792% (84.400–85.241) | +| 60% | 1,216 / 2,026 | 1,361 | 73 | 59.442% (58.948–59.752) | 7.958% (7.570–9.725) | 96.207% (93.962–97.478) | 81.117% (80.752–81.397) | +| 50% | 1,013 / 2,026 | 1,709 | 80 | 58.919% (58.534–59.450) | 8.146% (6.863–9.843) | 95.762% (94.260–96.690) | 77.217% (76.880–77.722) | +| 40% | 810 / 2,026 | 2,076 | 70 | 58.402% (58.382–58.903) | 8.121% (7.859–10.162) | 95.409% (93.395–95.424) | 73.232% (72.952–73.485) | +| 30% | 608 / 2,026 | 2,441 | 57 | 57.918% (57.681–58.251) | 8.317% (7.157–10.634) | 94.872% (93.138–95.306) | 69.136% (68.771–69.557) | +| 20% | 405 / 2,026 | 2,805 | 43 | 57.553% (57.468–57.596) | 8.870% (7.305–10.067) | 93.540% (92.742–94.776) | 65.039% (64.955–65.208) | +| 10% | 203 / 2,026 | 3,179 | 26 | 57.049% (56.368–58.131) | 7.778% (6.250–9.824) | 93.985% (93.341–94.615) | 60.915% (60.269–61.925) |
-Hebrew — he-il-default 1.0.0 +Turkish — tr-tr-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 2,358 / 2,358 | 0 | 0 | n/a | n/a | n/a | 98.250% | -| 90% | 2,122 / 2,358 | 5,914 | 196 | 21.813% (21.558–23.195) | 16.362% (15.846–17.917) | 88.785% (87.416–93.636) | 90.688% (90.578–90.888) | -| 80% | 1,886 / 2,358 | 11,868 | 390 | 21.715% (19.936–22.602) | 16.441% (14.182–17.242) | 89.628% (87.585–91.818) | 83.007% (82.678–83.339) | -| 70% | 1,651 / 2,358 | 17,846 | 499 | 21.488% (20.548–21.734) | 16.059% (15.072–16.256) | 88.713% (88.166–89.569) | 75.314% (75.106–75.538) | -| 60% | 1,415 / 2,358 | 23,856 | 551 | 20.530% (20.267–21.311) | 15.043% (14.721–15.824) | 88.702% (87.417–89.456) | 67.302% (67.264–67.705) | -| 50% | 1,179 / 2,358 | 29,985 | 576 | 20.205% (19.833–20.807) | 14.533% (14.185–15.123) | 88.771% (87.642–90.375) | 59.531% (59.185–59.763) | -| 40% | 943 / 2,358 | 36,073 | 547 | 19.758% (19.628–20.304) | 14.068% (13.841–14.608) | 89.236% (88.497–90.062) | 51.381% (51.289–51.799) | -| 30% | 707 / 2,358 | 42,248 | 465 | 19.125% (18.459–19.270) | 13.345% (12.730–13.462) | 88.961% (87.481–90.603) | 43.063% (42.627–43.264) | -| 20% | 472 / 2,358 | 48,464 | 365 | 17.734% (17.573–18.304) | 11.846% (11.656–12.416) | 89.000% (88.647–89.233) | 34.046% (33.884–34.388) | -| 10% | 236 / 2,358 | 54,723 | 206 | 16.532% (15.398–18.499) | 10.423% (9.169–12.678) | 89.756% (88.191–91.158) | 24.765% (23.666–26.661) | +| 100% | 3,017 / 3,017 | 0 | 0 | n/a | n/a | n/a | 99.263% | +| 90% | 2,715 / 3,017 | 22,024 | 288 | 82.355% (79.663–85.071) | 82.279% (79.549–85.021) | 88.776% (88.014–92.177) | 97.526% (97.031–97.798) | +| 80% | 2,414 / 3,017 | 43,638 | 442 | 81.792% (81.129–82.694) | 81.709% (81.009–82.607) | 88.985% (88.055–89.932) | 95.804% (95.608–95.959) | +| 70% | 2,112 / 3,017 | 66,129 | 600 | 81.756% (81.048–83.798) | 81.660% (80.939–83.734) | 88.737% (87.585–91.023) | 93.949% (93.831–94.503) | +| 60% | 1,810 / 3,017 | 88,540 | 733 | 81.961% (81.480–83.611) | 81.884% (81.381–83.550) | 88.067% (87.026–90.678) | 92.279% (92.054–92.910) | +| 50% | 1,509 / 3,017 | 111,440 | 911 | 81.608% (79.858–83.287) | 81.531% (79.726–83.240) | 87.754% (86.752–89.744) | 90.244% (89.262–91.086) | +| 40% | 1,207 / 3,017 | 133,132 | 900 | 81.522% (79.947–82.252) | 81.444% (79.832–82.181) | 87.290% (87.073–88.373) | 88.411% (87.502–89.019) | +| 30% | 905 / 3,017 | 155,798 | 763 | 81.433% (79.698–81.551) | 81.355% (79.593–81.479) | 87.189% (84.405–87.542) | 86.724% (85.562–86.851) | +| 20% | 603 / 3,017 | 178,334 | 575 | 79.471% (77.942–80.778) | 79.417% (77.814–80.700) | 86.502% (83.375–87.385) | 83.443% (82.135–84.477) | +| 10% | 302 / 3,017 | 201,851 | 285 | 77.397% (74.873–79.731) | 77.296% (74.748–79.658) | 84.869% (83.617–85.031) | 79.456% (77.232–81.684) |
-Hungarian — hu-hu-default 1.0.0 +Ukrainian — uk-ua-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 19,406 / 19,406 | 0 | 0 | n/a | n/a | n/a | 99.222% | -| 90% | 17,465 / 19,406 | 92,253 | 1,360 | 92.212% (91.551–92.510) | 92.203% (91.476–92.460) | 93.446% (92.426–93.786) | 98.479% (98.407–98.493) | -| 80% | 15,525 / 19,406 | 184,401 | 2,417 | 91.771% (91.652–92.138) | 91.756% (91.609–92.123) | 92.536% (92.147–92.726) | 97.646% (97.626–97.712) | -| 70% | 13,584 / 19,406 | 277,343 | 3,300 | 91.756% (91.390–92.053) | 91.744% (91.350–92.058) | 91.940% (90.827–92.376) | 96.861% (96.757–96.947) | -| 60% | 11,644 / 19,406 | 370,552 | 3,836 | 91.156% (91.058–91.366) | 91.187% (91.074–91.418) | 90.673% (90.092–91.129) | 95.888% (95.829–95.941) | -| 50% | 9,703 / 19,406 | 463,693 | 3,952 | 90.729% (90.537–90.756) | 90.777% (90.567–90.810) | 89.530% (89.200–89.806) | 94.834% (94.749–94.863) | -| 40% | 7,762 / 19,406 | 557,829 | 3,717 | 90.002% (89.679–90.181) | 90.053% (89.707–90.254) | 88.776% (88.318–89.016) | 93.526% (93.390–93.683) | -| 30% | 5,822 / 19,406 | 651,802 | 3,207 | 89.047% (88.909–89.354) | 89.126% (89.006–89.437) | 87.178% (86.603–87.420) | 92.006% (91.882–92.204) | -| 20% | 3,881 / 19,406 | 746,292 | 2,401 | 88.280% (87.847–88.842) | 88.393% (87.931–88.968) | 85.865% (85.589–85.920) | 90.395% (90.054–90.844) | -| 10% | 1,941 / 19,406 | 840,551 | 1,399 | 86.761% (86.100–87.668) | 86.876% (86.177–87.841) | 84.086% (83.633–85.050) | 87.963% (87.354–88.786) | +| 100% | 1,493 / 1,493 | 0 | 0 | n/a | n/a | n/a | 99.307% | +| 90% | 1,344 / 1,493 | 1,576 | 12 | 63.233% (61.940–65.736) | 57.218% (55.192–59.594) | 90.476% (88.514–93.243) | 95.673% (95.418–95.876) | +| 80% | 1,194 / 1,493 | 3,166 | 34 | 62.236% (61.727–64.985) | 56.335% (54.703–59.359) | 88.475% (87.879–93.255) | 91.987% (91.580–92.514) | +| 70% | 1,045 / 1,493 | 4,675 | 36 | 62.139% (60.712–63.062) | 55.456% (54.248–56.768) | 89.367% (85.811–90.562) | 87.990% (87.526–88.575) | +| 60% | 896 / 1,493 | 6,256 | 53 | 61.416% (58.976–62.308) | 54.708% (52.041–55.903) | 90.211% (87.669–91.525) | 84.076% (83.332–84.393) | +| 50% | 747 / 1,493 | 7,838 | 50 | 60.214% (59.232–62.314) | 53.268% (51.925–55.648) | 90.135% (88.140–91.180) | 79.793% (79.138–80.587) | +| 40% | 597 / 1,493 | 9,392 | 66 | 58.933% (57.994–60.117) | 51.880% (50.380–53.150) | 89.013% (87.486–90.578) | 75.097% (74.309–75.599) | +| 30% | 448 / 1,493 | 10,951 | 60 | 58.222% (57.492–59.390) | 51.048% (50.564–52.388) | 87.380% (86.917–89.403) | 70.566% (69.918–71.240) | +| 20% | 299 / 1,493 | 12,528 | 27 | 55.281% (54.207–58.249) | 47.743% (46.925–51.631) | 87.400% (85.313–88.161) | 64.002% (63.220–66.404) | +| 10% | 149 / 1,493 | 14,144 | 20 | 50.982% (50.365–54.332) | 42.530% (42.187–47.222) | 85.464% (84.667–88.227) | 55.754% (55.227–58.849) |
-Italian — it-it-default 1.0.0 +Uyghur — ug-cn-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 10,009 / 10,009 | 0 | 0 | n/a | n/a | n/a | 99.056% | -| 90% | 9,008 / 10,009 | 33,156 | 558 | 93.965% (93.205–94.474) | 93.600% (92.783–94.134) | 99.900% (99.600–100.000) | 98.473% (98.385–98.520) | -| 80% | 8,007 / 10,009 | 66,376 | 1,008 | 93.594% (93.308–93.801) | 93.192% (92.886–93.418) | 99.900% (99.750–99.950) | 97.816% (97.782–97.848) | -| 70% | 7,006 / 10,009 | 99,459 | 1,285 | 93.065% (92.894–93.176) | 92.631% (92.450–92.747) | 99.867% (99.800–99.900) | 97.072% (97.043–97.115) | -| 60% | 6,005 / 10,009 | 132,730 | 1,556 | 92.510% (92.196–92.768) | 92.037% (91.712–92.312) | 99.863% (99.750–99.900) | 96.267% (96.099–96.348) | -| 50% | 5,005 / 10,009 | 166,665 | 1,637 | 91.882% (91.717–92.283) | 91.375% (91.195–91.801) | 99.840% (99.780–99.920) | 95.296% (95.190–95.476) | -| 40% | 4,004 / 10,009 | 199,950 | 1,479 | 91.606% (91.107–91.863) | 91.077% (90.547–91.356) | 99.842% (99.783–99.883) | 94.458% (94.135–94.521) | -| 30% | 3,003 / 10,009 | 234,119 | 1,370 | 90.846% (90.583–91.131) | 90.272% (89.989–90.573) | 99.879% (99.800–99.929) | 93.137% (92.971–93.395) | -| 20% | 2,002 / 10,009 | 268,626 | 1,035 | 89.611% (89.163–90.122) | 88.964% (88.479–89.500) | 99.888% (99.725–99.963) | 91.388% (91.005–91.839) | -| 10% | 1,001 / 10,009 | 303,036 | 513 | 87.447% (87.014–87.906) | 86.665% (86.198–87.161) | 99.794% (99.556–99.917) | 88.555% (88.194–88.979) | +| 100% | 85 / 85 | 0 | 0 | n/a | n/a | n/a | 99.516% | +| 90% | 77 / 85 | 600 | 0 | 58.586% (47.333–59.500) | 58.020% (46.622–58.953) | 100.000% | 95.593% (94.577–95.718) | +| 80% | 68 / 85 | 1,287 | 0 | 56.410% (48.796–63.907) | 55.827% (48.110–63.407) | 100.000% | 90.795% (89.217–92.499) | +| 70% | 60 / 85 | 1,887 | 6 | 52.358% (46.962–60.201) | 51.719% (46.253–59.667) | 100.000% | 85.857% (83.826–87.779) | +| 60% | 51 / 85 | 2,562 | 6 | 53.747% (47.541–57.426) | 53.125% (46.835–56.845) | 100.000% | 80.997% (78.512–82.716) | +| 50% | 43 / 85 | 3,143 | 31 | 53.293% (47.211–63.662) | 52.660% (46.494–63.173) | 100.000% | 76.574% (73.637–81.950) | +| 40% | 34 / 85 | 3,818 | 31 | 54.861% (38.240–60.907) | 54.248% (37.404–60.380) | 100.000% | 72.668% (62.666–76.465) | +| 30% | 26 / 85 | 4,418 | 25 | 53.128% (36.151–58.328) | 52.493% (35.286–57.766) | 100.000% | 67.292% (55.587–71.011) | +| 20% | 17 / 85 | 5,099 | 25 | 55.295% (27.123–56.631) | 54.693% (26.138–56.047) | 100.000% | 64.151% (41.538–65.260) | +| 10% | 9 / 85 | 5,718 | 6 | 45.628% (44.741–46.523) | 44.895% (43.998–45.804) | 100.000% | 51.321% (50.570–52.164) |
-Norwegian Bokmål — nb-no-default 1.0.0 +Uzbek — uz-uz-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 17,929 / 17,929 | 0 | 0 | n/a | n/a | n/a | 96.852% | -| 90% | 16,136 / 17,929 | 8,682 | 459 | 79.616% (79.365–80.300) | 70.546% (70.229–71.560) | 95.510% (95.252–96.117) | 95.003% (94.958–95.075) | -| 80% | 14,343 / 17,929 | 17,357 | 841 | 79.282% (78.391–79.483) | 69.935% (68.919–70.124) | 95.430% (94.759–95.606) | 93.090% (92.952–93.176) | -| 70% | 12,550 / 17,929 | 26,235 | 1,056 | 78.616% (78.247–79.117) | 69.080% (68.975–69.623) | 95.083% (94.075–95.571) | 91.051% (91.022–91.248) | -| 60% | 10,757 / 17,929 | 35,053 | 1,222 | 77.888% (77.692–78.230) | 68.373% (67.934–68.693) | 94.544% (94.086–94.654) | 88.955% (88.860–89.131) | -| 50% | 8,965 / 17,929 | 44,080 | 1,264 | 77.213% (76.853–77.697) | 67.372% (67.269–68.071) | 94.168% (93.324–94.348) | 86.679% (86.578–86.965) | -| 40% | 7,172 / 17,929 | 53,261 | 1,266 | 76.064% (75.979–76.622) | 66.133% (65.907–66.950) | 93.222% (92.859–93.532) | 84.095% (83.980–84.407) | -| 30% | 5,379 / 17,929 | 62,496 | 1,103 | 74.899% (74.255–75.323) | 64.655% (63.730–65.508) | 92.387% (92.185–92.553) | 81.227% (80.765–81.536) | -| 20% | 3,586 / 17,929 | 71,829 | 816 | 72.851% (72.564–73.423) | 62.099% (61.353–62.757) | 91.669% (91.337–91.848) | 77.536% (77.244–77.940) | -| 10% | 1,793 / 17,929 | 81,276 | 458 | 69.855% (69.368–70.692) | 58.598% (57.048–59.496) | 89.971% (89.224–90.585) | 72.437% (72.066–73.258) | +| 100% | 295 / 295 | 0 | 0 | n/a | n/a | n/a | 99.970% | +| 90% | 266 / 295 | 979 | 0 | 94.592% (90.693–95.403) | 94.427% (90.723–95.263) | 100.000% (89.655–100.000) | 99.431% (98.973–99.512) | +| 80% | 236 / 295 | 1,949 | 0 | 92.825% (90.808–94.144) | 92.760% (90.510–93.966) | 100.000% (94.915–100.000) | 98.526% (98.181–98.790) | +| 70% | 207 / 295 | 2,928 | 0 | 89.857% (84.879–93.959) | 89.577% (84.539–93.877) | 98.864% (96.591–100.000) | 96.971% (95.162–98.191) | +| 60% | 177 / 295 | 3,916 | 1 | 91.037% (86.785–94.061) | 90.758% (86.470–93.878) | 99.153% (97.458–100.000) | 96.422% (94.450–97.591) | +| 50% | 148 / 295 | 4,940 | 1 | 89.758% (88.350–92.571) | 89.548% (88.063–92.364) | 98.639% (96.599–99.320) | 94.816% (93.983–96.239) | +| 40% | 118 / 295 | 5,951 | 2 | 90.388% (88.432–93.508) | 90.145% (88.165–93.325) | 98.305% (97.175–99.435) | 94.186% (92.936–96.087) | +| 30% | 89 / 295 | 6,899 | 0 | 89.955% (86.577–93.185) | 89.691% (86.226–92.989) | 98.544% (98.058–99.515) | 92.925% (90.527–95.243) | +| 20% | 59 / 295 | 7,897 | 1 | 88.920% (81.786–91.118) | 88.631% (81.346–90.883) | 98.729% (96.186–99.576) | 91.096% (85.251–92.905) | +| 10% | 30 / 295 | 8,871 | 0 | 88.164% (82.260–90.524) | 87.869% (81.784–90.242) | 98.868% (97.736–100.000) | 89.327% (83.970–91.513) |
-Dutch — nl-nl-default 1.0.0 +Voro — vro-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 4,992 / 4,992 | 0 | 0 | n/a | n/a | n/a | 99.120% | -| 90% | 4,493 / 4,992 | 3,085 | 57 | 74.468% (73.250–76.272) | 65.891% (63.190–67.730) | 94.512% (92.371–95.697) | 96.574% (96.437–96.777) | -| 80% | 3,994 / 4,992 | 6,160 | 110 | 74.369% (74.091–75.340) | 64.865% (64.550–66.383) | 94.408% (92.901–95.806) | 94.083% (94.025–94.245) | -| 70% | 3,494 / 4,992 | 9,298 | 148 | 74.137% (73.135–74.845) | 64.553% (63.215–66.338) | 94.412% (93.155–95.287) | 91.461% (91.190–91.734) | -| 60% | 2,995 / 4,992 | 12,412 | 171 | 73.828% (73.295–73.872) | 64.368% (63.661–64.668) | 94.242% (93.471–95.070) | 88.817% (88.524–88.845) | -| 50% | 2,496 / 4,992 | 15,581 | 174 | 73.408% (73.027–73.660) | 63.952% (63.070–64.202) | 94.186% (93.539–94.678) | 86.068% (85.791–86.153) | -| 40% | 1,997 / 4,992 | 18,714 | 171 | 72.501% (71.803–73.110) | 62.655% (61.524–63.525) | 93.853% (93.540–94.029) | 82.969% (82.495–83.280) | -| 30% | 1,498 / 4,992 | 21,856 | 141 | 71.097% (70.602–72.004) | 61.026% (59.937–61.731) | 93.565% (92.739–94.166) | 79.432% (79.003–79.969) | -| 20% | 998 / 4,992 | 25,065 | 109 | 69.543% (68.997–70.471) | 58.908% (57.908–60.208) | 92.781% (92.430–93.088) | 75.447% (74.855–76.168) | -| 10% | 499 / 4,992 | 28,267 | 62 | 67.039% (65.411–67.577) | 55.524% (53.812–55.891) | 92.616% (90.386–92.866) | 70.231% (68.760–70.692) | +| 100% | 63 / 63 | 0 | 0 | n/a | n/a | n/a | 99.758% | +| 90% | 57 / 63 | 41 | 0 | 16.667% (11.111–23.256) | 2.564% (0.000–10.811) | 100.000% (66.667–100.000) | 91.768% (90.073–92.494) | +| 80% | 50 / 63 | 81 | 0 | 15.957% (14.444–18.519) | 3.077% (0.000–4.938) | 92.308% (76.923–100.000) | 83.777% (80.630–84.262) | +| 70% | 44 / 63 | 123 | 0 | 16.667% (15.200–17.778) | 2.105% (1.031–6.897) | 89.474% (84.211–94.737) | 75.061% (72.881–76.755) | +| 60% | 38 / 63 | 162 | 1 | 16.352% (15.476–18.519) | 2.985% (1.504–5.839) | 88.000% (88.000–96.000) | 67.554% (65.375–67.797) | +| 50% | 32 / 63 | 203 | 1 | 16.256% (15.897–16.432) | 2.326% (1.220–4.396) | 93.548% (87.097–96.774) | 58.596% (56.659–60.048) | +| 40% | 25 / 63 | 249 | 1 | 15.936% (13.765–16.929) | 2.778% (1.493–3.318) | 86.842% (76.316–97.368) | 48.910% (48.184–51.332) | +| 30% | 19 / 63 | 288 | 1 | 16.495% (13.732–16.897) | 2.128% (0.410–3.644) | 88.636% (81.818–97.727) | 40.920% (39.952–43.584) | +| 20% | 13 / 63 | 331 | 1 | 16.566% (14.154–16.928) | 1.859% (0.712–3.546) | 92.000% (84.000–98.000) | 32.688% (31.477–35.593) | +| 10% | 6 / 63 | 373 | 0 | 15.385% (13.793–17.158) | 1.250% (0.000–2.532) | 94.737% (87.719–98.246) | 23.971% (21.308–24.939) |
-Norwegian Nynorsk — nn-no-default 1.0.0 +Western Highland Chatino — ctp-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 4,688 / 4,688 | 0 | 0 | n/a | n/a | n/a | 93.089% | -| 90% | 4,219 / 4,688 | 1,714 | 230 | 78.608% (77.071–80.395) | 70.338% (68.194–72.339) | 95.153% (94.473–97.054) | 91.278% (91.145–91.385) | -| 80% | 3,750 / 4,688 | 3,536 | 415 | 77.555% (74.632–78.831) | 68.830% (64.861–70.289) | 95.308% (94.801–95.734) | 89.242% (88.866–89.563) | -| 70% | 3,282 / 4,688 | 5,324 | 569 | 76.278% (73.779–76.665) | 67.482% (64.436–68.148) | 94.306% (93.322–94.907) | 87.054% (86.449–87.247) | -| 60% | 2,813 / 4,688 | 7,179 | 679 | 74.537% (72.582–75.521) | 65.383% (63.050–67.121) | 93.672% (92.678–95.267) | 84.601% (83.970–84.998) | -| 50% | 2,344 / 4,688 | 9,118 | 713 | 72.900% (71.989–74.007) | 62.808% (62.003–65.179) | 93.120% (92.681–94.000) | 81.986% (81.456–82.423) | -| 40% | 1,875 / 4,688 | 11,136 | 714 | 70.773% (70.483–71.887) | 60.799% (60.313–62.437) | 92.181% (91.901–92.710) | 78.826% (78.688–79.390) | -| 30% | 1,406 / 4,688 | 13,143 | 632 | 68.697% (68.490–69.926) | 58.151% (57.820–59.696) | 91.536% (91.227–91.890) | 75.436% (75.045–75.894) | -| 20% | 938 / 4,688 | 15,283 | 459 | 66.030% (65.878–67.801) | 55.037% (54.754–57.088) | 90.572% (89.668–91.189) | 71.121% (70.882–72.220) | -| 10% | 469 / 4,688 | 17,401 | 298 | 63.195% (60.378–64.426) | 51.191% (46.485–52.220) | 90.470% (89.673–91.396) | 66.017% (63.259–66.984) | +| 100% | 220 / 220 | 0 | 0 | n/a | n/a | n/a | 95.830% | +| 90% | 198 / 220 | 252 | 18 | 9.865% (6.567–11.792) | 0.000% (0.000–1.667) | 100.000% | 87.343% (84.280–89.004) | +| 80% | 176 / 220 | 494 | 37 | 9.278% (7.513–10.230) | 0.227% (0.000–1.660) | 100.000% | 79.410% (75.498–80.037) | +| 70% | 154 / 220 | 791 | 44 | 8.492% (7.933–9.355) | 0.138% (0.000–1.103) | 100.000% | 69.557% (67.749–71.107) | +| 60% | 132 / 220 | 1,044 | 46 | 8.867% (8.246–9.387) | 0.443% (0.216–1.046) | 100.000% | 61.402% (57.897–63.321) | +| 50% | 110 / 220 | 1,297 | 52 | 8.999% (8.538–9.484) | 0.509% (0.366–0.927) | 100.000% | 53.579% (50.148–56.642) | +| 40% | 88 / 220 | 1,582 | 57 | 8.910% (8.661–9.471) | 0.602% (0.490–0.929) | 100.000% | 44.022% (42.177–46.494) | +| 30% | 66 / 220 | 1,830 | 49 | 9.244% (8.502–9.344) | 0.731% (0.347–1.014) | 100.000% | 36.421% (34.022–37.934) | +| 20% | 44 / 220 | 2,109 | 40 | 9.226% (8.772–9.247) | 0.840% (0.466–1.164) | 100.000% | 27.122% (26.347–29.336) | +| 10% | 22 / 220 | 2,412 | 19 | 9.000% (8.708–9.731) | 0.859% (0.687–1.669) | 100.000% | 18.413% (16.384–19.299) |
-Polish — pl-pl-unimorph 1.0.0 +Xibe — sjo-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 9,990 / 9,990 | 0 | 0 | n/a | n/a | n/a | 98.837% | -| 90% | 8,991 / 9,990 | 12,963 | 296 | 73.070% (71.833–74.408) | 69.972% (68.686–71.530) | 90.985% (89.948–91.264) | 96.179% (96.039–96.269) | -| 80% | 7,992 / 9,990 | 25,947 | 560 | 73.298% (71.542–73.560) | 70.344% (68.301–70.589) | 90.225% (89.430–90.594) | 93.619% (93.242–93.666) | -| 70% | 6,993 / 9,990 | 39,130 | 726 | 72.217% (71.567–72.865) | 69.231% (68.375–70.016) | 89.428% (88.990–89.729) | 90.660% (90.510–90.869) | -| 60% | 5,994 / 9,990 | 52,228 | 831 | 71.858% (71.219–72.488) | 68.868% (68.033–69.513) | 89.261% (88.574–89.356) | 87.814% (87.665–88.121) | -| 50% | 4,995 / 9,990 | 65,434 | 856 | 71.160% (70.975–71.928) | 68.066% (67.805–69.051) | 88.517% (88.330–89.078) | 84.847% (84.707–85.211) | -| 40% | 3,996 / 9,990 | 78,368 | 857 | 70.344% (69.790–71.025) | 67.165% (66.449–67.971) | 88.216% (88.039–88.588) | 81.587% (81.365–81.994) | -| 30% | 2,997 / 9,990 | 91,501 | 747 | 69.088% (68.299–70.063) | 65.677% (64.723–66.909) | 88.256% (87.960–88.753) | 78.020% (77.407–78.451) | -| 20% | 1,998 / 9,990 | 105,119 | 565 | 68.340% (67.352–68.619) | 64.876% (63.830–65.097) | 87.827% (87.200–88.611) | 74.371% (73.606–74.469) | -| 10% | 999 / 9,990 | 118,711 | 291 | 65.763% (65.395–67.001) | 61.996% (61.861–63.283) | 86.633% (85.310–87.965) | 69.126% (68.742–70.110) | +| 100% | 1,892 / 1,892 | 0 | 0 | n/a | n/a | n/a | 98.561% | +| 90% | 1,703 / 1,892 | 318 | 9 | 79.866% (73.030–81.290) | 52.679% (43.537–61.333) | 96.721% (96.237–98.396) | 96.528% (95.777–96.747) | +| 80% | 1,514 / 1,892 | 635 | 16 | 76.850% (73.293–79.276) | 49.412% (43.007–50.704) | 96.748% (95.957–97.838) | 93.963% (93.150–94.683) | +| 70% | 1,324 / 1,892 | 961 | 22 | 75.739% (74.298–78.906) | 47.393% (42.750–49.112) | 96.791% (95.528–97.138) | 91.398% (90.960–92.774) | +| 60% | 1,135 / 1,892 | 1,248 | 24 | 75.594% (75.492–78.565) | 48.055% (44.356–50.538) | 96.390% (93.307–96.770) | 89.240% (88.771–90.835) | +| 50% | 946 / 1,892 | 1,563 | 25 | 75.446% (74.344–76.933) | 46.114% (43.153–48.837) | 95.708% (94.984–96.245) | 86.612% (86.519–87.989) | +| 40% | 757 / 1,892 | 1,900 | 26 | 75.258% (73.895–75.592) | 45.600% (43.445–47.842) | 95.009% (94.831–96.515) | 84.360% (83.516–84.673) | +| 30% | 568 / 1,892 | 2,233 | 21 | 74.563% (73.753–76.312) | 46.630% (44.066–48.229) | 94.985% (94.136–95.814) | 81.514% (81.045–83.015) | +| 20% | 378 / 1,892 | 2,554 | 15 | 73.761% (72.122–75.020) | 43.824% (41.563–46.476) | 93.563% (93.422–96.021) | 78.449% (77.166–80.044) | +| 10% | 189 / 1,892 | 2,870 | 8 | 72.485% (71.533–75.079) | 42.002% (36.325–46.127) | 94.879% (91.995–96.351) | 74.820% (74.195–77.385) |
-Portuguese — pt-pt-default 1.0.0 +Xipinawa — xty-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 4,001 / 4,001 | 0 | 0 | n/a | n/a | n/a | 99.815% | -| 90% | 3,601 / 4,001 | 21,519 | 66 | 94.620% (93.771–95.072) | 94.412% (93.530–94.882) | 100.000% (99.749–100.000) | 99.279% (99.194–99.326) | -| 80% | 3,201 / 4,001 | 43,021 | 113 | 94.273% (93.721–94.673) | 94.056% (93.478–94.472) | 100.000% (99.875–100.000) | 98.675% (98.576–98.762) | -| 70% | 2,801 / 4,001 | 64,495 | 146 | 93.982% (93.731–94.081) | 93.752% (93.489–93.852) | 100.000% (99.917–100.000) | 98.035% (97.952–98.049) | -| 60% | 2,401 / 4,001 | 85,970 | 216 | 93.987% (93.386–94.179) | 93.758% (93.130–93.957) | 99.938% (99.937–100.000) | 97.430% (97.204–97.509) | -| 50% | 2,001 / 4,001 | 107,492 | 243 | 93.601% (93.084–94.280) | 93.358% (92.822–94.061) | 99.900% (99.800–100.000) | 96.646% (96.404–97.001) | -| 40% | 1,600 / 4,001 | 129,092 | 242 | 93.154% (92.758–94.021) | 92.889% (92.480–93.795) | 99.958% (99.875–100.000) | 95.791% (95.526–96.281) | -| 30% | 1,200 / 4,001 | 150,645 | 201 | 92.584% (91.628–93.106) | 92.297% (91.306–92.854) | 99.893% (99.643–100.000) | 94.717% (94.035–95.078) | -| 20% | 800 / 4,001 | 172,258 | 154 | 91.775% (90.674–92.113) | 91.460% (90.315–91.809) | 99.938% (99.750–100.000) | 93.379% (92.481–93.616) | -| 10% | 400 / 4,001 | 193,881 | 56 | 90.423% (88.898–91.005) | 90.053% (88.476–90.666) | 99.833% (99.778–100.000) | 91.313% (89.988–91.878) | +| 100% | 585 / 585 | 0 | 0 | n/a | n/a | n/a | 99.029% | +| 90% | 527 / 585 | 269 | 5 | 28.839% (25.455–32.584) | 12.440% (10.599–16.746) | 89.474% (81.034–91.228) | 92.197% (91.658–92.557) | +| 80% | 468 / 585 | 545 | 9 | 28.962% (28.336–30.882) | 12.500% (11.601–14.988) | 89.744% (88.889–94.690) | 85.257% (84.933–85.509) | +| 70% | 410 / 585 | 816 | 11 | 28.360% (27.406–28.571) | 11.076% (10.526–12.112) | 89.714% (88.506–94.152) | 78.029% (77.778–78.317) | +| 60% | 351 / 585 | 1,098 | 14 | 26.993% (26.648–27.778) | 10.115% (9.186–10.814) | 90.598% (88.793–93.913) | 70.262% (70.191–70.802) | +| 50% | 293 / 585 | 1,378 | 17 | 26.084% (25.752–26.763) | 9.065% (8.489–9.446) | 91.034% (88.660–92.734) | 62.855% (62.531–62.999) | +| 40% | 234 / 585 | 1,656 | 15 | 25.260% (24.880–25.604) | 7.573% (6.870–8.187) | 90.831% (90.000–92.286) | 54.908% (54.585–55.304) | +| 30% | 176 / 585 | 1,930 | 14 | 24.271% (23.575–25.538) | 6.415% (5.581–7.842) | 91.667% (90.441–92.647) | 46.926% (46.422–47.717) | +| 20% | 117 / 585 | 2,208 | 12 | 23.535% (22.703–24.784) | 5.422% (4.107–6.452) | 91.863% (90.558–92.934) | 38.583% (38.044–39.842) | +| 10% | 59 / 585 | 2,490 | 8 | 22.427% (21.526–23.387) | 3.406% (3.093–3.478) | 94.667% (89.333–97.524) | 30.133% (29.414–31.140) |
-Russian — ru-ru-default 1.0.0 +Yamana — ame-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 37,410 / 37,410 | 0 | 0 | n/a | n/a | n/a | 98.807% | -| 90% | 33,669 / 37,410 | 79,477 | 1,784 | 82.007% (81.842–82.574) | 80.744% (80.515–81.372) | 94.506% (94.455–94.967) | 97.047% (97.012–97.088) | -| 80% | 29,928 / 37,410 | 158,745 | 3,165 | 81.665% (81.300–82.243) | 80.343% (79.956–81.034) | 94.359% (93.988–94.456) | 95.273% (95.172–95.333) | -| 70% | 26,187 / 37,410 | 238,018 | 4,273 | 81.575% (81.414–81.952) | 80.296% (80.115–80.758) | 94.006% (93.750–94.173) | 93.468% (93.397–93.575) | -| 60% | 22,446 / 37,410 | 318,625 | 4,709 | 81.182% (80.809–81.611) | 79.915% (79.506–80.371) | 93.632% (93.525–93.734) | 91.517% (91.394–91.717) | -| 50% | 18,705 / 37,410 | 398,725 | 5,007 | 80.854% (80.304–81.219) | 79.578% (78.976–79.972) | 93.316% (93.219–93.447) | 89.614% (89.371–89.752) | -| 40% | 14,964 / 37,410 | 479,920 | 4,763 | 80.312% (79.666–80.706) | 79.019% (78.331–79.445) | 93.010% (92.649–93.088) | 87.513% (87.184–87.695) | -| 30% | 11,223 / 37,410 | 560,108 | 4,217 | 79.739% (79.241–79.963) | 78.423% (77.874–78.684) | 92.571% (92.488–92.673) | 85.305% (84.992–85.415) | -| 20% | 7,482 / 37,410 | 640,968 | 3,381 | 78.648% (78.391–79.002) | 77.254% (76.977–77.673) | 91.951% (91.925–92.274) | 82.544% (82.404–82.868) | -| 10% | 3,741 / 37,410 | 723,165 | 1,916 | 76.473% (76.268–77.001) | 74.948% (74.735–75.545) | 91.268% (90.883–91.995) | 78.669% (78.459–79.115) | +| 100% | 326 / 326 | 0 | 0 | n/a | n/a | n/a | 98.763% | +| 90% | 293 / 326 | 259 | 5 | 16.732% (16.484–17.899) | 5.000% (4.762–6.250) | 96.970% (96.970–100.000) | 90.705% (90.217–90.855) | +| 80% | 261 / 326 | 516 | 7 | 16.667% (16.176–17.485) | 4.922% (4.595–5.856) | 98.462% (96.923–100.000) | 82.759% (81.672–83.021) | +| 70% | 228 / 326 | 781 | 14 | 16.710% (15.507–17.562) | 5.362% (4.161–5.417) | 96.939% (95.918–100.000) | 74.475% (72.939–75.187) | +| 60% | 196 / 326 | 1,039 | 18 | 16.957% (15.508–16.977) | 4.949% (3.961–5.476) | 97.692% (97.692–99.231) | 66.754% (65.180–67.091) | +| 50% | 163 / 326 | 1,307 | 17 | 16.333% (15.264–16.718) | 4.720% (3.729–5.340) | 97.546% (96.933–99.387) | 58.283% (56.259–58.508) | +| 40% | 130 / 326 | 1,577 | 18 | 16.337% (15.255–16.489) | 4.566% (3.617–5.147) | 98.469% (96.939–98.980) | 49.213% (47.939–50.637) | +| 30% | 98 / 326 | 1,846 | 18 | 15.855% (14.855–16.502) | 4.483% (3.006–4.825) | 98.246% (95.614–99.561) | 40.855% (39.843–42.466) | +| 20% | 65 / 326 | 2,118 | 12 | 14.932% (13.651–17.117) | 3.507% (2.716–5.628) | 97.318% (92.337–98.467) | 31.897% (30.210–34.183) | +| 10% | 33 / 326 | 2,398 | 6 | 14.807% (14.012–16.548) | 3.872% (2.670–5.048) | 98.976% (93.857–99.317) | 23.126% (22.151–25.037) |
-Swedish — sv-se-default 1.0.0 +Yiddish — yi-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 12,371 / 12,371 | 0 | 0 | n/a | n/a | n/a | 96.713% | -| 90% | 11,134 / 12,371 | 10,386 | 675 | 82.188% (81.351–82.660) | 78.458% (77.818–79.022) | 94.926% (93.807–95.482) | 95.085% (95.025–95.156) | -| 80% | 9,897 / 12,371 | 20,899 | 1,144 | 81.612% (81.194–82.294) | 78.137% (77.386–78.776) | 94.577% (93.139–94.972) | 93.421% (93.317–93.570) | -| 70% | 8,660 / 12,371 | 31,613 | 1,500 | 81.378% (80.737–81.694) | 77.755% (77.158–78.370) | 93.450% (93.333–94.124) | 91.702% (91.610–91.808) | -| 60% | 7,423 / 12,371 | 42,503 | 1,726 | 80.584% (80.441–80.710) | 77.113% (76.939–77.166) | 93.114% (92.742–93.192) | 89.845% (89.812–89.898) | -| 50% | 6,186 / 12,371 | 53,431 | 1,736 | 79.553% (79.081–79.924) | 75.813% (75.376–76.278) | 92.204% (92.067–92.705) | 87.777% (87.484–87.889) | -| 40% | 4,948 / 12,371 | 64,540 | 1,677 | 78.689% (78.113–78.970) | 74.894% (74.347–75.197) | 91.778% (91.278–92.147) | 85.502% (85.283–85.636) | -| 30% | 3,711 / 12,371 | 75,916 | 1,458 | 77.249% (76.663–77.381) | 73.248% (72.517–73.345) | 91.335% (90.486–91.670) | 82.749% (82.497–82.863) | -| 20% | 2,474 / 12,371 | 87,316 | 1,095 | 75.205% (74.488–75.890) | 70.861% (70.211–72.027) | 89.668% (89.346–90.344) | 79.323% (78.735–79.852) | -| 10% | 1,237 / 12,371 | 98,793 | 636 | 72.010% (70.983–72.505) | 67.214% (66.144–68.191) | 87.789% (86.974–88.659) | 74.396% (73.426–74.785) | +| 100% | 802 / 802 | 0 | 0 | n/a | n/a | n/a | 98.698% | +| 90% | 722 / 802 | 413 | 12 | 60.291% (58.962–66.986) | 45.318% (42.007–51.673) | 93.007% (87.500–94.631) | 94.907% (94.651–95.558) | +| 80% | 642 / 802 | 823 | 24 | 60.402% (59.246–64.234) | 45.180% (42.938–49.067) | 90.508% (84.028–92.388) | 90.930% (90.907–91.767) | +| 70% | 561 / 802 | 1,258 | 26 | 61.924% (60.048–63.121) | 46.748% (43.499–48.335) | 90.562% (86.577–92.511) | 87.512% (87.070–87.860) | +| 60% | 481 / 802 | 1,687 | 34 | 59.833% (59.576–62.234) | 45.164% (42.091–46.740) | 89.933% (86.464–91.792) | 83.070% (82.767–83.930) | +| 50% | 401 / 802 | 2,115 | 38 | 60.189% (57.729–64.180) | 44.428% (39.314–49.554) | 90.483% (88.108–91.345) | 79.256% (77.953–81.326) | +| 40% | 321 / 802 | 2,542 | 34 | 60.386% (59.274–63.676) | 43.712% (43.647–48.674) | 88.840% (87.723–91.081) | 75.419% (74.884–77.488) | +| 30% | 241 / 802 | 2,983 | 26 | 57.138% (55.881–62.209) | 39.886% (37.598–46.865) | 88.752% (87.027–90.258) | 69.535% (68.535–72.930) | +| 20% | 160 / 802 | 3,423 | 19 | 56.968% (54.863–59.445) | 38.615% (36.885–43.362) | 88.588% (86.935–90.646) | 65.186% (63.488–67.140) | +| 10% | 80 / 802 | 3,853 | 14 | 56.133% (53.401–59.770) | 37.767% (32.786–44.588) | 89.466% (84.791–91.028) | 60.233% (57.884–63.419) |
-Ukrainian — uk-ua-default 1.0.0 +Zacatepec Chatino — zpv-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 1,493 / 1,493 | 0 | 0 | n/a | n/a | n/a | 99.307% | -| 90% | 1,344 / 1,493 | 1,576 | 12 | 63.233% (61.940–65.736) | 57.218% (55.192–59.594) | 90.476% (88.514–93.243) | 95.673% (95.418–95.876) | -| 80% | 1,194 / 1,493 | 3,166 | 34 | 62.236% (61.727–64.985) | 56.335% (54.703–59.359) | 88.475% (87.879–93.255) | 91.987% (91.580–92.514) | -| 70% | 1,045 / 1,493 | 4,675 | 36 | 62.139% (60.712–63.062) | 55.456% (54.248–56.768) | 89.367% (85.811–90.562) | 87.990% (87.526–88.575) | -| 60% | 896 / 1,493 | 6,256 | 53 | 61.416% (58.976–62.308) | 54.708% (52.041–55.903) | 90.211% (87.669–91.525) | 84.076% (83.332–84.393) | -| 50% | 747 / 1,493 | 7,838 | 50 | 60.214% (59.232–62.314) | 53.268% (51.925–55.648) | 90.135% (88.140–91.180) | 79.793% (79.138–80.587) | -| 40% | 597 / 1,493 | 9,392 | 66 | 58.933% (57.994–60.117) | 51.880% (50.380–53.150) | 89.013% (87.486–90.578) | 75.097% (74.309–75.599) | -| 30% | 448 / 1,493 | 10,951 | 60 | 58.222% (57.492–59.390) | 51.048% (50.564–52.388) | 87.380% (86.917–89.403) | 70.566% (69.918–71.240) | -| 20% | 299 / 1,493 | 12,528 | 27 | 55.281% (54.207–58.249) | 47.743% (46.925–51.631) | 87.400% (85.313–88.161) | 64.002% (63.220–66.404) | -| 10% | 149 / 1,493 | 14,144 | 20 | 50.982% (50.365–54.332) | 42.530% (42.187–47.222) | 85.464% (84.667–88.227) | 55.754% (55.227–58.849) | +| 100% | 204 / 204 | 0 | 0 | n/a | n/a | n/a | 99.272% | +| 90% | 184 / 204 | 79 | 2 | 25.000% (23.077–25.641) | 0.000% | 100.000% | 91.990% (91.990–92.233) | +| 80% | 163 / 204 | 164 | 2 | 24.845% (23.171–25.625) | 0.000% | 100.000% (97.436–100.000) | 84.345% (83.981–84.830) | +| 70% | 143 / 204 | 243 | 3 | 24.797% (23.108–25.417) | 0.000% | 100.000% (96.667–100.000) | 77.184% (75.971–77.549) | +| 60% | 122 / 204 | 328 | 4 | 24.847% (23.582–25.231) | 0.000% | 98.780% (97.531–100.000) | 69.175% (68.325–69.782) | +| 50% | 102 / 204 | 409 | 3 | 24.450% (23.798–24.938) | 0.000% | 98.039% (98.020–99.020) | 61.893% (60.922–62.500) | +| 40% | 82 / 204 | 489 | 3 | 24.089% (23.313–24.846) | 0.000% | 98.347% (94.215–99.180) | 54.005% (53.762–54.976) | +| 30% | 61 / 204 | 576 | 3 | 24.394% (23.438–24.956) | 0.000% | 98.601% (95.070–100.000) | 46.602% (45.995–47.451) | +| 20% | 41 / 204 | 657 | 2 | 24.468% (23.860–24.924) | 0.000% | 98.773% (96.319–100.000) | 39.442% (38.956–40.170) | +| 10% | 20 / 204 | 741 | 0 | 24.462% (24.324–24.865) | 0.000% | 98.913% (97.826–100.000) | 31.796% (31.553–32.524) |
-English — us-uk-default 1.0.1 +Zarma — dje-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 396,939 / 396,939 | 0 | 0 | n/a | n/a | n/a | 97.668% | -| 90% | 357,245 / 396,939 | 96,702 | 3,746 | 94.680% (94.609–94.847) | 78.854% (78.538–79.603) | 98.744% (98.642–98.790) | 97.241% (97.237–97.256) | -| 80% | 317,551 / 396,939 | 193,781 | 6,659 | 94.608% (94.514–94.637) | 78.795% (78.580–79.033) | 98.592% (98.577–98.658) | 96.818% (96.816–96.831) | -| 70% | 277,857 / 396,939 | 291,831 | 8,726 | 94.425% (94.290–94.438) | 78.505% (77.971–78.614) | 98.454% (98.436–98.500) | 96.386% (96.367–96.394) | -| 60% | 238,163 / 396,939 | 390,750 | 10,090 | 94.178% (94.121–94.242) | 77.976% (77.824–78.311) | 98.314% (98.268–98.319) | 95.919% (95.916–95.953) | -| 50% | 198,470 / 396,939 | 490,475 | 10,590 | 93.934% (93.888–94.011) | 77.643% (77.410–77.726) | 98.133% (98.089–98.192) | 95.434% (95.421–95.484) | -| 40% | 158,776 / 396,939 | 591,318 | 10,198 | 93.699% (93.682–93.712) | 77.308% (77.149–77.479) | 97.940% (97.912–97.948) | 94.946% (94.916–94.960) | -| 30% | 119,082 / 396,939 | 692,940 | 8,930 | 93.440% (93.409–93.459) | 76.944% (76.715–76.961) | 97.741% (97.718–97.755) | 94.413% (94.372–94.433) | -| 20% | 79,388 / 396,939 | 795,287 | 6,816 | 93.155% (93.122–93.220) | 76.608% (76.306–76.669) | 97.509% (97.464–97.541) | 93.824% (93.807–93.898) | -| 10% | 39,694 / 396,939 | 898,572 | 3,741 | 92.805% (92.757–92.843) | 76.010% (75.505–76.378) | 97.184% (97.156–97.293) | 93.157% (93.123–93.211) | +| 100% | 27 / 27 | 0 | 0 | n/a | n/a | n/a | 100.000% | +| 90% | 24 / 27 | 9 | 0 | 33.333% | 0.000% | 100.000% | 92.593% | +| 80% | 22 / 27 | 15 | 0 | 33.333% | 0.000% | 100.000% | 87.654% | +| 70% | 19 / 27 | 24 | 0 | 33.333% | 0.000% | 100.000% | 80.247% | +| 60% | 16 / 27 | 33 | 0 | 33.333% | 0.000% | 100.000% | 72.840% | +| 50% | 14 / 27 | 39 | 0 | 33.333% | 0.000% | 100.000% | 67.901% | +| 40% | 11 / 27 | 48 | 0 | 33.333% | 0.000% | 100.000% | 60.494% | +| 30% | 8 / 27 | 57 | 0 | 33.333% | 0.000% | 100.000% | 53.086% | +| 20% | 5 / 27 | 66 | 0 | 33.333% | 0.000% | 100.000% | 45.679% | +| 10% | 3 / 27 | 72 | 0 | 33.333% | 0.000% | 100.000% | 40.741% |
-Yiddish — yi-default 1.0.0 +Zenzontepec Chatino — czn-default 1.0.0
| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 802 / 802 | 0 | 0 | n/a | n/a | n/a | 98.698% | -| 90% | 722 / 802 | 413 | 12 | 60.291% (58.962–66.986) | 45.318% (42.007–51.673) | 93.007% (87.500–94.631) | 94.907% (94.651–95.558) | -| 80% | 642 / 802 | 823 | 24 | 60.402% (59.246–64.234) | 45.180% (42.938–49.067) | 90.508% (84.028–92.388) | 90.930% (90.907–91.767) | -| 70% | 561 / 802 | 1,258 | 26 | 61.924% (60.048–63.121) | 46.748% (43.499–48.335) | 90.562% (86.577–92.511) | 87.512% (87.070–87.860) | -| 60% | 481 / 802 | 1,687 | 34 | 59.833% (59.576–62.234) | 45.164% (42.091–46.740) | 89.933% (86.464–91.792) | 83.070% (82.767–83.930) | -| 50% | 401 / 802 | 2,115 | 38 | 60.189% (57.729–64.180) | 44.428% (39.314–49.554) | 90.483% (88.108–91.345) | 79.256% (77.953–81.326) | -| 40% | 321 / 802 | 2,542 | 34 | 60.386% (59.274–63.676) | 43.712% (43.647–48.674) | 88.840% (87.723–91.081) | 75.419% (74.884–77.488) | -| 30% | 241 / 802 | 2,983 | 26 | 57.138% (55.881–62.209) | 39.886% (37.598–46.865) | 88.752% (87.027–90.258) | 69.535% (68.535–72.930) | -| 20% | 160 / 802 | 3,423 | 19 | 56.968% (54.863–59.445) | 38.615% (36.885–43.362) | 88.588% (86.935–90.646) | 65.186% (63.488–67.140) | -| 10% | 80 / 802 | 3,853 | 14 | 56.133% (53.401–59.770) | 37.767% (32.786–44.588) | 89.466% (84.791–91.028) | 60.233% (57.884–63.419) | +| 100% | 386 / 386 | 0 | 0 | n/a | n/a | n/a | 97.337% | +| 90% | 347 / 386 | 185 | 9 | 20.000% (19.149–20.000) | 0.000% | 94.737% (94.737–97.368) | 89.661% (89.399–89.817) | +| 80% | 309 / 386 | 365 | 17 | 19.780% (19.452–20.596) | 0.000% | 96.000% (94.667–98.701) | 82.089% (81.723–82.298) | +| 70% | 270 / 386 | 553 | 21 | 20.253% (19.821–20.615) | 0.000% | 97.391% (95.690–99.130) | 74.465% (74.047–74.674) | +| 60% | 232 / 386 | 739 | 25 | 20.298% (20.134–20.353) | 0.000% | 98.039% (97.368–98.693) | 66.945% (66.632–67.206) | +| 50% | 193 / 386 | 931 | 26 | 20.364% (20.151–20.408) | 0.000% | 98.446% (97.906–98.958) | 59.164% (58.956–59.582) | +| 40% | 154 / 386 | 1,126 | 26 | 20.213% (20.125–20.321) | 0.000% | 98.701% (98.261–99.134) | 51.227% (51.175–51.593) | +| 30% | 116 / 386 | 1,318 | 23 | 20.243% (20.091–20.258) | 0.000% | 99.254% (98.513–99.627) | 43.655% (43.603–43.864) | +| 20% | 77 / 386 | 1,515 | 19 | 20.198% (19.947–20.305) | 0.000% | 99.673% (98.377–100.000) | 35.822% (35.718–35.979) | +| 10% | 39 / 386 | 1,710 | 9 | 20.175% (19.977–20.258) | 0.000% | 99.711% (98.844–100.000) | 28.094% (27.937–28.251) | + +
+
+ +
+Zulu — zu-za-default 1.0.0 +
+ +| Training | Selected / total rows | Median unseen occurrences | Overlap excluded | Unseen all exact | Unseen changed exact | Unseen root preserved | Whole all exact | +| ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +| 100% | 563 / 563 | 0 | 0 | n/a | n/a | n/a | 98.931% | +| 90% | 507 / 563 | 3,174 | 75 | 2.148% (2.079–2.192) | 0.417% (0.186–0.657) | 100.000% (98.182–100.000) | 89.436% (88.300–90.728) | +| 80% | 450 / 563 | 6,423 | 133 | 2.143% (2.114–2.191) | 0.441% (0.327–0.534) | 100.000% (99.083–100.000) | 79.810% (79.654–81.304) | +| 70% | 394 / 563 | 9,576 | 172 | 2.141% (2.062–2.200) | 0.424% (0.390–0.514) | 100.000% (98.780–100.000) | 70.398% (69.478–71.745) | +| 60% | 338 / 563 | 12,719 | 181 | 2.123% (2.112–2.171) | 0.408% (0.399–0.499) | 100.000% (99.095–100.000) | 61.089% (60.698–61.881) | +| 50% | 282 / 563 | 15,762 | 174 | 2.115% (2.049–2.155) | 0.384% (0.304–0.495) | 99.639% (99.632–100.000) | 52.099% (50.626–52.276) | +| 40% | 225 / 563 | 19,546 | 187 | 2.089% (2.047–2.159) | 0.379% (0.315–0.463) | 99.698% (99.398–100.000) | 40.939% (40.078–42.903) | +| 30% | 169 / 563 | 22,899 | 156 | 2.114% (2.055–2.162) | 0.387% (0.359–0.474) | 99.239% (98.966–100.000) | 30.861% (29.877–33.369) | +| 20% | 113 / 563 | 26,177 | 87 | 2.116% (2.021–2.137) | 0.406% (0.345–0.438) | 99.552% (98.876–99.555) | 21.271% (20.272–24.018) | +| 10% | 56 / 563 | 29,332 | 43 | 2.041% (1.957–2.090) | 0.320% (0.262–0.378) | 99.604% (99.208–99.803) | 11.951% (11.147–13.167) |
## Provenance -- Radixor/Java: `4.2.0-6-g84e57fb` -- Core source revision: `84e57fb27ae40913569e826858a8ebb07cf2ea01` -- Source state: `dirty` -- Generalization generator SHA-256: `126964f216a48d2164d136a490698301111aae7f050209ced21caf73f8c01fb8` -- Measured-source manifest: [`dictionary-generalization-sources.sha256`](data/dictionary-generalization-sources.sha256) +- Measurement provenance is recorded per row. The active snapshot deliberately combines the frozen 20-language archive with the separately measured 123-language continuation. +- Historical 20-language source manifest: [`dictionary-generalization-sources.sha256`](data/dictionary-generalization-sources.sha256) +- Continuation source manifest: [`dictionary-generalization-2026-09-11-sources.txt`](data/dictionary-generalization-2026-09-11-sources.txt) - Split protocol: `radixor-generalization-v1` - Splits per coverage level: 5 -- Authoritative raw counters: [`dictionary-generalization.csv`](data/dictionary-generalization.csv) -- CSV SHA-256: `e6479840b9307ae03bd0873e55f397811e975125d621a8b8716d4c1a166b3ff2` +- Authoritative raw counters: [`dictionary-generalization-2026-09-11.csv`](data/dictionary-generalization-2026-09-11.csv) +- CSV SHA-256: `187a9fe7b7bb292ac8d36b43784dc63a6e6e749942e86f20e7cb26364b7ea8d5` - Model artifact IDs, independent versions, and SHA-256 values are recorded on every raw row. - Runtime speed is intentionally excluded: speed does not establish generalization. The [English coverage deep dive](reference/english-coverage.md) retains its separately measured diff --git a/docs/benchmarks/index.md b/docs/benchmarks/index.md index 9a4eb65..93c2fbb 100644 --- a/docs/benchmarks/index.md +++ b/docs/benchmarks/index.md @@ -54,7 +54,7 @@ Each language page contains: - throughput metrics, - pairwise linguistic-quality metrics showing whether same-group forms share a stem and forms with no shared dictionary-group membership remain separated, -- a language-specific edit-cost and dictionary-knowledge evidence table, factor analysis, and conclusion, +- for the historical 20-language cohort only, the separately retained edit-cost and dictionary-knowledge analysis, - interpretation notes for the compared stemmers. Open [Language Benchmark Pages](languages/index.md) for the complete language list. @@ -63,8 +63,8 @@ Open [Language Benchmark Pages](languages/index.md) for the complete language li The English dictionary coverage benchmark shows the current contracted-trie operating curve. With the full English dictionary, Radixor reaches `97.668%` all-token exactness and `98.110%` -changed-token exactness at `97.9 ns/token`. Even with a deterministic 10% dictionary slice, it -keeps `93.057%` all-token exactness and `77.327%` changed-token exactness at `68.9 ns/token`. +changed-token exactness at `120.4 ns/token`. Even with a deterministic 10% dictionary slice, it +keeps `93.057%` all-token exactness and `77.327%` changed-token exactness at `71.6 ns/token`. Those figures should not be reduced to a single speed badge. The professional interpretation is a quality/speed envelope: the amount and quality of dictionary knowledge affect stemming precision, @@ -72,7 +72,7 @@ while contracted tries reduce lookup cost in uniform regions of the compiled gra ## Quality versus performance -Each language page keeps exact-root accuracy, JMH latency, and pairwise linguistic-quality results in separate tables. No undocumented scalar combines them. The 2026-08-25 language tables are generated from the current corpus/command report, exact-root and speed JMH reports, and pairwise-quality snapshot produced for this refresh. The Snowball 3.1.0 matrix includes direct Czech, Persian, and Polish stemmers; all published Java stemmers were measured in the same run. Readers should inspect the quality and speed dimensions side by side; no cross-language Pareto ranking is inferred from workloads with different dictionaries and token counts. +Each language page keeps exact-root accuracy, JMH latency, and pairwise linguistic-quality results in separate tables. No undocumented scalar combines them. The 2026-09-10 Java tables are generated from the published corpus/command, exact-root, and speed reports; the richer pairwise-quality sections on the original 20 pages remain bound to their separately frozen snapshot. The Snowball 3.1.0 rows appear only where the implementation exactly matches the model language. Readers should inspect quality and speed side by side; no cross-language Pareto ranking is inferred from workloads with different dictionaries and token counts. ### New Snowball 3.1.0 rows @@ -88,55 +88,301 @@ These rows describe exact agreement with each Radixor model dictionary and the m ## Pairwise Quality Findings -The validated snapshot is a broad multilingual comparison covering the complete 20-language Radixor default-model universe, with one benchmark page per language. The direct ranking below uses only deterministic `PRIMARY_OUTPUT` rows over identical per-language inputs. Candidate-aware rows are intentionally excluded from this claim. +The validated snapshot is a broad multilingual comparison covering the complete 143-language Radixor default-model universe, with one benchmark page per language. The direct ranking below uses only deterministic `PRIMARY_OUTPUT` rows over identical per-language inputs. Candidate-aware rows are intentionally excluded from this claim. !!! success "Evidence-based primary-output result" - Radixor achieved the highest balanced accuracy among the evaluated deterministic stemmers for every documented language in both `ALL_WORDS` and `LOWERCASE_GROUPS_ONLY`: **first place in all 40 evaluated language-mode matrices, with no exact first-place ties**. 38 matrices include at least one direct comparator; the two Hebrew modes report Radixor independently because no same-language adapter is configured. This statement is limited to the evaluated implementations, versions, dictionaries, adapters, and balanced-accuracy metric; it is not a universal claim about every stemming use case. + Radixor achieved the highest balanced accuracy among the evaluated deterministic stemmers for every documented language in both `ALL_WORDS` and `LOWERCASE_GROUPS_ONLY`: **first place in all 282 evaluated language-mode matrices, with no exact first-place ties**. 60 matrices include at least one direct comparator; the two Hebrew modes report Radixor independently because no same-language adapter is configured. This statement is limited to the evaluated implementations, versions, dictionaries, adapters, and balanced-accuracy metric; it is not a universal claim about every stemming use case. ### Per-language winner matrix | Language | Dictionary mode | Winner | Balanced accuracy | Runner-up | Difference | Exact tie | Deterministic stemmers | |---|---|---|---:|---|---:|---|---:| +|Adyghe (`ADY`)|ALL_WORDS|Radixor|0.999051|n/a|n/a|no|1| +|Adyghe (`ADY`)|LOWERCASE_GROUPS_ONLY|Radixor|0.999051|n/a|n/a|no|1| +|Afrikaans (`AF_ZA`)|ALL_WORDS|Radixor|0.982452|n/a|n/a|no|1| +|Afrikaans (`AF_ZA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.982452|n/a|n/a|no|1| +|Aimele (`AIL`)|ALL_WORDS|Radixor|0.965734|n/a|n/a|no|1| +|Aimele (`AIL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.965734|n/a|n/a|no|1| +|Akan (`AK`)|ALL_WORDS|Radixor|0.997522|n/a|n/a|no|1| +|Akan (`AK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997522|n/a|n/a|no|1| +|Albanian (`SQ_AL`)|ALL_WORDS|Radixor|0.996960|n/a|n/a|no|1| +|Albanian (`SQ_AL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996960|n/a|n/a|no|1| +|Alsatian (`GSW`)|ALL_WORDS|Radixor|0.995913|n/a|n/a|no|1| +|Alsatian (`GSW`)|LOWERCASE_GROUPS_ONLY|Radixor|0.995913|n/a|n/a|no|1| +|Amharic (`AM_ET`)|ALL_WORDS|Radixor|0.980052|n/a|n/a|no|1| +|Amharic (`AM_ET`)|LOWERCASE_GROUPS_ONLY|Radixor|0.980052|n/a|n/a|no|1| +|Amol Azerbaijani (`AZG`)|ALL_WORDS|Radixor|0.994697|n/a|n/a|no|1| +|Amol Azerbaijani (`AZG`)|LOWERCASE_GROUPS_ONLY|Radixor|0.994697|n/a|n/a|no|1| +|Ancient Greek (`GRC`)|ALL_WORDS|Radixor|0.995540|n/a|n/a|no|1| +|Ancient Greek (`GRC`)|LOWERCASE_GROUPS_ONLY|Radixor|0.995540|n/a|n/a|no|1| +|Anglo-Norman (`XNO`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Anglo-Norman (`XNO`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Arabic (`AR`)|ALL_WORDS|Radixor|0.987101|SNOWBALL ARABIC DIRECT|0.315268705|no|3| +|Arabic (`AR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.987101|SNOWBALL ARABIC DIRECT|0.315268705|no|3| +|Armenian (`HY_AM`)|ALL_WORDS|Radixor|0.993984|SNOWBALL ARMENIAN DIRECT|0.427825813|no|3| +|Armenian (`HY_AM`)|LOWERCASE_GROUPS_ONLY|Radixor|0.993984|SNOWBALL ARMENIAN DIRECT|0.427825813|no|3| +|Ashaninka (`CNI`)|ALL_WORDS|Radixor|0.998172|n/a|n/a|no|1| +|Ashaninka (`CNI`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998172|n/a|n/a|no|1| +|Assamese (`AS_IN`)|ALL_WORDS|Radixor|0.991853|n/a|n/a|no|1| +|Assamese (`AS_IN`)|LOWERCASE_GROUPS_ONLY|Radixor|0.991853|n/a|n/a|no|1| +|Asturian (`AST`)|ALL_WORDS|Radixor|0.994745|n/a|n/a|no|1| +|Asturian (`AST`)|LOWERCASE_GROUPS_ONLY|Radixor|0.994745|n/a|n/a|no|1| +|Aymara (`AYM`)|ALL_WORDS|Radixor|0.996564|n/a|n/a|no|1| +|Aymara (`AYM`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996564|n/a|n/a|no|1| +|Azerbaijani (`AZ_AZ`)|ALL_WORDS|Radixor|0.999948|n/a|n/a|no|1| +|Azerbaijani (`AZ_AZ`)|LOWERCASE_GROUPS_ONLY|Radixor|0.999948|n/a|n/a|no|1| +|Bashkir (`BAK`)|ALL_WORDS|Radixor|0.998782|n/a|n/a|no|1| +|Bashkir (`BAK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998782|n/a|n/a|no|1| +|Belarusian (`BE_BY`)|ALL_WORDS|Radixor|0.996350|n/a|n/a|no|1| +|Belarusian (`BE_BY`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996350|n/a|n/a|no|1| +|Bengali (`BN_BD`)|ALL_WORDS|Radixor|0.997025|n/a|n/a|no|1| +|Bengali (`BN_BD`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997025|n/a|n/a|no|1| +|Bininj Kun-wok (`GUP`)|ALL_WORDS|Radixor|0.997625|n/a|n/a|no|1| +|Bininj Kun-wok (`GUP`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997625|n/a|n/a|no|1| +|Braj (`BRA`)|ALL_WORDS|Radixor|0.989171|n/a|n/a|no|1| +|Braj (`BRA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989171|n/a|n/a|no|1| +|Breton (`BRE`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Breton (`BRE`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Bulgarian (`BG_BG`)|ALL_WORDS|Radixor|0.998003|n/a|n/a|no|1| +|Bulgarian (`BG_BG`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998003|n/a|n/a|no|1| +|Catalan (`CA_ES`)|ALL_WORDS|Radixor|0.987704|SNOWBALL CATALAN DIRECT|0.076784766|no|3| +|Catalan (`CA_ES`)|LOWERCASE_GROUPS_ONLY|Radixor|0.987704|SNOWBALL CATALAN DIRECT|0.076784766|no|3| +|Cebuano (`CEB`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Cebuano (`CEB`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Chichewa (`NY_MW`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Chichewa (`NY_MW`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Chukchi (`CKT`)|ALL_WORDS|Radixor|0.990506|n/a|n/a|no|1| +|Chukchi (`CKT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.990506|n/a|n/a|no|1| +|Church Slavonic (`CHU`)|ALL_WORDS|Radixor|0.918731|n/a|n/a|no|1| +|Church Slavonic (`CHU`)|LOWERCASE_GROUPS_ONLY|Radixor|0.918731|n/a|n/a|no|1| +|Classical Armenian (`XCL`)|ALL_WORDS|Radixor|0.949840|n/a|n/a|no|1| +|Classical Armenian (`XCL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.949840|n/a|n/a|no|1| +|Classical Syriac (`SYC`)|ALL_WORDS|Radixor|0.993956|n/a|n/a|no|1| +|Classical Syriac (`SYC`)|LOWERCASE_GROUPS_ONLY|Radixor|0.993956|n/a|n/a|no|1| +|Congo Swahili (`SWC`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Congo Swahili (`SWC`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Copala Triqui (`CPA`)|ALL_WORDS|Radixor|0.895664|n/a|n/a|no|1| +|Copala Triqui (`CPA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.895664|n/a|n/a|no|1| +|Cornish (`COR`)|ALL_WORDS|Radixor|0.997378|n/a|n/a|no|1| +|Cornish (`COR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997378|n/a|n/a|no|1| +|Cree (`CRE`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Cree (`CRE`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Crimean Tatar (`CRH`)|ALL_WORDS|Radixor|0.999524|n/a|n/a|no|1| +|Crimean Tatar (`CRH`)|LOWERCASE_GROUPS_ONLY|Radixor|0.999524|n/a|n/a|no|1| |Czech (`CS_CZ`)|ALL_WORDS|Radixor|0.996617|HUNSPELL CZECH LUCENE FILTER|0.142485045|no|4| |Czech (`CS_CZ`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997195|HUNSPELL CZECH LUCENE FILTER|0.144045088|no|4| +|Dakota (`DAK`)|ALL_WORDS|Radixor|0.994385|n/a|n/a|no|1| +|Dakota (`DAK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.994385|n/a|n/a|no|1| |Danish (`DA_DK`)|ALL_WORDS|Radixor|0.996243|SNOWBALL DANISH DIRECT|0.053760569|no|3| |Danish (`DA_DK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996482|SNOWBALL DANISH DIRECT|0.054099342|no|3| |Dutch (`NL_NL`)|ALL_WORDS|Radixor|0.988733|SNOWBALL DUTCH DIRECT|0.261639748|no|4| |Dutch (`NL_NL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989114|SNOWBALL DUTCH DIRECT|0.258605347|no|4| +|Eastern Chatino (`CLY`)|ALL_WORDS|Radixor|0.924863|n/a|n/a|no|1| +|Eastern Chatino (`CLY`)|LOWERCASE_GROUPS_ONLY|Radixor|0.924863|n/a|n/a|no|1| +|Egyptian Arabic (`ARZ`)|ALL_WORDS|Radixor|0.978579|n/a|n/a|no|1| +|Egyptian Arabic (`ARZ`)|LOWERCASE_GROUPS_ONLY|Radixor|0.978579|n/a|n/a|no|1| |English (`US_UK`)|ALL_WORDS|Radixor|0.976120|ENGLISH LUCENE PORTER COPIED|0.010985401|no|11| |English (`US_UK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.976863|ENGLISH LUCENE PORTER COPIED|0.011393230|no|11| +|Estonian (`ET_EE`)|ALL_WORDS|Radixor|0.997030|SNOWBALL ESTONIAN DIRECT|0.255774903|no|3| +|Estonian (`ET_EE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997030|SNOWBALL ESTONIAN DIRECT|0.255774903|no|3| +|Evenki (`EVN`)|ALL_WORDS|Radixor|0.996169|n/a|n/a|no|1| +|Evenki (`EVN`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996169|n/a|n/a|no|1| +|Faroese (`FO_FO`)|ALL_WORDS|Radixor|0.979257|n/a|n/a|no|1| +|Faroese (`FO_FO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.979257|n/a|n/a|no|1| |Finnish (`FI_FI`)|ALL_WORDS|Radixor|0.984838|SNOWBALL FINNISH LUCENE FILTER|0.244558928|no|4| |Finnish (`FI_FI`)|LOWERCASE_GROUPS_ONLY|Radixor|0.988242|SNOWBALL FINNISH DIRECT|0.249699076|no|4| |French (`FR_FR`)|ALL_WORDS|Radixor|0.958627|SNOWBALL FRENCH DIRECT|0.109964908|no|6| |French (`FR_FR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.958856|SNOWBALL FRENCH DIRECT|0.110030565|no|6| +|Friulian (`FUR`)|ALL_WORDS|Radixor|0.997938|n/a|n/a|no|1| +|Friulian (`FUR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997938|n/a|n/a|no|1| +|Ga (`GAA`)|ALL_WORDS|Radixor|0.997872|n/a|n/a|no|1| +|Ga (`GAA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997872|n/a|n/a|no|1| +|Galolen (`GAL`)|ALL_WORDS|Radixor|0.983508|n/a|n/a|no|1| +|Galolen (`GAL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.983508|n/a|n/a|no|1| |German (`DE_DE`)|ALL_WORDS|Radixor|0.910445|GERMAN CISTEM|0.031918024|no|8| |German (`DE_DE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.966959|GERMAN CISTEM|0.052231588|no|8| +|Gothic (`GOT`)|ALL_WORDS|Radixor|0.981471|n/a|n/a|no|1| +|Gothic (`GOT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.981471|n/a|n/a|no|1| +|Greek (`EL_GR`)|ALL_WORDS|Radixor|0.978296|SNOWBALL GREEK DIRECT|0.106884319|no|3| +|Greek (`EL_GR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.978296|SNOWBALL GREEK DIRECT|0.106884319|no|3| +|Gulf Arabic (`AFB`)|ALL_WORDS|Radixor|0.986901|n/a|n/a|no|1| +|Gulf Arabic (`AFB`)|LOWERCASE_GROUPS_ONLY|Radixor|0.986901|n/a|n/a|no|1| +|Haida (`HAI`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Haida (`HAI`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| |Hebrew (`HE_IL`)|ALL_WORDS|Radixor|0.986253|n/a|n/a|no|1| |Hebrew (`HE_IL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.986253|n/a|n/a|no|1| +|Hiligaynon (`HIL`)|ALL_WORDS|Radixor|0.985981|n/a|n/a|no|1| +|Hiligaynon (`HIL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.985981|n/a|n/a|no|1| +|Hsilimo (`HSI`)|ALL_WORDS|Radixor|0.996914|n/a|n/a|no|1| +|Hsilimo (`HSI`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996914|n/a|n/a|no|1| |Hungarian (`HU_HU`)|ALL_WORDS|Radixor|0.995555|SNOWBALL HUNGARIAN LUCENE FILTER|0.172591951|no|4| |Hungarian (`HU_HU`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996227|SNOWBALL HUNGARIAN DIRECT|0.174150583|no|4| +|Icelandic (`IS_IS`)|ALL_WORDS|Radixor|0.981831|n/a|n/a|no|1| +|Icelandic (`IS_IS`)|LOWERCASE_GROUPS_ONLY|Radixor|0.981831|n/a|n/a|no|1| +|Indonesian (`ID_ID`)|ALL_WORDS|Radixor|0.999914|SNOWBALL INDONESIAN DIRECT|0.128496605|no|3| +|Indonesian (`ID_ID`)|LOWERCASE_GROUPS_ONLY|Radixor|0.999914|SNOWBALL INDONESIAN DIRECT|0.128496605|no|3| +|Ingrian (`IZH`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Ingrian (`IZH`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Irish (`GA_IE`)|ALL_WORDS|Radixor|0.975677|SNOWBALL IRISH DIRECT|0.462889616|no|3| +|Irish (`GA_IE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.975677|SNOWBALL IRISH DIRECT|0.462889616|no|3| |Italian (`IT_IT`)|ALL_WORDS|Radixor|0.996651|SNOWBALL ITALIAN DIRECT|0.130360651|no|4| |Italian (`IT_IT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996656|SNOWBALL ITALIAN DIRECT|0.130349693|no|4| -|Norwegian Bokmal (`NB_NO`)|ALL_WORDS|Radixor|0.976021|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.101762107|no|5| -|Norwegian Bokmal (`NB_NO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.976240|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.101954266|no|5| +|Itelmen (`ITL`)|ALL_WORDS|Radixor|0.999389|n/a|n/a|no|1| +|Itelmen (`ITL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.999389|n/a|n/a|no|1| +|Japanese (`JA_JP`)|ALL_WORDS|Radixor|0.919503|n/a|n/a|no|1| +|Japanese (`JA_JP`)|LOWERCASE_GROUPS_ONLY|Radixor|0.919503|n/a|n/a|no|1| +|Kabardian (`KBD`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Kabardian (`KBD`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Kalaallisut (`KL_GL`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Kalaallisut (`KL_GL`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Kannada (`KN_IN`)|ALL_WORDS|Radixor|0.898695|n/a|n/a|no|1| +|Kannada (`KN_IN`)|LOWERCASE_GROUPS_ONLY|Radixor|0.898695|n/a|n/a|no|1| +|Karelian (`KRL`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Karelian (`KRL`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Kashubian (`CSB`)|ALL_WORDS|Radixor|0.995801|n/a|n/a|no|1| +|Kashubian (`CSB`)|LOWERCASE_GROUPS_ONLY|Radixor|0.995801|n/a|n/a|no|1| +|Kazakh (`KK_KZ`)|ALL_WORDS|Radixor|0.998095|n/a|n/a|no|1| +|Kazakh (`KK_KZ`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998095|n/a|n/a|no|1| +|Khakas (`KJH`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Khakas (`KJH`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Khinalugh (`KLR`)|ALL_WORDS|Radixor|0.944247|n/a|n/a|no|1| +|Khinalugh (`KLR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.944247|n/a|n/a|no|1| +|Kodi (`KOD`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Kodi (`KOD`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Kongo (`KON`)|ALL_WORDS|Radixor|0.998208|n/a|n/a|no|1| +|Kongo (`KON`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998208|n/a|n/a|no|1| +|Kyrgyz (`KY_KG`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Kyrgyz (`KY_KG`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Ladin (`LLD`)|ALL_WORDS|Radixor|0.997844|n/a|n/a|no|1| +|Ladin (`LLD`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997844|n/a|n/a|no|1| +|Latin (`LA`)|ALL_WORDS|Radixor|0.963898|n/a|n/a|no|1| +|Latin (`LA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.963898|n/a|n/a|no|1| +|Latvian (`LV_LV`)|ALL_WORDS|Radixor|0.941147|n/a|n/a|no|1| +|Latvian (`LV_LV`)|LOWERCASE_GROUPS_ONLY|Radixor|0.941147|n/a|n/a|no|1| +|Lingala (`LIN`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Lingala (`LIN`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Lithuanian (`LT_LT`)|ALL_WORDS|Radixor|0.998532|SNOWBALL LITHUANIAN DIRECT|0.212439399|no|3| +|Lithuanian (`LT_LT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998532|SNOWBALL LITHUANIAN DIRECT|0.212439399|no|3| +|Livonian (`LIV`)|ALL_WORDS|Radixor|0.997482|n/a|n/a|no|1| +|Livonian (`LIV`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997482|n/a|n/a|no|1| +|Low German (`NDS`)|ALL_WORDS|Radixor|0.980207|n/a|n/a|no|1| +|Low German (`NDS`)|LOWERCASE_GROUPS_ONLY|Radixor|0.980207|n/a|n/a|no|1| +|Lower Sorbian (`DSB`)|ALL_WORDS|Radixor|0.992702|n/a|n/a|no|1| +|Lower Sorbian (`DSB`)|LOWERCASE_GROUPS_ONLY|Radixor|0.992702|n/a|n/a|no|1| +|Luganda (`LG_UG`)|ALL_WORDS|Radixor|0.994127|n/a|n/a|no|1| +|Luganda (`LG_UG`)|LOWERCASE_GROUPS_ONLY|Radixor|0.994127|n/a|n/a|no|1| +|Macedonian (`MK_MK`)|ALL_WORDS|Radixor|0.996059|n/a|n/a|no|1| +|Macedonian (`MK_MK`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996059|n/a|n/a|no|1| +|Magahi (`MAG`)|ALL_WORDS|Radixor|0.986749|n/a|n/a|no|1| +|Magahi (`MAG`)|LOWERCASE_GROUPS_ONLY|Radixor|0.986749|n/a|n/a|no|1| +|Malagasy (`MG_MG`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Malagasy (`MG_MG`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Maltese (`MT_MT`)|ALL_WORDS|Radixor|0.996750|n/a|n/a|no|1| +|Maltese (`MT_MT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996750|n/a|n/a|no|1| +|Manx (`GV_IM`)|ALL_WORDS|n/a|n/a|n/a|n/a|n/a|0| +|Manx (`GV_IM`)|LOWERCASE_GROUPS_ONLY|n/a|n/a|n/a|n/a|n/a|0| +|Maori (`MI_NZ`)|ALL_WORDS|Radixor|0.995192|n/a|n/a|no|1| +|Maori (`MI_NZ`)|LOWERCASE_GROUPS_ONLY|Radixor|0.995192|n/a|n/a|no|1| +|Mapudungun (`ARN`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Mapudungun (`ARN`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Middle French (`FRM`)|ALL_WORDS|Radixor|0.996418|n/a|n/a|no|1| +|Middle French (`FRM`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996418|n/a|n/a|no|1| +|Middle High German (`GMH`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Middle High German (`GMH`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Middle Low German (`GML`)|ALL_WORDS|Radixor|0.978543|n/a|n/a|no|1| +|Middle Low German (`GML`)|LOWERCASE_GROUPS_ONLY|Radixor|0.978543|n/a|n/a|no|1| +|Mongolian (`MN_MN`)|ALL_WORDS|Radixor|0.979541|n/a|n/a|no|1| +|Mongolian (`MN_MN`)|LOWERCASE_GROUPS_ONLY|Radixor|0.979541|n/a|n/a|no|1| +|Murrinh-Patha (`MWF`)|ALL_WORDS|Radixor|0.861762|n/a|n/a|no|1| +|Murrinh-Patha (`MWF`)|LOWERCASE_GROUPS_ONLY|Radixor|0.861762|n/a|n/a|no|1| +|Navajo (`NAV`)|ALL_WORDS|Radixor|0.976188|n/a|n/a|no|1| +|Navajo (`NAV`)|LOWERCASE_GROUPS_ONLY|Radixor|0.976188|n/a|n/a|no|1| +|Neapolitan (`NAP`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Neapolitan (`NAP`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|North Frisian (`FRR`)|ALL_WORDS|Radixor|0.954614|n/a|n/a|no|1| +|North Frisian (`FRR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.954614|n/a|n/a|no|1| +|Northern Sami (`SME`)|ALL_WORDS|Radixor|0.991171|n/a|n/a|no|1| +|Northern Sami (`SME`)|LOWERCASE_GROUPS_ONLY|Radixor|0.991171|n/a|n/a|no|1| +|Norwegian Bokmål (`NB_NO`)|ALL_WORDS|Radixor|0.976021|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.101762107|no|5| +|Norwegian Bokmål (`NB_NO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.976240|SNOWBALL NORWEGIAN BOKMAL DIRECT|0.101954266|no|5| |Norwegian Nynorsk (`NN_NO`)|ALL_WORDS|Radixor|0.950991|SNOWBALL NORWEGIAN NYNORSK DIRECT|0.082896791|no|3| |Norwegian Nynorsk (`NN_NO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.951104|SNOWBALL NORWEGIAN NYNORSK DIRECT|0.082851757|no|3| +|Old English (`ANG`)|ALL_WORDS|Radixor|0.970779|n/a|n/a|no|1| +|Old English (`ANG`)|LOWERCASE_GROUPS_ONLY|Radixor|0.970779|n/a|n/a|no|1| +|Old French (`FRO`)|ALL_WORDS|Radixor|0.965650|n/a|n/a|no|1| +|Old French (`FRO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.965650|n/a|n/a|no|1| +|Old High German (`GOH`)|ALL_WORDS|Radixor|0.992668|n/a|n/a|no|1| +|Old High German (`GOH`)|LOWERCASE_GROUPS_ONLY|Radixor|0.992668|n/a|n/a|no|1| +|Old Irish (`SGA`)|ALL_WORDS|Radixor|0.997869|n/a|n/a|no|1| +|Old Irish (`SGA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997869|n/a|n/a|no|1| +|Old Norse (`NON`)|ALL_WORDS|Radixor|0.940817|n/a|n/a|no|1| +|Old Norse (`NON`)|LOWERCASE_GROUPS_ONLY|Radixor|0.940817|n/a|n/a|no|1| +|Old Saxon (`OSX`)|ALL_WORDS|Radixor|0.987927|n/a|n/a|no|1| +|Old Saxon (`OSX`)|LOWERCASE_GROUPS_ONLY|Radixor|0.987927|n/a|n/a|no|1| +|Oodham (`OOD`)|ALL_WORDS|Radixor|0.988769|n/a|n/a|no|1| +|Oodham (`OOD`)|LOWERCASE_GROUPS_ONLY|Radixor|0.988769|n/a|n/a|no|1| +|Pashto (`PS_AF`)|ALL_WORDS|Radixor|0.955597|n/a|n/a|no|1| +|Pashto (`PS_AF`)|LOWERCASE_GROUPS_ONLY|Radixor|0.955597|n/a|n/a|no|1| |Persian (`FA_IR`)|ALL_WORDS|Radixor|0.975610|SNOWBALL PERSIAN DIRECT|0.440486794|no|3| |Persian (`FA_IR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.975610|SNOWBALL PERSIAN DIRECT|0.440486794|no|3| |Polish (`PL_PL`)|ALL_WORDS|Radixor|0.991105|POLISH LUCENE MORFOLOGIK FILTER|0.042712804|no|6| |Polish (`PL_PL`)|LOWERCASE_GROUPS_ONLY|Radixor|0.991301|POLISH LUCENE MORFOLOGIK FILTER|0.042883749|no|6| |Portuguese (`PT_PT`)|ALL_WORDS|Radixor|0.998542|SNOWBALL PORTUGUESE DIRECT|0.059619854|no|6| |Portuguese (`PT_PT`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998542|SNOWBALL PORTUGUESE DIRECT|0.059619854|no|6| +|Quechua (`QUE`)|ALL_WORDS|Radixor|0.986835|n/a|n/a|no|1| +|Quechua (`QUE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.986835|n/a|n/a|no|1| +|Romanian (`RO_RO`)|ALL_WORDS|Radixor|0.994415|SNOWBALL ROMANIAN DIRECT|0.145283621|no|3| +|Romanian (`RO_RO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.994415|SNOWBALL ROMANIAN DIRECT|0.145283621|no|3| |Russian (`RU_RU`)|ALL_WORDS|Radixor|0.990188|SNOWBALL RUSSIAN LUCENE FILTER|0.155623602|no|4| |Russian (`RU_RU`)|LOWERCASE_GROUPS_ONLY|Radixor|0.990213|SNOWBALL RUSSIAN DIRECT|0.155670422|no|4| +|Seneca (`SEE`)|ALL_WORDS|Radixor|0.998027|n/a|n/a|no|1| +|Seneca (`SEE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998027|n/a|n/a|no|1| +|Serbo-Croatian (`HBS`)|ALL_WORDS|Radixor|0.987756|n/a|n/a|no|1| +|Serbo-Croatian (`HBS`)|LOWERCASE_GROUPS_ONLY|Radixor|0.987756|n/a|n/a|no|1| +|Shipibo-Conibo (`SHP`)|ALL_WORDS|Radixor|0.978699|n/a|n/a|no|1| +|Shipibo-Conibo (`SHP`)|LOWERCASE_GROUPS_ONLY|Radixor|0.978699|n/a|n/a|no|1| +|Shona (`SN_ZW`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Shona (`SN_ZW`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Sotho, Southern (`ST_ZA`)|ALL_WORDS|Radixor|1.000000|SNOWBALL SESOTHO DIRECT|0.496582532|no|2| +|Sotho, Southern (`ST_ZA`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|SNOWBALL SESOTHO DIRECT|0.496582532|no|2| +|Southern Kurdish (`SDH`)|ALL_WORDS|n/a|n/a|n/a|n/a|n/a|0| +|Southern Kurdish (`SDH`)|LOWERCASE_GROUPS_ONLY|n/a|n/a|n/a|n/a|n/a|0| |Spanish (`ES_ES`)|ALL_WORDS|Radixor|0.989448|SNOWBALL SPANISH LUCENE FILTER|0.337009985|no|7| |Spanish (`ES_ES`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989580|SNOWBALL SPANISH DIRECT|0.337037572|no|7| |Swedish (`SV_SE`)|ALL_WORDS|Radixor|0.977619|SNOWBALL SWEDISH DIRECT|0.169075635|no|5| |Swedish (`SV_SE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.977573|SNOWBALL SWEDISH DIRECT|0.168961385|no|5| +|Tagalog (`TL_PH`)|ALL_WORDS|Radixor|0.997116|n/a|n/a|no|1| +|Tagalog (`TL_PH`)|LOWERCASE_GROUPS_ONLY|Radixor|0.997116|n/a|n/a|no|1| +|Temoaya Otomi (`OTE`)|ALL_WORDS|Radixor|0.953277|n/a|n/a|no|1| +|Temoaya Otomi (`OTE`)|LOWERCASE_GROUPS_ONLY|Radixor|0.953277|n/a|n/a|no|1| +|Turkish (`TR_TR`)|ALL_WORDS|Radixor|0.991555|SNOWBALL TURKISH DIRECT|0.378971891|no|3| +|Turkish (`TR_TR`)|LOWERCASE_GROUPS_ONLY|Radixor|0.991555|SNOWBALL TURKISH DIRECT|0.378971891|no|3| |Ukrainian (`UK_UA`)|ALL_WORDS|Radixor|0.995816|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.066909929|no|4| |Ukrainian (`UK_UA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.995815|UKRAINIAN LUCENE MORFOLOGIK FILTER|0.066926372|no|4| +|Uyghur (`UG_CN`)|ALL_WORDS|Radixor|0.996210|n/a|n/a|no|1| +|Uyghur (`UG_CN`)|LOWERCASE_GROUPS_ONLY|Radixor|0.996210|n/a|n/a|no|1| +|Uzbek (`UZ_UZ`)|ALL_WORDS|Radixor|0.999762|n/a|n/a|no|1| +|Uzbek (`UZ_UZ`)|LOWERCASE_GROUPS_ONLY|Radixor|0.999762|n/a|n/a|no|1| +|Voro (`VRO`)|ALL_WORDS|Radixor|0.998340|n/a|n/a|no|1| +|Voro (`VRO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.998340|n/a|n/a|no|1| +|Western Highland Chatino (`CTP`)|ALL_WORDS|Radixor|0.971433|n/a|n/a|no|1| +|Western Highland Chatino (`CTP`)|LOWERCASE_GROUPS_ONLY|Radixor|0.971433|n/a|n/a|no|1| +|Xibe (`SJO`)|ALL_WORDS|Radixor|0.970067|n/a|n/a|no|1| +|Xibe (`SJO`)|LOWERCASE_GROUPS_ONLY|Radixor|0.970067|n/a|n/a|no|1| +|Xipinawa (`XTY`)|ALL_WORDS|Radixor|0.990824|n/a|n/a|no|1| +|Xipinawa (`XTY`)|LOWERCASE_GROUPS_ONLY|Radixor|0.990824|n/a|n/a|no|1| +|Yamana (`AME`)|ALL_WORDS|Radixor|0.991071|n/a|n/a|no|1| +|Yamana (`AME`)|LOWERCASE_GROUPS_ONLY|Radixor|0.991071|n/a|n/a|no|1| |Yiddish (`YI`)|ALL_WORDS|Radixor|0.989079|SNOWBALL YIDDISH DIRECT|0.097960961|no|3| |Yiddish (`YI`)|LOWERCASE_GROUPS_ONLY|Radixor|0.989079|SNOWBALL YIDDISH DIRECT|0.097960961|no|3| +|Zacatepec Chatino (`ZPV`)|ALL_WORDS|Radixor|0.992863|n/a|n/a|no|1| +|Zacatepec Chatino (`ZPV`)|LOWERCASE_GROUPS_ONLY|Radixor|0.992863|n/a|n/a|no|1| +|Zarma (`DJE`)|ALL_WORDS|Radixor|1.000000|n/a|n/a|no|1| +|Zarma (`DJE`)|LOWERCASE_GROUPS_ONLY|Radixor|1.000000|n/a|n/a|no|1| +|Zenzontepec Chatino (`CZN`)|ALL_WORDS|Radixor|0.973968|n/a|n/a|no|1| +|Zenzontepec Chatino (`CZN`)|LOWERCASE_GROUPS_ONLY|Radixor|0.973968|n/a|n/a|no|1| +|Zulu (`ZU_ZA`)|ALL_WORDS|Radixor|0.993145|n/a|n/a|no|1| +|Zulu (`ZU_ZA`)|LOWERCASE_GROUPS_ONLY|Radixor|0.993145|n/a|n/a|no|1| ### Secondary-metric trade-offs @@ -157,7 +403,7 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa | Stemmer | Evaluated languages | Wins | Exact first-place ties | Top-three placements | Average rank | Median rank | |---|---:|---:|---:|---:|---:|---:| -|Radixor|20|20|0|20|1.000|1.000| +|Radixor|141|141|0|141|1.000|1.000| |CZECH LUCENE CZECH STEM FILTER|1|0|0|1|3.000|3.000| |ENGLISH LUCENE KSTEM FILTER|1|0|0|0|8.000|8.000| |ENGLISH LUCENE MINIMAL FILTER|1|0|0|0|9.000|9.000| @@ -195,21 +441,37 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa |PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|1|0|0|0|6.000|6.000| |PORTUGUESE LUCENE PORTUGUESE STEM FILTER|1|0|0|0|4.000|4.000| |RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|1|0|0|0|4.000|4.000| +|SNOWBALL ARABIC DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL ARABIC LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL ARMENIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL ARMENIAN LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL CATALAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL CATALAN LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL CZECH DIRECT|1|0|0|0|4.000|4.000| |SNOWBALL DANISH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL DANISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL DUTCH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL DUTCH LUCENE FILTER|1|0|0|0|4.000|4.000| +|SNOWBALL ESTONIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL ESTONIAN LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL FINNISH DIRECT|1|0|0|1|3.000|3.000| |SNOWBALL FINNISH LUCENE FILTER|1|0|0|1|2.000|2.000| |SNOWBALL FRENCH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL FRENCH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL GERMAN DIRECT|1|0|0|1|3.000|3.000| |SNOWBALL GERMAN LUCENE FILTER|1|0|0|0|4.000|4.000| +|SNOWBALL GREEK DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL GREEK LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL HUNGARIAN DIRECT|1|0|0|1|3.000|3.000| |SNOWBALL HUNGARIAN LUCENE FILTER|1|0|0|1|2.000|2.000| +|SNOWBALL INDONESIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL INDONESIAN LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL IRISH DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL IRISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL ITALIAN DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL ITALIAN LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL LITHUANIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL LITHUANIAN LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL NORWEGIAN BOKMAL DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL NORWEGIAN NYNORSK DIRECT|1|0|0|1|2.000|2.000| @@ -218,12 +480,17 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa |SNOWBALL POLISH DIRECT|1|0|0|0|6.000|6.000| |SNOWBALL PORTUGUESE DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL PORTUGUESE LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL ROMANIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL ROMANIAN LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL RUSSIAN DIRECT|1|0|0|1|3.000|3.000| |SNOWBALL RUSSIAN LUCENE FILTER|1|0|0|1|2.000|2.000| +|SNOWBALL SESOTHO DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL SPANISH DIRECT|1|0|0|1|3.000|3.000| |SNOWBALL SPANISH LUCENE FILTER|1|0|0|1|2.000|2.000| |SNOWBALL SWEDISH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL SWEDISH LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL TURKISH DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL TURKISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL YIDDISH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL YIDDISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SPANISH LUCENE SPANISH LIGHT STEM FILTER|1|0|0|0|5.000|5.000| @@ -240,7 +507,7 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa | Stemmer | Evaluated languages | Wins | Exact first-place ties | Top-three placements | Average rank | Median rank | |---|---:|---:|---:|---:|---:|---:| -|Radixor|20|20|0|20|1.000|1.000| +|Radixor|141|141|0|141|1.000|1.000| |CZECH LUCENE CZECH STEM FILTER|1|0|0|1|3.000|3.000| |ENGLISH LUCENE KSTEM FILTER|1|0|0|0|8.000|8.000| |ENGLISH LUCENE MINIMAL FILTER|1|0|0|0|9.000|9.000| @@ -278,21 +545,37 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa |PORTUGUESE LUCENE PORTUGUESE MINIMAL STEM FILTER|1|0|0|0|6.000|6.000| |PORTUGUESE LUCENE PORTUGUESE STEM FILTER|1|0|0|0|4.000|4.000| |RUSSIAN LUCENE RUSSIAN LIGHT STEM FILTER|1|0|0|0|4.000|4.000| +|SNOWBALL ARABIC DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL ARABIC LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL ARMENIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL ARMENIAN LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL CATALAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL CATALAN LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL CZECH DIRECT|1|0|0|0|4.000|4.000| |SNOWBALL DANISH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL DANISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL DUTCH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL DUTCH LUCENE FILTER|1|0|0|0|4.000|4.000| +|SNOWBALL ESTONIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL ESTONIAN LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL FINNISH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL FINNISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL FRENCH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL FRENCH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL GERMAN DIRECT|1|0|0|1|3.000|3.000| |SNOWBALL GERMAN LUCENE FILTER|1|0|0|0|4.000|4.000| +|SNOWBALL GREEK DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL GREEK LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL HUNGARIAN DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL HUNGARIAN LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL INDONESIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL INDONESIAN LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL IRISH DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL IRISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL ITALIAN DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL ITALIAN LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL LITHUANIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL LITHUANIAN LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL NORWEGIAN BOKMAL DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL NORWEGIAN BOKMAL LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL NORWEGIAN NYNORSK DIRECT|1|0|0|1|2.000|2.000| @@ -301,12 +584,17 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa |SNOWBALL POLISH DIRECT|1|0|0|0|6.000|6.000| |SNOWBALL PORTUGUESE DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL PORTUGUESE LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL ROMANIAN DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL ROMANIAN LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL RUSSIAN DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL RUSSIAN LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL SESOTHO DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL SPANISH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL SPANISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL SWEDISH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL SWEDISH LUCENE FILTER|1|0|0|1|3.000|3.000| +|SNOWBALL TURKISH DIRECT|1|0|0|1|2.000|2.000| +|SNOWBALL TURKISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SNOWBALL YIDDISH DIRECT|1|0|0|1|2.000|2.000| |SNOWBALL YIDDISH LUCENE FILTER|1|0|0|1|3.000|3.000| |SPANISH LUCENE SPANISH LIGHT STEM FILTER|1|0|0|0|5.000|5.000| @@ -322,17 +610,17 @@ Counts use `PRIMARY_OUTPUT` only and retain each adapter configuration as a sepa ### Radixor full-coverage aggregates -These aggregates cover all 20 documented languages. Macro balanced accuracy gives each language equal weight. Micro metrics first sum raw pair counts across languages. Unsupported third-party languages are never inserted as zero results, so this full-coverage table is not presented as a cross-stemmer common-language ranking. +These aggregates cover all 143 documented languages. Macro balanced accuracy gives each language equal weight. Micro metrics first sum raw pair counts across languages. Unsupported third-party languages are never inserted as zero results, so this full-coverage table is not presented as a cross-stemmer common-language ranking. | Dictionary mode | Languages | Macro balanced accuracy | Micro balanced accuracy | Micro precision | Micro recall | Micro F1 | |---|---:|---:|---:|---:|---:|---:| -|ALL_WORDS|20|0.981225|0.988003|0.999988|0.976005|0.987851| -|LOWERCASE_GROUPS_ONLY|20|0.984395|0.989641|0.999992|0.979282|0.989529| +|ALL_WORDS|141 / 143|0.984680|0.987120|0.999993|0.974239|0.986948| +|LOWERCASE_GROUPS_ONLY|141 / 143|0.985130|0.988026|0.999996|0.976053|0.987879| ### Reproducible data -- [Machine-readable quality snapshot](data/stemming-quality.csv) -- SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- [Machine-readable quality snapshot](data/stemming-quality-2026-09-11.csv) +- SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - [Linguistic quality methodology](reference/linguistic-quality.md) - [Tested stemmer inventory](reference/tested-stemmers.md) - [Reproducibility and raw data](reference/reproducibility.md) diff --git a/docs/benchmarks/languages/czech.md b/docs/benchmarks/languages/czech.md index ef7e261..894f474 100644 --- a/docs/benchmarks/languages/czech.md +++ b/docs/benchmarks/languages/czech.md @@ -1,8 +1,12 @@ -# Czech Stemmer Benchmarks +# Czech Stemmer Benchmarks ★★★★☆ This page reports same-language stemming benchmarks for Czech. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★☆. The exact count is **51,401 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `cs-cz-default` | `1.0.0` | `CS_CZ` | 5,113 | 56,612 | 10,049 | 46,563 | 46,563 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `cs-cz-default` | `1.0.0` | `CS_CZ` | 5,113 | 51,401 | 56,612 | 10,049 | 46,563 | changed tokens | 46,563 | ## Radixor Patch Command Distribution @@ -42,14 +46,14 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `czechRadixor` | 3.292 | 0.065 | 70.7 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 378.764 | 38.312 | 8134.4 | 115.055 | Benchmark-only Czech Hunspell dictionary compared via Lucene HunspellStemFilter. | -| Lucene CzechStemFilter | `czechLuceneCzechStemFilter` | 2.958 | 0.050 | 63.5 | 0.898 | Czech suffix stemmer implemented as a Lucene TokenFilter. | -| Official Snowball direct | `snowballDirect[CZECH]` | 4.057 | 0.516 | 87.1 | 1.232 | Official Snowball 3.1.0 generated Java stemmer; direct API. | +| Radixor | `czechRadixor` | 3.361 | 0.088 | 72.2 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 352.480 | 40.873 | 7570.0 | 104.877 | Benchmark-only Czech Hunspell dictionary compared via Lucene HunspellStemFilter. | +| Lucene CzechStemFilter | `czechLuceneCzechStemFilter` | 3.134 | 0.042 | 67.3 | 0.933 | Czech suffix stemmer implemented as a Lucene TokenFilter. | +| Official Snowball direct | `snowballDirect[CZECH]` | 4.308 | 0.654 | 92.5 | 1.282 | Official Snowball 3.1.0 generated Java stemmer; direct API. | ## Interpretation Notes @@ -87,18 +91,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **65.439%** - at 10% training knowledge to **69.900%** at 90%, a measured - **+4.461 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+3.349 pp** and - preservation of unseen already-root forms changes by **+1.660 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **65.439%** at 10% training knowledge to **69.900%** at 90%, a measured **+4.461 percentage-point** change. +- Unseen all-form exactness moves from **69.921%** at 10% training knowledge to **73.270%** at 90%, a measured **+3.349 percentage-point** change. +- Preservation of unseen already-root forms moves from **89.636%** at 10% training knowledge to **91.296%** at 90%, a measured **+1.660 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -191,7 +192,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `CS_CZ` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -460,8 +461,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `CS_CZ` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/danish.md b/docs/benchmarks/languages/danish.md index 3897a92..a16b4a2 100644 --- a/docs/benchmarks/languages/danish.md +++ b/docs/benchmarks/languages/danish.md @@ -1,8 +1,12 @@ -# Danish Stemmer Benchmarks +# Danish Stemmer Benchmarks ★★★★☆ This page reports same-language stemming benchmarks for Danish. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★☆. The exact count is **27,921 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `da-dk-default` | `1.0.0` | `DA_DK` | 4,179 | 32,256 | 8,356 | 23,900 | 23,900 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `da-dk-default` | `1.0.0` | `DA_DK` | 4,179 | 27,921 | 32,256 | 8,356 | 23,900 | changed tokens | 23,900 | ## Radixor Patch Command Distribution @@ -41,13 +45,13 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `radixor[DANISH]` | 1.146 | 0.171 | 48.0 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Official Snowball direct | `snowballDirect[DANISH]` | 2.617 | 0.255 | 109.5 | 2.283 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[DANISH]` | 3.013 | 0.332 | 126.1 | 2.629 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `radixor[DANISH]` | 1.361 | 0.216 | 57.0 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Official Snowball direct | `snowballDirect[DANISH]` | 2.712 | 0.277 | 113.5 | 1.992 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[DANISH]` | 3.164 | 0.421 | 132.4 | 2.325 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -85,18 +89,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **51.055%** - at 10% training knowledge to **67.504%** at 90%, a measured - **+16.449 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+13.087 pp** and - preservation of unseen already-root forms changes by **+4.480 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **51.055%** at 10% training knowledge to **67.504%** at 90%, a measured **+16.449 percentage-point** change. +- Unseen all-form exactness moves from **61.104%** at 10% training knowledge to **74.192%** at 90%, a measured **+13.087 percentage-point** change. +- Preservation of unseen already-root forms moves from **88.238%** at 10% training knowledge to **92.718%** at 90%, a measured **+4.480 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -187,7 +188,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `DA_DK` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -434,8 +435,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `DA_DK` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/dutch.md b/docs/benchmarks/languages/dutch.md index d20ded9..1db4bb1 100644 --- a/docs/benchmarks/languages/dutch.md +++ b/docs/benchmarks/languages/dutch.md @@ -1,8 +1,12 @@ -# Dutch Stemmer Benchmarks +# Dutch Stemmer Benchmarks ★★★★☆ This page reports same-language stemming benchmarks for Dutch. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★☆. The exact count is **26,201 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `nl-nl-default` | `1.0.0` | `NL_NL` | 4,992 | 31,466 | 9,981 | 21,485 | 21,485 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `nl-nl-default` | `1.0.0` | `NL_NL` | 4,992 | 26,201 | 31,466 | 9,981 | 21,485 | changed tokens | 21,485 | ## Radixor Patch Command Distribution @@ -42,14 +46,14 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `radixor[DUTCH]` | 1.395 | 0.185 | 64.9 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 26.133 | 4.062 | 1216.3 | 18.735 | Benchmark-only Dutch Hunspell dictionary compared via Lucene HunspellStemFilter. | -| Official Snowball direct | `snowballDirect[DUTCH]` | 4.299 | 0.274 | 200.1 | 3.082 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[DUTCH]` | 7.270 | 0.290 | 338.4 | 5.212 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `radixor[DUTCH]` | 1.540 | 0.278 | 71.7 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 28.377 | 4.971 | 1320.8 | 18.426 | Benchmark-only Dutch Hunspell dictionary compared via Lucene HunspellStemFilter. | +| Official Snowball direct | `snowballDirect[DUTCH]` | 4.102 | 0.293 | 190.9 | 2.663 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[DUTCH]` | 7.211 | 0.376 | 335.6 | 4.682 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -87,18 +91,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **55.524%** - at 10% training knowledge to **65.891%** at 90%, a measured - **+10.366 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+7.429 pp** and - preservation of unseen already-root forms changes by **+1.896 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **55.524%** at 10% training knowledge to **65.891%** at 90%, a measured **+10.366 percentage-point** change. +- Unseen all-form exactness moves from **67.039%** at 10% training knowledge to **74.468%** at 90%, a measured **+7.429 percentage-point** change. +- Preservation of unseen already-root forms moves from **92.616%** at 10% training knowledge to **94.512%** at 90%, a measured **+1.896 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -191,7 +192,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NL_NL` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -460,8 +461,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `NL_NL` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/english.md b/docs/benchmarks/languages/english.md index faf0ccb..94fdbaa 100644 --- a/docs/benchmarks/languages/english.md +++ b/docs/benchmarks/languages/english.md @@ -1,8 +1,12 @@ -# English Stemmer Benchmarks +# English Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for English. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **591,946 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `us-uk-default` | `1.0.1` | `US_UK` | 396,939 | 1,002,414 | 793,874 | 208,540 | 208,540 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `us-uk-default` | `1.0.1` | `US_UK` | 396,939 | 591,946 | 1,002,414 | 793,874 | 208,540 | changed tokens | 208,540 | ## Radixor Patch Command Distribution @@ -49,26 +53,26 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. !!! note "Separate English speed suites" - The `80.8 ns/token` Radixor value below is from the multilingual same-language comparison suite. The [coverage experiment](../reference/english-coverage.md) reports its own full-knowledge point from a separate benchmark method and run. Treat both as suite-specific estimates with their published uncertainty, not as interchangeable values. + The `84.7 ns/token` Radixor value below is from the multilingual same-language comparison suite. The [coverage experiment](../reference/english-coverage.md) reports its own full-knowledge point from a separate benchmark method and run. Treat both as suite-specific estimates with their published uncertainty, not as interchangeable values. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `radixorUsUkProfiPreferredStem` | 16.840 | 1.732 | 80.8 | 1.000 | Full dictionary patch-command stemmer using compiled patch commands. | -| Lucene EnglishPossessiveFilter | `luceneEnglishPossessiveFilter` | 16.439 | 0.771 | 78.8 | 0.976 | Possessive-ending remover only; not a full stemmer. | -| Lucene EnglishMinimalStemFilter | `luceneEnglishMinimalStemFilter` | 18.665 | 0.717 | 89.5 | 1.108 | Narrow plural reduction filter; not a full stemmer. | -| Lucene PorterStemmer direct copy | `lucenePorterStemmerCopied` | 16.770 | 0.260 | 80.4 | 0.996 | Benchmark-only generated copy of Lucene package-private Porter implementation. | -| OpenNLP PorterStemmer | `opennlpPorterStemmer` | 16.888 | 0.247 | 81.0 | 1.003 | Apache OpenNLP Porter implementation. | -| Snowball original Porter | `snowballOriginalPorter` | 32.842 | 2.612 | 157.5 | 1.950 | Classic Porter suffix-rule stemmer; historical English baseline, not a dictionary-equivalent stemmer. | -| Lucene PorterStemFilter | `lucenePorterStemFilter` | 31.412 | 0.750 | 150.6 | 1.865 | Lucene TokenFilter integration path for Porter; includes TokenStream overhead. | -| Lucene KStemFilter | `luceneKStemFilter` | 43.402 | 1.146 | 208.1 | 2.577 | Krovetz-style English TokenFilter; broader than minimal suffix filters. | -| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 72.329 | 1.052 | 346.8 | 4.295 | Benchmark-only English Hunspell comparison using the benchmark Hunspell corpus. | -| Snowball English / Porter2 | `snowballEnglishPorter2` | 46.742 | 2.724 | 224.1 | 2.776 | Porter2 suffix-rule stemmer, distinct from original Porter. | -| Paice/Husk Lancaster | `paiceHuskLancaster` | 138.158 | 3.983 | 662.5 | 8.204 | Aggressive rule-based English stemmer. | +| Radixor | `radixorUsUkProfiPreferredStem` | 17.654 | 2.053 | 84.7 | 1.000 | Full dictionary patch-command stemmer using compiled patch commands. | +| Lucene EnglishPossessiveFilter | `luceneEnglishPossessiveFilter` | 15.732 | 0.282 | 75.4 | 0.891 | Possessive-ending remover only; not a full stemmer. | +| Lucene EnglishMinimalStemFilter | `luceneEnglishMinimalStemFilter` | 17.249 | 0.344 | 82.7 | 0.977 | Narrow plural reduction filter; not a full stemmer. | +| Lucene PorterStemmer direct copy | `lucenePorterStemmerCopied` | 19.164 | 3.414 | 91.9 | 1.086 | Benchmark-only generated copy of Lucene package-private Porter implementation. | +| OpenNLP PorterStemmer | `opennlpPorterStemmer` | 18.592 | 2.908 | 89.2 | 1.053 | Apache OpenNLP Porter implementation. | +| Snowball original Porter | `snowballOriginalPorter` | 34.518 | 2.150 | 165.5 | 1.955 | Classic Porter suffix-rule stemmer; historical English baseline, not a dictionary-equivalent stemmer. | +| Lucene PorterStemFilter | `lucenePorterStemFilter` | 29.831 | 0.457 | 143.0 | 1.690 | Lucene TokenFilter integration path for Porter; includes TokenStream overhead. | +| Lucene KStemFilter | `luceneKStemFilter` | 42.618 | 1.848 | 204.4 | 2.414 | Krovetz-style English TokenFilter; broader than minimal suffix filters. | +| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 74.214 | 6.598 | 355.9 | 4.204 | Benchmark-only English Hunspell comparison using the benchmark Hunspell corpus. | +| Snowball English / Porter2 | `snowballEnglishPorter2` | 46.211 | 1.606 | 221.6 | 2.618 | Porter2 suffix-rule stemmer, distinct from original Porter. | +| Paice/Husk Lancaster | `paiceHuskLancaster` | 139.249 | 3.975 | 667.7 | 7.888 | Aggressive rule-based English stemmer. | ## Interpretation Notes @@ -106,18 +110,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **76.010%** - at 10% training knowledge to **78.854%** at 90%, a measured - **+2.843 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+1.875 pp** and - preservation of unseen already-root forms changes by **+1.560 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **76.010%** at 10% training knowledge to **78.854%** at 90%, a measured **+2.843 percentage-point** change. +- Unseen all-form exactness moves from **92.805%** at 10% training knowledge to **94.680%** at 90%, a measured **+1.875 percentage-point** change. +- Preservation of unseen already-root forms moves from **97.184%** at 10% training knowledge to **98.744%** at 90%, a measured **+1.560 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -210,7 +211,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `US_UK` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -535,8 +536,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `US_UK` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/finnish.md b/docs/benchmarks/languages/finnish.md index 2881f9e..0921a2d 100644 --- a/docs/benchmarks/languages/finnish.md +++ b/docs/benchmarks/languages/finnish.md @@ -1,8 +1,12 @@ -# Finnish Stemmer Benchmarks +# Finnish Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Finnish. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **1,788,784 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `fi-fi-default` | `1.0.0` | `FI_FI` | 57,027 | 1,865,215 | 110,525 | 1,754,690 | 1,754,690 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `fi-fi-default` | `1.0.0` | `FI_FI` | 57,027 | 1,788,784 | 1,865,215 | 110,525 | 1,754,690 | changed tokens | 1,754,690 | ## Radixor Patch Command Distribution @@ -42,14 +46,14 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `finnishRadixor` | 261.169 | 4.729 | 148.8 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene FinnishLightStemFilter | `finnishLuceneFinnishLightStemFilter` | 169.322 | 7.190 | 96.5 | 0.648 | Light Finnish suffix stemmer. | -| Official Snowball direct | `snowballDirect[FINNISH]` | 263.094 | 24.236 | 149.9 | 1.007 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[FINNISH]` | 322.750 | 12.740 | 183.9 | 1.236 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `finnishRadixor` | 296.399 | 10.829 | 168.9 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene FinnishLightStemFilter | `finnishLuceneFinnishLightStemFilter` | 177.154 | 8.445 | 101.0 | 0.598 | Light Finnish suffix stemmer. | +| Official Snowball direct | `snowballDirect[FINNISH]` | 286.169 | 15.290 | 163.1 | 0.965 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[FINNISH]` | 311.412 | 18.031 | 177.5 | 1.051 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -87,18 +91,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **78.644%** - at 10% training knowledge to **86.177%** at 90%, a measured - **+7.532 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+7.434 pp** and - preservation of unseen already-root forms changes by **+6.588 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **78.644%** at 10% training knowledge to **86.177%** at 90%, a measured **+7.532 percentage-point** change. +- Unseen all-form exactness moves from **79.210%** at 10% training knowledge to **86.644%** at 90%, a measured **+7.434 percentage-point** change. +- Preservation of unseen already-root forms moves from **88.247%** at 10% training knowledge to **94.835%** at 90%, a measured **+6.588 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -191,7 +192,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FI_FI` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -446,8 +447,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `FI_FI` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/french.md b/docs/benchmarks/languages/french.md index f3426aa..f534134 100644 --- a/docs/benchmarks/languages/french.md +++ b/docs/benchmarks/languages/french.md @@ -1,8 +1,12 @@ -# French Stemmer Benchmarks +# French Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for French. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **404,011 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `fr-fr-default` | `1.0.0` | `FR_FR` | 59,240 | 474,110 | 108,141 | 365,969 | 365,969 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `fr-fr-default` | `1.0.0` | `FR_FR` | 59,240 | 404,011 | 474,110 | 108,141 | 365,969 | changed tokens | 365,969 | ## Radixor Patch Command Distribution @@ -44,16 +48,16 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `frenchRadixor` | 42.603 | 1.019 | 116.4 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 1699.026 | 35.808 | 4642.5 | 39.881 | Benchmark-only French Hunspell dictionary compared via Lucene HunspellStemFilter. | -| Lucene FrenchMinimalStemFilter | `frenchLuceneFrenchMinimalStemFilter` | 17.239 | 0.422 | 47.1 | 0.405 | Minimal French suffix reducer; narrow baseline. | -| Lucene FrenchLightStemFilter | `frenchLuceneFrenchLightStemFilter` | 28.180 | 1.108 | 77.0 | 0.661 | Light French suffix stemmer. | -| Official Snowball direct | `snowballDirect[FRENCH]` | 110.345 | 5.041 | 301.5 | 2.590 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[FRENCH]` | 124.104 | 5.294 | 339.1 | 2.913 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `frenchRadixor` | 46.930 | 1.367 | 128.2 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 1762.626 | 129.299 | 4816.3 | 37.559 | Benchmark-only French Hunspell dictionary compared via Lucene HunspellStemFilter. | +| Lucene FrenchMinimalStemFilter | `frenchLuceneFrenchMinimalStemFilter` | 18.828 | 1.281 | 51.4 | 0.401 | Minimal French suffix reducer; narrow baseline. | +| Lucene FrenchLightStemFilter | `frenchLuceneFrenchLightStemFilter` | 29.206 | 1.693 | 79.8 | 0.622 | Light French suffix stemmer. | +| Official Snowball direct | `snowballDirect[FRENCH]` | 107.915 | 5.830 | 294.9 | 2.299 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[FRENCH]` | 121.681 | 5.945 | 332.5 | 2.593 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -91,18 +95,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **69.627%** - at 10% training knowledge to **80.031%** at 90%, a measured - **+10.404 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+9.193 pp** and - preservation of unseen already-root forms changes by **+6.651 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **69.627%** at 10% training knowledge to **80.031%** at 90%, a measured **+10.404 percentage-point** change. +- Unseen all-form exactness moves from **74.276%** at 10% training knowledge to **83.469%** at 90%, a measured **+9.193 percentage-point** change. +- Preservation of unseen already-root forms moves from **90.013%** at 10% training knowledge to **96.664%** at 90%, a measured **+6.651 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -195,7 +196,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FR_FR` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -480,8 +481,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `FR_FR` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/german.md b/docs/benchmarks/languages/german.md index a4ba23e..671aa18 100644 --- a/docs/benchmarks/languages/german.md +++ b/docs/benchmarks/languages/german.md @@ -1,8 +1,12 @@ -# German Stemmer Benchmarks +# German Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for German. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **277,266 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `de-de-default` | `1.0.0` | `DE_DE` | 54,092 | 333,036 | 90,535 | 242,501 | 242,501 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `de-de-default` | `1.0.0` | `DE_DE` | 54,092 | 277,266 | 333,036 | 90,535 | 242,501 | changed tokens | 242,501 | ## Radixor Patch Command Distribution @@ -46,18 +50,18 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `germanRadixor` | 36.523 | 1.174 | 150.6 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| CISTEM | `germanCistem` | 289.327 | 8.094 | 1193.1 | 7.922 | Benchmark-only CISTEM implementation. | -| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 276.775 | 22.204 | 1141.3 | 7.578 | Benchmark-only German Hunspell dictionary compared via Lucene HunspellStemFilter. | -| Lucene GermanMinimalStemFilter | `germanLuceneGermanMinimalStemFilter` | 22.697 | 0.295 | 93.6 | 0.621 | Minimal German suffix reduction; narrow baseline. | -| Lucene GermanLightStemFilter | `germanLuceneGermanLightStemFilter` | 23.477 | 0.264 | 96.8 | 0.643 | Light German suffix stemmer; narrower than Radixor's lexicon-trained transformation model. | -| Lucene GermanStemFilter | `germanLuceneGermanStemFilter` | 67.524 | 0.851 | 278.4 | 1.849 | Older German stemming TokenFilter with normalization requirements. | -| Lucene SnowballFilter | `luceneSnowballFilter[GERMAN]` | 104.691 | 3.041 | 431.7 | 2.866 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | -| Official Snowball direct | `snowballDirect[GERMAN]` | 95.094 | 3.149 | 392.1 | 2.604 | Official Snowball generated Java stemmer; direct API. | +| Radixor | `germanRadixor` | 38.055 | 1.638 | 156.9 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| CISTEM | `germanCistem` | 289.668 | 11.748 | 1194.5 | 7.612 | Benchmark-only CISTEM implementation. | +| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 308.274 | 46.165 | 1271.2 | 8.101 | Benchmark-only German Hunspell dictionary compared via Lucene HunspellStemFilter. | +| Lucene GermanMinimalStemFilter | `germanLuceneGermanMinimalStemFilter` | 23.477 | 2.303 | 96.8 | 0.617 | Minimal German suffix reduction; narrow baseline. | +| Lucene GermanLightStemFilter | `germanLuceneGermanLightStemFilter` | 23.050 | 0.269 | 95.1 | 0.606 | Light German suffix stemmer; narrower than Radixor's lexicon-trained transformation model. | +| Lucene GermanStemFilter | `germanLuceneGermanStemFilter` | 66.984 | 2.175 | 276.2 | 1.760 | Older German stemming TokenFilter with normalization requirements. | +| Lucene SnowballFilter | `luceneSnowballFilter[GERMAN]` | 103.452 | 6.867 | 426.6 | 2.718 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Official Snowball direct | `snowballDirect[GERMAN]` | 93.896 | 4.248 | 387.2 | 2.467 | Official Snowball generated Java stemmer; direct API. | ## Interpretation Notes @@ -95,18 +99,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **49.432%** - at 10% training knowledge to **59.108%** at 90%, a measured - **+9.676 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+8.042 pp** and - preservation of unseen already-root forms changes by **+6.967 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **49.432%** at 10% training knowledge to **59.108%** at 90%, a measured **+9.676 percentage-point** change. +- Unseen all-form exactness moves from **59.280%** at 10% training knowledge to **67.322%** at 90%, a measured **+8.042 percentage-point** change. +- Preservation of unseen already-root forms moves from **86.150%** at 10% training knowledge to **93.117%** at 90%, a measured **+6.967 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -199,7 +200,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `DE_DE` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -500,8 +501,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `DE_DE` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/hebrew.md b/docs/benchmarks/languages/hebrew.md index 8cdc6f3..9cffb96 100644 --- a/docs/benchmarks/languages/hebrew.md +++ b/docs/benchmarks/languages/hebrew.md @@ -1,8 +1,12 @@ -# Hebrew Stemmer Benchmarks +# Hebrew Stemmer Benchmarks ★★★★☆ This page reports same-language stemming benchmarks for Hebrew. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★☆. The exact count is **57,658 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ The default Hebrew model currently has no same-language third-party adapter in t ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `he-il-default` | `1.0.0` | `HE_IL` | 2,358 | 61,071 | 4,715 | 56,356 | 56,356 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `he-il-default` | `1.0.0` | `HE_IL` | 2,358 | 57,658 | 61,071 | 4,715 | 56,356 | changed tokens | 56,356 | ## Radixor Patch Command Distribution @@ -39,11 +43,11 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `hebrewRadixor` | 4.966 | 0.167 | 88.1 | 1.000 | Full default-model Radixor dictionary patch-command stemmer. | +| Radixor | `hebrewRadixor` | 5.073 | 0.131 | 90.0 | 1.000 | Full default-model Radixor dictionary patch-command stemmer. | ## Interpretation Notes @@ -80,18 +84,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **10.423%** - at 10% training knowledge to **16.362%** at 90%, a measured - **+5.939 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+5.281 pp** and - preservation of unseen already-root forms changes by **-0.971 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **10.423%** at 10% training knowledge to **16.362%** at 90%, a measured **+5.939 percentage-point** change. +- Unseen all-form exactness moves from **16.532%** at 10% training knowledge to **21.813%** at 90%, a measured **+5.281 percentage-point** change. +- Preservation of unseen already-root forms moves from **89.756%** at 10% training knowledge to **88.785%** at 90%, a measured **-0.971 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -184,7 +185,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `HE_IL` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -415,8 +416,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `HE_IL` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/hungarian.md b/docs/benchmarks/languages/hungarian.md index c1b86ce..06ee7b5 100644 --- a/docs/benchmarks/languages/hungarian.md +++ b/docs/benchmarks/languages/hungarian.md @@ -1,8 +1,12 @@ -# Hungarian Stemmer Benchmarks +# Hungarian Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Hungarian. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **910,688 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `hu-hu-default` | `1.0.0` | `HU_HU` | 19,406 | 935,713 | 38,775 | 896,938 | 896,938 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `hu-hu-default` | `1.0.0` | `HU_HU` | 19,406 | 910,688 | 935,713 | 38,775 | 896,938 | changed tokens | 896,938 | ## Radixor Patch Command Distribution @@ -42,14 +46,14 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `hungarianRadixor` | 55.040 | 0.877 | 61.4 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene HungarianLightStemFilter | `hungarianLuceneHungarianLightStemFilter` | 88.848 | 3.854 | 99.1 | 1.614 | Light Hungarian suffix stemmer. | -| Official Snowball direct | `snowballDirect[HUNGARIAN]` | 167.949 | 9.811 | 187.2 | 3.051 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[HUNGARIAN]` | 186.896 | 11.344 | 208.4 | 3.396 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `hungarianRadixor` | 57.672 | 1.428 | 64.3 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene HungarianLightStemFilter | `hungarianLuceneHungarianLightStemFilter` | 92.463 | 5.822 | 103.1 | 1.603 | Light Hungarian suffix stemmer. | +| Official Snowball direct | `snowballDirect[HUNGARIAN]` | 160.725 | 14.076 | 179.2 | 2.787 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[HUNGARIAN]` | 195.427 | 22.425 | 217.9 | 3.389 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -87,18 +91,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **86.876%** - at 10% training knowledge to **92.203%** at 90%, a measured - **+5.327 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+5.450 pp** and - preservation of unseen already-root forms changes by **+9.360 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **86.876%** at 10% training knowledge to **92.203%** at 90%, a measured **+5.327 percentage-point** change. +- Unseen all-form exactness moves from **86.761%** at 10% training knowledge to **92.212%** at 90%, a measured **+5.450 percentage-point** change. +- Preservation of unseen already-root forms moves from **84.086%** at 10% training knowledge to **93.446%** at 90%, a measured **+9.360 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -191,7 +192,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `HU_HU` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -446,8 +447,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `HU_HU` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/index.md b/docs/benchmarks/languages/index.md index eae21fc..8aecf02 100644 --- a/docs/benchmarks/languages/index.md +++ b/docs/benchmarks/languages/index.md @@ -1,8 +1,8 @@ # Language Benchmark Pages -This section splits Radixor stemmer benchmark results by language. Each of the 20 registered default models has one language page containing the refreshed corpus, patch-command distribution, exact-root accuracy, runtime performance, pairwise stemming-quality tables, and a language-specific interpretation of the edit-cost and dictionary-knowledge experiment. +This section covers all 143 registered language defaults. The original 20 pages retain their complete measured and experimental sections; newly added pages publish corpus, accuracy, and speed evidence from the current checked-in canonical reports. -The experimental sections do not merely repeat the cross-language macro values. They publish each language's five-split 10%–90% knowledge curve, exact command-equivalence census, selected edit costs, command-count effect, within-language factor associations, evidence coverage, and bounded conclusion. Non-baseline settings are exploratory candidates until independently validated. +The original 20 measured pages include experimental sections that do not merely repeat the cross-language macro values. They publish each language's five-split 10%–90% knowledge curve, exact command-equivalence census, selected edit costs, command-count effect, within-language factor associations, evidence coverage, and bounded conclusion. Non-baseline settings are exploratory candidates until independently validated. Equivalent sections for newly registered models remain reserved for later deterministic publication. !!! info "Two knowledge curves, two questions" The dictionary-family curve tests transfer to withheld lexical families under the fixed @@ -26,28 +26,151 @@ The experimental sections do not merely repeat the cross-language macro values. ## Languages -| Language | Resource | Benchmark page | -| --- | --- | --- | -| Czech | `CS_CZ` | [Czech](czech.md) | -| Danish | `DA_DK` | [Danish](danish.md) | -| Dutch | `NL_NL` | [Dutch](dutch.md) | -| English | `US_UK` | [English](english.md) | -| Finnish | `FI_FI` | [Finnish](finnish.md) | -| French | `FR_FR` | [French](french.md) | -| German | `DE_DE` | [German](german.md) | -| Hebrew | `HE_IL` | [Hebrew](hebrew.md) | -| Hungarian | `HU_HU` | [Hungarian](hungarian.md) | -| Italian | `IT_IT` | [Italian](italian.md) | -| Norwegian Bokmal | `NB_NO` | [Norwegian Bokmal](norwegian-bokmal.md) | -| Norwegian Nynorsk | `NN_NO` | [Norwegian Nynorsk](norwegian-nynorsk.md) | -| Persian | `FA_IR` | [Persian](persian.md) | -| Polish | `PL_PL` | [Polish](polish.md) | -| Portuguese | `PT_PT` | [Portuguese](portuguese.md) | -| Russian | `RU_RU` | [Russian](russian.md) | -| Spanish | `ES_ES` | [Spanish](spanish.md) | -| Swedish | `SV_SE` | [Swedish](swedish.md) | -| Ukrainian | `UK_UA` | [Ukrainian](ukrainian.md) | -| Yiddish | `YI` | [Yiddish](yiddish.md) | +| Language | Java enum | Model ID | Relative dictionary size | Availability | Benchmark page | +| --- | --- | --- | --- | --- | --- | +| Adyghe | `ADY` | `ady-default` | ★★★★☆ (20,334) | Individual artifact; outside standard aggregate | [Adyghe](ady.md) | +| Afrikaans | `AF_ZA` | `af-za-default` | ★★★★★ (289,303) | Individual artifact; outside standard aggregate | [Afrikaans](af-za.md) | +| Aimele | `AIL` | `ail-default` | ★★★☆☆ (2,978) | Individual artifact; outside standard aggregate | [Aimele](ail.md) | +| Akan | `AK` | `aka-default` | ★★☆☆☆ (1,890) | Individual artifact; outside standard aggregate | [Akan](aka.md) | +| Albanian | `SQ_AL` | `sq-al-default` | ★★★☆☆ (10,218) | Individual artifact; outside standard aggregate | [Albanian](sq-al.md) | +| Alsatian | `GSW` | `gsw-default` | ★★☆☆☆ (1,171) | Individual artifact; outside standard aggregate | [Alsatian](gsw.md) | +| Amharic | `AM_ET` | `am-et-default` | ★★★★☆ (41,308) | Individual artifact; outside standard aggregate | [Amharic](am-et.md) | +| Amol Azerbaijani | `AZG` | `azg-default` | ★★★☆☆ (9,357) | Individual artifact; outside standard aggregate | [Amol Azerbaijani](azg.md) | +| Ancient Greek | `GRC` | `grc-default` | ★★★★☆ (11,109) | Individual artifact; outside standard aggregate | [Ancient Greek](grc.md) | +| Anglo-Norman | `XNO` | `xno-default` | ★☆☆☆☆ (186) | Individual artifact; outside standard aggregate | [Anglo-Norman](xno.md) | +| Arabic | `AR` | `ar-default` | ★★★★★ (452,974) | Individual artifact; outside standard aggregate | [Arabic](ar.md) | +| Armenian | `HY_AM` | `hy-am-default` | ★★★★★ (242,091) | Individual artifact; outside standard aggregate | [Armenian](hy-am.md) | +| Ashaninka | `CNI` | `cni-default` | ★★★☆☆ (10,095) | Individual artifact; outside standard aggregate | [Ashaninka](cni.md) | +| Assamese | `AS_IN` | `as-in-default` | ★☆☆☆☆ (328) | Individual artifact; outside standard aggregate | [Assamese](as-in.md) | +| Asturian | `AST` | `ast-default` | ★★★★☆ (21,584) | Individual artifact; outside standard aggregate | [Asturian](ast.md) | +| Aymara | `AYM` | `aym-default` | ★★★★★ (231,247) | Individual artifact; outside standard aggregate | [Aymara](aym.md) | +| Azerbaijani | `AZ_AZ` | `az-az-default` | ★★★☆☆ (6,669) | Individual artifact; outside standard aggregate | [Azerbaijani](az-az.md) | +| Bashkir | `BAK` | `bak-default` | ★★★★☆ (10,616) | Individual artifact; outside standard aggregate | [Bashkir](bak.md) | +| Belarusian | `BE_BY` | `be-by-default` | ★★☆☆☆ (678) | Individual artifact; outside standard aggregate | [Belarusian](be-by.md) | +| Bengali | `BN_BD` | `bn-bd-default` | ★☆☆☆☆ (3) | Individual artifact; outside standard aggregate | [Bengali](bn-bd.md) | +| Bininj Kun-wok | `GUP` | `gup-default` | ★☆☆☆☆ (370) | Individual artifact; outside standard aggregate | [Bininj Kun-wok](gup.md) | +| Braj | `BRA` | `bra-default` | ★☆☆☆☆ (115) | Individual artifact; outside standard aggregate | [Braj](bra.md) | +| Breton | `BRE` | `bre-default` | ★★☆☆☆ (1,682) | Individual artifact; outside standard aggregate | [Breton](bre.md) | +| Bulgarian | `BG_BG` | `bg-bg-default` | ★★★★☆ (37,335) | Individual artifact; outside standard aggregate | [Bulgarian](bg-bg.md) | +| Catalan | `CA_ES` | `ca-es-default` | ★★★★★ (61,495) | Individual artifact; outside standard aggregate | [Catalan](ca-es.md) | +| Cebuano | `CEB` | `ceb-default` | ★☆☆☆☆ (392) | Individual artifact; outside standard aggregate | [Cebuano](ceb.md) | +| Chichewa | `NY_MW` | `ny-mw-default` | ★★★☆☆ (3,150) | Individual artifact; outside standard aggregate | [Chichewa](ny-mw.md) | +| Chukchi | `CKT` | `ckt-default` | ★☆☆☆☆ (302) | Individual artifact; outside standard aggregate | [Chukchi](ckt.md) | +| Church Slavonic | `CHU` | `chu-default` | ★★☆☆☆ (1,632) | Individual artifact; outside standard aggregate | [Church Slavonic](chu.md) | +| Classical Armenian | `XCL` | `xcl-default` | ★★★★☆ (55,946) | Individual artifact; outside standard aggregate | [Classical Armenian](xcl.md) | +| Classical Syriac | `SYC` | `syc-default` | ★★★☆☆ (2,139) | Individual artifact; outside standard aggregate | [Classical Syriac](syc.md) | +| Congo Swahili | `SWC` | `swc-default` | ★★★☆☆ (6,700) | Individual artifact; outside standard aggregate | [Congo Swahili](swc.md) | +| Copala Triqui | `CPA` | `cpa-default` | ★★☆☆☆ (1,751) | Individual artifact; outside standard aggregate | [Copala Triqui](cpa.md) | +| Cornish | `COR` | `cor-default` | ★☆☆☆☆ (159) | Individual artifact; outside standard aggregate | [Cornish](cor.md) | +| Cree | `CRE` | `cre-default` | ★★☆☆☆ (514) | Individual artifact; outside standard aggregate | [Cree](cre.md) | +| Crimean Tatar | `CRH` | `crh-default` | ★★★☆☆ (7,183) | Individual artifact; outside standard aggregate | [Crimean Tatar](crh.md) | +| Czech | `CS_CZ` | `cs-cz-default` | ★★★★☆ (51,401) | Standard aggregate | [Czech](czech.md) | +| Dakota | `DAK` | `dak-default` | ★★★☆☆ (2,942) | Individual artifact; outside standard aggregate | [Dakota](dak.md) | +| Danish | `DA_DK` | `da-dk-default` | ★★★★☆ (27,921) | Standard aggregate | [Danish](danish.md) | +| Dutch | `NL_NL` | `nl-nl-default` | ★★★★☆ (26,201) | Standard aggregate | [Dutch](dutch.md) | +| Eastern Chatino | `CLY` | `cly-default` | ★★☆☆☆ (2,030) | Individual artifact; outside standard aggregate | [Eastern Chatino](cly.md) | +| Egyptian Arabic | `ARZ` | `arz-default` | ★☆☆☆☆ (135) | Individual artifact; outside standard aggregate | [Egyptian Arabic](arz.md) | +| English | `US_UK` | `us-uk-default` | ★★★★★ (591,946) | Standard aggregate | [English](english.md) | +| Estonian | `ET_EE` | `et-ee-default` | ★★★★☆ (24,811) | Individual artifact; outside standard aggregate | [Estonian](et-ee.md) | +| Evenki | `EVN` | `evn-default` | ★★★★☆ (11,965) | Individual artifact; outside standard aggregate | [Evenki](evn.md) | +| Faroese | `FO_FO` | `fo-fo-default` | ★★★★☆ (31,366) | Individual artifact; outside standard aggregate | [Faroese](fo-fo.md) | +| Finnish | `FI_FI` | `fi-fi-default` | ★★★★★ (1,788,784) | Standard aggregate | [Finnish](finnish.md) | +| French | `FR_FR` | `fr-fr-default` | ★★★★★ (404,011) | Standard aggregate | [French](french.md) | +| Friulian | `FUR` | `fur-default` | ★★★☆☆ (5,007) | Individual artifact; outside standard aggregate | [Friulian](fur.md) | +| Ga | `GAA` | `gaa-default` | ★☆☆☆☆ (469) | Individual artifact; outside standard aggregate | [Ga](gaa.md) | +| Galolen | `GAL` | `gal-default` | ★★★★☆ (25,436) | Individual artifact; outside standard aggregate | [Galolen](gal.md) | +| German | `DE_DE` | `de-de-default` | ★★★★★ (277,266) | Standard aggregate | [German](german.md) | +| Gothic | `GOT` | `got-default` | ★★★★★ (134,162) | Individual artifact; outside standard aggregate | [Gothic](got.md) | +| Greek | `EL_GR` | `el-gr-default` | ★★★★★ (76,851) | Individual artifact; outside standard aggregate | [Greek](el-gr.md) | +| Gulf Arabic | `AFB` | `afb-default` | ★☆☆☆☆ (355) | Individual artifact; outside standard aggregate | [Gulf Arabic](afb.md) | +| Haida | `HAI` | `hai-default` | ★★☆☆☆ (520) | Individual artifact; outside standard aggregate | [Haida](hai.md) | +| Hebrew | `HE_IL` | `he-il-default` | ★★★★☆ (57,658) | Standard aggregate | [Hebrew](hebrew.md) | +| Hiligaynon | `HIL` | `hil-default` | ★☆☆☆☆ (308) | Individual artifact; outside standard aggregate | [Hiligaynon](hil.md) | +| Hsilimo | `HSI` | `hsi-default` | ★☆☆☆☆ (158) | Individual artifact; outside standard aggregate | [Hsilimo](hsi.md) | +| Hungarian | `HU_HU` | `hu-hu-default` | ★★★★★ (910,688) | Standard aggregate | [Hungarian](hungarian.md) | +| Icelandic | `IS_IS` | `is-is-default` | ★★★★☆ (52,132) | Individual artifact; outside standard aggregate | [Icelandic](is-is.md) | +| Indonesian | `ID_ID` | `id-id-default` | ★★★★☆ (21,296) | Individual artifact; outside standard aggregate | [Indonesian](id-id.md) | +| Ingrian | `IZH` | `izh-default` | ★★☆☆☆ (1,024) | Individual artifact; outside standard aggregate | [Ingrian](izh.md) | +| Irish | `GA_IE` | `ga-ie-default` | ★★★★☆ (21,829) | Individual artifact; outside standard aggregate | [Irish](ga-ie.md) | +| Italian | `IT_IT` | `it-it-default` | ★★★★★ (324,366) | Standard aggregate | [Italian](italian.md) | +| Itelmen | `ITL` | `itl-default` | ★★★☆☆ (3,546) | Individual artifact; outside standard aggregate | [Itelmen](itl.md) | +| Japanese | `JA_JP` | `ja-jp-default` | ★★★★☆ (10,848) | Individual artifact; outside standard aggregate | [Japanese](ja-jp.md) | +| Kabardian | `KBD` | `kbd-default` | ★★★☆☆ (3,054) | Individual artifact; outside standard aggregate | [Kabardian](kbd.md) | +| Kalaallisut | `KL_GL` | `kl-gl-default` | ★☆☆☆☆ (321) | Individual artifact; outside standard aggregate | [Kalaallisut](kl-gl.md) | +| Kannada | `KN_IN` | `kn-in-default` | ★☆☆☆☆ (11) | Individual artifact; outside standard aggregate | [Kannada](kn-in.md) | +| Karelian | `KRL` | `krl-default` | ★★☆☆☆ (566) | Individual artifact; outside standard aggregate | [Karelian](krl.md) | +| Kashubian | `CSB` | `csb-default` | ★☆☆☆☆ (350) | Individual artifact; outside standard aggregate | [Kashubian](csb.md) | +| Kazakh | `KK_KZ` | `kk-kz-default` | ★☆☆☆☆ (357) | Individual artifact; outside standard aggregate | [Kazakh](kk-kz.md) | +| Khakas | `KJH` | `kjh-default` | ★★☆☆☆ (1,172) | Individual artifact; outside standard aggregate | [Khakas](kjh.md) | +| Khinalugh | `KLR` | `klr-default` | ★★★★☆ (41,185) | Individual artifact; outside standard aggregate | [Khinalugh](klr.md) | +| Kodi | `KOD` | `kod-default` | ★★☆☆☆ (524) | Individual artifact; outside standard aggregate | [Kodi](kod.md) | +| Kongo | `KON` | `kon-default` | ★★☆☆☆ (585) | Individual artifact; outside standard aggregate | [Kongo](kon.md) | +| Kyrgyz | `KY_KG` | `ky-kg-default` | ★★★☆☆ (2,997) | Individual artifact; outside standard aggregate | [Kyrgyz](ky-kg.md) | +| Ladin | `LLD` | `lld-default` | ★★★☆☆ (4,819) | Individual artifact; outside standard aggregate | [Ladin](lld.md) | +| Latin | `LA` | `la-default` | ★★★★★ (485,259) | Individual artifact; outside standard aggregate | [Latin](la.md) | +| Latvian | `LV_LV` | `lv-lv-default` | ★★★★★ (75,492) | Individual artifact; outside standard aggregate | [Latvian](lv-lv.md) | +| Lingala | `LIN` | `lin-default` | ★☆☆☆☆ (230) | Individual artifact; outside standard aggregate | [Lingala](lin.md) | +| Lithuanian | `LT_LT` | `lt-lt-default` | ★★★★☆ (27,335) | Individual artifact; outside standard aggregate | [Lithuanian](lt-lt.md) | +| Livonian | `LIV` | `liv-default` | ★★★☆☆ (2,841) | Individual artifact; outside standard aggregate | [Livonian](liv.md) | +| Low German | `NDS` | `nds-default` | ★★★☆☆ (2,493) | Individual artifact; outside standard aggregate | [Low German](nds.md) | +| Lower Sorbian | `DSB` | `dsb-default` | ★★★★☆ (12,038) | Individual artifact; outside standard aggregate | [Lower Sorbian](dsb.md) | +| Luganda | `LG_UG` | `lg-ug-default` | ★★★☆☆ (4,671) | Individual artifact; outside standard aggregate | [Luganda](lg-ug.md) | +| Macedonian | `MK_MK` | `mk-mk-default` | ★★★★★ (135,590) | Individual artifact; outside standard aggregate | [Macedonian](mk-mk.md) | +| Magahi | `MAG` | `mag-default` | ★☆☆☆☆ (206) | Individual artifact; outside standard aggregate | [Magahi](mag.md) | +| Malagasy | `MG_MG` | `mg-mg-default` | ★★☆☆☆ (636) | Individual artifact; outside standard aggregate | [Malagasy](mg-mg.md) | +| Maltese | `MT_MT` | `mt-mt-default` | ★★☆☆☆ (1,421) | Individual artifact; outside standard aggregate | [Maltese](mt-mt.md) | +| Manx | `GV_IM` | `gv-im-default` | ★☆☆☆☆ (15) | Individual artifact; outside standard aggregate | [Manx](gv-im.md) | +| Maori | `MI_NZ` | `mi-nz-default` | ★☆☆☆☆ (207) | Individual artifact; outside standard aggregate | [Maori](mi-nz.md) | +| Mapudungun | `ARN` | `arn-default` | ★★☆☆☆ (548) | Individual artifact; outside standard aggregate | [Mapudungun](arn.md) | +| Middle French | `FRM` | `frm-default` | ★★★★☆ (27,002) | Individual artifact; outside standard aggregate | [Middle French](frm.md) | +| Middle High German | `GMH` | `gmh-default` | ★☆☆☆☆ (384) | Individual artifact; outside standard aggregate | [Middle High German](gmh.md) | +| Middle Low German | `GML` | `gml-default` | ★★☆☆☆ (565) | Individual artifact; outside standard aggregate | [Middle Low German](gml.md) | +| Mongolian | `MN_MN` | `mn-mn-default` | ★★★★☆ (17,231) | Individual artifact; outside standard aggregate | [Mongolian](mn-mn.md) | +| Murrinh-Patha | `MWF` | `mwf-default` | ★★☆☆☆ (592) | Individual artifact; outside standard aggregate | [Murrinh-Patha](mwf.md) | +| Navajo | `NAV` | `nav-default` | ★★★☆☆ (9,826) | Individual artifact; outside standard aggregate | [Navajo](nav.md) | +| Neapolitan | `NAP` | `nap-default` | ★★☆☆☆ (1,497) | Individual artifact; outside standard aggregate | [Neapolitan](nap.md) | +| North Frisian | `FRR` | `frr-default` | ★☆☆☆☆ (340) | Individual artifact; outside standard aggregate | [North Frisian](frr.md) | +| Northern Sami | `SME` | `sme-default` | ★★★★☆ (53,034) | Individual artifact; outside standard aggregate | [Northern Sami](sme.md) | +| Norwegian Bokmål | `NB_NO` | `nb-no-default` | ★★★★★ (73,170) | Standard aggregate | [Norwegian Bokmål](norwegian-bokmal.md) | +| Norwegian Nynorsk | `NN_NO` | `nn-no-default` | ★★★★☆ (16,937) | Standard aggregate | [Norwegian Nynorsk](norwegian-nynorsk.md) | +| Old English | `ANG` | `ang-default` | ★★★★★ (65,100) | Individual artifact; outside standard aggregate | [Old English](ang.md) | +| Old French | `FRO` | `fro-default` | ★★★★★ (93,998) | Individual artifact; outside standard aggregate | [Old French](fro.md) | +| Old High German | `GOH` | `goh-default` | ★★★☆☆ (5,267) | Individual artifact; outside standard aggregate | [Old High German](goh.md) | +| Old Irish | `SGA` | `sga-default` | ★★☆☆☆ (929) | Individual artifact; outside standard aggregate | [Old Irish](sga.md) | +| Old Norse | `NON` | `non-default` | ★★★★☆ (46,067) | Individual artifact; outside standard aggregate | [Old Norse](non.md) | +| Old Saxon | `OSX` | `osx-default` | ★★★★☆ (12,361) | Individual artifact; outside standard aggregate | [Old Saxon](osx.md) | +| Oodham | `OOD` | `ood-default` | ★☆☆☆☆ (455) | Individual artifact; outside standard aggregate | [Oodham](ood.md) | +| Pashto | `PS_AF` | `ps-af-default` | ★★★☆☆ (2,945) | Individual artifact; outside standard aggregate | [Pashto](ps-af.md) | +| Persian | `FA_IR` | `fa-ir-default` | ★★★☆☆ (3,544) | Standard aggregate | [Persian](persian.md) | +| Polish | `PL_PL` | `pl-pl-unimorph`; optional `pl-pl-polimorf` (★★★★★) | ★★★★★ (120,867) | Standard aggregate; optional model is individual only | [Polish](polish.md) | +| Portuguese | `PT_PT` | `pt-pt-default` | ★★★★★ (211,091) | Standard aggregate | [Portuguese](portuguese.md) | +| Quechua | `QUE` | `que-default` | ★★★★★ (122,820) | Individual artifact; outside standard aggregate | [Quechua](que.md) | +| Romanian | `RO_RO` | `ro-ro-default` | ★★★★☆ (48,497) | Individual artifact; outside standard aggregate | [Romanian](ro-ro.md) | +| Russian | `RU_RU` | `ru-ru-default` | ★★★★★ (759,333) | Standard aggregate | [Russian](russian.md) | +| Seneca | `SEE` | `see-default` | ★☆☆☆☆ (483) | Individual artifact; outside standard aggregate | [Seneca](see.md) | +| Serbo-Croatian | `HBS` | `hbs-default` | ★★★★★ (272,515) | Individual artifact; outside standard aggregate | [Serbo-Croatian](hbs.md) | +| Shipibo-Conibo | `SHP` | `shp-default` | ★★★☆☆ (7,705) | Individual artifact; outside standard aggregate | [Shipibo-Conibo](shp.md) | +| Shona | `SN_ZW` | `sn-zw-default` | ★★★☆☆ (2,640) | Individual artifact; outside standard aggregate | [Shona](sn-zw.md) | +| Sotho, Southern | `ST_ZA` | `st-za-default` | ★☆☆☆☆ (416) | Individual artifact; outside standard aggregate | [Sotho, Southern](st-za.md) | +| Southern Kurdish | `SDH` | `sdh-default` | ★☆☆☆☆ (165) | Individual artifact; outside standard aggregate | [Southern Kurdish](sdh.md) | +| Spanish | `ES_ES` | `es-es-default` | ★★★★★ (849,661) | Standard aggregate | [Spanish](spanish.md) | +| Swedish | `SV_SE` | `sv-se-default` | ★★★★★ (95,181) | Standard aggregate | [Swedish](swedish.md) | +| Tagalog | `TL_PH` | `tl-ph-default` | ★★★☆☆ (2,042) | Individual artifact; outside standard aggregate | [Tagalog](tl-ph.md) | +| Temoaya Otomi | `OTE` | `ote-default` | ★★☆☆☆ (1,785) | Individual artifact; outside standard aggregate | [Temoaya Otomi](ote.md) | +| Turkish | `TR_TR` | `tr-tr-default` | ★★★★★ (222,207) | Individual artifact; outside standard aggregate | [Turkish](tr-tr.md) | +| Ukrainian | `UK_UA` | `uk-ua-default` | ★★★★☆ (14,150) | Standard aggregate | [Ukrainian](ukrainian.md) | +| Uyghur | `UG_CN` | `ug-cn-default` | ★★★☆☆ (6,200) | Individual artifact; outside standard aggregate | [Uyghur](ug-cn.md) | +| Uzbek | `UZ_UZ` | `uz-uz-default` | ★★☆☆☆ (1,080) | Individual artifact; outside standard aggregate | [Uzbek](uz-uz.md) | +| Voro | `VRO` | `vro-default` | ★☆☆☆☆ (277) | Individual artifact; outside standard aggregate | [Voro](vro.md) | +| Western Highland Chatino | `CTP` | `ctp-default` | ★★★☆☆ (2,456) | Individual artifact; outside standard aggregate | [Western Highland Chatino](ctp.md) | +| Xibe | `SJO` | `sjo-default` | ★★★☆☆ (3,151) | Individual artifact; outside standard aggregate | [Xibe](sjo.md) | +| Xipinawa | `XTY` | `xty-default` | ★★☆☆☆ (1,536) | Individual artifact; outside standard aggregate | [Xipinawa](xty.md) | +| Yamana | `AME` | `ame-default` | ★★☆☆☆ (1,489) | Individual artifact; outside standard aggregate | [Yamana](ame.md) | +| Yiddish | `YI` | `yi-default` | ★★★☆☆ (3,532) | Standard aggregate | [Yiddish](yiddish.md) | +| Zacatepec Chatino | `ZPV` | `zpv-default` | ★★☆☆☆ (812) | Individual artifact; outside standard aggregate | [Zacatepec Chatino](zpv.md) | +| Zarma | `DJE` | `dje-default` | ★☆☆☆☆ (75) | Individual artifact; outside standard aggregate | [Zarma](dje.md) | +| Zenzontepec Chatino | `CZN` | `czn-default` | ★☆☆☆☆ (68) | Individual artifact; outside standard aggregate | [Zenzontepec Chatino](czn.md) | +| Zulu | `ZU_ZA` | `zu-za-default` | ★★★★☆ (32,384) | Individual artifact; outside standard aggregate | [Zulu](zu-za.md) | ## Methodology Notes diff --git a/docs/benchmarks/languages/italian.md b/docs/benchmarks/languages/italian.md index 0a7b806..2e62bc0 100644 --- a/docs/benchmarks/languages/italian.md +++ b/docs/benchmarks/languages/italian.md @@ -1,8 +1,12 @@ -# Italian Stemmer Benchmarks +# Italian Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Italian. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **324,366 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `it-it-default` | `1.0.0` | `IT_IT` | 10,009 | 337,546 | 20,004 | 317,542 | 317,542 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `it-it-default` | `1.0.0` | `IT_IT` | 10,009 | 324,366 | 337,546 | 20,004 | 317,542 | changed tokens | 317,542 | ## Radixor Patch Command Distribution @@ -41,14 +45,14 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `italianRadixor` | 23.234 | 0.511 | 73.2 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene ItalianLightStemFilter | `italianLuceneItalianLightStemFilter` | 14.961 | 0.201 | 47.1 | 0.644 | Light Italian suffix stemmer. | -| Official Snowball direct | `snowballDirect[ITALIAN]` | 110.443 | 3.741 | 347.8 | 4.754 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[ITALIAN]` | 115.937 | 4.929 | 365.1 | 4.990 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `italianRadixor` | 24.797 | 0.581 | 78.1 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene ItalianLightStemFilter | `italianLuceneItalianLightStemFilter` | 16.243 | 0.373 | 51.2 | 0.655 | Light Italian suffix stemmer. | +| Official Snowball direct | `snowballDirect[ITALIAN]` | 110.766 | 5.309 | 348.8 | 4.467 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[ITALIAN]` | 112.716 | 3.913 | 355.0 | 4.545 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -86,18 +90,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **86.665%** - at 10% training knowledge to **93.600%** at 90%, a measured - **+6.935 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+6.518 pp** and - preservation of unseen already-root forms changes by **+0.105 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **86.665%** at 10% training knowledge to **93.600%** at 90%, a measured **+6.935 percentage-point** change. +- Unseen all-form exactness moves from **87.447%** at 10% training knowledge to **93.965%** at 90%, a measured **+6.518 percentage-point** change. +- Preservation of unseen already-root forms moves from **99.794%** at 10% training knowledge to **99.900%** at 90%, a measured **+0.105 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -190,7 +191,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `IT_IT` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -445,8 +446,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `IT_IT` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/norwegian-bokmal.md b/docs/benchmarks/languages/norwegian-bokmal.md index 5a5baa1..f157622 100644 --- a/docs/benchmarks/languages/norwegian-bokmal.md +++ b/docs/benchmarks/languages/norwegian-bokmal.md @@ -1,8 +1,12 @@ -# Norwegian Bokmal Stemmer Benchmarks +# Norwegian Bokmal Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Norwegian Bokmal. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **73,170 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `nb-no-default` | `1.0.0` | `NB_NO` | 17,929 | 90,757 | 33,376 | 57,381 | 57,381 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `nb-no-default` | `1.0.0` | `NB_NO` | 17,929 | 73,170 | 90,757 | 33,376 | 57,381 | changed tokens | 57,381 | ## Radixor Patch Command Distribution @@ -43,15 +47,15 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `norwegianBokmalRadixor` | 3.315 | 0.067 | 57.8 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene NorwegianMinimalStemFilter | `norwegianBokmalLuceneNorwegianMinimalStemFilter` | 2.828 | 0.055 | 49.3 | 0.853 | Minimal Norwegian suffix reducer. | -| Lucene NorwegianLightStemFilter | `norwegianBokmalLuceneNorwegianLightStemFilter` | 3.260 | 0.093 | 56.8 | 0.983 | Light Norwegian suffix stemmer. | -| Official Snowball direct | `snowballDirect[NORWEGIAN_BOKMAL]` | 4.903 | 0.592 | 85.4 | 1.479 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_BOKMAL]` | 5.856 | 0.384 | 102.1 | 1.766 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `norwegianBokmalRadixor` | 3.217 | 0.114 | 56.1 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene NorwegianMinimalStemFilter | `norwegianBokmalLuceneNorwegianMinimalStemFilter` | 2.991 | 0.079 | 52.1 | 0.930 | Minimal Norwegian suffix reducer. | +| Lucene NorwegianLightStemFilter | `norwegianBokmalLuceneNorwegianLightStemFilter` | 3.389 | 0.059 | 59.1 | 1.054 | Light Norwegian suffix stemmer. | +| Official Snowball direct | `snowballDirect[NORWEGIAN_BOKMAL]` | 5.233 | 0.760 | 91.2 | 1.627 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_BOKMAL]` | 6.614 | 1.014 | 115.3 | 2.056 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -89,18 +93,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **58.598%** - at 10% training knowledge to **70.546%** at 90%, a measured - **+11.949 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+9.761 pp** and - preservation of unseen already-root forms changes by **+5.538 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **58.598%** at 10% training knowledge to **70.546%** at 90%, a measured **+11.949 percentage-point** change. +- Unseen all-form exactness moves from **69.855%** at 10% training knowledge to **79.616%** at 90%, a measured **+9.761 percentage-point** change. +- Preservation of unseen already-root forms moves from **89.971%** at 10% training knowledge to **95.510%** at 90%, a measured **+5.538 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -193,7 +194,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NB_NO` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -456,8 +457,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `NB_NO` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/norwegian-nynorsk.md b/docs/benchmarks/languages/norwegian-nynorsk.md index ba29b2e..0fe4438 100644 --- a/docs/benchmarks/languages/norwegian-nynorsk.md +++ b/docs/benchmarks/languages/norwegian-nynorsk.md @@ -1,8 +1,12 @@ -# Norwegian Nynorsk Stemmer Benchmarks +# Norwegian Nynorsk Stemmer Benchmarks ★★★★☆ This page reports same-language stemming benchmarks for Norwegian Nynorsk. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★☆. The exact count is **16,937 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `nn-no-default` | `1.0.0` | `NN_NO` | 4,688 | 19,651 | 6,089 | 13,562 | 13,562 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `nn-no-default` | `1.0.0` | `NN_NO` | 4,688 | 16,937 | 19,651 | 6,089 | 13,562 | changed tokens | 13,562 | ## Radixor Patch Command Distribution @@ -41,13 +45,13 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `radixor[NORWEGIAN_NYNORSK]` | 0.596 | 0.083 | 44.0 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Official Snowball direct | `snowballDirect[NORWEGIAN_NYNORSK]` | 1.130 | 0.142 | 83.3 | 1.895 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_NYNORSK]` | 1.306 | 0.146 | 96.3 | 2.191 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `radixor[NORWEGIAN_NYNORSK]` | 0.681 | 0.133 | 50.2 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Official Snowball direct | `snowballDirect[NORWEGIAN_NYNORSK]` | 1.197 | 0.171 | 88.2 | 1.757 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[NORWEGIAN_NYNORSK]` | 1.461 | 0.255 | 107.7 | 2.145 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -85,18 +89,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **51.191%** - at 10% training knowledge to **70.338%** at 90%, a measured - **+19.147 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+15.413 pp** and - preservation of unseen already-root forms changes by **+4.683 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **51.191%** at 10% training knowledge to **70.338%** at 90%, a measured **+19.147 percentage-point** change. +- Unseen all-form exactness moves from **63.195%** at 10% training knowledge to **78.608%** at 90%, a measured **+15.413 percentage-point** change. +- Preservation of unseen already-root forms moves from **90.470%** at 10% training knowledge to **95.153%** at 90%, a measured **+4.683 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -189,7 +190,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `NN_NO` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -436,8 +437,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `NN_NO` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/persian.md b/docs/benchmarks/languages/persian.md index 7eac6ea..7e575d0 100644 --- a/docs/benchmarks/languages/persian.md +++ b/docs/benchmarks/languages/persian.md @@ -1,8 +1,12 @@ -# Persian Stemmer Benchmarks +# Persian Stemmer Benchmarks ★★★☆☆ This page reports same-language stemming benchmarks for Persian. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★☆☆. The exact count is **3,544 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `fa-ir-default` | `1.0.0` | `FA_IR` | 69 | 3,770 | 138 | 3,632 | 5,000 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `fa-ir-default` | `1.0.0` | `FA_IR` | 69 | 3,544 | 3,770 | 138 | 3,632 | changed tokens | 5,000 | ## Radixor Patch Command Distribution @@ -41,13 +45,13 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `persianRadixor` | 0.265 | 0.006 | 53.0 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene PersianStemFilter | `persianLucenePersianStemFilter` | 0.458 | 0.008 | 91.6 | 1.728 | Persian suffix stemmer with Lucene normalization in the measured path. | -| Official Snowball direct | `snowballDirect[PERSIAN]` | 1.531 | 0.073 | 306.2 | 5.777 | Official Snowball 3.1.0 generated Java stemmer; direct API. | +| Radixor | `persianRadixor` | 0.280 | 0.007 | 56.1 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene PersianStemFilter | `persianLucenePersianStemFilter` | 0.477 | 0.009 | 95.4 | 1.702 | Persian suffix stemmer with Lucene normalization in the measured path. | +| Official Snowball direct | `snowballDirect[PERSIAN]` | 1.511 | 0.089 | 302.3 | 5.392 | Official Snowball 3.1.0 generated Java stemmer; direct API. | ## Interpretation Notes @@ -85,18 +89,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **14.842%** - at 10% training knowledge to **15.493%** at 90%, a measured - **+0.651 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+0.712 pp** and - preservation of unseen already-root forms changes by **+0.000 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **14.842%** at 10% training knowledge to **15.493%** at 90%, a measured **+0.651 percentage-point** change. +- Unseen all-form exactness moves from **17.987%** at 10% training knowledge to **18.699%** at 90%, a measured **+0.712 percentage-point** change. +- Preservation of unseen already-root forms moves from **100.000%** at 10% training knowledge to **100.000%** at 90%, a measured **+0.000 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -189,7 +190,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `FA_IR` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -436,8 +437,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `FA_IR` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/polish.md b/docs/benchmarks/languages/polish.md index 740695f..fd3861c 100644 --- a/docs/benchmarks/languages/polish.md +++ b/docs/benchmarks/languages/polish.md @@ -1,8 +1,12 @@ -# Polish Stemmer Benchmarks +# Polish Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Polish. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **120,867 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `pl-pl-unimorph` | `1.0.0` | `PL_PL` | 9,990 | 132,308 | 19,957 | 112,351 | 112,351 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `pl-pl-unimorph` | `1.0.0` | `PL_PL` | 9,990 | 120,867 | 132,308 | 19,957 | 112,351 | changed tokens | 112,351 | ## Radixor Patch Command Distribution @@ -44,16 +48,16 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `polishRadixor` | 8.470 | 0.176 | 75.4 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 537.060 | 32.708 | 4780.2 | 63.408 | Benchmark-only Polish Hunspell dictionary compared via Lucene HunspellStemFilter. | -| Lucene StempelStemmer direct | `polishLuceneStempelStemmerDirect` | 33.209 | 0.662 | 295.6 | 3.921 | Direct table-driven Polish Stempel stemmer API. | -| Lucene StempelFilter | `polishLuceneStempelFilter` | 41.041 | 0.656 | 365.3 | 4.846 | Lucene TokenFilter integration path for table-driven Polish Stempel. | -| Lucene MorfologikFilter | `polishLuceneMorfologikFilter` | 140.092 | 1.931 | 1246.9 | 16.540 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. | -| Official Snowball direct | `snowballDirect[POLISH]` | 10.168 | 1.442 | 90.5 | 1.201 | Official Snowball 3.1.0 generated Java stemmer; direct API. | +| Radixor | `polishRadixor` | 8.121 | 0.140 | 72.3 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 544.375 | 51.396 | 4845.3 | 67.036 | Benchmark-only Polish Hunspell dictionary compared via Lucene HunspellStemFilter. | +| Lucene StempelStemmer direct | `polishLuceneStempelStemmerDirect` | 33.200 | 1.703 | 295.5 | 4.088 | Direct table-driven Polish Stempel stemmer API. | +| Lucene StempelFilter | `polishLuceneStempelFilter` | 40.165 | 0.872 | 357.5 | 4.946 | Lucene TokenFilter integration path for table-driven Polish Stempel. | +| Lucene MorfologikFilter | `polishLuceneMorfologikFilter` | 145.100 | 7.257 | 1291.5 | 17.868 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. | +| Official Snowball direct | `snowballDirect[POLISH]` | 10.920 | 1.787 | 97.2 | 1.345 | Official Snowball 3.1.0 generated Java stemmer; direct API. | ## Interpretation Notes @@ -63,6 +67,37 @@ Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 in - Morfologik rows are dictionary-based and can emit multiple terms for one input token. Quality rows use the first returned term when no ranking weight is available. - Snowball rows are rule-based generated suffix stemmers; they are useful algorithmic baselines, not dictionary-root equivalence guarantees. + + +## Optional PoliMorf Model + +PoliMorf is an independently available optional Polish model and is not the `PL_PL` default or a member of either standard aggregate. Its evidence is kept separate from the UniMorph default above. + +Optional dictionary size: ★★★★★. The exact count is **4,668,685 distinct usable word forms**; the rating is relative size, not linguistic quality. + +### PoliMorf Corpus + +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `pl-pl-polimorf` | `1.0.0` | `PL_PL` | 315,639 | 4,668,685 | 4,812,023 | 315,639 | 4,496,384 | changed tokens | 4,496,384 | + +### PoliMorf Exact-root Accuracy + +| Stemmer | All exact | Changed exact | Root preserved | Note | +| --- | ---: | ---: | ---: | --- | +| Radixor PoliMorf | 98.837% | 98.744% | 99.359% | Exact optional model. | +| PoliMorf Lucene MorfologikFilter | 87.729% | 86.606% | 94.047% | Matching Polish dictionary adapter. | +| PoliMorf Official Snowball direct | 22.315% | 20.225% | 34.078% | Matching Polish Snowball algorithm. | + +### PoliMorf Speed + +| Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor PoliMorf | Note | +| --- | --- | ---: | ---: | ---: | ---: | --- | +| Radixor PoliMorf | `radixor[pl-pl-polimorf]` | 802.691 | 12.562 | 178.5 | 1.000 | Exact optional-model baseline. | +| Lucene MorfologikFilter | `polishPolimorfLuceneMorfologikFilter` | 6199.653 | 99.810 | 1378.8 | 7.724 | PoliMorf-derived corpus. | + + + ## Dictionary-Family Generalization Conclusion @@ -91,18 +126,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **61.996%** - at 10% training knowledge to **69.972%** at 90%, a measured - **+7.976 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+7.307 pp** and - preservation of unseen already-root forms changes by **+4.352 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **61.996%** at 10% training knowledge to **69.972%** at 90%, a measured **+7.976 percentage-point** change. +- Unseen all-form exactness moves from **65.763%** at 10% training knowledge to **73.070%** at 90%, a measured **+7.307 percentage-point** change. +- Preservation of unseen already-root forms moves from **86.633%** at 10% training knowledge to **90.985%** at 90%, a measured **+4.352 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -195,7 +227,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `PL_PL` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -494,8 +526,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `PL_PL` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/portuguese.md b/docs/benchmarks/languages/portuguese.md index db9d7a2..7ec13e0 100644 --- a/docs/benchmarks/languages/portuguese.md +++ b/docs/benchmarks/languages/portuguese.md @@ -1,8 +1,12 @@ -# Portuguese Stemmer Benchmarks +# Portuguese Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Portuguese. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **211,091 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `pt-pt-default` | `1.0.0` | `PT_PT` | 4,001 | 215,490 | 8,002 | 207,488 | 207,488 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `pt-pt-default` | `1.0.0` | `PT_PT` | 4,001 | 211,091 | 215,490 | 8,002 | 207,488 | changed tokens | 207,488 | ## Radixor Patch Command Distribution @@ -44,16 +48,16 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `portugueseRadixor` | 11.444 | 0.204 | 55.2 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene PortugueseLightStemFilter | `portugueseLucenePortugueseLightStemFilter` | 10.557 | 0.211 | 50.9 | 0.923 | Light Portuguese suffix stemmer. | -| Lucene PortugueseMinimalStemFilter | `portugueseLucenePortugueseMinimalStemFilter` | 14.744 | 0.203 | 71.1 | 1.288 | Minimal Portuguese suffix reducer. | -| Official Snowball direct | `snowballDirect[PORTUGUESE]` | 51.852 | 2.250 | 249.9 | 4.531 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[PORTUGUESE]` | 62.553 | 1.666 | 301.5 | 5.466 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | -| Lucene PortugueseStemFilter | `portugueseLucenePortugueseStemFilter` | 144.379 | 3.988 | 695.8 | 12.616 | Portuguese RSLP-style Lucene TokenFilter. | +| Radixor | `portugueseRadixor` | 10.856 | 0.302 | 52.3 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene PortugueseLightStemFilter | `portugueseLucenePortugueseLightStemFilter` | 10.333 | 0.232 | 49.8 | 0.952 | Light Portuguese suffix stemmer. | +| Lucene PortugueseMinimalStemFilter | `portugueseLucenePortugueseMinimalStemFilter` | 14.489 | 0.250 | 69.8 | 1.335 | Minimal Portuguese suffix reducer. | +| Official Snowball direct | `snowballDirect[PORTUGUESE]` | 51.535 | 2.448 | 248.4 | 4.747 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[PORTUGUESE]` | 59.921 | 4.127 | 288.8 | 5.520 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Lucene PortugueseStemFilter | `portugueseLucenePortugueseStemFilter` | 145.578 | 9.659 | 701.6 | 13.410 | Portuguese RSLP-style Lucene TokenFilter. | ## Interpretation Notes @@ -91,18 +95,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **90.053%** - at 10% training knowledge to **94.412%** at 90%, a measured - **+4.359 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+4.197 pp** and - preservation of unseen already-root forms changes by **+0.167 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **90.053%** at 10% training knowledge to **94.412%** at 90%, a measured **+4.359 percentage-point** change. +- Unseen all-form exactness moves from **90.423%** at 10% training knowledge to **94.620%** at 90%, a measured **+4.197 percentage-point** change. +- Preservation of unseen already-root forms moves from **99.833%** at 10% training knowledge to **100.000%** at 90%, a measured **+0.167 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -195,7 +196,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `PT_PT` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -466,8 +467,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `PT_PT` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/russian.md b/docs/benchmarks/languages/russian.md index 83dbe9d..6adca5a 100644 --- a/docs/benchmarks/languages/russian.md +++ b/docs/benchmarks/languages/russian.md @@ -1,8 +1,12 @@ -# Russian Stemmer Benchmarks +# Russian Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Russian. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **759,333 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `ru-ru-default` | `1.0.0` | `RU_RU` | 37,410 | 806,279 | 74,808 | 731,471 | 731,471 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `ru-ru-default` | `1.0.0` | `RU_RU` | 37,410 | 759,333 | 806,279 | 74,808 | 731,471 | changed tokens | 731,471 | ## Radixor Patch Command Distribution @@ -42,14 +46,14 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `russianRadixor` | 78.162 | 2.240 | 106.9 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene RussianLightStemFilter | `russianLuceneRussianLightStemFilter` | 57.502 | 2.358 | 78.6 | 0.736 | Light Russian suffix stemmer. | -| Official Snowball direct | `snowballDirect[RUSSIAN]` | 107.685 | 11.649 | 147.2 | 1.378 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[RUSSIAN]` | 133.100 | 6.117 | 182.0 | 1.703 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `russianRadixor` | 75.341 | 2.119 | 103.0 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene RussianLightStemFilter | `russianLuceneRussianLightStemFilter` | 56.914 | 3.211 | 77.8 | 0.755 | Light Russian suffix stemmer. | +| Official Snowball direct | `snowballDirect[RUSSIAN]` | 119.356 | 12.591 | 163.2 | 1.584 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[RUSSIAN]` | 141.677 | 16.761 | 193.7 | 1.880 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -87,18 +91,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **74.948%** - at 10% training knowledge to **80.744%** at 90%, a measured - **+5.796 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+5.534 pp** and - preservation of unseen already-root forms changes by **+3.238 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **74.948%** at 10% training knowledge to **80.744%** at 90%, a measured **+5.796 percentage-point** change. +- Unseen all-form exactness moves from **76.473%** at 10% training knowledge to **82.007%** at 90%, a measured **+5.534 percentage-point** change. +- Preservation of unseen already-root forms moves from **91.268%** at 10% training knowledge to **94.506%** at 90%, a measured **+3.238 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -191,7 +192,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `RU_RU` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -446,8 +447,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `RU_RU` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/spanish.md b/docs/benchmarks/languages/spanish.md index 4b80d68..c384913 100644 --- a/docs/benchmarks/languages/spanish.md +++ b/docs/benchmarks/languages/spanish.md @@ -1,8 +1,12 @@ -# Spanish Stemmer Benchmarks +# Spanish Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Spanish. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **849,661 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `es-es-default` | `1.0.0` | `ES_ES` | 65,059 | 926,393 | 120,121 | 806,272 | 806,272 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `es-es-default` | `1.0.0` | `ES_ES` | 65,059 | 849,661 | 926,393 | 120,121 | 806,272 | changed tokens | 806,272 | ## Radixor Patch Command Distribution @@ -45,17 +49,17 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `spanishRadixor` | 70.807 | 1.533 | 87.8 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 1983.491 | 40.315 | 2460.1 | 28.013 | Benchmark-only Spanish Hunspell dictionary compared via Lucene HunspellStemFilter. | -| Lucene SpanishMinimalStemFilter | `spanishLuceneSpanishMinimalStemFilter` | 39.667 | 1.296 | 49.2 | 0.560 | Minimal Spanish suffix reducer; narrow baseline. | -| Lucene SpanishLightStemFilter | `spanishLuceneSpanishLightStemFilter` | 42.394 | 1.768 | 52.6 | 0.599 | Light Spanish suffix stemmer. | -| Lucene SpanishPluralStemFilter | `spanishLuceneSpanishPluralStemFilter` | 93.092 | 4.240 | 115.5 | 1.315 | Plural-oriented Spanish suffix reducer. | -| Official Snowball direct | `snowballDirect[SPANISH]` | 180.797 | 8.287 | 224.2 | 2.553 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[SPANISH]` | 201.614 | 17.917 | 250.1 | 2.847 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `spanishRadixor` | 64.311 | 0.840 | 79.8 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 2038.148 | 88.781 | 2527.9 | 31.692 | Benchmark-only Spanish Hunspell dictionary compared via Lucene HunspellStemFilter. | +| Lucene SpanishMinimalStemFilter | `spanishLuceneSpanishMinimalStemFilter` | 40.128 | 2.577 | 49.8 | 0.624 | Minimal Spanish suffix reducer; narrow baseline. | +| Lucene SpanishLightStemFilter | `spanishLuceneSpanishLightStemFilter` | 41.989 | 2.354 | 52.1 | 0.653 | Light Spanish suffix stemmer. | +| Lucene SpanishPluralStemFilter | `spanishLuceneSpanishPluralStemFilter` | 93.070 | 6.243 | 115.4 | 1.447 | Plural-oriented Spanish suffix reducer. | +| Official Snowball direct | `snowballDirect[SPANISH]` | 186.104 | 10.262 | 230.8 | 2.894 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[SPANISH]` | 206.679 | 23.811 | 256.3 | 3.214 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -93,18 +97,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **80.995%** - at 10% training knowledge to **89.176%** at 90%, a measured - **+8.181 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+7.735 pp** and - preservation of unseen already-root forms changes by **+5.348 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **80.995%** at 10% training knowledge to **89.176%** at 90%, a measured **+8.181 percentage-point** change. +- Unseen all-form exactness moves from **82.066%** at 10% training knowledge to **89.801%** at 90%, a measured **+7.735 percentage-point** change. +- Preservation of unseen already-root forms moves from **89.364%** at 10% training knowledge to **94.711%** at 90%, a measured **+5.348 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -197,7 +198,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `ES_ES` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -490,8 +491,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `ES_ES` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/swedish.md b/docs/benchmarks/languages/swedish.md index 5421b6d..75fbcf0 100644 --- a/docs/benchmarks/languages/swedish.md +++ b/docs/benchmarks/languages/swedish.md @@ -1,8 +1,12 @@ -# Swedish Stemmer Benchmarks +# Swedish Stemmer Benchmarks ★★★★★ This page reports same-language stemming benchmarks for Swedish. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★★. The exact count is **95,181 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `sv-se-default` | `1.0.0` | `SV_SE` | 12,371 | 110,468 | 24,731 | 85,737 | 85,737 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `sv-se-default` | `1.0.0` | `SV_SE` | 12,371 | 95,181 | 110,468 | 24,731 | 85,737 | changed tokens | 85,737 | ## Radixor Patch Command Distribution @@ -43,15 +47,15 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `swedishRadixor` | 5.232 | 0.112 | 61.0 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene SwedishMinimalStemFilter | `swedishLuceneSwedishMinimalStemFilter` | 4.504 | 0.065 | 52.5 | 0.861 | Minimal Swedish suffix reducer. | -| Lucene SwedishLightStemFilter | `swedishLuceneSwedishLightStemFilter` | 4.682 | 0.092 | 54.6 | 0.895 | Light Swedish suffix stemmer. | -| Official Snowball direct | `snowballDirect[SWEDISH]` | 8.027 | 0.979 | 93.6 | 1.534 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[SWEDISH]` | 10.296 | 0.977 | 120.1 | 1.968 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `swedishRadixor` | 4.890 | 0.124 | 57.0 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene SwedishMinimalStemFilter | `swedishLuceneSwedishMinimalStemFilter` | 4.436 | 0.130 | 51.7 | 0.907 | Minimal Swedish suffix reducer. | +| Lucene SwedishLightStemFilter | `swedishLuceneSwedishLightStemFilter` | 4.668 | 0.198 | 54.4 | 0.955 | Light Swedish suffix stemmer. | +| Official Snowball direct | `snowballDirect[SWEDISH]` | 8.255 | 1.129 | 96.3 | 1.688 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[SWEDISH]` | 11.391 | 1.699 | 132.9 | 2.329 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -89,18 +93,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **67.214%** - at 10% training knowledge to **78.458%** at 90%, a measured - **+11.244 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+10.177 pp** and - preservation of unseen already-root forms changes by **+7.137 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **67.214%** at 10% training knowledge to **78.458%** at 90%, a measured **+11.244 percentage-point** change. +- Unseen all-form exactness moves from **72.010%** at 10% training knowledge to **82.188%** at 90%, a measured **+10.177 percentage-point** change. +- Preservation of unseen already-root forms moves from **87.789%** at 10% training knowledge to **94.926%** at 90%, a measured **+7.137 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -194,7 +195,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `SV_SE` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -457,8 +458,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `SV_SE` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/ukrainian.md b/docs/benchmarks/languages/ukrainian.md index b86fd23..7fc60e1 100644 --- a/docs/benchmarks/languages/ukrainian.md +++ b/docs/benchmarks/languages/ukrainian.md @@ -1,8 +1,12 @@ -# Ukrainian Stemmer Benchmarks +# Ukrainian Stemmer Benchmarks ★★★★☆ This page reports same-language stemming benchmarks for Ukrainian. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★★☆. The exact count is **14,150 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `uk-ua-default` | `1.0.0` | `UK_UA` | 1,493 | 15,737 | 2,985 | 12,752 | 12,752 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `uk-ua-default` | `1.0.0` | `UK_UA` | 1,493 | 14,150 | 15,737 | 2,985 | 12,752 | changed tokens | 12,752 | ## Radixor Patch Command Distribution @@ -42,14 +46,14 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `ukrainianRadixor` | 0.625 | 0.017 | 49.0 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 49.515 | 6.147 | 3883.0 | 79.197 | Benchmark-only Ukrainian Hunspell dictionary compared via Lucene HunspellStemFilter. | -| Morfologik direct | `ukrainianMorfologikDirect` | 8.299 | 0.124 | 650.8 | 13.275 | Direct Morfologik dictionary lookup; first returned stem is used for quality. | -| Lucene MorfologikFilter | `ukrainianLuceneMorfologikFilter` | 14.580 | 0.206 | 1143.4 | 23.320 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. | +| Radixor | `ukrainianRadixor` | 0.620 | 0.011 | 48.6 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Lucene HunspellStemFilter | `luceneHunspellStemFilter` | 55.033 | 8.824 | 4315.7 | 88.803 | Benchmark-only Ukrainian Hunspell dictionary compared via Lucene HunspellStemFilter. | +| Morfologik direct | `ukrainianMorfologikDirect` | 8.224 | 0.157 | 644.9 | 13.270 | Direct Morfologik dictionary lookup; first returned stem is used for quality. | +| Lucene MorfologikFilter | `ukrainianLuceneMorfologikFilter` | 14.808 | 0.346 | 1161.2 | 23.895 | Dictionary-based Morfologik TokenFilter; may emit multiple terms. | ## Interpretation Notes @@ -87,18 +91,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **42.530%** - at 10% training knowledge to **57.218%** at 90%, a measured - **+14.689 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+12.251 pp** and - preservation of unseen already-root forms changes by **+5.012 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **42.530%** at 10% training knowledge to **57.218%** at 90%, a measured **+14.689 percentage-point** change. +- Unseen all-form exactness moves from **50.982%** at 10% training knowledge to **63.233%** at 90%, a measured **+12.251 percentage-point** change. +- Preservation of unseen already-root forms moves from **85.464%** at 10% training knowledge to **90.476%** at 90%, a measured **+5.012 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -190,7 +191,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `UK_UA` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -487,8 +488,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `UK_UA` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/languages/yiddish.md b/docs/benchmarks/languages/yiddish.md index 948c0d7..b4a61cc 100644 --- a/docs/benchmarks/languages/yiddish.md +++ b/docs/benchmarks/languages/yiddish.md @@ -1,8 +1,12 @@ -# Yiddish Stemmer Benchmarks +# Yiddish Stemmer Benchmarks ★★★☆☆ This page reports same-language stemming benchmarks for Yiddish. Accuracy is listed first because speed without root agreement is not enough to interpret stemmer quality. -All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-08-25 Radixor/Java `4.2.0-6-g84e57fb` snapshot. Speed benchmark operations process changed dictionary tokens only. Accuracy uses the complete Radixor dictionary for the language. + +Dictionary size: ★★★☆☆. The exact count is **3,532 distinct usable word forms** after parser-compatible filtering and exact, case-preserved deduplication. Stars rank dictionary size relative to all 167 user-facing models in five nearly equal groups; they do **not** measure linguistic quality or benchmark accuracy. + + +All speed values are environment-specific and were measured on the hardware and JVM listed in the [benchmark overview](../index.md). The command distribution, exact-root accuracy, and speed tables belong to the published 2026-09-11 Radixor/Java `4.3.0.dirty` snapshot. Speed benchmark operations process changed tokens. Accuracy uses the complete Radixor dictionary for the language. !!! info "How to read this page" @@ -13,9 +17,9 @@ Runtime and exact-root agreement measure different properties. Light, minimal, p ## Dictionary Corpus -| Model ID | Model version | Language | Dictionary rows | Complete quality tokens | Already-root tokens | Changed tokens | JMH timing tokens | -| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `yi-default` | `1.0.0` | `YI` | 802 | 4,300 | 1,524 | 2,776 | 5,000 | +| Model ID | Model version | Language | Dictionary rows | Distinct usable forms | Complete quality tokens | Already-root tokens | Changed tokens | Timing workload | JMH timing tokens | +| --- | --- | --- | ---: | ---: | ---: | ---: | ---: | --- | ---: | +| `yi-default` | `1.0.0` | `YI` | 802 | 3,532 | 4,300 | 1,524 | 2,776 | changed tokens | 5,000 | ## Radixor Patch Command Distribution @@ -41,13 +45,13 @@ Accuracy is computed from JMH auxiliary counters in the current report. The coun ## Speed -Speed uses JMH average time, 5 warmup iterations, 7 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. +Speed uses JMH average time, 3 warmup iterations, 5 measurement iterations, 3 independent forks, and 1 thread. Relative factor is computed against the single Radixor row on this language page. Values below 1.000 are faster than that Radixor baseline; values above 1.000 are slower. | Stemmer | Benchmark method | Score ms/op | Error ms | ns/token | Relative vs Radixor | Note | | --- | --- | ---: | ---: | ---: | ---: | --- | -| Radixor | `radixor[YIDDISH]` | 0.214 | 0.002 | 42.8 | 1.000 | Radixor dictionary-trained patch-command stemmer. | -| Official Snowball direct | `snowballDirect[YIDDISH]` | 1.506 | 0.074 | 301.3 | 7.044 | Official Snowball generated Java stemmer; direct API. | -| Lucene SnowballFilter | `luceneSnowballFilter[YIDDISH]` | 1.744 | 0.094 | 348.8 | 8.155 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | +| Radixor | `radixor[YIDDISH]` | 0.257 | 0.080 | 51.3 | 1.000 | Radixor dictionary-trained patch-command stemmer. | +| Official Snowball direct | `snowballDirect[YIDDISH]` | 1.523 | 0.096 | 304.6 | 5.936 | Official Snowball generated Java stemmer; direct API. | +| Lucene SnowballFilter | `luceneSnowballFilter[YIDDISH]` | 1.709 | 0.113 | 341.7 | 6.659 | Lucene TokenFilter path around Snowball; includes TokenStream overhead. | ## Interpretation Notes @@ -85,18 +89,15 @@ also appeared in training. Parentheses show the observed split minimum–maximum ### Generalization conclusion -- Median exactness on genuinely unseen changed forms moves from **37.767%** - at 10% training knowledge to **45.318%** at 90%, a measured - **+7.551 percentage-point** change for this dictionary. -- Over the same endpoints, unseen all-form exactness changes by **+4.158 pp** and - preservation of unseen already-root forms changes by **+3.541 pp**. These separate - outcomes show whether the changed-form result coexists with preservation behavior. +- Median exactness on genuinely unseen changed forms moves from **37.767%** at 10% training knowledge to **45.318%** at 90%, a measured **+7.551 percentage-point** change. +- Unseen all-form exactness moves from **56.133%** at 10% training knowledge to **60.291%** at 90%, a measured **+4.158 percentage-point** change. +- Preservation of unseen already-root forms moves from **89.466%** at 10% training knowledge to **93.007%** at 90%, a measured **+3.541 percentage-point** change. - The evidence establishes within-resource transfer across withheld dictionary families. It does not estimate unrelated domains, misspellings, arbitrary compounds, or external corpora. The complete ten-level table and split ranges remain in the [independent generalization report](../generalization.md); raw counters and provenance are in -[`dictionary-generalization.csv`](../data/dictionary-generalization.csv). The +[active machine-readable snapshot](../data/dictionary-generalization-2026-09-11.csv). The [frozen methodology](../reference/generalization-methodology.md) defines family-level splitting, unseen-surface leakage control, aggregation, and the limits of the claim. @@ -189,7 +190,7 @@ The complete evidence is available in the [raw logical matrix](../data/edit-cost Runtime performance and linguistic grouping quality are independent dimensions. This section evaluates language `YI` using the complete validated stemming-quality result matrix. Every distinct surface form is one evaluated item and can belong to several dictionary groups. Two forms are a positive pair when their group-membership sets intersect and a negative pair when those sets are disjoint. A pair shared through several groups is counted once. Exact equality with a predetermined lemma is not required. -`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality.csv). +`ALL_WORDS` includes every valid group and its original forms. `LOWERCASE_GROUPS_ONLY` excludes an entire group when any Unicode code point is uppercase or titlecase; retained words are not lowercased or otherwise rewritten. This isolates case-handling effects without changing retained inputs. [Download the complete machine-readable result snapshot](../data/stemming-quality-2026-09-11.csv). ### Evaluation Scope and Key Findings @@ -436,8 +437,8 @@ Standard ARI, homogeneity, completeness, V-measure, and NMI are not calculated: ### Provenance -- Authoritative source: `docs/benchmarks/data/stemming-quality.csv` -- Source SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` +- Authoritative source: `docs/benchmarks/data/stemming-quality-2026-09-11.csv` +- Source SHA-256: `24bddfeed06a60bb3eeed58e1bfc93aed46c1d32e7bebd293aec2dacfddfef5b` - Evaluation command: `./gradlew stemmingQuality --no-daemon` - Dictionary language: `YI` - Processing modes: `ALL_WORDS`, `LOWERCASE_GROUPS_ONLY` diff --git a/docs/benchmarks/reference/corpora.md b/docs/benchmarks/reference/corpora.md index 669bc15..98295a3 100644 --- a/docs/benchmarks/reference/corpora.md +++ b/docs/benchmarks/reference/corpora.md @@ -1,28 +1,175 @@ # Benchmark Corpora -The table below describes the Radixor resources used to build speed and quality corpora. `Total tokens` is the complete dictionary token count used by quality benchmarks. `Already-root tokens` counts fields where the token is already equal to the line root. `Changed tokens` is the speed workload before the minimum-size repeat rule. +The table below describes the Radixor resources used to build speed and quality corpora. `Total benchmark tokens` is the complete dictionary token count used by quality benchmarks. `Changed benchmark tokens` counts fields whose token differs from the line root. Timing normally uses that changed population; a root-only model instead uses all root-preservation tokens. `Timing workload` makes this choice explicit. -| Default model ID | Version | SHA-256 | Language | Dictionary rows | Total tokens | Already-root tokens | Changed tokens | Speed timing tokens | -| --- | --- | --- | --- | ---: | ---: | ---: | ---: | ---: | -| `cs-cz-default` | `1.0.0` | `62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4` | `CS_CZ` | 5,113 | 56,612 | 10,049 | 46,563 | 46,563 | -| `da-dk-default` | `1.0.0` | `3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90` | `DA_DK` | 4,179 | 32,256 | 8,356 | 23,900 | 23,900 | -| `nl-nl-default` | `1.0.0` | `c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e` | `NL_NL` | 4,992 | 31,466 | 9,981 | 21,485 | 21,485 | -| `us-uk-default` | `1.0.1` | `0a3ec1fddd5ddb859f5ffa8003efe9200a5ef6203df1fddaf53a58003f33582c` | `US_UK` | 396,939 | 1,002,414 | 793,874 | 208,540 | 208,540 | -| `fi-fi-default` | `1.0.0` | `ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f` | `FI_FI` | 57,027 | 1,865,215 | 110,525 | 1,754,690 | 1,754,690 | -| `fr-fr-default` | `1.0.0` | `a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9` | `FR_FR` | 59,240 | 474,110 | 108,141 | 365,969 | 365,969 | -| `de-de-default` | `1.0.0` | `cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5` | `DE_DE` | 54,092 | 333,036 | 90,535 | 242,501 | 242,501 | -| `he-il-default` | `1.0.0` | `9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256` | `HE_IL` | 2,358 | 61,071 | 4,715 | 56,356 | 56,356 | -| `hu-hu-default` | `1.0.0` | `359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e` | `HU_HU` | 19,406 | 935,713 | 38,775 | 896,938 | 896,938 | -| `it-it-default` | `1.0.0` | `5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12` | `IT_IT` | 10,009 | 337,546 | 20,004 | 317,542 | 317,542 | -| `nb-no-default` | `1.0.0` | `f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9` | `NB_NO` | 17,929 | 90,757 | 33,376 | 57,381 | 57,381 | -| `nn-no-default` | `1.0.0` | `900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398` | `NN_NO` | 4,688 | 19,651 | 6,089 | 13,562 | 13,562 | -| `fa-ir-default` | `1.0.0` | `b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1` | `FA_IR` | 69 | 3,770 | 138 | 3,632 | 5,000 | -| `pl-pl-unimorph` | `1.0.0` | `8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721` | `PL_PL` | 9,990 | 132,308 | 19,957 | 112,351 | 112,351 | -| `pt-pt-default` | `1.0.0` | `7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d` | `PT_PT` | 4,001 | 215,490 | 8,002 | 207,488 | 207,488 | -| `ru-ru-default` | `1.0.0` | `df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40` | `RU_RU` | 37,410 | 806,279 | 74,808 | 731,471 | 731,471 | -| `es-es-default` | `1.0.0` | `7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721` | `ES_ES` | 65,059 | 926,393 | 120,121 | 806,272 | 806,272 | -| `sv-se-default` | `1.0.0` | `d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c` | `SV_SE` | 12,371 | 110,468 | 24,731 | 85,737 | 85,737 | -| `uk-ua-default` | `1.0.0` | `cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae` | `UK_UA` | 1,493 | 15,737 | 2,985 | 12,752 | 12,752 | -| `yi-default` | `1.0.0` | `f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3` | `YI` | 802 | 4,300 | 1,524 | 2,776 | 5,000 | +| Model ID | Role | Version | SHA-256 | Language | Dictionary rows | Distinct usable forms | Total benchmark tokens | Changed benchmark tokens | Timing workload | Speed timing tokens | +| --- | --- | --- | --- | --- | ---: | ---: | ---: | ---: | --- | ---: | +| `ady-default` | `standalone` | `1.0.0` | `952fbbe1fc0a4f58e110d5ebed3848939c31b6f414199c0658398f369a84ba3d` | `ADY` | 1,634 | 20,334 | 21,988 | 18,720 | changed tokens | 18,720 | +| `af-za-default` | `standalone` | `1.0.0` | `2360e30adf722e9a8a6fd428427ce650ac6bd883c8f186e27c4f2c72d09400d9` | `AF_ZA` | 166,498 | 289,303 | 460,165 | 127,169 | changed tokens | 127,169 | +| `afb-default` | `standalone` | `1.0.0` | `8ad86d7a99e8c0a42c1f799f9b20caa5367e6b6504ef319a580a87b4533a78a9` | `AFB` | 198 | 355 | 553 | 157 | changed tokens | 5,000 | +| `ail-default` | `standalone` | `1.0.0` | `364ae64f3c6df69cc6ed22d892762ffe7c875651b08cfef95abfbb388e0ebacd` | `AIL` | 622 | 2,978 | 3,672 | 2,428 | changed tokens | 5,000 | +| `aka-default` | `standalone` | `1.0.0` | `364fbdf2e356f1c5474d8ff2c83666bd6b8eb5193983f53437d2e3c556fe3c02` | `AK` | 79 | 1,890 | 1,975 | 1,817 | changed tokens | 5,000 | +| `am-et-default` | `standalone` | `1.0.0` | `ff154b83f4e30c28811b0d05933d455c40d45e2264b985ca90fcb0ad655f2b4c` | `AM_ET` | 2,461 | 41,308 | 44,923 | 40,001 | changed tokens | 40,001 | +| `ame-default` | `standalone` | `1.0.0` | `4876a2f73d99478d1c7c7572729e817738246d2f3ace4f51f0284bda82430dc0` | `AME` | 212 | 1,489 | 1,711 | 1,287 | changed tokens | 5,000 | +| `ang-default` | `standalone` | `1.0.0` | `ed45aecbcb8e394df8739e6f48e9333383c9d542f900b2ad3e3c8c3d7dc4d996` | `ANG` | 7,135 | 65,100 | 74,638 | 60,368 | changed tokens | 60,368 | +| `ar-default` | `standalone` | `1.0.1` | `36fe6a7a80e922f423e80761e27f21c61d86cc7f1b2837a1989761af4d51c904` | `AR` | 12,815 | 452,974 | 460,343 | 447,528 | changed tokens | 447,528 | +| `arn-default` | `standalone` | `1.0.0` | `aa1472b03d2cf04094814484e2d57a908c95b761c5c2ea6955623536bbe84077` | `ARN` | 26 | 548 | 574 | 522 | changed tokens | 5,000 | +| `arz-default` | `standalone` | `1.0.0` | `a8024efa9fb52a91209a97a17589ae3b52e2b5547de98c6cc5e2cb82a37b58eb` | `ARZ` | 105 | 135 | 241 | 31 | changed tokens | 5,000 | +| `as-in-default` | `standalone` | `1.0.0` | `5211ee56f3d91029bf9d7e9626aaf56cec45d163ab51086250883ac3cde75c11` | `AS_IN` | 81 | 328 | 409 | 247 | changed tokens | 5,000 | +| `ast-default` | `standalone` | `1.0.0` | `263bca1a264ad6a2b00616fcfaa064f9e43e6a433224220af884340b5ad0b86d` | `AST` | 436 | 21,584 | 22,256 | 21,384 | changed tokens | 21,384 | +| `aym-default` | `standalone` | `1.0.0` | `2fd438798d8ceac034af4aa2eb6a52e27ded6d84b84117d2476906fc8cf79400` | `AYM` | 2,482 | 231,247 | 234,796 | 229,832 | changed tokens | 229,832 | +| `az-az-default` | `standalone` | `1.0.0` | `5327f8dde212ac34c3c4573cdd2fb6b21f908cd7c0cdf919e400b6f59c36abb9` | `AZ_AZ` | 340 | 6,669 | 7,011 | 6,331 | changed tokens | 6,331 | +| `azg-default` | `standalone` | `1.0.0` | `43594f9d451d5e7ee630895efe7b452dc54fec3d98c610392faed1a8ed1b16fe` | `AZG` | 258 | 9,357 | 9,685 | 9,169 | changed tokens | 9,169 | +| `bak-default` | `standalone` | `1.0.0` | `46b76859f6bbbcaa9bc22ff228bbadb9d0149048c17a47f415c7efb37d6ad1f8` | `BAK` | 1,048 | 10,616 | 11,683 | 9,587 | changed tokens | 9,587 | +| `be-by-default` | `standalone` | `1.0.0` | `52ccbdd9a5105910ac06fc279d02d184a21e5a27a7ddbb84e5a3d25df646bf86` | `BE_BY` | 196 | 678 | 876 | 484 | changed tokens | 5,000 | +| `bg-bg-default` | `standalone` | `1.0.0` | `fcc4bb4e765a2a2fdf2c8631e9f065bd0b7f269e60beac68f101dea1bc91975e` | `BG_BG` | 2,412 | 37,335 | 39,868 | 35,044 | changed tokens | 35,044 | +| `bn-bd-default` | `standalone` | `1.0.0` | `dcaff09bdd0a1c10bf6da6c51b7608edc3021380ebbfe3707cf5284002d8f880` | `BN_BD` | 3 | 3 | 6 | 0 | root-only tokens | 5,000 | +| `bra-default` | `standalone` | `1.0.0` | `46d9f30d4509fddb9a9c91b4de97f4eca2348d7191ee99261af3183808baef3c` | `BRA` | 88 | 115 | 204 | 28 | changed tokens | 5,000 | +| `bre-default` | `standalone` | `1.0.0` | `184e58faab57319b5877bb2d6057cfb87aa0f2b6c17b3c1fdb2ce2c4e76e37d6` | `BRE` | 42 | 1,682 | 1,724 | 1,640 | changed tokens | 5,000 | +| `ca-es-default` | `standalone` | `1.0.0` | `e8736229250fedd586a1651a730ea99036f7adb8a7f3e85ed6234af742dcff6d` | `CA_ES` | 1,480 | 61,495 | 63,314 | 60,354 | changed tokens | 60,354 | +| `ceb-default` | `standalone` | `1.0.0` | `12e337c1d22c7efec4012e6473516973c972f9c4a0d5fd8d1a2c1312ade45ea9` | `CEB` | 90 | 392 | 482 | 302 | changed tokens | 5,000 | +| `chu-default` | `standalone` | `1.0.0` | `064d51bafb5ebf7fdbb1fbdf3a5f471dde7457164a4f680ae6448c1b2207baa0` | `CHU` | 150 | 1,632 | 2,142 | 1,842 | changed tokens | 5,000 | +| `ckt-default` | `standalone` | `1.0.0` | `1315617dbf5ba569eea7a05e6413052754fb929b18bed329c0a577cc7c837ca2` | `CKT` | 191 | 302 | 495 | 113 | changed tokens | 5,000 | +| `cly-default` | `standalone` | `1.0.0` | `5fb7fa5ea1061e47ca22e37119f3316bac9608ad0b3c4390e41c5c1c83fd3f66` | `CLY` | 185 | 2,030 | 2,431 | 2,061 | changed tokens | 5,000 | +| `cni-default` | `standalone` | `1.0.0` | `e793997cce34ccfa1786088d89421128857b3c9d8b50a5643d8def8533183c91` | `CNI` | 407 | 10,095 | 10,536 | 9,722 | changed tokens | 9,722 | +| `cor-default` | `standalone` | `1.0.0` | `c6c88a61c66ce8c846ff92180e2a469d5a3dbee36897ef85ecbe5ad1480ec575` | `COR` | 8 | 159 | 177 | 161 | changed tokens | 5,000 | +| `cpa-default` | `standalone` | `1.0.0` | `c52b1d26c06a11eb82192ad374df94666317a67b465ee5e0e429df61e0af1d24` | `CPA` | 555 | 1,751 | 2,535 | 1,425 | changed tokens | 5,000 | +| `cre-default` | `standalone` | `1.0.0` | `2a858e3097c89e458a36661e39b5cca5d73db3f421e50df8cdfdc7609344b030` | `CRE` | 27 | 514 | 541 | 487 | changed tokens | 5,000 | +| `crh-default` | `standalone` | `1.0.0` | `828cdc3a52e98f0a93739e57a1400e880bdb22e564bda7cfc51aac5dcf418fd8` | `CRH` | 1,208 | 7,183 | 8,395 | 5,979 | changed tokens | 5,979 | +| `cs-cz-default` | `default` | `1.0.0` | `62afdaa6dc7a721b54a0dc278a0c648a63ad52a34a412d27b5b52fbcde9c1ce4` | `CS_CZ` | 5,113 | 51,401 | 56,612 | 46,563 | changed tokens | 46,563 | +| `csb-default` | `standalone` | `1.0.0` | `ef7b728f0963360c1e6fd7fdbc8c44f87c31d0ad95e424f127db10936a381bd5` | `CSB` | 37 | 350 | 392 | 318 | changed tokens | 5,000 | +| `ctp-default` | `standalone` | `1.0.0` | `a8cc8316dc040985e757cd8eb47894dc72128dd3fdd30a2b629b54a26f3239fe` | `CTP` | 217 | 2,456 | 2,786 | 2,352 | changed tokens | 5,000 | +| `cy-gb-default` | `standalone` | `1.0.0` | `394475eedc3c0fe26d395d06821efa2e8ff0c2ea4b5c63b72bbe3c8a4efd3d5d` | `CY_GB` | 183 | 6,240 | 6,425 | 6,059 | changed tokens | 6,059 | +| `czn-default` | `standalone` | `1.0.0` | `30e6473c472452d4105c99e162142949a2c1ff975ca35035a5ee439c977f07ba` | `CZN` | 39 | 68 | 108 | 30 | changed tokens | 5,000 | +| `da-dk-default` | `default` | `1.0.0` | `3f7b670a0e7b872bda0381f5154ce058a4656297b39b7157b4ccf6560257cb90` | `DA_DK` | 4,179 | 27,921 | 32,256 | 23,900 | changed tokens | 23,900 | +| `dak-default` | `standalone` | `1.0.0` | `5cc5ea06b28df6b18aca534fe06701fac953023f34217aae5df6452c335af53f` | `DAK` | 419 | 2,942 | 3,386 | 2,548 | changed tokens | 5,000 | +| `de-de-default` | `default` | `1.0.0` | `cbfa038122823f02e4bdb54b0035492c356b6ecd80f11eb11290d7a7248a59f5` | `DE_DE` | 54,092 | 277,266 | 333,036 | 242,501 | changed tokens | 242,501 | +| `dje-default` | `standalone` | `1.0.0` | `1efb975e9f6ab2c58583eaf573a4a3f150a2d6a1ffb9bbfeb2a224e0d51f205d` | `DJE` | 25 | 75 | 100 | 50 | changed tokens | 5,000 | +| `dsb-default` | `standalone` | `1.0.0` | `c251bbd00618946b5d40c49605a2b6d3fe305019c691d2859d0dad19910237e8` | `DSB` | 990 | 12,038 | 13,185 | 11,205 | changed tokens | 11,205 | +| `el-gr-default` | `standalone` | `1.0.0` | `995d9f80ae3e4810447696407d01e9b6b10b7d5088d123df8b8175cf616974aa` | `EL_GR` | 11,792 | 76,851 | 91,268 | 67,684 | changed tokens | 67,684 | +| `es-es-default` | `default` | `1.0.0` | `7a1ec94cfdb1e9a95431289d62dc5579cb2a532d99532eeda90290072e569721` | `ES_ES` | 65,059 | 849,661 | 926,393 | 806,272 | changed tokens | 806,272 | +| `et-ee-default` | `standalone` | `1.0.0` | `3179b5135c9efb4fd83d1f4840de337c5b8d8e330243f4b69c0289cdec435fdf` | `ET_EE` | 886 | 24,811 | 25,776 | 24,004 | changed tokens | 24,004 | +| `eus-default` | `standalone` | `1.0.0` | `29ea20d68e085f9403c32330259f02599f6c510f47db0475f1e3ea73a276a98b` | `EUS` | 26 | 10,401 | 11,535 | 11,483 | changed tokens | 11,483 | +| `evn-default` | `standalone` | `1.0.0` | `bf3dd82443f2dc033e553fe9685cc7d9df8806fcbf86b724953f3b7c6d74b258` | `EVN` | 4,072 | 11,965 | 16,135 | 7,991 | changed tokens | 7,991 | +| `fa-ir-default` | `default` | `1.0.0` | `b29a0d168a6a97f980666aa40b74a0edd8b6be4ab3320a7abfbb76b3529f4ea1` | `FA_IR` | 69 | 3,544 | 3,770 | 3,632 | changed tokens | 5,000 | +| `fi-fi-default` | `default` | `1.0.0` | `ca2628b3db31fee92f1b612ebbbd5e956a6dbbfb10e721325e55ef528f26072f` | `FI_FI` | 57,027 | 1,788,784 | 1,865,215 | 1,754,690 | changed tokens | 1,754,690 | +| `fo-fo-default` | `standalone` | `1.0.0` | `7d52c5ef7a69d8a4b3f5fd4725f8c2f75e1e978f021351df1b67773cc58ebb13` | `FO_FO` | 3,077 | 31,366 | 36,410 | 30,256 | changed tokens | 30,256 | +| `fr-fr-default` | `default` | `1.0.0` | `a988658758952fd599dc7360e0234178a6d65ac46e5cedc7dcd325a7cb7e71d9` | `FR_FR` | 59,240 | 404,011 | 474,110 | 365,969 | changed tokens | 365,969 | +| `frm-default` | `standalone` | `1.0.0` | `ebe1ce025d74d7cf5eee3ce328840911d01f227ff6834fd039461763c9f3ac18` | `FRM` | 602 | 27,002 | 27,816 | 26,612 | changed tokens | 26,612 | +| `fro-default` | `standalone` | `1.0.0` | `58e05dd60f4c616db2c28e34394171c6015c07de2d28bfeae7cfd2c60d7393d8` | `FRO` | 1,699 | 93,998 | 106,654 | 103,256 | changed tokens | 103,256 | +| `frr-default` | `standalone` | `1.0.0` | `8ad47d2806f6ef2926e2270389609b3fc77b696ee1f09df6c1c395a06d9b64bb` | `FRR` | 51 | 340 | 408 | 306 | changed tokens | 5,000 | +| `fur-default` | `standalone` | `1.0.0` | `ee6eb34acd2db625e91282629040da78f2a2f596ef34d12686acf76b4d3d9f68` | `FUR` | 168 | 5,007 | 5,185 | 4,849 | changed tokens | 5,000 | +| `fy-nl-default` | `standalone` | `1.0.0` | `24dbc346fc258d7315048b02d24e8cc3675739e6563613695dac3c548fcfab68` | `FY_NL` | 85 | 696 | 784 | 614 | changed tokens | 5,000 | +| `ga-ie-default` | `standalone` | `1.0.0` | `cc4ffc97b026107936fee34938b02273330ec23e61a1c9835cb3fd38b5319702` | `GA_IE` | 7,023 | 21,829 | 29,342 | 15,296 | changed tokens | 15,296 | +| `gaa-default` | `standalone` | `1.0.0` | `052af2955659a954f1c3036e564b405ec75d93fa858e96c5cb8f2ce8bac8a534` | `GAA` | 94 | 469 | 564 | 376 | changed tokens | 5,000 | +| `gal-default` | `standalone` | `1.0.0` | `91fcd0fe4215d310749795879d4d441fa23b12f4bcbbc3734ae83565c9c8b3c4` | `GAL` | 486 | 25,436 | 26,681 | 25,709 | changed tokens | 25,709 | +| `gd-gb-default` | `standalone` | `1.0.0` | `089c9cb590ef564fee033fcaa59c50b7317aaed5e55ebd5493679a60b6458c33` | `GD_GB` | 70 | 265 | 335 | 195 | changed tokens | 5,000 | +| `gmh-default` | `standalone` | `1.0.0` | `91ccdb971ed6905cf710362b208537c12e7aed0b39fdf2e881cef7e3114cd109` | `GMH` | 29 | 384 | 413 | 355 | changed tokens | 5,000 | +| `gml-default` | `standalone` | `1.0.0` | `7a6d0ce9b18d3a0aa6cf8c6d4179d8473ba86c9d8b36133eb2370dfd4d257aec` | `GML` | 50 | 565 | 624 | 524 | changed tokens | 5,000 | +| `goh-default` | `standalone` | `1.0.0` | `7dc85288accc2a0b66298254e461a7b5328e6443e8334e6e5ad0746d62d00e6b` | `GOH` | 482 | 5,267 | 5,782 | 4,818 | changed tokens | 5,000 | +| `got-default` | `standalone` | `1.0.0` | `1e68dbfe6c47af09142952282110d5f953c416d34e54185ddc4459c16850eb09` | `GOT` | 6,225 | 134,162 | 143,969 | 131,519 | changed tokens | 131,519 | +| `grc-default` | `standalone` | `1.0.0` | `ea83c5b51c7795f6c8e628ec6cac95578948175726c51d71710e751940f11aca` | `GRC` | 1,909 | 11,109 | 13,092 | 9,274 | changed tokens | 9,274 | +| `gsw-default` | `standalone` | `1.0.0` | `770d3af9a6c62efb3c90bf8654066b262491d8d957e7bdb230a1b04c7a21f17f` | `GSW` | 143 | 1,171 | 1,318 | 1,032 | changed tokens | 5,000 | +| `gup-default` | `standalone` | `1.0.0` | `c9dfd43c26952aad6abd283b8ec2982d2d71763d92e85b9db14542126b3b7876` | `GUP` | 73 | 370 | 444 | 298 | changed tokens | 5,000 | +| `gv-im-default` | `standalone` | `1.0.0` | `3f51e5929a22c339dc21d309f0b7b8f73b2e3f95207af8514f591c50d37e9e15` | `GV_IM` | 1 | 15 | 16 | 14 | changed tokens | 5,000 | +| `hai-default` | `standalone` | `1.0.0` | `27de8438d05cc3c6f86087145212bbfbf044041f155e973c0ef1aad49baf2cfe` | `HAI` | 28 | 520 | 548 | 492 | changed tokens | 5,000 | +| `hbs-default` | `standalone` | `1.0.0` | `edeb7fff29f64fbe76b2087818d4838d36f4606857acbadc577a28774419f213` | `HBS` | 24,382 | 272,515 | 302,286 | 253,522 | changed tokens | 253,522 | +| `he-il-default` | `default` | `1.0.0` | `9a47dc69bb7dab21aba0266b73cd74cdaeb17db94363796a0a56111ac8518256` | `HE_IL` | 2,358 | 57,658 | 61,071 | 56,356 | changed tokens | 56,356 | +| `hil-default` | `standalone` | `1.0.0` | `43ccfab3a54e725d44a1c0c03154dcb474ffa33a716e6fd92ab5f59ba5d57d5c` | `HIL` | 80 | 308 | 388 | 228 | changed tokens | 5,000 | +| `hsb-default` | `standalone` | `1.0.0` | `dd085232541899920df2acfcb11ef8fd914fea6894351f08a1e99f8fb84c3950` | `HSB` | 343 | 662 | 1,006 | 320 | changed tokens | 5,000 | +| `hsi-default` | `standalone` | `1.0.0` | `40099c2852ee4407f1da7e2ebc9a396765d51cfeee6868a83c04abe169d1b057` | `HSI` | 49 | 158 | 208 | 110 | changed tokens | 5,000 | +| `hu-hu-default` | `default` | `1.0.0` | `359d46a01d751ec823705ad7f3dd1cc8f6663feb1a9d13cb04d0c6fb51ab646e` | `HU_HU` | 19,406 | 910,688 | 935,713 | 896,938 | changed tokens | 896,938 | +| `hy-am-default` | `standalone` | `1.0.0` | `dd2cef8bd372c2fc32a3127f0fa5ba7a4829c59e54f508b14ffd054e04fc1c00` | `HY_AM` | 6,946 | 242,091 | 250,040 | 236,148 | changed tokens | 236,148 | +| `id-id-default` | `standalone` | `1.0.0` | `dc285b60aaa4e149f3d77fc93fede8bb6d20bae599b01304a8f552cab925ddde` | `ID_ID` | 3,877 | 21,296 | 25,175 | 17,421 | changed tokens | 17,421 | +| `is-is-default` | `standalone` | `1.0.0` | `920ca13c98fe07f141f6b646c5858c5e0143ec94cfde08fd885ae66923076ebf` | `IS_IS` | 4,766 | 52,132 | 58,089 | 48,557 | changed tokens | 48,557 | +| `it-it-default` | `default` | `1.0.0` | `5e03be31c9761e30dbf24a47a5ced3d6ec949dabd31e92632fdd9f7c67fc2e12` | `IT_IT` | 10,009 | 324,366 | 337,546 | 317,542 | changed tokens | 317,542 | +| `itl-default` | `standalone` | `1.0.0` | `e794b033969008c47de41f60506075e2e8f3e4a50120a30c708e74ba012abceb` | `ITL` | 1,574 | 3,546 | 5,124 | 1,976 | changed tokens | 5,000 | +| `izh-default` | `standalone` | `1.0.0` | `31b8cdd20bb1be97b44a83c8d3b529119dae4b9204bd93866894d6c30d0ce2b2` | `IZH` | 50 | 1,024 | 1,074 | 974 | changed tokens | 5,000 | +| `ja-jp-default` | `standalone` | `1.0.0` | `4eb7421a20914b847177e9fd02c798fe58e088d2bb212136efcce576c9dd180b` | `JA_JP` | 107 | 10,848 | 11,699 | 11,485 | changed tokens | 11,485 | +| `ka-ge-default` | `standalone` | `1.0.0` | `f8205f3c31e3dc6a0f908b3839956c72688ae04b672ed75ba9083e4280b99c3b` | `KA_GE` | 3,847 | 75,914 | 79,967 | 72,273 | changed tokens | 72,273 | +| `kbd-default` | `standalone` | `1.0.0` | `2cdd9c1b8d88369667243c7b27c9eab06c02497893d6190deb7850d245b2fb89` | `KBD` | 249 | 3,054 | 3,303 | 2,805 | changed tokens | 5,000 | +| `kca-default` | `standalone` | `1.0.0` | `22be165f4e2d9805a8db4b84ca943fe66eb6e1c3dbf390bd8f135a03678107e9` | `KCA` | 661 | 2,175 | 2,859 | 1,537 | changed tokens | 5,000 | +| `ket-default` | `standalone` | `1.0.0` | `4916e12863d306cbda2240606bd667e77d1c0d96bb8a3981c1d72a7d565887eb` | `KET` | 519 | 1,100 | 1,622 | 584 | changed tokens | 5,000 | +| `kjh-default` | `standalone` | `1.0.0` | `241daa9381a3a3d0cb28defa7872ec3f913195782bc343d185d60e28de09b622` | `KJH` | 74 | 1,172 | 1,246 | 1,098 | changed tokens | 5,000 | +| `kk-kz-default` | `standalone` | `1.0.0` | `46bada2596fbdc80e42617ae1c4fbf638b436271a37c2d4253072b399e5c3d66` | `KK_KZ` | 26 | 357 | 383 | 331 | changed tokens | 5,000 | +| `kl-gl-default` | `standalone` | `1.0.0` | `e82c2bf70b7e7378002469f79dab280af2d14f8db558cec395b776ae423f2deb` | `KL_GL` | 23 | 321 | 344 | 298 | changed tokens | 5,000 | +| `klr-default` | `standalone` | `1.0.0` | `f1395204bd22af53c50b07e632e4c330877c4018b67bc4441cee5aa199bbaddc` | `KLR` | 591 | 41,185 | 45,299 | 44,117 | changed tokens | 44,117 | +| `kmr-default` | `standalone` | `1.0.0` | `147729f9f3eb36dd1eaf35095cc57086e0aebf42ad4780aaf80034a5c0492948` | `KMR` | 13,981 | 191,104 | 206,245 | 178,283 | changed tokens | 178,283 | +| `kn-in-default` | `standalone` | `1.0.0` | `c6922ab4612042b71af377b03692c6787e07cec25ddd1b1cea757f2e0de095c8` | `KN_IN` | 4 | 11 | 15 | 7 | changed tokens | 5,000 | +| `ko-kr-default` | `standalone` | `1.0.0` | `bbcb468402ee54327dfd2101df5ef90869db06fe0543d85c1f62ac4d3beba155` | `KO_KR` | 2,672 | 192,673 | 196,302 | 190,958 | changed tokens | 190,958 | +| `kod-default` | `standalone` | `1.0.0` | `b0a2de585785e54665a193e707bfd4b90e9f0c675f44817d5465278553880867` | `KOD` | 65 | 524 | 589 | 459 | changed tokens | 5,000 | +| `kon-default` | `standalone` | `1.0.0` | `0adce159fa3b67b38dbc181eb3d2d2e0d6016cb62b3266fe6e47e901bd2a80e0` | `KON` | 200 | 585 | 786 | 386 | changed tokens | 5,000 | +| `krl-default` | `standalone` | `1.0.0` | `a98c4446dd7eaec464369f6c2a6691748c7844629b74efbebf7437a8454c4c5a` | `KRL` | 20 | 566 | 586 | 546 | changed tokens | 5,000 | +| `ky-kg-default` | `standalone` | `1.0.0` | `ef11ea53a79367b3c156ffb284d3e9101f30948e705c77660fe01810b0b00b86` | `KY_KG` | 72 | 2,997 | 3,069 | 2,925 | changed tokens | 5,000 | +| `la-default` | `standalone` | `1.0.0` | `fd906e27def14dcb23ae0771a968555c1abf9f8d1de6bb156099be2425d3b4af` | `LA` | 46,067 | 485,259 | 545,673 | 453,539 | changed tokens | 453,539 | +| `lg-ug-default` | `standalone` | `1.0.0` | `6a9974b58b7b5b5781c61efa728ef599c87d921f4f016ffdecf820c1b65e2cbb` | `LG_UG` | 87 | 4,671 | 4,831 | 4,657 | changed tokens | 5,000 | +| `lin-default` | `standalone` | `1.0.0` | `f6eda2e6cce620baf0bd1baaa6ee8142497e702a094a5b92b8295c4aa9732439` | `LIN` | 57 | 230 | 287 | 173 | changed tokens | 5,000 | +| `liv-default` | `standalone` | `1.0.0` | `9545c84208002654360f49184d96f8553a9e9b8abd0cddb85db41b1109397e33` | `LIV` | 201 | 2,841 | 3,059 | 2,657 | changed tokens | 5,000 | +| `lld-default` | `standalone` | `1.0.0` | `2866be97f9b5ff688865d927aa1c47975dd1a69a151f9e643f2e6221fbb89e51` | `LLD` | 180 | 4,819 | 5,034 | 4,674 | changed tokens | 5,000 | +| `lt-lt-default` | `standalone` | `1.0.0` | `5702f06a6340b06ebb98443686b7053968d1b1b51ec178866488a6f7a6791257` | `LT_LT` | 1,348 | 27,335 | 28,771 | 26,075 | changed tokens | 26,075 | +| `lv-lv-default` | `standalone` | `1.0.0` | `157444ef19affe7ed5d1b0d1e356683f025f611b3cb687a8c8e8be26bb24287f` | `LV_LV` | 7,238 | 75,492 | 87,465 | 72,989 | changed tokens | 72,989 | +| `mag-default` | `standalone` | `1.0.0` | `9cbefd751c52eb3f6b322f2ed17b34edb9c46ac3dd2358c89434c3dca2ce91f6` | `MAG` | 110 | 206 | 318 | 98 | changed tokens | 5,000 | +| `mg-mg-default` | `standalone` | `1.0.0` | `d446dd8d28922e7a3d54e3f059845e2165bfbe758d8b3869021ad47e8e7c46f2` | `MG_MG` | 159 | 636 | 795 | 477 | changed tokens | 5,000 | +| `mi-nz-default` | `standalone` | `1.0.0` | `4f55a59ba208cd87743e18fbca89af50c731baed6148ff7940ddc51c95acc9de` | `MI_NZ` | 104 | 207 | 312 | 104 | changed tokens | 5,000 | +| `mk-mk-default` | `standalone` | `1.0.0` | `56aed4f1e6df14829a2bbc92961247b002404cd8a73068ec05f27a52cda972bd` | `MK_MK` | 9,776 | 135,590 | 146,166 | 126,614 | changed tokens | 126,614 | +| `mn-mn-default` | `standalone` | `1.0.0` | `ac94039864a6b0e4fb2008dcba2a695875409e6382091f4c820d917cb1707ecb` | `MN_MN` | 2,140 | 17,231 | 19,733 | 15,453 | changed tokens | 15,453 | +| `mt-mt-default` | `standalone` | `1.0.0` | `c3ff6a3014b3765a698c48b6c85d84c729fb4c552c50bb40cc641002dc959beb` | `MT_MT` | 107 | 1,421 | 1,537 | 1,323 | changed tokens | 5,000 | +| `mwf-default` | `standalone` | `1.0.0` | `6247cb8e87c3d534c07e6843f2f7671fc958a785712bc47d607fb86233fcf897` | `MWF` | 29 | 592 | 749 | 691 | changed tokens | 5,000 | +| `nap-default` | `standalone` | `1.0.0` | `3cb6cfa9bd14b91af99e845cbb75f0ffa62bd4c9916e8c3733be9c313b95e097` | `NAP` | 40 | 1,497 | 1,537 | 1,457 | changed tokens | 5,000 | +| `nav-default` | `standalone` | `1.0.0` | `ad0db7cb940f7d76d2c1ffbc4da52b0bffa725592f0357a09cf1434c88ef8b84` | `NAV` | 585 | 9,826 | 10,706 | 9,536 | changed tokens | 9,536 | +| `nb-no-default` | `default` | `1.0.0` | `f495bffb44e79d27993e6e2e65d4b1204b29365dc93f481b2d8b96766fc90fd9` | `NB_NO` | 17,929 | 73,170 | 90,757 | 57,381 | changed tokens | 57,381 | +| `nds-default` | `standalone` | `1.0.0` | `358c11faeb80d31662f3a3cca65a87c7f764382381a00e715add8f78b83ae8eb` | `NDS` | 325 | 2,493 | 2,868 | 2,218 | changed tokens | 5,000 | +| `nl-nl-default` | `default` | `1.0.0` | `c098034adc42da2ca3e419160e6dd2c2b3868f8af334303b3a191e09caadaf5e` | `NL_NL` | 4,992 | 26,201 | 31,466 | 21,485 | changed tokens | 21,485 | +| `nn-no-default` | `default` | `1.0.0` | `900cf2005605aea2a3d8d731ec0b0c1f47fb4469b4ba6b9134145d4d026a0398` | `NN_NO` | 4,688 | 16,937 | 19,651 | 13,562 | changed tokens | 13,562 | +| `non-default` | `standalone` | `1.0.0` | `0e4cc499f1414b304a0a91d74e1315c951a93344a74ca70c27c9d339c90daa52` | `NON` | 2,664 | 46,067 | 52,239 | 46,911 | changed tokens | 46,911 | +| `ny-mw-default` | `standalone` | `1.0.0` | `2f6bf10a59475dbf8c925fc27cdd23712c3d914d6b0fa5a9d6542f51873afadd` | `NY_MW` | 225 | 3,150 | 3,375 | 2,925 | changed tokens | 5,000 | +| `oci-default` | `standalone` | `1.0.0` | `7c5dcd14b47020d7676a1fc634a1d9f38a06728d69407386d8864cd909046c3c` | `OCI` | 174 | 7,465 | 7,784 | 7,436 | changed tokens | 7,436 | +| `om-et-default` | `standalone` | `1.0.0` | `df01f908af60ba4a63933a91845d1241a37d366467026bd826ebec08689406bb` | `OM_ET` | 84 | 656 | 740 | 572 | changed tokens | 5,000 | +| `ood-default` | `standalone` | `1.0.0` | `41a559d1a6ae83c0f2579793a2a4b883b2e974cfd705f90ad856b395346c368b` | `OOD` | 156 | 455 | 614 | 302 | changed tokens | 5,000 | +| `osx-default` | `standalone` | `1.0.0` | `7ad2453964fa001ec17b73eb1156c25513961a4683d5d7abaaaec477d3e8755e` | `OSX` | 863 | 12,361 | 13,416 | 11,690 | changed tokens | 11,690 | +| `ote-default` | `standalone` | `1.0.0` | `8cd8e5d6bfe7c5848e39ff334b49580a769ba78d1576d2861436641e8cec5a58` | `OTE` | 1,110 | 1,785 | 2,991 | 771 | changed tokens | 5,000 | +| `pl-pl-polimorf` | `optional` | `1.0.0` | `4fe4bf5e6c22c1beea5b3d57f1ce4c9ea5aac1ed8ab24c616fb06df745e40d15` | `PL_PL` | 315,639 | 4,668,685 | 4,812,023 | 4,496,384 | changed tokens | 4,496,384 | +| `pl-pl-unimorph` | `default` | `1.0.0` | `8191ed727097839cc808cbc5c56a1bd78b3c851e7733ad226ad9a51519a54721` | `PL_PL` | 9,990 | 120,867 | 132,308 | 112,351 | changed tokens | 112,351 | +| `ps-af-default` | `standalone` | `1.0.0` | `34c08ecbe51c274996a2e8be119cbf323317fe2eb390454730e60a4b776e9d33` | `PS_AF` | 381 | 2,945 | 3,445 | 2,683 | changed tokens | 5,000 | +| `pt-pt-default` | `default` | `1.0.0` | `7a035ff330a6f0548f446cd0d6617bc1cf4751292125a3564d3a255c5d6f516d` | `PT_PT` | 4,001 | 211,091 | 215,490 | 207,488 | changed tokens | 207,488 | +| `que-default` | `standalone` | `1.0.0` | `4aca805b20dce58d1cbd80a824efca214ada72a8955df738a271b08a5bfbeca8` | `QUE` | 814 | 122,820 | 125,491 | 123,863 | changed tokens | 123,863 | +| `ro-ro-default` | `standalone` | `1.0.0` | `fe5215778c84f871d0f6408dcb663a7cdef6571aaa0b96dba88ed7bfd09a3461` | `RO_RO` | 4,377 | 48,497 | 53,278 | 44,524 | changed tokens | 44,524 | +| `ru-ru-default` | `default` | `1.0.0` | `df7ea25e63a875eeec7a4185be685bd5372a3c568db85c34c44fdf5d8d980a40` | `RU_RU` | 37,410 | 759,333 | 806,279 | 731,471 | changed tokens | 731,471 | +| `sah-default` | `standalone` | `1.0.0` | `c5b361c9e1df64d60813c96cbea4d228d97fbc76e71bbaccb87609a13e6b67e8` | `SAH` | 5,622 | 526,431 | 534,266 | 523,022 | changed tokens | 523,022 | +| `san-default` | `standalone` | `1.0.0` | `bf94e3c36639ce16b0c11a6e9ab62baebc8acea18d14926a597585cbb179e708` | `SAN` | 54 | 55 | 109 | 1 | changed tokens | 5,000 | +| `sdh-default` | `standalone` | `1.0.0` | `c19f455c815ae94776d54dba1c504d41abea342c15ba9e8714dd370cad98be2a` | `SDH` | 1 | 165 | 166 | 164 | changed tokens | 5,000 | +| `see-default` | `standalone` | `1.0.0` | `329f57fb8c306ab2883f5640641c9a361f04a0a50fc3e9f62c0d8c6e40ddf380` | `SEE` | 26 | 483 | 509 | 457 | changed tokens | 5,000 | +| `sga-default` | `standalone` | `1.0.0` | `01ea73ea683f0864976e43375d3ff828c009349e17c5b4c34fb832950f29e7ca` | `SGA` | 49 | 929 | 980 | 882 | changed tokens | 5,000 | +| `shp-default` | `standalone` | `1.0.0` | `6139156cc7e79c30520d70b5a40efbcf784ea878c8e2668b278a01dee9ebf073` | `SHP` | 1,997 | 7,705 | 9,940 | 5,946 | changed tokens | 5,946 | +| `sjo-default` | `standalone` | `1.0.0` | `22101d045412a65a834bce25963181ec1cc3b7bf7f2ae8a1cf1d31950b11840c` | `SJO` | 1,892 | 3,151 | 5,089 | 1,305 | changed tokens | 5,000 | +| `sl-si-default` | `standalone` | `1.0.0` | `b59e7199dce258b404729e4d708172c535733cfae430a26f1c4faa3d25dc323e` | `SL_SI` | 2,531 | 25,100 | 27,734 | 22,672 | changed tokens | 22,672 | +| `slp-default` | `standalone` | `1.0.0` | `fb4cc41d286579f2c0992e282bbfa7e0ccc251526119cf51ad752f363ac352be` | `SLP` | 21 | 41 | 62 | 20 | changed tokens | 5,000 | +| `sme-default` | `standalone` | `1.0.0` | `77abb417f0c67f00cf44a1d129b8b578b9381a2e6b223558124cb39954830685` | `SME` | 2,082 | 53,034 | 55,703 | 51,539 | changed tokens | 51,539 | +| `sn-zw-default` | `standalone` | `1.0.0` | `a9aae37550b9230283ca7c62056ba6c7d0b5cf3f3bf20f30905f723c50addd98` | `SN_ZW` | 86 | 2,640 | 2,726 | 2,554 | changed tokens | 5,000 | +| `sq-al-default` | `standalone` | `1.0.0` | `dda602d80fe10911f4aec2f70196f429eed1a5c232ece4620065fe96274512c4` | `SQ_AL` | 587 | 10,218 | 10,851 | 9,677 | changed tokens | 9,677 | +| `st-za-default` | `standalone` | `1.0.0` | `86230d154bada51128d1c03a631fd182170e5d2b8070d57b65ebee2db91c6ce9` | `ST_ZA` | 26 | 416 | 442 | 390 | changed tokens | 5,000 | +| `sv-se-default` | `default` | `1.0.0` | `d9be72e3d67c776622c4281e04e4063b9381e8f84a823d98ebf08888b82dff0c` | `SV_SE` | 12,371 | 95,181 | 110,468 | 85,737 | changed tokens | 85,737 | +| `swc-default` | `standalone` | `1.0.0` | `28f8e90aea44ced44da8ca797ea40c3e822d580bd6aaa71f3dce6d847a02b9a9` | `SWC` | 100 | 6,700 | 6,800 | 6,600 | changed tokens | 6,600 | +| `syc-default` | `standalone` | `1.0.0` | `8eda4d03d168a0908033eacff07d2fbe3332692157c9d1f862cd8ca07fcafdd1` | `SYC` | 215 | 2,139 | 2,365 | 1,935 | changed tokens | 5,000 | +| `tat-default` | `standalone` | `1.0.0` | `1617b08152da422aa295ef7f990e9e30ec8834eeeb9555cba805d96142462779` | `TAT` | 1,258 | 7,483 | 8,745 | 6,229 | changed tokens | 6,229 | +| `tg-tj-default` | `standalone` | `1.0.0` | `4809cdc83d745d0db145d74c6654e4732b83bf70e32df3a208aac22a0db4d6b1` | `TG_TJ` | 75 | 78 | 153 | 3 | changed tokens | 5,000 | +| `tk-tm-default` | `standalone` | `1.0.0` | `3821730499cd42fa7974c907d08377763e1cc1a6f85784b638d22f87124317e2` | `TK_TM` | 68 | 804 | 872 | 736 | changed tokens | 5,000 | +| `tl-ph-default` | `standalone` | `1.0.0` | `e9c291d64e9359d58257cdd5342b7a4fdb94b123e5b23be3343a97073661aa78` | `TL_PH` | 289 | 2,042 | 2,336 | 1,758 | changed tokens | 5,000 | +| `tr-tr-default` | `standalone` | `1.0.0` | `29db7f67de1b5560187ab496998aaf886277b96f8c10cec7f49790c2afd23be1` | `TR_TR` | 3,014 | 222,207 | 226,873 | 220,845 | changed tokens | 220,845 | +| `tyv-default` | `standalone` | `1.0.0` | `7b4189ea74cbc151a0dfa22c01dd30b105a53f9f1be561033184df0d5fecd7b6` | `TYV` | 5,032 | 491,421 | 499,024 | 488,960 | changed tokens | 488,960 | +| `ug-cn-default` | `standalone` | `1.0.0` | `a76f8114c54be66c7ea564c79bc81f88d62a9ab9dd2c7028e542aa1d04e9b22f` | `UG_CN` | 85 | 6,200 | 6,316 | 6,146 | changed tokens | 6,146 | +| `uk-ua-default` | `default` | `1.0.0` | `cf3f612cfff16cb7763f99c55851069489b883c3bdd1a6576cd8c57a97e07eae` | `UK_UA` | 1,493 | 14,150 | 15,737 | 12,752 | changed tokens | 12,752 | +| `ur-pk-default` | `standalone` | `1.0.0` | `588e4e2644f121073adc8a62f8f5c14e9fcaa5e57ed29ada787596c0248eb426` | `UR_PK` | 162 | 881 | 1,047 | 723 | changed tokens | 5,000 | +| `us-uk-default` | `default` | `1.0.1` | `0a3ec1fddd5ddb859f5ffa8003efe9200a5ef6203df1fddaf53a58003f33582c` | `US_UK` | 396,939 | 591,946 | 1,002,414 | 208,540 | changed tokens | 208,540 | +| `uz-uz-default` | `standalone` | `1.0.0` | `f8e520d4f2741f4c8a52013923850696838cdffa9c433827e942cc779d86f725` | `UZ_UZ` | 15 | 1,080 | 1,095 | 1,065 | changed tokens | 5,000 | +| `vec-default` | `standalone` | `1.0.0` | `29d9dee4846c6fee65c69116c9ac2cfa370719af04cf206050ce08b9c5acfb0d` | `VEC` | 368 | 8,595 | 9,056 | 8,320 | changed tokens | 8,320 | +| `vot-default` | `standalone` | `1.0.0` | `f05693d86412bf46ca79606c033018d488553c30a180f3d00bee394867ad667b` | `VOT` | 55 | 1,392 | 1,448 | 1,338 | changed tokens | 5,000 | +| `vro-default` | `standalone` | `1.0.0` | `01a1408466bf5c549721a9bffa976dda00fb66d89a1a70d91022296162d9ba94` | `VRO` | 50 | 277 | 328 | 228 | changed tokens | 5,000 | +| `xcl-default` | `standalone` | `1.0.0` | `aca4a0452f876a3c7fc4372c8253f6d73c77e493191e5e5f62aeb69cecbc92e4` | `XCL` | 4,299 | 55,946 | 63,418 | 54,820 | changed tokens | 54,820 | +| `xno-default` | `standalone` | `1.0.0` | `eb6fa9f98fbc2b3d74714032555f5ab4bad394fdec1ddb2d386dabeed1812fbf` | `XNO` | 5 | 186 | 191 | 181 | changed tokens | 5,000 | +| `xty-default` | `standalone` | `1.0.0` | `f79b31e96b351170504e1a28ca6665be22725f1c50e3f2b30324f37de84f5431` | `XTY` | 417 | 1,536 | 1,965 | 1,131 | changed tokens | 5,000 | +| `yi-default` | `default` | `1.0.0` | `f47de665c27dcd72833a82904e49c68a945bb5aca769a7ec5a0164e2c981a6d3` | `YI` | 802 | 3,532 | 4,300 | 2,776 | changed tokens | 5,000 | +| `zpv-default` | `standalone` | `1.0.0` | `0dba154558eede2bec318eb5c33edc856483b1ebd999f433f0d16508d5cc62e9` | `ZPV` | 203 | 812 | 1,021 | 615 | changed tokens | 5,000 | +| `zu-za-default` | `standalone` | `1.0.0` | `da7922f2232206c74f0fe22bdc539c5b6f28a2fc781b40945a13be9bb1da4c75` | `ZU_ZA` | 563 | 32,384 | 33,297 | 32,171 | changed tokens | 32,171 | -Speed benchmarks process the complete changed-token dictionary sequence for the language. Only resources with fewer than 5,000 changed tokens are repeated to reach the minimum timing size; larger resources are not sampled or truncated. +Speed benchmarks process the complete selected timing sequence for the model. Changed tokens are selected whenever any exist. For root-only dictionaries, the complete root-preservation sequence is selected so the model still receives a speed measurement. Only selected populations with fewer than 5,000 tokens are repeated to reach the minimum timing size; larger populations are not sampled or truncated. diff --git a/docs/benchmarks/reference/english-coverage.md b/docs/benchmarks/reference/english-coverage.md index e1ac0de..9975880 100644 --- a/docs/benchmarks/reference/english-coverage.md +++ b/docs/benchmarks/reference/english-coverage.md @@ -15,21 +15,21 @@ The speed cells are JMH point estimates with their 99.9% score errors. The 100% !!! note "Separate English speed suites" - The full-knowledge `97.9 ns/token` point on this page belongs to the coverage suite. The [English language comparison](../languages/english.md#speed) reports `80.8 ns/token` from a separate JMH method and run. Their uncertainty intervals overlap; neither point estimate should replace the other. + The full-knowledge `120.4 ns/token` point on this page belongs to the coverage suite. The [English language comparison](../languages/english.md#speed) reports `84.7 ns/token` from a separate JMH method and run. Their uncertainty intervals overlap; neither point estimate should replace the other. | Used rows | Actual row ratio | All exact | Changed exact | Root preserved | Speed ms/op | Error ms | ns/token | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | -| 100% | 100.000% | 97.668% | 98.110% | 97.552% | 20.425 | 3.636 | 97.9 | -| 90% | 90.000% | 97.239% | 95.821% | 97.612% | 17.779 | 1.827 | 85.3 | -| 80% | 80.000% | 96.827% | 93.673% | 97.656% | 15.343 | 1.321 | 73.6 | -| 70% | 70.000% | 96.392% | 91.430% | 97.695% | 16.444 | 2.027 | 78.9 | -| 60% | 60.000% | 95.935% | 89.244% | 97.693% | 14.330 | 1.350 | 68.7 | -| 50% | 50.000% | 95.453% | 86.979% | 97.678% | 14.953 | 2.625 | 71.7 | -| 40% | 40.000% | 94.939% | 84.667% | 97.638% | 12.919 | 1.155 | 62.0 | -| 30% | 30.000% | 94.398% | 82.443% | 97.538% | 12.166 | 1.305 | 58.3 | -| 20% | 20.000% | 93.821% | 80.174% | 97.406% | 11.549 | 1.535 | 55.4 | -| 10% | 10.000% | 93.057% | 77.327% | 97.190% | 14.360 | 3.524 | 68.9 | +| 100% | 100.000% | 97.668% | 98.110% | 97.552% | 25.107 | 4.358 | 120.4 | +| 90% | 90.000% | 97.239% | 95.821% | 97.612% | 24.021 | 1.865 | 115.2 | +| 80% | 80.000% | 96.827% | 93.673% | 97.656% | 23.593 | 2.656 | 113.1 | +| 70% | 70.000% | 96.392% | 91.430% | 97.695% | 18.965 | 1.559 | 90.9 | +| 60% | 60.000% | 95.935% | 89.244% | 97.693% | 18.588 | 1.040 | 89.1 | +| 50% | 50.000% | 95.453% | 86.979% | 97.678% | 18.479 | 2.304 | 88.6 | +| 40% | 40.000% | 94.939% | 84.667% | 97.638% | 17.345 | 1.504 | 83.2 | +| 30% | 30.000% | 94.398% | 82.443% | 97.538% | 17.164 | 1.252 | 82.3 | +| 20% | 20.000% | 93.821% | 80.174% | 97.406% | 14.912 | 1.512 | 71.5 | +| 10% | 10.000% | 93.057% | 77.327% | 97.190% | 14.937 | 2.320 | 71.6 | ## Column Meanings diff --git a/docs/benchmarks/reference/environment.md b/docs/benchmarks/reference/environment.md index b1644d9..8883221 100644 --- a/docs/benchmarks/reference/environment.md +++ b/docs/benchmarks/reference/environment.md @@ -6,16 +6,20 @@ The values below are environment-specific and must not be read as universal perf The generalization report is deterministic and publishes no runtime score, so CPU frequency, background load, and elapsed time do not affect its accuracy -counters. It was nevertheless generated on the same identified workstation: +counters. The active snapshot combines the immutable 20-language campaign with +the separately measured 146-language continuation: | Item | Value | | --- | --- | -| Evaluation date | 2026-08-25 (Europe/Prague) | -| Command | `./gradlew --no-daemon -PdictionaryGeneralizationReleaseVersion=4.2.0-6-g84e57fb dictionaryGeneralization` | -| Radixor/Java source identity | `4.2.0-6-g84e57fb` | -| Report | `build/reports/generalization/dictionary-generalization.csv` | -| Published snapshot | `docs/benchmarks/data/dictionary-generalization.csv` | -| Scenarios | 1,000: 20 default models × 10 coverage levels × 5 frozen splits | +| Evaluation dates | Historical 20 languages: 2026-08-25; 146-language continuation: 2026-09-11 (Europe/Prague) | +| Continuation command | `./gradlew --no-daemon -PdictionaryGeneralizationReleaseVersion=4.3.0.dirty dictionaryGeneralizationStandalone` | +| Merge/publication command | `./gradlew --no-daemon mergeDictionaryGeneralizationSnapshots publishDictionaryGeneralizationDocumentation` | +| Radixor/Java source identities | Historical rows: `4.2.0-6-g84e57fb`; continuation rows: `4.3.0.dirty` at base revision `14f61beb106bca21d78f828e2f8e35fb457d3d90` | +| Continuation report | `build/reports/generalization/dictionary-generalization-standalone-2026-09-11.csv` | +| Active published snapshot | `docs/benchmarks/data/dictionary-generalization-2026-09-11.csv` | +| Historical archive | `docs/benchmarks/data/dictionary-generalization.csv` | +| Scenarios | 8,300: 166 default models × 10 coverage levels × 5 frozen splits | +| Continuation runtime | 6 min 34 s; informational only | | Evaluation type | Raw deterministic exact-root counters; no elapsed-time value is interpreted | | Java runtime | OpenJDK 25.0.4, Red Hat build 25.0.4+7 | | Operating system | Fedora Linux 44 (MATE-Compiz) | @@ -23,12 +27,12 @@ counters. It was nevertheless generated on the same identified workstation: | CPU | AMD Ryzen 5 8600G, 6 physical / 12 logical CPUs | | Architecture | x86_64 | | Split protocol | `radixor-generalization-v1`; exact-size nested row prefixes | -| Provenance | Every CSV row records model ID, independent model version, and compressed-resource SHA-256 | +| Provenance | Every CSV row records model ID, independent model version, compressed-resource SHA-256, source revision/state, and generator SHA-256; each campaign has its own source manifest | The CPU was configured with the `performance` governor, but that fact is not used to support any generalization claim. The new report deliberately excludes speed; the Java timing environment below remains the provenance for the -published 2026-08-25 JMH tables. +published 2026-09-10 JMH tables. ## Edit-Cost Sensitivity Run @@ -52,49 +56,50 @@ Elapsed time and memory describe the cost of reproducing the experiment and are | Item | Value | | --- | --- | -| Benchmark date | 2026-08-25 (Europe/Prague) | +| Benchmark date | 2026-09-10 (the run completed on 2026-09-11, Europe/Prague) | | Corpus command | `./gradlew benchmarkCorpusReport --no-daemon` | -| Exact-root accuracy command | `tools/run-published-accuracy-benchmarks.sh 2026-08-25`; all four `*BenchmarkQuality` classes are selected and timing scores are discarded | -| Stemming-quality command | `./gradlew stemmingQuality --no-daemon` | -| Published speed command | `tools/run-published-speed-benchmarks.sh 2026-08-25 4.2.0-6-g84e57fb` | -| Published speed run interval | 2026-08-25 14:33:15 to 16:27:55 Europe/Prague (1 h 54 min 40 s, including idle intervals and both JMH suites) | -| Stabilization intervals | 30 s before the main speed matrix; 15 s between the main matrix and coverage-speed suite | +| Exact-root accuracy command | `tools/run-published-accuracy-benchmarks.sh 2026-09-10`; the exact-root benchmark suite is selected and timing scores are discarded | +| Pairwise stemming quality | Complete 166-language default-model snapshot published as `stemming-quality-2026-09-11.csv`; the prior 20-language CSV remains an immutable archive | +| Published speed command | `tools/run-published-speed-benchmarks.sh 2026-09-10 4.3.0.dirty` | +| Main speed run | Started 2026-09-10 22:21:14; the completed main CSV was written on 2026-09-11 at approximately 01:21:00 Europe/Prague | +| Clean coverage-speed run | 2026-09-11 02:26:14 to 02:31:25 Europe/Prague | +| Stabilization intervals | 30 s before the main speed matrix; no additional delay before the clean recovery run because the preceding failed post-run validation had already left the machine idle | | Corpus and command report | `build/reports/jmh/benchmark-corpora.csv` | -| Exact-root reports | `build/reports/jmh/stemmer-accuracy-2026-08-25.csv` and `.txt` | -| Speed reports | `build/reports/jmh/stemmer-speed-2026-08-25.csv` and `.txt` | -| English coverage accuracy reports | `build/reports/jmh/english-coverage-accuracy-2026-08-25.csv` and `.txt` | -| English coverage speed reports | `build/reports/jmh/english-coverage-speed-2026-08-25.csv` and `.txt` | +| Exact-root reports | `build/reports/jmh/stemmer-accuracy-2026-09-10.csv` and `.txt` | +| Speed reports | `build/reports/jmh/stemmer-speed-2026-09-10.csv` and `.txt` | +| English coverage accuracy reports | `build/reports/jmh/english-coverage-accuracy-2026-09-10.csv` and `.txt` | +| English coverage speed reports | `build/reports/jmh/english-coverage-speed-2026-09-10.csv` and `.txt` | | Stemming-quality reports | `build/reports/stemming-quality/stemming-quality.csv` and `.md` | -| Environment report | `build/reports/jmh/performance-environment-2026-08-25.txt` | -| Selected speed methods | `build/reports/jmh/published-speed-benchmarks-2026-08-25.txt` | -| Comparison scope | Same-language methods used by the 20 language pages; `PolishPolimorfStemmerComparisonBenchmark`, all quality methods, the separate CISTEM gold-standard experiment, and internal trie microbenchmarks are excluded | -| Model scope | Exactly the 20 IDs declared by `Language.defaultModelId()`; Polish uses `pl-pl-unimorph`, and `pl-pl-polimorf` is not measured | -| Core base commit | `84e57fb27ae40913569e826858a8ebb07cf2ea01` | -| Release identity | Radixor/Java `4.2.0-6-g84e57fb`; exact measured tracked changes and untracked-source checksums are retained as `measured-source-2026-08-25.patch` and `measured-untracked-2026-08-25.sha256` | +| Environment report | `build/reports/jmh/performance-environment-2026-09-10.txt` | +| Selected speed methods | `build/reports/jmh/published-speed-benchmarks-2026-09-10.txt` | +| Comparison scope | Parameterized Radixor measurements for all 167 user-facing model IDs plus approved authoritative same-language comparator paths, including the separately identified optional PoliMorf Morfologik row; quality methods, the CISTEM gold-standard experiment, ambiguous duplicate baselines, and internal trie microbenchmarks are excluded | +| Model scope | All 167 user-facing IDs: 166 language defaults plus the optional `pl-pl-polimorf`; default and optional Polish results retain distinct model identities | +| Core base commit | `14f61beb106bca21d78f828e2f8e35fb457d3d90` | +| Release identity | Radixor/Java `4.3.0.dirty`; exact measured tracked changes and untracked-source checksums are retained as `measured-source-2026-09-10.patch` and `measured-untracked-2026-09-10.sha256` | | JMH version | 1.37 | | Speed benchmark mode | Average time, `time/op` | | Score unit | `ns/op`; language pages additionally derive `ms/op` and `ns/token` | -| Speed warmup | 5 iterations, 1 s each, independently in every fork | -| Speed measurement | 7 iterations, 1 s each, independently in every fork | +| Speed warmup | 3 iterations, 1 s each, independently in every fork | +| Speed measurement | 5 iterations, 1 s each, independently in every fork | | Speed forks | 3 independent JVM forks | | Speed threads | 1 | | Speed fork heap | Fixed `-Xms6g -Xmx6g` | -| Reported uncertainty | JMH `Score Error (99.9%)` over 21 measured samples | -| Observed relative uncertainty | Main speed matrix: maximum 15.542%, with 18 of 105 rows above 10% and none above 20%; coverage-speed curve: maximum 24.542%, with 7 of 10 rows above 10% and 1 above 20% | +| Reported uncertainty | JMH `Score Error (99.9%)` over 15 measured samples | +| Observed relative uncertainty | Main speed matrix: maximum 34.021%, with 207 of 307 rows above 10% and 106 above 20%; clean coverage-speed curve: maximum 17.358%, with 5 of 10 rows above 10% and none above 20% | | Deterministic measurements | Corpus, patch-command distribution, exact-root counters, coverage accuracy, and pairwise stemming quality are evaluated without interpreting runtime scores; no warmup is required | -| JVM reported by JMH | JDK 25.0.4, OpenJDK 64-Bit Server VM, 25.0.4+7 | -| Java runtime | OpenJDK Runtime Environment, Red Hat build 25.0.4+7 | +| JVM reported by JMH | JDK 25.0.4.1, OpenJDK 64-Bit Server VM, 25.0.4.1+1 | +| Java runtime | OpenJDK Runtime Environment, Red Hat build 25.0.4.1+1 | | JVM invoker | `/usr/lib/jvm/java-25-openjdk/bin/java` | | Operating system | Fedora Linux 44 (MATE-Compiz) | -| Kernel | Linux 7.1.8-200.fc44.x86_64 | +| Kernel | Linux 7.1.13-200.fc44.x86_64 | | Architecture | x86_64 | | CPU | AMD Ryzen 5 8600G w/ Radeon 760M Graphics | | Physical / logical CPUs | 6 / 12 | | CPU frequency policy | `amd-pstate-epp`; governor `performance` on every logical CPU; EPP `performance`; boost enabled | | CPU affinity | Scheduler default; no explicit pinning | | Installed memory | 61 GiB reported by the operating system | -| Pre-run idle state | Load average 0.32 / 0.56 / 1.14 after the 30 s idle interval; CPU Tctl 42.8 degrees Celsius; swap unused | -| End-of-run state | Load average 1.17 / 1.21 / 1.27; CPU Tctl 68.9 degrees Celsius | +| Pre-run idle state | Load average 0.43 / 0.47 / 0.59 after the 30 s idle interval; CPU Tctl 46.6 degrees Celsius; swap unused | +| End of clean coverage run | Load average 1.28 / 0.97 / 0.56; CPU Tctl 61.4 degrees Celsius; swap unused | | Power and idle policy | Developer workstation on stable power; screensaver, suspend, and hibernation disabled | | Concurrent project work | None during the published speed and coverage-speed run | @@ -106,33 +111,33 @@ All Radixor rows use contracted compiled patch tries. During compilation, a subt ## Model And Source Identity -`benchmark-corpora.csv` records the model ID, independent artifact version, and descriptor SHA-256 for every language. Every stemming-quality CSV row repeats the same three fields. The performance environment report additionally records checksums of the executable JMH JAR, runtime classpath manifest, corpus report, quality report, measured source patch, and untracked-source manifest. +`benchmark-corpora.csv` records the model ID, independent artifact version, and descriptor SHA-256 for all 167 user-facing models. Every active stemming-quality CSV row repeats the same three fields for its complete 166-default-language scope. The performance environment report additionally records checksums of the executable JMH JAR, runtime classpath manifest, corpus report, quality report, measured source patch, and untracked-source manifest. -The JMH runtime classpath contains the optional model artifact because it is a separately testable project dependency. It is not selected by any published benchmark. The selected-method manifest rejects `PolishPolimorf`, and the corpus/quality publication validators reject any non-default Polish model. +The parameterized Radixor speed benchmark selects every one of the 167 model IDs, including optional PoliMorf under its separate identity. Pairwise stemming quality covers all 166 default languages; the prior 20-language file is retained only as an immutable archive. A complete 189-entry runtime-classpath content manifest and the immutable main speed CSV were pinned retrospectively after the completed main run and before the clean coverage recovery. Their SHA-256 values are `0ae88fef5c30d58b90679ed149c6570e2d7bf2eb79f347d2daaf61c8d00f6a12` and `165b653523cdf04ebf8430044876efbade80aaddb5893edaad943b24858547c0`, respectively. This retrospective pin protects the recovery boundary but is explicitly not contemporaneous main-run provenance. ## Report Files Generated local report files for this benchmark update: -- `build/reports/generalization/dictionary-generalization.csv` +- `build/reports/generalization/dictionary-generalization-standalone-2026-09-11.csv` - `build/reports/generalization/edit-cost-sensitivity.csv` - `build/reports/jmh/benchmark-corpora.csv` -- `build/reports/jmh/stemmer-accuracy-2026-08-25.csv` -- `build/reports/jmh/stemmer-accuracy-2026-08-25.txt` -- `build/reports/jmh/stemmer-speed-2026-08-25.csv` -- `build/reports/jmh/stemmer-speed-2026-08-25.txt` -- `build/reports/jmh/english-coverage-accuracy-2026-08-25.csv` -- `build/reports/jmh/english-coverage-accuracy-2026-08-25.txt` -- `build/reports/jmh/english-coverage-speed-2026-08-25.csv` -- `build/reports/jmh/english-coverage-speed-2026-08-25.txt` -- `build/reports/jmh/performance-environment-2026-08-25.txt` +- `build/reports/jmh/stemmer-accuracy-2026-09-10.csv` +- `build/reports/jmh/stemmer-accuracy-2026-09-10.txt` +- `build/reports/jmh/stemmer-speed-2026-09-10.csv` +- `build/reports/jmh/stemmer-speed-2026-09-10.txt` +- `build/reports/jmh/english-coverage-accuracy-2026-09-10.csv` +- `build/reports/jmh/english-coverage-accuracy-2026-09-10.txt` +- `build/reports/jmh/english-coverage-speed-2026-09-10.csv` +- `build/reports/jmh/english-coverage-speed-2026-09-10.txt` +- `build/reports/jmh/performance-environment-2026-09-10.txt` - `build/reports/stemming-quality/stemming-quality.csv` - `build/reports/stemming-quality/stemming-quality.md` - `build/reports/stemming-quality/metric-correlations-pearson.csv` - `build/reports/stemming-quality/metric-correlations-spearman.csv` The versioned documentation snapshots under `docs/benchmarks/data/` preserve the complete -stemming-quality matrix, all 1,000 generalization scenarios, the compressed edit-cost raw matrix +stemming-quality matrix, all 8,300 active generalization scenarios, the compressed historical edit-cost raw matrix plus derived analyses, and the dated CSV inputs for the published Java and Python performance tables. The Python provenance JSON records its environment and run parameters. Detailed JMH TXT logs and machine-state reports remain local build artifacts. The complete versioned input list and diff --git a/docs/benchmarks/reference/generalization-methodology.md b/docs/benchmarks/reference/generalization-methodology.md index 80c936c..45c1bb2 100644 --- a/docs/benchmarks/reference/generalization-methodology.md +++ b/docs/benchmarks/reference/generalization-methodology.md @@ -78,13 +78,17 @@ English and Finnish dictionaries from dominating small resources. Run: ```bash -./gradlew dictionaryGeneralization +./gradlew dictionaryGeneralizationStandalone +./gradlew mergeDictionaryGeneralizationSnapshots ./gradlew publishDictionaryGeneralizationDocumentation ./gradlew verifyDictionaryGeneralizationDocumentation ``` -The first task writes `build/reports/generalization/dictionary-generalization.csv`. -Publication validates the complete 20 × 10 × 5 matrix, exact selected-row +The first task writes the topology-derived 146-language continuation without +rerunning the frozen original 20 defaults. The merge task validates disjoint +20/146 model sets, exact 1,000 + 7,300 scenario counts, headers, corpus identities, +and row-level provenance before creating the dated 8,300-row active snapshot. +Publication validates the complete 166 × 10 × 5 matrix, exact selected-row cardinality, count partitions, overlap arithmetic, model provenance, and the 100% boundary before checking in the CSV and its SHA-256 checksum. Verification re-renders the page from the checked-in counters and fails if either the page or diff --git a/docs/benchmarks/reference/methodology.md b/docs/benchmarks/reference/methodology.md index 48a25fc..12c9537 100644 --- a/docs/benchmarks/reference/methodology.md +++ b/docs/benchmarks/reference/methodology.md @@ -2,26 +2,28 @@ The stemmer comparison suite measures Radixor and Java stemmers on the same language and deterministic Radixor model dictionary-derived data. Published Radixor rows in this refresh use contracted compiled patch tries, where uniform preferred-command subtrees are collapsed into accepting leaves before the trie is frozen for lookup. For each language, the registered default model resource stores the expected root as the first tab-separated field on a line and its surface forms on the same line. Every single-token field on that line can therefore be paired with the same expected root. -Published speed results come only from the exact method selection retained in `published-speed-benchmarks-2026-08-25.txt`. Internal `FrequencyTrie*` microbenchmarks, quality methods, the CISTEM gold-standard experiment, and the optional `PolishPolimorfStemmerComparisonBenchmark` are not part of those results. The Snowball 3.1.0 matrix includes direct Czech, Persian, and Polish workloads; all published Java comparators were measured in the same refresh. +Published speed results come only from the exact method selection retained in `published-speed-benchmarks-2026-09-10.txt`. The parameterized Radixor method covers all 167 user-facing model IDs, while comparator rows are limited to authoritative same-language implementations; this includes the separately identified optional PoliMorf Morfologik row. Internal `FrequencyTrie*` microbenchmarks, quality methods, the CISTEM gold-standard experiment, and the ambiguous PoliMorf Radixor comparison method are excluded. ## Benchmark Passes There are two distinct benchmark passes: -- Speed benchmarks process only changed dictionary pairs where `token != expectedRoot`. This removes already-root tokens from timing so a stemmer is measured on words that actually require a transformation. If a language has fewer than 5,000 changed pairs, the complete changed-pair sequence is repeated in stable order until the timing corpus has at least 5,000 tokens. Larger changed-pair corpora are not sampled or truncated. +- Speed benchmarks preferentially process changed dictionary pairs where `token != expectedRoot`. This removes already-root tokens when the dictionary contains a transformation. A root-only model has no changed population, so its complete root-preservation corpus is used instead; the corpus report and language page identify that basis explicitly. If the selected population has fewer than 5,000 pairs, its complete sequence is repeated in stable order until the timing corpus has at least 5,000 tokens. Larger populations are not sampled or truncated. - Quality benchmarks process the complete dictionary for the language. They report exact agreement over all tokens, exact agreement over changed tokens only, and preservation of tokens that are already roots. Timing corpora are generated once per JMH JVM and kept in memory as shared `{token, expectedRoot}` arrays. Corpus construction, dictionary loading, trie loading, table loading, and analyzer construction are setup work and are not included in measured benchmark methods. -The deterministic and timed workloads are executed separately. Corpus statistics, patch-command counts, exact-root counters, coverage accuracy, and pairwise quality do not use or interpret warmup or runtime scores. Published speed and coverage-speed methods use three independent forks, five one-second warmup iterations and seven one-second measurement iterations per fork, one benchmark thread, and a fixed 6 GiB heap. +The deterministic and timed workloads are executed separately. Corpus statistics, patch-command counts, exact-root counters, coverage accuracy, and pairwise quality do not use or interpret warmup or runtime scores. Published speed and coverage-speed methods use three independent forks, three one-second warmup iterations and five one-second measurement iterations per fork, one benchmark thread, and a fixed 6 GiB heap. Performance is interpreted as average time per input token: ```text -timePerChangedTokenNs = JMH score ns/op / changedTimingTokenCount +timePerTimingTokenNs = JMH score ns/op / timingTokenCount ``` -This is necessary because Radixor dictionaries have different token counts by language. +For transformed dictionaries, `timingTokenCount` is the changed-form timing population. For a +root-only dictionary it is the explicitly reported complete root-preservation timing population. +This normalization is necessary because Radixor dictionaries have different token counts by language. ## Exact-root quality and interpretation @@ -41,11 +43,11 @@ Trie metadata records the language writing direction for inspection and intercha The quality pass reports exact-root agreement against the expected root from the default-model dictionary line. External-stemmer counters are written locally to: -- `build/reports/jmh/stemmer-accuracy-2026-08-25.csv` -- `build/reports/jmh/stemmer-accuracy-2026-08-25.txt` +- `build/reports/jmh/stemmer-accuracy-2026-09-10.csv` +- `build/reports/jmh/stemmer-accuracy-2026-09-10.txt` The tables in this documentation are verified against the checked-in -[dated accuracy CSV](../data/java-stemmer-accuracy-2026-08-25.csv), not against the mutable local +[dated accuracy CSV](../data/java-stemmer-accuracy-2026-09-10.csv), not against the mutable local report directory. Accuracy is computed from standard JMH secondary rows: @@ -60,7 +62,9 @@ rootPreservedPercent = rootPreservedMatches / rootEvaluatedTokens * 100 Morfologik can emit multiple terms for one input token. The quality benchmark uses the first emitted term for exact-root accounting when no ranking weight is exposed. Throughput benchmarks for Morfologik TokenFilter paths consume all emitted terms. -External-stemmer quality reports use JMH auxiliary counter rows from one deterministic evaluation. Radixor exact-root counts are computed directly while the default-model corpus and preferred patch commands are audited, so all 20 default models have the same coverage even where no older JMH quality adapter existed. Documentation uses counter ratios and does not interpret quality benchmark timing scores. +When `changedEvaluatedTokens` is zero, changed exactness is reported as `n/a`; an empty changed population is not a measured 0% or 100%. The overall and root-preservation ratios remain numeric when their populations are non-empty. + +External-stemmer quality reports use JMH auxiliary counter rows from one deterministic evaluation. Radixor exact-root counts are computed directly while the model corpus and preferred patch commands are audited, so all registered models have auditable exact-root coverage even where no older JMH quality adapter existed. Documentation uses counter ratios and does not interpret quality benchmark timing scores. Pairwise over-stemming, under-stemming, candidate-aware policies, and relation metrics are a separate analytical evaluation. See [Linguistic Quality Methodology](linguistic-quality.md); exact-root accuracy must not be interpreted as the complement of pairwise under-stemming. Default rows use `Language.defaultModelId()`. Optional variants require a separate model field; `pl-pl-unimorph` and `pl-pl-polimorf` must never share an ambiguous Polish label. The benchmark runtime receives each resource exactly once from its individual model JAR through direct JMH runtime dependencies. See [Model Selection and Loading](../../model-selection-and-loading.md). diff --git a/docs/benchmarks/reference/reproducibility.md b/docs/benchmarks/reference/reproducibility.md index 659811b..01484dc 100644 --- a/docs/benchmarks/reference/reproducibility.md +++ b/docs/benchmarks/reference/reproducibility.md @@ -2,24 +2,32 @@ ## Published quality snapshot -- Machine-readable CSV: [stemming-quality.csv](../data/stemming-quality.csv) -- SHA-256 record: [stemming-quality.sha256](../data/stemming-quality.sha256) -- SHA-256: `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` -- Complete scenarios: 314 -- Authoritative language universe: 20 languages -- Language-page scenarios: 314 across 20 benchmark pages +- Active machine-readable CSV: [stemming-quality-2026-09-11.csv](../data/stemming-quality-2026-09-11.csv) +- SHA-256 record: [stemming-quality-2026-09-11.sha256](../data/stemming-quality-2026-09-11.sha256) +- SHA-256: `1e893f8e16b0e96616d5b16c4b58d0a6e6c55b560e69bd06d433a3d213bbc772` +- Complete scenarios: 1,236 +- Authoritative universe: 166 language defaults; optional PoliMorf is explicitly outside this snapshot +- Historical archive: [stemming-quality.csv](../data/stemming-quality.csv), preserved byte-for-byte with SHA-256 `85763189eab4d0fbb047c2d5d3554c66abf9732182bd0d8fd758d7aef680e66f` -The CSV contains the model ID, independent model version, descriptor SHA-256, raw pair counts, raw over/under numerators and denominators, candidate statistics, and relation metrics. Reserved partition-metric columns remain empty because the gold standard is an overlapping cover. Documentation is regenerated from this file rather than manually transcribed. Publication fails when any row uses a model other than the language's registered default. +The CSV contains the model ID, independent model version, descriptor SHA-256, raw pair counts, raw over/under numerators and denominators, candidate statistics, and relation metrics. Reserved partition-metric columns remain empty because the gold standard is an overlapping cover. Documentation is regenerated from this file rather than manually transcribed. Publication fails when any row uses a model other than the language's registered default. Empty metric fields with zero denominators render as `n/a`; the raw counts remain published. `ANY_CANDIDATE` is an oracle-bound policy and has no coherent confusion matrix. + +The [active snapshot manifest](../data/active-snapshots.properties) selects the complete current +performance campaign (corpus, exact-root accuracy, speed, and both English coverage inputs), +quality, generalization, its measured-source manifest, and the comparator catalog. A refresh adds +dated files and changes the pointers; it never repurposes an archived file name. ## Published generalization snapshot -- Machine-readable CSV: [dictionary-generalization.csv](../data/dictionary-generalization.csv) -- SHA-256 record: [dictionary-generalization.sha256](../data/dictionary-generalization.sha256) -- SHA-256: `e6479840b9307ae03bd0873e55f397811e975125d621a8b8716d4c1a166b3ff2` -- Measured-source manifest: [dictionary-generalization-sources.sha256](../data/dictionary-generalization-sources.sha256) -- Complete scenarios: 1,000 -- Matrix: 20 default models × 10 coverage levels × 5 frozen splits -- Measured source identity: Radixor/Java `4.2.0-6-g84e57fb` +- Active machine-readable CSV: [dictionary-generalization-2026-09-11.csv](../data/dictionary-generalization-2026-09-11.csv) +- SHA-256 record: [dictionary-generalization-2026-09-11.sha256](../data/dictionary-generalization-2026-09-11.sha256) +- SHA-256: `0c64dda032a48a04c3e890b0669d00dd8e323000bf66274105ec7a34e87790a5` +- Continuation measured-source manifest: [dictionary-generalization-2026-09-11-sources.txt](../data/dictionary-generalization-2026-09-11-sources.txt) +- Continuation source-manifest checksum: [dictionary-generalization-2026-09-11-sources.sha256](../data/dictionary-generalization-2026-09-11-sources.sha256), SHA-256 `5b1612856d22ee6436864332968a726eedc0f625cede1d4f2b3a88f13392a222` +- Complete scenarios: 8,300 +- Matrix: 166 default models × 10 coverage levels × 5 frozen splits +- Measured source identities: the archived 20-language Radixor/Java `4.2.0-6-g84e57fb` campaign plus the separately recorded 146-language `4.3.0.dirty` continuation +- Historical archive: [dictionary-generalization.csv](../data/dictionary-generalization.csv), preserved byte-for-byte with SHA-256 `e6479840b9307ae03bd0873e55f397811e975125d621a8b8716d4c1a166b3ff2` +- Historical measured-source manifest: [dictionary-generalization-sources.sha256](../data/dictionary-generalization-sources.sha256) This CSV retains integer numerators and denominators for complete-dictionary, withheld-row, and unseen-surface scopes. It also records selected and total rows, @@ -27,10 +35,10 @@ overlap exclusions, split seed, protocol version, and exact model provenance. The [generalization methodology](generalization-methodology.md) defines the frozen split and the limits of the claim. -The accompanying manifest records the byte identity of the generator and the -Java implementation files that determine trie construction, traversal, patch -encoding, and lookup. The base revision and measured source state are retained -instead of inferring a clean release tag for this experiment. +The two campaign-specific manifests record the byte identity of the generator and the Java +implementation files that determine trie construction, traversal, patch encoding, and lookup. +Every row retains its campaign's base revision and measured source state; the active snapshot does +not infer a clean release tag for the dirty continuation. ## Published edit-cost snapshot @@ -54,11 +62,16 @@ The Java comparison tables, English coverage curve, Python runtime page, landing and technology summary are regenerated from the dated inputs below rather than from disposable files under `build/reports/`: -- [Java corpus and command report](../data/java-benchmark-corpora-2026-08-25.csv) -- [Java exact-root report](../data/java-stemmer-accuracy-2026-08-25.csv) -- [Java speed report](../data/java-stemmer-speed-2026-08-25.csv) -- [English coverage accuracy report](../data/java-english-coverage-accuracy-2026-08-25.csv) -- [English coverage speed report](../data/java-english-coverage-speed-2026-08-25.csv) +- [Current Java corpus and command report](../data/java-benchmark-corpora-2026-09-10.csv) +- [Current Java exact-root report](../data/java-stemmer-accuracy-2026-09-10.csv) +- [Current Java speed report](../data/java-stemmer-speed-2026-09-10.csv) +- [Current English coverage accuracy report](../data/java-english-coverage-accuracy-2026-09-10.csv) +- [Current English coverage speed report](../data/java-english-coverage-speed-2026-09-10.csv) +- [Previous Java corpus and command report](../data/java-benchmark-corpora-2026-08-25.csv) +- [Previous Java exact-root report](../data/java-stemmer-accuracy-2026-08-25.csv) +- [Previous Java speed report](../data/java-stemmer-speed-2026-08-25.csv) +- [Previous English coverage accuracy report](../data/java-english-coverage-accuracy-2026-08-25.csv) +- [Previous English coverage speed report](../data/java-english-coverage-speed-2026-08-25.csv) - [Python all-language batch CSV](../data/python-all-languages-batch-2026-08-25.csv) - [Python all-language provenance JSON](../data/python-all-languages-batch-2026-08-25.json) - [SHA-256 manifest](../data/performance-snapshots.sha256) @@ -76,12 +89,14 @@ these checked-in inputs and verifies every checksum in the manifest. ./gradlew --no-daemon publishStemmingQualityDocumentation \ verifyStemmingQualityDocumentation ./gradlew --no-daemon \ - -PdictionaryGeneralizationReleaseVersion=4.2.0-6-g84e57fb \ - dictionaryGeneralization + -PdictionaryGeneralizationReleaseVersion=4.3.0.dirty \ + dictionaryGeneralizationStandalone +./gradlew --no-daemon mergeDictionaryGeneralizationSnapshots \ + publishDictionaryGeneralizationDocumentation python3 tools/update-generalization-documentation.py \ - build/reports/generalization/dictionary-generalization.csv docs update -python3 tools/update-generalization-documentation.py \ - docs/benchmarks/data/dictionary-generalization.csv docs verify + docs/benchmarks/data/dictionary-generalization-2026-09-11.csv docs update \ + --corpus docs/benchmarks/data/java-benchmark-corpora-2026-09-10.csv \ + --snapshot-name dictionary-generalization-2026-09-11.csv ./gradlew --no-daemon \ -PdictionaryGeneralizationReleaseVersion=4.2.0-6-g84e57fb \ editCostSensitivity @@ -90,19 +105,19 @@ python3 tools/update-edit-cost-documentation.py \ python3 tools/update-edit-cost-documentation.py \ docs/benchmarks/data/edit-cost-sensitivity.csv.gz docs verify ./gradlew --no-daemon benchmarkCorpusReport writeJmhRuntimeClasspath -tools/run-published-accuracy-benchmarks.sh 2026-08-25 -tools/run-published-speed-benchmarks.sh 2026-08-25 4.2.0-6-g84e57fb +tools/run-published-accuracy-benchmarks.sh 2026-09-10 +tools/run-published-speed-benchmarks.sh 2026-09-10 4.3.0.dirty ./gradlew --no-daemon pythonBenchmarkAllLanguagesBatch cp build/reports/jmh/benchmark-corpora.csv \ - docs/benchmarks/data/java-benchmark-corpora-2026-08-25.csv -cp build/reports/jmh/stemmer-accuracy-2026-08-25.csv \ - docs/benchmarks/data/java-stemmer-accuracy-2026-08-25.csv -cp build/reports/jmh/stemmer-speed-2026-08-25.csv \ - docs/benchmarks/data/java-stemmer-speed-2026-08-25.csv -cp build/reports/jmh/english-coverage-accuracy-2026-08-25.csv \ - docs/benchmarks/data/java-english-coverage-accuracy-2026-08-25.csv -cp build/reports/jmh/english-coverage-speed-2026-08-25.csv \ - docs/benchmarks/data/java-english-coverage-speed-2026-08-25.csv + docs/benchmarks/data/java-benchmark-corpora-2026-09-10.csv +cp build/reports/jmh/stemmer-accuracy-2026-09-10.csv \ + docs/benchmarks/data/java-stemmer-accuracy-2026-09-10.csv +cp build/reports/jmh/stemmer-speed-2026-09-10.csv \ + docs/benchmarks/data/java-stemmer-speed-2026-09-10.csv +cp build/reports/jmh/english-coverage-accuracy-2026-09-10.csv \ + docs/benchmarks/data/java-english-coverage-accuracy-2026-09-10.csv +cp build/reports/jmh/english-coverage-speed-2026-09-10.csv \ + docs/benchmarks/data/java-english-coverage-speed-2026-09-10.csv cp build/reports/python-benchmarks/all-languages-batch.csv \ docs/benchmarks/data/python-all-languages-batch-2026-08-25.csv cp build/reports/python-benchmarks/all-languages-batch.json \ @@ -115,17 +130,22 @@ sed -i "s#${PWD}##g" \ java-stemmer-speed-2026-08-25.csv \ java-english-coverage-accuracy-2026-08-25.csv \ java-english-coverage-speed-2026-08-25.csv \ + java-benchmark-corpora-2026-09-10.csv \ + java-stemmer-accuracy-2026-09-10.csv \ + java-stemmer-speed-2026-09-10.csv \ + java-english-coverage-accuracy-2026-09-10.csv \ + java-english-coverage-speed-2026-09-10.csv \ python-all-languages-batch-2026-08-25.csv \ python-all-languages-batch-2026-08-25.json \ > performance-snapshots.sha256) python3 tools/update-benchmark-documentation.py \ - --corpus docs/benchmarks/data/java-benchmark-corpora-2026-08-25.csv \ - --accuracy docs/benchmarks/data/java-stemmer-accuracy-2026-08-25.csv \ - --speed docs/benchmarks/data/java-stemmer-speed-2026-08-25.csv \ - --coverage-accuracy docs/benchmarks/data/java-english-coverage-accuracy-2026-08-25.csv \ - --coverage-speed docs/benchmarks/data/java-english-coverage-speed-2026-08-25.csv \ - --date 2026-08-25 \ - --release-version 4.2.0-6-g84e57fb \ + --corpus docs/benchmarks/data/java-benchmark-corpora-2026-09-10.csv \ + --accuracy docs/benchmarks/data/java-stemmer-accuracy-2026-09-10.csv \ + --speed docs/benchmarks/data/java-stemmer-speed-2026-09-10.csv \ + --coverage-accuracy docs/benchmarks/data/java-english-coverage-accuracy-2026-09-10.csv \ + --coverage-speed docs/benchmarks/data/java-english-coverage-speed-2026-09-10.csv \ + --date 2026-09-10 \ + --release-version 4.3.0.dirty \ --mode update python3 tools/update-python-benchmark-documentation.py \ --csv docs/benchmarks/data/python-all-languages-batch-2026-08-25.csv \ @@ -157,7 +177,7 @@ the checked-in catalog and must never rewrite it ad hoc. Audit mode is enabled with `-PstemmingQualityAudit=true`. Language, stemmer, dictionary-mode, output-policy, and ranking filters are documented on the central [stemming-quality page](../../stemming-quality.md). Filtered reports use separate filenames and cannot be accepted as publication sources. -`publishStemmingQualityDocumentation` validates the complete build CSV, copies a versioned documentation snapshot, and replaces only marked generated sections. `verifyStemmingQualityDocumentation` re-renders from the checked-in snapshot and fails on changed values, ordering, missing pages, duplicate keys, arithmetic inconsistencies, policy violations, or stale sections. +`publishStemmingQualityDocumentation` validates the complete build CSV, copies it to a new dated documentation snapshot, updates the active pointer, and replaces only marked generated sections. It never overwrites the historical `stemming-quality.csv`. `verifyStemmingQualityDocumentation` re-renders from the active checked-in snapshot and fails on changed values, ordering, missing pages, duplicate keys, arithmetic inconsistencies, policy violations, or stale sections. The staged site under `build/` is disposable output. The canonical model catalog is the reviewed, checked-in `docs/stemmer-model-catalog.md`; the staged copy must remain identical to it. @@ -181,10 +201,11 @@ not recorded for the legacy UniMorph imports; that limitation is disclosed with upstream revision and source-archive checksum. This reproducibility limitation does not replace or weaken the packaged license and attribution requirements. -Each UniMorph-derived model artifact carries its own notice with the canonical CC BY-SA 3.0 URI, -upstream attribution, transformations, ShareAlike statement, and Leo Galambos contribution notice. -The full CC legal text is not duplicated or presented as a root-project license. PoliMorf retains -its separately packaged BSD-2-Clause license. +Each UniMorph-derived model artifact carries its own notice with the exact audited license URI, +upstream attribution, transformations, applicable distribution statement, and Leo Galambos +contribution notice. Most are CC BY-SA 3.0; CC BY-SA 4.0, CC BY 4.0, and LGPLLR exceptions are +identified exactly. Khaling additionally packages the canonical LGPLLR text and legible dictionary +form. PoliMorf retains its separately packaged BSD-2-Clause license. For a future full PoliMorf measurement, also record the startup heap separately from benchmark parameters. Complete runtime construction is currently verified with a dedicated 6 GiB maximum heap; this limit is neither a retained-trie measurement nor a setting applied to ordinary JMH runs. @@ -196,15 +217,15 @@ The current accuracy, speed, and coverage commands are: ```bash ./gradlew --no-daemon benchmarkCorpusReport writeJmhRuntimeClasspath -tools/run-published-accuracy-benchmarks.sh 2026-08-25 -tools/run-published-speed-benchmarks.sh 2026-08-25 4.2.0-6-g84e57fb +tools/run-published-accuracy-benchmarks.sh 2026-09-10 +tools/run-published-speed-benchmarks.sh 2026-09-10 4.3.0.dirty ``` -The speed runner refuses to start unless every CPU uses the `performance` governor, materializes the exact selected benchmark list, rejects quality/Polimorf/gold-standard methods, and requires the Hebrew speed path. It records hardware, JVM, source-state, JAR, classpath, corpus, quality, load, temperature, and governor provenance before running. The accuracy runner evaluates all four exact-root benchmark classes and verifies that every new Snowball 3.1.0 candidate exposes all six accuracy counters. The exact JMH configuration is listed in [Environment and reports](environment.md). Quality and performance reports are separate datasets and are not combined into an undocumented scalar. +The speed runner refuses to start unless every CPU uses the `performance` governor, materializes the exact selected benchmark list, and rejects quality, gold-standard, and internal microbenchmark methods. It validates that the parameterized Radixor method covers all 167 user-facing model IDs and selects only the approved same-language comparator paths, including the separately identified optional PoliMorf comparator. It records hardware, JVM, source state, executable JAR, the complete runtime-classpath content manifest, corpus, quality, load, temperature, and governor provenance. After the main CSV passes model-coverage validation, the runner also records its SHA-256 before the coverage run. The accuracy runner evaluates the exact-root benchmark suite and verifies the complete counter rows for every selected Snowball 3.1.0 candidate. The exact JMH configuration is listed in [Environment and reports](environment.md). Quality and performance reports are separate datasets and are not combined into an undocumented scalar. ## Recorded and unavailable provenance -The performance documentation records its 2026-08-25 environment, JDK, operating system, hardware, base revision, exact measured-source patch, untracked-source checksums, executable JMH JAR checksum, and model descriptor checksums. The quality CSV embeds model identity and checksum in every row; run date, core source state, JVM, OS, and hardware are shared provenance on the environment page. +The performance documentation records its 2026-09-10 environment, JDK, operating system, hardware, base revision, exact measured-source patch, untracked-source checksums, executable JMH JAR checksum, complete runtime-classpath content manifest, main-report checksum, and model descriptor checksums. The quality CSV embeds model identity and checksum in every row; run date, core source state, JVM, OS, and hardware are shared provenance on the environment page. Exact immutable upstream revisions were not recorded for every legacy UniMorph import. That limitation remains explicit in model descriptors and cannot be repaired from filesystem timestamps. Dependency versions reproducible from repository configuration include Apache Lucene 10.5.0, Morfologik 2.1.9, the Ukrainian dictionary artifact 4.9.1, and JMH 1.37. diff --git a/docs/benchmarks/reference/tested-stemmers.md b/docs/benchmarks/reference/tested-stemmers.md index 90100e6..2c62d6e 100644 --- a/docs/benchmarks/reference/tested-stemmers.md +++ b/docs/benchmarks/reference/tested-stemmers.md @@ -4,10 +4,10 @@ The JMH adapter registry is authoritative for evaluated implementations and lang | Family or implementation | Upstream / attribution | Tested version or revision | Evaluated scope | Output capability and adapter behaviour | Interpretation notes | | --- | --- | --- | --- | --- | --- | -| Radixor | Egothor / Radixor project | Base commit and measured working-tree state recorded on the environment page | All 20 reconciled default model languages; all 20 have benchmark pages | Deterministic preferred patch via `get`; ranked distinct alternatives via `getAll`; primary is always included | Dictionary-trained compiled patch trie. Default rows use each language's stable default model ID. | +| Radixor | Egothor / Radixor project | Base commit and measured working-tree state recorded on the environment page | All 166 default languages have quality and benchmark pages; speed covers 167 user-facing models including optional PoliMorf | Deterministic preferred patch via `get`; ranked distinct alternatives via `getAll`; primary is always included | Dictionary-trained compiled patch trie. Default rows use each language's stable default model ID. | | Apache Lucene language stem filters | Apache Lucene project | 10.5.0 | Adapter-declared language-specific subsets | TokenFilter lifecycle and language normalization match JMH; normally single-output | Light, minimal, possessive, and language stem filters deliberately implement different scopes. Narrow scope is not a defect. | | Apache Lucene SnowballFilter | Apache Lucene project using Snowball algorithms | Lucene 10.5.0 | Snowball-supported subset of Radixor languages | Single primary token emitted through the Lucene TokenFilter path | Includes TokenStream overhead and required normalization. | -| Official Snowball Java | Snowball project | 3.1.0 source distribution; SHA-256 `5dab34d491f55f47b6e971569ffe6aadf5991512c648ddfe5d331b494cf6d655` | 17 same-language direct adapters, including the Czech, Persian, and Polish stemmers added in 3.1.0 | Direct generated Java API; single output | Rule-based suffix algorithms provide broad baselines rather than dictionary-root guarantees. Lucene 10.5.0 does not yet expose the three new algorithms through `SnowballFilter`, so those rows are direct-only. | +| Official Snowball Java | Snowball project | 3.1.0 source distribution; SHA-256 `5dab34d491f55f47b6e971569ffe6aadf5991512c648ddfe5d331b494cf6d655` | 29 exact-language direct adapters; 28 participate in the homepage aggregate and root-only Arabic is charted separately | Direct generated Java API; single output | Rule-based suffix algorithms provide broad baselines rather than dictionary-root guarantees. Lucene 10.5.0 exposes 25 of these algorithms through `SnowballFilter`; the remaining four comparisons use the direct API only. | | Lucene Stempel | Apache Lucene / Polish stemming tables | Lucene 10.5.0 | Polish | Direct and TokenFilter paths where registered; single primary output | Table-driven Polish implementation. | | Morfologik | Morfologik project; Lucene integration by Apache Lucene | Morfologik 2.1.9, Lucene integration 10.5.0; Ukrainian dictionary artifact 4.9.1 | Registered Polish and Ukrainian paths | Deterministic first lemma for primary comparison; all distinct lemma strings for candidate policies | Several analyses may share a lemma and are deduplicated by exact string equality. | | Hunspell via Lucene | Hunspell dictionaries from the `wooorm/dictionaries` repository; adapter by Apache Lucene | Lucene 10.5.0; dictionary repository revision was not recorded | Configured German, English, Spanish, French, Dutch, Polish, and Ukrainian dictionaries | First emitted stem is primary; all distinct stems at the token position are candidates | Dictionary content and affix rules differ by language. | diff --git a/docs/built-in-languages.md b/docs/built-in-languages.md index 8eb027f..68eeb21 100644 --- a/docs/built-in-languages.md +++ b/docs/built-in-languages.md @@ -1,10 +1,10 @@ # Built-in Languages and Default Models -“Supported language” means that the repository maintains a default dictionary -and runtime mapping for that language. Packaging differs by runtime: Java keeps -the core dictionary-free and resolves external model artifacts, while the -Python installs a separate `radixor-models-standard` data package containing -the 20 default dictionaries in precompiled version 7 form. +“Registered language” means that the repository maintains a dictionary and a +stable Java runtime mapping for that language. Packaging differs by runtime: +Java keeps the core dictionary-free and resolves individual model artifacts, +while the Java and Python `radixor-models-standard` aggregates intentionally +remain limited to the original 20 standard dictionaries. The Java language enum carries language identity, writing direction, a legacy resource-directory name, and the stable default model ID. A Java model @@ -13,33 +13,158 @@ Python accepts the short alias or the same full model ID. See [Model Selection and Loading](model-selection-and-loading.md) for Java and [Python Usage and API](python/usage.md) for Python. -## Defaults and variants - -| Language | Java enum | Python alias | Default model ID | Java default artifact | Optional variants | -|---|---|---|---|---|---| -| Czech | `CS_CZ` | `cs` | `cs-cz-default` | `org.egothor:radixor-model-cs-cz-default` | — | -| Danish | `DA_DK` | `da` | `da-dk-default` | `org.egothor:radixor-model-da-dk-default` | — | -| German | `DE_DE` | `de` | `de-de-default` | `org.egothor:radixor-model-de-de-default` | — | -| Spanish | `ES_ES` | `es` | `es-es-default` | `org.egothor:radixor-model-es-es-default` | — | -| Persian | `FA_IR` | `fa` | `fa-ir-default` | `org.egothor:radixor-model-fa-ir-default` | — | -| Finnish | `FI_FI` | `fi` | `fi-fi-default` | `org.egothor:radixor-model-fi-fi-default` | — | -| French | `FR_FR` | `fr` | `fr-fr-default` | `org.egothor:radixor-model-fr-fr-default` | — | -| Hebrew | `HE_IL` | `he` | `he-il-default` | `org.egothor:radixor-model-he-il-default` | — | -| Hungarian | `HU_HU` | `hu` | `hu-hu-default` | `org.egothor:radixor-model-hu-hu-default` | — | -| Italian | `IT_IT` | `it` | `it-it-default` | `org.egothor:radixor-model-it-it-default` | — | -| Norwegian Bokmål | `NB_NO` | `nb` | `nb-no-default` | `org.egothor:radixor-model-nb-no-default` | — | -| Dutch | `NL_NL` | `nl` | `nl-nl-default` | `org.egothor:radixor-model-nl-nl-default` | — | -| Norwegian Nynorsk | `NN_NO` | `nn` | `nn-no-default` | `org.egothor:radixor-model-nn-no-default` | — | -| Polish | `PL_PL` | `pl` | `pl-pl-unimorph` | `org.egothor:radixor-model-pl-pl-unimorph` | `pl-pl-polimorf` / `org.egothor:radixor-model-pl-pl-polimorf` | -| Portuguese | `PT_PT` | `pt` | `pt-pt-default` | `org.egothor:radixor-model-pt-pt-default` | — | -| Russian | `RU_RU` | `ru` | `ru-ru-default` | `org.egothor:radixor-model-ru-ru-default` | — | -| Swedish | `SV_SE` | `sv` | `sv-se-default` | `org.egothor:radixor-model-sv-se-default` | — | -| Ukrainian | `UK_UA` | `uk` | `uk-ua-default` | `org.egothor:radixor-model-uk-ua-default` | — | -| English | `US_UK` | `en` | `us-uk-default` | `org.egothor:radixor-model-us-uk-default` | — | -| Yiddish | `YI` | `yi` | `yi-default` | `org.egothor:radixor-model-yi-default` | — | - -The maintained table deliberately avoids duplicating mutable provenance and checksum fields. Those values come from module metadata and are generated into the model catalog. - +## Registered languages and models + +The standard Java and Python aggregates remain fixed at the original 20 `default` entries. Every `standalone` entry is a registered Java language default and an individually publishable artifact, but is not installed by either standard aggregate. The optional PoliMorf model is also individual only. + +| Language | Java enum | Model ID | Topology role | Relative dictionary size | +| --- | --- | --- | --- | --- | +| Adyghe | `ADY` | `ady-default` | `standalone` | ★★★★☆ (20,334 distinct forms) | +| Afrikaans | `AF_ZA` | `af-za-default` | `standalone` | ★★★★★ (289,303 distinct forms) | +| Aimele | `AIL` | `ail-default` | `standalone` | ★★★☆☆ (2,978 distinct forms) | +| Akan | `AK` | `aka-default` | `standalone` | ★★☆☆☆ (1,890 distinct forms) | +| Albanian | `SQ_AL` | `sq-al-default` | `standalone` | ★★★☆☆ (10,218 distinct forms) | +| Alsatian | `GSW` | `gsw-default` | `standalone` | ★★☆☆☆ (1,171 distinct forms) | +| Amharic | `AM_ET` | `am-et-default` | `standalone` | ★★★★☆ (41,308 distinct forms) | +| Amol Azerbaijani | `AZG` | `azg-default` | `standalone` | ★★★☆☆ (9,357 distinct forms) | +| Ancient Greek | `GRC` | `grc-default` | `standalone` | ★★★★☆ (11,109 distinct forms) | +| Anglo-Norman | `XNO` | `xno-default` | `standalone` | ★☆☆☆☆ (186 distinct forms) | +| Arabic | `AR` | `ar-default` | `standalone` | ★★★★★ (452,974 distinct forms) | +| Armenian | `HY_AM` | `hy-am-default` | `standalone` | ★★★★★ (242,091 distinct forms) | +| Ashaninka | `CNI` | `cni-default` | `standalone` | ★★★☆☆ (10,095 distinct forms) | +| Assamese | `AS_IN` | `as-in-default` | `standalone` | ★☆☆☆☆ (328 distinct forms) | +| Asturian | `AST` | `ast-default` | `standalone` | ★★★★☆ (21,584 distinct forms) | +| Aymara | `AYM` | `aym-default` | `standalone` | ★★★★★ (231,247 distinct forms) | +| Azerbaijani | `AZ_AZ` | `az-az-default` | `standalone` | ★★★☆☆ (6,669 distinct forms) | +| Bashkir | `BAK` | `bak-default` | `standalone` | ★★★★☆ (10,616 distinct forms) | +| Belarusian | `BE_BY` | `be-by-default` | `standalone` | ★★☆☆☆ (678 distinct forms) | +| Bengali | `BN_BD` | `bn-bd-default` | `standalone` | ★☆☆☆☆ (3 distinct forms) | +| Bininj Kun-wok | `GUP` | `gup-default` | `standalone` | ★☆☆☆☆ (370 distinct forms) | +| Braj | `BRA` | `bra-default` | `standalone` | ★☆☆☆☆ (115 distinct forms) | +| Breton | `BRE` | `bre-default` | `standalone` | ★★☆☆☆ (1,682 distinct forms) | +| Bulgarian | `BG_BG` | `bg-bg-default` | `standalone` | ★★★★☆ (37,335 distinct forms) | +| Catalan | `CA_ES` | `ca-es-default` | `standalone` | ★★★★★ (61,495 distinct forms) | +| Cebuano | `CEB` | `ceb-default` | `standalone` | ★☆☆☆☆ (392 distinct forms) | +| Chichewa | `NY_MW` | `ny-mw-default` | `standalone` | ★★★☆☆ (3,150 distinct forms) | +| Chukchi | `CKT` | `ckt-default` | `standalone` | ★☆☆☆☆ (302 distinct forms) | +| Church Slavonic | `CHU` | `chu-default` | `standalone` | ★★☆☆☆ (1,632 distinct forms) | +| Classical Armenian | `XCL` | `xcl-default` | `standalone` | ★★★★☆ (55,946 distinct forms) | +| Classical Syriac | `SYC` | `syc-default` | `standalone` | ★★★☆☆ (2,139 distinct forms) | +| Congo Swahili | `SWC` | `swc-default` | `standalone` | ★★★☆☆ (6,700 distinct forms) | +| Copala Triqui | `CPA` | `cpa-default` | `standalone` | ★★☆☆☆ (1,751 distinct forms) | +| Cornish | `COR` | `cor-default` | `standalone` | ★☆☆☆☆ (159 distinct forms) | +| Cree | `CRE` | `cre-default` | `standalone` | ★★☆☆☆ (514 distinct forms) | +| Crimean Tatar | `CRH` | `crh-default` | `standalone` | ★★★☆☆ (7,183 distinct forms) | +| Czech | `CS_CZ` | `cs-cz-default` | `default` | ★★★★☆ (51,401 distinct forms) | +| Dakota | `DAK` | `dak-default` | `standalone` | ★★★☆☆ (2,942 distinct forms) | +| Danish | `DA_DK` | `da-dk-default` | `default` | ★★★★☆ (27,921 distinct forms) | +| Dutch | `NL_NL` | `nl-nl-default` | `default` | ★★★★☆ (26,201 distinct forms) | +| Eastern Chatino | `CLY` | `cly-default` | `standalone` | ★★☆☆☆ (2,030 distinct forms) | +| Egyptian Arabic | `ARZ` | `arz-default` | `standalone` | ★☆☆☆☆ (135 distinct forms) | +| English | `US_UK` | `us-uk-default` | `default` | ★★★★★ (591,946 distinct forms) | +| Estonian | `ET_EE` | `et-ee-default` | `standalone` | ★★★★☆ (24,811 distinct forms) | +| Evenki | `EVN` | `evn-default` | `standalone` | ★★★★☆ (11,965 distinct forms) | +| Faroese | `FO_FO` | `fo-fo-default` | `standalone` | ★★★★☆ (31,366 distinct forms) | +| Finnish | `FI_FI` | `fi-fi-default` | `default` | ★★★★★ (1,788,784 distinct forms) | +| French | `FR_FR` | `fr-fr-default` | `default` | ★★★★★ (404,011 distinct forms) | +| Friulian | `FUR` | `fur-default` | `standalone` | ★★★☆☆ (5,007 distinct forms) | +| Ga | `GAA` | `gaa-default` | `standalone` | ★☆☆☆☆ (469 distinct forms) | +| Galolen | `GAL` | `gal-default` | `standalone` | ★★★★☆ (25,436 distinct forms) | +| German | `DE_DE` | `de-de-default` | `default` | ★★★★★ (277,266 distinct forms) | +| Gothic | `GOT` | `got-default` | `standalone` | ★★★★★ (134,162 distinct forms) | +| Greek | `EL_GR` | `el-gr-default` | `standalone` | ★★★★★ (76,851 distinct forms) | +| Gulf Arabic | `AFB` | `afb-default` | `standalone` | ★☆☆☆☆ (355 distinct forms) | +| Haida | `HAI` | `hai-default` | `standalone` | ★★☆☆☆ (520 distinct forms) | +| Hebrew | `HE_IL` | `he-il-default` | `default` | ★★★★☆ (57,658 distinct forms) | +| Hiligaynon | `HIL` | `hil-default` | `standalone` | ★☆☆☆☆ (308 distinct forms) | +| Hsilimo | `HSI` | `hsi-default` | `standalone` | ★☆☆☆☆ (158 distinct forms) | +| Hungarian | `HU_HU` | `hu-hu-default` | `default` | ★★★★★ (910,688 distinct forms) | +| Icelandic | `IS_IS` | `is-is-default` | `standalone` | ★★★★☆ (52,132 distinct forms) | +| Indonesian | `ID_ID` | `id-id-default` | `standalone` | ★★★★☆ (21,296 distinct forms) | +| Ingrian | `IZH` | `izh-default` | `standalone` | ★★☆☆☆ (1,024 distinct forms) | +| Irish | `GA_IE` | `ga-ie-default` | `standalone` | ★★★★☆ (21,829 distinct forms) | +| Italian | `IT_IT` | `it-it-default` | `default` | ★★★★★ (324,366 distinct forms) | +| Itelmen | `ITL` | `itl-default` | `standalone` | ★★★☆☆ (3,546 distinct forms) | +| Japanese | `JA_JP` | `ja-jp-default` | `standalone` | ★★★★☆ (10,848 distinct forms) | +| Kabardian | `KBD` | `kbd-default` | `standalone` | ★★★☆☆ (3,054 distinct forms) | +| Kalaallisut | `KL_GL` | `kl-gl-default` | `standalone` | ★☆☆☆☆ (321 distinct forms) | +| Kannada | `KN_IN` | `kn-in-default` | `standalone` | ★☆☆☆☆ (11 distinct forms) | +| Karelian | `KRL` | `krl-default` | `standalone` | ★★☆☆☆ (566 distinct forms) | +| Kashubian | `CSB` | `csb-default` | `standalone` | ★☆☆☆☆ (350 distinct forms) | +| Kazakh | `KK_KZ` | `kk-kz-default` | `standalone` | ★☆☆☆☆ (357 distinct forms) | +| Khakas | `KJH` | `kjh-default` | `standalone` | ★★☆☆☆ (1,172 distinct forms) | +| Khinalugh | `KLR` | `klr-default` | `standalone` | ★★★★☆ (41,185 distinct forms) | +| Kodi | `KOD` | `kod-default` | `standalone` | ★★☆☆☆ (524 distinct forms) | +| Kongo | `KON` | `kon-default` | `standalone` | ★★☆☆☆ (585 distinct forms) | +| Kyrgyz | `KY_KG` | `ky-kg-default` | `standalone` | ★★★☆☆ (2,997 distinct forms) | +| Ladin | `LLD` | `lld-default` | `standalone` | ★★★☆☆ (4,819 distinct forms) | +| Latin | `LA` | `la-default` | `standalone` | ★★★★★ (485,259 distinct forms) | +| Latvian | `LV_LV` | `lv-lv-default` | `standalone` | ★★★★★ (75,492 distinct forms) | +| Lingala | `LIN` | `lin-default` | `standalone` | ★☆☆☆☆ (230 distinct forms) | +| Lithuanian | `LT_LT` | `lt-lt-default` | `standalone` | ★★★★☆ (27,335 distinct forms) | +| Livonian | `LIV` | `liv-default` | `standalone` | ★★★☆☆ (2,841 distinct forms) | +| Low German | `NDS` | `nds-default` | `standalone` | ★★★☆☆ (2,493 distinct forms) | +| Lower Sorbian | `DSB` | `dsb-default` | `standalone` | ★★★★☆ (12,038 distinct forms) | +| Luganda | `LG_UG` | `lg-ug-default` | `standalone` | ★★★☆☆ (4,671 distinct forms) | +| Macedonian | `MK_MK` | `mk-mk-default` | `standalone` | ★★★★★ (135,590 distinct forms) | +| Magahi | `MAG` | `mag-default` | `standalone` | ★☆☆☆☆ (206 distinct forms) | +| Malagasy | `MG_MG` | `mg-mg-default` | `standalone` | ★★☆☆☆ (636 distinct forms) | +| Maltese | `MT_MT` | `mt-mt-default` | `standalone` | ★★☆☆☆ (1,421 distinct forms) | +| Manx | `GV_IM` | `gv-im-default` | `standalone` | ★☆☆☆☆ (15 distinct forms) | +| Maori | `MI_NZ` | `mi-nz-default` | `standalone` | ★☆☆☆☆ (207 distinct forms) | +| Mapudungun | `ARN` | `arn-default` | `standalone` | ★★☆☆☆ (548 distinct forms) | +| Middle French | `FRM` | `frm-default` | `standalone` | ★★★★☆ (27,002 distinct forms) | +| Middle High German | `GMH` | `gmh-default` | `standalone` | ★☆☆☆☆ (384 distinct forms) | +| Middle Low German | `GML` | `gml-default` | `standalone` | ★★☆☆☆ (565 distinct forms) | +| Mongolian | `MN_MN` | `mn-mn-default` | `standalone` | ★★★★☆ (17,231 distinct forms) | +| Murrinh-Patha | `MWF` | `mwf-default` | `standalone` | ★★☆☆☆ (592 distinct forms) | +| Navajo | `NAV` | `nav-default` | `standalone` | ★★★☆☆ (9,826 distinct forms) | +| Neapolitan | `NAP` | `nap-default` | `standalone` | ★★☆☆☆ (1,497 distinct forms) | +| North Frisian | `FRR` | `frr-default` | `standalone` | ★☆☆☆☆ (340 distinct forms) | +| Northern Sami | `SME` | `sme-default` | `standalone` | ★★★★☆ (53,034 distinct forms) | +| Norwegian Bokmål | `NB_NO` | `nb-no-default` | `default` | ★★★★★ (73,170 distinct forms) | +| Norwegian Nynorsk | `NN_NO` | `nn-no-default` | `default` | ★★★★☆ (16,937 distinct forms) | +| Old English | `ANG` | `ang-default` | `standalone` | ★★★★★ (65,100 distinct forms) | +| Old French | `FRO` | `fro-default` | `standalone` | ★★★★★ (93,998 distinct forms) | +| Old High German | `GOH` | `goh-default` | `standalone` | ★★★☆☆ (5,267 distinct forms) | +| Old Irish | `SGA` | `sga-default` | `standalone` | ★★☆☆☆ (929 distinct forms) | +| Old Norse | `NON` | `non-default` | `standalone` | ★★★★☆ (46,067 distinct forms) | +| Old Saxon | `OSX` | `osx-default` | `standalone` | ★★★★☆ (12,361 distinct forms) | +| Oodham | `OOD` | `ood-default` | `standalone` | ★☆☆☆☆ (455 distinct forms) | +| Pashto | `PS_AF` | `ps-af-default` | `standalone` | ★★★☆☆ (2,945 distinct forms) | +| Persian | `FA_IR` | `fa-ir-default` | `default` | ★★★☆☆ (3,544 distinct forms) | +| Polish | `PL_PL` | `pl-pl-polimorf` | `optional` | ★★★★★ (4,668,685 distinct forms) | +| Polish | `PL_PL` | `pl-pl-unimorph` | `default` | ★★★★★ (120,867 distinct forms) | +| Portuguese | `PT_PT` | `pt-pt-default` | `default` | ★★★★★ (211,091 distinct forms) | +| Quechua | `QUE` | `que-default` | `standalone` | ★★★★★ (122,820 distinct forms) | +| Romanian | `RO_RO` | `ro-ro-default` | `standalone` | ★★★★☆ (48,497 distinct forms) | +| Russian | `RU_RU` | `ru-ru-default` | `default` | ★★★★★ (759,333 distinct forms) | +| Seneca | `SEE` | `see-default` | `standalone` | ★☆☆☆☆ (483 distinct forms) | +| Serbo-Croatian | `HBS` | `hbs-default` | `standalone` | ★★★★★ (272,515 distinct forms) | +| Shipibo-Conibo | `SHP` | `shp-default` | `standalone` | ★★★☆☆ (7,705 distinct forms) | +| Shona | `SN_ZW` | `sn-zw-default` | `standalone` | ★★★☆☆ (2,640 distinct forms) | +| Sotho, Southern | `ST_ZA` | `st-za-default` | `standalone` | ★☆☆☆☆ (416 distinct forms) | +| Southern Kurdish | `SDH` | `sdh-default` | `standalone` | ★☆☆☆☆ (165 distinct forms) | +| Spanish | `ES_ES` | `es-es-default` | `default` | ★★★★★ (849,661 distinct forms) | +| Swedish | `SV_SE` | `sv-se-default` | `default` | ★★★★★ (95,181 distinct forms) | +| Tagalog | `TL_PH` | `tl-ph-default` | `standalone` | ★★★☆☆ (2,042 distinct forms) | +| Temoaya Otomi | `OTE` | `ote-default` | `standalone` | ★★☆☆☆ (1,785 distinct forms) | +| Turkish | `TR_TR` | `tr-tr-default` | `standalone` | ★★★★★ (222,207 distinct forms) | +| Ukrainian | `UK_UA` | `uk-ua-default` | `default` | ★★★★☆ (14,150 distinct forms) | +| Uyghur | `UG_CN` | `ug-cn-default` | `standalone` | ★★★☆☆ (6,200 distinct forms) | +| Uzbek | `UZ_UZ` | `uz-uz-default` | `standalone` | ★★☆☆☆ (1,080 distinct forms) | +| Voro | `VRO` | `vro-default` | `standalone` | ★☆☆☆☆ (277 distinct forms) | +| Western Highland Chatino | `CTP` | `ctp-default` | `standalone` | ★★★☆☆ (2,456 distinct forms) | +| Xibe | `SJO` | `sjo-default` | `standalone` | ★★★☆☆ (3,151 distinct forms) | +| Xipinawa | `XTY` | `xty-default` | `standalone` | ★★☆☆☆ (1,536 distinct forms) | +| Yamana | `AME` | `ame-default` | `standalone` | ★★☆☆☆ (1,489 distinct forms) | +| Yiddish | `YI` | `yi-default` | `default` | ★★★☆☆ (3,532 distinct forms) | +| Zacatepec Chatino | `ZPV` | `zpv-default` | `standalone` | ★★☆☆☆ (812 distinct forms) | +| Zarma | `DJE` | `dje-default` | `standalone` | ★☆☆☆☆ (75 distinct forms) | +| Zenzontepec Chatino | `CZN` | `czn-default` | `standalone` | ★☆☆☆☆ (68 distinct forms) | +| Zulu | `ZU_ZA` | `zu-za-default` | `standalone` | ★★★★☆ (32,384 distinct forms) | + +The maintained table deliberately avoids duplicating mutable provenance fields. Those values come from model module metadata and the generated model catalog. ## The Polish dual-model case `PL_PL` represents Polish. It is not an alias for either source dictionary. @@ -73,7 +198,7 @@ dependencies { } ``` -All documented defaults: +All 20 standard-aggregate defaults: ```groovy dependencies { @@ -85,11 +210,13 @@ dependencies { The standard pack is metadata-only and excludes optional PoliMorf. Every individual model artifact carries its own provenance and licensing material. UniMorph -models carry different model-specific CC BY-SA 3.0 notices because their official language -repositories identify different lexical sources and contributors. Each notice preserves upstream +models carry model-specific audited licenses and notices because their official language +repositories identify different lexical sources, contributors, and applicable terms. Each notice preserves upstream attribution and records the Radixor transformations and Leo Galambos contribution statement. Legacy imports disclose when an exact historical revision was not recorded; this is a -reproducibility limitation, not a claim that the source or license is unknown. +reproducibility limitation, not a claim that the source or license is unknown. The active new-model +set includes CC BY-SA 3.0, CC BY-SA 4.0, CC BY 4.0, and LGPLLR material; unsupported evidence is +quarantined rather than assigned a generic license. ## Loading a language default @@ -105,7 +232,7 @@ The call discovers the default descriptor from the runtime classpath, verifies i ## Writing direction -Persian, Hebrew, and Yiddish declare right-to-left writing metadata for presentation. Writing direction does not reorder characters in a Java `String`: all built-in natural-language models therefore use backward traversal from the stored sequence end, where suffixes remain located. Explicit forward traversal is reserved for deliberately prefix-oriented custom data. The selected traversal must remain aligned across dictionary parsing, trie lookup, patch generation, persistence, and application; model identity and writing direction are separate concerns. +Arabic (including Gulf and Egyptian Arabic), Persian, Hebrew, Pashto, Southern Kurdish, Classical Syriac, Uyghur, Urdu, and Yiddish declare right-to-left writing metadata for presentation. Writing direction does not reorder characters in a Java `String`: all built-in natural-language models therefore use backward traversal from the stored sequence end, where suffixes remain located. Explicit forward traversal is reserved for deliberately prefix-oriented custom data. The selected traversal must remain aligned across dictionary parsing, trie lookup, patch generation, persistence, and application; model identity and writing direction are separate concerns. ## Custom and persisted alternatives diff --git a/docs/dictionary-format.md b/docs/dictionary-format.md index ce9fffd..4da0e31 100644 --- a/docs/dictionary-format.md +++ b/docs/dictionary-format.md @@ -23,8 +23,8 @@ Consequently, compressed size is not a construction-memory estimate. The PoliMor Comment headers in maintained model inputs summarize provenance but do not replace packaged legal material. Each UniMorph-derived artifact includes a language-specific notice describing its -official repository, lexical source, upstream attribution, CC BY-SA 3.0 canonical URI, ShareAlike -status, Radixor transformations, and Leo Galambos's protectable model-data contributions. The +official repository, lexical source, upstream attribution, exact audited license and canonical +URI, Radixor transformations, and Leo Galambos's protectable model-data contributions. The notice does not claim ownership over the underlying data. GZip packaging and descriptor/checksum generation are disclosed transformations; the in-memory trie is a Radixor runtime structure. diff --git a/docs/index.md b/docs/index.md index daa37e9..0e527ea 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,12 +1,12 @@ --- template: landing.html title: Radixor -description: Java flagship and two native Python runtimes using interoperable compiled patch-command tries across 20 languages. +description: Java flagship and two native Python runtimes using interoperable compiled patch-command tries across 143 language defaults. --- # Radixor Learned transformation stemming with a flagship Java implementation and two native Python choices: batch-oriented `radixor` and scalar-oriented -`radixor-c`. All use compact reduced tries of patch commands across 20 -languages. +`radixor-c`. Java publishes 144 individually available models across 143 +language pages; the standard aggregate remains the original 20-model pack. diff --git a/docs/integration-deep-dive.md b/docs/integration-deep-dive.md index 67a52cf..c367138 100644 --- a/docs/integration-deep-dive.md +++ b/docs/integration-deep-dive.md @@ -78,7 +78,13 @@ module example.search { The core ships no language dictionary. Add one or more `radixor-model-` artifacts, or the optional metadata-only standard pack. Each model JAR contains an indexed descriptor and a namespaced GZip dictionary. `StemmerPatchTrieLoader.Language` represents language properties and a stable default model ID; it does not own embedded data. -The standard option is specifically a POM-only runtime dependency aggregate, not an all-model binary JAR. It resolves one default model JAR per language and excludes optional PoliMorf. The separate POM-only `radixor-models-bom` manages recommended versions without adding runtime artifacts. Repository tests and JMH attach individual model projects directly to non-production configurations, so neither path changes the root publication's dependency graph. +The standard option is specifically a POM-only runtime dependency aggregate, +not an all-model binary JAR. It resolves the original 20 standard model JARs +and excludes the 123 standalone language models and optional PoliMorf. The +separate POM-only `radixor-models-bom` manages recommended versions for all 144 +user-facing models without adding runtime artifacts. Repository tests and JMH +attach individual model projects directly to non-production configurations, so +neither path changes the root publication's dependency graph. For minimal deployments choose only required model artifacts. For multiple Polish variants add both `pl-pl-unimorph` and `pl-pl-polimorf`, retain UniMorph as the language default, and request PoliMorf explicitly. See [Model Selection and Loading](model-selection-and-loading.md) for complete dependencies and [Built-in Languages](built-in-languages.md) for mappings. diff --git a/docs/migration-and-backward-compatibility.md b/docs/migration-and-backward-compatibility.md index a6eddb4..4e32b4d 100644 --- a/docs/migration-and-backward-compatibility.md +++ b/docs/migration-and-backward-compatibility.md @@ -92,7 +92,7 @@ UniMorph and PoliMorf are not interchangeable quality datasets. They can differ Model migration does not erase source obligations. Each migrated UniMorph artifact packages its language-specific notice with upstream attribution, Radixor modifications and contribution -statement, ShareAlike terms, and the canonical CC BY-SA 3.0 URI. The original imports did not +statement, applicable license terms, and the canonical license URI. The original imports did not record exact UniMorph commits, so descriptors use `source.revision=not-recorded-in-legacy-import` and disclose that fact. Future model imports must record an exact upstream revision and source-archive checksum. @@ -135,7 +135,12 @@ git tag -a "models-catalog@" -m "Release Radixor model catalog A core tag publishes only the root `org.egothor:radixor` software artifacts, never model JARs. A model tag validates and publishes exactly its matching module, never core, standard, BOM, JMH, or the multilingual quality suite. A catalog tag publishes only BOM and standard aggregate metadata. Local model dry-run: -The catalog artifacts are POM-only: `radixor-models-standard` carries runtime dependencies on the 20 defaults, while `radixor-models-bom` carries dependency-management constraints for all 21 individual models. Neither publishes an empty binary, sources, or Javadoc JAR. This Maven BOM is distinct from the root CycloneDX SBOM report under `build/reports/sbom/`. +The catalog artifacts are POM-only: `radixor-models-standard` carries runtime +dependencies on the original 20 standard models, while `radixor-models-bom` +carries dependency-management constraints for all 144 user-facing individual +models. Neither publishes an empty binary, sources, or Javadoc JAR. This Maven +BOM is distinct from the root CycloneDX SBOM report under +`build/reports/sbom/`. ```bash ./tools/parse-model-release-tag.sh "model/pl-pl-polimorf@" . diff --git a/docs/model-selection-and-loading.md b/docs/model-selection-and-loading.md index c11b8dc..9fc61f4 100644 --- a/docs/model-selection-and-loading.md +++ b/docs/model-selection-and-loading.md @@ -27,8 +27,8 @@ These identifiers answer different questions: One language can have several models. `Language.PL_PL` is neither UniMorph nor PoliMorf. `loadCompiled(Language.PL_PL, ...)` resolves the stable default ID declared by `Language.defaultModelId()`. An explicit lookup requests exactly one ID. Registry ordering never changes either decision. Licensing follows the selected artifact. Radixor Java software is BSD-3-Clause; UniMorph-derived -model data carries a model-specific CC BY-SA 3.0 notice, while PoliMorf carries its separate -BSD-2-Clause license. The UniMorph notice preserves upstream attribution and identifies the +model data carries its model-specific audited license and notice, while PoliMorf carries its +separate BSD-2-Clause license. The UniMorph notice preserves upstream attribution and identifies the Radixor transformations and limited protectable contributions without claiming the underlying data. ## Choose runtime dependencies @@ -78,7 +78,10 @@ dependencies { } ``` -The standard aggregate is POM-only. Its POM supplies exactly one default model per supported language as transitive runtime dependencies and excludes optional PoliMorf. It publishes no empty binary JAR. +The standard aggregate is POM-only. Its POM supplies exactly the original 20 +standard models as transitive runtime dependencies. It excludes the 123 +standalone language models and optional PoliMorf; all remain available through +individual artifacts. It publishes no empty binary JAR. ### BOM-managed versions diff --git a/docs/overrides/landing.html b/docs/overrides/landing.html index 9a987f7..65c5f3b 100644 --- a/docs/overrides/landing.html +++ b/docs/overrides/landing.html @@ -41,7 +41,7 @@
Dictionary-trained. Not dictionary-bound.

High-quality multilingual stemming.
Built for Java and Python.

-

Radixor learns word-to-stem transformations from lexical evidence, compiles them into compact tries of patch commands, and applies those transformations algorithmically at runtime. It combines deterministic results, small deployable models, and high throughput across 20 languages. The training dictionary supplies evidence—it does not define a closed runtime vocabulary.

+

Radixor learns word-to-stem transformations from lexical evidence, compiles them into compact tries of patch commands, and applies those transformations algorithmically at runtime. It combines deterministic results, small deployable models, and high throughput across 166 language defaults. The training dictionary supplies evidence—it does not define a closed runtime vocabulary.

@@ -69,13 +69,53 @@