{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,13]],"date-time":"2026-09-13T23:03:11Z","timestamp":1789340591347,"version":"build-2803163510"},"reference-count":26,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,14]],"date-time":"2026-09-14T00:00:00Z","timestamp":1789344000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Genet."],"abstract":"<jats:p>Automated platforms that classify sequence variants under the American College of Medical Genetics and Genomics\/Association for Molecular Pathology (ACMG\/AMP) framework are increasingly used in rare-disease genomics, yet methodology for characterizing their performance at an early organizational stage is underdeveloped. The two framings most common in the literature are poorly suited to this stage: single-source comparison treats one peer laboratory or curated set as ground truth, which is difficult to defend at evidence depths where qualified laboratories disagree 25%\u201335% of the time; and formal regulatory adjudication requires multi-adjudicator panels and quality-management infrastructure that small or single-founder developers do not yet have. This paper proposes a methodological framework for Real-World Performance Studies that occupies the space between informal in-house testing and formal regulatory adjudication. The framework has four components: a three-layer performance model that separates analytical, classification, and clinical performance and forces every observation to a locus of attribution; a multi-source ground-truth construction with an explicit, evidence-strength-ordered weighting hierarchy; a six-category methodological-disposition taxonomy that resolves platform-versus-comparator disagreement into characterized categories with distinct action implications, only one of which denotes a classifier defect; and a phased pathway that carries early-stage evidence forward toward eventual regulatory submission. The framework is demonstrated, not validated, on three real-world cohorts (97 scored cases across three classifier versions) of a variant classification platform developed by Helena Bioinformatics. The demonstration illustrates how the method is operationalized. It does not test, validate, or claim superiority for any system. The framework is non-proprietary and is offered for independent adoption and adaptation. Its principal limitations - single-platform demonstration, single peer comparator, a taxonomy developed on the same cohorts that illustrate it, and structurally challenging conflicts of interest - are stated explicitly and are the reason the paper claims a method, not a result. The proposed framework is a conceptual methodology operationalized as a structured human-in-the-loop protocol; it is not a turnkey software package, and empirical use requires version-locked automated outputs plus documented expert review.<\/jats:p>","DOI":"10.3389\/fgene.2026.1925492","type":"journal-article","created":{"date-parts":[[2026,9,13]],"date-time":"2026-09-13T22:10:12Z","timestamp":1789337412000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["A methodological framework for real-world performance studies of clinical variant classification platforms at early organizational stages"],"prefix":"10.3389","volume":"17","author":[{"given":"Vladimir","family":"Mitev","sequence":"first","affiliation":[{"name":"Helena Bioinformatics","place":["Sofia, Bulgaria"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,14]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1517","DOI":"10.1002\/humu.23626","article-title":"Recommendations for interpreting the loss of function PVS1 ACMG\/AMP variant criterion","volume":"39","author":"Abou Tayoun","year":"2018","journal-title":"Hum. Mutat."},{"key":"B2","doi-asserted-by":"publisher","first-page":"1067","DOI":"10.1016\/j.ajhg.2016.03.024","article-title":"Performance of ACMG-AMP variant-interpretation guidelines among nine laboratories in the clinical sequencing exploratory research consortium","volume":"98","author":"Amendola","year":"2016","journal-title":"Am. J. Hum. Genet."},{"key":"B3","doi-asserted-by":"publisher","first-page":"932","DOI":"10.1016\/j.ajhg.2020.09.011","article-title":"Variant classification concordance using the ACMG-AMP variant interpretation guidelines across nine genomic implementation research studies","volume":"107","author":"Amendola","year":"2020","journal-title":"Am. J. Hum. Genet."},{"key":"B4","doi-asserted-by":"publisher","first-page":"101402","DOI":"10.1016\/j.gim.2025.101402","article-title":"Calibration of additional computational tools expands ClinGen recommendation options for variant classification with PP3\/BP4 criteria","volume":"27","author":"Bergquist","year":"2025","journal-title":"Genet. Med."},{"key":"B5","doi-asserted-by":"publisher","first-page":"1687","DOI":"10.1038\/gim.2018.42","article-title":"The ACMG\/AMP reputable source criteria for the interpretation of sequence variants","volume":"20","author":"Biesecker","year":"2018","journal-title":"Genet. Med."},{"key":"B6","volume-title":"Human Genetic and Genomic Testing Using Traditional and High-Throughput Nucleic Acid Sequencing Methods","year":"2023"},{"key":"B7","doi-asserted-by":"publisher","first-page":"e0295010","DOI":"10.1371\/journal.pone.0295010","article-title":"A taxonomic schema of potential pitfalls in clinical variant analysis based on real-world evidence","volume":"18","author":"Coovadia","year":"2023","journal-title":"PLoS One"},{"key":"B8","unstructured":"ACGS best practice guidelines for variant classification in rare disease 2024, version 1.2. association for clinical genomic science\n          \n          \n            \n              Durkie\n              M.\n            \n            \n              Cassidy\n              E.-J.\n            \n            \n              Berry\n              I.\n            \n            \n              Owens\n              M.\n            \n            \n              Turnbull\n              C.\n            \n            \n              Scott\n              R. H.\n            \n          \n          \n          2024"},{"key":"B9","doi-asserted-by":"publisher","first-page":"btaf623","DOI":"10.1093\/bioinformatics\/btaf623","article-title":"Comprehensive evaluation of ACMG\/AMP-based variant classification tools","volume":"42","author":"Ghasemnejad","year":"2026","journal-title":"Bioinformatics"},{"key":"B10","doi-asserted-by":"publisher","first-page":"3216","DOI":"10.1093\/bioinformatics\/btr540","article-title":"Kaviar: an accessible system for testing SNV novelty","volume":"27","author":"Glusman","year":"2011","journal-title":"Bioinformatics"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1096","DOI":"10.1038\/gim.2017.14","article-title":"Clinical laboratories collaborate to resolve differences in variant interpretations submitted to ClinVar","volume":"19","author":"Harrison","year":"2017","journal-title":"Genet. Med."},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1101\/2025.11.24.25340888","article-title":"The ten year evolution of sherloc, a points-based framework for genetic variant classification","author":"Kobayashi","year":"2025"},{"key":"B13","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1186\/s13073-025-01543-4","article-title":"Recommendations for bioinformatics in clinical practice","volume":"17","author":"Lavrichenko","year":"2025","journal-title":"Genome Med."},{"key":"B14","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1038\/gim.2017.119","article-title":"Improved diagnostic yield compared with targeted gene sequencing panels suggests a role for whole-genome sequencing as a first-tier genetic test","volume":"20","author":"Lionel","year":"2018","journal-title":"Genet. Med."},{"key":"B15","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1186\/s12920-022-01166-3","article-title":"GeneTerpret: a customizable multilayer approach to genomic variant prioritization and interpretation","volume":"15","author":"Manshaei","year":"2022","journal-title":"BMC Med. Genomics"},{"key":"B16","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1186\/s13073-020-00748-z","article-title":"The medical genome initiative: moving whole-genome sequencing for rare disease diagnosis to the clinic","volume":"12","author":"Marshall","year":"2020","journal-title":"Genome Med."},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.21105027","article-title":"A methodological framework for real-world performance studies of clinical variant classification platforms at early organizational stages","author":"Mitev","year":"2026","journal-title":"Zenodo"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1105","DOI":"10.1038\/gim.2017.37","article-title":"Sherloc: a comprehensive refinement of the ACMG-AMP variant classification criteria","volume":"19","author":"Nykamp","year":"2017","journal-title":"Genet. Med."},{"key":"B19","doi-asserted-by":"publisher","first-page":"e0170365","DOI":"10.1371\/journal.pone.0170365","article-title":"Clinical practice guidelines for rare diseases: the orphanet database","volume":"12","author":"Pavan","year":"2017","journal-title":"PLoS One"},{"key":"B20","doi-asserted-by":"publisher","first-page":"2163","DOI":"10.1016\/j.ajhg.2022.10.013","article-title":"Calibration of computational tools for missense variant pathogenicity classification and ClinGen recommendations for PP3\/BP4 criteria","volume":"109","author":"Pejaver","year":"2022","journal-title":"Am. J. Hum. Genet."},{"key":"B21","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1038\/gim.2015.30","article-title":"Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American college of medical genetics and genomics and the association for molecular pathology","volume":"17","author":"Richards","year":"2015","journal-title":"Genet. Med."},{"key":"B22","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/j.jmoldx.2017.11.003","article-title":"Standards and guidelines for validating next-generation sequencing bioinformatics pipelines: a joint recommendation of the association for molecular pathology and the college of American pathologists","volume":"20","author":"Roy","year":"2018","journal-title":"J. Mol. Diagn."},{"key":"B23","doi-asserted-by":"publisher","first-page":"1054","DOI":"10.1038\/gim.2017.210","article-title":"Modeling the ACMG\/AMP variant classification guidelines as a Bayesian classification framework","volume":"20","author":"Tavtigian","year":"2018","journal-title":"Genet. Med."},{"key":"B24","doi-asserted-by":"publisher","first-page":"1734","DOI":"10.1002\/humu.24088","article-title":"Fitting a naturally scaled point system to the ACMG\/AMP variant classification guidelines","volume":"41","author":"Tavtigian","year":"2020","journal-title":"Hum. Mutat."},{"key":"B25","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1038\/s41436-019-0705-9","article-title":"Variant interpretation is a component of clinical practice among genetic counselors in multiple specialties","volume":"22","author":"Wain","year":"2020","journal-title":"Genet. Med."},{"key":"B26","doi-asserted-by":"publisher","first-page":"1046","DOI":"10.1016\/j.ajhg.2023.06.002","article-title":"Using the ACMG\/AMP framework to capture evidence related to predicted and observed impact on splicing: recommendations from the ClinGen SVI splicing subgroup","volume":"110","author":"Walker","year":"2023","journal-title":"Am. J. Hum. Genet."}],"container-title":["Frontiers in Genetics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fgene.2026.1925492\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,13]],"date-time":"2026-09-13T22:10:14Z","timestamp":1789337414000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fgene.2026.1925492\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,14]]},"references-count":26,"alternative-id":["10.3389\/fgene.2026.1925492"],"URL":"https:\/\/doi.org\/10.3389\/fgene.2026.1925492","relation":{},"ISSN":["1664-8021"],"issn-type":[{"value":"1664-8021","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,14]]},"article-number":"1925492"}}