Berlin51 language metrics faced a cross-language test.
Coherence scores rely on word-embedding models trained on one language’s corpus, so the same score can mean different things elsewhere.
Simple word-count measures outperformed complex ones, the study’s authors say, undercutting fancier diagnostic models.
Sources: Nature