{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:10:38Z","timestamp":1760123438531,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,2,10]],"date-time":"2023-02-10T00:00:00Z","timestamp":1675987200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>A growing number of papers on style transfer for texts rely on information decomposition. The performance of the resulting systems is usually assessed empirically in terms of the output quality or requires laborious experiments. This paper suggests a straightforward information theoretical framework to assess the quality of information decomposition for latent representations in the context of style transfer. Experimenting with several state-of-the-art models, we demonstrate that such estimates could be used as a fast and straightforward health check for the models instead of more laborious empirical experiments.<\/jats:p>","DOI":"10.3390\/e25020322","type":"journal-article","created":{"date-parts":[[2023,2,10]],"date-time":"2023-02-10T02:33:27Z","timestamp":1675996407000},"page":"322","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Quick Estimate of Information Decomposition for Text Style Transfer"],"prefix":"10.3390","volume":"25","author":[{"given":"Viacheslav","family":"Shibaev","sequence":"first","affiliation":[{"name":"Department of Intelligent Information Technologies, Ural Federal University, 620075 Ekaterinburg, Russia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3987-1271","authenticated-orcid":false,"given":"Eckehard","family":"Olbrich","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Mathematics in the Sciences Leipzig, 04103 Leipzig, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5258-6590","authenticated-orcid":false,"given":"J\u00fcrgen","family":"Jost","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Mathematics in the Sciences Leipzig, 04103 Leipzig, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3784-0671","authenticated-orcid":false,"given":"Ivan P.","family":"Yamshchikov","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Mathematics in the Sciences Leipzig, 04103 Leipzig, Germany"},{"name":"CEMAPRE, University of Lisbon, 1649-004 Lisboa, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,10]]},"reference":[{"key":"ref_1","unstructured":"Hu, Z., Yang, Z., Liang, X., Salakhutdinov, R., and Xing, E.P. (2017, January 13\u201316). Toward Controlled Generation of Text. Proceedings of the International Conference on Machine Learning, Ho Chi Minh City, Vietnam."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bowman, S.R., Angeli, G., Potts, C., and Manning, C.D. (2015, January 17\u201321). A large annotated corpus for learning natural language inference. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1075"},{"key":"ref_3","unstructured":"Shen, T., Lei, T., Barzilay, R., and Jaakkola, T. (2017, January 4\u20139). Style transfer from non-parallel text by cross-alignment. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Fu, Z., Tan, X., Peng, N., Zhao, D., and Yan, R. (2018, January 2\u20133). Style transfer in text: Exploration and evaluation. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11330"},{"key":"ref_5","unstructured":"Tian, Y., Hu, Z., and Yu, Z. (2018). Structured Content Preservation for Unsupervised Text Style Transfer. arXiv."},{"key":"ref_6","unstructured":"Romanov, A., Rumshisky, A., Rogers, A., and Donahue, D. (2019). Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Association for Computational Linguistics."},{"key":"ref_7","unstructured":"John, V., Mou, L., Bahuleyan, H., and Vechtomova, O. (August, January 28). Disentangled Representation Learning for Non-Parallel Text Style Transfer. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Halevi, G., and Wadhawan, K. (2019, January 4\u20136). Text Style Transfer Using Partly-Shared Decoder. Proceedings of the 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), Portland, OR, USA.","DOI":"10.1109\/ICTAI.2019.00225"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Tikhonov, A., Shibaev, V., Nagaev, A., Nugmanova, A., and Yamshchikov, I.P. (2019, January 3\u20137). Style Transfer for Texts: Retrain, Report Errors, Compare with Rewrites. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1406"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1145\/3544903.3544906","article-title":"Text style transfer: A review and experimental evaluation","volume":"24","author":"Hu","year":"2022","journal-title":"ACM Sigkdd Explor. Newsl."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rao, S., and Tetreault, J. (2018, January 1\u20136). Dear Sir or Madam, May I Introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-1012"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lyu, Y., Liang, P.P., Pham, H., Hovy, E., Pocz\u00f3s, B., Salakhutdinov, R., and Morency, L.P. (2021, January 6\u201311). StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.","DOI":"10.18653\/v1\/2021.naacl-main.171"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Briakou, E., Lu, D., Zhang, K., and Tetreault, J. (2021, January 6\u201311). Ol\u00e1, bonjour, salve! XFORMAL: A benchmark for multilingual formality style transfer. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Online.","DOI":"10.18653\/v1\/2021.naacl-main.256"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Yamshchikov, I.P., Shibaev, V., Nagaev, A., Jost, J., and Tikhonov, A. (2019, January 4). Decomposing Textual Information For Style Transfer. Proceedings of the 3rd Workshop on Neural Generation and Translation, Hong Kong, China.","DOI":"10.18653\/v1\/D19-5613"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002, January 7\u201312). GBLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Xu, W. (2017, January 8). From Shakespeare to Twitter: What are Language Styles all about?. Proceedings of the Workshop on Stylistic Variation, Copenhagen, Denmark.","DOI":"10.18653\/v1\/W17-4901"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"7682","DOI":"10.1073\/pnas.1115407109","article-title":"Quantitative patterns of stylistic influence in the evolution of literature","volume":"109","author":"Hughes","year":"2012","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Potash, P., Romanov, A., and Rumshisky, A. (2015, January 17\u201321). GhostWriter: Using an LSTM for Automatic Rap Lyric Generation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1221"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"115","DOI":"10.25046\/aj040614","article-title":"Learning Literary Style End-to-end with Artificial Neural Networks","volume":"4","author":"Yamshchikov","year":"2019","journal-title":"Adv. Sci. Technol. Eng. Syst. J."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Sennrich, R., Haddow, B., and Birch, A. (2016, January 12\u201317). Controlling politeness in neural machine translation via side constraints. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, CA, USA.","DOI":"10.18653\/v1\/N16-1005"},{"key":"ref_21","unstructured":"Xu, W., Ritter, A., Dolan, W.B., Grishman, R., and Cherry, C. (2012, January 8\u201315). Paraphrasing for style. Proceedings of the COLING, Mumbai, India."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jhamtani, H., Gangal, V., Hovy, E., and Nyberg, E. (2017, January 8). Shakespearizing Modern Language Using Copy-Enriched Sequence-to-Sequence Models. Proceedings of the Workshop on Stylistic Variation, Copenhagen, Denmark.","DOI":"10.18653\/v1\/W17-4902"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Prabhumoye, S., Tsvetkov, Y., Black, A.W., and Salakhutdinov, R. (2018). Style Transfer Through Back-Translation. arXiv.","DOI":"10.18653\/v1\/P18-1080"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ficler, J., and Goldberg, Y. (2017, January 8). Controlling linguistic style aspects in neural language generation. Proceedings of the Workshop on Stylistic Variation, Copenhagen, Denmark.","DOI":"10.18653\/v1\/W17-4912"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lin, K., Liu, M.Y., Sun, M.T., and Kautz, J. (2020, January 10). Learning to Generate Multiple Style Transfer Outputs for an Input Sentence. Proceedings of the Fourth Workshop on Neural Generation and Translation, Online.","DOI":"10.18653\/v1\/2020.ngt-1.2"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Cheng, P., Min, M.R., Shen, D., Malon, C., Zhang, Y., Li, Y., and Carin, L. (2020). Improving Disentangled Text Representation Learning with Information-Theoretic Guidance. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.673"},{"key":"ref_27","unstructured":"Qian, K., Zhang, Y., Chang, S., Hasegawa-Johnson, M., and Cox, D. (2020, January 13\u201318). Unsupervised speech decomposition via triple information bottleneck. Proceedings of the International Conference on Machine Learning. PMLR, Virtual."},{"key":"ref_28","unstructured":"Tikhonov, A., and Yamshchikov, I.P. (2018). What is wrong with style transfer for texts?. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Kabbara, J., and Cheung, J.C.K. (2016, January 5). Stylistic transfer in natural language generation systems using recurrent neural networks. Proceedings of the Workshop on Uphill Battles in Language Processing: Scaling Early Achievements to Robust Methods, Austin, TX, USA.","DOI":"10.18653\/v1\/W16-6010"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Li, J., Jia, R., He, H., and Liang, P. (2018, January 1\u20136). Delete, Retrieve, Generate: A Simple Approach to Sentiment and Style Transfer. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, LA, USA.","DOI":"10.18653\/v1\/N18-1169"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Xu, J., Sun, X., Zeng, Q., Ren, X., Zhang, X., Wang, H., and Li, W. (2018). Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach. arXiv.","DOI":"10.18653\/v1\/P18-1090"},{"key":"ref_32","unstructured":"Yamshchikov, I.P., Shibaev, V., Khlebnikov, N., and Tikhonov, A. (March, January 22). Style-transfer and Paraphrase: Looking for a Sensible Semantic Similarity Metric. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_34","unstructured":"Williams, P., and Beer, R. (2010). Nonnegative Decomposition of Multivariate Information. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2161","DOI":"10.3390\/e16042161","article-title":"Quantifying unique information","volume":"16","author":"Bertschinger","year":"2014","journal-title":"Entropy"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1109\/TIT.1954.1057469","article-title":"Multivariate information transmission","volume":"4","author":"McGill","year":"1954","journal-title":"Trans. Ire Prof. Group Inf. Theory"},{"key":"ref_37","unstructured":"Bell, A.J. (2003, January 22\u201324). The co-information lattice. Proceedings of the Fifth International Workshop on Independent Component Analysis and Blind Signal Separation: ICA, Granada, Spain."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"066138","DOI":"10.1103\/PhysRevE.69.066138","article-title":"Estimating mutual information","volume":"69","author":"Kraskov","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ross, B.C. (2014). Mutual information between discrete and continuous data sets. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0087357"},{"key":"ref_40","unstructured":"Subramanian, S., Lample, G., Smith, E.M., Denoyer, L., Ranzato, M.A., and Boureau, Y.L. (2018). Multiple-Attribute Text Style Transfer. arXiv."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/322\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:29:36Z","timestamp":1760120976000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/322"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,10]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["e25020322"],"URL":"https:\/\/doi.org\/10.3390\/e25020322","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2023,2,10]]}}}