{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T17:14:05Z","timestamp":1774631645773,"version":"3.50.1"},"reference-count":60,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T00:00:00Z","timestamp":1709251200000},"content-version":"vor","delay-in-days":60,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Changing an attribute of a text without changing the content usually requires first disentangling the text into irrelevant attributes and content representations. After that, in the inference phase, the representation of one attribute is tuned to a different value, expecting that the corresponding attribute of the text can also be changed accordingly. The usual way of disentanglement is to add some constraints on the latent space of an encoder-decoder architecture, including adversarial-based constraints and mutual-information-based constraints. However, previous semi-supervised processes of attribute change are usually not enough to guarantee the success of attribute change and content preservation. In this paper, we propose a novel approach to achieve a robust control of attributes while enhancing content preservation. In this approach, we use a semi-supervised contrastive learning method to encourage the disentanglement of attributes in latent spaces. Differently from previous works, we re-disentangle the reconstructed sentence and compare the re-disentangled latent space with the original latent space, which makes a closed-loop disentanglement process. This also helps content preservation. In addition, the contrastive learning method is also able to replace the role of minimizing mutual information and adversarial training in the disentanglement process, which alleviates the computation cost. We conducted experiments on three text datasets, including the Yelp Service review dataset, the Amazon Product review dataset, and the GoEmotions dataset. The experimental results show the effectiveness of our model.<\/jats:p>","DOI":"10.1162\/tacl_a_00640","type":"journal-article","created":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T18:22:56Z","timestamp":1709317376000},"page":"190-209","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":3,"title":["Text Attribute Control via Closed-Loop Disentanglement"],"prefix":"10.1162","volume":"12","author":[{"given":"Lei","family":"Sha","sequence":"first","affiliation":[{"name":"Institute of Artificial Intelligence, Beihang University, China"},{"name":"Department of Computer Science, University of Oxford, UK"},{"name":"Zhongguancun Laboratory, Beijing, China. shalei@buaa.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"Lukasiewicz","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Oxford, UK"},{"name":"Institute of Logic and Computation, Vienna University of Technology, Austria. thomas.lukasiewicz@tuwien.ac.at"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2024,3,1]]},"reference":[{"key":"2024030118224361000_bib1","first-page":"8769","article-title":"Unsupervised state representation learning in Atari","volume-title":"Proceedings of the 33rd International Conference on Neural Information Processing Systems","author":"Anand","year":"2019"},{"key":"2024030118224361000_bib2","doi-asserted-by":"publisher","first-page":"194","DOI":"10.18653\/v1\/P19-1019","article-title":"An effective approach to unsupervised machine translation","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Artetxe","year":"2019"},{"key":"2024030118224361000_bib3","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1399","article-title":"Unsupervised neural machine translation","volume-title":"International Conference on Learning Representations","author":"Artetxe","year":"2018"},{"key":"2024030118224361000_bib4","first-page":"15535","article-title":"Learning representations by maximizing mutual information across views","volume":"32","author":"Bachman","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024030118224361000_bib5","doi-asserted-by":"publisher","first-page":"10","DOI":"10.18653\/v1\/K16-1002","article-title":"Generating sentences from a continuous space","volume-title":"Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning","author":"Bowman","year":"2016"},{"key":"2024030118224361000_bib6","article-title":"Unsupervised learning of visual features by contrasting cluster assignments","volume-title":"34th Conference on Neural Information Processing Systems (NeurIPS)","author":"Caron","year":"2020"},{"issue":"3","key":"2024030118224361000_bib7","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02172-5_2","article-title":"Large scale online learning of image similarity through ranking","volume":"11","author":"Chechik","year":"2010","journal-title":"Journal of Machine Learning Research"},{"key":"2024030118224361000_bib8","first-page":"2610","article-title":"Isolating sources of disentanglement in variational autoencoders","volume-title":"Advances in Neural Information Processing Systems","author":"Chen","year":"2018"},{"key":"2024030118224361000_bib9","first-page":"1597","article-title":"A simple framework for contrastive learning of visual representations","volume-title":"International Conference on Machine Learning","author":"Chen","year":"2020"},{"key":"2024030118224361000_bib10","first-page":"22243","article-title":"Big self-supervised models are strong semi-supervised learners","volume":"33","author":"Chen","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024030118224361000_bib11","first-page":"2172","article-title":"InfoGAN: Interpretable representation learning by information maximizing generative adversarial nets","volume-title":"Advances in Neural Information Processing Systems","author":"Xi","year":"2016"},{"key":"2024030118224361000_bib12","article-title":"Improved baselines with momentum contrastive learning","author":"Chen","year":"2020","journal-title":"arXiv preprint arXiv:2003.04297"},{"key":"2024030118224361000_bib13","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00950","article-title":"An empirical study of training self-supervised vision transformers","author":"Chen","year":"2021","journal-title":"arXiv preprint arXiv:2104.02057"},{"issue":"4","key":"2024030118224361000_bib14","doi-asserted-by":"publisher","DOI":"10.3390\/e24040456","article-title":"Ctrl: Closed-loop transcription to an ldr via minimaxing rate reduction","volume":"24","author":"Dai","year":"2022","journal-title":"Entropy"},{"key":"2024030118224361000_bib15","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.372","article-title":"GoEmotions: A dataset of fine-grained emotions","volume-title":"58th Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Demszky","year":"2020"},{"key":"2024030118224361000_bib16","volume-title":"Feedback and Control Systems","author":"Di Stefano","year":"1967"},{"key":"2024030118224361000_bib17","article-title":"Structured disentangled representations","author":"Esmaeili","year":"2018","journal-title":"arXiv preprint arXiv:1804.02086"},{"key":"2024030118224361000_bib18","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11330","article-title":"Style transfer in text: Exploration and evaluation","volume-title":"Proceedings of the 32th AAAI Conference on Artificial Intelligence","author":"Zhenxin","year":"2018"},{"key":"2024030118224361000_bib19","article-title":"Watching the world go by: Representation learning from unlabeled videos","author":"Gordon","year":"2020","journal-title":"arXiv preprint arXiv:2003.07990"},{"key":"2024030118224361000_bib20","first-page":"297","article-title":"Noise-contrastive estimation: A new estimation principle for unnormalized statistical models","volume-title":"Proceedings of the 13th International Conference on Artificial Intelligence and Statistics","author":"Gutmann","year":"2010"},{"issue":"2","key":"2024030118224361000_bib21","article-title":"Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics","volume":"13","author":"Gutmann","year":"2012","journal-title":"Journal of Machine Learning Research"},{"key":"2024030118224361000_bib22","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1109\/CVPR.2006.100","article-title":"Dimensionality reduction by learning an invariant mapping","volume-title":"2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906)","author":"Hadsell","year":"2006"},{"key":"2024030118224361000_bib23","article-title":"A probabilistic formulation of unsupervised text style transfer","volume-title":"International Conference on Learning Representations","author":"He","year":"2019"},{"key":"2024030118224361000_bib24","doi-asserted-by":"publisher","first-page":"9729","DOI":"10.1109\/CVPR42600.2020.00975","article-title":"Momentum contrast for unsupervised visual representation learning","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"He","year":"2020"},{"issue":"5","key":"2024030118224361000_bib25","first-page":"6","article-title":"Beta-VAE: Learning basic visual concepts with a constrained variational framework","volume":"2","author":"Higgins","year":"2017","journal-title":"International Conference on Learning Representations"},{"key":"2024030118224361000_bib26","first-page":"3","article-title":"Autoencoders, minimum description length, and Helmholtz free energy","volume":"6","author":"Hinton","year":"1994","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024030118224361000_bib27","article-title":"Representation learning with video deep infomax","author":"Devon Hjelm","year":"2020","journal-title":"arXiv preprint arXiv:2007.13278"},{"key":"2024030118224361000_bib28","article-title":"Learning deep representations by mutual information estimation and maximization","author":"Devon Hjelm","year":"2018","journal-title":"arXiv preprint arXiv:1808.06670"},{"key":"2024030118224361000_bib29","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1007\/978-3-319-24261-3_7","article-title":"Deep metric learning using triplet network","volume-title":"International Workshop on Similarity-based Pattern Recognition","author":"Hoffer","year":"2015"},{"key":"2024030118224361000_bib30","article-title":"ELBO surgery: Yet another way to carve up the variational evidence lower bound","volume-title":"Proceedings of the Workshop in Advances in Approximate Bayesian Inference, NIPS","author":"Hoffman","year":"2016"},{"key":"2024030118224361000_bib31","doi-asserted-by":"publisher","first-page":"424","DOI":"10.18653\/v1\/P19-1041","article-title":"Disentangled representation learning for non-parallel text style transfer","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"John","year":"2019"},{"key":"2024030118224361000_bib32","article-title":"Supervised contrastive learning","volume":"33","author":"Khosla","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024030118224361000_bib33","article-title":"Disentangling by factorising","author":"Kim","year":"2018","journal-title":"arXiv preprint arXiv:1802.05983"},{"key":"2024030118224361000_bib34","doi-asserted-by":"publisher","first-page":"1746","DOI":"10.3115\/v1\/D14-1181","article-title":"Convolutional neural networks for sentence classification","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Kim","year":"2014"},{"key":"2024030118224361000_bib35","article-title":"Auto-encoding variational Bayes","author":"Kingma","year":"2014","journal-title":"Proceedings of the International Conference on Learning Representations"},{"key":"2024030118224361000_bib36","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1109\/ICASSP.1995.479394","article-title":"Improved backing-off for m-gram language modeling","volume-title":"1995 International Conference on Acoustics, Speech, and Signal Processing","author":"Kneser","year":"1995"},{"key":"2024030118224361000_bib37","volume-title":"Content Analysis: An Introduction to Its Methodology","author":"Krippendorff","year":"2004"},{"key":"2024030118224361000_bib38","doi-asserted-by":"publisher","first-page":"737","DOI":"10.18653\/v1\/2020.emnlp-main.55","article-title":"Reformulating unsupervised style transfer as paraphrase generation","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Krishna","year":"2020"},{"key":"2024030118224361000_bib39","article-title":"Variational inference of disentangled latent concepts from unlabeled observations","author":"Kumar","year":"2017","journal-title":"arXiv preprint arXiv:1711.00848"},{"key":"2024030118224361000_bib40","article-title":"Multiple-attribute text rewriting","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Lample","year":"2019"},{"key":"2024030118224361000_bib41","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3031549","article-title":"Contrastive representation learning: A framework and review","author":"Le-Khac","year":"2020","journal-title":"IEEE Access"},{"key":"2024030118224361000_bib42","article-title":"Prototypical contrastive learning of unsupervised representations","volume-title":"International Conference on Learning Representations","author":"Li","year":"2020"},{"key":"2024030118224361000_bib43","first-page":"5103","article-title":"Content preserving text generation with attribute controls","volume-title":"Advances in Neural Information Processing Systems","author":"Logeswaran","year":"2018"},{"key":"2024030118224361000_bib44","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1194","article-title":"Towards Fine-grained text sentiment transfer","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Luo","year":"2019"},{"key":"2024030118224361000_bib45","article-title":"Prompt-based editing for text style transfer","author":"Luo","year":"2023","journal-title":"arXiv preprint arXiv:2301.11997"},{"key":"2024030118224361000_bib46","article-title":"Disentangling disentanglement in variational autoencoders","author":"Mathieu","year":"2018","journal-title":"arXiv preprint arXiv:1812.02833"},{"key":"2024030118224361000_bib47","first-page":"9084","article-title":"Invariant representations without adversarial training","volume-title":"Advances in Neural Information Processing Systems","author":"Moyer","year":"2018"},{"key":"2024030118224361000_bib48","first-page":"5925","article-title":"Learning disentangled representations with semi-supervised deep generative models","volume-title":"Advances in Neural Information Processing Systems","author":"Narayanaswamy","year":"2017"},{"key":"2024030118224361000_bib49","article-title":"Representation learning with contrastive predictive coding","author":"van den Oord","year":"2018","journal-title":"arXiv preprint arXiv:1807.03748"},{"key":"2024030118224361000_bib50","doi-asserted-by":"publisher","first-page":"2912","DOI":"10.18653\/v1\/2022.findings-acl.229","article-title":"Controllable natural language generation with contrastive prefixes","volume-title":"Findings of the Association for Computational Linguistics: ACL 2022","author":"Qian","year":"2022"},{"issue":"1","key":"2024030118224361000_bib51","first-page":"5485","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"The Journal of Machine Learning Research"},{"key":"2024030118224361000_bib52","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-short.94","article-title":"A recipe for arbitrary text style transfer with large language models","author":"Reif","year":"2021","journal-title":"arXiv preprint arXiv:2109.03910"},{"key":"2024030118224361000_bib53","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1109\/ICRA.2018.8462891","article-title":"Time-contrastive networks: Self-supervised learning from video","volume-title":"2018 IEEE International Conference on Robotics and Automation (ICRA)","author":"Sermanet","year":"2018"},{"key":"2024030118224361000_bib54","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i11.17146","article-title":"Multi-type disentanglement without adversarial training","volume-title":"Proceedings of the 35th AAAI Conference on Artificial Intelligence","author":"Sha","year":"2021"},{"key":"2024030118224361000_bib55","first-page":"8655","article-title":"ControlVAE: Controllable variational autoencoder","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Shao","year":"2020"},{"key":"2024030118224361000_bib56","first-page":"6833","article-title":"Style transfer from non-parallel text by cross-alignment","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Shen","year":"2017"},{"issue":"86","key":"2024030118224361000_bib57","first-page":"2579","article-title":"Visualizing data using T-SNE","volume":"9","author":"van der Maaten","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"2024030118224361000_bib58","doi-asserted-by":"publisher","first-page":"2794","DOI":"10.1109\/ICCV.2015.320","article-title":"Unsupervised learning of visual representations using videos","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Wang","year":"2015"},{"key":"2024030118224361000_bib59","article-title":"Delving into inter-image invariance for unsupervised visual representations","author":"Xie","year":"2020","journal-title":"arXiv preprint arXiv:2008.11702"},{"key":"2024030118224361000_bib60","doi-asserted-by":"publisher","first-page":"6002","DOI":"10.1109\/ICCV.2019.00610","article-title":"Local aggregation for unsupervised learning of visual embeddings","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Zhuang","year":"2019"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00640\/2342331\/tacl_a_00640.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00640\/2342331\/tacl_a_00640.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T18:23:18Z","timestamp":1709317398000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00640\/119813\/Text-Attribute-Control-via-Closed-Loop"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":60,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00640","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}