{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:01:58Z","timestamp":1760148118634,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T00:00:00Z","timestamp":1680134400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Educational Fund Management Institution\u2014Ministry of Finance, Indonesia"},{"name":"Educational Fund Management Institution\u2014Ministry of Finance and Bandung Institute of Technology"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>The paraphrase generator for citation sentences is used to produce several sentence alternatives to avoid plagiarism. Furthermore, the generation results need to pay attention to semantic similarity and lexical divergence standards. This study proposed the StoPGEN model as an algorithm for generating citation paraphrase sentences with stochastic output. The generation process is guided by an objective function using a simulated annealing algorithm to maintain the properties of semantic similarity and lexical divergence. The objective function is created by combining the two factors that maintain these properties. This study combined METEOR and PINC Scores in a linear weighting function that can be adjusted for its value tendency in one of the matrix functions. The dataset of citation sentences that had been labeled with paraphrases was used to test StoPGEN and other models for comparison. The StoPGEN model, with the citation sentences dataset, produced a BLEU score of 55.37, outperforming the bidirectional LSTM method with a value of 28.93. StoPGEN was also tested using Quora data by changing the language source in the architecture section resulting in a BLEU score of 22.37, outperforming UPSA 18.21. In addition, the qualitative evaluation results of the citation sentence generation based on respondents obtained an acceptance value of 50.80.<\/jats:p>","DOI":"10.3390\/informatics10020034","type":"journal-article","created":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T02:08:01Z","timestamp":1680228481000},"page":"34","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Generating Paraphrase Using Simulated Annealing for Citation Sentences"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5359-260X","authenticated-orcid":false,"given":"Ridwan","family":"Ilyas","sequence":"first","affiliation":[{"name":"School of Electrical Engineering and Informatics, Bandung Institute of Technology, Bandung 40312, Indonesia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8765-8109","authenticated-orcid":false,"given":"Masayu","family":"Khodra","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Informatics, Bandung Institute of Technology, Bandung 40312, Indonesia"}]},{"given":"Rinaldi","family":"Munir","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Informatics, Bandung Institute of Technology, Bandung 40312, Indonesia"}]},{"given":"Rila","family":"Mandala","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Informatics, Bandung Institute of Technology, Bandung 40312, Indonesia"}]},{"given":"Dwi","family":"Widyantoro","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Informatics, Bandung Institute of Technology, Bandung 40312, Indonesia"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1613\/jair.2985","article-title":"A Survey of Paraphrasing and Textual Entailment Methods","volume":"38","author":"Androutsopoulos","year":"2010","journal-title":"J. Artif. Intell. Res."},{"key":"ref_2","unstructured":"Quirk, C., Brockett, C., and Dolan, W. (2003). Monolingual Machine Translation for Paraphrase Generation, Association for Computational Linguistics."},{"key":"ref_3","unstructured":"Lisa, C.M.L. (2009). Merging Corpus Linguistics and Collaborative Knowledge. [Ph.D. Thesis, University of Birmingham]."},{"key":"ref_4","first-page":"1","article-title":"Plagiarism Meets Paraphrasing: Insights for the Next Generation in Automatic Plagiarism Detection","volume":"10","author":"Vila","year":"2013","journal-title":"Assoc. Comput. Linguist."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/S1532-0464(03)00016-9","article-title":"Paraphrasing for Condensation in Journal Abstracting","volume":"35","author":"Kittredge","year":"2002","journal-title":"J. Biomed. Inform."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Ilyas, R., Widiyantoro, D.H., and Khodra, M.L. (2018, January 15\u201317). Building Candidate Monolingual Parallel Corpus from Scientific Papers. Proceedings of the 2018 International Conference on Asian Language Processing, IALP, Bandung, Indonesia.","DOI":"10.1109\/IALP.2018.8629246"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Teufel, S., Siddharthan, A., and Tidhar, D. (2006, January 15\u201316). An Annotation Scheme for Citation Function. Proceedings of the COLING\/ACL 2006\u2013SIGdial06: 7th SIGdial Workshop on Discourse and Dialogue, Sydney, Australia.","DOI":"10.3115\/1654595.1654612"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Liu, X., Mou, L., Meng, F., Zhou, H., Zhou, J., and Song, S. (2019). Unsupervised Paraphrasing by Simulated Annealing. arXiv, 302\u2013312.","DOI":"10.18653\/v1\/2020.acl-main.28"},{"key":"ref_9","unstructured":"Banerjee, S., and Lavie, A. (2005). Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization, Association for Computational Linguistics."},{"key":"ref_10","unstructured":"Chen, D.L., and Dolan, W.B. (2011, January 19\u201324). Collecting Highly Parallel Data for Paraphrase Evaluation. Proceedings of the ACL-HLT 2011, 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, OR, USA."},{"key":"ref_11","unstructured":"Carbonell, J., and Goldstein, J. (1998). SIGIR Forum (ACM Special Interest Group on Information Retrieval), ACM."},{"key":"ref_12","unstructured":"Mikolov, T., Chen, K., Corrado, G., and Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv."},{"key":"ref_13","unstructured":"NADAS, A. (1984). IEEE Transactions on Acoustics Speech and Signal Processing, IEEE."},{"key":"ref_14","first-page":"1","article-title":"Paraphrasing Questions Using Given and New Information","volume":"9","author":"McKeown","year":"1983","journal-title":"Am. J. Comput. Linguist."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1017\/S1351324901002765","article-title":"Discovery of Inference Rules for Question-Answering","volume":"7","author":"Lin","year":"2001","journal-title":"Nat. Lang. Eng."},{"key":"ref_16","unstructured":"Kauchak, D., and Barzilay, R. (2006). Proceedings of the Main Conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, Association for Computational Linguistics."},{"key":"ref_17","unstructured":"Prakash, A., Hasan, S.A., Lee, K., Datla, V., Qadir, A., Liu, J., and Farri, O. (2016). Neural Paraphrase Generation with Stacked Residual LSTM Networks. arXiv."},{"key":"ref_18","unstructured":"Vizcarra, G., and Ochoa-Luna, J. (2020). Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020), Association for Computational Linguistics."},{"key":"ref_19","first-page":"5999","article-title":"Attention Is All You Need","volume":"2017","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","first-page":"7176","article-title":"A Task in a Suit and a Tie: Paraphrase Generation with Semantic Augmentation","volume":"33","author":"Wang","year":"2019","journal-title":"Proc. Conf. AAAI Artif. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Luong, M.T., Pham, H., and Manning, C.D. (2015, January 17\u201321). Effective Approaches to Attention-Based Neural Machine Translation. Proceedings of the EMNLP 2015: Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal.","DOI":"10.18653\/v1\/D15-1166"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ma, X., and Hovy, E. (2016). End-to-End Sequence Labeling via Bi-Directional LSTM-CNNs-CRF, Association for Computational Linguistics.","DOI":"10.18653\/v1\/P16-1101"},{"key":"ref_23","first-page":"6834","article-title":"CGMH: Constrained Sentence Generation by Metropolis-Hastings Sampling","volume":"33","author":"Miao","year":"2019","journal-title":"AAAI Conf. Artif. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lan, W., Qiu, S., He, H., and Xu, W. (2017, January 7\u201311). A Continuously Growing Dataset of Sentential Paraphrases. Proceedings of the EMNLP 2017\u2014Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1126"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bowman, S.R., Vilnis, L., Vinyals, O., Dai, A.M., Jozefowicz, R., and Bengio, S. (2016, January 11\u201312). Generating Sentences from a Continuous Space. Proceedings of the CoNLL 2016\u201420th SIGNLL Conference on Computational Natural Language Learning, Berlin, Germany.","DOI":"10.18653\/v1\/K16-1002"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Bhagat, R., and Hovy, E. (2013). What Is Paraphrase, Association for Computational Linguistics.","DOI":"10.1162\/COLI_a_00166"},{"key":"ref_27","unstructured":"Ganitkevitch, J., Van Durme, B., and Callison-Burch, C. (2013, January 9\u201314). PPDB: The Paraphrase Database. Proceedings of the NAACL-HLT\u2013Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, USA."},{"key":"ref_28","unstructured":"Dolan, W.B., and Brockett, C. (2005, January 14). Automatically Constructing a Corpus of Sentential Paraphrases. Proceedings of the Third International Workshop on Paraphrasing, Yamamoto, Japan."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1162\/tacl_a_00194","article-title":"Extracting Lexically Divergent Paraphrases from Twitter","volume":"2","author":"Xu","year":"2014","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Pavlick, E., Rastogi, P., Ganitkevitch, J., Durme, B.V., and Callison-Burch, C. (2015, January 26\u201331). PPDB 2.0: Better Paraphrase Ranking, Fine-Grained Entailment Relations, Word Embeddings, and Style Classification. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers), Beijing, China.","DOI":"10.3115\/v1\/P15-2070"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.jslw.2012.03.003","article-title":"Rewriting and Paraphrasing Source Texts in Second Language Writing","volume":"21","author":"Shi","year":"2012","journal-title":"J. Second Lang. Writ."},{"key":"ref_32","unstructured":"Teufel, S. (2017, January 7\u201311). Do \u201cFuture Work\u201d Sections Have a Purpose? Citation Links and Entailment for Global Scientometric Questions. Proceedings of the 2nd Joint Workshop on Bibliometric-enhanced Information Retrieval and Natural Language Processing for Digital Libraries, Tokyo, Japan."},{"key":"ref_33","unstructured":"Takahashi, K., Ishibashi, Y., Sudoh, K., and Nakamura, S. (2021, January 10\u201311). Multilingual Machine Translation Evaluation Metrics Fine-Tuned on Pseudo-Negative Examples for WMT 2021 Metrics Task. Proceedings of the WMT 2021\u20136th Conference on Machine Translation, Online."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W. (2002, January 6\u201312). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadephia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Doddington, G. (2002). Automatic Evaluation of Machine Translation Quality Using N-Gram Co-Occurrence Statistics, Morgan Kaufmann Publishers Inc.","DOI":"10.3115\/1289189.1289273"},{"key":"ref_36","unstructured":"Madnani, N., Tetreault, J., and Chodorow, M. (2012, January 10\u201315). Re-Examining Machine Translation Metrics for Paraphrase Identification. Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Seattle, WA, USA."},{"key":"ref_37","unstructured":"Ji, Y., and Eisenstein, J. (2013, January 18\u201321). Discriminative Improvements to Distributional Sentence Similarity. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP 2013), Seattle, WA, USA."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Brad, F. (2017). Neural Paraphrase Generation Using Transfer Learning, Association for Computational Linguistics.","DOI":"10.18653\/v1\/W17-3542"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Aziz, A.A., Djamal, E.C., and Ilyas, R. (2019, January 20\u201322). Siamese Similarity Between Two Sentences Using Manhattan\u2019s Recurrent Neural Networks. Proceedings of the 2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA), Yogyakarta, Indonesia.","DOI":"10.1109\/ICAICTA.2019.8904412"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Saputro, W.F., Djamal, E.C., and Ilyas, R. (2019, January 8\u201310). Paraphrase Identification Between Two Sentence Using Support Vector Machine. Proceedings of the 2019 International Conference on Electrical Engineering and Informatics (ICEEI), Nanjing, China.","DOI":"10.1109\/ICEEI47359.2019.8988874"},{"key":"ref_41","unstructured":"Wubben, S., Bosch, A.V.D., and Krahmer, E. (2010, January 7\u20139). Paraphrase Generation as Monolingual Translation: Data and Evaluation. Proceedings of the 6th International Natural Language Generation Conference, Meath, Ireland."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Mallinson, J., Sennrich, R., and Lapata, M. (2017, January 3\u20137). Paraphrasing Revisited with Neural Machine Translation. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain.","DOI":"10.18653\/v1\/E17-1083"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhao, S., Lan, X., Liu, T., and Li, S. (2009, January 2\u20137). Application-Driven Statistical Paraphrase Generation. Proceedings of the 47th Annual Meeting of the ACL and the 4th IJCNLP of the AFNLP, Suntec, Singapore.","DOI":"10.3115\/1690219.1690263"},{"key":"ref_44","first-page":"3104","article-title":"Sequence to Sequence Learning with Neural Networks","volume":"27","author":"Sutskever","year":"2014","journal-title":"Nips"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Parikh, A.P., T\u00e4ckstr\u00f6m, O., Das, D., and Uszkoreit, J. (2016). A Decomposable Attention Model for Natural Language Inference. arXiv.","DOI":"10.18653\/v1\/D16-1244"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1126\/science.220.4598.671","article-title":"Optimization by Simulated Annealing","volume":"220","author":"Kirkpatrick","year":"1983","journal-title":"Science"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"652","DOI":"10.1109\/34.295910","article-title":"Simulated Annealing: A Proof of Convergence","volume":"16","author":"Granville","year":"1994","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_48","first-page":"1","article-title":"Distributed Representations Ofwords and Phrases and Their Compositionality","volume":"26","author":"Mikolov","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_49","unstructured":"Donoghue, D.P., Saggion, H., Dong, F., Hurley, D., Abgaz, Y., Zheng, X., Corcho, O., Careil, J.-M., Mahdian, B., and Zhao, X. (2014). Towards Dr Inventor: A Tool for Promoting Scientific Creativity, Jozef Stefan Institute."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"100","DOI":"10.2307\/2346830","article-title":"Algorithm AS 136: A K-Means Clustering Algorithm","volume":"28","author":"Hartigan","year":"1979","journal-title":"Applied Statistics"},{"key":"ref_51","unstructured":"He, J., Spokoyny, D., Neubig, G., and Berg-Kirkpatrick, T. (2019, January 6\u20139). Lagging Inference Networks and Posterior Collapse in Variational Autoencoders. Proceedings of the 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA."},{"key":"ref_52","unstructured":"Lin, C.-Y. (2004). Proceedings of the Text Summarization Branches Out, Association for Computational Linguistics."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/10\/2\/34\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:07:11Z","timestamp":1760123231000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/10\/2\/34"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,30]]},"references-count":52,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,6]]}},"alternative-id":["informatics10020034"],"URL":"https:\/\/doi.org\/10.3390\/informatics10020034","relation":{},"ISSN":["2227-9709"],"issn-type":[{"type":"electronic","value":"2227-9709"}],"subject":[],"published":{"date-parts":[[2023,3,30]]}}}