{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T14:27:10Z","timestamp":1785421630855,"version":"3.56.0"},"reference-count":25,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T00:00:00Z","timestamp":1782345600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002338","name":"Ministry of Education of China","doi-asserted-by":"crossref","award":["JYB2025XDXM402"],"award-info":[{"award-number":["JYB2025XDXM402"]}],"id":[{"id":"10.13039\/501100002338","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["32327801"],"award-info":[{"award-number":["32327801"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"award":["32327801"],"award-info":[{"award-number":["32327801"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Diffusion-based generative models achieve high-fidelity image synthesis; however, controlling internal representations for abstract visual concepts remains challenging due to the ambiguity of textual descriptions. In this work, we propose a prompt-guided concept-vector learning framework for the controllable manipulation of such concepts without requiring external human-annotated image pairs, segmentation masks, identity labels, or manually annotated editing targets. The method introduces a learnable concept vector optimized in the bottleneck (mid-block) feature space of a pretrained Stable Diffusion U-Net, while keeping all pretrained model parameters frozen. A multi-prompt data generation strategy based on paired positive and neutral prompts provides weak semantic guidance for capturing the target concept direction and reducing dependence on a single prompt formulation. The learned vector is further applied in an image-to-image setting through controlled noise injection and concept-guided denoising, enabling the semantic modification of real images while preserving structural content. The concept strength is controlled by a scaling parameter \u03b3, while the image-to-image noise strength is controlled by \u03b2, allowing for a practical balance between semantic modification and structural fidelity. Experiments are conducted on two main abstract concepts, perfect skin and peaceful lake, with additional qualitative analysis on subjective portrait-level concepts. Quantitative evaluation using SSIM, LPIPS, and CLIP similarity demonstrates that the proposed method improves semantic alignment while maintaining structural preservation compared with Stable Diffusion image-to-image baselines. A human preference study further shows that concept-injected outputs are preferred in 76.0% of responses for perfect skin and 85.7% for peaceful lake. Ablation studies further demonstrate the controllability and robustness of the proposed framework. Overall, the method provides a simple and parameter-efficient approach for interpretable concept-level manipulation in diffusion models.<\/jats:p>","DOI":"10.3390\/jimaging12070279","type":"journal-article","created":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T12:37:55Z","timestamp":1782391075000},"page":"279","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Prompt-Guided Semantic Latent Direction Learning in Diffusion Models for Abstract Visual Concept Manipulation"],"prefix":"10.3390","volume":"12","author":[{"given":"Mahzaib","family":"Khalid","sequence":"first","affiliation":[{"name":"Department of Computer Science, East China University of Science and Technology, Shanghai 200237, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8390-3229","authenticated-orcid":false,"given":"Fangli","family":"Ying","sequence":"additional","affiliation":[{"name":"Department of Computer Science, East China University of Science and Technology, Shanghai 200237, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Al-Garadi Ahmed Mohammed","family":"Atef","sequence":"additional","affiliation":[{"name":"Department of Computer Science, East China University of Science and Technology, Shanghai 200237, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2289-3116","authenticated-orcid":false,"given":"Aniwat","family":"Phaphuangwittayakul","sequence":"additional","affiliation":[{"name":"International College of Digital Innovation, Chiang Mai University, Chiang Mai 50200, Thailand"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4779-0268","authenticated-orcid":false,"given":"Riyad","family":"Dhuny","sequence":"additional","affiliation":[{"name":"Department of Creative Arts, Film and Media Technologies, University of Technology, Mauritius, Port Louis 11134, Mauritius"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,25]]},"reference":[{"key":"ref_1","first-page":"8780","article-title":"Diffusion Models Beat GANs on Image Synthesis","volume":"34","author":"Dhariwal","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst. (NeurIPS)"},{"key":"ref_2","first-page":"6840","article-title":"Denoising Diffusion Probabilistic Models","volume":"33","author":"Ho","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst. (NeurIPS)"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. (2022, January 18\u201324). High-Resolution Image Synthesis with Latent Diffusion Models. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"ref_4","unstructured":"Ho, J., and Salimans, T. (2022). Classifier-Free Diffusion Guidance. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"4771","DOI":"10.1109\/TPAMI.2025.3646548","article-title":"Controllable Generation with Text-to-Image Diffusion Models: A Survey","volume":"48","author":"Cao","year":"2026","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_6","unstructured":"Meng, C., He, Y., Song, Y., Song, J., Wu, J., Zhu, J.-Y., and Ermon, S. (2022, January 25\u201329). SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations. Proceedings of the Tenth International Conference on Learning Representations, Virtual."},{"key":"ref_7","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning Transferable Visual Models from Natural Language Supervision. Proceedings of the 38th International Conference on Machine Learning, Virtual."},{"key":"ref_8","unstructured":"Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. (2022). Hierarchical Text-Conditional Image Generation with CLIP Latents. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Haas, R., Huberman-Spiegelglas, I., Mulayoff, R., Gra\u00dfhof, S., Brandt, S.S., and Michaeli, T. (2024, January 27\u201331). Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models. Proceedings of the 2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG), Istanbul, Turkiye.","DOI":"10.1109\/FG59268.2024.10581912"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"24129","DOI":"10.52202\/075280-1048","article-title":"Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry","volume":"36","author":"Park","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst. (NeurIPS)"},{"key":"ref_11","unstructured":"Kwon, M., Jeong, J., and Uh, Y. (2022). Diffusion Models Already Have a Semantic Latent Space. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Brooks, T., Holynski, A., and Efros, A.A. (2023, January 18\u201322). InstructPix2Pix: Learning to Follow Image Editing Instructions. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01764"},{"key":"ref_13","unstructured":"Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., Pritch, Y., and Cohen-Or, D. (2022). Prompt-to-Prompt Image Editing with Cross-Attention Control. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, L., Rao, A., and Agrawala, M. (2023, January 1\u20136). Adding Conditional Control to Text-to-Image Diffusion Models. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.00355"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mou, C., Wang, X., Xie, L., Wu, Y., Zhang, J., Qi, Z., and Shan, Y. (2024, January 20\u201327). T2I-Adapter: Learning Adapters for Controllable Text-to-Image Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i5.28226"},{"key":"ref_16","unstructured":"Traub, J. (2022). Representation Learning with Diffusion Models. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Preechakul, K., Chatthee, N., Wizadwongsa, S., and Suwajanakorn, S. (2022, January 18\u201324). Diffusion Autoencoders: Toward a Meaningful and Decodable Representation. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01036"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Samuel, D., Ben-Ari, R., Raviv, S., Darshan, N., and Chechik, G. (2024, January 20\u201327). Generating Images of Rare Concepts Using Pre-trained Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i5.28270"},{"key":"ref_19","unstructured":"Zeng, E.Z., Chen, Y., and Wong, A. (2024). Decoding Diffusion: A Scalable Framework for Unsupervised Analysis of Latent Space Representations. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Li, H., Shen, C., Torr, P., Tresp, V., and Gu, J. (2024, January 16\u201322). Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR52733.2024.01141"},{"key":"ref_21","unstructured":"Shuai, Z., Wu, C., Tang, Z., Song, B., and Shen, L. (2024). Latent Space Disentanglement in Diffusion Transformers Enables Zero-shot Semantic Editing. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yang, Z., Yu, H., Li, B., Zhang, J., Huang, J., and Zhao, F. (2024). Unleashing the Potential of the Semantic Latent Space in Diffusion Models for Image Dehazing. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-031-72784-9_21"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image Quality Assessment: From Error Visibility to Structural Similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18\u201322). The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"ref_25","unstructured":"Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A.H., Chechik, G., and Cohen-Or, D. (2023, January 1\u20135). An Image Is Worth One Word: Personalizing Text-to-Image Generation Using Textual Inversion. Proceedings of the Eleventh International Conference on Learning Representations, Kigali, Rwanda."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/12\/7\/279\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T05:23:09Z","timestamp":1782537789000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/12\/7\/279"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,25]]},"references-count":25,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["jimaging12070279"],"URL":"https:\/\/doi.org\/10.3390\/jimaging12070279","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,25]]}}}