{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T14:13:44Z","timestamp":1760710424381,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2020,11,30]],"date-time":"2020-11-30T00:00:00Z","timestamp":1606694400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Hashtag-based image descriptions are a popular approach for labeling images on social media platforms. In practice, images are often described by more than one hashtag. Due the rapid development of deep neural networks specialized in image embedding and classification, it is now possible to generate those descriptions automatically. In this paper we propose a novel Voting Deep Neural Network with Associative Rules Mining (VDNN-ARM) algorithm that can be used to solve multi-label hashtag recommendation problems. VDNN-ARM is a machine learning approach that utilizes an ensemble of deep neural networks to generate image features, which are then classified to potential hashtag sets. Proposed hashtags are then filtered by a voting schema. The remaining hashtags might be included in a final recommended hashtags dataset by application of associative rules mining, which explores dependencies in certain hashtag groups. Our approach is evaluated on a HARRISON benchmark dataset as a multi-label classification problem. The highest values of our evaluation parameters, including precision, recall, and accuracy, have been obtained for VDNN-ARM with a confidence threshold 0.95. VDNN-ARM outperforms state-of-the-art algorithms, including VGG-Object + VGG-Scene precision by 17.91% as well as ensemble\u2013FFNN (intersection) recall by 32.33% and accuracy by 27.00%. Both the dataset and all source codes we implemented for this research are available for download, and our results can be reproduced.<\/jats:p>","DOI":"10.3390\/e22121351","type":"journal-article","created":{"date-parts":[[2020,11,30]],"date-time":"2020-11-30T10:26:12Z","timestamp":1606731972000},"page":"1351","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Image Hashtag Recommendations Using a Voting Deep Neural Network and Associative Rules Mining Approach"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1390-9021","authenticated-orcid":false,"given":"Tomasz","family":"Hachaj","sequence":"first","affiliation":[{"name":"Institute of Computer Science, Pedagogical University of Krakow, 2 Podchorazych Ave, 30-084 Krakow, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1094-2854","authenticated-orcid":false,"given":"Justyna","family":"Miazga","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, Pedagogical University of Krakow, 2 Podchorazych Ave, 30-084 Krakow, Poland"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,30]]},"reference":[{"key":"ref_1","unstructured":"Park, M., Li, H., and Kim, J. (2016). HARRISON: A Benchmark on HAshtag Recommendation for Real-world Images in Social Networks. arXiv."},{"key":"ref_2","unstructured":"Zangerle, E., Gassler, W., and Specht, G. (2011). Recommending#-tags in twitter. Proceedings of the Workshop on Semantic Adaptive Social Web (SASWeb 2011), RWTH. CEUR Workshop Proceedings."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.neucom.2017.06.056","article-title":"Hashtag recommendation for multimodal microblog posts","volume":"272","author":"Gong","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"10569","DOI":"10.1109\/ACCESS.2020.2964682","article-title":"Exploring Pattern Mining Algorithms for Hashtag Retrieval Problem","volume":"8","author":"Belhadi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Miazga, J., and Hachaj, T. (2019, January 8). Evaluation of Most Popular Sentiment Lexicons Coverage on Various Datasets. Proceedings of the 2019 2nd International Conference on Sensors, Signal and Image Processing, Prague, Czech Republic.","DOI":"10.1145\/3365245.3365251"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, Q., Shah, S., Nourbakhsh, A., Liu, X., and Fang, R. (2016, January 24). Hashtag Recommendation Based on Topic Enhanced Embedding, Tweet Entity Data and Learning to Rank. Proceedings of the 25th ACM International Conference on Information and Knowledge Management (CIKM 2016), Indianapolis, IN, USA.","DOI":"10.1145\/2983323.2983915"},{"key":"ref_7","unstructured":"Liu, Z., Chen, X., and Sun, M. (2011, January 27\u201331). A Simple Word Trigger Method for Social Tag Suggestion. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, Edinburgh, UK."},{"key":"ref_8","unstructured":"Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., and Bengio, Y. (2015, January 6\u201311). Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. Proceedings of the 32nd International Conference on Machine Learning, Lille, France."},{"key":"ref_9","unstructured":"Joci\u0107, M., \u0110or\u0111e, O., Malbasa, V., and Konjovic, Z. (2017, January 12\u201315). Image tagging with an ensemble of deep convolutional neural networks. Proceedings of the 2017 International Conference on Information Society and Technology, Budapest, Hungary."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Gaur, S. (2019, January 8\u201312). Generation of a Short Narrative Caption for an Image Using the Suggested Hashtag. Proceedings of the 2019 IEEE 35th International Conference on Data Engineering Workshops (ICDEW), Macao, China.","DOI":"10.1109\/ICDEW.2019.00060"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"4001","DOI":"10.1007\/s00521-017-3322-z","article-title":"Character-level Recurrent Neural Networks in Practice: Comparing Training and Sampling Schemes","volume":"31","author":"Dhoedt","year":"2019","journal-title":"Neural Comput. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hwang, K., and Sung, W. (2017, January 5\u20139). Character-level language modeling with hierarchical recurrent neural networks. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.","DOI":"10.1109\/ICASSP.2017.7953252"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Nguyen, H., Wistuba, M., and Schmidt-Thieme, L. (2017). Personalized Tag Recommendation for Images Using Deep Transfer Learning. ECML PKDD 2017: Machine Learning and Knowledge Discovery in Databases, Springer.","DOI":"10.1007\/978-3-319-71246-8_43"},{"key":"ref_14","first-page":"5805","article-title":"Hashtag Recommendation for Photo Sharing Services","volume":"33","author":"Zhang","year":"2019","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1007\/s11390-018-1851-2","article-title":"Hashtag Recommendation Based on Multi-Features of Microblogs","volume":"33","author":"Kou","year":"2018","journal-title":"J. Comput. Sci. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Denton, E., Weston, J., Paluri, M., Bourdev, L., and Fergus, R. (2015, January 10). User Conditional Hashtag Prediction for Images. Proceedings of the 21th ACM SIGKDD International Conference on kKnowledge Discovery and Data Mining, Sydney, NSW, Australia.","DOI":"10.1145\/2783258.2788576"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kao, D., Lai, K.T., and Chen, M.S. (2019). An Efficient and Resource-Aware Hashtag Recommendation Using Deep Neural Networks. Pacific-Asia Conference on Knowledge Discovery and Data Mining, PAKDD 2019: Advances in Knowledge Discovery and Data Mining, Springer.","DOI":"10.1007\/978-3-030-16145-3_12"},{"key":"ref_18","unstructured":"Gaosheng, W., Li, Y., Yan, W., Li, R., Gu, X., and Yang, Q. (2018, January 13\u201316). Hashtag Recommendation with Attention-Based Neural Image Hashtagging Network. Proceedings of the 25th International Conference, ICONIP 2018, Siem Reap, Cambodia."},{"key":"ref_19","unstructured":"Nguyen, T. (2018). Recommending Tags for Images: Deep Learning Approaches for Personalized Tag Recommendation. [Ph.D. Thesis, University of Hildesheim]."},{"key":"ref_20","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. (2017). Places: A 10 Million Image Database for Scene Recognition. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1167\/17.10.296"},{"key":"ref_23","unstructured":"Kalliatakis, G. (2020, November 29). Keras-VGG16-Places365. Available online: https:\/\/github.com\/GKalliatakis\/Keras-VGG16-places365."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Panboonyuen, T., Jitkajornwanich, K., Lawawirojwong, S., Srestasathiern, P., and Vateekul, P. (2019). Semantic Segmentation on Remotely Sensed Images Using an Enhanced Global Convolutional Network with Channel Attention and Domain Specific Transfer Learning. Remote Sens., 11.","DOI":"10.20944\/preprints201812.0090.v3"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lin, D., Wang, Y., Xu, G., Li, J., and Fu, K. (2018). Transform a Simple Sketch to a Chinese Painting by a Multiscale Deep Neural Network. Algorithms, 11.","DOI":"10.3390\/a11010004"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2016). Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. arXiv.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2017, January 21\u201326). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Zoph, B., Vasudevan, V., Shlens, J., and Le, Q.V. (2018, January 18\u201323). Learning Transferable Architectures for Scalable Image Recognition. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00907"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. (2018, January 18\u201323). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Agrawal, R., Imielinski, T., and Swami, A. (1993, January 26\u201328). Mining Associations Between Sets of Items in Massive Databases. Proceedings of the ACM-SIGMOD 1993 International Conference on Management of Data, Washington, DC, USA.","DOI":"10.1145\/170035.170072"},{"key":"ref_33","unstructured":"Agrawal, R., and Srikant, R. (1994). Fast Algorithms for Mining Association Rules in Large Databases. Proceedings of the 20th International Conference on Very Large Data Bases, Morgan Kaufmann Publishers Inc."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"638","DOI":"10.21105\/joss.00638","article-title":"MLxtend: Providing machine learning and data science utilities and extensions to Python\u2019s scientific computing stack","volume":"3","author":"Raschka","year":"2018","journal-title":"J. Open Source Softw."},{"key":"ref_35","unstructured":"Kingma, D.P., and Ba, J. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_36","first-page":"361","article-title":"Gephi: An Open Source Software for Exploring and Manipulating Networks","volume":"8","author":"Bastian","year":"2009","journal-title":"Int. AAAI Conf. Weblogs Soc. Media"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Jacomy, M., Venturini, T., Heymann, S., and Bastian, M. (2014). ForceAtlas2, a Continuous Graph Layout Algorithm for Handy Network Visualization Designed for the Gephi Software. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0098679"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Giannoulakis, S., and Tsapatsoulis, N. (2016). Defining and Identifying Stophashtags in Instagram. INNS 2016: Advances in Big Data, Springer.","DOI":"10.1007\/978-3-319-47898-2_31"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/12\/1351\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:39:29Z","timestamp":1760179169000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/12\/1351"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,30]]},"references-count":38,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["e22121351"],"URL":"https:\/\/doi.org\/10.3390\/e22121351","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,11,30]]}}}