{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T11:00:01Z","timestamp":1788865201107,"version":"build-2803163510"},"reference-count":31,"publisher":"American Association for the Advancement of Science (AAAS)","license":[{"start":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T00:00:00Z","timestamp":1649289600000},"content-version":"vor","delay-in-days":96,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation","award":["1704337"],"award-info":[{"award-number":["1704337"]}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"crossref","award":["R01AR054385"],"award-info":[{"award-number":["R01AR054385"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BME Front"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:p>\n            <jats:italic>Objective and Impact Statement<\/jats:italic>\n            . We adopt a deep learning model for bone osteolysis prediction on computed tomography (CT) images of murine breast cancer bone metastases. Given the bone CT scans at previous time steps, the model incorporates the bone-cancer interactions learned from the sequential images and generates future CT images. Its ability of predicting the development of bone lesions in cancer-invading bones can assist in assessing the risk of impending fractures and choosing proper treatments in breast cancer bone metastasis.\n            <jats:italic>Introduction<\/jats:italic>\n            . Breast cancer often metastasizes to bone, causes osteolytic lesions, and results in skeletal-related events (SREs) including severe pain and even fatal fractures. Although current imaging techniques can detect macroscopic bone lesions, predicting the occurrence and progression of bone lesions remains a challenge.\n            <jats:italic>Methods<\/jats:italic>\n            . We adopt a temporal variational autoencoder (T-VAE) model that utilizes a combination of variational autoencoders and long short-term memory networks to predict bone lesion emergence on our micro-CT dataset containing sequential images of murine tibiae. Given the CT scans of murine tibiae at early weeks, our model can learn the distribution of their future states from data.\n            <jats:italic>Results<\/jats:italic>\n            . We test our model against other deep learning-based prediction models on the bone lesion progression prediction task. Our model produces much more accurate predictions than existing models under various evaluation metrics.\n            <jats:italic>Conclusion<\/jats:italic>\n            . We develop a deep learning framework that can accurately predict and visualize the progression of osteolytic bone lesions. It will assist in planning and evaluating treatment strategies to prevent SREs in breast cancer patients.\n          <\/jats:p>","DOI":"10.34133\/2022\/9763284","type":"journal-article","created":{"date-parts":[[2022,4,7]],"date-time":"2022-04-07T23:45:06Z","timestamp":1649375106000},"source":"Crossref","is-referenced-by-count":9,"title":["Breast Cancer Induced Bone Osteolysis Prediction Using Temporal Variational Autoencoders"],"prefix":"10.34133","volume":"2022","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0669-5029","authenticated-orcid":false,"given":"Wei","family":"Xiong","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Rochester, Rochester, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9123-1960","authenticated-orcid":false,"given":"Neil","family":"Yeung","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Rochester, Rochester, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shubo","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of Delaware, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haofu","family":"Liao","sequence":"additional","affiliation":[{"name":"Amazon Web ServicesUSA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liyun","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of Delaware, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiebo","family":"Luo","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Rochester, Rochester, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"221","published-online":{"date-parts":[[2022,4,7]]},"reference":[{"issue":"1","key":"e_1_3_4_1_2","first-page":"7","article-title":"Cancer statistics, 2020","volume":"70","author":"Siegel R. L.","year":"2020","unstructured":"R. L.Siegel, K. D.Miller, and A.Jemal, \u201cCancer statistics, 2020,\u201d CA: a Cancer Journal for Clinicians, vol. 70, no. 1, pp. 7\u201330, 2020","journal-title":"CA: a Cancer Journal for Clinicians"},{"issue":"8","key":"e_1_3_4_2_2","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1038\/nrc867","article-title":"Metastasis to bone: causes, consequences and therapeutic opportunities","volume":"2","author":"Mundy G. R.","year":"2002","unstructured":"G. R.Mundy, \u201cMetastasis to bone: causes, consequences and therapeutic opportunities,\u201d Nature Reviews. Cancer, vol. 2, no. 8, pp. 584\u2013593, 2002","journal-title":"Nature Reviews. Cancer"},{"issue":"5","key":"e_1_3_4_3_2","doi-asserted-by":"crossref","first-page":"259","DOI":"10.3747\/co.19.1011","article-title":"Bone-targeted agents and skeletal-related events in breast cancer patients with bone metastases: the state of the art","volume":"19","author":"Clemons M.","year":"2012","unstructured":"M.Clemons, K. A.Gelmon, K. I.Pritchard, and A. H.Paterson, \u201cBone-targeted agents and skeletal-related events in breast cancer patients with bone metastases: the state of the art,\u201d Current Oncology, vol. 19, no. 5, pp. 259\u2013268, 2012","journal-title":"Current Oncology"},{"issue":"1","key":"e_1_3_4_4_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13244-020-00885-4","article-title":"Imaging diagnosis of metastatic breast cancer","volume":"11","author":"Pesapane F.","year":"2020","unstructured":"F.Pesapane, K.Downey, A.Rotili, E.Cassano, and D.-M.Koh, \u201cImaging diagnosis of metastatic breast cancer,\u201d Insights Into Imaging, vol. 11, no. 1, pp. 1\u201314, 2020","journal-title":"Insights Into Imaging"},{"issue":"6","key":"e_1_3_4_5_2","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1002\/jor.24660","article-title":"Fracture risk assessment and clinical decision making for patients with metastatic bone disease","volume":"38","author":"Damron T. A.","year":"2020","unstructured":"T. A.Damron, and K. A.Mann, \u201cFracture risk assessment and clinical decision making for patients with metastatic bone disease,\u201d Journal of Orthopaedic Research\u00ae, vol. 38, no. 6, pp. 1175\u20131190, 2020","journal-title":"Journal of Orthopaedic Research\u00ae"},{"issue":"11","key":"e_1_3_4_6_2","doi-asserted-by":"crossref","first-page":"2514","DOI":"10.1158\/1078-0432.CCR-14-2668","article-title":"Treatment planning and fracture prediction in patients with skeletal metastasis with CT-based rigidity analysis","volume":"21","author":"Nazarian A.","year":"2015","unstructured":"A.Nazarian, V.Entezari, D.Zurakowski, N.Calderon, J. A.Hipp, J. C.Villa-Camacho, P. P.Lin, F. H.Cheung, A. J.Aboulafia, R.Turcotte, M. E.Anderson, M. C.Gebhardt, E. Y.Cheng, R. M.Terek, M.Yaszemski, T. A.Damron, and B. D.Snyder, \u201cTreatment planning and fracture prediction in patients with skeletal metastasis with CT-based rigidity analysis,\u201d Clinical Cancer Research, vol. 21, no. 11, pp. 2514\u20132519, 2015","journal-title":"Clinical Cancer Research"},{"issue":"8","key":"e_1_3_4_7_2","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1302\/0301-620X.94B8.28449","article-title":"The assessment of the risk of fracture in femora with metastatic lesions: comparing case-specific finite element analyses with predictions by clinical experts","volume":"94","author":"Derikx L. C.","year":"2012","unstructured":"L. C.Derikx, J. B.van Aken, D.Janssen, A.Snyers, Y. M.van der Linden, N.Verdonschot, and E.Tanck, \u201cThe assessment of the risk of fracture in femora with metastatic lesions: comparing case-specific finite element analyses with predictions by clinical experts,\u201d The Journal of Bone and Joint Surgery. British volume, vol. 94-B, no. 8, pp. 1135\u20131142, 2012","journal-title":"The Journal of Bone and Joint Surgery. British volume"},{"key":"e_1_3_4_8_2","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.bone.2018.02.011","article-title":"Pathological fracture risk assessment in patients with femoral metastases using CT-based finite element methods. A retrospective clinical study","volume":"110","author":"Sternheim A.","year":"2018","unstructured":"A.Sternheim, O.Giladi, Y.Gortzak, M.Drexler, M.Salai, N.Trabelsi, C.Milgrom, and Z.Yosibash, \u201cPathological fracture risk assessment in patients with femoral metastases using CT-based finite element methods. A retrospective clinical study,\u201d Bone, vol. 110, pp. 215\u2013220, 2018","journal-title":"Bone"},{"issue":"6","key":"e_1_3_4_9_2","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1302\/2046-3758.76.BJR-2017-0325.R2","article-title":"Can patient-specific finite element models better predict fractures in metastatic bone disease than experienced clinicians? Towards computational modelling in daily clinical practice","volume":"7","author":"Eggermont F.","year":"2018","unstructured":"F.Eggermont, L. C.Derikx, N.Verdonschot, I. C. M.van der Geest, M. A. A.de Jong, A.Snyers, Y. M.van der Linden, and E.Tanck, \u201cCan patient-specific finite element models better predict fractures in metastatic bone disease than experienced clinicians? Towards computational modelling in daily clinical practice,\u201d Bone & joint research, vol. 7, no. 6, pp. 430\u2013439, 2018","journal-title":"Bone & joint research"},{"key":"e_1_3_4_10_2","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1016\/j.bone.2018.11.008","article-title":"Prediction of early metastatic disease in experimental breast cancer bone metastasis by combining PET\/CT and MRI parameters to a model-averaged neural network","volume":"120","author":"Ellmann S.","year":"2019","unstructured":"S.Ellmann, L.Seyler, J.Evers, H.Heinen, A.Bozec, O.Prante, T.Kuwert, M.Uder, and T.B\u00e4uerle, \u201cPrediction of early metastatic disease in experimental breast cancer bone metastasis by combining PET\/CT and MRI parameters to a model-averaged neural network,\u201d Bone, vol. 120, pp. 254\u2013261, 2019","journal-title":"Bone"},{"key":"e_1_3_4_11_2","doi-asserted-by":"crossref","first-page":"804","DOI":"10.1038\/bonekey.2016.31","article-title":"Murine models of breast cancer bone metastasis","volume":"5","author":"Wright L. E.","year":"2016","unstructured":"L. E.Wright, P. D.Ottewell, N.Rucci, O.Peyruchaud, G. M.Pagnotti, A.Chiechi, J. T.Buijs, and J. A.Sterling, \u201cMurine models of breast cancer bone metastasis,\u201d BoneKEy reports, vol. 5, p. 804, 2016","journal-title":"BoneKEy reports"},{"key":"e_1_3_4_12_2","doi-asserted-by":"crossref","unstructured":"O.Ronneberger P.Fischer and T.Brox \u201cU-net: convolutional networks for biomedical image segmentation \u201d Lecture Notes in Computer Science Springer pp. 234\u2013241 2015","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_3_4_13_2","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.neunet.2020.09.004","article-title":"GP-GAN: brain tumor growth prediction using stacked 3D generative adversarial networks from longitudinal MR images","volume":"132","author":"Elazab A.","year":"2020","unstructured":"A.Elazab, C.Wang, S. J. S.Gardezi, H.Bai, Q.Hu, T.Wang, C.Chang, and B.Lei, \u201cGP-GAN: brain tumor growth prediction using stacked 3D generative adversarial networks from longitudinal MR images,\u201d Neural Networks, vol. 132, pp. 321\u2013332, 2020","journal-title":"Neural Networks"},{"key":"e_1_3_4_14_2","doi-asserted-by":"crossref","unstructured":"W.Li V.-D.Nguyen H.Liao M.Wilder K.Cheng and J.Luo \u201cPatch transformer for multitagging whole slide histopathology images \u201d Lecture Notes in Computer Science Springer pp. 532\u2013540 2019","DOI":"10.1007\/978-3-030-32239-7_59"},{"key":"e_1_3_4_15_2","doi-asserted-by":"crossref","unstructured":"W.Li Y.Lu K.Zheng H.Liao C.Lin J.Luo C. T.Cheng J.Xiao L.Lu C. F.Kuo and S.Miao \u201cStructured landmark detection via topology-adapting deep graph learning \u201d Computer Vision \u2013 ECCV 2020 Springer pp. 266\u2013283 2020","DOI":"10.1007\/978-3-030-58545-7_16"},{"key":"e_1_3_4_16_2","doi-asserted-by":"crossref","unstructured":"W.Xiong W.Luo L.Ma W.Liu and J.Luo \u201cLearning to generate time-lapse videos using multi-stage dynamic generative adversarial networks \u201d in in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Salt Lake City UT USA 2018 pp. 2364\u20132373","DOI":"10.1109\/CVPR.2018.00251"},{"key":"e_1_3_4_17_2","unstructured":"L.Zhang \u201cSpatio-temporal convolutional LSTMs for tumor growth prediction by learning 4D longitudinal patient data \u201d 2019 https:\/\/arxiv.org\/abs\/1902.08716."},{"key":"e_1_3_4_18_2","unstructured":"Y.Wang \u201cPredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning \u201d 2021 https:\/\/arxiv.org\/abs\/2103.09504."},{"key":"e_1_3_4_19_2","article-title":"Stochastic video generation with a learned prior","author":"Denton E.","year":"2018","unstructured":"E.Denton, and R.Fergus, \u201cStochastic video generation with a learned prior,\u201d CoRR, 2018, http:\/\/arxiv.org\/abs\/1802.07687.","journal-title":"CoRR"},{"key":"e_1_3_4_20_2","unstructured":"D. P.Kingma and M.Welling \u201cAuto-encoding variational bayes \u201d 2013 https:\/\/arxiv.org\/abs\/1312.6114."},{"issue":"4","key":"e_1_3_4_21_2","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1561\/2200000056","article-title":"An introduction to variational autoencoders","volume":"12","author":"Kingma D.","year":"2019","unstructured":"D.Kingma, and M.Welling, \u201cAn introduction to variational autoencoders,\u201d Foundations and Trends\u00ae in Machine Learning, vol. 12, no. 4, pp. 307\u2013392, 2019","journal-title":"Foundations and Trends\u00ae in Machine Learning"},{"key":"e_1_3_4_22_2","doi-asserted-by":"crossref","unstructured":"R.Zhang P.Isola A. A.Efros E.Shechtman and O.Wang \u201cThe unreasonable effectiveness of deep features as a perceptual metric \u201d in in Proceedings of the IEEE conference on computer vision and pattern recognition Salt Lake City UT USA 2018 pp. 586\u2013595","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_3_4_23_2","article-title":"Convolutional LSTM network: a machine learning approach for precipitation nowcasting","author":"Shi X.","year":"2015","unstructured":"X.Shi, Z.Chen, H.Wang, D.Yeung, W.Wong, and W.Woo, \u201cConvolutional LSTM network: a machine learning approach for precipitation nowcasting,\u201d CoRR, 2015, https:\/\/arxiv.org\/abs\/1506.04214.","journal-title":"CoRR"},{"key":"e_1_3_4_24_2","article-title":"Generative adversarial nets","volume":"27","author":"Goodfellow I.","year":"2014","unstructured":"I.Goodfellow, \u201cGenerative adversarial nets,\u201d Advances in Neural Information Processing Systems, vol. 27, 2014","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_4_25_2","unstructured":"A.Radford L.Metz and S.Chintala \u201cUnsupervised representation learning with deep convolutional generative adversarial networks \u201d 2016 https:\/\/arxiv.org\/abs\/1511.06434."},{"key":"e_1_3_4_26_2","doi-asserted-by":"crossref","unstructured":"W.Xiong Y.He Y.Zhang W.Luo L.Ma and J.Luo \u201cFine-grained image-to-image transformation towards visual recognition \u201d in in Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Seattle WA USA 2020 pp. 5840\u20135849","DOI":"10.1109\/CVPR42600.2020.00588"},{"key":"e_1_3_4_27_2","doi-asserted-by":"crossref","first-page":"116389","DOI":"10.1016\/j.neuroimage.2019.116389","article-title":"QSMGAN: improved quantitative susceptibility mapping using 3D generative adversarial networks with increased receptive field","volume":"207","author":"Chen Y.","year":"2020","unstructured":"Y.Chen, A.Jakary, S.Avadiappan, C. P.Hess, and J. M.Lupo, \u201cQSMGAN: improved quantitative susceptibility mapping using 3D generative adversarial networks with increased receptive field,\u201d NeuroImage, vol. 207, p. 116389, 2020","journal-title":"NeuroImage"},{"key":"e_1_3_4_28_2","author":"Y Z B H R","year":"2019","unstructured":"Y Z B H RSimpleITK image-analysis notebooks: a collaborative environment for education and reproducible research, 2019","journal-title":"SimpleITK image-analysis notebooks: a collaborative environment for education and reproducible research"},{"key":"e_1_3_4_29_2","unstructured":"J.Shlens \u201cNotes on Kullback-Leibler Divergence and Likelihood \u201d 2014 https:\/\/arxiv.org\/abs\/1404.2000."},{"key":"e_1_3_4_30_2","article-title":"Focal loss for dense object detection","author":"Lin T.","year":"2017","unstructured":"T.Lin, P.Goyal, R. B.Girshick, K.He, and P.Dollar, \u201cFocal loss for dense object detection,\u201d CoRR, 2017, https:\/\/arxiv.org\/abs\/1708.02002.","journal-title":"CoRR"},{"key":"e_1_3_4_31_2","doi-asserted-by":"crossref","first-page":"116100","DOI":"10.1016\/j.bone.2021.116100","article-title":"Moderate tibial loading and treadmill running, but not overloading, protect adult murine bone from destruction by metastasized breast cancer","volume":"153","author":"Wang S.","year":"2021","unstructured":"S.Wang, S.Pei, M.Wasi, A.Parajuli, A.Yee, L.You, and L.Wang, \u201cModerate tibial loading and treadmill running, but not overloading, protect adult murine bone from destruction by metastasized breast cancer,\u201d Bone, vol. 153, p. 116100, 2021","journal-title":"Bone"}],"container-title":["BME Frontiers"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.spj.sciencemag.org\/bmef\/2022\/9763284.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.spj.sciencemag.org\/bmef\/2022\/9763284.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/spj.science.org\/doi\/pdf\/10.34133\/2022\/9763284","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,9]],"date-time":"2024-01-09T13:30:22Z","timestamp":1704807022000},"score":1,"resource":{"primary":{"URL":"https:\/\/spj.science.org\/doi\/10.34133\/2022\/9763284"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1]]},"references-count":31,"alternative-id":["10.34133\/2022\/9763284"],"URL":"https:\/\/doi.org\/10.34133\/2022\/9763284","relation":{},"ISSN":["2765-8031"],"issn-type":[{"value":"2765-8031","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1]]},"article-number":"9763284"}}