{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T11:31:16Z","timestamp":1790681476181,"version":"4.1.0"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2018,12,4]],"date-time":"2018-12-04T00:00:00Z","timestamp":1543881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Basic Research Program of China","doi-asserted-by":"crossref","award":["2011CB302400"],"award-info":[{"award-number":["2011CB302400"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Research Grant Council of Hong Kong","award":["717012,17209815,17211017"],"award-info":[{"award-number":["717012,17209815,17211017"]}]},{"DOI":"10.13039\/501100011002","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61272019"],"award-info":[{"award-number":["61272019"]}],"id":[{"id":"10.13039\/501100011002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,12,31]]},"abstract":"<jats:p>Sketching provides an intuitive user interface for communicating free form shapes. While human observers can easily envision the shapes they intend to communicate, replicating this process algorithmically requires resolving numerous ambiguities. Existing sketch-based modeling methods resolve these ambiguities by either relying on expensive user annotations or by restricting the modeled shapes to specific narrow categories. We present an approach for modeling generic freeform 3D surfaces from sparse, expressive 2D sketches that overcomes both limitations by incorporating convolution neural networks (CNN) into the sketch processing workflow.<\/jats:p>\n          <jats:p>Given a 2D sketch of a 3D surface, we use CNNs to infer the depth and normal maps representing the surface. To combat ambiguity we introduce an intermediate CNN layer that models the dense curvature direction, or flow, field of the surface, and produce an additional output confidence map along with depth and normal. The flow field guides our subsequent surface reconstruction for improved regularity; the confidence map trained unsupervised measures ambiguity and provides a robust estimator for data fitting. To reduce ambiguities in input sketches users can refine their input by providing optional depth values at sparse points and curvature hints for strokes. Our CNN is trained on a large dataset generated by rendering sketches of various 3D shapes using non-photo-realistic line rendering (NPR) method that mimics human sketching of free-form shapes. We use the CNN model to process both single- and multi-view sketches. Using our multi-view framework users progressively complete the shape by sketching in different views, generating complete closed shapes. For each new view, the modeling is assisted by partial sketches and depth cues provided by surfaces generated in earlier views. The partial surfaces are fused into a complete shape using predicted confidence levels as weights.<\/jats:p>\n          <jats:p>We validate our approach, compare it with previous methods and alternative structures, and evaluate its performance with various modeling tasks. The results demonstrate our method is a new approach for efficiently modeling freeform shapes with succinct but expressive 2D sketches.<\/jats:p>","DOI":"10.1145\/3272127.3275051","type":"journal-article","created":{"date-parts":[[2018,11,28]],"date-time":"2018-11-28T19:16:10Z","timestamp":1543432570000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":63,"title":["Robust flow-guided neural prediction for sketch-based freeform surface modeling"],"prefix":"10.1145","volume":"37","author":[{"given":"Changjian","family":"Li","sequence":"first","affiliation":[{"name":"The University of Hong Kong, Hong Kong and Microsoft Research Asia, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Pan","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yang","family":"Liu","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xin","family":"Tong","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alla","family":"Sheffer","sequence":"additional","affiliation":[{"name":"University of British Columbia, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenping","family":"Wang","sequence":"additional","affiliation":[{"name":"The University of Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,12,4]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1449715.1449740"},{"key":"e_1_2_2_2_1","volume-title":"Matisse: Painting 2D regions for Modeling Free-Form Shapes. In SBIM. 57--64.","author":"Bernhardt Adrien","year":"2008","unstructured":"Adrien Bernhardt , Adeline Pihuit , Marie-Paule Cani , and Lo\u00efc Barthe . 2008 . Matisse: Painting 2D regions for Modeling Free-Form Shapes. In SBIM. 57--64. Adrien Bernhardt, Adeline Pihuit, Marie-Paule Cani, and Lo\u00efc Barthe. 2008. Matisse: Painting 2D regions for Modeling Free-Form Shapes. In SBIM. 57--64."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00131148"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cag.2015.05.026"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508378"},{"key":"e_1_2_2_6_1","unstructured":"Christopher B Choy Danfei Xu JunYoung Gwak Kevin Chen and Silvio Savarese. 2016. 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction. In ECCV.  Christopher B Choy Danfei Xu JunYoung Gwak Kevin Chen and Silvio Savarese. 2016. 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction. In ECCV."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360687"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531334"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882354"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3203197"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311576"},{"key":"e_1_2_2_12_1","first-page":"1","article-title":"Designing N-PolyVector Fields with Complex Polynomials","volume":"33","author":"Diamanti Olga","year":"2014","unstructured":"Olga Diamanti , Amir Vaxman , Daniele Panozzo , and Olga Sorkine-Hornung . 2014 . Designing N-PolyVector Fields with Complex Polynomials . SGP 33 , 5 (2014), 1 -- 11 . Olga Diamanti, Amir Vaxman, Daniele Panozzo, and Olga Sorkine-Hornung. 2014. Designing N-PolyVector Fields with Complex Polynomials. SGP 33, 5 (2014), 1--11.","journal-title":"SGP"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.304"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185527"},{"key":"e_1_2_2_15_1","volume-title":"Guibas","author":"Fan Haoqiang","year":"2017","unstructured":"Haoqiang Fan , Hao Su , and Leonidas J . Guibas . 2017 . A Point Set Generation Network for 3D Object Reconstruction from a Single Image. In CVPR. Haoqiang Fan, Hao Su, and Leonidas J. Guibas. 2017. A Point Set Generation Network for 3D Object Reconstruction from a Single Image. In CVPR."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618494"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882276"},{"key":"e_1_2_2_18_1","doi-asserted-by":"crossref","unstructured":"Thibault Groueix Matthew Fisher Vladimir G. Kim Bryan Russell and Mathieu Aubry. 2018. AtlasNet: A Papier-M\u00e2ch\u00e9 Approach to Learning 3D Surface Generation. In CVPR.  Thibault Groueix Matthew Fisher Vladimir G. Kim Bryan Russell and Mathieu Aubry. 2018. AtlasNet: A Papier-M\u00e2ch\u00e9 Approach to Learning 3D Surface Generation. In CVPR.","DOI":"10.1109\/CVPR.2018.00030"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12966"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073629"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.345074"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2467771"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2710026"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311602"},{"key":"e_1_2_2_25_1","volume-title":"Carr","author":"Joshi Pushkar","year":"2008","unstructured":"Pushkar Joshi and Nathan A . Carr . 2008 . Repouss\u00e9 : Automatic Inflation of 2D Artwork. In SBIM (SBM '08). Pushkar Joshi and Nathan A. Carr. 2008. Repouss\u00e9: Automatic Inflation of 2D Artwork. In SBIM (SBM'08)."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276401"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2749458"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487237"},{"key":"e_1_2_2_29_1","unstructured":"Alex Kendall and Yarin Gal. 2017. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?. In NIPS.   Alex Kendall and Yarin Gal. 2017. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?. In NIPS."},{"key":"e_1_2_2_30_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A Method for Stochastic Optimization . arXiv preprint arXiv:1412.6980 (2014). Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_2_31_1","unstructured":"Jeehyung Lee and Thomas Funkhouser. 2008. Sketch-based Search and Composition of 3D Models. In SBIM (SBM'08). 97--104.   Jeehyung Lee and Thomas Funkhouser. 2008. Sketch-based Search and Composition of 3D Models. In SBIM (SBM'08). 97--104."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073632"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2017.00018"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276429"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073226"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925951"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2011.84"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_39_1","unstructured":"Szymon Rusinkiewicz and Doug DeCarlo. 2013. Real-time suggestive contours. (2013). http:\/\/www.cs.princeton.edu\/gfx\/proj\/sugcon\/  Szymon Rusinkiewicz and Doug DeCarlo. 2013. Real-time suggestive contours. (2013). http:\/\/www.cs.princeton.edu\/gfx\/proj\/sugcon\/"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618495"},{"key":"e_1_2_2_41_1","volume-title":"Mario Costa Sousa, and Joaquim A. Jorge","author":"Schmidt Ryan","year":"2005","unstructured":"Ryan Schmidt , Brian Wyvill , Mario Costa Sousa, and Joaquim A. Jorge . 2005 . ShapeShop: Sketch- Based Solid Modeling with BlobTrees. In SBIM. Ryan Schmidt, Brian Wyvill, Mario Costa Sousa, and Joaquim A. Jorge. 2005. ShapeShop: Sketch-Based Solid Modeling with BlobTrees. In SBIM."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185541"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12044"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.114"},{"key":"e_1_2_2_45_1","unstructured":"Wanchao Su Dong Du Xin Yang Shizhe Zhou and Hongbo Fu. 2018. Interactive Sketch-Based Normal Map Generation with Deep Neural Networks. In ACM i3D.  Wanchao Su Dong Du Xin Yang Shizhe Zhou and Hongbo Fu. 2018. Interactive Sketch-Based Normal Map Generation with Deep Neural Networks. In ACM i3D."},{"key":"e_1_2_2_46_1","doi-asserted-by":"crossref","unstructured":"Maxim Tatarchenko Alexey Dosovitskiy and Thomas Brox. 2016. Multi-view 3D Models from Single Images with a Convolutional Network. In ECCV. 322--337.  Maxim Tatarchenko Alexey Dosovitskiy and Thomas Brox. 2016. Multi-view 3D Models from Single Images with a Convolutional Network. In ECCV. 322--337.","DOI":"10.1007\/978-3-319-46478-7_20"},{"key":"e_1_2_2_47_1","volume-title":"Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs. arXiv preprint arXiv:1703.09438","author":"Tatarchenko Maxim","year":"2017","unstructured":"Maxim Tatarchenko , Alexey Dosovitskiy , and Thomas Brox . 2017. Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs. arXiv preprint arXiv:1703.09438 ( 2017 ). Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. 2017. Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs. arXiv preprint arXiv:1703.09438 (2017)."},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298797"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818068"},{"key":"e_1_2_2_50_1","doi-asserted-by":"crossref","unstructured":"X. Wang D. F. Fouhey and A. Gupta. 2015a. Designing deep networks for surface normal estimation. In CVPR. 539--547.  X. Wang D. F. Fouhey and A. Gupta. 2015a. Designing deep networks for surface normal estimation. In CVPR. 539--547.","DOI":"10.1109\/CVPR.2015.7298652"},{"key":"e_1_2_2_51_1","unstructured":"Jiajun Wu Chengkai Zhang Tianfan Xue William T Freeman and Joshua B Tenenbaum. 2016. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In NIPS. 82--90.   Jiajun Wu Chengkai Zhang Tianfan Xue William T Freeman and Joshua B Tenenbaum. 2016. Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. In NIPS. 82--90."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12200"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601128"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461968"},{"key":"e_1_2_2_55_1","first-page":"1","article-title":"Interactive High-Relief Reconstruction for Organic and Double-sided Objects from a Photo","volume":"99","author":"Yeh C. K.","year":"2016","unstructured":"C. K. Yeh , S. Y. Huang , P. K. Jayaraman , C. W. Fu , and T. Y. Lee . 2016 . Interactive High-Relief Reconstruction for Organic and Double-sided Objects from a Photo . IEEE. T. Vis. Comput. Gr. 99 (2016), 1 -- 1 . C. K. Yeh, S. Y. Huang, P. K. Jayaraman, C. W. Fu, and T. Y. Lee. 2016. Interactive High-Relief Reconstruction for Organic and Double-sided Objects from a Photo. IEEE. T. Vis. Comput. Gr. 99 (2016), 1--1.","journal-title":"IEEE. T. Vis. Comput. Gr."},{"key":"e_1_2_2_56_1","volume-title":"CVPR","volume":"1","author":"Zhang Li","unstructured":"Li Zhang , G. Dugas-Phocion , J. S. Samson , and S. M. Seitzt . 2001. Single view modeling of free-form scenes . In CVPR , Vol. 1 . I-990--I-997. Li Zhang, G. Dugas-Phocion, J. S. Samson, and S. M. Seitzt. 2001. Single view modeling of free-form scenes. In CVPR, Vol. 1. I-990--I-997."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3272127.3275051","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3272127.3275051","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:44:04Z","timestamp":1750207444000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3272127.3275051"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12,4]]},"references-count":56,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2018,12,31]]}},"alternative-id":["10.1145\/3272127.3275051"],"URL":"https:\/\/doi.org\/10.1145\/3272127.3275051","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,12,4]]},"assertion":[{"value":"2018-12-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}