{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"id":[{"id":"https:\/\/ror.org\/03mb6wj31","id-type":"ROR","asserted-by":"publisher"},{"id":"https:\/\/www.isni.org\/000000041937028X","id-type":"ISNI","asserted-by":"publisher"},{"id":"https:\/\/www.wikidata.org\/entity\/Q1640731","id-type":"wikidata","asserted-by":"publisher"}],"name":"Universitat Polit\u00e8cnica de Catalunya","acronym":["UPC"]}],"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T17:08:07Z","timestamp":1785431287530,"version":"3.56.0"},"reference-count":0,"publisher":"Universitat Polit\u00e8cnica de Catalunya","license":[{"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-sa\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>Computer vision plays a key role in Artificial Intelligence because of the rich semantic information contained in pixels and the ubiquity of cameras nowadays. Multimedia content is on a rise since social networks have such a strong impact in our society and access to the internet becomes more widespread. This context allows the gathering of large datasets which have fostered great advancements in the computer vision field thanks to deep neural networks. These models can effectively exploit large amounts of data to reach a high expressive power. Since the breakout of Imagenet, a large dataset for image classification, most computer vision tasks have benefited from deep neural networks. Among the different tasks in the computer vision field, locating objects in images and videos is a central one, as it has many applications in autonomous driving, surveillance, image and video edition, medical diagnosis and biometrics along with others. Localization of objects can be obtained with bounding boxes around the target objects, or with accurate pixel-level masks that delineate the instances. The latter is a more challenging task, but fundamental for certain applications where edges of objects need to be determined. The main task addressed in this thesis is instance segmentation, that consists in, given an image or video, providing pixel-level masks for each instance of certain semantic object classes.\u00a0\r\n\r\nIn order to train a segmentation model, current solutions rely on large amounts of pixel-wise annotations, which demand significant human effort to collect. Furthermore, expert knowledge is needed to gather certain annotations, such as labels for medical images. In consequence, there is a huge interest for systems that work with less-demanding forms of supervision, such as weakly or semi-supervised pipelines.\u00a0 Besides, in some segmentation tasks, human effort is not only needed for training the models, but also at inference. In semi-automatic systems, user input may be required as guidance to start the system. One example is the task of one-shot Video Object Segmentation (osVOS), which expects that the end-user provides a pixel-level mask for each object to be tracked in the first frame of the video. Following, the model must predict the segmentation mask of the tracked objects for the remaining frames. These initialization cues are crucial for high accuracy, but they are arduous to obtain. An alternative are models that depend on weaker input signals that are user-friendlier.\r\n\r\nThis thesis explores different supervision scenarios for the instance segmentation task, distinguishing between supervision during training and at inference, and focusing on low-supervision setups. In the first part of the thesis we present a novel recurrent architecture for video object segmentation that is end-to-end trainable in a fully-supervised setup, and that does not require any post-processing step, i.e., the output of the model directly solves the addressed task. The second part of the thesis aims at lowering the annotation cost, in terms of labeling time, needed to train image segmentation models. We explore semi-supervised pipelines and show results when a very limited budget is available. The third part of the dissertation attempts to alleviate the supervision required by semi-automatic systems at inference time. Particularly, we focus on semi-supervised video object segmentation, which typically requires generating a binary mask for each instance to be tracked. In contrast, we present a model for language-guided video object segmentation, which identifies the object to segment with a natural language expression. We study current benchmarks, propose a novel categorization of referring expressions for video, and identify the main challenges posed by the video task.<\/jats:p>\n                <jats:p>La visi\u00f3 per computador t\u00e9 un paper clau en la intel\u00b7lig\u00e8ncia artificial pel ric contingut sem\u00e0ntic dels p\u00edxels i la ubiq\u00fcitat de c\u00e0meres avui en dia.  El contingut multim\u00e8dia creix exponencialment degut a qu\u00e8 les xarxes socials tenen una gran influ\u00e8ncia en la nostra societat, i a que l\u2019acc\u00e9s a l\u2019internat \u00e9s cada cop m\u00e9s generalitzat. Aquesta contextura permet  la recol\u00b7lecci\u00f3 de grans bases de dades que impulsen aven\u00e7os gr\u00e0cies a les xarxes d\u2019aprenentatge profund. Des de generaci\u00f3 d\u2019Imagenet, una base de dades de gran escala pel problema de classificaci\u00f3 d\u2019imatges, moltes tasques de visi\u00f3 s\u2019han beneficiat de les xarxes d\u2019aprenentatge profund. D\u2019entre les diferents tasques de visi\u00f3, localitzar objectes en imatges i v\u00eddeos \u00e9s de les m\u00e9s rellevants, ja que t\u00e9 moltes aplicacions en \u00e0mbits com la conducci\u00f3 aut\u00f2noma, la videovigil\u00e0ncia, l\u2019edici\u00f3 d\u2019imatges i v\u00eddeos, el diagn\u00f2stic m\u00e8dic per imatge, i aplicacions biom\u00e8triques d\u2019entre altres. La localitzaci\u00f3 d\u2019objectes es pot resoldre amb el que popularment s'anomenen bounding boxes, o amb segmentacions a nivell de p\u00edxel, sent aquesta segona una tasca m\u00e9s complexa. En aquesta tesis investiguem la segmentaci\u00f3 d\u2019inst\u00e0ncies, que consisteix en, a partir d\u2019una imatge o un v\u00eddeo, predir segmentacions de cada inst\u00e0ncia que hi apareix de certes categories sem\u00e0ntiques. Per tal d\u2019entrenar un model de segmentaci\u00f3, les solucions actuals entrenen amb grans bases de dades que tenen anotacions a nivell de p\u00edxel. Aquests anotacions s\u00f3n molt costoses d\u2019obtenir ja que es requereix molt temps de feina manual. A m\u00e9s, per certes anotacions \u00e9s necess\u00e0ria la intervenci\u00f3 d\u2019experts, com per exemple per imatges m\u00e8diques. Per aquests motius hi ha molt inter\u00e8s tant en sistemes que es puguin entrenar amb formes m\u00e9s senzilles d\u2019anotacions com en sistemes semi-supervisats. En algunes tasques l\u2019esfor\u00e7 d\u2019anotar no nom\u00e9s es requereix per les dades d\u2019entrenament, sin\u00f3 que tamb\u00e9 \u00e9s necessari en fase de testeig. En certs models semi-autom\u00e0tics l\u2019usuari ha d\u2019introduir alguna mena d\u2019anotaci\u00f3 per tal de qu\u00e8 funcioni el sistema. Un exemple d\u2019aquest tipus de tasca \u00e9s la segmentaci\u00f3 semi-supervisada d'objectes en v\u00eddeos, on es necessita com a entrada al sistema una m\u00e0scara a nivell de p\u00edxel per cada objecte que es vulgui segmentar en la primera imatge del v\u00eddeo. A continuaci\u00f3, el model prediu segmentacions per la resta del v\u00eddeo. Aquesta inicialitzaci\u00f3 \u00e9s imprescindible per obtenir m\u00e0scares precises, per\u00f2 \u00e9s molt costosa. Una alternativa \u00e9s treballar amb senyals m\u00e9s f\u00e0cils d\u2019obtenir. Aquesta tesis explora diferents nivells de supervisi\u00f3 per la tasca de segmentaci\u00f3 d\u2019inst\u00e0ncies, diferenciant entre supervisi\u00f3 en entrenament i en infer\u00e8ncia. En concret, el nostre objectiu \u00e9s treballar amb poca supervisi\u00f3. En la primera part de la tesis presentem una arquitectura recurrent per la tasca de segmentaci\u00f3 d\u2019objectes en v\u00eddeos, que es pot entrenar end-to-end de forma totalment supervisada, i que no requereix cap post-processament, \u00e9s a dir, la sortida del model directament solventa la tasca final. La segona part es centra en reduir el cost d\u2019anotacions de bases de dades per entrenar models de segmentaci\u00f3 d\u2019imatges. Explorem arquitectures semi-supervisades i presentem resultats quan nom\u00e9s es disposa d\u2019un pressupost d\u2019anotaci\u00f3 molt limitat. La tercera part de la tesis es centra en reduir el nivell de supervisi\u00f3 en sistemes semi-autom\u00e0tics en infer\u00e8ncia. En concret, investiguem la tasca de segmentaci\u00f3 semi-supervisada d\u2019objectes en v\u00eddeos, tasca que tradicionalment requereix que l\u2019usuari indiqui quins objectes cal segmentar amb m\u00e0scares a nivell de p\u00edxel. En canvi, nosaltres presentem un model que utilitza el llenguatge natural. Estudiem les bases de dades actuals i proposem una categoritzaci\u00f3 d\u2019expressions de llenguatge per tal d\u2019identificar els majors reptes.<\/jats:p>","DOI":"10.5821\/dissertation-2117-347912","type":"dissertation","created":{"date-parts":[[2023,7,19]],"date-time":"2023-07-19T03:53:55Z","timestamp":1689738835000},"approved":{"date-parts":[[2021,3,26]]},"source":"Crossref","is-referenced-by-count":0,"title":["Image and video object segmentation in low supervision scenarios"],"prefix":"10.5821","author":[{"given":"M\u00edriam","family":"Bellver Bueno","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"3865","container-title":[],"original-title":[],"contributor":[{"sequence":"additional","affiliation":[],"role":[null]}],"deposited":{"date-parts":[[2026,2,6]],"date-time":"2026-02-06T06:31:05Z","timestamp":1770359465000},"score":1,"resource":{"primary":{"URL":"https:\/\/hdl.handle.net\/2117\/347912"}},"subtitle":[],"editor":[{"given":"Jordi","family":"Torres Vi\u00f1als","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]},{"given":"Xavier","family":"Gir\u00f3 Nieto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[null]]},"references-count":0,"URL":"https:\/\/doi.org\/10.5821\/dissertation-2117-347912","relation":{},"subject":[]}}