SUMAC 2020
The 2nd workshop on Structuring and Understanding of Multimedia heritAge Contents

12 October 2020

ACM MULTIMEDIA 2020
12-16 October 2020 Online

About the workshop

Aims and scope of the workshop

The digitization of large quantities of analogue data and the massive production of born-digital documents for many years now provide us with large volumes of varied multimedia data (images, maps, text, video, multisensor data, etc.), an important feature of which is that they are cross-domain. "Cross-domain" reflects the fact that these data may have been acquired in very different conditions: different acquisition systems, times and points of view (e.g. a 1962 postcard from the Arc de Triomphe vs. a recent street-view acquisition by mobile mapping of the same monument). These data represent an extremely rich heritage that can be exploited in a wide variety of fields, from SSH to land use and territorial policies, including smart city, urban planning, tourism, creative media and entertainment.
In terms of research in computer science, they address challenging problems related to the diversity and volume of the media across time, the variety of content descriptors (potentially including the time dimension), the veracity of the data, and the different user needs with respect to engaging with this rich material and the extraction of value out of the data. These challenges are reflected in research topics such as multimodal and mixed media search, automatic content analysis, multimedia linking and recommendation, and big data analysis and visualisation, where scientific bottlenecks may be exacerbated by the time dimension, which also provides topics of interest such as multimodal time series analysis.
The objective of the second edition of this workshop is to present and discuss the latest and most significant trends in the analysis, structuring and understanding of multimedia contents dedicated to the valorization of heritage, with emphasis on the unlocking of and access to the big data of the past.

Keynotes

Deep Image Features for Instance-level Recognition and Matching

André Araujo   (Google, USA)

André Araujo is a senior software engineer at Google Research. He obtained a PhD in electrical engineering from Stanford University, in 2016. His current research focuses on computer vision, mainly on techniques in the area of instance-level recognition such as local and global images features. He has recently served as a primary organizer of CVPR'18/19 workshops on landmark recognition and for the ECCV'20 workshop in instance-level recognition.


Towards the semantic-aware 3D digitisation of architectural heritage: the "Notre-Dame de Paris" digital twin project

Livio de Luca   (CNRS, France)

Architect, PhD in Engineering (Arts et Métiers ParisTech), HDR in Computer Science, Livio De Luca is a research director at CNRS (The French National Centre for Scientific Research) and director of CNRS-MAP unit. General Co-chair of the UNESCO/IEEE/EG DigitalHeritage international congress (Marseille 2013, Grenade 2015) and coordinator and member of national and international actions, his research activities focus on surveying, geometric modeling and semantic enrichment of digital representations of heritage objects. Associate editor of the Journal of Cultural Heritage and the Journal on Computing and Cultural Heritage, since 2016 he is an appointed member of the CoNRS. His work was rewarded in 2007 by the Pierre Bézier Prize (Arts et Métiers Foundation), in 2016 by the Medal for Research and Technology (french Academy of Architecture) and in 2019 by the CNRS Medal of Innovation. He is today coordinator of the “digital data” working group of the CNRS/Ministry of Culture scientific site for the restoration of Notre-Dame de Paris.

Organizers

Valerie Gouet-Brunet  (Université Gustave Eiffel, IGN-ENSG/LASTIG, France)

Margarita Khokhlova (Université Gustave Eiffel, IGN-ENSG/LASTIG, France), Centrale Lyon/LIRIS, France)

Ronak Kosti (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany)

Xu-Cheng Yin  (University of Science and Technology Beijing, China)

Liming Chen   (Centrale Lyon/LIRIS, France)


Program Committee

Nathalie Abadie (Université Gustave Eiffel, IGN-ENSG/LASTIG, France)

Peter Bell (Friedrich-Alexander-Universität, Germany)

Jenny Benois-Pineau (Université de Bordeaux/LABRI, France)

Véronique Eglin (INSA de Lyon/LIRIS, France)

Marin Ferecatu (Cnam Paris/CEDRIC, France)

Liangcai Gao (Peking University, China)

Sony George (The Norwegian Colour and Visual Computing Laboratory, Norway)

Leo Impett (EPFL/IVRL, Switzerland)

Pedro Jacobetty (University of Edinburgh, UK)

Martin Langner (Uni. Göttingen, Germany)

Prathmesh Madhu (Friedrich-Alexander-Universität, Germany)

Fabian Offert (FAU Erlangen, Germany/University of California, Santa Barbara)

Benjamin Renoust (Osaka University, Japan)

Fernanda Pires (Universitat Pompeu Fabra, Spain)

Li Weng (Hangzhou Dianzi University, China)

Chao Zhu (University of Science and Technology Beijing, China)

Chongsheng Zhang (Henan University, China)

Call for papers

The objective of this workshop is to present and discuss the latest and most significant trends in the analysis, structuring and understanding of multimedia contents dedicated to the valorization of heritage, with the emphasis on the unlocking of and access to the big data of the past. We welcome research contributions related to the following (but not limited to) topics:

● Multimedia and cross-domain data interlinking and recommendation
● Dating and geolocalization of historical data
● Mixed media data access and indexing
● Deep learning in adverse conditions (transfer learning, learning with side information, etc.)
● Multi-modal time series analysis, evolution modelling
● Multi-modal and multi-temporal data rendering
● HCI / Interfaces for large scale data sets
● Smart digitization of massive quantities of data
● Benchmarking, open data movement

Submission

Submission Due: Thursday 30 July 2020

Acceptance Notification: Wednesday 26 August 2020

Camera Ready Submission: Wednesday 2 September 2020

Workshop Date: Monday 12 October 2020

Submission formats

All submissions must be original work not under review at any other workshop, conference, or journal. The workshop will accept papers describing completed work as well as work in progress. One submission format is accepted: full paper, which must follow the formatting guidelines of the main conference ACM MM 2020. Full papers should be from 6 to 8 pages (plus 2 additional pages for the references), encoded as PDF and using the ACM Article Template. For paper guidelines, please visit the conference website.

Peer Review and publication in ACM Digital Library

Paper submissions must conform with the “double-blind” review policy. All papers will be peer-reviewed by experts in the field, they will receive at least two reviews. Acceptance will be based on relevance to the workshop, scientific novelty, and technical quality. Depending on the number, maturity and topics of the accepted submissions, the work will be presented via oral or poster sessions. The workshop papers will be published in the ACM Digital Library.


Submit Now

List of accepted papers

PP-LinkNet: Improving Semantic Segmentation of High Resolution Satellite Imagery with Multi-stage Training An Tran (Grabtaxi Holdings), Ali Zonoozi (Grab), Jagannadan Varadarajan (Grab)

Semantics Preserving Hierarchy based Retrieval of Indian heritage monuments Ronak Gupta (IIT Delhi), Prerana Mukherjee (IIT DELHI), Brejesh Lall (IIT Delhi), Varshul Gupta (IIT DELHI)

An Automated Pipeline for a Browser-based, City-scale Mobile 4D VR Application based on Historical Images Sander Muenster (FSU Jena), Ferdinand Maiwald (TU Dresden), Christoph Lehmann (TU Dresden), Taras Lazariv (TU Dresden), Mathias Hofmann (TU Dresden), Florian Niebling (JMU Würzburg)

New Interactive Methods for Image Registration with Applications in Repeat Photography Axel Schaffland (University of Osnabrueck), Tri Hiep Bui (University of Osnabrueck), Oliver Vornberger (University of Osnabrueck), Gunther Heidemann (Institute of Cognitive Science, University of Osnabrück)

Hybrid Human-Machine Classification System for Cultural Heritage Data Shaban Shabani (University of Basel), Maria Sokhn (HES-SO Valais), Heiko Schuldt (University of Basel)

Face Detection on Pre-modern Japanese Artworks using R-CNN and Image Patching for Semi-Automatic Annotation Alexis Mermet (EPFL), Asanobu Kitamoto (National Institute of Informatics, Japan), Chikahiko Suzuki (National Institute of Informatics, Japan), Akira Takagishi (University of Tokyo)

A Generative Adversarial Approach with Residual Learning for Dust and Scratches Artifacts Removal Ionut Mironica (Adobe Research)

Program

This year, the workshop appears virtually as the main conference. To each accepted paper is associated a pre-recorded video oral presentation (15 min), which is already available here (see the YouTube link close to the title of each presentation).

This year, we have the pleasure to award a prize of 500 euros for the best article titled
'Semantics Preserving Hierarchy based Retrieval of Indian heritage monuments' by Ronak Gupta, Prerana Mukherjee, Brejesh Lall and Varshul Gupta (IIT DELHI).
The award is sponsored by Friedrich-Alexander-Universität Erlangen, the French Mapping Agency (IGN) and the French National Research Agency (ANR, Alegoria project).

Note also that all the program is scheduled according to the Eastern Time zone (UTC−05:00).

Full day workshop

09:00-09:15 Welcome & Introductory session by organizers V. Gouet-Brunet (IGN/LaSTIG, France), M. Khokhlova (IGN/LaSTIG, Centrale Lyon/LIRIS,France), Ronak Kosti (Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany), Xu-Cheng Yin (University of Science and Technology Beijing, China), L. Chen (Centrale Lyon/LIRIS, France)
09:15-10:30

Session 1 Q&A: Description and learning of multimedia heritage contents

09.15-09.30
PP-LinkNet: Improving Semantic Segmentation of High Resolution Satellite Imagery with Multi-stage Training (video) (paper)
An Tran (Grabtaxi Holdings), Ali Zonoozi (Grab), Jagannadan Varadarajan (Grab)
09.30-09.45
Face Detection on Pre-modern Japanese Artworks using R-CNN and Image Patching for Semi-Automatic Annotation (video) (paper)
Alexis Mermet (EPFL), Asanobu Kitamoto (National Institute of Informatics, Japan), Chikahiko Suzuki (National Institute of Informatics, Japan), Akira Takagishi (University of Tokyo)
09.45-10.00
A Generative Adversarial Approach with Residual Learning for Dust and Scratches Artifacts Removal (video) (paper)
Ionut Mironica (Adobe Research)
10.00-10.15
Semantics Preserving Hierarchy based Retrieval of Indian heritage monuments (video) (paper)
Ronak Gupta (IIT Delhi), Prerana Mukherjee (IIT DELHI), Brejesh Lall (IIT Delhi), Varshul Gupta (IIT DELHI)
10.15-10.30

Break

10:30-11:30

Keynote #1: Deep Image Features for Instance-level Recognition and Matching (video)

Andre Araujo (Google, USA)
11:30-13:00

Break

13:00-14:00

Keynote #2: Towards the semantic-aware 3D digitisation of architectural heritage: the "Notre-Dame de Paris" digital twin project (video)

Livio de Luca (CNRS, France)
14:00-14:15

Break

14:15-15:00

Session 2 Q&A: Browsing, visualization and interaction in multimedia heritage contents

14:15-15:00
Hybrid Human-Machine Classification System for Cultural Heritage Data (video) (paper)
Shaban Shabani (University of Basel), Maria Sokhn (HES-SO Valais), Heiko Schuldt (University of Basel)
14:30-14:45
New Interactive Methods for Image Registration with Applications in Repeat Photography (video) (paper)
Axel Schaffland (University of Osnabrück), Tri Hiep Bui (University of Osnabrück), Oliver Vornberger (University of Osnabrück), Gunther Heidemann (Institute of Cognitive Science, University of Osnabrück)
14.45-15.00
An Automated Pipeline for a Browser-based, City-scale Mobile 4D VR Application based on Historical Images (video) (paper)
Sander Muenster (FSU Jena), Ferdinand Maiwald (TU Dresden), Christoph Lehmann (TU Dresden), Taras Lazariv (TU Dresden), Mathias Hofmann (TU Dresden), Florian Niebling (JMU Würzburg)
15:15-15:30 Wrap-up and ceremony award

Contact Information

Any questions? Please contact us!

valerie.gouet@ign.fr
margarita.khokhlova@ign.fr