Multimedia Discourse Interaction
Official course page

The course is led by Lynda Hardman, and runs from 16th November 2016 until February 2017.

NOTE: This web page is dynamic and will change throughout the course.

Table of contents

• Schedule
• Seed Publications
• Interesting things to present
• Assignments
• Useful Web sites
• Semantic Web resources
• Random useful links

Schedule

Wednesday 16th November 13:15-15:00

13:15 Who is the teacher? What is the course about?
What is your background? What do you want to do in the course?
How does this course fit with other courses? (When are the work hours?)
14:15 What shall we do before Friday's lecture?

Friday 18th November 11:00-12:45

Talk From Linked Data to Stories presented by Lynda

Wednesday 23rd November 13:15-15:00

Presentations and discussions on 5 papers
Decide on papers for next week
Discuss potential publication venues.

Friday 25th November 11:00-12:45

Finalise plan

Seed publications for the seminar

• Vox Populi paper: Automatic generation of matter-of-opinion video documentaries, talk, demo and thesis by Stefano Bocconi.
• Artequakt: Generating Tailored Biographies with Automatically Annotated Fragments from the Web, S. Kim, H. Alani, W. Hall, P. Lewis, D. Millard, N. Shadbolt, and M. Weal. In Semantic Authoring, Annotation and Knowledge Markup (SAAKM) 2002 Workshop at the 15th European Conference on Artificial Intelligence (ECAI 2002), Lyon, France.
• Towards Ontology-Driven Discourse: From Semantic Graphs to Multimedia Presentations, Joost Geurts, Stefano Bocconi, Jacco van Ossenbruggen, Lynda Hardman. ISWC, 2nd International Semantic Web Conference 2003, 597-612.
• Context Aware Guidance for Multimedia Authoring: Harmonizing Domain and Discourse Knowledge by Kateryna Falkovych. Further articles on the topic all Kateryna's publications at CWI.
• Link-enriched video: user information needs for environmental opinion-forming and decision-making slides in pdf, technical report and bad quality video from 1hr 3mins by Ana Carina Palumbo
• Generation of abstract geometric art based on exact aesthetics, gestalt theory and graphic design principles, Michiel Kauw-A-Tjoe, CWI Tech report
• Genre Driven Multimedia Document Production by Means of Incremental Transformation Marc Nanard, Jocelyne Nanard, Peter R. King, Ludovic Gaillard, ACM Symposium on Document Engineering 2007: 111-120
• Where story and media meet: computer generation of narrative discourse, Rémi Ronfard, Nicolas Szilas, CMN 2014: 164-176
• Generation of video documentaries from discourse structures, Cesare Rocchi and Massimo Zancanaro, in proceedings of the 9th European Workshop on Natural Language Generation. 2003
• Can Computers Create Humor? Greame Ritchie. AI Magazine 30(3): 71-81 (2009)
• Enriching news for supporting users' information needs using schema-driven classification of entities and relations Viola Pinzi, MSc thesis, UvA 2014
• Papers by Marc Cavazza.

Interesting things to present

• Introduction to the LinkedTV project, Lynda and Lilia Pérez Romero
• TV screen of LinkedTV News demo and second screen application
• LinkedTV News video demo. Lilia's USI project report LinkedTV News: designing a second screen companion for web-enriched news broadcasts.
• Short video explanation of entities (What is Metaweb?)
• Linked Data - Connect Distributed Data across the Web, The Linking Open Data cloud diagram and its statistics.

• How the emergence of the semantic web changes our approach to information architecture by Silver Oliver, BBC.
• Linking hypervideos to Web content, Raphaël Troncy
• EventMedia Live: Exploring Events Connections in Real-Time to Enhance Content
• Experiencing Events Through User-Generated Media by André Fialho.
• Meaningful presentations.
• The next 5,000 days of the Web
• Modes of persuasion
• Rhetorical Structure Theory
• Investigation of methods for user adapted visualisation of information in a hypermedia generation system by Jana Werner has a figure of different (simple) discourse structures on p37.

New HTML5 video technologies for the future of TV by Silvia Pfeiffer.
Papers on finding and filtering links by Michiel Hildebrand: Thesaurus-based search in large heterogeneous collections, Searching in semantically rich linked data: a case study in cultural heritage Wordnet allows you to traverse the class hierarchy using hyponyms and hypernyms (after finding the definition of a word, click on the underlined S). More information on the linking and filtering:

• for the die hards Towards a pattern science for the Semantic Web
• for those who want something that just works, tools are available, in particular the relationship finder.

Assignments

Everything in Google doc, including links to contributions

Useful Web Sites

• europeana.
• CHIP allows exploratory access to (cultural heritage) information.

These show interesting interfaces:

• openAcademia shows several visualizations of search results (tagcloud, topic graph, relation browser.
• OntoWiki follows the wiki style with faceted navigation, map and calendar visualization.
• Tabulator (Tim Berners-Lee) is available as a Firefox extension. Very simple interface, which effectively shows (and allows navigation) of the raw graph structure (not included in the survey).

These are more oriented to search.

• Swoogle, semantic web search engine.
• Squiggle is a simple but effective interface for recommendation and search in music.
• SemSearch allows simple semi-structured input queries.
• Freebase, an open shared database of the world's knowledge (not strictly semantic web, but similar goals).

Vox Populi, a system for generating video sequences based on argument structures that uses explicit representation of argumentation structure.

Semantic Web Resources

How Will We Interact with the Web of Data? by Tom Heath.
Semantic Web Challenge, in particular the 2008 winner paggr.

Wordnet is an existing English language thesaurus from Princeton that has been converted to RDF.
Dublin Core (DC), VRA Core are vocabularies for describing resources. VRA Core is specialised for works of visual culture as well as the images that document them.
SKOS, Simple Knowledge Organisation System, is a family of formal languages, built on RDF and RDFS, designed for the representation of structured controlled vocabularies, such as thesauri or classification schemes.
FOAF (Friend of a Friend) a means of creating links among people on the Web, also the FOAF project, in particular the explorer.

Random Useful Links

These are some links that may not be useful now for the course, but are useful resources about relevant topics.

• Seminal article on the notion of mechanised support for access to information. Quoted by both the hypertext and digital library communities.
  As We May Think by Vannevar Bush.
• Weaving the Web, the story of the Web from Tim Berners-Lee himself.
• "Understanding Comics" by Scott McCloud for a discussion of the whys and wherefors of text and images and mixes of the two.
• Inaugural speech Smart Style for Conveying Information at TU/e)

Web Design

• Cool examples of CSS.
• "Don't Make Me Think : A Common Sense Approach to Web Usability" by Steve Krug for a simple but fundamental approach to information design work.
• Usability for the Web: Designing Web Sites that Work, The Morgan Kaufmann Series in Interactive Technologies.
• Exhibit allows you to combine different interfaces into a single one. For example, a time and place visualization of American Presidents using both a map and a timeline.

Semantic Web

• "The Semantic Web: an Introduction" by Sean B. Palmer
• Handy Semantic Web reference cheat sheet. Click on the pdf logo for an A4 version.
• Article explaining the (many) connections and similarities of hypermedia research and the Semantic Web.
• Article about Semantic Web applications
  "Usability and the Semantic Web" by Anthony Jameson. Slides of talk
• Semantic Web application examples RDFa bookmarklets
• Discussion of whether entities should be able to be modelled at the class and instance level simultaneously.
  Representing Classes As Property Values on the Semantic Web

Facet Browsing

• Flamenco paper Faceted Metadata for Image Search and Browsing and demo.
• Longwell and demo
• Facet browsing at Ebay
• E-culture's /facet