Multimedia Discourse Interaction
Official course page

The course is led by Lynda Hardman, and runs from 14th November 2018 until 1st February 2019.

NOTE: This web page is dynamic and will change throughout the course.

Table of contents

• Schedule
• Seed Publications
• Interesting things to present
• Assignments
• Useful Web sites
• Semantic Web resources
• Random useful links

Schedule

Wednesday 14th November 15:15-17:00

15:15 Who is the teacher? What is the course about?
What is your background? What do you want to do in the course? How does this course fit with your other courses?
When do we meet?
What shall we do before next Wednesday's meeting?
What do we need to keep track of? How will we keep track of it?
Who is in which group?
How can we communicate most efficiently? How do you get the most out of my time?
Do we want to use Slack for group communication.
Talk From Linked Data to Stories presented by Lynda

Friday 16th November 13:15-15:00

No session. If you still need to join the Slack group then email Lynda.

Wednesday 21st November 15:15-17:00

Presentations and discussions on 5 papers

Provide 3 good points and 3 points for improvement about each of the 5 papers to be read by Wed 21st 15:00. These answers are to help you think about how you will write your final report for the course.
Answers can be found via the Slack group.

Decide in class on communal papers to read for Wednesday 28th November.
Before Friday November 23rd come up with suggestions for 5 papers to read for Wednesday 28th November (and beyond). Put them in order of preference since you need only read 2 or 3 before 28th November.

Friday 23rd November 13:15-15:00

Potential project topics:
• Visualizations for Linked Brain Data
• Marc Geerards composition of video clips around a specific topic. Video clips together with topic & filmic annotations are available.
• Diana Guzijan, perception of cultural heritage
• National Geographic Making the Case for Nature

Talks (to be) presented by Lynda
Ana Carina Palumbo User information needs for environmental opinion-forming and decision-making in linked-enriched video
Helen Boots-Blankers Identifying Aspects of Informative Videos that Indicate Objectivity: Is Seeing Believing? Slide 15, video fragment 1 , video fragment 2. MSc thesis Is Seeing Believing? Identifying Aspects of Informative Videos that Indicate Objectivity.

Wednesday 28th November 15:15-17:00

Presentations and discussions on 5 papers. Discuss what should be in initial project proposals

Friday 30th November 13:15-15:00

Work on initial project proposals. Feedback to be supplied by Lynda asap.

Wednesday 5th December 15:15-17:00

Second draft of individual project proposals submitted.
Presentations of initial project proposals (5 minutes presentation and 3 minutes discussion per group).
Use of time to be planned. Schedule presentations of papers and how they relate to project.

Friday 7th December 13:15-16:00

To be planned

Wednesday 12th December 15:15-17:00

First draft of structured set of writing tips. [Link to be proposed by group]
To be planned

Friday 14th December 13:15-16:00

To be planned

Wednesday 19th December 15:15-17:00

To be planned

Friday 21st December 13:15-15:00

To be planned

Wednesday 9th January 15:15-17:00

Group project pilot studies.

Friday 11th January 13:15-16:00

Group project pilot studies.

Wednesday 16th January 15:15-17:00

To be planned

Friday 18th January 13:15-15:00

Lynda away - to be planned

Wednesday 23rd January 15:15-17:00

Lynda away - to be planned

Friday 25th January 13:15-15:00

Lynda away - to be planned
Draft reports submitted for feedback (by 26th January).

Wednesday 30th January 14:15-17:00

Final presentations 15 minutes per group - 1 minute for set-up, 12 minutes for the talk, 2 minutes for questions.

Friday 1st Feburary 13:15-16:00

No room booked. Final report to be handed in.

Seed publications for the seminar

Papers to read for week 2 of course

• Vox Populi paper:Automatic generation of matter-of-opinion video documentaries, talk, demo and thesis by Stefano Bocconi. Interview With America documentary.
• Artequakt: Generating Tailored Biographies with Automatically Annotated Fragments from the Web, S. Kim, H. Alani, W. Hall, P. Lewis, D. Millard, N. Shadbolt, and M. Weal. In Semantic Authoring, Annotation and Knowledge Markup (SAAKM) 2002 Workshop at the 15th European Conference on Artificial Intelligence (ECAI 2002), Lyon, France.
• Link-enriched video: user information needs for environmental opinion-forming and decision-making slides in pdf, technical report and bad quality video from 1hr 3mins by Ana Carina Palumbo
• Generation of abstract geometric art based on exact aesthetics, gestalt theory and graphic design principles, Michiel Kauw-A-Tjoe, CWI Tech report
• Genre Driven Multimedia Document Production by Means of Incremental Transformation Marc Nanard, Jocelyne Nanard, Peter R. King, Ludovic Gaillard, ACM Symposium on Document Engineering 2007: 111-120

Papers to read for week 3 of course

• Where story and media meet: computer generation of narrative discourse, Rémi Ronfard, Nicolas Szilas, CMN 2014: 164-176
• Towards Constructive Text, Diagram, and Layout Generation for Information Presentation, John Bateman, Thomas Kamps, Jörg Kleinz and Klaus Reichenberger, Computational Linguistics 27(3) pp409-449
• Can Computers Create Humor? Greame Ritchie. AI Magazine 30(3): 71-81 (2009)

Papers not yet read

• Towards Ontology-Driven Discourse: From Semantic Graphs to Multimedia Presentations, Joost Geurts, Stefano Bocconi, Jacco van Ossenbruggen, Lynda Hardman. ISWC, 2nd International Semantic Web Conference 2003, 597-612.
• Context Aware Guidance for Multimedia Authoring: Harmonizing Domain and Discourse Knowledge by Kateryna Falkovych. Further articles on the topic all Kateryna's publications at CWI.
• Generation of video documentaries from discourse structures, Cesare Rocchi and Massimo Zancanaro, in proceedings of the 9th European Workshop on Natural Language Generation. 2003
• Enriching news for supporting users' information needs using schema-driven classification of entities and relations, Viola Pinzi, MSc thesis, UvA 2014
• Papers by Marc Cavazza.

Interesting things to present

• Introduction to the LinkedTV project, Lynda and Lilia Pérez Romero
• TV screen of LinkedTV News demo and second screen application
• LinkedTV News video demo. Lilia's USI project report LinkedTV News: designing a second screen companion for web-enriched news broadcasts.
• Short video explanation of entities (What is Metaweb?)
• Linked Data - Connect Distributed Data across the Web, The Linking Open Data cloud diagram and its statistics.

• How the emergence of the semantic web changes our approach to information architecture by Silver Oliver, BBC.
• Linking hypervideos to Web content, Raphaël Troncy
• EventMedia Live: Exploring Events Connections in Real-Time to Enhance Content
• Experiencing Events Through User-Generated Media by André Fialho.
• Meaningful presentations.
• The next 5,000 days of the Web
• Modes of persuasion
• Rhetorical Structure Theory
• Investigation of methods for user adapted visualisation of information in a hypermedia generation system by Jana Werner has a figure of different (simple) discourse structures on p37.

New HTML5 video technologies for the future of TV by Silvia Pfeiffer.
Papers on finding and filtering links by Michiel Hildebrand: Thesaurus-based search in large heterogeneous collections, Searching in semantically rich linked data: a case study in cultural heritage Wordnet allows you to traverse the class hierarchy using hyponyms and hypernyms (after finding the definition of a word, click on the underlined S). More information on the linking and filtering:

• for the die hards Towards a pattern science for the Semantic Web
• for those who want something that just works, tools are available, in particular the relationship finder.

Assignments

Everything in Google doc, including links to contributions

Useful Web Sites

• europeana.
• CHIP allows exploratory access to (cultural heritage) information.

These show interesting interfaces:

• openAcademia shows several visualizations of search results (tagcloud, topic graph, relation browser.
• OntoWiki follows the wiki style with faceted navigation, map and calendar visualization.
• Tabulator (Tim Berners-Lee) is available as a Firefox extension. Very simple interface, which effectively shows (and allows navigation) of the raw graph structure (not included in the survey).

These are more oriented to search.

• Swoogle, semantic web search engine.
• Squiggle is a simple but effective interface for recommendation and search in music.
• SemSearch allows simple semi-structured input queries.
• Freebase, an open shared database of the world's knowledge (not strictly semantic web, but similar goals).

Vox Populi, a system for generating video sequences based on argument structures that uses explicit representation of argumentation structure.

Semantic Web Resources

How Will We Interact with the Web of Data? by Tom Heath.
Semantic Web Challenge, in particular the 2008 winner paggr.

Wordnet is an existing English language thesaurus from Princeton that has been converted to RDF.
Dublin Core (DC), VRA Core are vocabularies for describing resources. VRA Core is specialised for works of visual culture as well as the images that document them.
SKOS, Simple Knowledge Organisation System, is a family of formal languages, built on RDF and RDFS, designed for the representation of structured controlled vocabularies, such as thesauri or classification schemes.
FOAF (Friend of a Friend) a means of creating links among people on the Web, also the FOAF project, in particular the explorer.

Random Useful Links

These are some links that may not be useful now for the course, but are useful resources about relevant topics.

• Seminal article on the notion of mechanised support for access to information. Quoted by both the hypertext and digital library communities.
  As We May Think by Vannevar Bush.
• Weaving the Web, the story of the Web from Tim Berners-Lee himself.
• "Understanding Comics" by Scott McCloud for a discussion of the whys and wherefors of text and images and mixes of the two.
• Inaugural speech Smart Style for Conveying Information at TU/e)

Web Design

• Cool examples of CSS.
• "Don't Make Me Think : A Common Sense Approach to Web Usability" by Steve Krug for a simple but fundamental approach to information design work.
• Usability for the Web: Designing Web Sites that Work, The Morgan Kaufmann Series in Interactive Technologies.
• Exhibit allows you to combine different interfaces into a single one. For example, a time and place visualization of American Presidents using both a map and a timeline.

Semantic Web

• "The Semantic Web: an Introduction" by Sean B. Palmer
• Handy Semantic Web reference cheat sheet. Click on the pdf logo for an A4 version.
• Article explaining the (many) connections and similarities of hypermedia research and the Semantic Web.
• Article about Semantic Web applications
  "Usability and the Semantic Web" by Anthony Jameson. Slides of talk
• Semantic Web application examples RDFa bookmarklets
• Discussion of whether entities should be able to be modelled at the class and instance level simultaneously.
  Representing Classes As Property Values on the Semantic Web

Facet Browsing

• Flamenco paper Faceted Metadata for Image Search and Browsing and demo.
• Longwell and demo
• Facet browsing at Ebay
• E-culture's /facet