INTRO: concepts, precedents, method
Precedents: immersive fiction, doc, theater, art installation
Concepts: immersion, presence, spatializing story, multiple perspective & entry
Method: choosing story, strategy
Wk 01, Aug 31: Why immersion + documentary?
- Documenting & representing reality > immersive, augmented and virtual reality
- A timeline of ‘capturing life’ to ‘is it real?’
Assignment: for wk 2, Read NYT article, 2010, A.O. Scott’s How Real Does it Feel?
Wk 02, Sep 07: Aesthetics of Immersion
- Spatialization of story; how do you map narrative onto space? (30 min)
- Hierarchy of space mapping: 2D analog approach to physical mapping; 3D physical space of immersive theater; 3D CG space of video games; AR in locative space; VR
- How might one design for these affordances? (30 min)
- Feeling of presence, from literature, fictional world building, mobile AR
- Agency vs passivity, ie, traditional curated experience
- Spatialization of perspective: encountering story, multiple entry points, multiple POV
- Character; voice, unreliable narrator
- Time; reconstruction, reenactment
- Combine reality & cg , where to draw lines: Empathetic’s The Peace Process or Emblematic’s Crossing the Line (20 min)
- In class exercise: Silent Running, Echo Location (20)
- Create teams for flipbook exercise
Assignment for week 3
Prototype 1: Flipbook
Goals; to use the familiar 2D medium of photography to explore 3D space; to experiment with creating sense of immersion; to tell an immersive story about part of the city, including at least 2 characters, some contextual information and preferably focus on one specific environment.
Reference: NYT’s Walking in War’s Path
Read: Scott McCloud’s Understanding Comics ch 4, “Time Frames” to consider visualizing different beats / transitions from moment to moment
Wk 03, Sep 14: which stories? Intro to 360 vid
- Review of flipbook prototypes (5 x 10 min ~ 60 min)
- Dan’s report backs from Bangladesh, issues of environmental & economic distress, workers rights in garment factories, used as prompts, bringing realities home (30 min)
- What kinds of stories lend to this format?
- Crash Course in 360 video, Intro to Ricoh with Zahra Rasool (60 min)
Assignment for week 4
Prototype 2, 360 video: Reprise the 2D flipbook assignment in 360 video/photo spheres.
Read: Joel Beeson’s Inside the Empathy Machine; VR, Neuroscience, Race, Journalism
Check out: Bridging Selma
Write: a draft short pitch for your group immersive project, considering what stories would lend themselves best to this format.
Wk 04, Sep 21: VR UX
- Review 360 videos (60 min)
- Pitch different story ideas (30 min)
- In class workshop (45 min) Proto 3, VR / UX concept, utilizing first person perspectives Design: a scene for virtual reality that will operate in real time, except instead of a game engine, you will have fellow classmates serve as stagehands to operate the interactions.
Assignment for week 5
Prototype 3: VR UX prototype
Write: research & revise your 1 minute final project pitch
Read: Melina Gill’s Gabo Arora on Making VR With Vrse.Works & the UN, 2016, consider issues of invitation, appropriation, colonialism, staging
RESEARCH & TECHNICAL forms (3 weeks)
Researching event
Visual narrative prototype (2D)
Performance prototype in real space
Technical workshops
Wk 05, Sep 28: Final project pitches, 3D storyboard
- Present: VR / UX prototypes; presentation in 10 min slots (60 min)
- Final project pitches, present in 5 min slots; strategize how to refine and storyboard (40 min)
- Exercise: How to storyboard in 3D (30 min)
Assignment for week 6:
Storyboard: final project proposal groups should storyboard their respective pitches for a 5 minute presentation in class next week, when we will choose 1-2 final ideas / stories to pursue.
Wk 06, Class 6, Oct 05: Choose final project concept, Unity 1
- Review 5 pitches (5 minutes each); blind voting in rounds, choose 1-2 (60 min)
- Exercise: What different approaches would you take to the same story? 360 Video; Augmented Reality; CG VR; Transmedia; Video Game; Immersive Theatre (15)
- Unity, hands on training part 1 ** students require laptops (60 min)
Assignment for week 7, Midterms
- Final Project, prototype 1: Each final project group should create 2 aesthetic prototypes concerning some aspect of your concept, to be presented in a 10 minute demo / presentation.
Example formats: 360 short video; CG VR Experience in Unity; presentation describing tangible proof of digital production. - Group MOU: each final project group should collectively write, agree to and sign a Memorandum of Understanding that describes your group’s goals, individual roles, communication, decision making process, and timeline.
OCT 12 No Class; Yom Kippur
Wk 07, Oct 19: MIDTERM review
Each final project group presents two aesthetic prototypes; and outlines group goals, individual roles and tentative time line from their MOU. 10 minute presentations.
Read for wk 8, Vorwald and Wolff, How to Write a Short Story, “Developing Your Story Idea,” 2008
MAKING: 6 weeks
Prototype, iterate, present, document
Wk 08, Oct 26, Unity 2
- Groups present updated visual prototypes (45 min)
- Intro to 3D Scanning, Matterport, photogrammetry (30 min)
- Unity, working with photo textures & video (30)
- Trouble shooting (30)
Read, for wk 9: MIT Open Lab’s Stereoscopy to Condition One
Wk 09, Nov 02
Groups present prototype with revised aesthetics, proposed UI
Wk 10, Nov 09
Present Final Project Prototype 2: layout story considering UI/UX
Wk 11: Nov 16: Work in class
Wk 12, Nov 22 (Tuesday class!)
Present Final Project Prototype 3: user experience, interaction prototype
THANKSGIVING BREAK
Wk 13, Nov 30: Troubleshooting, on site work
Wk 14, Dec 07: Final User Test
Criteria for evaluating final work: Students will be assessed on ideation, design development, UI / UX, on-boarding experience, storytelling, aesthetics, arc of experience and technical production for immersive experience. Please keep in mind:
— The on-boarding experience should be as self-contained as possible
Users shouldn’t need to be prompted, design for people experiencing the work w/out the creator present
Is the first prompt easy and clear enough?
— Story, what is going on should be clear
Provide clear feedback
This might mean using text overlays, or having proper cues
Design for the experience to be guided but as exploratory as possible
Good to have some kind of conclusion to know when the piece is over
Good to have a title screen
— UI
Design with consistent clues to encourage people walk, and to know that something is live
— Be hardware ready
Be prepared with extra equipment: external power sources, google cardboards, over the ear head phones
Make sure controllers are working smoothly
Load builds on as many iPhones as possible and / or 2nd laptops
Wk 15, Dec 14: FINAL PRESENTATION, guest critic
Due: Final Deliverables:
URL for downloadable video play-through of project experience:
1-2 paragraph project description with 1-2 images in pdf format.