CMS.339 / CMS.839
Virtual Reality and Immersive Media Production
MORE ABOUT THE COURSE
Mens et manus ... mind and hand ... VR and Immersive Media Production will engage both. What better way to explore a new technology than to push it to its limits, re-imagine it, and to rebuild it in unexpected ways?
Offered as part of a relationship between MIT's Comparative Media Studies / Writing and in partnership with the Oculus NextGen Program and Unity, Hacking XR will enable students to experiment with VR/AR technologies and current developments - including Oculus Rift, touch controllers, 360 cameras and new this year, Mixed Reality headsets. The course will survey historical developments and present day innovation in virtual reality and immersive media - gear, software, and emerging storytelling techniques, languages and approaches. It will cover essential steps in immersive media production - from conception, storyboard, UX/UI and human-centered design, sound and story.
Finally, the class invites leading makers of the industry, to explore with them the fundamentals of immersive creation, and with luck, develop innovative ways to build immersive experiences. Join us!
Register for 12 units. HASS-A
Meets Mondays 7-10 pm
MORE ABOUT YOU
Pioneers, pirates and makers
Hacking XR is all about pushing the boundaries of immersive media production. We believe in fostering a creative and collaborative experience. Students from various disciplines are encouraged to enroll. We’re looking for a diversity of skills and backgrounds – coding, game design, programming experience but also storytelling and creative wildcards – to build complementary teams that will produce innovative and original VR experiences. Tell us more about yourself - fill in the questionnaire!
MEET THE INSTRUCTIONAL TEAM
Sandra Rodriguez, PhD., is a creative director (interactive/film) and sociologist of new media technology. A Lecturer/Visiting Scholar at CMS (Fellow of MIT OpenDocLab) she also heads a Creative Reality Lab at EyeSteelFilm, and Emmy Awarded company based in Montreal, where she explores the future of immersive documentary.
Emily Salvador is a MAS student at the Media Lab focusing on immersive storytelling and experiences. An MIT grad, she took a gap year to work for Walt Disney Imagineering and Universal Creative designing interactive technology for theme parks.
hours - TBD
IS PREVIOUS EXPERIENCE WITH VIRTUAL REALITY REQUIRED FOR THIS COURSE?
No, the course will move quickly from the basics of understanding VR production and workflow, and will encourage participants to go as far as possible during the semester. Projects will be completed by teams of 3 students, with each team having members with programming, design, and creative skills, so experience in any of these areas is far more important.
HOW DO I APPLY FOR THE COURSE?
This is the first course of its kind at MIT. Well, actually the second. Last year's iteration was listed as a ‘special topics’ course, but this year it is a fully regular part of the MIT curriculum.
IS THIS A CRITICAL COURSE OR A HANDS-ON PRODUCTION COURSE?
Both. The course will use hands on experimentation to push up against the hype surrounding VR, but also AR and MR. It will use critical and historical approaches to interrogate some of the key default assumptions about VR, while probing to find unexplored alternatives and under-considered consequences. What can we learn from current immersive media projects, and can VR/AR/XR enable new kinds of stories to be told? What kind of cognitive processes underlie immersion? Are there particular ethical considerations that this medium requires? What can touch controllers add to stories and games? Theory and practice will work together in the course, and the two will hopefully inform the semester-long development of VR/AR and MR projects.
DOES THE COURSE FULFILL ANY HASS ELECTIVES?
The course will count as a 12-unit general HASS-A.
WHAT IS THE COURSE’S RELATIONSHIP TO OCULUS OR OTHER COMPANIES?
Thanks to a special relationship with MIT's CMS/W and the Open Documentary Lab and the Oculus NextGen Program, companies like Oculus, Samsung, Unity and HP provide equipment and services to support the course. Our job is to push these technologies and their storytelling capacities tand to come up with fresh ideas. All intellectual property produced in the course will belong to the students who produce it, as per standard MIT policy. MIT is one of 11 campuses working with Oculus NextGen.
WHAT IS YOUR POLICY ON AUDITING SPECIFIC CLASSES WHILE NOT ENROLLED IN THE COURSE?
The course focuses on team production of VR or AR/MR experiences. Class attendance by all members of a team is required to push the projects gradually, and will be evaluated as part of participation. Auditing will not be permitted in regular classes. However, the course will have multiple guest-lectures and special workshops which will be opened to wider audiences, held the next day (Tuesday) at lunch. Stay tuned by following our blog!
HOW IS THIS CLASS PART OF AN OCULUS NEXT GEN NETWORK?
Beyond the course, there will be several ways to communicate and network with other Oculus NextGen participants in universities across the US :
A Facebook group to which you will be invited to register, contribute, post, share, comment, etc
Regular skype/in-person lectures and check-ins from Oculus chosen picks, artists and staff
If you have unanswered questions, please contact the instructor. Also, if you'd like to join us but aren't registered in the class, do note that we have a public and open HackingXR Speaker Series, featuring some of the class special guests (organized in collaboration with the MIT Open Documentary Lab and the MIT Center for Arts, Science and Technology).
INTRODUCTION TO IMMERSION
FEBRUARY 12, 2018
Introduction to course mechanics, definitions and the distinctions among various VR/AR/MR systems. This session will include a historical overview of immersive media; lessons learned from earlier media hypes; and an assessment of the current ecosystem. We will get acquainted with the lab, Oculus headsets and controllers, computers and schedules; explain assignments and the Kino method.
SO YOU HAVE AN IDEA...From ideation, conception and production
FEBRUARY 20, 2018 (TUESDAY!!)
How does one construct a VR experience? What makes an immersive experience "good"? What makes it stand out? We’ll explore some of the fundamental mechanics (visual, sensory, depth, sound and story!) that allow or impede presence and immersion. From the “Swayze effect” to simulator sickness, interactive or passive VR, this session offers important learning steps in creating immersive content - from ideation, conceptualization, storyboarding, to production. Get your bases covered!
WHOSE STORY? From empathy machines to human narratives
FEBRUARY 26, 2018
Should we create immersive experience that look or that feel real? After taking up issues such as user attention, interaction, and transitions, it is paramount to also think about what makes an immersive experience real, relevant or felt. Thinking about space, time, interaction and transitions are important. But what about empathy? Accessibility? Representation? As immersive media expands, diversity is key to create compelling, meaningful and thriving ecosystem of experiences.
BUILDING IMMERSIVE WORLDS IN UNITY
MARCH 5, 2018
Building worlds - from pros to beginners, you’ll need to be able to communicate in world building terms. Team projects will rely heavily on team work. And knowing of Unity is a very useful skill. This workshop will walk us through the essential steps into creating immersive worlds, spaces, light and character in Unity. The workshop is provided by Unity. Participation is mandatory.
PRODUCTION 1: MAKE IT HAPPEN!
Image capture, tinkering, prototyping
MARCH 12, 2018
There’s never a time like now to start your projects. Will you choose 360 live capture techniques? Real time capture systems such as laser scanning (LIDAR), photogrammetry, Kinekt can also offer significant possibilities. Hands-on session on examples of production and post-production methods. With 2 guest mentors pushing VR and AR, hackathon mode!
PRODUCTION 2- Sound, Interactivity, Assets
MARCH 19, 2018
Creating worlds digitally. Imagination, interactivity and coding. Some say sound is 50% of immersion in VR but also AR and XR. How does it translate to new platforms? How should we think of music, audio, space and texture in immersive media productions? Learn about grounding your 360 environments, plus imagination, interactivity and haptic feedback, from the studio behind some of the most acclaimed VR experiences today.
Sensory Storytelling - Inspiration from other medium
APRIL 2, 2018
What does it mean to be immersed in the “real”? What can the documentary art form tell us about real world, people and situations representations? How can we treat real-life issues and topics with interactive media? How do non-fiction ethics apply within emerging medium? What are artists’ responsibilities when creating immersive digital experiences? Exploring new venues in interactive, immersive, storytelling.
IMMERSIVE APPLICATIONS: EXTENDING REALITY
APRIL 9, 2018
Rethinking the rules of immersive storytelling. What else could we create? 3D printing, painting and sculpting in VR. What about scanning the real? What about touch? Smell? Taste? What can we learn from filmmakers, game designers, magicians and architects? What if you could see through the eyes of an insect? What if you could be a tree? What if you feel, touch, smell in immersive experiences?
IMMERSIVE APPLICATIONS: UX, HUMAN EXPERIENCE & DESIGN
APRIL 23, 2018
Game culture and design have long been thinking about user experience, interaction and emergent narratives. How can interactive UX/UI help create compelling immersive projects? What does VR add to gameplay? What are current developments in Social VR, collective experiences and installations?
IMMERSIVE APPLICATIONS: VR IN SCIENCE AND EDUCATION
APRIL 30, 2018
Is VR just a tool for entertainment? What does neuroscience teach us about virtual spaces and perception? And how could it be applied in health, in physics, in science? What kind of VR/AR/MR experiences do we want to produce and to what purpose?
IMPORTANT QUESTIONS FOR THE FUTURE - NEW DISTRIBUTION MODELS
MAY 7, 2018
VR is still in its infancy, but it’s taking flight quickly. What should pioneers think of when exploring, producing and distributing this new medium? This year has been a game changer in distribution of immersive content. From the popularization of VR cinemas and the creation of new distribution models, discover the development of new, quickly popularizing immersive distribution approaches, challenges and successes.
FINAL PROJECT PRESENTATIONS AND CRITIQUES
MAY 14, 2018
Projects will be presented and discussed in class. We can't wait to see them!
A CRITICAL LOOK
reviews of various and sundry VR projects
In order to hack VR, one has to understand its emergent norms and variations. This section includes course participants' responses to the projects they have experienced
FIND OUT MORE...
If you have specific questions about the course, please contact the course instructor Sandra Rodriguez.
TA: Emily Salvador.