CMS.339 / CMS.839

Virtual Reality and Immersive Media Production

HACKING XR

MORE ABOUT THE COURSE

Mens et manus ... mind and hand ... VR and Immersive Media Production will engage both. What better way to explore a new technology than to push it to its limits, re-imagine it, and to rebuild it in unexpected ways?

Offered as part of a relationship between MIT's Comparative Media Studies / Writing and in partnership with the Oculus NextGen Program and Unity, Hacking XR will enable students to experiment with VR/AR technologies and current developments - including Oculus Rift, touch controllers, 360 cameras and new this year, Mixed Reality headsets. The course will survey historical developments and present day innovation in virtual reality and immersive media - gear, software, and emerging storytelling techniques, languages and approaches. It will cover essential steps in immersive media production - from conception, storyboard, UX/UI and human-centered design, sound and story.

Finally, the class invites leading makers of the industry, to explore with them the fundamentals of immersive creation, and with luck, develop innovative ways to build immersive experiences. Join us!

Meets Mondays 7-10 pm

MORE ABOUT YOU

Pioneers, pirates and makers

Hacking XR is all about pushing the boundaries of immersive media production. We believe in fostering a creative and collaborative experience. Students from various disciplines are encouraged to enroll. We’re looking for a diversity of skills and backgrounds – coding, game design, programming experience but also storytelling and creative wildcards – to build complementary teams that will produce innovative and original VR experiences. Tell us more about yourself - fill in the questionnaire!

Fill in the Questionnaire

MEET THE INSTRUCTIONAL TEAM

SANDRA RODRIGUEZ

Instructor/Lecturer

Sandra Rodriguez, PhD., is a creative director (interactive/film) and sociologist of new media technology. A Lecturer/Visiting Scholar at CMS (Fellow of MIT OpenDocLab) she also heads a Creative Reality Lab at EyeSteelFilm, and Emmy Awarded company based in Montreal, where she explores the future of immersive documentary.

sanrodri@mit.edu

Emily Salvador is a MAS student at the Media Lab focusing on immersive storytelling and experiences. An MIT grad, she took a gap year to work for Walt Disney Imagineering and Universal Creative designing interactive technology for theme parks.

emilysa@mit.edu

hours - TBD

EMILY SALVADOR

Teacher Assistant

FAQS

IS PREVIOUS EXPERIENCE WITH VIRTUAL REALITY REQUIRED FOR THIS COURSE?

No, the course will move quickly from the basics of understanding VR production and workflow, and will encourage participants to go as far as possible during the semester. Projects will be completed by teams of 3 students, with each team having members with programming, design, and creative skills, so experience in any of these areas is far more important.

HOW DO I APPLY FOR THE COURSE?

This is the first course of its kind at MIT. Well, actually the second. Last year's iteration was listed as a ‘special topics’ course, but this year it is a fully regular part of the MIT curriculum.

IS THIS A CRITICAL COURSE OR A HANDS-ON PRODUCTION COURSE?

Both. The course will use hands on experimentation to push up against the hype surrounding VR, but also AR and MR. It will use critical and historical approaches to interrogate some of the key default assumptions about VR, while probing to find unexplored alternatives and under-considered consequences. What can we learn from current immersive media projects, and can VR/AR/XR enable new kinds of stories to be told? What kind of cognitive processes underlie immersion? Are there particular ethical considerations that this medium requires? What can touch controllers add to stories and games? Theory and practice will work together in the course, and the two will hopefully inform the semester-long development of VR/AR and MR projects.

DOES THE COURSE FULFILL ANY HASS ELECTIVES?

The course will count as a 12-unit general HASS-A.

WHAT IS THE COURSE’S RELATIONSHIP TO OCULUS OR OTHER COMPANIES?

Thanks to a special relationship with MIT's CMS/W and the Open Documentary Lab and the Oculus NextGen Program, companies like Oculus, Samsung, Unity and HP provide equipment and services to support the course. Our job is to push these technologies and their storytelling capacities tand to come up with fresh ideas. All intellectual property produced in the course will belong to the students who produce it, as per standard MIT policy. MIT is one of 11 campuses working with Oculus NextGen.

WHAT IS YOUR POLICY ON AUDITING SPECIFIC CLASSES WHILE NOT ENROLLED IN THE COURSE?

The course focuses on team production of VR or AR/MR experiences. Class attendance by all members of a team is required to push the projects gradually, and will be evaluated as part of participation. Auditing will not be permitted in regular classes. However, the course will have multiple guest-lectures and special workshops which will be opened to wider audiences, held the next day (Tuesday) at lunch. Stay tuned by following our blog!

HOW IS THIS CLASS PART OF AN OCULUS NEXT GEN NETWORK?

Beyond the course, there will be several ways to communicate and network with other Oculus NextGen participants in universities across the US :

A Facebook group to which you will be invited to register, contribute, post, share, comment, etc
Regular skype/in-person lectures and check-ins from Oculus chosen picks, artists and staff

WHAT'S AHEAD?

INTRODUCTION TO IMMERSION

FEBRUARY 12, 2018

Introduction to course mechanics, definitions and the distinctions among various VR/AR/MR systems. This session will include a historical overview of immersive media; lessons learned from earlier media hypes; and an assessment of the current ecosystem. We will get acquainted with the lab, Oculus headsets and controllers, computers and schedules; explain assignments and the Kino method.

Avatar of Jo Yardley, immersive media maker #secondlife

SO YOU HAVE AN IDEA...From ideation, conception and production

FEBRUARY 20, 2018 (TUESDAY!!)

How does one construct a VR experience? What makes an immersive experience "good"? What makes it stand out? We’ll explore some of the fundamental mechanics (visual, sensory, depth, sound and story!) that allow or impede presence and immersion. From the “Swayze effect” to simulator sickness, interactive or passive VR, this session offers important learning steps in creating immersive content - from ideation, conceptualization, storyboarding, to production. Get your bases covered!

WHOSE STORY? From empathy machines to human narratives

FEBRUARY 26, 2018

Should we create immersive experience that look or that feel real? After taking up issues such as user attention, interaction, and transitions, it is paramount to also think about what makes an immersive experience real, relevant or felt. Thinking about space, time, interaction and transitions are important. But what about empathy? Accessibility? Representation? As immersive media expands, diversity is key to create compelling, meaningful and thriving ecosystem of experiences.

BUILDING IMMERSIVE WORLDS IN UNITY

MARCH 5, 2018

Building worlds - from pros to beginners, you’ll need to be able to communicate in world building terms. Team projects will rely heavily on team work. And knowing of Unity is a very useful skill. This workshop will walk us through the essential steps into creating immersive worlds, spaces, light and character in Unity. The workshop is provided by Unity. Participation is mandatory.

PRODUCTION 1: MAKE IT HAPPEN!

Image capture, tinkering, prototyping

MARCH 12, 2018

There’s never a time like now to start your projects. Will you choose 360 live capture techniques? Real time capture systems such as laser scanning (LIDAR), photogrammetry, Kinekt can also offer significant possibilities. Hands-on session on examples of production and post-production methods. With 2 guest mentors pushing VR and AR, hackathon mode!

PRODUCTION 2- Sound, Interactivity, Assets

MARCH 19, 2018

Creating worlds digitally. Imagination, interactivity and coding. Some say sound is 50% of immersion in VR but also AR and XR. How does it translate to new platforms? How should we think of music, audio, space and texture in immersive media productions? Learn about grounding your 360 environments, plus imagination, interactivity and haptic feedback, from the studio behind some of the most acclaimed VR experiences today.

IMMERSIVE APPLICATIONS:

Sensory Storytelling - Inspiration from other medium

APRIL 2, 2018

What does it mean to be immersed in the “real”? What can the documentary art form tell us about real world, people and situations representations? How can we treat real-life issues and topics with interactive media? How do non-fiction ethics apply within emerging medium? What are artists’ responsibilities when creating immersive digital experiences? Exploring new venues in interactive, immersive, storytelling.

IMMERSIVE APPLICATIONS: EXTENDING REALITY

APRIL 9, 2018

Rethinking the rules of immersive storytelling. What else could we create? 3D printing, painting and sculpting in VR. What about scanning the real? What about touch? Smell? Taste? What can we learn from filmmakers, game designers, magicians and architects? What if you could see through the eyes of an insect? What if you could be a tree? What if you feel, touch, smell in immersive experiences?

IMMERSIVE APPLICATIONS: UX, HUMAN EXPERIENCE & DESIGN

APRIL 23, 2018

Game culture and design have long been thinking about user experience, interaction and emergent narratives. How can interactive UX/UI help create compelling immersive projects? What does VR add to gameplay? What are current developments in Social VR, collective experiences and installations?

IMMERSIVE APPLICATIONS: VR IN SCIENCE AND EDUCATION

APRIL 30, 2018

Is VR just a tool for entertainment? What does neuroscience teach us about virtual spaces and perception? And how could it be applied in health, in physics, in science? What kind of VR/AR/MR experiences do we want to produce and to what purpose?

IMPORTANT QUESTIONS FOR THE FUTURE - NEW DISTRIBUTION MODELS

MAY 7, 2018

VR is still in its infancy, but it’s taking flight quickly. What should pioneers think of when exploring, producing and distributing this new medium? This year has been a game changer in distribution of immersive content. From the popularization of VR cinemas and the creation of new distribution models, discover the development of new, quickly popularizing immersive distribution approaches, challenges and successes.

FINAL PROJECT PRESENTATIONS AND CRITIQUES

MAY 14, 2018

Projects will be presented and discussed in class. We can't wait to see them!

WORTH READING....

WORTH VIEWING....

TECHNOLOGY WORTH CONSIDERING...

Keep up with our latest news, activities, and events by checking the course blog.

A CRITICAL LOOK

reviews of various and sundry VR projects

In order to hack VR, one has to understand its emergent norms and variations. This section includes course participants' responses to the projects they have experienced