CMS.339 / CMS.839

Virtual Reality and Immersive Media Production

HACKING XR
 
 

MORE ABOUT YOU

Pioneers, pirates and makers

Hacking XR is all about pushing the boundaries of immersive media production.  We believe in fostering a creative and collaborative experience. Students from various disciplines are encouraged to enroll. We’re looking for a diversity of skills and backgrounds – coding, game design, programming experience but also storytelling and creative wildcards – to build complementary teams that will produce innovative and original VR experiences. Tell us more about yourself - fill in the questionnaire!

 

MORE ABOUT THE COURSE

Hacking XR course (CMS.339/839 - formerly known as HackingVR) is MIT’s first regularly offered course on VR, AR, and emerging immersive technologies. A hands-on course, it involves creative production as well as critical analysis. Mens et manus ... mind and hand,VR and Immersive Media Production will engage both. What better way to explore a new technology than to push it to its limits, re-imagine it, and to rebuild it in unexpected ways?

 

The course will enable students to explore VR/AR technologies and discover current XR developments. The course will survey historical developments and present day innovation in virtual reality and immersive media - gear, software, and emerging storytelling techniques, languages and approaches. It will cover essential steps in immersive media production - from conception, storyboard, UX/UI and human-centered design, sound and story. We hack assumptions and discover unexpected affordances.

 

The course includes regular meetings with artists, technologists, curators, and leading makers of the industry. You can follow the HackingXR Speaker Series here, or check out past iterations of the class in our archives  

Join us!

 

Register for 12 units. HASS-A

Meets VIRTUALLY and SYNCHRONOUSLY

Monday 12:00-3:00 pm EST

In partnership with XR@ODL 

CMS.339 / CMS.839

Virtual Reality and Immersive Media Production

HACKING XR

MEET THE INSTRUCTIONAL TEAM

 

SANDRA RODRIGUEZ

Instructor/Lecturer

Sandra Rodriguez, PhD, is a creative director, filmmaker and a sociologist of new media technology. A Sundance Lab Fellow, her work spans from feature films, webdocs, interactive, immersive and emergent media (VR/XR/AI). She was director or producer of digital experiences (DoNotTrack, Deprogrammed_VR, Big Picture VR, Manic_VR, Chomsky vs. Chomsky: First Encounter, FutureRites, and more) that have garnered multiple awards, including a Peabody (DoNotTrack, 2016), best immersive experience (IDFA DocLab 2016; Leipzig DokNeuland 2018), and the prestigious Golden Nica Award at Ars Electronica (2019).

 

Rodriguez was founder and head of the Creative Reality Lab (an Emmy-award-winning film company based in Montreal) where she explored futures of non-fiction storytelling. She is a regular speaker on digital storytelling for high visibility events including: Cannes Film Festival, Venice Film Festival, Sundance New Frontier, IDFA, SXSW_Edu, Mutek, Singapore Film Festival, Frontières IA,  AI&Arts, Korea: NRP Demo-day, and more. 

Rodriguez most recent works: Chomsky vs. Chomsky (Sundance 2020) and Future Rites (Creative XR 2020, IDFA Forum 2020), explore the sparks that fly at the crossroads of AI, and human creativity. 

sanrodri@mit.edu

office hours - Mon. 3:30-4:30pm EST

Davide_Zhang_profile.jpg

DAVIDE ZHANG

Teaching Assistant

Davide Zhang is a MArch and MDes concurrent degree candidate at Harvard Graduate School of Design. He works on prototyping at the Responsive Environments and Artifacts Lab at Harvard GSD and previously worked on AR prototyping at Columbia University Computer Graphics and User Interfaces Lab. He is interested in designing and prototyping experiences with emerging technologies like VR/AR.

davidezhang@gsd.harvard.edu

office hours : Wed/Fri 1pm - 2:30pm EST 

P1120711_edited.jpg

PATRICIA LAI

Teaching Assistant

Patricia Lai is an experienced UX designer, and a Human-computer interaction (HCI) researcher based in Boston. She is currently pursuing a Master degree in Design Engineering (MDE) at Harvard University. Before join MDE program, Patricia worked at ASUS Design Center, where she embarked on projects ranging from interface design in smartphones and laptops to interaction design in companion robots, and HTC Vive R&D Multimedia team, where she has designed the best sensation and perception experience for using virtual reality device. 

 

Patricia’s work focuses on UX design, interaction design, service design, and human factors research. She is always passionate about applying psychology and cognitive science to create innovative and intuitive products design.

patricialai@mde.harvard.edu  / plai@mit.edu

office hours: Mon. 9am-12pm EST

Both! The course will use hands on experimentation to push up against the hype surrounding VR, but also AR and MR. It will use critical and historical approaches to interrogate some of the key default assumptions about VR, while probing to find unexplored alternatives and under-considered consequences. What can we learn from current immersive media projects, and can VR/AR/XR enable new kinds of stories to be told? What kind of cognitive processes underlie immersion? Are there particular ethical considerations that this medium requires? What can touch controllers add to stories and games? Theory and practice will work together in the course, and the two will hopefully inform the semester-long development of VR/AR and MR projects.

 

FAQS

Both! The course will use hands on experimentation to push up against the hype surrounding VR, but also AR and MR. It will use critical and historical approaches to interrogate some of the key default assumptions about VR, while probing to find unexplored alternatives and under-considered consequences. What can we learn from current immersive media projects, and can VR/AR/XR enable new kinds of stories to be told? What kind of cognitive processes underlie immersion? Are there particular ethical considerations that this medium requires? What can touch controllers add to stories and games? Theory and practice will work together in the course, and the two will hopefully inform the semester-long development of VR/AR and MR projects.

IS THIS A CRITICAL COURSE OR A HANDS-ON PRODUCTION COURSE?

IS PREVIOUS EXPERIENCE WITH VIRTUAL REALITY REQUIRED FOR THIS COURSE?

No, the course will move quickly from the basics of understanding VR production and workflow, and will encourage participants to go as far as possible during the semester. Projects will be completed by teams of 3 students, with each team having members with programming, design, and creative skills, so experience in any of these areas is far more important.

HOW DO I APPLY FOR THE COURSE?

This is the first course of its kind at MIT. It was the 1st course offered at MIT on Immersive media and VR, and launched the beginning of new curriculum focused on VR/AR offered in today's MIT curriculum. You can apply for the class like you would any CMS class - see here - and you finally need to fill in this questionnaire. Why a questionnaire? Well, since the class is limited to 18 and we want a diversity of skills and brains, we need to know you better first! 

DOES THE COURSE FULFILL ANY HASS ELECTIVES?

The course will count as a 12-unit general HASS-A.

WHAT IS THE COURSE’S RELATIONSHIP TO OCULUS OR OTHER COMPANIES?

Thanks to a special relationship with MIT's CMS/W and the Open Documentary Lab and the Oculus NextGen Program, companies like Oculus, Samsung, Unity and HP provide equipment and services to support the course. MIT was one of the first universities to be part of Oculus Next Gen in 2017. The program was left pending in 2019 with more than 20 US campuses working with Oculus NextGen. The program has been on hold since 2020. But this being said, the class aims to always remain technology agnostic - we also have access to Microsoft MR headsets, HTC and new technology soon arriving. Our job is to push these technologies and their storytelling capacities and to come up with fresh ideas. All intellectual property produced in the course will belong to the students who produce it, as per standard MIT policy.

WHAT IS YOUR POLICY ON AUDITING SPECIFIC CLASSES WHILE NOT ENROLLED IN THE COURSE?

The course focuses on team production of VR or AR/MR experiences. Class attendance by all members of a team is required to push the projects gradually, and will be evaluated as part of participation. Auditing is therefore not always best suited for regular classes. However, the course does have multiple guest-lectures by artists and leaders of the XR industry, and special workshops which will be opened to wider audiences (often held the next day - Tuesday - in collaboration with XR@ODL, MIT Open Doc Lab). Stay tuned by following our blog and the HackingXR Speaker Series here above!

HOW IS THIS CLASS PART OF AN OCULUS NEXT GEN NETWORK?

Beyond the course, there will be several ways to communicate and network with other Oculus NextGen participants in universities across the US :

  1. A Facebook group to which you will be invited to register, contribute, post, share, comment, etc

  2. Regular skype/in-person lectures and check-ins from Oculus chosen picks, artists and staff

MORE QUESTIONS?

If you have unanswered questions, please contact the instructor. Also, if you'd like to join us but aren't registered in the class, do note that we have a public and open HackingXR Speaker Seriesfeaturing some of the class special guests (organized in collaboration with the MIT Open Documentary Lab).

 

WHAT'S AHEAD?

Building worlds - from pros to beginners, you’ll need to create worlds. How does one create worlds that are simultaneously compelling, immersive and comfortable for a user? What about interaction design? Pacing? Music and tone setting? How to plan the next steps needed to push your concept into worlds that enhance “suspension of disbelief”? In this session, we’ll think of development, prototyping, and getting to work (even without a complex understanding of game engines). Walk-through tutorial of A-Frame, a popular open-source web framework for building VR experiences in web browser, based on HTML. Guest visitor: Sonny Li, Senior Curriculum Developer at Codecademy.

MARCH 15, 2021

PRODUCTION SPRINT 1: MAKE IT HAPPEN (in VR)

MARCH 29, 2021

There’s never a time like now to start on your team projects! Will you choose 360 live capture techniques? Photogrammetry? Will you choose VR or AR? Hands-on session!! With examples of varied production and post-production methods, students work in class discussion and group breakouts to push their team AR and MR explorations. Friendly team project pitches and a hands-on session on examples of production and post-production methods - hackathon style! A special guide to overcoming the "impossible", with VR artist Ali Eslami.

theblu-wevr-natural-history-museum-la3-8

SO YOU HAVE AN IDEA...From ideation, conception and production

MARCH 9, 2021 (TUESDAY!)

XR is an emergent medium. What makes an experience stand out? How are current developments pushing the medium?  Think about space, time, sound, user attention, UX/UI, interaction, transitions. What about representation of body and self in this emergent medium? Think about creating with human affordances in mind. Who is your audience? What is their access? How are they represented?

And can you fail-proof for future? An peek into XR future with Ryot/Verizon, Jake Sally.

BUILDING IMMERSIVE WORLDS

PRODUCTION SPRINT 2- MAKE IT HAPPEN! (in AR)

APRIL 5, 2021

How do new platforms and hardware resolve obstacles that break down immersion? Exploring relationship between visual, audio, and textures in immersive content. We discover exploration merging outside the box interaction and uses (or misuses) of XR tech, with ANAGRAM, a duo of XR creation that has helped the industry rethink the rules and how we define immersive storytelling.

PRODUCTION SPRINT 3 - MAKE IT HAPPEN! (with heart)

APRIL 12, 2021

It’s not about which subjects fit to immersive media, but rather how you think of what is felt, experienced, and understood through technological affordances. We’ll revisit the concept of the “empathy-machine”, with a walk-through of the making of award-winning experiences Hangman at Home and Minimum Mass (Floreal Films).

Highres.jpg

Exploring our human experience and public issues in XR. What can non-fiction art form tell us about the way we represent the world, people, our past and future? How can we treat social issues with interactive and immersive media? What are artists’ and funders responsibilities when creating immersive digital experiences?

EXTENDING REALITY : STORYTELLING THE PAST & FUTURE

APRIL 26, 2021

EXTENDING REALITY: XR & THE MUSIC INDUSTRY

MAY 3, 2021

Game culture and game design have long been thinking about user experience, interaction and emergent narratives. But recent experiences show game-inspired interactions, UX, UI and are shifting the entertainment and music industry, in particular, helped with current developments in Social XR.

Is VR just a tool for entertainment? How does interaction, innovation and magic converge to create original works? Enabling a strong sense of presence and immersion also means designing virtual worlds that can respond quickly to a user’s action. What is to become of LBE? Is at home consumption facilitated with 5G? What about developments in Artificial Intelligence and Machine Learning, when combined with immersive storytelling techniques and tools, yo help create “Intelligent Virtual Environment” (IVE)?

MAY 10, 2021

EXTENDING REALITY: A SHIFTING FUTURE

MAY 17, 2021

FINAL PROJECT PRESENTATIONS AND CRITIQUES

Projects will be presented and discussed in class. We can't wait to see them!

 

INTRODUCTION TO IMMERSION

FEBRUARY 22, 2021

Introduction to course mechanics, definitions and the distinctions among various VR/AR/MR systems and their current developments. This session will include a short historical overview of immersive media; lessons learned from earlier hypes; and an assessment of the current ecosystem. We get acquainted with the lab, the students, the equipment and schedules; we explain assignments and our collaborative & creative method.

planetarium1.gif

CREATING in XR - some guidelines

MARCH 1, 2021

How does one construct a compelling VR/AR or immersive media experience? What makes it "good"? What makes it stand out? We’ll explore some of the fundamental mechanics (visual, sensory, depth, sound and story!) that allow or impede presence and immersion. From the “Swayze effect” to simulator sickness, interactive or passive UX, this session offers important learning steps in creating immersive content - with a guided tour of XR worlds. Get your bases covered!

A CRITICAL LOOK

reviews of various and sundry VR projects

In order to hack XR, one has to understand its emergent norms and variations.  This section includes course participants' responses to the diverse projects they have experienced over the last 4 years! An amazing review of genres, styles, interaction approaches and more... follow our Blog for updated weekly project reviews.

MUSICSYNTH.jpg

FANTASYNTH

download.jpg

ECLIPSE: EDGE OF LIGHT

vacation-simulator-screen-04-ps4-us-04ma

VACATION SIMULATOR

Screen Shot 2019-04-29 at 11.52.41 AM.pn
Screen Shot 2019-04-29 at 11.37.00 AM.pn

BLOCKS

WONDERFUL YOU

static1.squarespace.png

PROSPECT

VESTIGE

Screen Shot 2019-04-29 at 9.25.48 AM.png
1_GhostShell.jpg

GHOST IN THE SHELL VR

WORTH READING....

WORTH VIEWING....

TECHNOLOGY WORTH CONSIDERING...

Keep up with our latest news, activities, and events by checking the course blog.

 

FIND OUT MORE...

If you have specific questions about the course, please contact the course lecturer Sandra Rodriguez.

E15-313