A low-cost, virtual reality, 360 camera system along with a mobile web 360 player application, forms an end-to-end solution; from capture to viewing.
Designs based around using off-the-shelf hardware/software. Parameterized 3D model files allow for easy generation of camera mounts to fit most small, rectangular cameras like GoPro, SJCAM, Xiaomi and NoPro. Parameterization also allows scalability to reach 4K resolution and beyond. The example SJ4000 rig costs less than $500 USD to build and achieves 4K resolution.
Video stitching methods include using accessible software and custom scripts provided by this repository.
Project started Spring break 2015 with some good friends and further developed through a Computational Photography class, CS 534. Hardware resources/funding by Garage Physics at University of Wisconsin, Madison.
4K 60fps 8x Xiaomi Yi ITE Building
4K 30fps 7x SJ4000 Vibration Test
4K 30fps 6x SJ4000 Madison Capitol
360 video is an exciting medium that was recently supported by video sharing sites like YouTube, Facebook and Vimeo. Unfortunately, content creation has remained limited due the high cost-of-entry in acquiring 360 cameras and video stitching software. Professional solutions range from thousands to tens of thousands of dollars and has low accessibility. Cost-friendly, consumer 360 cameras exist on the market but are limited to sub 4K resolutions, which suffer from pixelation in 360 playback.
This repository aims to provide a high-quality solution while being accessible to 360 filmmakers on a budget. The optimal target cost for a full 360 video solution is under or around $500.
Additionally, concensus in the creative community is that a different approach from traditional filmmaking techniques must be used in developing meaningful 360 video. This is because applying panning motions and camera dollys like in traditional film may cause excessive disorientation in 360 video. Another challenge is that users may look in directions that are not intended for by the director/news-anchor.
Increasing accessibility of 360 cameras will allow more creative individuals to contemplate the artistic representation and navigation issues of 360 video. It is only beneficial to allow more people to address the challenge which can improve the collective understanding of art, filmmaking and journalism.
/360warper - A standalone image/video warper to be used with video compositing tools like AFX, Blender, Nuke
/cam - 3D printable 360 camera rig files (.stl), generator files (.scad) and camera control commands
/cpp - Automated video stitching (in development, see video tutorial below for manual solution)
/img - Documentation, reference and source images
/viewer - Basic VR 360 video/image player for web browsers
Version 2 left, pictured without upper mount. Version 3 on right with optional top camera expansion module.
Captures monoscopic video in 360 degrees horizontal and ~135 degrees vertical. Six individual streams are stitched and blended together into a cohesive panoramic video.
6 SJ4000 cameras arranged outwards is just barely enough information to create a 360 video and requires a decent amount of user input to stitch in After Effects. Using 5 SJ4000 cameras seated horizontally would help automated stitching a lot at the cost of some vertical field of view. Using more cameras is ideal, but drives up cost and requires a more powerful computer for stitching. Modifying lenses of the cameras with wider angle replacements is also a solution. If planning to build with out-of-box SJ4000s oriented vertically, using 8 cameras is a safe option, 7 will be bare minimum.
Listed below are items I have tested but any of them can be swapped out with other items that serve a similar purpose. Search 3DHubs to find access to a nearby 3D printer. Some libraries have printers as well. Or build your own RepRap as a support project.
a. 6 or more SJ4000 action cameras (~$60 each), Alternatives include GoPro, Xiaomi (may need to modify 3D models). Can use less cameras by replacing stock lenses with wide angle lenses for Xiaomi and SJ4000.
b. A lower bracket/apparatus to seat cameras. Modify existing CAD designs in /cam directory to meet camera choice.
c. A matching upper apparatus to secure cameras. Likewise, modify existing CAD designs in /cam directory to meet camera choice.
d. (Optional) Zenith module for housing a single additional zenith camera i.e. "Sky Camera."
e. Standard Tripod
f. Standard computer, relatively powerful GPU for image processing preferred
g. Any mobile device with a modern web browser
h. A headmounted display to house mobile device for VR viewing like Cardboard or print one from Thingiverse.
h1 - 1x 1/4"-20 bolt for securing top mount to bottom mount. For SJ4000s, length of bolt must be > 3". May be different for other camera selections. Ideally, choose a length that allows bolt to stick past bottom mount an inch or two to connect to tripod with coupling bolt 'h7.'
h2 - 2x Washers for 1/4"-20 bolt.
h3 - 1x 1/4"-20 nut for tightening under bottom mount.
h4 - h6 only applies if using top camera module
h4 - 2x 6-32 screws (> 1.5" length) to mount top camera expansion module.
h5 - 4x #6 washers.
h6 - 2x 6-32 nuts.
h7 - 1x 1/4"-20 coupling nut for joining entire setup to tripod head.
(Optional) Switch to gen-lock/sync record controls for cameras, necessary to mitigate rolling shutter when capturing scenes with fast motion, also provides convenience for starting/stopping recording, requires soldering
- OpenSCAD for rapid, parameterized prototyping of 3D printable VR camera mounts (Free)
- After Effects CC for basic stitching using warp plugins (Free trial, then monthly subscription)
- (Optional) Stitching plugins for After Effects like Skybox Mettle, PTGui, Kolor or VideoStitch for professional stitching results (> $90)
- Blender for alternative stitching method to After Effects, also includes capabilites to composite text and 3D. (Free)
- YouTube 360 Injector for tagging proper metadata to final 360 video for properly uploading to YouTube (Free)
Video Tutorial: Hand-stitching in After Effects (Quick)
Video Tutorial: Hand-stitching (Detailed using 360warper script)
or
For fully automated process,compile and run stitcher in /cpp (still in the works)
Stitcher uses ORB feature descriptor pairs to register overlapping images. ORB is also free from patent restrictions.
Web VR 360 Player - Demo Link
Visit `/viewer` directory for usage and source code.
Print camera mount yourself or find a local printer through 3DHubs or the local library.
Older version of mono_lower_6x_sj4000.stl printed on a Makerbot Replicator 2 with Red PLA. The mount takes about 4 hours to print if things go smoothly! Newer version of mount printed on an open-source Prusa i3 design, right, with a layer height of 0.27mm through a 0.4mm extruder nozzle.
Files are located in /cam directory, organized by camera model. To create a new camera mount, open the closest existing .scad file and adapt it by changing the camera trench dimensions. Camera dimensions are listed as variables at the top of the .scad files and should globally change the trench sizes across the mount. To add more cameras, simply increase the distance of the optical center and make additional copies of the trenches and assign each trench with the appropriate angle. The numbers are technically unitless but they default to millimeter for most printers. Be sure to add 1 to the value used for the camera dimension for a little breathing room.
- Create automated video stitching (Implement feature detector)
Top/bottom camera transform for AFX stitch method using Blender or scriptMake video tutorial for hand-stitching 360 videos in AFX