GSOC Panda3D: Project Proposal: Native Video Loading

Published

March 5, 2019

Panda3D: More options for native video loading

Problem Description

Panda3D’s current video solution FFmpeg while reliable limits users to only one choice for their video needs. FFmpeg uses a restrictive LGPL license which limits the freedom of the users and contributors, and the large dependency makes it quite hard to maintain. The proposed solution to the problem is to implement an alternative native video loader for Panda3D to give the user the ability to switch their video loader.

My proposed solution is to implement WebM’s libvpx as an option into Panda3D to load files, implement threading support for libvpx as done for FFmpeg already, and provide tests and documentation for the implemented video loader.

Project Goals

  • Implement WebM video loader into the existing Panda3D codebase
  • Implement threading support for WebM
  • Write up documentation, add new tests, and clean up the code

Stretch goals

These are the goals that are more ambitious, that I might work on, but do not constitute as the primary goals for the project as presented by this proposal.

  • Implement libtheora as a third option for video loading
  • Add threading support to the aforementioned libtheora
  • Provide documentation for the implemented libtheora library

Project Implementation Plan

  • Get more familiar with libvpx the literature on the subject and the codebase before the project officially starts.
  • Implement stubs for the classes required to implement libvpx to make it return blank output.
  • Get libvpx to be able to decode the initial frame of a test file
  • Get libvpx to decode the frames with a test file properly.
  • Get libvpx to start working with multiple .webm videos
  • Test and implement error handling features
  • Interface libvpx with existing audio libraries such as libopus and libvorbis
  • Add seeking functionality
  • Add in documentation for the integrated libvpx
  • Work on adding threading support11

Project Timeline

  • Prior to June 1

    • Reading up documentation for libvpx and learn more about video compression and codecs.

    • Gain more familiarity with Panda3D’s codebase

    • Write more tests for the existing Panda3D codebase

    • Relieve off any GSoC independent time commitments

    • Start working on Video Loader Implementation

  • June 1 - June 10

    • Implement a method stub for WebM loader, and lay groundwork for future work.

  • June 10 - June 24

    • Integrate library for loading Matroska files (possibly using libwebm)

    • Integrate libvpx for video decoding

    • Get the implementation to be able to decode the first frame of an example video.

  • June 24 - July 1

    • Get libvpx to decode rest of the frames of the example video.

  • July 1 - 3

    • Making sure the crafted implementation works as expected.

    • Deal with any errors that might arise, for example, corrupted files, changed extension names etc.

    • Turn in Phase 1 evaluation

  • July 3 - July 6

    • Read up literature on threading and to formulate a way to implement threading into the newly integrated libvpx

  • July 6 - July 21

    • Implement threading support for libvpx in Panda3D as is done in the ffmpeg implementation.

  • July 21 - August 3

    • Turn in Phase 2 Evaluations

  • August 6 - August 10

    • Review over the code base and write unit tests and integration tests in order to ensure everything is working as expected

    • Start writing documentation and cleaning up.

  • August 10 - August 24

    • Buffer time for writing tests, and adding features that are not in the scope of this proposal.

Existing Contributions to Panda3D

  • PR #663 [MERGED]: Added unit tests for audio and video functionality of Panda3D. Added tests for Added test for audio_length, audio_rate, audio seek, audio seek fast, audio channels and many more. More information about this PR can be found in PR #612

  • PR #595 [MERGED]: Update Header File Names to fit Panda3D’s naming convention.

  • PR #592 [MERGED]: Similar to PR #595

  • PR #596 [MERGED]: Tested OpenAL’s functionality on MacOS, and whether it actually needed FMOD by testing audio using the Media Sampler Program

  • PR #623: Expanding on the work done by @hitesh4 in PR #593 . Added a new test for testing setting modes for Clock Objects

  • PR #624: To remove deprecated methods, once a depreciation policy is decided upon.

Benefits to the community

This project will allow users to switch out their native video loader from FFmpeg to WebM’s libvpx if they choose. This helps the users to not be limited to only FFmpeg to satisfy their video needs and helps them get some of the freedom they experience when choosing an video loader for Panda3D. FFMpeg’s large codebase provides a lot of functionality that the user doesn’t often need for their games, and this causes a large overhead, which is something integration of libvpx aims to reduce.

Time Constraints

I plan to spend up to 40 hours a week working on this project. I might have to travel a bit in the early, or late stages of the project, but due to COVID-19 pandemic, it’d be hard to tell the exact dates on when this might occur. Aside from that I have no other commitments planned that might occupy my time.

About me

I’m a sophomore (junior by credit hours) studying Computer Science at Texas Tech University. I used to go by Dr.Euler on Github, but I go by FReeshabh now. I still go by Dr.Euler (or oiler) on Discord, and can be contacted there, or by email rishabhtewari1@gmail.com.

I mostly run my systems on Arch Linux these days, although I have access to MacOS, and Windows machine if the need arises. I’d either be available in the CDT Timezone, or the IST timezone, but due to the pandemic, I’m unable to say which one currently.