Add 3D Face Effects Twilio Video Processor and Menu

[eludadev]

Contributing to Twilio

All third party contributors acknowledge that any contributions they provide will be made under the same open source license that the open source project is provided under.

• I acknowledge that all my contributions will be made under the project's license.

Pull Request Details

Description

This PR adds native support for 3D face mask effects to this app. User can toggle a selection menu (like background effects) to pick from a list of face masks, and apply the chosen mask to cover their face. It uses WebGL for rendering the mask texture into the user's predicted face geometry.

The model used is TensorFlow's Face Landmarks model. The WebGL code is inspired by Yoshiteru’s implementation, which is under an MIT license and all credits were given. Their code was modified to take advantage of TypeScript and OOP.

Burndown

Before review

• Updated CHANGELOG.md if necessary
• Added unit tests if necessary
• Updated affected documentation
• Verified locally with npm test
• Manually sanity tested running locally
• Included screenshot as PR comment (if needed)
• Ready for review

Before merge

• Got one or more +1s
• Re-tested if necessary

[eludadev]

screencast-_00001.mp4