Building blocks for your AI use case

Translate any video or audio content with natural sounding translations and voices.

Smart, automatic cropping of a video to a given aspect ratio based on subject detection and speaker tracking.

High-quality background removal for images and videos.

A comprehensive solution for video lipsyncing with a suite of different model and enhancements options.

State-of-the-art audio-visual active speaker detection based on new, efficent face and speaker detection models.

Fast, high quality speech transcription with many available backends, word-level timestamps, speaker diarization, and translation capabilities.

Filters for removing background noise, enhancing speech, and more in audio files.

Correct eye contact in videos by redirecting the eyes to look at the camera.

Generate and render video or audio highlights for long-form content based on search phrases.

Given a video or audio, generate a title, chapters, summary, tags, and highlights.

A set of text-to-speech models and tooling that helps generate natural-sounding speech, clone voices, control emotions, access word timestamps, and more.

Moderate videos and images for harmful content.

LivePortrait is a video-driven portrait animation system that can animate a portrait video using another driving video. It can also retarget facial expressions from one image to another.

An active speaker detection model to detect which people are speaking in a video.

A highly customizable text moderation tool that combines AI and algorithmic methods to detect and manage harmful, inappropriate, or unwanted content in real-time.

Reliable customizable text translation supporting 200+ languages through complex tokenization and sentence splitting.

YOLOv8 real-time object detection model with COCO, face, and world variants.

Generate depth maps from images or videos.

High-quality speech recognition using major improvements on top of Whisper

YouTube downloader, download videos, audios, subtitles, and metadata at scale.

Detect scene transitions in a video

Generate a portrait avatar from a source image and driving audio with multiple backends and enhancement options.

A visual language foundation model that can perform a variety of image and video question-answer tasks, such as object detection, image captioning, segmentation, and OCR.

A comprehensive visual question answering app that integrates image and video analysis with text-based queries to provide accurate, structured, and context-aware responses.

This is an optimized implementation of Segment Anything 2, a model that can dynamically segment objects in an image or video.

MuseTalk is a lip-sync model that generates realistic talking faces from audio input.

A diffusion-based audio-driven portrait animation model

CodeFormer is a face restoration model that can restore low-resolution faces to high-resolution faces.

Demucs is a state-of-the-art music source separation model, currently capable of separating drums, bass, and vocals from the rest of the accompaniment.

An optimized version of VideoReTalking, an audio-based lip synchronization model for talking head video editing in the wild.

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

Detect faces in image and video with MediaPipe.

Diarize audio using pyannote-audio

Resemble Enhance is an AI-powered tool that aims to improve the overall quality of speech by performing denoising and enhancement