Sieve: The cloud for video & audio AI

The cloud for

video & audio

Leading product teams use Sieve's APIs, tools, and infrastructure to ship AI-powered capabilities faster, together.

Deploy custom appsAccelerate your R&D

Empower your team of developers, designers, and PMs to build custom AI apps with full ownership & control.

Many ready-to-use models and apps

State-of-the art models in just a few lines of code, and a curated set of production-ready apps for video and audio.

Featured apps

video_transcript_analyzer

public

Given a video or audio, generate a title, chapters, summary and tags

Featured

functionbuilt 9d ago

video_retalking

public

Sync lips in a video to any audio

Featured

functionbuilt 3d ago

audio_enhancement

public

Remove background noise from audio and upsample to 48kHz.

Featured

functionbuilt a month ago

dis_background_remover

public

Remove background from image and video

Featured

functionbuilt a month ago

text_to_video_lipsync

public

Generate a video of a person speaking a given text using lipsyncing.

Featured

functionbuilt 25d ago

live_speech_transcriber

public

Transcribe and translate live audio streams in real-time with high accuracy.

Featured

functionbuilt 9d ago

import sieve

audio = sieve.Audio(path="noisy_audio.wav")
enhancer = sieve.functions.get("sieve/audio_enhancement")
cleaned = enhancer.run(audio)

import sieve

audio = sieve.Audio(path="noisy_audio.wav")
enhancer = sieve.functions.get("sieve/audio_enhancement")
cleaned = enhancer.run(audio)

Build complex AI apps

What if you could import your favorite models like they were utilities? With Sieve, you can combine many models together in just a few lines of code.

import sieve

@sieve.function(name="video_dubbing", system_packages=["ffmpeg"])
def video_dubbing(source_video: sieve.Video, language: str):
  transcriber = sieve.function.get("sieve/speech_transcriber")
  translator = sieve.function.get("sieve/seamless_text2text")
  tts = sieve.function.get("sieve/xtts-v1")
  lipsyncer = sieve.function.get("sieve/video_retalking")

  # Extract audio from video
  import subprocess
  audio_path = 'temp.wav'
  subprocess.run(
    ["ffmpeg", "-i", source_video.path, audio_path, "-y"]
  )
  
  # transcribe audio
  transcript = list(transcriber.run(sieve.Audio(path=audio_path)))
  text = transcript_to_text(transcript)
  
  # Translate text
  translated_text = translator.run(text, "eng", language)
  
  # Generate new audio from translated text
  target_audio = tts.run(source_audio, language, translated_text)
  
  # Combine audio and video with Retalker
  return lipsyncer.run(source_video, target_audio)

import sieve

@sieve.function(name="video_dubbing", system_packages=["ffmpeg"])
def video_dubbing(source_video: sieve.Video, language: str):
  transcriber = sieve.function.get("sieve/speech_transcriber")
  translator = sieve.function.get("sieve/seamless_text2text")
  tts = sieve.function.get("sieve/xtts-v1")
  lipsyncer = sieve.function.get("sieve/video_retalking")

  # Extract audio from video
  import subprocess
  audio_path = 'temp.wav'
  subprocess.run(
    ["ffmpeg", "-i", source_video.path, audio_path, "-y"]
  )
  
  # transcribe audio
  transcript = list(transcriber.run(sieve.Audio(path=audio_path)))
  text = transcript_to_text(transcript)
  
  # Translate text
  translated_text = translator.run(text, "eng", language)
  
  # Generate new audio from translated text
  target_audio = tts.run(source_audio, language, translated_text)
  
  # Combine audio and video with Retalker
  return lipsyncer.run(source_video, target_audio)

Collaborative playgrounds

Each app comes with an auto-generated playground to share with your team. PMs and designers can experiment just as easily as engineers.

Deploy custom models with ease

Define your dependencies and compute type in-code, and deploy with a single command.

import sieve

@sieve.function(
  name="my_model",
  gpu=True,
  python_packages=["torch==1.8.1"]
)
def my_model(video: sieve.Video):
  # do stuff

import sieve

@sieve.function(
  name="my_model",
  gpu=True,
  python_packages=["torch==1.8.1"]
)
def my_model(video: sieve.Video):
  # do stuff

sieve deploy

sieve deploy

Infrastructure that just works

Fast, scalable infrastructure without the hassle. No AWS account required.

Run at any scale

We built Sieve to automatically scale as your traffic increases with zero extra configuration.

Stop worrying about Docker, CUDA, and GPUs

Package models with a simple Python decorator and deploy instantly.

Logs and metrics

A full-featured observability stack so you have full visibility of what’s happening under the hood.

Flexible, compute-based pricing

Pay only for what you use, by the second. Gain full control over your costs.

Built for your use case

Work with the experts

We're a team of machine learning experts from places like Berkeley, NVIDIA, Apple, Snapchat, Microsoft, and Scale AI. Work with us to support your custom use case and integrate Sieve into your existing pipeline.