Skip to content
Patchnotes

Speech models, image tools and video editing

Issue 63 min read

New open speech-to-text and voice models, image models for layouts and vector art, and tools that edit video for you.

In this issue

Voice assistants

Block’s Dessa is a voice AI trained directly on human conversation. It uses no text and no pretrained language model. It listens and talks at the same time, and it responds to emotion and pauses.

Because it learns from audio instead of text, it can use the tone and timing that a transcript drops. Dessa is not open to everyone yet. There is a waitlist.

ChatGPT Voice has a new Finances integration for talking to ChatGPT about your money.

Transcription and live translation

  • NVIDIA Nemotron 3.5 Transcribe. Open-weight speech-to-text in 40 languages. It handles recordings up to an hour long and adds translation, speaker labels and timestamps.
  • Tencent Hy-SimulMT. Translates spoken Chinese and English while the speaker is still talking. It outputs text and speech, and the speech keeps the speaker’s voice.
  • Underdog Bark 3.0. English speech-to-text on a Mac with a 163 MB model, for private meeting notes and dictation. The open weights also work outside the Underdog app.

Speech generation

  • Ant Group Ming-3.0-TTS. Downloadable speech models with voice cloning and control over how lines are performed. They cover seven languages, design voices from text descriptions and produce sounds like laughs and sighs. A small 0.5B version is available.
  • Audio8-TTS-Realistic. A compact voice model with more expressive speech in 23 languages. Its demos compare cloned voices across systems.
  • Paradee 1.2. Now has four English voices, adding American and British ones, in the same 8.5 MB model. The previous voice keeps its quality.

Image models

  • Canva Craft. Canva’s own image model, built for designs with readable text and arranged layouts. It outputs up to 2K and is available in Dream Lab and through a developer API.
  • Bria Fibo 2.0 Dev. Image generation and editing in one downloadable model, with precise edits, masks and multiple references. Bria says it trained the model only on licensed data. It is free for noncommercial use, and commercial use needs an agreement with Bria.
  • Ideogram Vector. Makes editable SVG artwork from prompts or images. The artwork scales without blur, and you can edit shapes and colors in a vector editor.

Making and cleaning up video

  • SenseNova 6.8 Code2Video. Makes short videos by writing and running code. That gives you direct control over text, timing and motion. The weights and the workflow are available.
  • MatAnyone-E. Separates people from video backgrounds and keeps fine detail like hair. In the researchers’ tests it ran at about 28.6 fps on native 4K video on an H200.
  • Open-H3-Regenerate-2K. From NVIDIA’s Sana team. It redraws low-resolution AI video at higher quality, up to 4K, for clips of 5 to 15 seconds.

Automatic video editing

  • Kino. Edit video with your agents.
  • Archon. A video foundation model that turns raw footage into a finished edit in minutes. It works zero-shot, which means it needs no extra training for the task.

The short version

  • Small speech models run on your own machine. Paradee fits four voices in 8.5 MB, and Bark 3.0 transcribes on a Mac with a 163 MB model.
  • Two image tools target design work over photos. Canva Craft focuses on readable text and layouts, and Ideogram Vector outputs editable SVG.
  • Video tools now cover generation from code, background removal, upscaling and full edits from raw footage.

Topics in this issue