Command Palette

Search for a command to run...

UnylyUnyly
Browse all

Midasheng Gen

FreeNot checked

MCP server generating coherent 16 kHz mixed audio scenes from text - speech, music, sound effects and ambience in one pass - powered by MiDashengLM-Gen (Xiaomi

GitHubEmbed

About

MCP server generating coherent 16 kHz mixed audio scenes from text - speech, music, sound effects and ambience in one pass - powered by MiDashengLM-Gen (Xiaomi Research). Local-first, runs entirely on your CUDA GPU. Apache-2.0.

README

MiDashengLM-Gen

midasheng-gen-mcp

Type a scene. Get the whole soundscape.
An MCP server and dark-theme webapp that generate coherent 16 kHz mixed audio scenes - speech, music, sound effects and ambience - in one pass from text, powered by MiDashengLM-Gen (Xiaomi Research). Runs entirely on your GPU.

License: Apache-2.0 Python 3.12+ FastMCP 3.4.4 Local CUDA inference 9 languages MCP hosts Webapp

Quick install · Example prompts · Documentation


One call in Claude, Cursor or opencode - "a comedy club scene with a punchline, crowd laughter and a jazz sting" - and you get a single 16 kHz WAV with all of it. Speech lands close to dedicated TTS quality in 9 languages with emotion control, and the whole ~6 GB model stays on your GPU until you unload it.

What this wraps

MiDashengLM-Gen - the first end-to-end trained general text-to-audio model (per the paper): an LLM drives per-token flow matching to generate variable-length audio scenes with speech intelligibility approaching dedicated TTS (Seed-TTS WER 12.15% -> 2.79% vs 1.24% for dedicated TTS). Supports 9 languages and emotion control. Checkpoint auto-downloads from Hugging Face (~6 GB). Model weights are never bundled - see docs/WRAPPEE.md.

Features

How it runs: a local FastMCP 3.4 server (stdio for Claude Desktop, HTTP /mcp for Cursor/webapp) with a React dashboard. The model loads lazily on first generate and stays on the GPU until unloaded.

Direction Artifacts Notes
Hands-in structured caption (caption/asr/speech/sfx/music/env views), guidance + seed params Via MCP tool, REST, or webapp
Hands-out 16 kHz mono WAV scenes, indexed in SQLite, browsable/exportable audio_scene(operation="export") or webapp
  • Generate whole soundscapes in one call - crowd laughter + jazz sting + comedy speech, or rain + thunder + forest ambience
  • Speech intelligibility near dedicated TTS quality, 9 languages, emotion control through the speech view
  • Scene library with pagination, audio playback, and export in the webapp
  • Async job API for long generations; Prefab UI cards in chat
  • Variable-length output via the learned stop head - no fixed-duration cuts

Quick Install

The fastest path is the .mcpb bundle for Claude Desktop (see INSTALL.md for all options):

  1. Download midasheng-gen-mcp-v0.1.0.mcpb from Releases
  2. Drag it onto Claude Desktop
  3. First use will download the ~6 GB checkpoint automatically

Or clone and double-click start.bat for the full stack (backend + webapp).

Example Prompts

  • "Generate a comedy club scene: a punchline, crowd laughter, and a jazz band sting" -> audio_scene(operation="generate", caption="A comedian delivering a punchline followed by uproarious crowd laughter", asr="And that is why I never buy cheap luggage anymore!", speech="expressive comedic male voice", music="sudden upbeat jazz band sting", sfx="crowd laughter", env="intimate comedy club")
  • "Make a thunderstorm at night in a forest" -> audio_scene(operation="generate", caption="A rolling thunderstorm in a forest at night", sfx="distant thunder and heavy rain", env="dense forest", seed=42)
  • "What's the model state?" -> audio_scene(operation="status")

Documentation

Doc Contents
Installation All install methods, prerequisites
Onboarding First-run model download, GPU checks, pitfalls
Wrapped app MiDashengLM-Gen paper, weights, license, demo
Architecture System architecture, ports, data flow
Configuration Env vars, config options
Tool Reference All available tools
Development Contributing, local setup
Troubleshooting Common issues

Requirements

  • Windows/Linux/macOS with a CUDA GPU (RTX 4090 class: ~12 GB VRAM (fp32); CPU inference works but is slow)
  • Python 3.12+ via uv, Node.js 20+, bun (auto-installed by start.ps1 on naked PCs)
  • ~10 GB free disk (checkpoint ~6 GB + torch stack)

License

Apache-2.0 (model and this wrapper). See the use-restrictions section in the upstream repo: no unlawful/military use, no harm to minors or groups.

from github.com/sandraschi/midasheng-gen-mcp

Installing Midasheng Gen

This server has no published package — it is built from source. Open the repository and follow its README.

▸ github.com/sandraschi/midasheng-gen-mcp

FAQ

Is Midasheng Gen MCP free?

Yes, Midasheng Gen MCP is free — one-click install via Unyly at no cost.

Does Midasheng Gen need an API key?

No, Midasheng Gen runs without API keys or environment variables.

Is Midasheng Gen hosted or self-hosted?

Self-hosted: the server runs locally on your machine via the install command above.

How do I install Midasheng Gen in Claude Desktop, Claude Code or Cursor?

Open Midasheng Gen on unyly.org, pick your client tab (Claude Desktop, Claude Code, Cursor) and press Install — the config is generated automatically, no JSON editing.

Related MCPs

Compare Midasheng Gen with

Not sure what to pick?

Find your stack in 60 seconds

Author?

Embed badge for your README

Browse similar

All media MCPs