Building on FluidSynth — and contributing back

Building on FluidSynth — and contributing back

When you press a button in the Neoharp app and hear a piano, that sound is synthesized by FluidSynth — an open-source engine that turns a note and an instrument’s samples into audio. It’s been around for over twenty years, and dozens of music programs run on it.

We built our app’s sound on FluidSynth. Running on phones surfaced a few mobile-specific needs — so we added them on our side, then contributed the changes back upstream, where they’re available to anyone building on FluidSynth, not just us. Three are already merged into FluidSynth; the fourth is a proposal that’s in review.

Here’s what we worked on.

Sound on iPhone

FluidSynth already shipped with an audio driver for Apple platforms — Apple’s own coreaudio — but it only ran on macOS. For Neoharp on iOS, we extended that driver to run on iOS too, so FluidSynth can now play through coreaudio on iPhone as well as on Mac. PR #1720 is merged into FluidSynth.

A limiter for loud playback

When many notes sound at once at full volume, the combined signal can get too loud and distort — digital clipping. We wired in a limiter from Signalsmith — an open-source DSP library — that eases the peaks back, so loud playback stays clean. PR #1719 is merged.

Logs on Android

FluidSynth emits diagnostic messages, but on Android they didn’t surface where Android developers normally look. We routed them through the standard Android log. Users won’t notice this one — but it makes tracking down issues on Android much easier for anyone developing on the engine. PR #1621 is merged.

A leaner Android build

Still in review, and the most exploratory of the four. FluidSynth uses glib — a general-purpose library — for basics like threads, files, and time, and on most platforms that works just fine. On our Android build we hit an issue with the C++11 threading fallback (likely specific to our toolchain) and wanted to keep the build lean without pulling glib in. So we proposed an optional, Android-focused layer for those basics, letting our Android build run without glib. It’s an admittedly niche need — as the maintainers noted, it’s a fairly specific use case for the amount of code it adds — so it may well stay out of FluidSynth itself. They’ve kept the PR open so anyone who runs into the same thing can weigh in or reuse our work. PR #1718 is with them now.

Why we contribute back

We could have kept all of this in our own fork. But then every FluidSynth update would mean re-applying our changes by hand. Upstreamed, the changes are maintained in the open alongside FluidSynth instead of in a private fork — and they’re there for anyone building on the same engine, not just us.

A bit of perspective: almost all of the sound you hear is FluidSynth and twenty years of its community’s work. Our additions are a few narrow, mobile-specific pieces on top. Thanks to the FluidSynth maintainers and to Signalsmith Audio — none of this would play without them.

Our fork and all the changes