Skip to content
Download

Audio and effects

Status: Implemented (API 1.7). Live control of every sound, filters, effect buses, default routing, the listener, voice chat streams, PCM streams and acoustic tracing, in Lua, C and C++. The server-side voice loudness event is pending (see the end).

This guide builds the pieces most acoustics mods need: a reverb for the whole world, sounds muffled through walls, a radio voice, your own synthesized sound, and reading the room with the tracer. Every call is listed in API Reference: Sound and Music.

All of it runs on the client (put it in a client-side mod or behind cl.has("side.client")), on the client thread: in client events, tick.client_post and the sound events. Nothing changes in the game’s audio until your mod makes its first audio call.

When to call what. Buses, routes and the default filter are mixer setup: make them as soon as your mod loads (top of the script, on_construct, mod_main); the game keeps them and creates them when its sound engine starts. Everything that needs a playing sound or the running engine (sound handles, streams, the listener, the tracer, the voice monitor) is called from a tick or an event; before the engine started it returns CL_E_NOT_LOADED.

Every sound is a handle

client.sound_play fires before any sound starts: vanilla, sent by a server, played by a mod, voice chat. e.handle is that sound, and stays usable while it plays:

cl.on("client.sound_play", function(e)
  local s = e.handle
  if e.category == "hostile" then
    s:set_volume(s:volume() * 0.5)   -- calmer monsters
  end
end)

static void on_sound(void* payload, void* user) {
    cl_ev_sound_play* e = CL_EVENT(cl_ev_sound_play, payload);
    (void)user;
    if (e->category == CL_SOUND_HOSTILE) cl_sound_set_volume(e->handle, e->volume * 0.5f);
}

clight::on<clight::ev::ClientSoundPlay>([](cl_ev_sound_play& e) {
    if (e.category == CL_SOUND_HOSTILE) clight::Sound(e.handle).set_volume(e.volume * 0.5f);
}).detach();

What you set inside the handler applies from the very first sample. Later, the same handle moves the sound (s:set_position(pos)), makes it follow an entity (s:attach(entity, cl.vec(0, 1.6, 0))), or changes its range and fall-off (s:set_range(48), s:set_distance_model("inverse", 1)). client.sound_stop tells you when it ended. cl.sound.playing() lists every sound playing right now.

Muffled through a wall: filters

A filter shapes the sound on its direct path. A low-pass takes out the highs (a wall, water), a high-pass the lows (a small radio speaker), a band-pass both:

s:set_filter{ type = "lowpass", gain = 0.6, gain_hf = 0.15 }    -- behind a wall
s:set_filter{ type = "highpass", gain = 1.0, gain_lf = 0.3 }    -- tinny
s:set_filter(nil)                                                -- back to normal

Gains go from 0 to 1. Change them as often as you like: the engine smooths every change over about 20 ms, so a filter that follows a moving mob never clicks.

A cave for the whole world: buses and default routing

An effect bus is a shared effect (a reverb, an echo…) that many sounds feed, like an effect send on a mixing desk. A default route sends every sound of some categories to it, so you don’t have to touch sounds one by one:

local room = cl.audio.bus_create("reverb")
room:set{ density = 1, diffusion = 0.9, gain = 0.4, decay_time = 3.2, decay_hf_ratio = 0.6, late_reverb_gain = 1.2 }
cl.audio.route_default(room, 0.5, { categories = { "blocks", "hostile", "neutral", "players", "ambient", "weather", "voice" } })

cl_audio_bus room;
cl_audio_bus_create(CL_STR("reverb"), &room);
cl_audio_bus_set(room, CL_STR("decay_time"), 3.2f);
cl_audio_bus_set(room, CL_STR("late_reverb_gain"), 1.2f);
cl_audio_route_default(room, 0.5f, 0 /* every world category */, true, NULL);

auto room = clight::audio::bus_create("reverb");
if (room) {
    room->set("decay_time", 3.2f);
    room->route_default(0.5f, CL_AUDIO_BLOCKS | CL_AUDIO_HOSTILE | CL_AUDIO_AMBIENT, true, nullptr);
}

Then, every tick, set the reverb to the room the player is in (see the tracer below): room:set{ decay_time = rt60, gain = wet }. Twelve effects are available (reverb, echo, chorus, distortion, flanger, ring_modulator, pitch_shifter, equalizer, compressor, autowah, frequency_shifter, vocal_morpher); the reference lists their parameters.

A single sound can send to more buses with s:set_send(bus, gain, filter), and s:set_send(room, 0) takes it out of a route. Buses chain: echo:set_output(room) makes the echoes reverberate.

cl.audio.set_default_filter{ ... } filters every positional sound (deafness, underwater, a stunned player); cl.audio.set_default_filter(nil) clears it.

Sends are limited

Each sound has a fixed number of sends. cl.audio.max_sends() is how many your buses can use: 6, or 2 while the built-in Sound Physics (Client Settings) is on, because it keeps 4 for its own reverb. Routes take the first ones, then each sound’s own sends; more are dropped. Use one or two buses per room rather than one per sound. Several mods can own buses at the same time.

Where the listener is

local l = cl.audio.listener()   -- { pos, forward, up, in_fluid = "water" | "lava" | nil }
if l.in_fluid == "water" then
  cl.audio.set_default_filter{ type = "lowpass", gain = 0.8, gain_hf = 0.1 }
end

cl.audio.hrtf_enabled() tells you whether the player turned on Directional Audio (useful for accessibility cues).

Reading the room: the tracer

Two native helpers save you thousands of block lookups in Lua:

-- One line: what is between the sound and you, in a few microseconds.
local t = cl.audio.trace_paths(sound_pos, l.pos, { max_blocks = 48 })
-- t.blocks, t.by_material = { stone = 3, wood = 1 }, t.hit, t.first_hit, t.occlusion, t.gain, t.gain_hf
s:set_filter{ type = "lowpass", gain = t.gain, gain_hf = t.gain_hf }

-- The full acoustics (reverb time, how closed the space is, the opening a sound comes through), on a worker thread.
local a = cl.audio.trace_acoustics(l.pos, cl.vec(l.pos.x, l.pos.y + 1, l.pos.z))
if a.ready then room:set{ decay_time = math.max(0.1, a.rt60), gain = 0.2 + 0.4 * a.enclosure } end

trace_acoustics returns ready = false the first time and queues the work; ask again next tick. Both use the Sound Physics material model and work with Sound Physics off.

Voice chat with effects

Simple Voice Chat streams (built into Celestial) are sounds too. Route category "voice" and players echo in caves; or give one stream a radio sound:

cl.on("voice.stream_open", function(e)
  -- e.speaker_uuid, e.speaker (an entity or nil), e.kind = "proximity" | "whisper" | "group" | "location" | "static"
  if e.kind == "group" then
    e.handle:set_filter{ type = "bandpass", gain = 1, gain_lf = 0.2, gain_hf = 0.4 }   -- walkie-talkie
  end
end)

cl.on("voice.talking", function(e)
  -- e.speaker_uuid, e.talking, e.whisper: "who is talking" HUDs, mobs that react to noise
end)

-- Your own voice, played back through the room only (the way others hear you).
cl.voice.monitor{ enabled = true, wet_only = true, gain = 0.5, bus = room }

cl.voice.is_available(), is_muted() and is_disabled() tell you the voice chat state.

Your own sound: PCM streams

A stream plays samples your mod makes: a synthesizer, an engine hum, text-to-speech.

local st
local phase = 0
cl.on("tick.client_post", function()
  st = st or cl.audio.stream_open{ rate = 48000, channels = 1, pos = cl.vec(0, 64, 0), category = "ambient" }
  if st:queued() < 4800 then              -- keep ~100 ms ahead
    local t = {}
    for i = 1, 2400 do
      t[i] = math.sin(phase) * 0.2         -- numbers from -1 to 1
      phase = phase + 2 * math.pi * 220 / 48000
    end
    st:push(t)
  end
end)

In C and C++, push 16-bit little-endian samples (cl_audio_stream_push(st, (cl_str){(const char*)buf, bytes})). st:sound() is the stream as a sound, for filters and sends. Two channels work for listener-relative streams (relative = true).

Distance models in sounds.json

Vanilla sounds fade linearly and stop at their range. Your own sounds can fall off like real sound:

{ "distant_bell": { "sounds": [{ "name": "my_mod:bell", "attenuation_distance": 96, "distance_model": "inverse", "rolloff": 1 }] } }

Vanilla and vanilla resource packs ignore these fields, so nothing changes for them.

Detecting the features

if cl.has("audio") then ... end              -- the API exists (Celestial API 1.7+)
if cl.has("audio.efx") then ... end          -- the device can play effects (else everything works silently)
if cl.has("audio.bus_output") then echo:set_output(room) end

Pending

A server-side voice.server_packet event (each speaking player’s loudness, without the audio, for “monsters hear you” gameplay) needs a voice chat server inside Celestial’s integrated and dedicated server. That server doesn’t exist yet, so the event isn’t declared; cl.has("voice.server_packet") will turn true when it ships.