From 38740b43a901022bf8cf0ede19e59ea606dc11a9 Mon Sep 17 00:00:00 2001 From: Aleksander Helgaker Date: Wed, 4 Nov 2026 10:25:57 +0100 Subject: [PATCH] craft-audio: core types, WAV/AIFF read and write, peaks, symphonia (#441) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * craft-audio: core types, WAV/AIFF read and write, peaks, symphonia The new crate without any codec module: what every later codec plugs into. - Shared types: `AudioBuffer`, `AudioInfo`, `ChannelLayout`/`Speaker`, `AudioError`, `EncodeOptions` (built with `new` and `with_*`), `EncodeFormat`, `BitDepth`, `BwfInfo`, `SampleFormat`. - WAV/BWF/RF64 and AIFF/AIFF-C reading and writing from SoundCraft's `soundcraft-audio-io`, kept over EffectCraft's and FilmCraft's smaller writers (only it has BWF, dither, any speaker mask and streaming without a known length). `encode` and `FileEncoder` behind `pcm-encode`; `lossless-encode` and `encode` turn it on. - `decode`, `probe`, `tracks` and the streaming, seekable `Decoder` (feature `decode-native`). The native path reads WAV, AIFF, and PCM in MP4/MOV and Matroska through craft-media's demuxers, packet by packet, with edit lists and `CodecDelay`/`DiscardPadding` applied. `TrackDecoder` decodes PCM; each codec PR adds its codec to it. - symphonia 0.6 (MPL-2.0, used unmodified) behind the `symphonia` feature, part of `decode`: the fallback for every format with no native decoder yet. Its size fields are checked before it reads, and its panics are caught where they unwind. - `peaks`: the min/max/RMS pyramid for waveform drawing. Workspace: craft-media is a workspace dependency without default features, so an encode-only craft-audio does not build the MP4 demuxer, and `symphonia-bundle-flac` builds without overflow checks in dev profiles (its 32-bit LPC restore wraps on valid input). CI: `audio.yml`, with clippy per feature, rustdoc, WASM, MSRV, and the tests with ffmpeg oracles. Co-Authored-By: Claude Opus 5.5 * craft-audio: review fixes (CodeRabbit on #441) - Reading: AIFF that declares frames without an SSND chunk is Malformed; a Matroska PCM track's length counts from its first frame; seek targets saturate; an empty FLAC has one track; a failed header read is an Io error; deinterleave allocates fallibly; concealed silence is capped like a decoded packet, and symphonia's uses the damaged packet's own duration when known. - Peaks sum squares in f64, so loud float samples give a finite RMS. - Writing: .aifc maps to AIFF; BWF text is cut at a character boundary; a WAV byte rate past 32 bits is an error; the FileEncoder docs say nothing may follow the sink's position. - README: the Limits list says which codecs go through symphonia and which MP4 edit lists apply. - Tests for each, alias and dither coverage, and a FLAC case in the routing oracle. Co-Authored-By: Claude Opus 5.5 * craft-audio: review fixes, second round (CodeRabbit on #441) - Every header read is checked: `Native::open`, `native::tracks` and the no-symphonia error read it with `head_checked`. craft-media demuxer errors that are I/O failures (not a file ending early) are returned as `AudioError::Io` instead of sending the file to the other path. - A Matroska track's presented length runs from its first frame plus `CodecDelay` (unclamped, in timestamp ticks with the delay rounded to one) to its end. - symphonia's concealment allocates fallibly, and a damaged first packet is concealed when the demuxer gives its duration. `deinterleave` reserves its channel list fallibly too. - `TrackDecoder::conceal` documents its cap at the decode limit. - Tests: an I/O failure past the first 64 KiB, codec delay with the first block at -20, 0 and 50 ms (checked with `probe`), a seek back after an overshoot, and the loud-sample RMS value. Co-Authored-By: Claude Opus 5.5 * craft-audio: TrackDecoder::conceal returns a Result Concealment allocates fallibly and fails with TooLarge when the silence would pass the 2^30-sample decode limit, instead of returning less than the packet's length. The native path, the README, the packets example and the doc example pass the error on. Co-Authored-By: Claude Opus 5.5 * craft-audio: review fixes, third round (CodeRabbit on #441) - Concealed silence is held to the decode limit inside pcm::silence, so symphonia's path checks frames × channels too, and symphonia's decoded audio is copied into fallibly allocated planes. - TrackDecoder::reset forgets the last packet's length. - Without symphonia, a damaged MP4, Matroska or Ogg file (truncated, invalid, no moov) is Malformed instead of Unsupported; a feature the demuxer does not read still names the build. - Tests: an MP4 with no moov in a native-only build, conceal after a reset and past the limit. Co-Authored-By: Claude Opus 5.5 * craft-audio: review fixes, fourth round (CodeRabbit on #441) - A file the native demuxers find damaged is Malformed with symphonia too, when symphonia calls it unsupported; before, an ftyp-only MP4 was Unsupported in symphonia builds. - symphonia's decoded blocks are held to the 2^30-sample decode limit before allocating. - Symph::seek forgets the last packet's length, so a lost packet after a seek is not concealed with the length of one before it. Co-Authored-By: Claude Opus 5.5 * craft-audio: review fixes, fifth round (CodeRabbit on #441) - The damaged-file test covers Matroska as well as MP4, and fails when either branch of native::damage is removed. - native::damage leaves out Ogg: ogg::open reports no damage (it skips damaged pages and calls a file with none it can read not Ogg), and symphonia reports damaged Ogg as malformed itself. Co-Authored-By: Claude Opus 5.5 * craft-audio: run the ffmpeg oracle tests on ubuntu-26.04 ubuntu-latest's apt ships ffmpeg 6.1.1, which writes MP2 and E-AC-3 in Matroska with ends a few milliseconds early, so the routing tests on the later codec PRs cannot pass on it. ubuntu-26.04 ships ffmpeg 8.0. Co-Authored-By: Claude Opus 5.5 --------- Co-authored-by: Claude Opus 5.5 --- .github/workflows/audio.yml | 112 +++ Cargo.lock | 232 +++++ Cargo.toml | 11 +- README.md | 5 +- crates/audio/Cargo.toml | 69 ++ crates/audio/README.md | 360 ++++++++ crates/audio/examples/encode.rs | 31 + crates/audio/examples/info.rs | 25 + crates/audio/examples/packets.rs | 34 + crates/audio/examples/stream.rs | 27 + crates/audio/src/buffer.rs | 72 ++ crates/audio/src/layout.rs | 298 +++++++ crates/audio/src/lib.rs | 319 +++++++ crates/audio/src/pcm.rs | 156 ++++ crates/audio/src/peaks.rs | 285 +++++++ crates/audio/src/read/aiff.rs | 149 ++++ crates/audio/src/read/caf.rs | 87 ++ crates/audio/src/read/flac.rs | 104 +++ crates/audio/src/read/mod.rs | 601 +++++++++++++ crates/audio/src/read/mp4.rs | 106 +++ crates/audio/src/read/native.rs | 660 ++++++++++++++ crates/audio/src/read/source.rs | 135 +++ crates/audio/src/read/symph.rs | 899 ++++++++++++++++++++ crates/audio/src/read/wav.rs | 320 +++++++ crates/audio/src/track.rs | 230 +++++ crates/audio/src/wav.rs | 7 + crates/audio/src/write/aiff.rs | 154 ++++ crates/audio/src/write/mod.rs | 183 ++++ crates/audio/src/write/pcm.rs | 124 +++ crates/audio/src/write/wav.rs | 241 ++++++ crates/audio/tests/audio/encode_format.rs | 18 + crates/audio/tests/audio/float_headroom.rs | 101 +++ crates/audio/tests/audio/main.rs | 8 + crates/audio/tests/audio/malformed.rs | 699 +++++++++++++++ crates/audio/tests/audio/roundtrip.rs | 274 ++++++ crates/audio/tests/audio/stream.rs | 259 ++++++ crates/audio/tests/audio/wav_empty_data.rs | 87 ++ crates/audio/tests/codecs/main.rs | 10 + crates/audio/tests/codecs/routing_oracle.rs | 320 +++++++ crates/audio/tests/codecs/testkit.rs | 61 ++ crates/media/README.md | 4 +- 41 files changed, 7873 insertions(+), 4 deletions(-) create mode 100644 .github/workflows/audio.yml create mode 100644 crates/audio/Cargo.toml create mode 100644 crates/audio/README.md create mode 100644 crates/audio/examples/encode.rs create mode 100644 crates/audio/examples/info.rs create mode 100644 crates/audio/examples/packets.rs create mode 100644 crates/audio/examples/stream.rs create mode 100644 crates/audio/src/buffer.rs create mode 100644 crates/audio/src/layout.rs create mode 100644 crates/audio/src/lib.rs create mode 100644 crates/audio/src/pcm.rs create mode 100644 crates/audio/src/peaks.rs create mode 100644 crates/audio/src/read/aiff.rs create mode 100644 crates/audio/src/read/caf.rs create mode 100644 crates/audio/src/read/flac.rs create mode 100644 crates/audio/src/read/mod.rs create mode 100644 crates/audio/src/read/mp4.rs create mode 100644 crates/audio/src/read/native.rs create mode 100644 crates/audio/src/read/source.rs create mode 100644 crates/audio/src/read/symph.rs create mode 100644 crates/audio/src/read/wav.rs create mode 100644 crates/audio/src/track.rs create mode 100644 crates/audio/src/wav.rs create mode 100644 crates/audio/src/write/aiff.rs create mode 100644 crates/audio/src/write/mod.rs create mode 100644 crates/audio/src/write/pcm.rs create mode 100644 crates/audio/src/write/wav.rs create mode 100644 crates/audio/tests/audio/encode_format.rs create mode 100644 crates/audio/tests/audio/float_headroom.rs create mode 100644 crates/audio/tests/audio/main.rs create mode 100644 crates/audio/tests/audio/malformed.rs create mode 100644 crates/audio/tests/audio/roundtrip.rs create mode 100644 crates/audio/tests/audio/stream.rs create mode 100644 crates/audio/tests/audio/wav_empty_data.rs create mode 100644 crates/audio/tests/codecs/main.rs create mode 100644 crates/audio/tests/codecs/routing_oracle.rs create mode 100644 crates/audio/tests/codecs/testkit.rs diff --git a/.github/workflows/audio.yml b/.github/workflows/audio.yml new file mode 100644 index 00000000..1dc76d17 --- /dev/null +++ b/.github/workflows/audio.yml @@ -0,0 +1,112 @@ +name: Shared audio + +on: + push: + branches: [main] + paths: ["crates/audio/**", "crates/media/**", "crates/media-testkit/**", "Cargo.toml", "Cargo.lock", ".github/workflows/audio.yml"] + pull_request: + paths: ["crates/audio/**", "crates/media/**", "crates/media-testkit/**", "Cargo.toml", "Cargo.lock", ".github/workflows/audio.yml"] + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: shared-audio-${{ github.ref }} + cancel-in-progress: true + +jobs: + lint: + timeout-minutes: 15 + name: Audio lint, feature sets, docs, WASM + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: dtolnay/rust-toolchain@b725eda525362e7cf34af9a7243cac1e3f01a7ec # stable + with: + components: rustfmt, clippy + targets: wasm32-unknown-unknown + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + with: + key: craft-audio-lint + - run: cargo fmt -p craft-audio --check + # Each feature on its own, so code that needs another feature fails here. + - name: Clippy, each feature alone + run: | + for f in "" decode decode-native symphonia encode lossless-encode pcm-encode peaks serde; do + echo "::group::features: ${f:-none}" + cargo clippy -p craft-audio --all-targets --locked --no-default-features ${f:+--features "$f"} -- -D warnings + echo "::endgroup::" + done + - run: cargo clippy -p craft-audio --all-targets --locked --all-features -- -D warnings + # missing_docs is a warning in the crate; -D warnings makes it (and broken links) fail here. + - run: cargo doc -p craft-audio --all-features --no-deps --locked + env: + RUSTDOCFLAGS: "-D warnings" + # Partial feature sets must document too (their links to other features are allowed to dangle). + - run: cargo doc -p craft-audio --no-default-features --features decode-native --no-deps --locked + env: + RUSTDOCFLAGS: "-D warnings" + - run: cargo doc -p craft-audio --no-default-features --features lossless-encode --no-deps --locked + env: + RUSTDOCFLAGS: "-D warnings" + - run: cargo check -p craft-audio --all-features --locked --target wasm32-unknown-unknown + - run: cargo check -p craft-audio --no-default-features --features lossless-encode --locked --target wasm32-unknown-unknown + msrv: + timeout-minutes: 10 + name: Audio on the minimum Rust version + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + # The workspace's rust-version. + - run: rustup toolchain install 1.95.0 --profile minimal + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + with: + key: craft-audio-msrv + - run: cargo +1.95.0 check -p craft-audio -p craft-media --all-features --locked + features: + timeout-minutes: 10 + name: Audio tests without symphonia and lossless-encode only + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: dtolnay/rust-toolchain@b725eda525362e7cf34af9a7243cac1e3f01a7ec # stable + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + with: + key: craft-audio-features + - run: cargo test -p craft-audio --no-default-features --features lossless-encode --locked + - run: cargo test -p craft-audio --no-default-features --features pcm-encode --locked --lib + - run: cargo test -p craft-audio --no-default-features --features decode-native --locked + test: + timeout-minutes: 20 + name: Audio tests with ffmpeg oracles + # Pinned for the ffmpeg its apt ships (8.0): the oracle's files come from this ffmpeg, and + # ffmpeg 6, on ubuntu-24.04, writes MP2 and E-AC-3 in Matroska with early ends. + runs-on: ubuntu-26.04 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: dtolnay/rust-toolchain@b725eda525362e7cf34af9a7243cac1e3f01a7ec # stable + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + with: + key: craft-audio-test + - name: Build tests while installing ffmpeg + run: | + # The man-db index rebuild after dpkg is slow on hosted runners. + sudo rm -f /var/lib/man-db/auto-update + (sudo apt-get update -qq && sudo apt-get install -y -qq --no-install-recommends ffmpeg) > "$RUNNER_TEMP/apt.log" 2>&1 & + apt=$! + cargo test -p craft-audio --all-features --locked --no-run + wait "$apt" || { cat "$RUNNER_TEMP/apt.log"; exit 1; } + ffmpeg -hide_banner -version | sed -n 1p + - run: cargo test -p craft-audio --all-features --locked + env: + CRAFT_MEDIA_REQUIRE_ORACLES: "1" + CRAFT_MEDIA_FIXTURES_DIR: ${{ runner.temp }}/media-fixtures diff --git a/Cargo.lock b/Cargo.lock index fd5e504f..4c12df4f 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -786,6 +786,20 @@ dependencies = [ "rustix 1.1.5", ] +[[package]] +name = "craft-audio" +version = "0.1.0" +dependencies = [ + "craft-media", + "craft-media-testkit", + "log", + "proptest", + "serde", + "serde_json", + "symphonia", + "thiserror 2.0.21", +] + [[package]] name = "craft-cms" version = "0.1.0" @@ -1759,6 +1773,12 @@ dependencies = [ "pin-project-lite", ] +[[package]] +name = "extended" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "af9673d8203fcb076b19dfd17e38b3d4ae9f44959416ea532ce72415a6020365" + [[package]] name = "fastrand" version = "2.5.0" @@ -3345,6 +3365,15 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "num-complex" +version = "0.4.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73f88a1307638156682bada9d7604135552957b7818057dcef22705b4d509495" +dependencies = [ + "num-traits", +] + [[package]] name = "num-conv" version = "0.2.2" @@ -4440,6 +4469,12 @@ dependencies = [ "thiserror 2.0.21", ] +[[package]] +name = "regex-lite" +version = "0.1.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cab834c73d247e67f4fae452806d17d3c7501756d98c8808d7c9c7aa7d18f973" + [[package]] name = "regex-syntax" version = "0.8.11" @@ -5343,6 +5378,203 @@ dependencies = [ "siphasher", ] +[[package]] +name = "symphonia" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7edef6a96b696d4e0cab5ee9ebb7ca155ed95f30a6b45bbb8b97d2727f02424" +dependencies = [ + "lazy_static", + "symphonia-bundle-flac", + "symphonia-bundle-mp3", + "symphonia-codec-aac", + "symphonia-codec-adpcm", + "symphonia-codec-alac", + "symphonia-codec-pcm", + "symphonia-codec-vorbis", + "symphonia-core", + "symphonia-format-caf", + "symphonia-format-isomp4", + "symphonia-format-mkv", + "symphonia-format-ogg", + "symphonia-format-riff", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-bundle-flac" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6405d5c34ff6f8f7ca08a4101efe66d21e180f7322ef9359900a0e55698a7892" +dependencies = [ + "log", + "symphonia-common", + "symphonia-core", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-bundle-mp3" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "98ea5ffc8716bff677dfb3b01b420c7b758de901a72b8c330bf2040ab74b4add" +dependencies = [ + "lazy_static", + "log", + "symphonia-core", +] + +[[package]] +name = "symphonia-codec-aac" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f5bf8e39552d34a3c4c98333370e62f48c92456d2e814f273b5c3ad7c4a5f45c" +dependencies = [ + "lazy_static", + "log", + "symphonia-common", + "symphonia-core", +] + +[[package]] +name = "symphonia-codec-adpcm" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "445932ecb0c59362fde9c082dd63a75380650c5077fdb8b80b8cac850442a74c" +dependencies = [ + "log", + "symphonia-core", +] + +[[package]] +name = "symphonia-codec-alac" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "920a78f96f3cf62932d0c497959c0cc2100ef36b03c7ca1417b260f432a482d8" +dependencies = [ + "log", + "symphonia-common", + "symphonia-core", +] + +[[package]] +name = "symphonia-codec-pcm" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e04ba75686acbe43542fdd374571195f0530c0b7785ca25cc6840e9c6c4b6eea" +dependencies = [ + "log", + "symphonia-core", +] + +[[package]] +name = "symphonia-codec-vorbis" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "73d90b4fcf796137cc683c538282804ff9629f8ad9dbfd881fcbba331ac4e986" +dependencies = [ + "log", + "symphonia-common", + "symphonia-core", +] + +[[package]] +name = "symphonia-common" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2acc3fcc18ec9b8cdd48614e259c4cf0d27b71d41e5d9b120b42c5adab12d7c4" +dependencies = [ + "log", + "symphonia-core", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-core" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "01c412864d599d4750d0c3d684d7e093ec05e5309681ef5252cc1096a437f6e0" +dependencies = [ + "bitflags 2.13.2", + "bytemuck", + "lazy_static", + "log", + "num-complex", + "smallvec", +] + +[[package]] +name = "symphonia-format-caf" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ab64327ee1920531c5bf86cf7e9cd1a599d57013ddc30691da62683cefc65191" +dependencies = [ + "log", + "symphonia-common", + "symphonia-core", +] + +[[package]] +name = "symphonia-format-isomp4" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0e681a70e1870d34e02abf1dbc51e4267c3f1827801474e8870be8c689fc4dc3" +dependencies = [ + "log", + "symphonia-common", + "symphonia-core", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-format-mkv" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d015c5c0558864665894b3f4cbd95e10abb01b9c868e751c72670f326a56360e" +dependencies = [ + "lazy_static", + "log", + "symphonia-common", + "symphonia-core", +] + +[[package]] +name = "symphonia-format-ogg" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b5495e7f7e3c7035328d82b6d6e377eef289bb0c4105bdeb557fc93a833f994" +dependencies = [ + "log", + "symphonia-common", + "symphonia-core", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-format-riff" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1ff70929083a8c1a5f6cd7c904b6071c7914ad04739b510c2f7239dfc9b7dabe" +dependencies = [ + "extended", + "log", + "symphonia-core", + "symphonia-metadata", +] + +[[package]] +name = "symphonia-metadata" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "83713a97705d77bdef7cdbc0768fd6e5a54e4cd7e48d60a806ae85639e2c87c6" +dependencies = [ + "lazy_static", + "log", + "regex-lite", + "smallvec", + "symphonia-core", +] + [[package]] name = "syn" version = "1.0.109" diff --git a/Cargo.toml b/Cargo.toml index 24e75777..7ebee590 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -37,7 +37,9 @@ craft-fonts-build = { path = "crates/fonts-build" } craft-i18n = { path = "crates/i18n" } craft-jpeg = { path = "crates/jpeg" } craft-keymap = { path = "crates/keymap" } -craft-media = { path = "crates/media" } +craft-audio = { path = "crates/audio" } +# No default features, so craft-audio's `encode` builds craft-media without `isobmff`. +craft-media = { path = "crates/media", default-features = false } craft-media-testkit = { path = "crates/media-testkit" } craft-menu = { path = "crates/menu" } craft-pcraft-format = { path = "crates/pcraft-format" } @@ -94,3 +96,10 @@ eframe = { git = "https://github.com/storytold/egui.git", rev = "6d518fc79968157 egui_demo_lib = { git = "https://github.com/storytold/egui.git", rev = "6d518fc799681571963842f95b55a6b68431b15a" } egui_inspection = { git = "https://github.com/storytold/egui.git", rev = "6d518fc799681571963842f95b55a6b68431b15a" } winit = { git = "https://github.com/storytold/winit.git", rev = "a517b3846ee8457d84ee4975110bfda6797c44dc" } + +# symphonia's FLAC decoder restores LPC samples as `residual + (prediction as i32)`. In a valid +# 32-bit stream the prediction can leave the i32 range while the sum does not, so a debug build +# panics where a release build wraps to the right sample. Apps that decode 32-bit FLAC through +# craft-audio's `decode` need the same override in their own workspace. +[profile.dev.package.symphonia-bundle-flac] +overflow-checks = false diff --git a/README.md b/README.md index 7747fb09..6c004096 100644 --- a/README.md +++ b/README.md @@ -78,9 +78,12 @@ craft- = { git = "https://github.com/storytold/craft-libs", rev = "` + per channel), in WAV channel order (L R C LFE, back, side) whenever the channel layout is known. +- Malformed input returns an error, never a panic (see [symphonia](#symphonia) for the one + exception). No `unsafe`. Readers and decoders are tested with corrupted and truncated input. +- Implemented from the published specifications; tested against ffmpeg. + Builds for `wasm32-unknown-unknown`. +- Containers (MP4/MOV, Matroska/WebM, Ogg), bit I/O and the `Container`/`CodecId`/`CodecParams` + types come from [`craft-media`](../media/README.md). This crate never depends on video codecs. + +## Cargo features + +No default features. The shared types (`AudioBuffer`, `AudioInfo`, `ChannelLayout`, `AudioError`, +`EncodeOptions`, `ByteSource`, …) build with none. + +| Feature | Provides | Pulls in | +|---|---|---| +| `decode` | `decode`, `probe`, `Decoder`, `tracks`, `TrackDecoder`; every format in [Format support](#format-support) | `decode-native`, `symphonia` | +| `decode-native` | the same, native formats only (others are `Unsupported`) | craft-media `isobmff`/`matroska`/`ogg`, `log` | +| `symphonia` | the fallback for formats with no native decoder; turning it on is the same as `decode` | `decode-native`, symphonia | +| `encode` | every encoder: the WAV and AIFF file writers | `lossless-encode` | +| `lossless-encode` | WAV and AIFF writing | `pcm-encode` | +| `pcm-encode` | `encode`, `FileEncoder` for WAV and AIFF; `wav_float_header` | none | +| `peaks` | `peaks::Peaks` | none | +| `serde` | `Serialize`/`Deserialize` on the shared types | `serde` | + +An app that decodes 32-bit FLAC with `decode` needs a profile override in its workspace; see +[symphonia](#symphonia). + +WAV and AIFF export only (no decoders, no demuxers, no symphonia): + +```toml +craft-audio = { git = "https://github.com/storytold/craft-libs", rev = "", features = ["lossless-encode"] } +``` + +Decode, every encoder and waveform overviews: + +```toml +craft-audio = { git = "https://github.com/storytold/craft-libs", rev = "", features = ["decode", "encode", "peaks"] } +``` + +craft-audio re-exports the craft-media types its API uses (`ByteSource`, `ReadSeekSource`, +`CodecParams`, …), and craft-media itself as `craft_audio::craft_media`. Use that to reach the +containers directly (for example to pass an `isobmff::Track`'s `codec_params()` to +`TrackDecoder`); `decode-native` turns on all three. A separate craft-media dependency must use +the **same `rev`** as craft-audio: with two revisions Cargo builds two copies, and their types do +not match. + +## Usage + +Runnable versions are in `examples/`: + +```bash +cargo run -p craft-audio --example info --features decode -- take.m4a +``` + +```bash +cargo run -p craft-audio --example encode --features lossless-encode -- out +``` + +```bash +cargo run -p craft-audio --example stream --features decode -- long-recording.m4a 60 +``` + +```bash +cargo run -p craft-audio --example packets --features decode -- take.mov +``` + +`stream` takes a file and a start time in seconds; `packets` takes an MP4 file. The entry points +(`decode`, `probe`, `tracks`, `Decoder`, `TrackDecoder`, `encode`, `FileEncoder`) and the +functions around them have rustdoc examples. + +### Decode a file + +```rust +use craft_audio::{decode, probe}; + +let bytes = std::fs::read("take.m4a")?; + +// Container, codec, rate, speaker layout and length. Read from the headers; when they give no +// length (an MP3 without a Xing frame, PCM in Matroska), the file is decoded to count it. +let info = probe(&bytes, Some("m4a"))?; +println!("{} in {}, {} Hz, {}", info.codec.name(), info.container.name(), info.sample_rate, info.layout); + +// The first audio track: planar f32 in WAV channel order, trimmed to its presentation length. +let (info, buf) = decode(&bytes, Some("m4a"))?; +let left: &[f32] = &buf.channels[0]; +``` + +`decode` returns `TooLarge` above 2³⁰ samples (4 GiB of `f32`). `info.damaged` counts packets +that were damaged and replaced by silence of their length. + +### Choose a track + +```rust +use craft_audio::{DecodeOptions, decode_with, tracks}; + +let list = tracks(&bytes, None)?; // codec, rate and channels of every audio track +if let Some(second) = list.get(1) { + let opts = DecodeOptions::default().with_track(second.index); + let (info, buf) = decode_with(&bytes, None, &opts)?; +} +``` + +A track index the file does not have is `Unsupported`. `DecodeOptions::with_max_samples` sets +another size limit. `tracks_of` lists the tracks of a file read from a `ByteSource`; +`detect_container` names a file's container from its first bytes. + +### Stream and seek + +```rust +use craft_audio::{DecodeOptions, Decoder}; + +let file = std::fs::File::open("long-recording.m4a")?; +let mut dec = Decoder::open(file, Some("m4a"), &DecodeOptions::default())?; +dec.seek(48_000 * 60)?; // sample-exact: the next block starts here +while let Some(block) = dec.next_block()? { + // block.channels: planar f32, WAV channel order +} +``` + +`Decoder::open` takes any `ByteSource` that is `Send + Sync` (`SharedSource`): a `Vec` or +`Arc<[u8]>`, a `ReadSeekSource` around any `Read + Seek`, or a `std::fs::File`. Prefer +`ReadSeekSource` for files: it takes the length once and reports a failure when it is made. On +single-threaded WebAssembly the source need not be `Send`, so a reader over a JS `Blob` works. +`Decoder::from_bytes` does the same for a borrowed buffer. + +`seek` returns the frame decoding resumes at. A seek to or past the end of a file whose length is +known stops at the end, and the next `next_block` returns `None`. When `length_known()` is false, +`info().frames` is 0: add up the block lengths, or call `Decoder::probe`, which decodes the rest +of the file to count them. `Decoder::probe` is also how to probe a file on disk. + +Native formats read their samples from the source packet by packet. The container's packet index +stays in memory (tens of bytes per packet), and opening a Matroska file reads through the whole +file to build it. Formats symphonia decodes are read whole first. + +A damaged packet is concealed and counted in `info().damaged`. A source that fails a read (a +network drop) returns `AudioError::Io` from `next_block` instead, and that packet is skipped. To +retry it, keep the position: start from the frame the last `seek` returned (0 without one), add +each block's frames, and `seek` to that position. + +### Decode packets from your own demuxer + +`TrackDecoder` takes a track's `CodecParams` and the first packets. The parameters come from any +craft-media demuxer's `codec_params()` (see +[craft-media's codec parameters](../media/README.md#containers-codecs-and-codec-parameters)), or +are built with `CodecParams::new` for another container: + +```rust +use craft_audio::{CodecId, CodecParams, PcmFormat, TrackDecoder}; + +let params = CodecParams::new(CodecId::Pcm).with_pcm(PcmFormat::int(24, true, true)).with_sample_rate(48_000).with_channels(2); +let lookahead: Vec<&[u8]> = packets.iter().take(craft_audio::LOOKAHEAD_PACKETS).map(Vec::as_slice).collect(); +let mut dec = TrackDecoder::new(¶ms, &lookahead)?; +for p in &packets { + let block = dec.decode(p).or_else(|_| dec.conceal(Some(p)))?; +} +``` + +It decodes PCM so far, and returns `AudioBuffer`s in WAV order when `layout()` is ordered. +`codec()`, `sample_rate()` and `layout()` describe the output. Container timing is the caller's. + +To seek: position your demuxer at the packet that starts `preroll()` frames or more before the +target, call `reset()`, decode from there, and drop the output before the target. + +### Encode a file + +```rust +use craft_audio::{AudioBuffer, BitDepth, ChannelLayout, EncodeFormat, EncodeOptions, FileEncoder, encode}; + +// One Vec of samples per channel, full scale ±1. `AudioBuffer::new(rate, channels, frames)` +// makes a silent one to fill. +let buf = AudioBuffer { sample_rate: 48_000, channels: vec![left, right] }; + +let opts = EncodeOptions::new(EncodeFormat::Aiff).with_bit_depth(BitDepth::Int24).with_dither(true); +let aiff: Vec = encode(&buf, &opts)?; + +// 5.1 with side surrounds: a layout other than the default for 6 channels, recorded in the WAV. +let wav = encode(&surround, &EncodeOptions::default().with_layout(ChannelLayout::SURROUND_5_1_SIDE))?; + +// Block by block into any Write + Seek with nothing after its position; the same bytes as `encode`. +let mut enc = FileEncoder::new(std::fs::File::create("take.wav")?, &EncodeOptions::default(), 48_000, 2)?; +enc.write(&buf.channels, buf.frames())?; // one slice (or Vec) per channel +enc.finish()?; +``` + +`EncodeOptions::default()` is 24-bit WAV. `with_bwf` adds a BWF `bext` chunk and +`with_force_rf64` writes RF64 below 4 GiB; both apply to WAV only. `encode_format_for("aiff")` +maps an extension to an `EncodeFormat`, and `ENCODE_EXTENSIONS` lists the extensions the enabled +features can write. `write` refuses a block without one slice per channel, or with a slice +shorter than `frames`, and writes nothing. After a write fails for another reason, every later +`write` and `finish` returns an error. + +### Waveform overviews + +```rust +use craft_audio::peaks::Peaks; + +let peaks = Peaks::build(&left); // once per channel +// Start sample, samples per column, overlap, columns; pass the same samples each time. One +// (min, max) per column, `None` past the end of the data. +let columns = peaks.spans(&left, 0.0, 512.0, 0.0, 800); +``` + +`Peaks` is a min/max/sum-of-squares pyramid (32-sample blocks, fan-out 4). Range queries are +exact, O(log n). `rms_spans` returns one RMS value per column. + +### Errors + +Functions return `craft_audio::Result`. + +| `AudioError` | Cause | +|---|---| +| `Unsupported` | valid input using an unsupported codec, feature or option, or a format whose feature is off | +| `Malformed` | corrupt or truncated input, or not the claimed format | +| `TooLarge` | over a size limit (`DecodeOptions::max_samples`, an allocation) | +| `Encode` | an encoder rejected the input or its settings (a buffer with no channels or a 0 Hz rate, a block shorter than its frame count, a wrong layout) | +| `Io` | the source failed a read, or the `FileEncoder` writer failed | + +`AudioError` is `#[non_exhaustive]`, so a `match` needs a `_` arm: + +```rust +match craft_audio::decode(&bytes, None) { + Ok((info, buf)) => { /* … */ } + Err(craft_audio::AudioError::Unsupported(what)) => eprintln!("cannot decode: {what}"), + Err(e) => eprintln!("{e}"), +} +``` + +## Format support + +| Format | Decode | Encode | +|---|---|---| +| WAV, BWF, RF64/BW64 | native | native | +| AIFF, AIFF-C | native | native | +| FLAC | symphonia | — | +| AAC-LC, HE-AAC | symphonia | — | +| MP1, MP2, MP3 (MPEG audio Layer I/II/III) | symphonia | — | +| ALAC, Vorbis, ADPCM, A-law, µ-law | symphonia | — | +| Opus | — | — | + +| Container | Demuxer | Timing | +|---|---|---| +| MP4, M4A, MOV | `craft_media::isobmff` for PCM; symphonia for the others (AAC, ALAC, MP3, FLAC) | edit lists | +| Matroska, WebM | `craft_media::matroska` for PCM; symphonia for the others (AAC, Vorbis, FLAC, MP3) | `CodecDelay`, `DiscardPadding` | +| Ogg | symphonia (Vorbis, FLAC) | — | +| Raw ADTS AAC, MP1/MP2/MP3 | symphonia | — | +| FLAC | symphonia | — | +| CAF | symphonia | — | + +## Limits + +- AIFF with more than four channels decodes with an unordered layout, in the file's order. +- Not decoded at all: AC-3, E-AC-3, Opus and Speex. +- `TrackDecoder` returns `Unsupported` for every codec but PCM; `decode` and `Decoder` read the + other codecs in the table above through symphonia. +- MP4 edit lists: a single leading media edit (encoder priming) is applied. Empty edits and lists + of more than one edit are not; the media plays as it is stored. + +## Codecs + +| Codec | Spec | Scope | Verified against | +|---|---|---|---| +| WAV, AIFF | Microsoft RIFF, Apple AIFF/AIFF-C | read: integer 8–32-bit, float 32/64-bit, `WAVE_FORMAT_EXTENSIBLE` masks, RF64/BW64, BWF `bext`, truncated files; write: 16/24/32-bit integer and 32-bit float, RF64 above 4 GiB, any layout, TPDF dither | round trips; hostile-input tests | + +## symphonia + +symphonia 0.6 (MPL-2.0, used unmodified as the Craft licensing rules require) decodes the formats +marked "symphonia" in [Format support](#format-support). Each is to be replaced by a native +decoder, after which the dependency is removed. The native path is tried first, so a new native +decoder takes its formats over without an API change. Build with `decode-native` instead of +`decode` to see what still needs symphonia. + +Notes while symphonia is in use: + +- symphonia reads from memory: a file it decodes is read whole first. Its decoded audio still + streams through `Decoder`. +- symphonia allocates sizes taken from the file. Container size fields of WAV, FLAC, MP4 and CAF + files are checked against the file length before symphonia reads them. +- A panic inside symphonia is caught and returned as `AudioError::Malformed` only where panics + unwind: native builds with the default `panic = "unwind"`. On `wasm32-unknown-unknown`, and in + an app built with `panic = "abort"`, such a panic aborts. The native decoders do not panic. +- `symphonia-bundle-flac` overflows an `i32` add on 32-bit LPC FLAC; the wrapped value is correct. + Release builds decode correctly; debug builds panic (returned as `Malformed`). Apps that decode + 32-bit FLAC need this in the workspace `Cargo.toml` (craft-libs has it): + + ```toml + [profile.dev.package.symphonia-bundle-flac] + overflow-checks = false + ``` + +## Tests + +Both integration test binaries need features, so a plain `cargo test -p craft-audio` runs only +the unit tests. Run everything with: + +```bash +cargo test -p craft-audio --all-features +``` + +Tests that need ffmpeg are skipped when it is missing. To fail instead, set +`CRAFT_MEDIA_REQUIRE_ORACLES=1`. The helpers and their variables are shared with craft-media (see +[its tests](../media/README.md#tests)). The test files describe what they check; +`tests/codecs/routing_oracle.rs` is the end-to-end check of containers, codecs, timing, seeking +and track choice against ffmpeg. + +CI (`.github/workflows/audio.yml`) runs clippy for each feature on its own, rustdoc with warnings +as errors, the WASM build, encode-only and native-only test runs, and an MSRV check. It also runs +the full tests with ffmpeg required. + +## Origin + +| Part | Source | +|---|---| +| WAV, AIFF, `decode`/`probe`, peaks | SoundCraft `soundcraft-audio-io` [7dc56a4](https://github.com/storytold/soundcraft/commit/7dc56a4dbeebf57eacece09536726c57844fc59d) | + +Where apps had overlapping implementations, the one that measured better was kept, with the +tests of both: + +| Format | Kept | Reason | +|---|---|---| +| WAV/AIFF writing | SoundCraft | only one with BWF, dither, arbitrary speaker masks, streaming without known length | + +## Migration notes for apps + +For the apps switching from their own code; to be removed once all three have switched. + +What an app adjusts: + +- `AudioInfo` has `container`, `codec`, `layout` and `damaged` instead of SoundCraft's `format`. + `detect_container` replaces `detect_format`. `probe` reports the layout instead of + `wav_channel_mask`. +- `EncodeOptions` is built with `EncodeOptions::new(format)` and `with_*` methods. It carries the + speaker layout and RF64, which replace `encode_with_layout`, `encode_wav_rf64` and + `FileEncoder::new`'s layout argument. `BwfInfo` is built the same way. `ENCODE_EXTENSIONS` is a + list. +- Arguments go sample rate, then channels, and channel counts are `u16`: + `FileEncoder::new(out, &opts, sample_rate, channels)`, `wav_float_header(sample_rate, + channels, frames)`, `AudioBuffer::new(sample_rate, channels, frames)`, + `AudioBuffer::from_interleaved` and `num_channels`, and the `ChannelLayout` constructors and + `channels()`. +- `AudioInfo`, `AudioError`, `EncodeOptions`, `BwfInfo`, `SampleFormat` and `BitDepth` are + `#[non_exhaustive]`. + `Speaker::ALL` is a slice. +- `TrackInfo::sample_rate` and `channels` are `Option`s. +- Features are per format. SoundCraft's `encode` is now `lossless-encode`, which turns on + `pcm-encode` (WAV, AIFF). `encode` turns on every encoder. Writing a format + whose feature is off returns `AudioError::Unsupported`. +- `FileEncoder::write` takes any per-channel slices, but refuses a block with missing channels or + short slices instead of filling them with silence. +- `DecodeOptions` is not `Copy`. +- `decode` tries the native path first (WAV, AIFF, and PCM in MP4 and Matroska), which decodes + MP4 and Matroska files to presentation length. A failed source read is `AudioError::Io`. diff --git a/crates/audio/examples/encode.rs b/crates/audio/examples/encode.rs new file mode 100644 index 00000000..a1c7b89d --- /dev/null +++ b/crates/audio/examples/encode.rs @@ -0,0 +1,31 @@ +//! Write a tone as WAV and AIFF, whole and streamed. +//! +//! `cargo run -p craft-audio --example encode --features lossless-encode -- out` + +use craft_audio::{AudioBuffer, BitDepth, ChannelLayout, EncodeFormat, EncodeOptions, FileEncoder, encode}; + +fn main() -> Result<(), Box> { + let dir = std::path::PathBuf::from(std::env::args().nth(1).unwrap_or_else(|| ".".into())); + std::fs::create_dir_all(&dir)?; + let rate = 48_000; + let tone: Vec = (0..rate).map(|i| (i as f32 * 440.0 * std::f32::consts::TAU / rate as f32).sin() * 0.5).collect(); + let stereo = AudioBuffer { sample_rate: rate as u32, channels: vec![tone.clone(), tone.clone()] }; + + // A whole buffer at once. + let wav = encode(&stereo, &EncodeOptions::new(EncodeFormat::Wav).with_bit_depth(BitDepth::Int24).with_dither(true))?; + std::fs::write(dir.join("tone.wav"), wav)?; + + // A 5.1 file whose layout a channel count alone cannot name (side surrounds). + let surround = AudioBuffer { sample_rate: rate as u32, channels: vec![tone.clone(); 6] }; + let wav = encode(&surround, &EncodeOptions::default().with_layout(ChannelLayout::SURROUND_5_1_SIDE))?; + std::fs::write(dir.join("tone-5.1.wav"), wav)?; + + // Streamed: the same bytes, block by block into a file. + let opts = EncodeOptions::new(EncodeFormat::Aiff); + let mut enc = FileEncoder::new(std::fs::File::create(dir.join("tone.aiff"))?, &opts, rate as u32, 2)?; + for chunk in tone.chunks(4096) { + enc.write(&[chunk, chunk], chunk.len())?; + } + enc.finish()?; + Ok(()) +} diff --git a/crates/audio/examples/info.rs b/crates/audio/examples/info.rs new file mode 100644 index 00000000..dd82fd8b --- /dev/null +++ b/crates/audio/examples/info.rs @@ -0,0 +1,25 @@ +//! Print what a file holds, then decode it whole. +//! +//! `cargo run -p craft-audio --example info --features decode -- song.flac` + +fn main() -> Result<(), Box> { + let path = std::env::args().nth(1).ok_or("usage: info ")?; + let bytes = std::fs::read(&path)?; + let ext = std::path::Path::new(&path).extension().and_then(|e| e.to_str()); + + // Container, codec, rate, speakers and length, from the headers where they say. + let info = craft_audio::probe(&bytes, ext)?; + println!("{} in {}, {} Hz, {}, {:.2} s", info.codec.name(), info.container.name(), info.sample_rate, info.layout, info.duration_secs()); + + // Every audio track, for files that have several. + for t in craft_audio::tracks(&bytes, ext)? { + let rate = t.sample_rate.map_or("?".into(), |r| r.to_string()); + let channels = t.channels.map_or("?".into(), |c| c.to_string()); + println!(" track {}: {} {rate} Hz {channels} ch", t.index, t.codec.name()); + } + + // The whole first track: planar f32, WAV channel order. + let (_, buf) = craft_audio::decode(&bytes, ext)?; + println!("decoded {} frames, peak {:.3}", buf.frames(), buf.peak()); + Ok(()) +} diff --git a/crates/audio/examples/packets.rs b/crates/audio/examples/packets.rs new file mode 100644 index 00000000..03fdd588 --- /dev/null +++ b/crates/audio/examples/packets.rs @@ -0,0 +1,34 @@ +//! Demux an MP4 with craft-media and decode its audio packets with `TrackDecoder`: what an app +//! with its own container code (or a network stream) does instead of `decode`. +//! +//! `cargo run -p craft-audio --example packets --features decode -- clip.mov` + +use craft_audio::TrackDecoder; +use craft_media::isobmff; + +fn main() -> Result<(), Box> { + let path = std::env::args().nth(1).ok_or("usage: packets ")?; + let bytes = std::fs::read(&path)?; + + let file = isobmff::open(bytes.as_slice())?; + let track = file.track_of_kind(isobmff::TrackKind::Audio).ok_or("no audio track")?; + let t = &file.tracks[track]; + // The container-neutral codec description every craft-media demuxer produces. + let params = t.codec_params(); + + let packets: Vec> = (0..t.samples.len()).map(|i| file.read_sample(bytes.as_slice(), track, i)).collect::>()?; + // Codecs that find their configuration in the stream read it from the first packets. + let lookahead: Vec<&[u8]> = packets.iter().take(craft_audio::LOOKAHEAD_PACKETS).map(Vec::as_slice).collect(); + let mut dec = TrackDecoder::new(¶ms, &lookahead)?; + println!("{} {} Hz {}", dec.codec().name(), dec.sample_rate(), dec.layout()); + + let mut frames = 0; + for p in &packets { + // A damaged packet is replaced by concealment of its length, so the timeline holds. + let block = dec.decode(p).or_else(|_| dec.conceal(Some(p)))?; + frames += block.frames(); + } + // Container timing (the edit list's priming) is the caller's to apply; `decode` does it. + println!("{} packets, {frames} decoded frames", packets.len()); + Ok(()) +} diff --git a/crates/audio/examples/stream.rs b/crates/audio/examples/stream.rs new file mode 100644 index 00000000..dafca370 --- /dev/null +++ b/crates/audio/examples/stream.rs @@ -0,0 +1,27 @@ +//! Decode a long file block by block from disk, starting at an offset, without loading it. +//! +//! `cargo run -p craft-audio --example stream --features decode -- recording.m4a 60` + +use craft_audio::{DecodeOptions, Decoder}; + +fn main() -> Result<(), Box> { + let mut args = std::env::args().skip(1); + let path = args.next().ok_or("usage: stream [start seconds]")?; + let start: f64 = args.next().map(|s| s.parse()).transpose()?.unwrap_or(0.0); + + let file = std::fs::File::open(&path)?; + let mut dec = Decoder::open(file, None, &DecodeOptions::default())?; + let rate = dec.info().sample_rate; + println!("{} {} Hz {}", dec.info().codec.name(), rate, dec.info().layout); + + // Seeking is sample-exact: the next block starts at exactly this frame. + dec.seek((start * f64::from(rate)) as u64)?; + + let (mut frames, mut peak) = (0usize, 0f32); + while let Some(block) = dec.next_block()? { + frames += block.frames(); + peak = peak.max(block.peak()); + } + println!("{frames} frames from {start} s, peak {peak:.3}"); + Ok(()) +} diff --git a/crates/audio/src/buffer.rs b/crates/audio/src/buffer.rs new file mode 100644 index 00000000..c7ac9319 --- /dev/null +++ b/crates/audio/src/buffer.rs @@ -0,0 +1,72 @@ +//! Planar `f32` audio buffer. + +/// Planar f32 audio. +/// +/// Every channel is expected to hold the same number of frames; [`AudioBuffer::frames`] reports the +/// shortest channel so a ragged buffer never causes out-of-bounds access. +#[derive(Debug, Clone, Default, PartialEq)] +pub struct AudioBuffer { + /// Sample rate in Hz. + pub sample_rate: u32, + /// One `Vec` of samples per channel. + pub channels: Vec>, +} + +impl AudioBuffer { + /// A silent buffer with `channels` channels of `frames` frames each. + pub fn new(sample_rate: u32, channels: u16, frames: usize) -> Self { + Self { sample_rate, channels: vec![vec![0.0; frames]; usize::from(channels)] } + } + + /// Number of frames (the length of the shortest channel; 0 when there are no channels). + pub fn frames(&self) -> usize { + self.channels.iter().map(Vec::len).min().unwrap_or(0) + } + + /// Number of channels. + pub fn num_channels(&self) -> u16 { + u16::try_from(self.channels.len()).unwrap_or(u16::MAX) + } + + /// Duration in seconds (0 when the sample rate is 0). + pub fn duration_secs(&self) -> f64 { + if self.sample_rate == 0 { + return 0.0; + } + self.frames() as f64 / f64::from(self.sample_rate) + } + + /// Interleaved copy of the samples (frame-major: `L0 R0 L1 R1 ...`). + pub fn interleaved(&self) -> Vec { + let frames = self.frames(); + let mut out = Vec::with_capacity(frames.saturating_mul(self.channels.len())); + for f in 0..frames { + for ch in &self.channels { + out.push(ch.get(f).copied().unwrap_or(0.0)); + } + } + out + } + + /// Build a planar buffer from interleaved samples. A trailing partial frame is dropped; + /// `channels == 0` yields an empty buffer. + pub fn from_interleaved(sr: u32, channels: u16, data: &[f32]) -> Self { + let channels = usize::from(channels); + if channels == 0 { + return Self { sample_rate: sr, channels: Vec::new() }; + } + let frames = data.len() / channels; + let mut out: Vec> = (0..channels).map(|_| Vec::with_capacity(frames)).collect(); + for frame in data.chunks_exact(channels) { + for (ch, &s) in out.iter_mut().zip(frame) { + ch.push(s); + } + } + Self { sample_rate: sr, channels: out } + } + + /// Absolute peak over all channels (NaN samples are ignored). + pub fn peak(&self) -> f32 { + self.channels.iter().flat_map(|c| c.iter()).fold(0.0f32, |acc, &s| acc.max(s.abs())) + } +} diff --git a/crates/audio/src/layout.rs b/crates/audio/src/layout.rs new file mode 100644 index 00000000..e4028f0e --- /dev/null +++ b/crates/audio/src/layout.rs @@ -0,0 +1,298 @@ +//! Channel layouts: which speaker each channel of a buffer feeds. +//! +//! A layout is a WAVE_FORMAT_EXTENSIBLE speaker mask (the same bits WAV files store in +//! `dwChannelMask`). Decoded audio is in WAV order: one channel per set bit, lowest bit first, so +//! L R C LFE, then back, side and height channels. A layout with no mask is *unordered*: the +//! channels are not assigned to speakers (Ambisonics, discrete multitrack, or a codec layout with +//! no WAV equivalent), and they come in the order the codec stores them. + +/// A speaker position, with its WAVE_FORMAT_EXTENSIBLE mask bit. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub enum Speaker { + /// Front left (`SPEAKER_FRONT_LEFT`, 0x1). + FrontLeft, + /// Front right (`SPEAKER_FRONT_RIGHT`, 0x2). + FrontRight, + /// Front centre (`SPEAKER_FRONT_CENTER`, 0x4). + FrontCenter, + /// Low-frequency effects (`SPEAKER_LOW_FREQUENCY`, 0x8). + Lfe, + /// Back left (`SPEAKER_BACK_LEFT`, 0x10). + BackLeft, + /// Back right (`SPEAKER_BACK_RIGHT`, 0x20). + BackRight, + /// Front left of centre (`SPEAKER_FRONT_LEFT_OF_CENTER`, 0x40). + FrontLeftOfCenter, + /// Front right of centre (`SPEAKER_FRONT_RIGHT_OF_CENTER`, 0x80). + FrontRightOfCenter, + /// Back centre (`SPEAKER_BACK_CENTER`, 0x100). + BackCenter, + /// Side left (`SPEAKER_SIDE_LEFT`, 0x200). + SideLeft, + /// Side right (`SPEAKER_SIDE_RIGHT`, 0x400). + SideRight, + /// Top centre (`SPEAKER_TOP_CENTER`, 0x800). + TopCenter, + /// Top front left (`SPEAKER_TOP_FRONT_LEFT`, 0x1000). + TopFrontLeft, + /// Top front centre (`SPEAKER_TOP_FRONT_CENTER`, 0x2000). + TopFrontCenter, + /// Top front right (`SPEAKER_TOP_FRONT_RIGHT`, 0x4000). + TopFrontRight, + /// Top back left (`SPEAKER_TOP_BACK_LEFT`, 0x8000). + TopBackLeft, + /// Top back centre (`SPEAKER_TOP_BACK_CENTER`, 0x10000). + TopBackCenter, + /// Top back right (`SPEAKER_TOP_BACK_RIGHT`, 0x20000). + TopBackRight, +} + +impl Speaker { + /// Every position, in mask-bit (and therefore WAV channel) order. + pub const ALL: &[Speaker] = &[ + Speaker::FrontLeft, + Speaker::FrontRight, + Speaker::FrontCenter, + Speaker::Lfe, + Speaker::BackLeft, + Speaker::BackRight, + Speaker::FrontLeftOfCenter, + Speaker::FrontRightOfCenter, + Speaker::BackCenter, + Speaker::SideLeft, + Speaker::SideRight, + Speaker::TopCenter, + Speaker::TopFrontLeft, + Speaker::TopFrontCenter, + Speaker::TopFrontRight, + Speaker::TopBackLeft, + Speaker::TopBackCenter, + Speaker::TopBackRight, + ]; + + /// The WAVE_FORMAT_EXTENSIBLE mask bit (`SPEAKER_FRONT_LEFT` = 0x1, …). + pub fn mask_bit(self) -> u32 { + let index = Speaker::ALL.iter().position(|&s| s == self).unwrap_or(0); + 1 << index + } + + /// A short label: `"L"`, `"R"`, `"C"`, `"LFE"`, `"Ls"`, … + pub fn label(self) -> &'static str { + match self { + Speaker::FrontLeft => "L", + Speaker::FrontRight => "R", + Speaker::FrontCenter => "C", + Speaker::Lfe => "LFE", + Speaker::BackLeft => "Lb", + Speaker::BackRight => "Rb", + Speaker::FrontLeftOfCenter => "Lc", + Speaker::FrontRightOfCenter => "Rc", + Speaker::BackCenter => "Cb", + Speaker::SideLeft => "Ls", + Speaker::SideRight => "Rs", + Speaker::TopCenter => "Tc", + Speaker::TopFrontLeft => "Tfl", + Speaker::TopFrontCenter => "Tfc", + Speaker::TopFrontRight => "Tfr", + Speaker::TopBackLeft => "Tbl", + Speaker::TopBackCenter => "Tbc", + Speaker::TopBackRight => "Tbr", + } + } +} + +/// The bits [`Speaker::ALL`] covers. +const KNOWN_BITS: u32 = (1 << 18) - 1; + +/// How the channels of a buffer map to speakers. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize), serde(try_from = "RawLayout", into = "RawLayout"))] +pub struct ChannelLayout { + channels: u16, + /// 0 when unordered; otherwise exactly `channels` bits. + mask: u32, +} + +/// The serialized form of a [`ChannelLayout`], checked on the way in. +#[cfg(feature = "serde")] +#[derive(serde::Serialize, serde::Deserialize)] +struct RawLayout { + channels: u16, + mask: u32, +} + +#[cfg(feature = "serde")] +impl From for RawLayout { + fn from(l: ChannelLayout) -> Self { + RawLayout { channels: l.channels, mask: l.mask } + } +} + +#[cfg(feature = "serde")] +impl TryFrom for ChannelLayout { + type Error = String; + + fn try_from(r: RawLayout) -> Result { + let ordered = r.mask & !KNOWN_BITS == 0 && r.mask.count_ones() == u32::from(r.channels); + if r.mask == 0 || ordered { + Ok(ChannelLayout { channels: r.channels, mask: r.mask }) + } else { + Err(format!("speaker mask {:#x} does not describe {} channels", r.mask, r.channels)) + } + } +} + +/// A channel count as the API holds it, saturating at `u16::MAX`. +#[cfg(any(feature = "decode-native", feature = "pcm-encode"))] +pub(crate) fn count(n: usize) -> u16 { + u16::try_from(n).unwrap_or(u16::MAX) +} + +impl ChannelLayout { + /// C. + pub const MONO: ChannelLayout = ChannelLayout { channels: 1, mask: 0x4 }; + /// L R. + pub const STEREO: ChannelLayout = ChannelLayout { channels: 2, mask: 0x3 }; + /// L R C. + pub const SURROUND_3_0: ChannelLayout = ChannelLayout { channels: 3, mask: 0x7 }; + /// L R Lb Rb. + pub const QUAD: ChannelLayout = ChannelLayout { channels: 4, mask: 0x33 }; + /// L R C Lb Rb. + pub const SURROUND_5_0: ChannelLayout = ChannelLayout { channels: 5, mask: 0x37 }; + /// L R C LFE Lb Rb. + pub const SURROUND_5_1: ChannelLayout = ChannelLayout { channels: 6, mask: 0x3F }; + /// L R C LFE Ls Rs (Dolby's 5.1, with side surrounds). + pub const SURROUND_5_1_SIDE: ChannelLayout = ChannelLayout { channels: 6, mask: 0x60F }; + /// L R C LFE Cb Ls Rs. + pub const SURROUND_6_1: ChannelLayout = ChannelLayout { channels: 7, mask: 0x70F }; + /// L R C LFE Lb Rb Ls Rs. + pub const SURROUND_7_1: ChannelLayout = ChannelLayout { channels: 8, mask: 0x63F }; + + /// The layout of a speaker mask: one channel per set bit. Bits above the 18 standard positions + /// make it unordered. + pub fn from_mask(mask: u32) -> ChannelLayout { + let channels = mask.count_ones() as u16; + if mask & !KNOWN_BITS != 0 { ChannelLayout::unordered(channels) } else { ChannelLayout { channels, mask } } + } + + /// `channels` channels described by `mask`, as WAV files store them: when the mask names a + /// different number of speakers than there are channels, the channels are unordered. + pub fn from_mask_and_channels(mask: u32, channels: u16) -> ChannelLayout { + let layout = ChannelLayout::from_mask(mask); + if mask != 0 && layout.is_ordered() && layout.channels() == channels { layout } else { ChannelLayout::unordered(channels) } + } + + /// The usual layout for a channel count, as WAV readers assume when a file has no mask: mono, + /// stereo, L R C, quad, 5.0, 5.1, 6.1 (with a back centre), 7.1 (backs and sides), 7.1.4. + /// Other counts are unordered. + pub fn default_for(channels: u16) -> ChannelLayout { + let mask = match channels { + 1 => 0x4, + 2 => 0x3, + 3 => 0x7, + 4 => 0x33, + 5 => 0x37, + 6 => 0x3F, + 7 => 0x13F, + 8 => 0x63F, + 12 => 0x2_D63F, + _ => 0, + }; + if mask == 0 { ChannelLayout::unordered(channels) } else { ChannelLayout::from_mask(mask) } + } + + /// `channels` channels with no speaker assignment. + pub fn unordered(channels: u16) -> ChannelLayout { + ChannelLayout { channels, mask: 0 } + } + + /// The number of channels. + pub fn channels(&self) -> u16 { + self.channels + } + + /// The speaker mask, or `None` for an unordered layout. + pub fn mask(&self) -> Option { + (self.mask != 0).then_some(self.mask) + } + + /// Whether the channels are assigned to speakers (and therefore in WAV order). + pub fn is_ordered(&self) -> bool { + self.mask != 0 + } + + /// The speaker of each channel in order; empty for an unordered layout. + pub fn speakers(&self) -> Vec { + Speaker::ALL.iter().copied().filter(|s| self.mask & s.mask_bit() != 0).collect() + } + + /// The channel that feeds `speaker`, if the layout has it. + pub fn index_of(&self, speaker: Speaker) -> Option { + self.speakers().iter().position(|&s| s == speaker) + } + + /// A conventional name: `"mono"`, `"stereo"`, `"5.1"`, `"7.1"`; `None` for other layouts. + pub fn name(&self) -> Option<&'static str> { + Some(match self.mask { + 0x4 => "mono", + 0x3 => "stereo", + 0xB => "2.1", + 0x7 => "3.0", + 0x33 => "quad", + 0x603 => "quad (side)", + 0x37 | 0x607 => "5.0", + 0x3F | 0x60F => "5.1", + 0x13F | 0x70F => "6.1", + 0x63F | 0xFF => "7.1", + 0x2_D63F => "7.1.4", + _ => return None, + }) + } +} + +impl std::fmt::Display for ChannelLayout { + /// The conventional name, or the channel count: `"5.1"`, `"10 channels"`. + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self.name() { + Some(name) => f.write_str(name), + None if self.channels == 1 => f.write_str("1 channel"), + None => write!(f, "{} channels", self.channels), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn speakers_follow_the_mask_in_wav_order() { + assert_eq!( + ChannelLayout::SURROUND_5_1_SIDE.speakers(), + [Speaker::FrontLeft, Speaker::FrontRight, Speaker::FrontCenter, Speaker::Lfe, Speaker::SideLeft, Speaker::SideRight] + ); + assert_eq!(ChannelLayout::SURROUND_7_1.index_of(Speaker::SideLeft), Some(6)); + assert_eq!(Speaker::SideRight.mask_bit(), 0x400); + } + + #[test] + fn a_mask_that_does_not_match_the_channels_is_unordered() { + assert_eq!(ChannelLayout::from_mask_and_channels(0x3F, 6), ChannelLayout::SURROUND_5_1); + assert!(!ChannelLayout::from_mask_and_channels(0x3, 4).is_ordered()); + assert!(!ChannelLayout::from_mask_and_channels(0, 4).is_ordered()); + assert!(!ChannelLayout::from_mask(1 << 20).is_ordered()); + assert_eq!(ChannelLayout::unordered(4).channels(), 4); + assert!(ChannelLayout::unordered(4).speakers().is_empty()); + } + + #[test] + fn defaults_and_names() { + assert_eq!(ChannelLayout::default_for(6), ChannelLayout::SURROUND_5_1); + assert!(!ChannelLayout::default_for(9).is_ordered()); + assert_eq!(ChannelLayout::default_for(2).to_string(), "stereo"); + assert_eq!(ChannelLayout::SURROUND_5_1_SIDE.to_string(), "5.1"); + assert_eq!(ChannelLayout::unordered(10).to_string(), "10 channels"); + } +} diff --git a/crates/audio/src/lib.rs b/crates/audio/src/lib.rs new file mode 100644 index 00000000..942a141b --- /dev/null +++ b/crates/audio/src/lib.rs @@ -0,0 +1,319 @@ +#![deny(clippy::unwrap_used, clippy::expect_used, clippy::panic, clippy::unimplemented, clippy::todo, clippy::unreachable)] +#![warn(missing_docs)] +// The docs link items of every feature; built with fewer features, those links have no target. +// CI and docs.rs build the docs with all features, where every link must resolve. +#![cfg_attr(not(all(feature = "decode", feature = "encode", feature = "peaks")), allow(rustdoc::broken_intra_doc_links))] +//! Audio file I/O and codecs for the Craft apps. Input and output are byte slices or a +//! [`craft_media::ByteSource`] (no filesystem access of its own); builds for +//! `wasm32-unknown-unknown`. +//! +//! - [`decode`], [`probe`], [`Decoder`] (streaming, seekable), [`tracks`], [`detect_container`] +//! (feature `decode`): WAV and AIFF are read natively; PCM in MP4/MOV and Matroska/WebM goes +//! through craft-media's demuxers and [`TrackDecoder`]; everything else (AAC, MP1/MP2/MP3, +//! FLAC, ALAC, Vorbis, CAF, ADPCM, A-law, µ-law) through `symphonia`, which is to be replaced +//! by native decoders. Feature `decode-native` is the same without symphonia. +//! - [`TrackDecoder`] (feature `decode-native`): one track's packets from any demuxer to planar +//! `f32` in WAV channel order. +//! - [`encode`], [`FileEncoder`]: WAV/BWF/RF64 and AIFF/AIFF-C (feature `pcm-encode`, which also +//! gives [`wav_float_header`]), whole-buffer or incremental into any `Write + Seek`. Features +//! `lossless-encode` and `encode` turn it on. +//! - [`peaks`] (feature `peaks`): min/max/RMS pyramid for waveform drawing. +//! +//! The shared types below build with no features. + +mod buffer; +mod layout; +#[cfg(feature = "decode-native")] +mod pcm; +#[cfg(feature = "peaks")] +pub mod peaks; +#[cfg(feature = "decode-native")] +mod read; +#[cfg(feature = "decode-native")] +mod track; +#[cfg(any(feature = "decode-native", feature = "pcm-encode"))] +mod wav; +#[cfg(feature = "pcm-encode")] +mod write; + +pub use buffer::AudioBuffer; +/// The craft-media this crate is built with, for its containers: using it rather than a separate +/// dependency keeps the types the two crates share identical. +pub use craft_media; +pub use craft_media::{ByteSource, CodecId, CodecParams, Container, PcmFormat, ReadSeekSource}; +pub use layout::{ChannelLayout, Speaker}; +#[cfg(feature = "decode-native")] +pub use read::SharedSource; +#[cfg(feature = "decode-native")] +pub use read::{DEFAULT_MAX_SAMPLES, DecodeOptions, Decoder, TrackInfo, decode, decode_with, detect_container, probe, probe_with, tracks, tracks_of}; +#[cfg(feature = "decode-native")] +pub use track::{LOOKAHEAD_PACKETS, TrackDecoder}; +#[cfg(feature = "pcm-encode")] +pub use write::{ENCODE_EXTENSIONS, FileEncoder, encode, encode_format_for, wav_float_header}; + +/// Errors from probing, decoding and encoding. +#[non_exhaustive] +#[derive(Debug, thiserror::Error)] +pub enum AudioError { + /// The format, codec or option is valid but not supported. + #[error("unsupported audio: {0}")] + Unsupported(String), + /// The input is corrupt or not what it claims to be. + #[error("malformed audio: {0}")] + Malformed(String), + /// The input (or requested output) exceeds a size limit. + #[error("audio too large: {0}")] + TooLarge(String), + /// Encoding failed (e.g. an empty buffer). + #[error("audio encode failed: {0}")] + Encode(String), + /// Reading the input or writing the output failed (a source that stopped serving reads, a + /// full disk). Bytes missing because the input is truncated are [`AudioError::Malformed`]. + #[error("audio I/O failed: {0}")] + Io(#[from] std::io::Error), +} + +/// Result alias for this module. +pub type Result = std::result::Result; + +/// The file formats [`encode`] writes (feature `pcm-encode`). +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub enum EncodeFormat { + /// WAV / BWF, RF64 above 4 GiB. + Wav, + /// AIFF (integer depths) or AIFF-C `fl32` (float). + Aiff, +} + +impl EncodeFormat { + /// The container this format writes. + pub fn container(self) -> Container { + match self { + EncodeFormat::Wav => Container::Wav, + EncodeFormat::Aiff => Container::Aiff, + } + } +} + +/// Stored sample representation. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub enum SampleFormat { + /// Integer PCM, 1-8 bits. + Int8, + /// Integer PCM, 9-16 bits. + Int16, + /// Integer PCM, 17-24 bits. + Int24, + /// Integer PCM, 25-32 bits. + Int32, + /// 32-bit IEEE float. + Float32, + /// 64-bit IEEE float. + Float64, + /// A lossy codec (MP3, AAC, Vorbis, ...): no fixed bit depth. + Compressed, +} + +/// Broadcast Wave (`bext`) metadata. +/// +/// `#[non_exhaustive]`: build it with `BwfInfo::default()` and the `with_*` methods. +#[non_exhaustive] +#[derive(Debug, Clone, Default, PartialEq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub struct BwfInfo { + /// `Description` (up to 256 bytes). + pub description: String, + /// `Originator` (up to 32 bytes). + pub originator: String, + /// `OriginatorReference` (up to 32 bytes). + pub originator_reference: String, + /// `yyyy-mm-dd`. + pub origination_date: String, + /// `hh:mm:ss`. + pub origination_time: String, + /// Sample count since midnight of the first sample (the timestamp Pro Tools spots to). + pub time_reference: u64, +} + +impl BwfInfo { + /// Sets `description`. + pub fn with_description(mut self, description: impl Into) -> Self { + self.description = description.into(); + self + } + + /// Sets `originator`. + pub fn with_originator(mut self, originator: impl Into) -> Self { + self.originator = originator.into(); + self + } + + /// Sets `originator_reference`. + pub fn with_originator_reference(mut self, reference: impl Into) -> Self { + self.originator_reference = reference.into(); + self + } + + /// `date` as `yyyy-mm-dd`, `time` as `hh:mm:ss`. + pub fn with_origination(mut self, date: impl Into, time: impl Into) -> Self { + (self.origination_date, self.origination_time) = (date.into(), time.into()); + self + } + + /// Sets `time_reference`. + pub fn with_time_reference(mut self, time_reference: u64) -> Self { + self.time_reference = time_reference; + self + } +} + +/// What `probe` learns about a file without decoding all of it. +/// +/// `#[non_exhaustive]`: fields may be added, so it is only built by this crate. +#[non_exhaustive] +#[derive(Debug, Clone, PartialEq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub struct AudioInfo { + /// The file format around the audio. + pub container: Container, + /// The codec of the audio track ([`CodecId::Pcm`] for uncompressed audio). + pub codec: CodecId, + /// The stored sample representation. + pub sample_format: SampleFormat, + /// The decoded sample rate (for HE-AAC, the rate after spectral band replication). + pub sample_rate: u32, + /// Decoded channel count. + pub channels: u16, + /// Which speaker each decoded channel feeds. Decoded audio is in WAV channel order when the + /// layout is ordered. + pub layout: ChannelLayout, + /// Length in frames (samples per channel), after container trimming. + pub frames: u64, + /// Broadcast Wave metadata from a WAV `bext` chunk. + pub bwf: Option, + /// Packets that were damaged or failed to decode and were replaced by concealment (silence of + /// their length), counted as audio is decoded: by [`decode`] for the whole file, and so far by + /// a [`Decoder`]. 0 from [`probe`] when it read only headers. + pub damaged: u64, +} + +impl AudioInfo { + /// Info for a stream with `layout`'s channels and no BWF metadata. + #[cfg(feature = "decode-native")] + pub(crate) fn new( + container: Container, + codec: CodecId, + sample_format: SampleFormat, + sample_rate: u32, + layout: ChannelLayout, + frames: u64, + ) -> AudioInfo { + let channels = layout.channels(); + AudioInfo { container, codec, sample_format, sample_rate, channels, layout, frames, bwf: None, damaged: 0 } + } + + /// Duration in seconds. + pub fn duration_secs(&self) -> f64 { + if self.sample_rate == 0 { 0.0 } else { self.frames as f64 / f64::from(self.sample_rate) } + } +} + +/// Bit depth for `encode`. +#[non_exhaustive] +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub enum BitDepth { + /// 16-bit integer. + Int16, + /// 24-bit integer. + Int24, + /// 32-bit integer. + Int32, + /// Preserve finite samples, including values above full scale; map NaN to 0 and infinities to ±1. + Float32, +} + +/// Options for `encode` and `FileEncoder`. +/// +/// `#[non_exhaustive]`: build it with [`EncodeOptions::new`] and the `with_*` methods. +/// +/// ``` +/// use craft_audio::{BitDepth, EncodeFormat, EncodeOptions}; +/// +/// let opts = EncodeOptions::new(EncodeFormat::Aiff).with_bit_depth(BitDepth::Int16).with_dither(true); +/// assert_eq!(opts.format, EncodeFormat::Aiff); +/// ``` +#[non_exhaustive] +#[derive(Debug, Clone, PartialEq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub struct EncodeOptions { + /// Output file format. + pub format: EncodeFormat, + /// Sample depth to write. + pub bit_depth: BitDepth, + /// Add TPDF dither when quantizing to an integer depth. + pub dither: bool, + /// Write a BWF `bext` chunk (WAV only; ignored for other formats). + pub bwf: Option, + /// The speaker layout to record, for layouts a channel count alone cannot identify (5.1 with + /// side vs back surrounds, 7.0 vs 6.1, Ambisonics as [`ChannelLayout::unordered`]). `None` is + /// the usual layout for the channel count. WAV stores it as `dwChannelMask`; AIFF and FLAC + /// have fixed channel orders and ignore it. It must have as many channels as the audio. + pub layout: Option, + /// Write WAV in the RF64 layout even below 4 GiB (WAV only; ignored for other formats). + pub force_rf64: bool, +} + +impl EncodeOptions { + /// Options for `format`: 24-bit, no dither, no BWF chunk, the usual layout. + pub fn new(format: EncodeFormat) -> Self { + Self { format, bit_depth: BitDepth::Int24, dither: false, bwf: None, layout: None, force_rf64: false } + } + + /// Sets `format`. + pub fn with_format(mut self, format: EncodeFormat) -> Self { + self.format = format; + self + } + + /// Sets `bit_depth`. + pub fn with_bit_depth(mut self, bit_depth: BitDepth) -> Self { + self.bit_depth = bit_depth; + self + } + + /// Sets `dither`. + pub fn with_dither(mut self, dither: bool) -> Self { + self.dither = dither; + self + } + + /// Sets `bwf`. + pub fn with_bwf(mut self, bwf: BwfInfo) -> Self { + self.bwf = Some(bwf); + self + } + + /// Sets `layout`. + pub fn with_layout(mut self, layout: ChannelLayout) -> Self { + self.layout = Some(layout); + self + } + + /// Sets `force_rf64`. + pub fn with_force_rf64(mut self, force_rf64: bool) -> Self { + self.force_rf64 = force_rf64; + self + } +} + +impl Default for EncodeOptions { + /// 24-bit WAV. + fn default() -> Self { + Self::new(EncodeFormat::Wav) + } +} diff --git a/crates/audio/src/pcm.rs b/crates/audio/src/pcm.rs new file mode 100644 index 00000000..397f0c07 --- /dev/null +++ b/crates/audio/src/pcm.rs @@ -0,0 +1,156 @@ +//! Byte-level PCM helpers shared by the WAV, AIFF and FLAC readers: bounded readers and sample +//! decoding to `f32`. + +use crate::{AudioError, Result}; + +/// Maximum number of decoded `f32` samples (all channels) we will produce: 4 GiB of output. +pub(crate) const MAX_DECODE_SAMPLES: u64 = (4u64 << 30) / 4; + +/// Little-endian `u16` at `pos`. +#[cfg(feature = "decode-native")] +pub(crate) fn le_u16(b: &[u8], pos: usize) -> Option { + let s = b.get(pos..pos.checked_add(2)?)?; + Some(u16::from_le_bytes([*s.first()?, *s.get(1)?])) +} + +/// Little-endian `u32` at `pos`. +#[cfg(feature = "decode-native")] +pub(crate) fn le_u32(b: &[u8], pos: usize) -> Option { + let s: [u8; 4] = b.get(pos..pos.checked_add(4)?)?.try_into().ok()?; + Some(u32::from_le_bytes(s)) +} + +/// Little-endian `u64` at `pos`. +#[cfg(feature = "decode-native")] +pub(crate) fn le_u64(b: &[u8], pos: usize) -> Option { + let s: [u8; 8] = b.get(pos..pos.checked_add(8)?)?.try_into().ok()?; + Some(u64::from_le_bytes(s)) +} + +/// Big-endian `u16` at `pos`. +#[cfg(feature = "decode-native")] +pub(crate) fn be_u16(b: &[u8], pos: usize) -> Option { + let s: [u8; 2] = b.get(pos..pos.checked_add(2)?)?.try_into().ok()?; + Some(u16::from_be_bytes(s)) +} + +/// Big-endian `u32` at `pos`. +#[cfg(feature = "decode-native")] +pub(crate) fn be_u32(b: &[u8], pos: usize) -> Option { + let s: [u8; 4] = b.get(pos..pos.checked_add(4)?)?.try_into().ok()?; + Some(u32::from_be_bytes(s)) +} + +/// Big-endian `u64` at `pos`. +#[cfg(feature = "symphonia")] +#[cfg(feature = "decode-native")] +pub(crate) fn be_u64(b: &[u8], pos: usize) -> Option { + let s: [u8; 8] = b.get(pos..pos.checked_add(8)?)?.try_into().ok()?; + Some(u64::from_be_bytes(s)) +} + +/// Four-byte tag at `pos`. +#[cfg(feature = "decode-native")] +pub(crate) fn tag(b: &[u8], pos: usize) -> Option<[u8; 4]> { + b.get(pos..pos.checked_add(4)?)?.try_into().ok() +} + +/// How the samples in a PCM byte stream are encoded. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub(crate) enum PcmKind { + /// Signed (or, for 1-byte WAV, unsigned) integers occupying `bytes` bytes each, left-justified. + Int { bytes: usize, big_endian: bool, unsigned: bool }, + /// IEEE float, 4 or 8 bytes. + Float { bytes: usize, big_endian: bool }, +} + +impl PcmKind { + pub(crate) fn bytes(self) -> usize { + match self { + PcmKind::Int { bytes, .. } | PcmKind::Float { bytes, .. } => bytes, + } + } +} + +/// Decode one sample from exactly `kind.bytes()` bytes. Returns 0 for a short slice. +fn decode_sample(s: &[u8], kind: PcmKind) -> f32 { + match kind { + PcmKind::Int { bytes, big_endian, unsigned } => { + if bytes == 0 || bytes > 4 || s.len() < bytes { + return 0.0; + } + let mut acc: u32 = 0; + for i in 0..bytes { + let idx = if big_endian { i } else { bytes - 1 - i }; + let mut byte = s.get(idx).copied().unwrap_or(0); + if unsigned && i == 0 { + byte ^= 0x80; + } + acc = (acc << 8) | u32::from(byte); + } + let shifted = acc << (32 - 8 * bytes as u32); + (f64::from(shifted as i32) / 2_147_483_648.0) as f32 + } + PcmKind::Float { bytes, big_endian } => { + let v = match bytes { + 4 => { + let Some(a) = s.get(..4).and_then(|x| <[u8; 4]>::try_from(x).ok()) else { return 0.0 }; + if big_endian { f32::from_be_bytes(a) } else { f32::from_le_bytes(a) } + } + 8 => { + let Some(a) = s.get(..8).and_then(|x| <[u8; 8]>::try_from(x).ok()) else { return 0.0 }; + (if big_endian { f64::from_be_bytes(a) } else { f64::from_le_bytes(a) }) as f32 + } + _ => 0.0, + }; + if v.is_finite() { v } else { 0.0 } + } + } +} + +/// `channels` empty channels with room for `frames` samples each, allocated fallibly. +pub(crate) fn planar(channels: usize, frames: usize) -> Result>> { + let too_large = || AudioError::TooLarge("cannot allocate decoded audio".into()); + let mut out = Vec::new(); + out.try_reserve_exact(channels).map_err(|_| too_large())?; + for _ in 0..channels { + let mut ch = Vec::new(); + ch.try_reserve_exact(frames).map_err(|_| too_large())?; + out.push(ch); + } + Ok(out) +} + +/// `channels` channels of `frames` samples of silence, allocated fallibly and held to the decode +/// limit like decoded audio. +pub(crate) fn silence(channels: usize, frames: usize) -> Result>> { + let total = (frames as u64).saturating_mul(channels as u64); + if total > MAX_DECODE_SAMPLES { + return Err(AudioError::TooLarge(format!("{total} samples of concealment exceed the {MAX_DECODE_SAMPLES}-sample decode limit"))); + } + let mut out = planar(channels, frames)?; + for ch in &mut out { + ch.resize(frames, 0.0); + } + Ok(out) +} + +/// Deinterleave `frames` frames of `channels`-channel PCM from `data` into planar `f32`. +pub(crate) fn deinterleave(data: &[u8], kind: PcmKind, channels: usize, frames: u64) -> Result>> { + let bytes = kind.bytes(); + let block = bytes.checked_mul(channels).filter(|&b| b > 0).ok_or_else(|| AudioError::Malformed("zero-sized sample frame".into()))?; + let available = (data.len() / block) as u64; + let frames = frames.min(available); + let total = frames.checked_mul(channels as u64).ok_or_else(|| AudioError::TooLarge("sample count overflows".into()))?; + if total > MAX_DECODE_SAMPLES { + return Err(AudioError::TooLarge(format!("{total} samples exceeds the {MAX_DECODE_SAMPLES}-sample decode limit"))); + } + let frames = usize::try_from(frames).map_err(|_| AudioError::TooLarge("frame count exceeds address space".into()))?; + let mut out = planar(channels, frames)?; + for frame in data.chunks_exact(block).take(frames) { + for (ch, s) in out.iter_mut().zip(frame.chunks_exact(bytes)) { + ch.push(decode_sample(s, kind)); + } + } + Ok(out) +} diff --git a/crates/audio/src/peaks.rs b/crates/audio/src/peaks.rs new file mode 100644 index 00000000..16a6c14b --- /dev/null +++ b/crates/audio/src/peaks.rs @@ -0,0 +1,285 @@ +//! Multi-resolution overview for waveform drawing. +//! +//! Level 0 sums up each block of [`BLOCK`] samples (min, max and sum of squares); each following +//! level merges [`FANOUT`] blocks of the level below, until a level has a single block. Range +//! queries combine the coarsest whole blocks that fit inside the range and read the partial blocks +//! at its edges from the samples, so a query is exact and costs O(log n) regardless of zoom. + +/// Samples per level-0 block. +pub const BLOCK: usize = 32; +/// Blocks of level `n` merged into one block of level `n + 1`. +pub const FANOUT: usize = 4; +/// Upper bound on the number of spans [`Peaks::spans`] will produce. +pub const MAX_COLUMNS: usize = 1 << 20; + +/// Multi-resolution overview of one channel for waveform drawing. +#[derive(Debug, Clone, Default, PartialEq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub struct Peaks { + len: usize, + levels: Vec>, +} + +/// What a run of samples sums up to; NaN samples count for nothing. +#[derive(Debug, Clone, Copy, PartialEq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +struct Block { + min: f32, + max: f32, + /// In f64: the square of a loud float sample, or the sum of many, can overflow f32. + squares: f64, +} + +impl Block { + const EMPTY: Block = Block { min: f32::INFINITY, max: f32::NEG_INFINITY, squares: 0.0 }; + + fn point(s: f32) -> Block { + Block { min: s, max: s, squares: if s.is_finite() { f64::from(s) * f64::from(s) } else { 0.0 } } + } + + fn of(samples: &[f32]) -> Block { + samples.iter().fold(Block::EMPTY, |b, &s| b.merge(Block::point(s))) + } + + fn merge(self, o: Block) -> Block { + Block { min: self.min.min(o.min), max: self.max.max(o.max), squares: self.squares + o.squares } + } + + /// (min, max), or (0, 0) when nothing was summed up. + fn range(self) -> (f32, f32) { + if self.min > self.max { (0.0, 0.0) } else { (self.min, self.max) } + } +} + +impl Peaks { + /// Build the overview of `samples` (one channel). NaN samples are ignored. + pub fn build(samples: &[f32]) -> Peaks { + if samples.is_empty() { + return Peaks::default(); + } + let mut levels: Vec> = vec![samples.chunks(BLOCK).map(Block::of).collect()]; + while let Some(last) = levels.last().filter(|l| l.len() > 1) { + let next = last.chunks(FANOUT).map(|c| c.iter().fold(Block::EMPTY, |a, &b| a.merge(b))).collect(); + levels.push(next); + } + Peaks { len: samples.len(), levels } + } + + /// Number of samples the overview was built from. + pub fn len_samples(&self) -> usize { + self.len + } + + /// Number of resolution levels (0 for an empty overview). + pub fn num_levels(&self) -> usize { + self.levels.len() + } + + /// (min, max) over the sample range `[start, end)`, clamped to the data. `samples` must be the + /// data the overview was built from. Returns `(0.0, 0.0)` for an empty range. + pub fn minmax(&self, samples: &[f32], start: usize, end: usize) -> (f32, f32) { + self.sum(samples, start, end).range() + } + + fn sum(&self, samples: &[f32], start: usize, end: usize) -> Block { + let end = end.min(self.len).min(samples.len()); + let (first, last) = (start.div_ceil(BLOCK), end / BLOCK); + let raw = |a: usize, b: usize| Block::of(samples.get(a..b).unwrap_or_default()); + if first >= last { + return raw(start, end); + } + raw(start, first * BLOCK).merge(raw(last * BLOCK, end)).merge(self.blocks(first, last)) + } + + /// The level-0 blocks `[lo, hi)`, summed from the coarsest blocks that fit. + fn blocks(&self, mut lo: usize, mut hi: usize) -> Block { + let mut acc = Block::EMPTY; + for (lvl, level) in self.levels.iter().enumerate() { + let mut add = |i: usize| acc = acc.merge(level.get(i).copied().unwrap_or(Block::EMPTY)); + if lvl + 1 == self.levels.len() || hi.saturating_sub(lo) < 2 * FANOUT { + (lo..hi).for_each(add); + break; + } + while !lo.is_multiple_of(FANOUT) { + add(lo); + lo += 1; + } + while !hi.is_multiple_of(FANOUT) { + hi -= 1; + add(hi); + } + lo /= FANOUT; + hi /= FANOUT; + } + acc + } + + /// The spans of sample positions `[start + c * step, start + (c + 1) * step + overlap]` that + /// reach the data, clamped to it. + fn windows(&self, samples: &[f32], start: f64, step: f64, overlap: f64, cols: usize) -> impl Iterator> { + let n = self.len.min(samples.len()); + let valid = n > 0 && start.is_finite() && step.is_finite() && step > 0.0 && overlap.is_finite() && overlap >= 0.0; + let last = n.saturating_sub(1) as f64; + (0..cols.min(MAX_COLUMNS)).map(move |c| { + let (a, b) = (start + step * c as f64, start + step * (c + 1) as f64 + overlap); + (valid && b >= 0.0 && a <= last).then(|| (a.max(0.0), b.min(last))) + }) + } + + /// (min, max) of the line through `samples` over each of `cols` spans of sample positions + /// `[start + c * step, start + (c + 1) * step + overlap]`; fractional positions interpolate + /// between samples. Neighbouring spans share at least their boundary, so a waveform drawn from + /// them stays connected at any zoom. `None` for spans outside the data. + pub fn spans(&self, samples: &[f32], start: f64, step: f64, overlap: f64, cols: usize) -> Vec> { + let at = |t: f64| { + let i = t.floor(); + let a = samples.get(i as usize).copied().unwrap_or(0.0); + let b = samples.get(i as usize + 1).copied().unwrap_or(a); + Block::point(a + (b - a) * (t - i) as f32) + }; + self.windows(samples, start, step, overlap, cols) + .map(|w| w.map(|(a, b)| self.sum(samples, a.ceil() as usize, b.floor() as usize + 1).merge(at(a)).merge(at(b)).range())) + .collect() + } + + /// The RMS of `samples` over each span [`Peaks::spans`] gives, including the samples just + /// outside a span's ends. + pub fn rms_spans(&self, samples: &[f32], start: f64, step: f64, overlap: f64, cols: usize) -> Vec> { + self.windows(samples, start, step, overlap, cols) + .map(|w| { + w.map(|(a, b)| { + let (s, e) = (a.floor() as usize, (b.ceil() as usize).saturating_add(1).min(self.len).min(samples.len())); + (self.sum(samples, s, e).squares / e.saturating_sub(s).max(1) as f64).sqrt() as f32 + }) + }) + .collect() + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn brute(samples: &[f32], start: usize, end: usize) -> (f32, f32) { + let s = &samples[start..end]; + (s.iter().copied().fold(f32::INFINITY, f32::min), s.iter().copied().fold(f32::NEG_INFINITY, f32::max)) + } + + fn signal(n: usize) -> Vec { + (0..n).map(|i| ((i as f32 * 0.037).sin() * (i as f32 * 0.0011).cos()) + if i % 997 == 0 { 0.5 } else { 0.0 }).collect() + } + + #[test] + fn every_range_is_exact() { + let x = signal(100_003); + let p = Peaks::build(&x); + for &(a, b) in &[(0, 32), (0, 100_003), (32, 64_000), (3200, 3232), (96, 99_968), (64, 100_003), (5, 6), (31, 33), (1000, 1031)] { + assert_eq!(p.minmax(&x, a, b), brute(&x, a, b), "range {a}..{b}"); + } + let mut seed = 12345u64; + for _ in 0..2000 { + seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + let a = (seed >> 33) as usize % 100_003; + seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407); + let b = a + 1 + (seed >> 33) as usize % (100_003 - a); + assert_eq!(p.minmax(&x, a, b), brute(&x, a, b), "range {a}..{b}"); + } + } + + #[test] + fn degenerate_inputs() { + let p = Peaks::build(&[]); + assert_eq!(p.len_samples(), 0); + assert_eq!(p.minmax(&[], 0, 10), (0.0, 0.0)); + assert_eq!(p.spans(&[], 0.0, 1.0, 0.0, 4), vec![None; 4]); + let x = [0.5, -0.25]; + let p = Peaks::build(&x); + assert_eq!(p.minmax(&x, 0, 2), (-0.25, 0.5)); + assert_eq!(p.minmax(&x, 5, 3), (0.0, 0.0)); + assert_eq!(p.minmax(&x, 0, usize::MAX), (-0.25, 0.5)); + assert_eq!(p.minmax(&x[..1], 0, 2), (0.5, 0.5)); + for (start, step, overlap) in [ + (f64::NAN, 1.0, 0.0), + (0.0, 0.0, 0.0), + (0.0, -1.0, 0.0), + (0.0, f64::INFINITY, 0.0), + (f64::NEG_INFINITY, 1.0, 0.0), + (0.0, 1.0, -0.5), + (0.0, 1.0, f64::NAN), + ] { + assert_eq!(p.spans(&x, start, step, overlap, 3), vec![None; 3]); + } + assert_eq!(p.spans(&x, 0.0, 1.0, 0.0, 0).len(), 0); + assert_eq!(p.spans(&x, 0.0, 1.0, 0.0, usize::MAX).len(), MAX_COLUMNS); + let x = [f32::NAN, 1.0]; + let p = Peaks::build(&x); + assert_eq!(p.minmax(&x, 0, 2), (1.0, 1.0)); + assert_eq!(p.spans(&x, 0.0, 1.0, 0.0, 1), vec![Some((1.0, 1.0))]); + assert!(p.rms_spans(&x, 0.0, 1.0, 0.0, 1)[0].is_some_and(f32::is_finite)); + } + + #[test] + fn loud_finite_samples_give_a_finite_rms() { + let x = [1e20f32, -1e20, 3e38, 0.5]; + let p = Peaks::build(&x); + let rms = p.rms_spans(&x, 0.0, 4.0, 0.0, 1)[0].unwrap(); + let want = (x.iter().map(|&v| f64::from(v) * f64::from(v)).sum::() / 4.0).sqrt() as f32; + assert!(rms.is_finite() && (rms / want - 1.0).abs() < 1e-6, "{rms} vs {want}"); + } + + #[test] + fn rms_spans_match_the_samples() { + let x = signal(20_000); + let p = Peaks::build(&x); + let rms = |s: usize, e: usize| (x[s..e].iter().map(|&v| f64::from(v) * f64::from(v)).sum::() / (e - s) as f64).sqrt() as f32; + for (step, overlap) in [(1.5, 0.0), (37.0, 37.0), (1000.0, 0.0), (4096.0, 512.0)] { + let cols = ((19_999.0 - overlap) / step) as usize; + for (c, got) in p.rms_spans(&x, 0.0, step, overlap, cols).into_iter().enumerate() { + let (a, b) = (c as f64 * step, ((c + 1) as f64 * step + overlap).min(19_999.0)); + let (got, want) = (got.unwrap(), rms(a.floor() as usize, (b.ceil() as usize + 1).min(20_000))); + assert!((got - want).abs() <= want * 1e-4 + 1e-6, "span {c} at step {step}: {got} vs {want}"); + } + } + let sine: Vec = (0..48_000).map(|i| (i as f32 * 0.01).sin()).collect(); + let p = Peaks::build(&sine); + let r = p.rms_spans(&sine, 0.0, 47_000.0, 0.0, 1)[0].unwrap(); + assert!((r - std::f32::consts::FRAC_1_SQRT_2).abs() < 1e-3, "{r}"); + assert_eq!(p.rms_spans(&sine, f64::NAN, 1.0, 0.0, 2), vec![None, None]); + assert_eq!(p.rms_spans(&sine, 50_000.0, 1.0, 0.0, 1), vec![None]); + } + + #[test] + fn spans_cover_their_samples_and_touch_their_neighbours() { + let x = signal(10_000); + let p = Peaks::build(&x); + for step in [0.3, 1.0, 7.5, 100.0, 333.3] { + let cols = (9_999.0 / step) as usize; + let spans = p.spans(&x, 0.0, step, 0.0, cols); + for (c, s) in spans.iter().enumerate() { + let (lo, hi) = s.unwrap_or_else(|| panic!("span {c} at step {step}")); + let (a, b) = ((c as f64 * step).ceil() as usize, ((c + 1) as f64 * step).floor() as usize); + if a <= b { + let truth = brute(&x, a, b + 1); + assert!(lo <= truth.0 && hi >= truth.1, "span {c} at step {step}"); + } + if let Some(Some((nlo, nhi))) = spans.get(c + 1) { + assert!(*nlo <= hi && *nhi >= lo, "spans {c} and {} at step {step} don't touch", c + 1); + } + } + let all = spans.iter().flatten().fold((f32::INFINITY, f32::NEG_INFINITY), |a, s| (a.0.min(s.0), a.1.max(s.1))); + assert_eq!(all, brute(&x, 0, ((cols as f64 * step).floor() as usize + 1).min(10_000)), "step {step}"); + } + } + + #[test] + fn spans_follow_the_line_between_samples() { + let x = [0.0, 1.0, -1.0, 0.5]; + let p = Peaks::build(&x); + assert_eq!(p.spans(&x, 0.0, 0.25, 0.0, 4), vec![Some((0.0, 0.25)), Some((0.25, 0.5)), Some((0.5, 0.75)), Some((0.75, 1.0))]); + assert_eq!(p.spans(&x, 0.5, 1.0, 0.0, 1), vec![Some((0.0, 1.0))]); + let spans = vec![None, Some((0.0, 0.0)), Some((0.0, 1.0)), Some((-1.0, 1.0)), Some((-1.0, 0.5)), Some((0.5, 0.5)), None]; + assert_eq!(p.spans(&x, -2.0, 1.0, 0.0, 7), spans); + assert_eq!(p.spans(&x, 0.0, 1.0, 1.0, 3), vec![Some((-1.0, 1.0)), Some((-1.0, 1.0)), Some((-1.0, 0.5))]); + assert_eq!(p.spans(&x, 0.0, 1.0, 0.5, 3), vec![Some((0.0, 1.0)), Some((-1.0, 1.0)), Some((-1.0, 0.5))]); + } +} diff --git a/crates/audio/src/read/aiff.rs b/crates/audio/src/read/aiff.rs new file mode 100644 index 00000000..a005f272 --- /dev/null +++ b/crates/audio/src/read/aiff.rs @@ -0,0 +1,149 @@ +//! Native AIFF / AIFF-C reader (Apple "Audio Interchange File Format" 1.3 and the AIFF-C +//! draft: big-endian IFF chunks, 80-bit IEEE extended sample rate). + +use super::source::Src; +use crate::pcm::{PcmKind, be_u16, be_u32, tag}; +use crate::{AudioError, AudioInfo, ChannelLayout, CodecId, Container, Result, SampleFormat}; + +const MAX_SAMPLE_RATE: f64 = 10_000_000.0; + +pub(crate) struct AiffLayout { + pub info: AudioInfo, + pub kind: PcmKind, + pub data_start: u64, + pub data_len: u64, +} + +pub(crate) enum AiffParse { + Native(AiffLayout), + /// Compressed AIFF-C (ulaw, alaw, ima4, ...): handed to symphonia. + Foreign, +} + +/// True when the bytes start like an AIFF or AIFF-C file. +pub(crate) fn is_aiff(b: &[u8]) -> bool { + tag(b, 0).is_some_and(|t| &t == b"FORM") && tag(b, 8).is_some_and(|t| &t == b"AIFF" || &t == b"AIFC") +} + +/// Decode an 80-bit IEEE 754 extended-precision float. +pub(crate) fn extended_to_f64(b: &[u8]) -> Option { + let se = be_u16(b, 0)?; + let hi = u64::from(be_u32(b, 2)?); + let lo = u64::from(be_u32(b, 6)?); + let mant = (hi << 32) | lo; + if mant == 0 { + return Some(0.0); + } + let exp = i32::from(se & 0x7FFF); + if exp == 0x7FFF { + return None; // inf / NaN + } + let v = mant as f64 * 2f64.powi(exp - 16383 - 63); + Some(if se & 0x8000 != 0 { -v } else { v }) +} + +/// Parse an AIFF file's chunks: its `COMM` format and where its `SSND` samples are. Only chunk +/// headers and the first bytes of `COMM` and `SSND` are read. +pub(crate) fn parse(src: &Src<'_>) -> Result { + let head = src.bytes_upto(0, 12); + if !is_aiff(&head) { + return Err(AudioError::Malformed("not an AIFF/AIFF-C file".into())); + } + let total = src.size(); + let is_aifc = tag(&head, 8).is_some_and(|t| &t == b"AIFC"); + let mut comm: Option<(i16, u32, i16, f64, [u8; 4])> = None; + let mut ssnd: Option<(u64, u64)> = None; + + let mut pos: u64 = 12; + let mut chunks = 0usize; + while let (Some(id), Some(size32)) = (src.tag(pos), src.be_u32(pos + 4)) { + chunks += 1; + if chunks > 100_000 { + break; + } + let body = pos + 8; + let remaining = total.saturating_sub(body); + let declared = u64::from(size32); + let len = declared.min(remaining); + match &id { + b"COMM" => { + let c = src.bytes(body, usize::try_from(len.min(64)).unwrap_or(0)).unwrap_or_default(); + let short = || AudioError::Malformed("COMM chunk too short".into()); + let channels = be_u16(&c, 0).ok_or_else(short)? as i16; + let frames = be_u32(&c, 2).ok_or_else(short)?; + let bits = be_u16(&c, 6).ok_or_else(short)? as i16; + let rate = extended_to_f64(c.get(8..18).ok_or_else(short)?).ok_or_else(|| AudioError::Malformed("invalid sample rate".into()))?; + let compression = if is_aifc { tag(&c, 18).ok_or_else(short)? } else { *b"NONE" }; + comm = Some((channels, frames, bits, rate, compression)); + } + b"SSND" if ssnd.is_none() => { + let offset = if len >= 4 { u64::from(src.be_u32(body).unwrap_or(0)) } else { 0 }; + let start = body.saturating_add(8).saturating_add(offset).min(total); + let data_len = len.saturating_sub(8).saturating_sub(offset); + ssnd = Some((start, data_len)); + } + _ => {} + } + if declared > remaining { + break; + } + let Some(next) = body.checked_add(declared).and_then(|p| p.checked_add(declared & 1)) else { break }; + pos = next; + } + + let (channels, frames, bits, rate, compression) = comm.ok_or_else(|| AudioError::Malformed("AIFF has no COMM chunk".into()))?; + if channels <= 0 { + return Err(AudioError::Malformed("AIFF declares zero or negative channels".into())); + } + if !(1.0..=MAX_SAMPLE_RATE).contains(&rate) { + return Err(AudioError::Malformed(format!("AIFF sample rate {rate} out of range"))); + } + let sample_rate = rate.round() as u32; + let int_kind = |big_endian: bool| -> Result<(PcmKind, SampleFormat)> { + if !(1..=32).contains(&bits) { + return Err(AudioError::Unsupported(format!("AIFF sample size {bits}"))); + } + let bytes = (bits as usize).div_ceil(8); + let sf = match bits { + 1..=8 => SampleFormat::Int8, + 9..=16 => SampleFormat::Int16, + 17..=24 => SampleFormat::Int24, + _ => SampleFormat::Int32, + }; + Ok((PcmKind::Int { bytes, big_endian, unsigned: false }, sf)) + }; + let (kind, sample_format) = match &compression { + b"NONE" | b"twos" => int_kind(true)?, + b"sowt" => int_kind(false)?, + b"fl32" | b"FL32" => (PcmKind::Float { bytes: 4, big_endian: true }, SampleFormat::Float32), + b"fl64" | b"FL64" => (PcmKind::Float { bytes: 8, big_endian: true }, SampleFormat::Float64), + _ => return Ok(AiffParse::Foreign), + }; + // SSND is required when COMM declares frames (AIFF 1.3); a file that ends before it is + // truncated, not empty + let (data_start, data_len) = match ssnd { + Some(ssnd) => ssnd, + None if frames > 0 => return Err(AudioError::Malformed(format!("AIFF declares {frames} frames but has no SSND chunk"))), + None => (total, 0), + }; + let block = kind.bytes().saturating_mul(channels as usize).max(1); + let frames = u64::from(frames).min(data_len / block as u64); + Ok(AiffParse::Native(AiffLayout { + info: AudioInfo::new(Container::Aiff, CodecId::Pcm, sample_format, sample_rate, layout(usize::from(channels as u16)), frames), + kind, + data_start, + data_len, + })) +} + +/// The AIFF channel order (AIFF 1.3, "Sample Points"): mono, stereo, L R C and quad map to +/// speakers; the six-channel order (L Lc C R Rc S) and larger counts have no WAV order. +fn layout(channels: usize) -> ChannelLayout { + match channels { + 1 => ChannelLayout::MONO, + 2 => ChannelLayout::STEREO, + 3 => ChannelLayout::SURROUND_3_0, + 4 => ChannelLayout::QUAD, + n => ChannelLayout::unordered(crate::layout::count(n)), + } +} diff --git a/crates/audio/src/read/caf.rs b/crates/audio/src/read/caf.rs new file mode 100644 index 00000000..94ae7f04 --- /dev/null +++ b/crates/audio/src/read/caf.rs @@ -0,0 +1,87 @@ +//! Size checks on CAF chunks before symphonia reads them: it allocates whatever a magic cookie, +//! a packet table or a packet claims, and a failed allocation aborts the process. + +use crate::pcm::{be_u16, be_u32, be_u64}; +use crate::{AudioError, Result}; + +/// Refuse the CAF stream at `at` if its magic cookie, packet table or packets claim more bytes +/// than the file holds. +pub(crate) fn check(b: &[u8], at: usize) -> Result<()> { + if be_u16(b, at + 4) == Some(1) { chunks(b, at + 8) } else { Ok(()) } +} + +fn malformed(what: &str) -> AudioError { + AudioError::Malformed(format!("CAF {what} is larger than the file")) +} + +/// Walk the chunks the way the CAF reader reads them: it advances by the bytes it parses, not by +/// the declared size, and after a `data` chunk of unknown size it reads on into the audio. +fn chunks(b: &[u8], mut pos: usize) -> Result<()> { + // Whether the packet table lists each packet's size and frame count (variable per packet). + let (mut lists_sizes, mut lists_frames, mut described) = (false, false, false); + while let (Some(kind), Some(size)) = (b.get(pos..pos + 4), be_u64(b, pos + 4)) { + let (body, size) = (pos + 12, size as i64); + let used = match kind { + b"desc" if size == 32 => { + lists_sizes = be_u32(b, body + 16) == Some(0); + lists_frames = be_u32(b, body + 20) == Some(0); + described = true; + Some(32) + } + b"data" if size == -1 => Some(4), + b"chan" if size >= 12 => be_u32(b, body + 8).map(|n| 12 + u64::from(n) * 20), + b"pakt" if size >= 24 && described => packet_table(b, body, lists_sizes, lists_frames)?, + b"kuki" if size >= 0 && size as u64 > b.len().saturating_sub(body) as u64 => return Err(malformed("magic cookie")), + b"desc" | b"pakt" | b"chan" => None, + _ => u64::try_from(size).ok(), + }; + match used.filter(|_| described).and_then(|n| usize::try_from(n).ok()).and_then(|n| body.checked_add(n)) { + Some(next) if next < b.len() => pos = next, + _ => return Ok(()), + } + } + Ok(()) +} + +/// Check a packet table's count and packet sizes; returns the bytes symphonia reads for it, or +/// None where symphonia stops. It reserves room for every packet before reading any, so a table +/// it would refuse after that, or that runs past the file, is refused first. +fn packet_table(b: &[u8], body: usize, lists_sizes: bool, lists_frames: bool) -> Result> { + let valid = |at: usize| be_u64(b, at).filter(|&n| n as i64 >= 0); + let (Some(packets), Some(_)) = (valid(body), valid(body + 8)) else { return Ok(None) }; + let mut at = body + 24; + if packets > b.len().saturating_sub(at) as u64 { + return Err(malformed("packet table")); + } + if !lists_sizes && !lists_frames { + return if packets == 0 { Ok(Some(24)) } else { Err(AudioError::Malformed("a constant bit rate CAF lists packets".into())) }; + } + for _ in 0..packets { + if lists_sizes { + let Some((size, len)) = varint(b, at) else { return Err(malformed("packet table")) }; + if size > b.len() as u64 { + return Err(malformed("packet")); + } + at += len; + } + if lists_frames { + let Some((_, len)) = varint(b, at) else { return Err(malformed("packet table")) }; + at += len; + } + } + Ok(Some((at - body) as u64)) +} + +/// A CAF variable-length integer at `at` and its length in bytes. +fn varint(b: &[u8], at: usize) -> Option<(u64, usize)> { + let mut v = 0u64; + for i in 0..9 { + let x = *b.get(at + i)?; + v |= u64::from(x & 0x7f); + if x & 0x80 == 0 { + return Some((v, i + 1)); + } + v <<= 7; + } + None +} diff --git a/crates/audio/src/read/flac.rs b/crates/audio/src/read/flac.rs new file mode 100644 index 00000000..896150e2 --- /dev/null +++ b/crates/audio/src/read/flac.rs @@ -0,0 +1,104 @@ +//! FLAC metadata checks run before symphonia decodes a stream (RFC 9639 metadata blocks). + +use crate::pcm::{be_u32, le_u32}; +use crate::{AudioError, AudioInfo, ChannelLayout, CodecId, Container, Result, SampleFormat}; + +/// Refuse the FLAC stream at `at` if a comment or picture field claims more bytes than its +/// metadata block holds: symphonia allocates each field's length before reading it. +pub(crate) fn check_metadata(b: &[u8], at: usize) -> Result<()> { + let mut pos = at + 4; + while let Some(head) = be_u32(b, pos) { + let (body, len) = (pos + 4, (head & 0x00ff_ffff) as usize); + let end = (body + len).min(b.len()); + let overruns = match (head >> 24) & 0x7f { + 4 => comments_overrun(b, body, end), + // Picture: type, MIME type, description, four size fields, data. + 6 => overruns(b, body + 4, end, be_u32, [0, 0, 16]), + _ => false, + }; + if overruns { + return Err(AudioError::Malformed("a FLAC metadata field is larger than its block".into())); + } + if head >> 31 == 1 { + break; + } + pos = body + len; + } + Ok(()) +} + +/// The info of a FLAC stream with no audio frames at all: STREAMINFO declares 0 samples and +/// nothing follows the metadata blocks. Symphonia's reader fails on these, as it looks for a first +/// frame, so the caller answers them itself. +pub(crate) fn empty_stream_info(b: &[u8], at: usize) -> Option { + let mut pos = at.checked_add(4)?; + let mut first = true; + loop { + let head = be_u32(b, pos)?; + let (body, len) = (pos.checked_add(4)?, (head & 0x00ff_ffff) as usize); + let end = body.checked_add(len)?; + if end > b.len() { + return None; + } + if first && ((head >> 24) & 0x7f != 0 || len != 34) { + return None; + } + first = false; + if head >> 31 == 1 { + if end != b.len() { + return None; + } + break; + } + pos = end; + } + // STREAMINFO, from the sample rate on: 20 bits rate, 3 channels - 1, 5 bits per sample - 1, 36 bits total. + let si = at.checked_add(4 + 4 + 10)?; + let packed = u64::from_be_bytes(b.get(si..si.checked_add(8)?)?.try_into().ok()?); + let sample_rate = (packed >> 44) as u32; + let channels = ((packed >> 41) & 0x7) as u16 + 1; + let bps = ((packed >> 36) & 0x1f) as u32 + 1; + if packed & 0xf_ffff_ffff != 0 || sample_rate == 0 { + return None; + } + let sample_format = match bps { + 1..=8 => SampleFormat::Int8, + 9..=16 => SampleFormat::Int16, + 17..=24 => SampleFormat::Int24, + _ => SampleFormat::Int32, + }; + Some(AudioInfo::new(Container::Flac, CodecId::Flac, sample_format, sample_rate, flac_layout(usize::from(channels)), 0)) +} + +/// The FLAC channel assignment for a channel count (RFC 9639 §9.1.3). +pub(crate) fn flac_layout(channels: usize) -> ChannelLayout { + match channels { + 7 => ChannelLayout::SURROUND_6_1, + 1..=8 => ChannelLayout::default_for(crate::layout::count(channels)), + n => ChannelLayout::unordered(crate::layout::count(n)), + } +} + +/// Whether a comment in a (little-endian) Vorbis comment block claims more than the block holds. +/// The vendor string is skipped unread. +fn comments_overrun(b: &[u8], body: usize, end: usize) -> bool { + let Some(vendor) = le_u32(b, body) else { return false }; + let at = (body + 4).saturating_add(vendor as usize); + let Some(count) = le_u32(b, at).filter(|_| at.saturating_add(4) <= end) else { return false }; + overruns(b, at + 4, end, le_u32, std::iter::repeat_n(0, count as usize)) +} + +/// Whether one of the length-prefixed fields from `at` claims more than `end` leaves; `gaps` are +/// the bytes skipped before each length, which `read` reads. +fn overruns(b: &[u8], mut at: usize, end: usize, read: fn(&[u8], usize) -> Option, gaps: impl IntoIterator) -> bool { + for gap in gaps { + at = at.saturating_add(gap); + let Some(n) = read(b, at).filter(|_| at.saturating_add(4) <= end) else { return false }; + at += 4; + if n as usize > end - at { + return true; + } + at += n as usize; + } + false +} diff --git a/crates/audio/src/read/mod.rs b/crates/audio/src/read/mod.rs new file mode 100644 index 00000000..99798120 --- /dev/null +++ b/crates/audio/src/read/mod.rs @@ -0,0 +1,601 @@ +//! Format detection, probing and decoding. The native path (craft-media's demuxers, the WAV and +//! AIFF readers, and [`TrackDecoder`]) comes first; symphonia decodes the rest. +//! +//! `native.rs` opens a file natively or returns `None`, and `symph.rs` is the fallback, so a new +//! native decoder takes its formats over without an API change. Removing symphonia means deleting +//! `symph.rs` and its size checks, the `symphonia` feature and the workspace's +//! `symphonia-bundle-flac` profile override. + +mod aiff; +#[cfg(feature = "symphonia")] +mod caf; +#[cfg(feature = "symphonia")] +mod flac; +#[cfg(feature = "symphonia")] +mod mp4; +mod native; +mod source; +#[cfg(feature = "symphonia")] +mod symph; +mod wav; + +use crate::{AudioBuffer, AudioError, AudioInfo, ChannelLayout, CodecId, Container, Result}; +use native::Native; +pub use source::SharedSource; +use source::Src; +use std::sync::Arc; +#[cfg(feature = "symphonia")] +use symph::Symph; + +/// The decode limit [`decode`] applies unless [`DecodeOptions::max_samples`] says otherwise: +/// 2^30 samples, 4 GiB of `f32`. +pub const DEFAULT_MAX_SAMPLES: u64 = crate::pcm::MAX_DECODE_SAMPLES; + +/// Size of an ID3v2 tag at the start of `b`, if there is one. The 10-byte footer flag (`0x10`) +/// exists only in ID3v2.4; in v2.2/v2.3 that bit is reserved and must not extend the skip. +pub(crate) fn id3v2_len(b: &[u8]) -> Option { + if b.get(..3)? != b"ID3" { + return None; + } + let version = *b.get(3)?; + let flags = *b.get(5)?; + let size = b.get(6..10)?.iter().fold(0usize, |acc, &x| (acc << 7) | usize::from(x & 0x7F)); + let footer = version == 4 && flags & 0x10 != 0; + Some(10 + size + if footer { 10 } else { 0 }) +} + +fn detect_by_magic(b: &[u8], depth: u32) -> Option { + let head = |n: usize| b.get(..n); + if wav::is_wav(b) { + return Some(Container::Wav); + } + if aiff::is_aiff(b) { + return Some(Container::Aiff); + } + match head(4)? { + b"fLaC" => return Some(Container::Flac), + b"OggS" => return Some(Container::Ogg), + b"caff" => return Some(Container::Caf), + // Matroska and WebM differ only in the EBML header's DocType, within its first bytes. + [0x1A, 0x45, 0xDF, 0xA3] => { + let header = b.get(..64.min(b.len())).unwrap_or_default(); + return Some(if header.windows(4).any(|w| w == b"webm") { Container::WebM } else { Container::Matroska }); + } + _ => {} + } + if b.get(4..8) == Some(b"ftyp") { + return Some(if b.get(8..12) == Some(b"qt ") { Container::QuickTime } else { Container::Mp4 }); + } + if let Some(len) = id3v2_len(b) { + if depth == 0 + && let Some(f) = b.get(len..).and_then(|rest| detect_by_magic(rest, 1)) + { + return Some(f); + } + return Some(Container::MpegAudio); + } + let (b0, b1) = (*b.first()?, *b.get(1)?); + if b0 == 0x0B && b1 == 0x77 { + return Some(Container::Ac3); + } + if b0 == 0xFF && b1 & 0xE0 == 0xE0 { + let layer = (b1 >> 1) & 0x3; + if b1 & 0xF0 == 0xF0 && layer == 0 { + return Some(Container::Adts); + } + if layer != 0 { + return Some(Container::MpegAudio); + } + } + None +} + +fn detect_by_ext(ext: &str) -> Option { + Some(match ext.trim_start_matches('.').to_ascii_lowercase().as_str() { + "wav" | "wave" | "bwf" | "rf64" => Container::Wav, + "aif" | "aiff" | "aifc" => Container::Aiff, + "flac" => Container::Flac, + "mp3" | "mp2" | "mp1" | "mpa" => Container::MpegAudio, + "ogg" | "oga" | "opus" => Container::Ogg, + "aac" | "adts" => Container::Adts, + "m4a" | "m4b" | "mp4" => Container::Mp4, + "mov" => Container::QuickTime, + "mkv" | "mka" => Container::Matroska, + "webm" => Container::WebM, + "ac3" | "eac3" | "ec3" => Container::Ac3, + "caf" => Container::Caf, + _ => return None, + }) +} + +/// The container of a file, by its magic bytes, falling back to the extension hint (`"wav"`, +/// `".flac"`); `None` when neither says. +/// +/// ``` +/// use craft_audio::{Container, detect_container}; +/// +/// assert_eq!(detect_container(b"fLaC\0\0\0\x22", None), Some(Container::Flac)); +/// assert_eq!(detect_container(&[0; 16], Some("wav")), Some(Container::Wav)); +/// assert_eq!(detect_container(&[0; 16], None), None); +/// ``` +pub fn detect_container(bytes: &[u8], ext_hint: Option<&str>) -> Option { + detect_by_magic(bytes, 0).or_else(|| ext_hint.and_then(detect_by_ext)) +} + +/// Options for [`Decoder`], [`decode_with`] and [`probe_with`]. +/// +/// `#[non_exhaustive]`: start from `DecodeOptions::default()` and use the `with_*` methods. +#[non_exhaustive] +#[derive(Debug, Clone, PartialEq, Eq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub struct DecodeOptions { + /// Which audio track to decode, counted among the file's audio tracks as [`tracks`] lists + /// them; `None` is the first. + pub track: Option, + /// The most samples (frames × channels) [`decode_with`] returns; a longer file is + /// [`AudioError::TooLarge`]. The streaming [`Decoder`] has no limit. + pub max_samples: u64, +} + +impl Default for DecodeOptions { + fn default() -> Self { + DecodeOptions { track: None, max_samples: DEFAULT_MAX_SAMPLES } + } +} + +impl DecodeOptions { + /// Decode the audio track at index `track` (see [`DecodeOptions::track`]). + pub fn with_track(mut self, track: usize) -> DecodeOptions { + self.track = Some(track); + self + } + + /// Set [`DecodeOptions::max_samples`]. + pub fn with_max_samples(mut self, max_samples: u64) -> DecodeOptions { + self.max_samples = max_samples; + self + } +} + +/// One audio track of a file, as [`tracks`] lists it. +#[non_exhaustive] +#[derive(Debug, Clone, PartialEq, Eq)] +#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))] +pub struct TrackInfo { + /// The value for [`DecodeOptions::track`]. + pub index: usize, + /// The codec the container names (an AAC track that turns out to be HE-AAC is `Aac` here). + pub codec: CodecId, + /// The sample rate the container declares, if it does. + pub sample_rate: Option, + /// The channel count the container declares, if it does. + pub channels: Option, +} + +/// The decoder behind a [`Decoder`]. +enum Inner<'a> { + Native(Box>), + #[cfg(feature = "symphonia")] + Symph(Box), +} + +/// A file's audio, decoded block by block: the streaming counterpart of [`decode`], with seeking +/// and track choice. +/// +/// Native formats (WAV, AIFF, and PCM in MP4/MOV and Matroska/WebM) read their sample data from +/// the source packet by packet. The container's packet index is held in memory (tens of bytes per +/// packet), and opening a Matroska file reads through the whole file to build it; WAV and AIFF +/// need no index. Formats symphonia decodes (AAC, MP1/MP2/MP3, FLAC, ALAC, Vorbis, CAF, ADPCM, +/// A-law, µ-law) read the whole source into memory first; their decoded audio still streams. +/// +/// ```no_run +/// # fn demo() -> craft_audio::Result<()> { +/// use craft_audio::{DecodeOptions, Decoder}; +/// +/// let file = std::fs::File::open("long-recording.m4a")?; +/// let mut dec = Decoder::open(file, Some("m4a"), &DecodeOptions::default())?; +/// println!("{} {} Hz {}", dec.info().codec.name(), dec.info().sample_rate, dec.info().layout); +/// dec.seek(48_000 * 60)?; // one minute in +/// while let Some(block) = dec.next_block()? { +/// // block.channels: planar f32, WAV channel order +/// } +/// # Ok(()) } +/// ``` +pub struct Decoder<'a> { + inner: Inner<'a>, +} + +impl std::fmt::Debug for Decoder<'_> { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("Decoder").field("info", self.info()).field("length_known", &self.length_known()).finish_non_exhaustive() + } +} + +impl Decoder<'static> { + /// Open a file read from `source`: a `std::fs::File`, a shared buffer, a + /// [`crate::ReadSeekSource`] around any `Read + Seek`, or any [`crate::ByteSource`] that is + /// [`SharedSource`] (`Send + Sync`, except on single-threaded WebAssembly). `ext_hint` is the + /// file's extension, used only when its bytes do not say what it is. + pub fn open(source: impl SharedSource, ext_hint: Option<&str>, opts: &DecodeOptions) -> Result> { + Decoder::from_src(Src::Shared(Arc::new(source)), ext_hint, opts) + } +} + +impl<'a> Decoder<'a> { + /// Open a file held in memory. + pub fn from_bytes(bytes: &'a [u8], ext_hint: Option<&str>, opts: &DecodeOptions) -> Result> { + Decoder::from_src(Src::Slice(bytes), ext_hint, opts) + } + + fn from_src(src: Src<'a>, ext_hint: Option<&str>, opts: &DecodeOptions) -> Result> { + let head = src.head_checked()?; + let container = detect_container(&head, ext_hint); + if let Some(n) = Native::open(&src, container, opts.track) { + return Ok(Decoder { inner: Inner::Native(Box::new(n?)) }); + } + Decoder::fallback(src, container, opts) + } + + /// A file the native path does not read: symphonia's. + #[cfg(feature = "symphonia")] + fn fallback(src: Src<'a>, container: Option, opts: &DecodeOptions) -> Result> { + let mut s = Symph::open(src.whole()?, container, opts.track).map_err(|e| damaged_or(&src, container, e))?; + // A WAV with a codec symphonia reads (ADPCM, A-law, ...) keeps its BWF metadata. + if container == Some(Container::Wav) + && let Ok(wav::WavParse::Foreign { bwf }) = wav::parse(&src) + { + s.set_bwf(bwf); + } + Ok(Decoder { inner: Inner::Symph(Box::new(s)) }) + } + + /// A file the native path does not read, without symphonia. + #[cfg(not(feature = "symphonia"))] + fn fallback(src: Src<'a>, _container: Option, _opts: &DecodeOptions) -> Result> { + Err(not_built(&src)) + } + + /// What the file holds. `frames` is 0 until decoded when the container does not say how long + /// the track is ([`Decoder::length_known`]). + pub fn info(&self) -> &AudioInfo { + match &self.inner { + Inner::Native(n) => n.info(), + #[cfg(feature = "symphonia")] + Inner::Symph(s) => s.info(), + } + } + + /// Whether [`AudioInfo::frames`] came from the file rather than being unknown (an MP3 without + /// a Xing frame, PCM in Matroska). + pub fn length_known(&self) -> bool { + match &self.inner { + Inner::Native(n) => n.frames_known(), + #[cfg(feature = "symphonia")] + Inner::Symph(s) => s.frames_known(), + } + } + + /// What [`probe`] reports, for a decoder opened on any source: the info from the headers, or, + /// when they give no length, the info after decoding the rest of the track to count it. + /// + /// ```no_run + /// # fn demo() -> craft_audio::Result<()> { + /// use craft_audio::{DecodeOptions, Decoder}; + /// + /// let info = Decoder::open(std::fs::File::open("take.mp3")?, Some("mp3"), &DecodeOptions::default())?.probe()?; + /// println!("{:.1} s", info.duration_secs()); + /// # Ok(()) } + /// ``` + pub fn probe(mut self) -> Result { + if self.length_known() { + return Ok(self.info().clone()); + } + let (mut frames, mut channels) = (0u64, None); + while let Some(b) = self.next_block()? { + frames = frames.saturating_add(b.frames() as u64); + channels.get_or_insert(b.channels.len()); + } + let mut info = self.info().clone(); + info.frames = frames; + fit_channels(&mut info, channels); + Ok(info) + } + + /// The next block of decoded audio (planar `f32`, WAV channel order when the layout is + /// ordered), or `None` at the end. Blocks follow the codec's packets, so their length varies. + /// A damaged packet is concealed and counted in [`AudioInfo::damaged`]; a source that fails a read gives + /// [`AudioError::Io`] and that packet is skipped, so call [`Decoder::seek`] to retry it. + pub fn next_block(&mut self) -> Result> { + match &mut self.inner { + Inner::Native(n) => n.next_block(), + #[cfg(feature = "symphonia")] + Inner::Symph(s) => s.next_block(), + } + } + + /// Continue decoding from frame `frame` (samples per channel from the start); the next block + /// starts exactly there. Returns the frame decoding resumes at. + pub fn seek(&mut self, frame: u64) -> Result { + match &mut self.inner { + Inner::Native(n) => n.seek(frame), + #[cfg(feature = "symphonia")] + Inner::Symph(s) => s.seek(frame), + } + } +} + +/// Read a file's container, codec, sample rate, channel layout, length and BWF metadata from its +/// headers. When the headers give no length (an MP3 without a Xing frame, PCM in Matroska), the +/// file is decoded to count it. For a file on disk, use [`Decoder::probe`]. +/// +/// ``` +/// # #[cfg(feature = "pcm-encode")] { +/// use craft_audio::{AudioBuffer, ChannelLayout, CodecId, Container, EncodeOptions, encode, probe}; +/// +/// let wav = encode(&AudioBuffer::new(48_000, 2, 4800), &EncodeOptions::default()).unwrap(); +/// let info = probe(&wav, Some("wav")).unwrap(); +/// assert_eq!((info.container, info.codec, info.layout), (Container::Wav, CodecId::Pcm, ChannelLayout::STEREO)); +/// assert_eq!((info.sample_rate, info.frames), (48_000, 4800)); +/// # } +/// ``` +pub fn probe(bytes: &[u8], ext_hint: Option<&str>) -> Result { + probe_with(bytes, ext_hint, &DecodeOptions::default()) +} + +/// [`probe`] for the track `opts` names. +/// +/// ``` +/// # #[cfg(feature = "pcm-encode")] { +/// use craft_audio::{AudioBuffer, EncodeOptions, encode}; +/// let wav = encode(&AudioBuffer::new(48_000, 2, 4800), &EncodeOptions::default()).unwrap(); +/// use craft_audio::{DecodeOptions, probe_with}; +/// +/// let info = probe_with(&wav, Some("wav"), &DecodeOptions::default().with_track(0)).unwrap(); +/// assert_eq!((info.channels, info.frames), (2, 4800)); +/// assert!(probe_with(&wav, Some("wav"), &DecodeOptions::default().with_track(1)).is_err()); +/// # } +/// ``` +pub fn probe_with(bytes: &[u8], ext_hint: Option<&str>, opts: &DecodeOptions) -> Result { + Decoder::from_bytes(bytes, ext_hint, opts)?.probe() +} + +/// Decode the first audio track of a file to planar `f32`, trimmed to its presentation length, in +/// WAV channel order when [`AudioInfo::layout`] is ordered. Use [`Decoder`] to stream a long +/// file instead, and [`decode_with`] for another track or size limit. +/// +/// ``` +/// # #[cfg(feature = "pcm-encode")] { +/// use craft_audio::{AudioBuffer, EncodeFormat, EncodeOptions, decode, encode}; +/// +/// let tone: Vec = (0..4800).map(|i| (i as f32 * 0.05).sin() * 0.5).collect(); +/// let aiff = encode(&AudioBuffer { sample_rate: 48_000, channels: vec![tone] }, &EncodeOptions::new(EncodeFormat::Aiff)).unwrap(); +/// let (info, buf) = decode(&aiff, None).unwrap(); +/// assert_eq!((info.channels, buf.frames()), (1, 4800)); +/// # } +/// ``` +pub fn decode(bytes: &[u8], ext_hint: Option<&str>) -> Result<(AudioInfo, AudioBuffer)> { + decode_with(bytes, ext_hint, &DecodeOptions::default()) +} + +/// [`decode`] with options: another track, or another size limit. +/// +/// ``` +/// # #[cfg(feature = "pcm-encode")] { +/// use craft_audio::{AudioBuffer, EncodeOptions, encode}; +/// let wav = encode(&AudioBuffer::new(48_000, 2, 4800), &EncodeOptions::default()).unwrap(); +/// use craft_audio::{AudioError, DecodeOptions, decode_with}; +/// +/// let (_, buf) = decode_with(&wav, None, &DecodeOptions::default()).unwrap(); +/// assert_eq!(buf.frames(), 4800); +/// // 2 × 4800 samples are over a limit of 1000. +/// let small = DecodeOptions::default().with_max_samples(1000); +/// assert!(matches!(decode_with(&wav, None, &small), Err(AudioError::TooLarge(_)))); +/// # } +/// ``` +pub fn decode_with(bytes: &[u8], ext_hint: Option<&str>, opts: &DecodeOptions) -> Result<(AudioInfo, AudioBuffer)> { + let mut d = Decoder::from_bytes(bytes, ext_hint, opts)?; + let declared = d.info().frames.saturating_mul(u64::from(d.info().channels.max(1))); + if d.length_known() && declared > opts.max_samples { + return Err(AudioError::TooLarge(format!("{declared} samples exceeds the decode limit"))); + } + let mut planar: Vec> = Vec::new(); + let (mut frames, mut damaged) = (0u64, 0u64); + while let Some(block) = d.next_block()? { + if planar.is_empty() { + planar = vec![Vec::new(); block.channels.len()]; + } + // A block with another channel count cannot join the buffer: it is concealed like a + // damaged packet, so the audio after it keeps its timing. + let block = if block.channels.len() == planar.len() { + block + } else { + damaged += 1; + AudioBuffer { sample_rate: block.sample_rate, channels: vec![vec![0.0; block.frames()]; planar.len()] } + }; + frames = frames.saturating_add(block.frames() as u64); + if frames.saturating_mul(planar.len() as u64) > opts.max_samples { + return Err(AudioError::TooLarge("decoded audio exceeds the decode limit".into())); + } + for (dst, src) in planar.iter_mut().zip(block.channels) { + dst.try_reserve(src.len()).map_err(|_| AudioError::TooLarge("cannot allocate decoded audio".into()))?; + dst.extend(src); + } + } + let mut info = d.info().clone(); + info.damaged += damaged; + let channels = (!planar.is_empty()).then_some(planar.len()); + fit_channels(&mut info, channels); + if planar.is_empty() { + planar = vec![Vec::new(); usize::from(info.channels)]; + } + info.frames = planar.iter().map(Vec::len).min().unwrap_or(0) as u64; + let sample_rate = info.sample_rate; + Ok((info, AudioBuffer { sample_rate, channels: planar })) +} + +/// The decoded channel count is authoritative: a layout that names another count is dropped. +fn fit_channels(info: &mut AudioInfo, decoded: Option) { + if let Some(ch) = decoded + && ch != usize::from(info.channels) + { + info.channels = u16::try_from(ch).unwrap_or(u16::MAX); + info.layout = ChannelLayout::unordered(crate::layout::count(ch)); + } +} + +/// The audio tracks of a file, in the order [`DecodeOptions::track`] counts them. +/// +/// ``` +/// # #[cfg(feature = "pcm-encode")] { +/// use craft_audio::{AudioBuffer, EncodeOptions, encode}; +/// let wav = encode(&AudioBuffer::new(48_000, 2, 4800), &EncodeOptions::default()).unwrap(); +/// use craft_audio::{CodecId, tracks}; +/// +/// let list = tracks(&wav, None).unwrap(); +/// assert_eq!((list.len(), list[0].codec, list[0].channels), (1, CodecId::Pcm, Some(2))); +/// # } +/// ``` +pub fn tracks(bytes: &[u8], ext_hint: Option<&str>) -> Result> { + tracks_in(&Src::Slice(bytes), ext_hint) +} + +/// [`tracks`] for a file read from a source. +/// +/// ```no_run +/// # fn demo() -> craft_audio::Result<()> { +/// for t in craft_audio::tracks_of(std::fs::File::open("interview.mkv")?, Some("mkv"))? { +/// println!("track {}: {}", t.index, t.codec.name()); +/// } +/// # Ok(()) } +/// ``` +pub fn tracks_of(source: impl SharedSource, ext_hint: Option<&str>) -> Result> { + tracks_in(&Src::Shared(Arc::new(source)), ext_hint) +} + +fn tracks_in(src: &Src<'_>, ext_hint: Option<&str>) -> Result> { + let container = detect_container(&src.head_checked()?, ext_hint); + let list = |v: Vec<(CodecId, Option, Option)>| -> Vec { + v.into_iter().enumerate().map(|(index, (codec, sample_rate, channels))| TrackInfo { index, codec, sample_rate, channels }).collect() + }; + if let Some(params) = native::tracks(src, container) { + let params = params?; + return Ok(list(params.into_iter().map(|p| (p.codec, p.sample_rate, p.channels)).collect())); + } + match Native::open(src, container, None) { + // Single-track formats: WAV, AIFF, raw streams. + Some(n) => { + let n = n?; + let i = n.info(); + Ok(list(vec![(i.codec, (i.sample_rate > 0).then_some(i.sample_rate), (i.channels > 0).then_some(i.channels))])) + } + #[cfg(feature = "symphonia")] + None => Ok(list( + symph::tracks(src.whole()?, container) + .map_err(|e| damaged_or(src, container, e))? + .into_iter() + .map(|(c, r, n)| (c, (r > 0).then_some(r), (n > 0).then_some(n))) + .collect(), + )), + #[cfg(not(feature = "symphonia"))] + None => Err(not_built(src)), + } +} + +/// symphonia's error for a file it cannot read, or the native demuxer's finding when symphonia +/// calls unsupported a file the native path found damaged: a damaged file is malformed whichever +/// path last tried it. +#[cfg(feature = "symphonia")] +fn damaged_or(src: &Src<'_>, container: Option, e: AudioError) -> AudioError { + if matches!(e, AudioError::Unsupported(_)) + && let Some(m) = native::damage(src, container) + { + return AudioError::Malformed(m); + } + e +} + +/// The error for a file the native path does not read, in a build without symphonia: the build +/// is blamed only when the bytes name a format. +#[cfg(not(feature = "symphonia"))] +fn not_built(src: &Src<'_>) -> AudioError { + let head = match src.head_checked() { + Ok(head) => head, + Err(e) => return e, + }; + match detect_container(&head, None) { + Some(c) => AudioError::Unsupported(format!("{} needs a decoder this build leaves out (the `decode` feature adds symphonia)", c.name())), + None => AudioError::Unsupported("not a recognized audio format".into()), + } +} + +#[cfg(all(test, not(feature = "symphonia")))] +mod native_only_tests { + use super::*; + + /// Without symphonia, a format with no native decoder is unsupported, not malformed. + #[test] + fn formats_left_to_symphonia_are_unsupported() { + let flac_header = [&b"fLaC"[..], &[0x80, 0, 0, 34], &[0; 34]].concat(); + assert!(matches!(decode(&flac_header, None), Err(AudioError::Unsupported(m)) if m.contains("FLAC"))); + assert!(matches!(probe(&flac_header, None), Err(AudioError::Unsupported(_)))); + // A track list says the same instead of listing nothing. + assert!(matches!(tracks(&flac_header, None), Err(AudioError::Unsupported(m)) if m.contains("FLAC"))); + // Bytes that name no format are not blamed on the build. + assert!(matches!(tracks(&[7u8; 64], None), Err(AudioError::Unsupported(m)) if m.contains("not a recognized"))); + } + + /// A source that cannot be read is an I/O error, not an unrecognized format. + #[test] + fn a_failed_header_read_is_an_io_error() { + struct Failing; + impl craft_media::ByteSource for Failing { + fn len(&self) -> u64 { + 1000 + } + fn read_at(&self, _: u64, _: &mut [u8]) -> std::io::Result<()> { + Err(std::io::Error::other("disk gone")) + } + } + assert!(matches!(tracks_of(Failing, None), Err(AudioError::Io(_)))); + assert!(matches!(Decoder::open(Failing, None, &DecodeOptions::default()), Err(AudioError::Io(_)))); + } + + /// A damaged file in a container the native path reads is malformed, not left to a decoder + /// this build lacks. + #[test] + fn a_damaged_mp4_is_malformed() { + // an `ftyp` box and nothing after it: no `moov` + let mp4 = [&16u32.to_be_bytes()[..], b"ftypisom", &[0, 0, 2, 0]].concat(); + assert!(matches!(decode(&mp4, None), Err(AudioError::Malformed(_)))); + assert!(matches!(tracks(&mp4, None), Err(AudioError::Malformed(_)))); + } + + /// A source that fails past its first 64 KiB fails inside the demuxer: still an I/O error. + #[test] + fn a_failed_read_inside_the_demuxer_is_an_io_error() { + use craft_media::matroska::{LacingMode, MkvWriter, MuxOptions, TrackKind, TrackSpec}; + struct FailsLate(Vec); + impl craft_media::ByteSource for FailsLate { + fn len(&self) -> u64 { + self.0.len() as u64 + } + fn read_at(&self, offset: u64, buf: &mut [u8]) -> std::io::Result<()> { + if offset + buf.len() as u64 > 64 * 1024 { + return Err(std::io::Error::other("disk gone")); + } + self.0.read_at(offset, buf) + } + } + // a Matroska PCM file of about 400 KB, whose cues and later clusters lie past 64 KiB + let mut spec = TrackSpec::new(TrackKind::Audio, "A_PCM/FLOAT/IEEE"); + spec.audio = Some((48_000.0, 1, Some(32))); + let opts = MuxOptions { lacing: LacingMode::None, ..MuxOptions::default() }; + let mut w = MkvWriter::new(std::io::Cursor::new(Vec::new()), vec![spec], opts).unwrap(); + for k in 0..100i64 { + w.write_frame(0, k * 20_000_000, true, &[0; 960 * 4], Some(20_000_000)).unwrap(); + } + let file = w.finish().unwrap().into_inner(); + assert!(matches!(tracks_of(FailsLate(file.clone()), None), Err(AudioError::Io(_)))); + assert!(matches!(Decoder::open(FailsLate(file), None, &DecodeOptions::default()), Err(AudioError::Io(_)))); + } +} diff --git a/crates/audio/src/read/mp4.rs b/crates/audio/src/read/mp4.rs new file mode 100644 index 00000000..06e68ddc --- /dev/null +++ b/crates/audio/src/read/mp4.rs @@ -0,0 +1,106 @@ +//! Size checks on MP4 atoms before symphonia reads them: it allocates whatever an atom, a sample +//! table or a sample claims, and a failed allocation aborts the process. + +use crate::pcm::{be_u32, be_u64}; +use crate::{AudioError, Result}; + +#[derive(Clone, Copy)] +enum Level { + Top, + Moov, + Trak, + Mdia, + Minf, + Stbl, + Udta, + Meta, + Ilst, + Tag, +} + +/// Refuse the MP4 file whose top-level atoms start at `at` if an atom symphonia reads whole, a +/// sample table or a sample claims more bytes than the file holds. +pub(crate) fn check(b: &[u8], at: usize) -> Result<()> { + atoms(b, at, b.len(), Level::Top).map(|_| ()) +} + +fn malformed(what: &str) -> AudioError { + AudioError::Malformed(format!("MP4 {what} is larger than the file")) +} + +/// Walk the atoms in `pos..end` that symphonia descends into, as it reads them. False where +/// symphonia stops reading: at a header it refuses, or after an atom that runs past `end`. +fn atoms(b: &[u8], mut pos: usize, end: usize, level: Level) -> Result { + // A header that starts inside the container is read whole, even past its end. + while pos < end { + let (Some(size), Some(kind)) = (be_u32(b, pos), b.get(pos + 4..pos + 8)) else { return Ok(false) }; + let head = if size == 1 { 16 } else { 8 }; + // Size 0: no data, and the next atom starts at the container's end. + let len = match size { + 0 => head, + 1 => be_u64(b, pos + 8).unwrap_or(0), + n => u64::from(n), + }; + if len < head { + return Ok(false); + } + let body = pos + head as usize; + let data_end = usize::try_from(len).ok().and_then(|n| pos.checked_add(n)); + let next = if size == 0 { Some(end) } else { data_end }; + let inner = data_end.map_or(b.len(), |n| n.min(b.len())); + let more = match (level, kind) { + (Level::Top, b"moov") => atoms(b, body, inner, Level::Moov)?, + (Level::Top | Level::Udta, b"meta") => atoms(b, body + 4, inner, Level::Meta)?, + (Level::Moov, b"trak") => atoms(b, body, inner, Level::Trak)?, + (Level::Moov, b"udta") => atoms(b, body, inner, Level::Udta)?, + (Level::Trak, b"mdia") => atoms(b, body, inner, Level::Mdia)?, + (Level::Mdia, b"minf") => atoms(b, body, inner, Level::Minf)?, + (Level::Minf, b"stbl") => atoms(b, body, inner, Level::Stbl)?, + (Level::Meta, b"ilst") => atoms(b, body, inner, Level::Ilst)?, + (Level::Ilst, _) => atoms(b, body, inner, Level::Tag)?, + (Level::Stbl, b"stts" | b"co64") => table(b, body + 4, 8, inner)?, + (Level::Stbl, b"stsc") => table(b, body + 4, 12, inner)?, + (Level::Stbl, b"stco") => table(b, body + 4, 4, inner)?, + (Level::Stbl, b"stsz") => sample_sizes(b, body, inner)?, + (Level::Mdia, b"hdlr") | (Level::Tag, b"data" | b"name" | b"mean") if len > (b.len() - pos) as u64 => { + return Err(malformed(&format!("`{}` atom", String::from_utf8_lossy(kind)))); + } + _ => true, + }; + match next { + Some(n) if more && n <= end => pos = n, + _ => return Ok(false), + } + } + Ok(true) +} + +/// A sample table's entries, counted at `count`, must fit in the file: symphonia reserves room for +/// all of them first. False if they run past the table's atom (`end`), where symphonia stops. +fn table(b: &[u8], count: usize, entry: u64, end: usize) -> Result { + let Some(n) = be_u32(b, count) else { return Ok(false) }; + let (first, bytes) = (count + 4, u64::from(n) * entry); + if bytes > b.len().saturating_sub(first) as u64 { + return Err(malformed("sample table")); + } + Ok(first as u64 + bytes <= end as u64) +} + +/// Every sample's size must fit in the file: symphonia reads each one into a buffer of that size. +/// Sizes past the atom are never used, as symphonia stops before reading a packet. +fn sample_sizes(b: &[u8], body: usize, end: usize) -> Result { + let (Some(size), Some(count)) = (be_u32(b, body + 4), be_u32(b, body + 8)) else { return Ok(false) }; + let file = b.len() as u64; + if u64::from(size) > file { + return Err(malformed("sample")); + } + if size != 0 { + return Ok(true); + } + let fits = table(b, body + 8, 4, end)?; + let (sizes, _) = b.get(body + 12..end).unwrap_or_default().as_chunks::<4>(); + if sizes.iter().take(count as usize).any(|s| u64::from(u32::from_be_bytes(*s)) > file) { + return Err(malformed("sample")); + } + Ok(fits) +} diff --git a/crates/audio/src/read/native.rs b/crates/audio/src/read/native.rs new file mode 100644 index 00000000..67588476 --- /dev/null +++ b/crates/audio/src/read/native.rs @@ -0,0 +1,660 @@ +//! The native decode path: craft-media's demuxers (ISO BMFF / QuickTime, Matroska / WebM) and +//! WAV / AIFF PCM, feeding [`TrackDecoder`]. +//! [`Native::open`] answers `None` for a stream with no native codec (AAC, MP1/MP2/MP3, FLAC, +//! ALAC, Vorbis, Opus, ...) or a container these demuxers do not read; the caller then uses symphonia. +//! +//! Every container is a [`Packets`] source over a [`Src`], read packet by packet, so a decode +//! streams. Timing follows the containers: MP4 edit lists (AAC priming) and Matroska `CodecDelay` and +//! per-block `DiscardPadding` are applied, so a file decodes to exactly its presentation length. + +use super::aiff::{self, AiffParse}; +use super::id3v2_len; +use super::source::{Src, io_error}; +use super::wav::{self, WavParse}; +use crate::track::TrackDecoder; +use crate::{AudioBuffer, AudioError, AudioInfo, CodecId, CodecParams, Container, PcmFormat, Result}; +use craft_media::{isobmff, matroska, ogg}; + +/// Corrupt packets tolerated before a stream that has produced nothing is an error. +const MAX_ERRORS: usize = 1000; +/// The most frames a packet may declare (as on the symphonia path): decoders allocate a packet's +/// frames up front. +const MAX_PACKET_FRAMES: u64 = 1 << 20; +/// Frames per packet read from a WAV or AIFF file. +const PCM_PACKET_FRAMES: u64 = 4096; + +/// A packet's bytes and the frames to drop from the end of its decoded audio (Matroska +/// `DiscardPadding`). +struct Packet { + data: Vec, + trim_end: u64, +} + +/// Where a track's packets come from. +enum Packets<'a> { + Mp4 { + file: Box, + src: Src<'a>, + track: usize, + next: usize, + /// Media timescale and output rate, for packet start times. + timescale: u32, + rate: u32, + }, + /// Matroska is read packet by packet, which carries each block's `DiscardPadding`. + Mkv { + demux: Box>>, + track: usize, + rate: u32, + /// The track's first packet's timestamp (ffmpeg starts AAC tracks before 0), the decoded + /// timeline's origin. + first_ns: i64, + /// How a packet's place on the decoded timeline is found after a seek. + frames: MkvFrames, + /// A packet read ahead by a seek, returned next. + pending: Option, + }, + /// Interleaved PCM in `start..start + frames * frame_bytes` (WAV, AIFF). + Pcm { src: Src<'a>, start: u64, frames: u64, frame_bytes: u64, next_frame: u64 }, +} + +impl Packets<'_> { + /// The next packet of the track, or `None` at its end. + fn next(&mut self) -> Option> { + let whole = |data: Vec| Packet { data, trim_end: 0 }; + match self { + Packets::Mp4 { file, src, track, next, .. } => { + let i = *next; + (i < file.tracks.get(*track).map_or(0, |t| t.samples.len())).then(|| { + *next += 1; + file.read_sample(src, *track, i).map(whole).map_err(mp4_error) + }) + } + Packets::Pcm { src, start, frames, frame_bytes, next_frame } => { + let n = (*frames - *next_frame).min(PCM_PACKET_FRAMES); + if n == 0 { + return None; + } + let at = *start + *next_frame * *frame_bytes; + *next_frame += n; + // A truncated file holds fewer bytes than its header says: decode what is there. + Some(src.read_upto(at, usize::try_from(n * *frame_bytes).unwrap_or(0)).map(whole)) + } + Packets::Mkv { demux, track, rate, pending, .. } => match pending.take() { + Some(p) => Some(Ok(p)), + None => mkv_next(demux, *track, *rate).map(|r| r.map(|(p, _)| p)), + }, + } + } + + /// Position the source at the latest packet whose decoded audio starts at or before output + /// frame `frame` (counted from the track's first decoded sample) and return where that packet + /// starts. + fn seek(&mut self, frame: u64) -> Result { + // The last index whose start is <= frame, given the starts in order. + fn at_or_before(starts: impl Iterator, frame: u64) -> (usize, u64) { + let mut found = (0, 0); + for (i, s) in starts.enumerate() { + if s > frame { + break; + } + found = (i, s); + } + found + } + match self { + Packets::Mp4 { file, track, next, timescale, rate, .. } => { + let samples = file.tracks.get(*track).map(|t| t.samples.as_slice()).unwrap_or_default(); + let first = samples.first().map_or(0, |s| s.dts); + let starts = + samples.iter().map(|s| rescale(u64::try_from(s.dts.saturating_sub(first)).unwrap_or(0), u64::from(*timescale), u64::from(*rate))); + let (i, s) = at_or_before(starts, frame); + *next = i; + Ok(s) + } + Packets::Pcm { frames, next_frame, .. } => { + *next_frame = frame.min(*frames); + Ok(*next_frame) + } + Packets::Mkv { demux, track, rate, first_ns, frames, pending } => { + let ns = i64::try_from(rescale(frame, u64::from(*rate), 1_000_000_000)).unwrap_or(i64::MAX).saturating_add(*first_ns); + let point = demux.seek(*track, ns).map_err(mkv_error)?; + match mkv_next(demux, *track, *rate) { + Some(Ok((p, pts_ns))) => { + *pending = Some(p); + // Timestamps are rounded to the file's timestamp scale (often 1 ms), so the + // packet's index places it exactly where packets have a known length. + let by_index = point.sample.and_then(|k| frames.start(k)); + Ok(by_index.unwrap_or_else(|| { + rescale_round(u64::try_from(pts_ns.saturating_sub(*first_ns)).unwrap_or(0), 1_000_000_000, u64::from(*rate)) + })) + } + Some(Err(e)) => Err(e), + None => Ok(frame), + } + } + } + } +} + +/// How the decoded-timeline start of a Matroska packet is found from its index. +enum MkvFrames { + /// Every packet decodes to this many frames. + Constant(u64), + /// PCM: the stored sizes of the track's packets, and the bytes per frame. + Pcm { sizes: Vec, frame_bytes: u64 }, + /// From the timestamps. + Timestamps, +} + +impl MkvFrames { + /// The decoded-timeline start of packet `k`, when the packets say. + fn start(&self, k: usize) -> Option { + match self { + MkvFrames::Constant(n) => Some(k as u64 * *n), + MkvFrames::Pcm { sizes, frame_bytes } => Some(sizes.iter().take(k).map(|&s| u64::from(s)).sum::() / *frame_bytes), + MkvFrames::Timestamps => None, + } + } +} + +/// The next packet of Matroska track `track` and its timestamp in nanoseconds. +fn mkv_next(demux: &mut matroska::Demuxer>, track: usize, rate: u32) -> Option> { + loop { + match demux.next_packet() { + Ok(Some(p)) if p.track == track => { + let pad = p.discard_padding_ns.and_then(|ns| u64::try_from(ns).ok()).unwrap_or(0); + let packet = Packet { data: p.data, trim_end: rescale_round(pad, 1_000_000_000, u64::from(rate)) }; + return Some(Ok((packet, p.pts_ns))); + } + Ok(Some(_)) => continue, + Ok(None) => return None, + Err(e) => return Some(Err(mkv_error(e))), + } + } +} + +/// Container read errors: the source failing is [`AudioError::Io`], anything else in the file is +/// [`AudioError::Malformed`]. +fn mp4_error(e: isobmff::Error) -> AudioError { + match e { + isobmff::Error::Io(e) => io_error(e), + e => AudioError::Malformed(e.to_string()), + } +} + +fn mkv_error(e: matroska::Error) -> AudioError { + match e { + matroska::Error::Io(e) => io_error(e), + e => AudioError::Malformed(e.to_string()), + } +} + +/// A track the native path decodes: its packets, its decoder and its container timing. +pub(crate) struct Native<'a> { + packets: Packets<'a>, + dec: TrackDecoder, + info: AudioInfo, + /// Whether `info.frames` came from the container (else it is 0 until decoded). + frames_known: bool, + /// Decoded frames at the start that are not presented (AAC priming). + skip: u64, + /// Presented frames, when the container says. + limit: Option, + /// Set by [`Native::seek`]: running out of packets afterwards is the end of the stream, even + /// when nothing was decoded (a seek to the last frame). + seeked: bool, + /// Decoded-timeline frame (from the first decoded sample) of the next decoded sample. + pos: u64, + /// The first decoded-timeline frame to present: `skip`, or a seek target. + from: u64, + errors: usize, + decoded: bool, + /// The first error that said the stream is unsupported, returned if nothing decodes. + unsupported: Option, +} + +impl<'a> Native<'a> { + /// Open the audio track `track` (the first when `None`) of the file in `src`; `None` when the + /// native path does not decode it. + pub(crate) fn open(src: &Src<'a>, detected: Option, track: Option) -> Option>> { + let head = match src.head_checked() { + Ok(head) => head, + Err(e) => return Some(Err(e)), + }; + let at = after_id3(&head); + let body = head.get(at..).unwrap_or_default(); + // The bytes decide the container when they say; the detection (which may come from the + // extension) only picks among raw streams. + // Formats with one track: any other track index is an error, as for a multi-track file. + let single = |opened: Option>>| match (track, opened) { + (Some(n), Some(Ok(_))) if n > 0 => Some(Err(AudioError::Unsupported(format!("no audio track {n}: the file has one")))), + (_, opened) => opened, + }; + match super::detect_container(&head, None).or(detected) { + Some(Container::Wav) if wav::is_wav(&head) => single(open_wav(src)), + Some(Container::Aiff) if aiff::is_aiff(&head) => single(open_aiff(src)), + Some(c @ (Container::Mp4 | Container::QuickTime)) if head.get(4..8) == Some(b"ftyp") => open_mp4(src, c, track), + Some(c @ (Container::Matroska | Container::WebM)) if body.starts_with(&[0x1A, 0x45, 0xDF, 0xA3]) => open_mkv(src, c, track), + _ => None, + } + } + + /// What the stream is; `frames` is 0 when the container does not say (see `frames_known`). + pub(crate) fn info(&self) -> &AudioInfo { + &self.info + } + + pub(crate) fn frames_known(&self) -> bool { + self.frames_known + } + + /// The next block of presented audio, or `None` at the end. A packet that is damaged or cannot + /// be decoded is replaced by the decoder's concealment (silence of its length), so the audio + /// after it keeps its timing; a stream where nothing decodes is an error. A failing source is + /// returned as [`AudioError::Io`] and skips that packet. + pub(crate) fn next_block(&mut self) -> Result> { + loop { + if self.limit.is_some_and(|l| self.pos >= l.saturating_add(self.skip)) { + return Ok(None); + } + let Some(next) = self.packets.next() else { + // A track that declares no frames (an empty WAV or AIFF) is empty, not broken. + let empty = self.frames_known && self.info.frames == 0 && self.errors == 0; + if !self.decoded && !empty && !(self.seeked && self.errors == 0) { + return Err(self.no_audio("stream contains no audio")); + } + return Ok(None); + }; + let (block, trim_end) = match next { + Ok(p) => match self.dec.decode(&p.data) { + Ok(block) => { + if block.channels.is_empty() { + continue; + } + self.decoded = true; + (block, p.trim_end) + } + Err(e) => { + self.error(e)?; + (self.dec.conceal(Some(&p.data))?, p.trim_end) + } + }, + // The source failing is not damage in the file: the caller decides whether to retry. + Err(e @ AudioError::Io(_)) => return Err(e), + Err(e) => { + self.error(e)?; + (self.dec.conceal(None)?, 0) + } + }; + if let Some(out) = self.present(block, trim_end) { + return Ok(Some(out)); + } + } + } + + /// The part of a decoded `block` (less its last `trim_end` frames) that is presented. + fn present(&mut self, mut block: AudioBuffer, trim_end: u64) -> Option { + let n = (block.frames() as u64).saturating_sub(trim_end); + let start = self.pos; + self.pos += n; + let end = match self.limit { + Some(l) => self.pos.min(l.saturating_add(self.skip)), + None => self.pos, + }; + let first = start.max(self.from); + if first >= end { + return None; + } + let (a, b) = ((first - start) as usize, (end - start) as usize); + for c in &mut block.channels { + c.truncate(b); + c.drain(..a); + for s in c.iter_mut().filter(|s| !s.is_finite()) { + *s = 0.0; + } + } + Some(block) + } + + /// Note a packet that could not be read or decoded. A stream where nothing decodes is an + /// error: unsupported when a packet said so, else malformed. + fn error(&mut self, e: AudioError) -> Result<()> { + self.errors += 1; + self.info.damaged += 1; + log::debug!("concealing a packet that failed: {e}"); + let msg = e.to_string(); + if matches!(e, AudioError::Unsupported(_)) && self.unsupported.is_none() { + self.unsupported = Some(e); + } + if self.errors > MAX_ERRORS && !self.decoded { + return Err(self.no_audio(&msg)); + } + Ok(()) + } + + fn no_audio(&mut self, msg: &str) -> AudioError { + self.unsupported.take().unwrap_or_else(|| AudioError::Malformed(format!("no decodable packets: {msg}"))) + } + + /// Continue from presented frame `frame`: back up the decoder's pre-roll, decode from the + /// packet before it and drop what comes before `frame`. Returns the frame decoding resumes at + /// (`frame`, or the end of the stream). + pub(crate) fn seek(&mut self, frame: u64) -> Result { + let frame = match self.limit { + Some(l) => frame.min(l), + None => frame, + }; + let frame = if self.frames_known { frame.min(self.info.frames) } else { frame }; + self.seeked = true; + let target = frame.saturating_add(self.skip); + let start = self.packets.seek(target.saturating_sub(self.dec.preroll() as u64))?; + self.dec.reset(); + self.pos = start; + self.from = target; + Ok(frame) + } +} + +/// Where a stream starts: after any ID3v2 tags. +fn after_id3(b: &[u8]) -> usize { + let mut at = 0; + while let Some(len) = b.get(at..).and_then(id3v2_len) { + at = at.saturating_add(len); + } + at +} + +/// A native track from its parts; `None` when [`TrackDecoder`] does not decode the codec (it is +/// then left to symphonia). +fn native<'a>( + container: Container, + packets: Packets<'a>, + params: &CodecParams, + lookahead: &[Vec], + skip: u64, + limit: Option, + frames: Option, +) -> Option> { + let lookahead: Vec<&[u8]> = lookahead.iter().map(Vec::as_slice).collect(); + let dec = TrackDecoder::new(params, &lookahead).ok()?; + let mut info = AudioInfo::new(container, dec.codec(), dec.sample_format(), dec.sample_rate(), dec.layout(), frames.unwrap_or(0)); + if dec.channels() == 0 || dec.sample_rate() == 0 { + info.frames = 0; + } + Some(Native { + packets, + info, + frames_known: frames.is_some() && dec.channels() > 0 && dec.sample_rate() > 0, + dec, + skip, + limit, + pos: 0, + from: skip, + errors: 0, + decoded: false, + seeked: false, + unsupported: None, + }) +} + +fn open_wav<'a>(src: &Src<'a>) -> Option>> { + let layout = match wav::parse(src) { + Ok(WavParse::Native(l)) => l, + Ok(WavParse::Foreign { .. }) => return None, + Err(e) => return Some(Err(e)), + }; + let mut n = pcm_track(src, Container::Wav, layout.info.clone(), layout.kind, layout.data_start, layout.data_len); + if let Some(Ok(n)) = &mut n { + n.info.bwf = layout.info.bwf; + } + n +} + +fn open_aiff<'a>(src: &Src<'a>) -> Option>> { + match aiff::parse(src) { + Ok(AiffParse::Native(l)) => pcm_track(src, Container::Aiff, l.info, l.kind, l.data_start, l.data_len), + Ok(AiffParse::Foreign) => None, + Err(e) => Some(Err(e)), + } +} + +/// A WAV or AIFF file's samples as a PCM track. +fn pcm_track<'a>( + src: &Src<'a>, + container: Container, + info: AudioInfo, + kind: crate::pcm::PcmKind, + start: u64, + len: u64, +) -> Option>> { + let channels = usize::from(info.channels); + let frame_bytes = (kind.bytes() * channels) as u64; + let (bits, float, big_endian, unsigned) = match kind { + crate::pcm::PcmKind::Int { bytes, big_endian, unsigned } => ((bytes * 8) as u16, false, big_endian, unsigned), + crate::pcm::PcmKind::Float { bytes, big_endian } => ((bytes * 8) as u16, true, big_endian, false), + }; + let params = CodecParams::new(CodecId::Pcm).with_sample_rate(info.sample_rate).with_channels(info.channels).with_pcm(if float { + PcmFormat::float(bits, big_endian) + } else { + PcmFormat::int(bits, !unsigned, big_endian) + }); + let frames = len.checked_div(frame_bytes).unwrap_or(0).min(info.frames); + let packets = Packets::Pcm { src: src.clone(), start, frames, frame_bytes, next_frame: 0 }; + let mut n = native(container, packets, ¶ms, &[], 0, None, Some(frames))?; + // The file's own description wins: its layout (an extensible mask) and stored sample format. + n.info = AudioInfo { frames, ..info }; + Some(Ok(n)) +} + +/// The `n`-th audio track (the first when `None`) among `kinds`, as an index into them. +fn audio_track(kinds: impl Iterator, n: Option) -> Option> { + let audio: Vec = kinds.enumerate().filter(|&(_, audio)| audio).map(|(i, _)| i).collect(); + if audio.is_empty() { + return None; + } + let n = n.unwrap_or(0); + Some(audio.get(n).copied().ok_or_else(|| AudioError::Unsupported(format!("no audio track {n}: the file has {}", audio.len())))) +} + +/// An MP4 / QuickTime file's audio track, if its codec is native. Samples that claim to lie +/// beyond the end of the file are refused rather than read leniently. +fn open_mp4<'a>(src: &Src<'a>, container: Container, track: Option) -> Option>> { + let file = match demuxed(isobmff::open(src), mp4_failure)? { + Ok(file) => file, + Err(e) => return Some(Err(e)), + }; + let track = match audio_track(file.tracks.iter().map(|t| t.kind == isobmff::TrackKind::Audio), track)? { + Ok(t) => t, + Err(e) => return Some(Err(e)), + }; + let t = file.tracks.get(track)?; + // A track with no samples (or sample sizes the demuxer refused) goes to the other path, whose + // size checks name what is wrong. + if t.samples.is_empty() { + return None; + } + if t.audio().is_some_and(|a| u64::from(a.samples_per_packet.max(a.const_frames_per_packet)) > MAX_PACKET_FRAMES) { + return Some(Err(AudioError::Malformed("the file declares packets far longer than any codec uses".into()))); + } + let size = src.size(); + if t.samples.iter().any(|s| s.offset.checked_add(u64::from(s.size)).is_none_or(|end| end > size)) { + return Some(Err(AudioError::Malformed("MP4 sample is larger than the file".into()))); + } + let params = t.codec_params(); + let lookahead: Vec> = + (0..t.samples.len().min(crate::track::LOOKAHEAD_PACKETS)).filter_map(|i| file.read_sample(src, track, i).ok()).collect(); + let timescale = t.timescale; + // Probe the decoder for the output rate before the timing is worked out. + let rate = TrackDecoder::new(¶ms, &lookahead.iter().map(Vec::as_slice).collect::>()).ok()?.sample_rate(); + // A leading media edit (AAC priming) is skipped, and the edit's duration is the + // presentation length. Empty edits and multi-segment lists play the media as it is. + let (skip, limit) = match t.edits.as_slice() { + [e] if e.media_time >= 0 && e.media_rate == 0x0001_0000 => { + let skip = rescale(e.media_time.unsigned_abs(), u64::from(timescale), u64::from(rate)); + let limit = (e.segment_duration > 0).then(|| rescale(e.segment_duration, u64::from(file.timescale), u64::from(rate))); + (skip, limit) + } + _ => (0, None), + }; + let media: u64 = t.samples.iter().map(|s| u64::from(s.duration)).sum(); + let frames = limit.unwrap_or_else(|| rescale(media, u64::from(timescale), u64::from(rate)).saturating_sub(skip)); + let packets = Packets::Mp4 { file: Box::new(file), src: src.clone(), track, next: 0, timescale, rate }; + native(container, packets, ¶ms, &lookahead, skip, limit, Some(frames)).map(Ok) +} + +/// A Matroska / WebM file's audio track, if its codec is native. +fn open_mkv<'a>(src: &Src<'a>, container: Container, track: Option) -> Option>> { + let demux = match demuxed(matroska::Demuxer::new(src.clone()), mkv_failure)? { + Ok(demux) => demux, + Err(e) => return Some(Err(e)), + }; + let file = demux.file(); + let track = match audio_track(file.tracks.iter().map(|t| t.kind == matroska::TrackKind::Audio), track)? { + Ok(t) => t, + Err(e) => return Some(Err(e)), + }; + let t = file.tracks.get(track)?; + if !t.frames_readable() { + return None; + } + let params = t.codec_params(); + let lookahead: Vec> = + (0..t.samples.len().min(crate::track::LOOKAHEAD_PACKETS)).filter_map(|i| demux.read_sample(track, i).ok()).collect(); + let probe = TrackDecoder::new(¶ms, &lookahead.iter().map(Vec::as_slice).collect::>()).ok()?; + let rate = probe.sample_rate(); + let frames_of = match (probe.frames_per_packet(), params.pcm) { + (Some(n), _) => MkvFrames::Constant(n as u64), + (None, Some(pcm)) => { + let frame_bytes = u64::from(pcm.bits.div_ceil(8)) * u64::from(params.channels.unwrap_or(1).max(1)); + MkvFrames::Pcm { sizes: t.samples.iter().map(|s| s.size).collect(), frame_bytes: frame_bytes.max(1) } + } + _ => MkvFrames::Timestamps, + }; + // `CodecDelay` (AAC priming) is decoded audio that is not presented. + let skip = rescale_round(t.codec_delay_ns, 1_000_000_000, u64::from(rate)); + // The presentation ends with the last frame (timestamps, with `CodecDelay` already taken off, + // are in ticks of `timebase` seconds); known only when the last frame has a duration. + // The decoded timeline starts at the first frame (negative when `CodecDelay` was taken off), + // and the presentation starts `CodecDelay` later. Both are counted in timestamp ticks, the + // delay rounded to one, so a file whose first frame is stored at 0 presents from exactly 0 + // although its timestamps are rounded (AAC's 1024-frame priming is 21.33 ms, stored as -21). + let (num, den) = t.timebase; + let last = t.samples.iter().max_by_key(|s| s.pts); + let delay_ticks = i64::try_from(rescale_round(t.codec_delay_ns.saturating_mul(den), num.saturating_mul(1_000_000_000), 1)).unwrap_or(i64::MAX); + let start = t.samples.iter().map(|s| s.pts).min().unwrap_or(0).saturating_add(delay_ticks); + let frames = last.filter(|s| s.duration > 0).and_then(|s| { + let end = u64::try_from(s.pts.saturating_add(i64::try_from(s.duration).ok()?).saturating_sub(start)).ok()?; + Some(rescale_round(end.saturating_mul(num), den, u64::from(rate))) + }); + let mut demux = demux; + // The first packet's own timestamp is the decoded timeline's origin; read it and rewind. + let first = mkv_next(&mut demux, track, rate); + let first_ns = match first { + Some(Ok((_, pts_ns))) => pts_ns, + _ => 0, + }; + demux.seek(track, i64::MIN).ok()?; + let packets = Packets::Mkv { demux: Box::new(demux), track, rate, first_ns, frames: frames_of, pending: None }; + native(container, packets, ¶ms, &lookahead, skip, frames, frames).map(Ok) +} + +/// The audio tracks of `src` the native path reads, described for a track list; `None` when it +/// does not read the container. +pub(crate) fn tracks(src: &Src<'_>, detected: Option) -> Option>> { + let head = match src.head_checked() { + Ok(head) => head, + Err(e) => return Some(Err(e)), + }; + match super::detect_container(&head, None).or(detected) { + Some(Container::Mp4 | Container::QuickTime) => demuxed(isobmff::open(src), mp4_failure) + .map(|r| r.map(|file| file.tracks.iter().filter(|t| t.kind == isobmff::TrackKind::Audio).map(isobmff::Track::codec_params).collect())), + Some(Container::Matroska | Container::WebM) => demuxed(matroska::open(src.clone()), mkv_failure) + .map(|r| r.map(|file| file.tracks.iter().filter(|t| t.kind == matroska::TrackKind::Audio).map(matroska::Track::codec_params).collect())), + Some(Container::Ogg) => demuxed(ogg::open(src), ogg_failure) + .map(|r| r.map(|file| file.streams.iter().filter(|s| s.codec.codec_id().is_audio()).map(ogg::Stream::codec_params).collect())), + _ => None, + } +} + +/// What is damaged in `src`, when it is in a container the native demuxers read and they find it +/// truncated or structurally invalid: the error to return when symphonia cannot read it either. +/// Ogg is left out: `ogg::open` skips damaged pages and calls a file with none it can read not +/// Ogg, and symphonia reports a damaged Ogg stream as malformed itself. +#[cfg(feature = "symphonia")] +pub(crate) fn damage(src: &Src<'_>, detected: Option) -> Option { + let head = src.head_checked().ok()?; + let failure = match super::detect_container(&head, None).or(detected) { + Some(Container::Mp4 | Container::QuickTime) => mp4_failure(isobmff::open(src).err()?), + Some(Container::Matroska | Container::WebM) => mkv_failure(matroska::open(src.clone()).err()?), + _ => return None, + }; + match failure { + Failure::Damaged(m) => Some(m), + Failure::Io(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => Some(e.to_string()), + _ => None, + } +} + +/// Why a craft-media demuxer could not read a file. +enum Failure { + /// The source failed to read. + Io(std::io::Error), + /// The file is damaged: truncated, or structurally invalid. + Damaged(String), + /// Anything else: not this container, or a feature the demuxer does not read. + Other, +} + +/// A craft-media demuxer's result for the native path: the source's failure to read as +/// [`AudioError::Io`], and `None` when the demuxer does not read the file, so the other path +/// tries it. Without symphonia there is no other path, so a damaged file is +/// [`AudioError::Malformed`]; with it, the file is malformed when symphonia cannot read it either +/// ([`damage`]). +fn demuxed(r: std::result::Result, why: fn(E) -> Failure) -> Option> { + let e = match r { + Ok(v) => return Some(Ok(v)), + Err(e) => why(e), + }; + match e { + Failure::Io(e) if e.kind() != std::io::ErrorKind::UnexpectedEof => Some(Err(AudioError::Io(e))), + Failure::Io(e) if cfg!(not(feature = "symphonia")) => Some(Err(AudioError::Malformed(e.to_string()))), + Failure::Damaged(m) if cfg!(not(feature = "symphonia")) => Some(Err(AudioError::Malformed(m))), + _ => None, + } +} + +fn mp4_failure(e: isobmff::Error) -> Failure { + match e { + isobmff::Error::Io(e) => Failure::Io(e), + e @ (isobmff::Error::Truncated(_) | isobmff::Error::Invalid(_) | isobmff::Error::NoMoov) => Failure::Damaged(e.to_string()), + _ => Failure::Other, + } +} + +fn mkv_failure(e: matroska::Error) -> Failure { + match e { + matroska::Error::Io(e) => Failure::Io(e), + e @ matroska::Error::Invalid(_) => Failure::Damaged(e.to_string()), + _ => Failure::Other, + } +} + +fn ogg_failure(e: ogg::Error) -> Failure { + match e { + ogg::Error::Io(e) => Failure::Io(e), + e @ ogg::Error::Invalid(_) => Failure::Damaged(e.to_string()), + _ => Failure::Other, + } +} + +/// `v` in units of `from` per second, in units of `to` per second (truncated). +fn rescale(v: u64, from: u64, to: u64) -> u64 { + if from == 0 { 0 } else { u64::try_from(u128::from(v) * u128::from(to) / u128::from(from)).unwrap_or(u64::MAX) } +} + +/// [`rescale`], rounded to the nearest unit: for durations stored in a coarser unit than samples +/// (Matroska's nanoseconds hold 1024 samples at 48 kHz as 21 333 333 ns). +fn rescale_round(v: u64, from: u64, to: u64) -> u64 { + if from == 0 { 0 } else { u64::try_from((u128::from(v) * u128::from(to) + u128::from(from) / 2) / u128::from(from)).unwrap_or(u64::MAX) } +} diff --git a/crates/audio/src/read/source.rs b/crates/audio/src/read/source.rs new file mode 100644 index 00000000..a7a4ccca --- /dev/null +++ b/crates/audio/src/read/source.rs @@ -0,0 +1,135 @@ +//! What the decoders read from: borrowed bytes or a shared [`ByteSource`], with small typed reads. + +use crate::{AudioError, Result}; +use craft_media::ByteSource; +#[cfg(feature = "symphonia")] +use std::borrow::Cow; +use std::io; +use std::sync::Arc; + +/// A source a [`crate::Decoder`] can own: a [`ByteSource`] that is `Send + Sync + 'static`, so +/// the decoder can move between threads. On single-threaded WebAssembly (no `atomics`), where +/// sources such as a JS `Blob` reader are not `Send`, any `ByteSource + 'static` is one. +/// Implemented for every type that qualifies. +#[cfg(not(all(target_arch = "wasm32", not(target_feature = "atomics"))))] +pub trait SharedSource: ByteSource + Send + Sync + 'static {} +#[cfg(not(all(target_arch = "wasm32", not(target_feature = "atomics"))))] +impl SharedSource for T {} + +/// A source a [`crate::Decoder`] can own: a [`ByteSource`] that is `Send + Sync + 'static`, so +/// the decoder can move between threads. On single-threaded WebAssembly (no `atomics`), where +/// sources such as a JS `Blob` reader are not `Send`, any `ByteSource + 'static` is one. +/// Implemented for every type that qualifies. +#[cfg(all(target_arch = "wasm32", not(target_feature = "atomics")))] +pub trait SharedSource: ByteSource + 'static {} +#[cfg(all(target_arch = "wasm32", not(target_feature = "atomics")))] +impl SharedSource for T {} + +/// The bytes a decoder reads: a borrowed buffer, or a source shared with the caller (a file, a +/// network-backed buffer). +#[derive(Clone)] +pub(crate) enum Src<'a> { + Slice(&'a [u8]), + Shared(Arc), +} + +impl ByteSource for Src<'_> { + fn len(&self) -> u64 { + match self { + Src::Slice(b) => <[u8]>::len(b) as u64, + Src::Shared(s) => s.len(), + } + } + + fn read_at(&self, offset: u64, buf: &mut [u8]) -> io::Result<()> { + match self { + Src::Slice(b) => b.read_at(offset, buf), + Src::Shared(s) => s.read_at(offset, buf), + } + } +} + +impl Src<'_> { + /// The source's length in bytes. + pub(crate) fn size(&self) -> u64 { + ByteSource::len(self) + } + + /// `n` bytes at `pos`, or `None` when they are not all there. + pub(crate) fn bytes(&self, pos: u64, n: usize) -> Option> { + if let Src::Slice(b) = self { + let start = usize::try_from(pos).ok()?; + return b.get(start..start.checked_add(n)?).map(<[u8]>::to_vec); + } + let mut v = vec![0; n]; + self.read_at(pos, &mut v).ok()?; + Some(v) + } + + /// `n` bytes at `pos` for decoding: missing bytes are [`AudioError::Malformed`], any other + /// failure of the source is [`AudioError::Io`]. + pub(crate) fn read(&self, pos: u64, n: usize) -> Result> { + let mut v = vec![0; n]; + self.read_at(pos, &mut v).map_err(io_error)?; + Ok(v) + } + + /// Up to `n` bytes at `pos` for decoding, fewer near the end of the source; errors as + /// [`Src::read`]. + pub(crate) fn read_upto(&self, pos: u64, n: usize) -> Result> { + let n = n.min(usize::try_from(self.size().saturating_sub(pos)).unwrap_or(usize::MAX)); + self.read(pos, n) + } + + /// Up to `n` bytes at `pos`: fewer near the end of the source. + pub(crate) fn bytes_upto(&self, pos: u64, n: usize) -> Vec { + let n = n.min(usize::try_from(self.size().saturating_sub(pos)).unwrap_or(usize::MAX)); + self.bytes(pos, n).unwrap_or_default() + } + + fn array(&self, pos: u64) -> Option<[u8; N]> { + let mut a = [0; N]; + self.read_at(pos, &mut a).ok()?; + Some(a) + } + + pub(crate) fn tag(&self, pos: u64) -> Option<[u8; 4]> { + self.array(pos) + } + + pub(crate) fn le_u32(&self, pos: u64) -> Option { + self.array(pos).map(u32::from_le_bytes) + } + + pub(crate) fn be_u32(&self, pos: u64) -> Option { + self.array(pos).map(u32::from_be_bytes) + } + + /// The first bytes of the source, enough to detect its format; a source that fails to read + /// them is [`AudioError::Io`], so the failure is not taken for an unknown format. + pub(crate) fn head_checked(&self) -> Result> { + self.read_upto(0, 64 * 1024) + } + + /// The whole source in memory: borrowed for a buffer, read once for a shared source. + #[cfg(feature = "symphonia")] + pub(crate) fn whole(&self) -> Result> { + match self { + Src::Slice(b) => Ok(Cow::Borrowed(b)), + Src::Shared(s) => { + let n = usize::try_from(s.len()).map_err(|_| AudioError::TooLarge("the source does not fit in memory".into()))?; + let mut v = Vec::new(); + v.try_reserve_exact(n).map_err(|_| AudioError::TooLarge("the source does not fit in memory".into()))?; + v.resize(n, 0); + s.read_at(0, &mut v).map_err(io_error)?; + Ok(Cow::Owned(v)) + } + } + } +} + +/// A read failure as an [`AudioError`]: bytes past the end are a truncated or lying file +/// ([`AudioError::Malformed`]); anything else is the source failing ([`AudioError::Io`]). +pub(crate) fn io_error(e: io::Error) -> AudioError { + if e.kind() == io::ErrorKind::UnexpectedEof { AudioError::Malformed(e.to_string()) } else { AudioError::Io(e) } +} diff --git a/crates/audio/src/read/symph.rs b/crates/audio/src/read/symph.rs new file mode 100644 index 00000000..f53aca1b --- /dev/null +++ b/crates/audio/src/read/symph.rs @@ -0,0 +1,899 @@ +//! Decode path for everything that isn't native PCM WAV/AIFF, backed by `symphonia`. + +use std::borrow::Cow; +use std::io::{Cursor, Seek, SeekFrom}; +use std::panic::{AssertUnwindSafe, catch_unwind}; + +use symphonia::core::audio::Channels; +use symphonia::core::codecs::audio::AudioDecoder; +use symphonia::core::codecs::audio::well_known::{ + CODEC_ID_AAC, CODEC_ID_ADPCM_G722, CODEC_ID_ADPCM_G726, CODEC_ID_ADPCM_G726LE, CODEC_ID_ADPCM_IMA_QT, CODEC_ID_ADPCM_IMA_WAV, CODEC_ID_ADPCM_MS, + CODEC_ID_ALAC, CODEC_ID_FLAC, CODEC_ID_MP1, CODEC_ID_MP2, CODEC_ID_MP3, CODEC_ID_OPUS, CODEC_ID_PCM_ALAW, CODEC_ID_PCM_F32BE, CODEC_ID_PCM_F32LE, + CODEC_ID_PCM_F64BE, CODEC_ID_PCM_F64BE_PLANAR, CODEC_ID_PCM_F64LE, CODEC_ID_PCM_MULAW, CODEC_ID_PCM_S32LE, CODEC_ID_VORBIS, +}; +use symphonia::core::codecs::audio::{AudioCodecId, AudioCodecParameters, AudioDecoderOptions, CODEC_ID_NULL_AUDIO}; +use symphonia::core::errors::Error as SymError; +use symphonia::core::formats::probe::{Hint, ProbeOptions, ProbeableFormat, Score, Scoreable}; +use symphonia::core::formats::{FormatOptions, FormatReader, SeekMode, SeekTo, Track}; +use symphonia::core::io::{MediaSourceStream, MediaSourceStreamOptions, ScopedStream}; +use symphonia::core::meta::MetadataOptions; +use symphonia::core::units::{TimeBase, Timestamp}; + +use super::id3v2_len; +use crate::pcm::{MAX_DECODE_SAMPLES, be_u32}; +use crate::{AudioBuffer, AudioError, AudioInfo, ChannelLayout, CodecId, Container, Result, SampleFormat}; + +/// The most frames a packet may declare: decoders allocate a packet's frames up front, and ALAC +/// takes its frame length from the file (encoders use 4096). +const MAX_PACKET_FRAMES: u64 = 1 << 20; + +/// How far [`first_marker`] looks for the first format marker: the same 1 MiB a default symphonia +/// probe scans. +const MAX_PREFIX_SCAN: usize = 1 << 20; + +/// How many MPEG/ADTS candidates [`first_marker`] may hand to the readers' expensive `Scoreable` +/// implementations. A crafted 1 MiB prefix of repeated sync markers otherwise asks for hundreds +/// of thousands of scores, each allowed up to symphonia's 16 KiB score depth. Once the budget is +/// spent, the scan stops at the next audio candidate: a container-like byte string later in an +/// unscored audio payload must not become authoritative. A container marker reached first is still +/// checked; an unchecked one the probe selects is refused after probing. +const MAX_AUDIO_SCORE_ATTEMPTS: usize = 256; + +/// A per-scan budget for `Scoreable::score` calls. Taking a slot consumes it even when the score +/// rejects the candidate, so a stream of false syncs cannot multiply the work. +struct ScoreBudget { + used: usize, +} + +impl ScoreBudget { + /// Start a scan with no expensive score attempts spent. + fn new() -> Self { + Self { used: 0 } + } + + /// Spend one score attempt, returning false once the per-scan budget is exhausted. + fn take(&mut self) -> bool { + if self.exhausted() { + return false; + } + self.used = self.used.saturating_add(1); + true + } + + /// Whether no attempt is left for this scan. + fn exhausted(&self) -> bool { + self.used >= MAX_AUDIO_SCORE_ATTEMPTS + } +} + +/// Turn a symphonia error into the audio layer's error, keeping "unsupported" and "too large" +/// distinct from a malformed file. +fn map_err(e: SymError) -> AudioError { + match e { + SymError::Unsupported(m) => AudioError::Unsupported(m.to_string()), + SymError::LimitError(m) => AudioError::TooLarge(m.to_string()), + other => AudioError::Malformed(other.to_string()), + } +} + +/// Run `f`, turning a panic inside the third-party decoder into an error. +fn guarded(f: impl FnOnce() -> Result) -> Result { + catch_unwind(AssertUnwindSafe(f)).unwrap_or_else(|_| Err(AudioError::Malformed("decoder failed on malformed input".into()))) +} + +/// What [`open`] found: a reader, or a stream that holds no audio frames and has no reader. +enum Opened { + Reader(Box), + Empty(AudioInfo), +} + +/// Decoder options for every compressed format. Gapless playback used to be a +/// `FormatOptions` flag in symphonia 0.5; in 0.6 the decoder option (on by default) trims the +/// encoder delay and padding that the demuxer reports on each packet. +fn decoder_options() -> AudioDecoderOptions { + AudioDecoderOptions::default().gapless(true) +} + +/// The first byte after any ID3v2 tags at the start of `b`. Symphonia's tag reader allocates +/// whatever a frame claims, so the tags are skipped, not read: they are not used here. +fn skip_id3v2(b: &[u8]) -> usize { + let mut at = 0usize; + loop { + match b.get(at..).and_then(id3v2_len) { + Some(n) if n > 0 => at = at.saturating_add(n), + _ => return at, + } + } +} + +/// The first format marker the probe can reach from `start`, within its 1 MiB scan limit. +#[derive(Clone, Copy)] +enum FirstMarker { + Flac(usize), + Caf(usize), + Mp4(usize), + Wav(usize), + /// MPEG audio, Ogg, Matroska or AIFF: formats with no separate container size check. + Audio, +} + +/// What the preflight container checks covered before the probe built a reader. +enum Precheck { + /// A zero-frame FLAC: returned without a reader, because Symphonia 0.6's reader needs a first + /// audio frame to resynchronize to. + Empty(AudioInfo), + /// The size checks ran for the container the probe reports under this short name. + Checked(&'static str), + /// No container size check was needed. + Unchecked, +} + +/// Probe short names whose demuxer allocates from sizes in the file, so the matching container +/// check has to run before the stream is decoded. +const SIZE_CHECKED_PROBE_FORMATS: &[&str] = &["caf", "flac", "isomp4", "wave"]; + +/// The container markers to check for the family `detected` names. `None` is a raw audio stream +/// (MP3, Ogg, AIFF), whose payload may contain container-like bytes: the unrestricted scan below +/// finds a container only when one starts the stream, before any audio marker. +fn container_markers(detected: Option) -> Option<&'static [&'static [u8; 4]]> { + match detected { + // RF64 and BW64 are the >4 GiB RIFF forms. + Some(Container::Wav) => Some(&[b"RIFF", b"RF64", b"BW64"]), + Some(Container::Flac) => Some(&[b"fLaC"]), + Some(Container::Caf) => Some(&[b"caff"]), + // MP4/M4A put `ftyp` after the first atom's size field (an `.aac` name may hide one). + Some(Container::Mp4 | Container::QuickTime | Container::Adts) => Some(&[b"ftyp"]), + Some(Container::Aiff | Container::MpegAudio | Container::Ogg) => None, + // Unknown files may carry junk before the container; symphonia's probe scans for markers. + _ => Some(&[b"caff", b"ftyp", b"fLaC", b"RIFF"]), + } +} + +/// The probe short name a container marker stands for. +fn expected_probe_format(marker: &[u8; 4]) -> &'static str { + match marker { + b"caff" => "caf", + // The isomp4 reader reads the `ftyp` atom. + b"ftyp" => "isomp4", + b"fLaC" => "flac", + // RIFF, RF64 and BW64. + _ => "wave", + } +} + +/// Find the first marker of `markers` at or after `at`, within symphonia's probe depth. A RIFF +/// form other than WAVE is skipped, as the probe skips it and reads a WAVE behind it. +fn find_container_marker<'a>(bytes: &[u8], at: usize, markers: &'a [&'static [u8; 4]]) -> Option<(usize, &'a [u8; 4])> { + let end = bytes.len().min(at.saturating_add(MAX_PREFIX_SCAN)); + for pos in at..end.saturating_sub(3) { + let window = bytes.get(pos..pos + 4)?; + if let Some(marker) = markers.iter().find(|marker| window == marker.as_slice()) { + if matches!(*marker, b"RIFF" | b"RF64" | b"BW64") && bytes.get(pos + 8..pos + 12) != Some(&b"WAVE"[..]) { + continue; + } + return Some((pos, *marker)); + } + } + None +} + +/// The scan variant for a marker found by [`find_container_marker`], with the probe short name its +/// size check stands for. `ftyp` sits four bytes into its atom, so the MP4 check starts there. +fn marker_kind(at: usize, marker: &[u8; 4]) -> (FirstMarker, &'static str) { + match marker { + b"fLaC" => (FirstMarker::Flac(at), expected_probe_format(marker)), + b"caff" => (FirstMarker::Caf(at), expected_probe_format(marker)), + b"ftyp" => (FirstMarker::Mp4(at.saturating_sub(4)), expected_probe_format(marker)), + _ => (FirstMarker::Wav(at), expected_probe_format(marker)), + } +} + +/// The probe short name a container scan variant stands for. +fn marker_name(marker: FirstMarker) -> &'static str { + match marker { + FirstMarker::Flac(_) => "flac", + FirstMarker::Caf(_) => "caf", + FirstMarker::Mp4(_) => "isomp4", + FirstMarker::Wav(_) => "wave", + FirstMarker::Audio => "", + } +} + +/// Whether `rest` starts with one of `R`'s exact registered probe markers. +fn has_probe_marker>(rest: &[u8]) -> bool { + R::probe_data().iter().any(|data| data.spec.markers.iter().any(|marker| rest.starts_with(marker))) +} + +/// Score `R` at `at` exactly as symphonia's private probe does: `Supported` wins, EOF and every +/// other scoring error mean "not this format", while a real I/O error aborts the probe. +fn scores_supported(stream: &mut MediaSourceStream<'_>, at: usize, depth: u16) -> Result { + stream.seek(SeekFrom::Start(u64::try_from(at).unwrap_or(u64::MAX)))?; + match R::score(ScopedStream::new(stream, u64::from(depth))) { + Ok(Score::Supported(_)) => Ok(true), + Err(SymError::IoError(e)) if e.kind() != std::io::ErrorKind::UnexpectedEof => Err(AudioError::Malformed(e.to_string())), + _ => Ok(false), + } +} + +/// Whether `b[at..]` is an MPEG/ADTS candidate the probe would actually select: an exact +/// `probe_data()` marker and a `Supported` score from the same reader. A loose `0xFFEx` pair is +/// not enough: `0xFFE0` is neither a registered marker nor a valid frame, so it must not stop the +/// scan before a later FLAC/CAF/MP4 candidate. +fn audio_sync_at<'a>(b: &'a [u8], at: usize, stream: &mut Option>, budget: &mut ScoreBudget) -> Result { + let rest = b.get(at..).unwrap_or_default(); + let adts = has_probe_marker::>(rest); + let mpa = has_probe_marker::>(rest); + if !adts && !mpa { + return Ok(false); + } + if stream.is_none() { + *stream = Some(MediaSourceStream::new(Box::new(Cursor::new(b)), MediaSourceStreamOptions::default())); + } + let Some(mss) = stream.as_mut() else { return Ok(false) }; + let depth = ProbeOptions::default().max_score_depth; + if adts && budget.take() && scores_supported::>(mss, at, depth)? { + return Ok(true); + } + if mpa && budget.take() && scores_supported::>(mss, at, depth)? { + return Ok(true); + } + Ok(false) +} + +/// Find the first marker that can start a container or audio stream. Symphonia's `Probe::probe` +/// constructs the reader itself and exposes no pre-construction hook, while checking every magic +/// in the scan window would validate bytes the probe never reaches (for example a valid MP3's +/// audio payload). Stopping at the first marker avoids that: it is the candidate the probe selects +/// for FLAC, CAF, MP4, Ogg and Matroska, and for WAV/AIFF once their full form is verified below. +/// MPEG/ADTS candidates are checked with the readers' own `probe_data()`/`Scoreable` APIs, so a +/// false sync byte pair cannot stop the scan. If that scoring budget runs out, the scan stops at +/// the next audio candidate instead of trusting a container-like byte string later in the unscored +/// audio payload. A container marker reached before any such candidate is still preflighted; if +/// the probe selects a size-checked container the scan did not check, `open` refuses it. +/// +/// WAV candidates include RF64/BW64 so `check_info` can run, although symphonia 0.6 registers only +/// the RIFF marker. Native PCM RF64/BW64 is parsed by `wav::parse` before this path; a foreign +/// RF64/BW64 stream therefore remains unsupported by the probe rather than misdetected. +fn first_marker(b: &[u8], start: usize) -> Result> { + first_marker_with_budget(b, start, &mut ScoreBudget::new()) +} + +/// [`first_marker`] with an explicit score budget, so a test can prove a prefix crowded with +/// false sync markers performs only the bounded number of reader scores. +fn first_marker_with_budget(b: &[u8], start: usize, budget: &mut ScoreBudget) -> Result> { + let end = b.len().min(start.saturating_add(MAX_PREFIX_SCAN)); + let mut at = start; + let mut audio_probe = None; + while at.saturating_add(4) <= end { + let rest = b.get(at..).unwrap_or_default(); + if rest.get(..4) == Some(b"fLaC".as_slice()) { + return Ok(Some(FirstMarker::Flac(at))); + } + if rest.get(..4) == Some(b"caff".as_slice()) { + return Ok(Some(FirstMarker::Caf(at))); + } + if rest.get(..4) == Some(b"OggS".as_slice()) || rest.get(..4) == Some(b"\x1A\x45\xDF\xA3".as_slice()) || super::aiff::is_aiff(rest) { + return Ok(Some(FirstMarker::Audio)); + } + if super::wav::is_wav(rest) { + return Ok(Some(FirstMarker::Wav(at))); + } + if let (Some(&b0), Some(&b1)) = (b.get(at), b.get(at + 1)) + && b0 == 0xFF + && b1 & 0xE0 == 0xE0 + { + // A candidate after the cap cannot be scored. Stop rather than let a container-like + // byte string later in an unscored audio payload become preflight-authoritative; the + // probe will select the reader, and `open` refuses an unchecked size-checked one. + if budget.exhausted() { + return Ok(None); + } + if audio_sync_at(b, at, &mut audio_probe, budget)? { + return Ok(Some(FirstMarker::Audio)); + } + } + if rest.get(4..8) == Some(b"ftyp".as_slice()) { + return Ok(Some(FirstMarker::Mp4(at))); + } + at += 1; + } + Ok(None) +} + +/// Run the early size checks on the container the probe can select. `start` is the first byte +/// after leading ID3v2 tags. A zero-frame FLAC is returned as an empty stream because Symphonia +/// 0.6's reader requires a first audio frame to resynchronize to. +fn precheck(b: &[u8], start: usize, detected: Option) -> Result { + let any_marker = |b: &[u8], start: usize| -> Result> { + Ok(match first_marker(b, start)? { + // Stop at the first marker of any kind, so container magic incidental to an audio + // payload is not mistaken for the stream's container. + Some(FirstMarker::Audio) | None => None, + Some(marker) => Some((marker, marker_name(marker))), + }) + }; + let found = match container_markers(detected) { + // Follow the detection: check the first marker of the family it names, so a misleading + // marker cannot let a differently sized container reach the probe unchecked. A detection + // is only a hint, though: when the stream holds none of its markers (a FLAC behind a + // `.mp3` name), the probe can still select another container, which is checked here. + Some(markers) => match find_container_marker(b, start, markers) { + Some((at, marker)) => Some(marker_kind(at, marker)), + None => any_marker(b, start)?, + }, + None => any_marker(b, start)?, + }; + let Some((marker, expected)) = found else { return Ok(Precheck::Unchecked) }; + match marker { + FirstMarker::Flac(at) => { + super::flac::check_metadata(b, at)?; + if let Some(info) = super::flac::empty_stream_info(b, at) { + return Ok(Precheck::Empty(info)); + } + Ok(Precheck::Checked(expected)) + } + FirstMarker::Caf(at) => { + super::caf::check(b, at)?; + Ok(Precheck::Checked(expected)) + } + FirstMarker::Mp4(at) => { + super::mp4::check(b, at)?; + Ok(Precheck::Checked(expected)) + } + FirstMarker::Wav(at) => { + super::wav::check_info(b, at)?; + Ok(Precheck::Checked(expected)) + } + FirstMarker::Audio => Ok(Precheck::Unchecked), + } +} + +/// Skip leading ID3v2 tags, run the container size preflight, then probe with symphonia; a zero-frame FLAC found by +/// the preflight comes back as [`Opened::Empty`] without probing. A size-checked container is refused when probing +/// selects a different reader or one that was never checked, so incidental payload magic cannot bypass those limits. +fn open(bytes: Cow<'_, [u8]>, detected: Option) -> Result { + let start = skip_id3v2(&bytes); + if start > 0 && start >= bytes.len() { + return Err(AudioError::Malformed("ID3v2 tag extends past the end of the file".into())); + } + let checked = precheck(&bytes, start, detected)?; + if let Precheck::Empty(info) = checked { + return Ok(Opened::Empty(info)); + } + // symphonia reads from an owned stream: a borrowed buffer is copied once, a buffer read from + // a shared source is moved in. + let mut mss = MediaSourceStream::new(Box::new(Cursor::new(bytes.into_owned())), MediaSourceStreamOptions::default()); + // Seek (rather than pre-positioning the cursor) so the stream's absolute position is `start` + // too: container offsets such as MP4's `stco` are absolute file offsets. + mss.seek(SeekFrom::Start(u64::try_from(start).unwrap_or(u64::MAX)))?; + let reader: Box = + symphonia::default::get_probe().probe(&Hint::new(), mss, FormatOptions::default(), MetadataOptions::default()).map_err(map_err)?; + let probed = reader.format_info().short_name; + match checked { + // Only containers whose readers allocate from sizes in the file need the check. A stream + // the probe reads with another reader (MP3, Ogg, AIFF, ...) has no such size fields, so a + // marker found in its audio data must not reject it. + Precheck::Checked(expected) if SIZE_CHECKED_PROBE_FORMATS.contains(&probed) && probed != expected => { + return Err(AudioError::Malformed(format!("the probed container ({probed}) is not the one whose size checks ran"))); + } + Precheck::Unchecked if SIZE_CHECKED_PROBE_FORMATS.contains(&probed) => { + return Err(AudioError::Malformed(format!("the probed container ({probed}) was not size-checked"))); + } + _ => {} + } + Ok(Opened::Reader(reader)) +} + +/// The codec symphonia reports, as a [`CodecId`]. +fn codec_id_of(c: AudioCodecId) -> CodecId { + match c { + c if (CODEC_ID_PCM_S32LE..=CODEC_ID_PCM_F64BE_PLANAR).contains(&c) => CodecId::Pcm, + c if c == CODEC_ID_PCM_ALAW => CodecId::ALaw, + c if c == CODEC_ID_PCM_MULAW => CodecId::MuLaw, + c if [CODEC_ID_ADPCM_G722, CODEC_ID_ADPCM_G726, CODEC_ID_ADPCM_G726LE, CODEC_ID_ADPCM_IMA_QT, CODEC_ID_ADPCM_IMA_WAV, CODEC_ID_ADPCM_MS] + .contains(&c) => + { + CodecId::Adpcm + } + c if c == CODEC_ID_FLAC => CodecId::Flac, + c if c == CODEC_ID_ALAC => CodecId::Alac, + c if c == CODEC_ID_AAC => CodecId::Aac, + c if c == CODEC_ID_MP1 => CodecId::Mp1, + c if c == CODEC_ID_MP2 => CodecId::Mp2, + c if c == CODEC_ID_MP3 => CodecId::Mp3, + c if c == CODEC_ID_OPUS => CodecId::Opus, + c if c == CODEC_ID_VORBIS => CodecId::Vorbis, + _ => CodecId::Unknown, + } +} + +/// The container a symphonia reader stands for, refined by the detection (which tells MP4 from +/// QuickTime and Matroska from WebM). +fn container_of(short_name: &str, detected: Option) -> Container { + match short_name { + "wave" => Container::Wav, + "aiff" => Container::Aiff, + "flac" => Container::Flac, + "caf" => Container::Caf, + "ogg" => Container::Ogg, + "isomp4" if detected == Some(Container::QuickTime) => Container::QuickTime, + "isomp4" => Container::Mp4, + "matroska" if detected == Some(Container::WebM) => Container::WebM, + "matroska" => Container::Matroska, + "mp1" | "mp2" | "mp3" => Container::MpegAudio, + "aac" | "adts" => Container::Adts, + _ => detected.unwrap_or(Container::MpegAudio), + } +} + +/// The speakers of symphonia's channel description; its first 18 position bits are the WAV mask +/// bits, and its decoded planes follow them. +fn layout_of(channels: Option<&Channels>) -> ChannelLayout { + match channels { + Some(Channels::Positioned(p)) => { + let bits = p.bits(); + match u32::try_from(bits) { + Ok(mask) if mask != 0 => ChannelLayout::from_mask(mask), + _ => ChannelLayout::unordered(bits.count_ones() as u16), + } + } + Some(c) => ChannelLayout::unordered(crate::layout::count(c.count())), + None => ChannelLayout::unordered(0), + } +} + +/// The stored sample format, using the codec and its declared bit depth. Lossy and unusual +/// codecs decode to `Compressed`. +fn sample_format_of(p: &AudioCodecParameters) -> SampleFormat { + let c = p.codec; + if c == CODEC_ID_PCM_F32LE || c == CODEC_ID_PCM_F32BE { + return SampleFormat::Float32; + } + if c == CODEC_ID_PCM_F64LE || c == CODEC_ID_PCM_F64BE { + return SampleFormat::Float64; + } + let lossy = [ + CODEC_ID_MP1, + CODEC_ID_MP2, + CODEC_ID_MP3, + CODEC_ID_AAC, + CODEC_ID_VORBIS, + CODEC_ID_OPUS, + CODEC_ID_ADPCM_MS, + CODEC_ID_ADPCM_IMA_WAV, + CODEC_ID_ADPCM_IMA_QT, + CODEC_ID_ADPCM_G722, + CODEC_ID_ADPCM_G726, + CODEC_ID_ADPCM_G726LE, + CODEC_ID_PCM_ALAW, + CODEC_ID_PCM_MULAW, + ]; + let lossless = !lossy.contains(&c); + match (lossless, p.bits_per_sample) { + (true, Some(1..=8)) => SampleFormat::Int8, + (true, Some(9..=16)) => SampleFormat::Int16, + (true, Some(17..=24)) => SampleFormat::Int24, + (true, Some(25..=32)) => SampleFormat::Int32, + _ => SampleFormat::Compressed, + } +} + +/// The audio parameters of `track`, if it carries any. +fn audio_params(track: &Track) -> Option<&AudioCodecParameters> { + track.codec_params.as_ref()?.audio() +} + +/// The audio tracks with a decodable codec. +fn audio_tracks(reader: &dyn FormatReader) -> impl Iterator { + reader.tracks().iter().filter(|t| audio_params(t).is_some_and(|p| p.codec != CODEC_ID_NULL_AUDIO)) +} + +/// The `n`-th audio track (the first when `None`), with the stream information the UI shows. +fn info_of(reader: &dyn FormatReader, detected: Option, n: Option) -> Result<(u32, AudioInfo)> { + let container = container_of(reader.format_info().short_name, detected); + let track = audio_tracks(reader).nth(n.unwrap_or(0)).ok_or_else(|| AudioError::Unsupported("no decodable audio track".into()))?; + let p = audio_params(track).ok_or_else(|| AudioError::Unsupported("no decodable audio track".into()))?; + let sample_rate = p.sample_rate.filter(|&r| r > 0).ok_or_else(|| AudioError::Malformed("unknown sample rate".into()))?; + let layout = layout_of(p.channels.as_ref()); + let frames = track.num_frames.unwrap_or(0); + Ok((track.id, AudioInfo::new(container, codec_id_of(p.codec), sample_format_of(p), sample_rate, layout, frames))) +} + +/// A demux error ends the decode; it is only an error when nothing was decoded yet. +fn demux_failure(frames: u64, e: SymError) -> Result<()> { + if frames == 0 { + return Err(map_err(e)); + } + log::warn!("stopping decode after demux error: {e}"); + Ok(()) +} + +/// Note a corrupt packet (the `errors`-th); a stream that never decodes is malformed. +fn corrupt_packet(errors: usize, frames: u64, msg: &str) -> Result<()> { + log::debug!("concealing corrupt packet: {msg}"); + if errors > 1000 && frames == 0 { + return Err(AudioError::Malformed(format!("no decodable packets: {msg}"))); + } + Ok(()) +} + +/// A file decoded by symphonia, block by block. +pub(crate) struct Symph { + /// `None` for a stream with no audio frames (a zero-frame FLAC). + reader: Option>, + decoder: Option>, + track_id: u32, + time_base: Option, + info: AudioInfo, + frames_known: bool, + /// Frames decoded so far (for the "nothing decoded" checks). + decoded: u64, + errors: usize, + /// Frames of the last decoded packet: the length of the silence that replaces a packet that + /// fails to decode when the demuxer gives no duration for it, so the audio after it keeps its + /// timing (0 before the first packet decodes). + last_frames: usize, + /// Frames still to drop after a seek landed before its target. + discard: u64, + ended: bool, + interleaved: Vec, +} + +impl Symph { + /// Open audio track `track` (the first when `None`) of the file in `bytes`. + pub(crate) fn open(bytes: Cow<'_, [u8]>, detected: Option, track: Option) -> Result { + guarded(move || { + let reader = match open(bytes, detected)? { + Opened::Reader(reader) => reader, + Opened::Empty(info) => { + // a stream with no audio still has one track, as WAV and AIFF do + if let Some(n) = track.filter(|&n| n > 0) { + return Err(AudioError::Unsupported(format!("no audio track {n}: the file has one"))); + } + return Ok(Symph { + reader: None, + decoder: None, + track_id: 0, + time_base: None, + info, + frames_known: true, + decoded: 0, + errors: 0, + last_frames: 0, + discard: 0, + ended: true, + interleaved: Vec::new(), + }); + } + }; + let (track_id, info) = info_of(reader.as_ref(), detected, track)?; + let t = reader.tracks().iter().find(|t| t.id == track_id).ok_or_else(|| AudioError::Malformed("track vanished".into()))?; + let params = audio_params(t).cloned().ok_or_else(|| AudioError::Malformed("track vanished".into()))?; + let alac = params.extra_data.as_deref().filter(|_| params.codec == CODEC_ID_ALAC).and_then(|d| be_u32(d, 0)); + if params.max_frames_per_packet.into_iter().chain(alac.map(u64::from)).any(|n| n > MAX_PACKET_FRAMES) { + return Err(AudioError::Malformed("the file declares packets far longer than any codec uses".into())); + } + let decoder = symphonia::default::get_codecs().make_audio_decoder(¶ms, &decoder_options()).map_err(map_err)?; + let frames_known = info.frames > 0 && info.channels > 0; + Ok(Symph { + time_base: t.time_base, + reader: Some(reader), + decoder: Some(decoder), + track_id, + info, + frames_known, + decoded: 0, + errors: 0, + last_frames: 0, + discard: 0, + ended: false, + interleaved: Vec::new(), + }) + }) + } + + pub(crate) fn info(&self) -> &AudioInfo { + &self.info + } + + pub(crate) fn frames_known(&self) -> bool { + self.frames_known + } + + /// The BWF metadata of the WAV around the stream (symphonia does not read `bext`). + pub(crate) fn set_bwf(&mut self, bwf: Option) { + self.info.bwf = bwf; + } + + /// The next block of decoded audio, or `None` at the end. + pub(crate) fn next_block(&mut self) -> Result> { + guarded(|| self.next_block_inner()) + } + + fn next_block_inner(&mut self) -> Result> { + let (Some(reader), Some(decoder)) = (self.reader.as_mut(), self.decoder.as_mut()) else { return Ok(None) }; + loop { + if self.ended { + if self.info.channels == 0 { + return Err(AudioError::Malformed("stream contains no audio".into())); + } + return Ok(None); + } + let packet = match reader.next_packet() { + Ok(Some(packet)) => packet, + Ok(None) | Err(SymError::ResetRequired) => { + self.ended = true; + continue; + } + Err(SymError::IoError(e)) if e.kind() == std::io::ErrorKind::UnexpectedEof => { + self.ended = true; + continue; + } + Err(e) => { + demux_failure(self.decoded, e)?; + self.ended = true; + continue; + } + }; + if packet.track_id != self.track_id { + continue; + } + let block = match decoder.decode(&packet) { + Ok(decoded) => { + // The interleaved copy below is laid out by planes: use their count so a + // decoder whose channel spec disagrees with its planes cannot trip the copy's + // length assertion. + let ch = decoded.num_planes(); + if ch == 0 { + continue; + } + let n = decoded.frames(); + let needed = n + .checked_mul(ch) + .filter(|&t| t as u64 <= MAX_DECODE_SAMPLES) + .ok_or_else(|| AudioError::TooLarge(format!("a decoded block exceeds the {MAX_DECODE_SAMPLES}-sample decode limit")))?; + if self.interleaved.len() < needed { + self.interleaved + .try_reserve(needed - self.interleaved.len()) + .map_err(|_| AudioError::TooLarge("cannot allocate decoded audio".into()))?; + self.interleaved.resize(needed, 0.0); + } + decoded.copy_to_slice_interleaved(&mut self.interleaved[..needed]); + let mut planar = crate::pcm::planar(ch, n)?; + for frame in self.interleaved[..needed].chunks_exact(ch) { + for (dst, &s) in planar.iter_mut().zip(frame) { + dst.push(if s.is_finite() { s } else { 0.0 }); + } + } + // The decoded stream is authoritative for the channel count. + if usize::from(self.info.channels) != ch { + self.info.channels = u16::try_from(ch).map_err(|_| AudioError::Unsupported("too many channels".into()))?; + self.info.layout = ChannelLayout::unordered(crate::layout::count(ch)); + } + self.decoded = self.decoded.saturating_add(n as u64); + self.last_frames = n; + planar + } + Err(SymError::DecodeError(m)) => { + self.errors += 1; + self.info.damaged += 1; + corrupt_packet(self.errors, self.decoded, m)?; + // the packet's own duration when the demuxer gives one (Vorbis packets vary in + // length; it leaves out priming), otherwise the last packet's: a damaged first + // packet is concealed only when its duration is known + let rate = u64::from(self.info.sample_rate.max(1)); + let (numer, denom) = self.time_base.map_or((1, rate), |tb| (u64::from(tb.numer.get()), u64::from(tb.denom.get()))); + let own = u128::from(packet.dur.get()) * u128::from(numer) * u128::from(rate) / u128::from(denom); + let frames = match usize::try_from(own) { + Ok(n) if n > 0 && n as u64 <= MAX_PACKET_FRAMES => n, + _ => self.last_frames, + }; + if frames == 0 || self.info.channels == 0 { + continue; + } + crate::pcm::silence(usize::from(self.info.channels), frames)? + } + Err(SymError::IoError(_)) => { + self.ended = true; + continue; + } + Err(e) => return Err(map_err(e)), + }; + let mut block = AudioBuffer { sample_rate: self.info.sample_rate, channels: block }; + if self.discard > 0 { + let drop = (self.discard as usize).min(block.frames()); + for c in &mut block.channels { + c.drain(..drop); + } + self.discard -= drop as u64; + if block.frames() == 0 { + continue; + } + } + return Ok(Some(block)); + } + } + + /// Continue from frame `frame`: an accurate seek lands at or before it, and the frames up to + /// it are dropped. + pub(crate) fn seek(&mut self, frame: u64) -> Result { + // At or past a known end: nothing is left to decode. + if self.frames_known && frame >= self.info.frames { + self.ended = true; + self.discard = 0; + return Ok(self.info.frames); + } + guarded(|| { + let (Some(reader), Some(decoder)) = (self.reader.as_mut(), self.decoder.as_mut()) else { return Ok(0) }; + let rate = u64::from(self.info.sample_rate.max(1)); + // Timestamps count `numer / denom` seconds; without a time base they count frames. + let (numer, denom) = self.time_base.map_or((1, rate), |tb| (u64::from(tb.numer.get()), u64::from(tb.denom.get()))); + let to_ts = |f: u64| u128::from(f) * u128::from(denom) / (u128::from(numer) * u128::from(rate)); + let ts = i64::try_from(to_ts(frame)).unwrap_or(i64::MAX); + let seeked = match reader.seek(SeekMode::Accurate, SeekTo::Timestamp { ts: Timestamp::new(ts), track_id: self.track_id }) { + Ok(s) => s, + // Past the end of a stream whose length is unknown: the stream is over. + Err(SymError::SeekError(_)) => { + self.ended = true; + return Ok(frame); + } + Err(SymError::IoError(e)) if e.kind() == std::io::ErrorKind::UnexpectedEof => { + self.ended = true; + return Ok(frame); + } + Err(e) => return Err(map_err(e)), + }; + decoder.reset(); + // the packet before the seek says nothing about the length of one lost after it + self.last_frames = 0; + let actual = u64::try_from(seeked.actual_ts.get()).unwrap_or(0); + let actual = u64::try_from(u128::from(actual) * u128::from(numer) * u128::from(rate) / u128::from(denom)).unwrap_or(u64::MAX); + self.discard = frame.saturating_sub(actual); + self.ended = false; + Ok(frame) + }) + } +} + +/// The codec parameters of each audio track symphonia finds, for a track list. +pub(crate) fn tracks(bytes: Cow<'_, [u8]>, detected: Option) -> Result> { + guarded(move || { + let reader = match open(bytes, detected)? { + Opened::Reader(reader) => reader, + Opened::Empty(info) => return Ok(vec![(info.codec, info.sample_rate, info.channels)]), + }; + Ok(audio_tracks(reader.as_ref()) + .filter_map(audio_params) + .map(|p| (codec_id_of(p.codec), p.sample_rate.unwrap_or(0), p.channels.as_ref().map_or(0, |c| u16::try_from(c.count()).unwrap_or(0)))) + .collect()) + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use symphonia::core::codecs::audio::well_known::{ + CODEC_ID_AAC, CODEC_ID_ADPCM_MS, CODEC_ID_ALAC, CODEC_ID_FLAC, CODEC_ID_MP3, CODEC_ID_PCM_F32LE, CODEC_ID_PCM_F64LE, CODEC_ID_PCM_S16LE, + CODEC_ID_VORBIS, + }; + + /// Build codec parameters for a test, optionally declaring a bit depth. + fn params(codec: AudioCodecId, bits_per_sample: Option) -> AudioCodecParameters { + let mut params = AudioCodecParameters::new(); + params.for_codec(codec); + if let Some(bits) = bits_per_sample { + params.with_bits_per_sample(bits); + } + params + } + + /// symphonia's codec ids map to ours, PCM by its id range. + #[test] + fn codecs_map_to_codec_ids() { + assert_eq!(codec_id_of(CODEC_ID_FLAC), CodecId::Flac); + assert_eq!(codec_id_of(CODEC_ID_MP3), CodecId::Mp3); + assert_eq!(codec_id_of(CODEC_ID_ALAC), CodecId::Alac); + assert_eq!(codec_id_of(CODEC_ID_VORBIS), CodecId::Vorbis); + assert_eq!(codec_id_of(CODEC_ID_PCM_S16LE), CodecId::Pcm); + assert_eq!(codec_id_of(CODEC_ID_PCM_F64LE), CodecId::Pcm); + assert_eq!(codec_id_of(CODEC_ID_ADPCM_MS), CodecId::Adpcm); + assert_eq!(codec_id_of(CODEC_ID_PCM_ALAW), CodecId::ALaw); + } + + /// The reader names the container; the detection refines MP4 into QuickTime and Matroska into WebM. + #[test] + fn readers_map_to_containers() { + assert_eq!(container_of("isomp4", None), Container::Mp4); + assert_eq!(container_of("isomp4", Some(Container::QuickTime)), Container::QuickTime); + assert_eq!(container_of("matroska", Some(Container::WebM)), Container::WebM); + assert_eq!(container_of("mp3", None), Container::MpegAudio); + assert_eq!(container_of("wave", Some(Container::Wav)), Container::Wav); + } + + /// PCM and FLAC keep their declared bit depth; lossy codecs report `Compressed` regardless of their decoded + /// width. + #[test] + fn sample_formats_keep_their_depth_and_lossy_marker() { + assert_eq!(sample_format_of(¶ms(CODEC_ID_PCM_S16LE, Some(16))), SampleFormat::Int16); + assert_eq!(sample_format_of(¶ms(CODEC_ID_FLAC, Some(24))), SampleFormat::Int24); + assert_eq!(sample_format_of(¶ms(CODEC_ID_FLAC, Some(8))), SampleFormat::Int8); + assert_eq!(sample_format_of(¶ms(CODEC_ID_FLAC, Some(32))), SampleFormat::Int32); + assert_eq!(sample_format_of(¶ms(CODEC_ID_PCM_F32LE, Some(32))), SampleFormat::Float32); + assert_eq!(sample_format_of(¶ms(CODEC_ID_PCM_F64LE, Some(64))), SampleFormat::Float64); + // Lossy codecs report no fixed bit depth, whatever their decoded sample format is. + assert_eq!(sample_format_of(¶ms(CODEC_ID_AAC, Some(16))), SampleFormat::Compressed); + assert_eq!(sample_format_of(¶ms(CODEC_ID_ADPCM_MS, Some(4))), SampleFormat::Compressed); + assert_eq!(sample_format_of(¶ms(CODEC_ID_ALAC, None)), SampleFormat::Compressed); + } + + /// Decoder options keep gapless trimming enabled so the encoder delay and padding reported by the demuxer are + /// removed. + #[test] + fn gapless_decoding_stays_enabled() { + assert!(decoder_options().gapless); + } + + /// A complete MPEG frame header is a candidate; loose or unregistered sync pairs are rejected before scoring, + /// while a registered but truncated header is rejected by the reader's score after consuming a budget slot. + #[test] + fn only_the_readers_own_audio_markers_are_candidates() { + /// Whether the preflight would score `b` as an audio candidate at offset 0. + fn candidate(b: &[u8]) -> bool { + let mut stream = None; + let mut budget = ScoreBudget::new(); + matches!(audio_sync_at(b, 0, &mut stream, &mut budget), Ok(true)) + } + let mut mp3_frame = vec![0u8; 417]; + mp3_frame[..4].copy_from_slice(&[0xFF, 0xFB, 0x90, 0x00]); // MPEG-1 Layer III, 128 kbps, 44.1 kHz + assert!(candidate(&[&mp3_frame[..], &mp3_frame].concat())); + for false_sync in [vec![0xFF, 0xE0, 0x00], vec![0xFF, 0xF0, 0x00], vec![0xFF, 0xFB, 0xF0, 0x00]] { + assert!(!candidate(&false_sync), "{false_sync:?} is not a probe candidate"); + } + } + + /// A prefix made of false MPEG sync markers must not ask the readers to score every byte + /// position: the per-scan budget caps the expensive work. A cheap container marker reached + /// before the next audio candidate is still found, but once the cap is spent an audio + /// candidate stops the scan so later payload magic cannot be preflight-authoritative. + #[test] + fn a_prefix_crowded_with_sync_markers_scores_only_the_budget() { + let mut crowded = vec![0u8; 1 << 16]; + for pair in crowded.as_chunks_mut::<2>().0 { + pair.copy_from_slice(&[0xFF, 0xE2]); + } + let mut budget = ScoreBudget::new(); + assert!(matches!(first_marker_with_budget(&crowded, 0, &mut budget), Ok(None))); + assert_eq!(budget.used, MAX_AUDIO_SCORE_ATTEMPTS, "no candidate after the cap is scored"); + + // At the cap boundary, with no later audio candidate, the cheap marker is still found. + let mut at_cap: Vec = (0..MAX_AUDIO_SCORE_ATTEMPTS).flat_map(|_| [0xFFu8, 0xE2]).collect(); + at_cap.extend_from_slice(b"fLaC"); + let mut budget = ScoreBudget::new(); + assert!( + matches!(first_marker_with_budget(&at_cap, 0, &mut budget), Ok(Some(FirstMarker::Flac(_)))), + "a container marker reached before the next audio candidate is still found" + ); + assert_eq!(budget.used, MAX_AUDIO_SCORE_ATTEMPTS); + + // One more false candidate between the cap and the magic makes the scan stop unchecked: + // the same magic could instead sit in a valid audio payload. + let mut after_candidate = at_cap[..2 * MAX_AUDIO_SCORE_ATTEMPTS].to_vec(); + after_candidate.extend_from_slice(&[0xFF, 0xE2]); + after_candidate.extend_from_slice(b"fLaC"); + let mut budget = ScoreBudget::new(); + assert!(matches!(first_marker_with_budget(&after_candidate, 0, &mut budget), Ok(None)), "an audio candidate after the cap stops the scan"); + assert_eq!(budget.used, MAX_AUDIO_SCORE_ATTEMPTS); + } + + /// ID3v2 sizes are read in their 7-bit syncsafe form; consecutive tags are skipped by their declared sizes, and + /// the v2.3 reserved footer bit does not extend a tag. + #[test] + fn leading_id3_tags_are_skipped_by_their_declared_size() { + let tag = [&b"ID3\x04\0\0\0\0\0\x05"[..], b"hello"].concat(); + assert_eq!(skip_id3v2(&[&tag[..], &tag[..]].concat()), 30); + assert_eq!(skip_id3v2(b"fLaC"), 0); + // The 10-byte footer flag exists only in v2.4; the same bit in v2.3 is reserved. + assert_eq!(id3v2_len(b"ID3\x03\0\x10\0\0\0\0"), Some(10)); + assert_eq!(id3v2_len(b"ID3\x04\0\x10\0\0\0\0"), Some(20)); + } +} diff --git a/crates/audio/src/read/wav.rs b/crates/audio/src/read/wav.rs new file mode 100644 index 00000000..9208aaaa --- /dev/null +++ b/crates/audio/src/read/wav.rs @@ -0,0 +1,320 @@ +//! Native WAV / BWF / RF64 reader (RIFF chunk layout per the Microsoft multimedia +//! specification, `bext` per EBU Tech 3285, `ds64` per EBU Tech 3306). + +use super::source::Src; +use crate::pcm::{PcmKind, le_u16, le_u32, le_u64, tag}; +use crate::wav::{GUID_TAIL, WAVE_FORMAT_EXTENSIBLE, WAVE_FORMAT_IEEE_FLOAT, WAVE_FORMAT_PCM}; +use crate::{AudioError, AudioInfo, BwfInfo, ChannelLayout, CodecId, Container, Result, SampleFormat}; + +/// Tail of the Ambisonic subtype GUIDs {0000000x-0721-11D3-8644-C8C1CA000000}: PCM (x = 1) and float (x = 3) as well. +const AMBISONIC_TAIL: [u8; 14] = [0x00, 0x00, 0x21, 0x07, 0xD3, 0x11, 0x86, 0x44, 0xC8, 0xC1, 0xCA, 0x00, 0x00, 0x00]; + +/// Parsed WAV layout (no sample data decoded yet). +pub(crate) struct WavLayout { + pub info: AudioInfo, + pub kind: PcmKind, + pub data_start: u64, + pub data_len: u64, +} + +/// Outcome of parsing a WAV header. +pub(crate) enum WavParse { + /// A PCM / float layout we decode natively. + Native(WavLayout), + /// A valid WAV whose codec we hand to symphonia (ADPCM, A-law, ...). + #[cfg_attr(not(feature = "symphonia"), allow(dead_code))] + Foreign { bwf: Option }, +} + +struct Fmt { + tag: u16, + channels: u16, + sample_rate: u32, + block_align: u16, + bits: u16, + valid_bits: u16, + /// `dwChannelMask` of a WAVE_FORMAT_EXTENSIBLE header. + mask: Option, + ambisonic: bool, +} + +impl Fmt { + /// The speaker layout: the extensible mask when there is one (0 or a mask naming another + /// number of speakers leaves the channels unordered), unordered for Ambisonics, else the + /// usual layout for the channel count. + fn layout(&self) -> ChannelLayout { + let channels = usize::from(self.channels); + match self.mask { + _ if self.ambisonic => ChannelLayout::unordered(crate::layout::count(channels)), + Some(mask) => ChannelLayout::from_mask_and_channels(mask, crate::layout::count(channels)), + None => ChannelLayout::default_for(crate::layout::count(channels)), + } + } +} + +fn parse_fmt(c: &[u8]) -> Result { + let short = || AudioError::Malformed("fmt chunk too short".into()); + let mut fmt = Fmt { + tag: le_u16(c, 0).ok_or_else(short)?, + channels: le_u16(c, 2).ok_or_else(short)?, + sample_rate: le_u32(c, 4).ok_or_else(short)?, + block_align: le_u16(c, 12).ok_or_else(short)?, + bits: le_u16(c, 14).unwrap_or(0), + valid_bits: 0, + mask: None, + ambisonic: false, + }; + fmt.valid_bits = fmt.bits; + if fmt.tag == WAVE_FORMAT_EXTENSIBLE { + // cbSize(2) validBits(2) channelMask(4) subFormat GUID(16) + let (Some(cb_size), Some(valid), Some(sub)) = (le_u16(c, 16), le_u16(c, 18), le_u16(c, 24)) else { + return Err(AudioError::Malformed("WAVE_FORMAT_EXTENSIBLE fmt chunk too short".into())); + }; + let Some(tail) = c.get(26..40) else { + return Err(AudioError::Malformed("WAVE_FORMAT_EXTENSIBLE fmt chunk too short".into())); + }; + if cb_size < 22 { + return Err(AudioError::Malformed("WAVE_FORMAT_EXTENSIBLE cbSize is below 22".into())); + } + if valid > fmt.bits { + return Err(AudioError::Malformed("WAVE_FORMAT_EXTENSIBLE valid bits exceed the container size".into())); + } + if valid != 0 { + fmt.valid_bits = valid; + } + // A subtype outside the KSDATAFORMAT_SUBTYPE_* family is not PCM/float, whatever its first two bytes say: + // keep the extensible tag so the file goes to the foreign decoder. + if tail == GUID_TAIL || tail == AMBISONIC_TAIL { + fmt.tag = sub; + } + fmt.mask = le_u32(c, 20); + fmt.ambisonic = tail == AMBISONIC_TAIL; + } + Ok(fmt) +} + +/// Read a NUL-padded ASCII field. +fn fixed_str(b: &[u8]) -> String { + let end = b.iter().position(|&c| c == 0).unwrap_or(b.len()); + String::from_utf8_lossy(b.get(..end).unwrap_or_default()).trim_end().to_string() +} + +fn parse_bext(c: &[u8]) -> Option { + if c.len() < 346 { + return None; + } + let lo = u64::from(le_u32(c, 338)?); + let hi = u64::from(le_u32(c, 342)?); + Some(BwfInfo { + description: fixed_str(c.get(0..256)?), + originator: fixed_str(c.get(256..288)?), + originator_reference: fixed_str(c.get(288..320)?), + origination_date: fixed_str(c.get(320..330)?), + origination_time: fixed_str(c.get(330..338)?), + time_reference: (hi << 32) | lo, + }) +} + +/// Refuse the WAVE file at `at`, which symphonia reads (A-law, ADPCM, ...), if an INFO entry +/// claims more bytes than the file holds: symphonia allocates each entry's length. Chunks are +/// walked where symphonia reads them: after the bytes a chunk's parser used, not its length. +#[cfg(feature = "symphonia")] +pub(crate) fn check_info(b: &[u8], at: usize) -> Result<()> { + if tag(b, at + 8) != Some(*b"WAVE") { + return Ok(()); + } + let mut pos = at + 12; + while let Some(id) = tag(b, pos) { + let Some(len) = le_u32(b, pos + 4).map(|n| n as usize) else { break }; + let (body, pad) = (pos + 8, len & 1); + let used = match &id { + b"data" => break, + b"fmt " => fmt_used(b, body, len), + b"LIST" if len < 4 => break, + b"LIST" => { + let entries = if tag(b, body) == Some(*b"INFO") { info_entries(b, body + 4, len - 4)? } else { 0 }; + // The list pads itself, then the chunk is padded again. + 4 + entries.max(len - 4) + pad + } + _ => len, + }; + pos = body.saturating_add(used).saturating_add(pad); + } + Ok(()) +} + +/// The bytes symphonia reads of a `fmt ` chunk: some formats' parsers go by their own extra size. +#[cfg(feature = "symphonia")] +fn fmt_used(b: &[u8], body: usize, len: usize) -> usize { + match le_u16(b, body) { + Some(2 | 6 | 7 | 0x11) => le_u16(b, body + 16).map_or(len, |extra| 18 + usize::from(extra)), + Some(0xfffe) => 40, + _ => len, + } +} + +/// Check the INFO entries in the `len` bytes from `start`; returns the bytes symphonia reads of +/// them, with the pad it reads after an odd entry. +#[cfg(feature = "symphonia")] +fn info_entries(b: &[u8], start: usize, len: usize) -> Result { + let mut used = 0; + while used + 8 <= len { + let Some(n) = le_u32(b, start + used + 4).map(|n| n as usize) else { break }; + if n > b.len().saturating_sub(start + used + 8) { + return Err(AudioError::Malformed("a WAV INFO entry is larger than the file".into())); + } + used += 8 + n; + used += used & 1; + } + Ok(used) +} + +/// True when the bytes start like a RIFF/RF64/BW64 WAVE file. +pub(crate) fn is_wav(b: &[u8]) -> bool { + matches!(tag(b, 0), Some(t) if &t == b"RIFF" || &t == b"RF64" || &t == b"BW64") && tag(b, 8).is_some_and(|t| &t == b"WAVE") +} + +/// True when everything from `from` to the end of the file is whole RIFF chunks (four printable +/// ASCII id bytes, a size that fits, the pad byte after an odd size; the last chunk's pad may be +/// missing): the metadata of a finished file, not samples. +fn only_chunks_follow(src: &Src<'_>, from: u64) -> bool { + let total = src.size(); + let mut pos = from; + while pos < total { + let (Some(id), Some(size)) = (src.tag(pos), src.le_u32(pos + 4)) else { return false }; + if !id.iter().all(|c| (0x20..0x7f).contains(c)) { + return false; + } + let size = u64::from(size); + let end = pos + 8 + size; + if end > total { + return false; + } + let next = end + (size & 1); + if end == total || next == total { + return true; + } + pos = next; + } + false +} + +/// A `data` chunk header as found in the file, for [`data_chunk_len`]. +struct DataChunk { + body: u64, + size32: u32, + is_rf64: bool, + remaining: u64, + declared: u64, +} + +/// Bytes of audio a `data` chunk holds: leniently, whatever is present when the size is wrong or +/// the recording was never finalised. +fn data_chunk_len(src: &Src<'_>, c: DataChunk) -> u64 { + let finished = u64::from(src.le_u32(4).unwrap_or(0)) + 8 == src.size(); + let empty = c.size32 == 0 && !c.is_rf64 && finished && only_chunks_follow(src, c.body); + if c.size32 == 0 && !c.is_rf64 && !empty { c.remaining } else { c.declared.min(c.remaining) } +} + +/// The sample layout of a `fmt` chunk, `None` for a format we do not read natively. +fn pcm_layout(fmt: &Fmt) -> Result> { + Ok(Some(match (fmt.tag, fmt.bits) { + (WAVE_FORMAT_PCM, bits @ 1..=32) => { + let bytes = usize::from(bits.div_ceil(8)); + let sf = match fmt.valid_bits.min(bits) { + 0..=8 => SampleFormat::Int8, + 9..=16 => SampleFormat::Int16, + 17..=24 => SampleFormat::Int24, + _ => SampleFormat::Int32, + }; + (PcmKind::Int { bytes, big_endian: false, unsigned: bytes == 1 }, sf) + } + (WAVE_FORMAT_IEEE_FLOAT, 32) => (PcmKind::Float { bytes: 4, big_endian: false }, SampleFormat::Float32), + (WAVE_FORMAT_IEEE_FLOAT, 64) => (PcmKind::Float { bytes: 8, big_endian: false }, SampleFormat::Float64), + (WAVE_FORMAT_PCM | WAVE_FORMAT_IEEE_FLOAT, bits) => { + return Err(AudioError::Unsupported(format!("WAV format {} with {bits} bits per sample", fmt.tag))); + } + _ => return Ok(None), + })) +} + +/// Parse a WAV file's chunks: its format, `bext` metadata and where its samples are. Only chunk +/// headers and the small `fmt `, `bext` and `ds64` bodies are read. +pub(crate) fn parse(src: &Src<'_>) -> Result { + if !is_wav(&src.bytes_upto(0, 12)) { + return Err(AudioError::Malformed("not a RIFF/RF64 WAVE file".into())); + } + let total = src.size(); + let is_rf64 = src.tag(0).is_some_and(|t| &t != b"RIFF"); + let mut ds64_data: Option = None; + let mut fmt: Option = None; + let mut bwf: Option = None; + let mut data: Option<(u64, u64)> = None; + + let mut pos: u64 = 12; + // Bound the number of chunks we look at so a file of tiny chunks can't spin forever. + let mut chunks = 0usize; + while let (Some(id), Some(size32)) = (src.tag(pos), src.le_u32(pos + 4)) { + chunks += 1; + if chunks > 100_000 { + break; + } + let body = pos + 8; + let remaining = total.saturating_sub(body); + let mut declared = u64::from(size32); + if &id == b"data" && is_rf64 && size32 == u32::MAX { + declared = ds64_data.unwrap_or(remaining); + } + if &id == b"data" { + // Lenient: a truncated recording (or a streaming writer's 0 / 0xFFFFFFFF size) still + // yields whatever audio is actually present. + // A size of 0 in a file whose RIFF size is right and that has only chunks after it is an + // empty data chunk followed by metadata (LIST, bext, …); otherwise it is a recording + // that was never finalised, and what follows is audio. + let len = data_chunk_len(src, DataChunk { body, size32, is_rf64, remaining, declared }); + if data.is_none() { + data = Some((body, len)); + } + if declared > remaining { + break; + } + } else { + if declared > remaining { + // A truncated non-audio chunk at the end of the file: stop scanning. + break; + } + // The fields read from these chunks lie in their first bytes. + let small = |n: u64| src.bytes(body, usize::try_from(declared.min(n)).unwrap_or(0)).unwrap_or_default(); + match &id { + b"fmt " => fmt = Some(parse_fmt(&small(256))?), + b"bext" => bwf = parse_bext(&small(4096)), + b"ds64" => ds64_data = le_u64(&small(64), 8), + _ => {} + } + } + let Some(next) = body.checked_add(declared).and_then(|p| p.checked_add(declared & 1)) else { break }; + pos = next; + } + + let fmt = fmt.ok_or_else(|| AudioError::Malformed("WAV has no fmt chunk".into()))?; + if fmt.channels == 0 { + return Err(AudioError::Malformed("WAV declares zero channels".into())); + } + if fmt.sample_rate == 0 { + return Err(AudioError::Malformed("WAV declares a zero sample rate".into())); + } + let channels = usize::from(fmt.channels); + let Some((kind, sample_format)) = pcm_layout(&fmt)? else { return Ok(WavParse::Foreign { bwf }) }; + let block = kind.bytes().checked_mul(channels).ok_or_else(|| AudioError::Malformed("block size overflow".into()))?; + if usize::from(fmt.block_align) != block { + log::debug!("WAV block_align {} disagrees with computed {block}; using computed", fmt.block_align); + } + let (data_start, data_len) = data.ok_or_else(|| AudioError::Malformed("WAV has no data chunk".into()))?; + let frames = data_len / block as u64; + Ok(WavParse::Native(WavLayout { + info: AudioInfo { bwf, ..AudioInfo::new(Container::Wav, CodecId::Pcm, sample_format, fmt.sample_rate, fmt.layout(), frames) }, + kind, + data_start, + data_len, + })) +} diff --git a/crates/audio/src/track.rs b/crates/audio/src/track.rs new file mode 100644 index 00000000..34495b00 --- /dev/null +++ b/crates/audio/src/track.rs @@ -0,0 +1,230 @@ +//! [`TrackDecoder`]: one track's packets in, planar `f32` in WAV channel order out, whatever the +//! codec. The file-level [`crate::decode`] and [`crate::Decoder`] use it for every native codec; +//! an app with its own demuxer (MXF, MPEG-TS, a network stream) uses it directly. + +use crate::pcm::{PcmKind, deinterleave}; +use crate::{AudioBuffer, AudioError, ChannelLayout, CodecId, CodecParams, Result, SampleFormat}; + +/// How many leading packets [`TrackDecoder::new`] looks at. +pub const LOOKAHEAD_PACKETS: usize = 8; + +/// The codec-specific decoder behind a [`TrackDecoder`]. +enum Inner { + Pcm { kind: PcmKind, channels: usize }, +} + +/// Decodes the packets of one audio track to planar `f32`, in WAV channel order when +/// [`TrackDecoder::layout`] is ordered. +/// +/// It takes the track's [`CodecParams`] (from `isobmff::Track::codec_params`, +/// `matroska::Track::codec_params`, `ogg::Stream::codec_params`, or built by hand for another +/// container) and decodes PCM. Container timing (priming, end trimming) is the caller's: every +/// decoded sample is returned. +/// +/// ```no_run +/// # fn demo(params: &craft_audio::CodecParams, packets: &[Vec]) -> craft_audio::Result<()> { +/// use craft_audio::TrackDecoder; +/// +/// let lookahead: Vec<&[u8]> = packets.iter().take(8).map(Vec::as_slice).collect(); +/// let mut dec = TrackDecoder::new(params, &lookahead)?; +/// println!("{} at {} Hz, {}", dec.codec().name(), dec.sample_rate(), dec.layout()); +/// for p in packets { +/// let block = match dec.decode(p) { +/// Ok(block) => block, +/// Err(_) => dec.conceal(Some(p))?, // keep the timing across a damaged packet +/// }; +/// // block.channels: one Vec per channel +/// } +/// # Ok(()) } +/// ``` +pub struct TrackDecoder { + inner: Inner, + codec: CodecId, + sample_rate: u32, + layout: ChannelLayout, + sample_format: SampleFormat, + /// Frames of the last decoded packet, the length of a concealed one when the codec does not + /// say. + last_frames: usize, +} + +impl std::fmt::Debug for TrackDecoder { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("TrackDecoder") + .field("codec", &self.codec) + .field("sample_rate", &self.sample_rate) + .field("layout", &self.layout) + .finish_non_exhaustive() + } +} + +impl TrackDecoder { + /// A decoder for the track `params` describes. `lookahead` holds the track's first packets + /// (up to [`LOOKAHEAD_PACKETS`]; empty when they are not at hand), for codecs that find their + /// configuration in the stream; PCM needs none. Missing configuration is + /// [`AudioError::Malformed`]. + /// + /// Codecs without a native packet decoder yet (AAC, MP3, FLAC, ALAC, Vorbis, Opus) are + /// [`AudioError::Unsupported`]; `craft_audio::decode` decodes them through symphonia when the + /// `symphonia` feature is on. + pub fn new(params: &CodecParams, lookahead: &[&[u8]]) -> Result { + let rate = params.sample_rate.unwrap_or(0); + let channels = usize::from(params.channels.unwrap_or(0)); + // PCM needs no lookahead; the codec decoders take their configuration from it. + let _ = lookahead; + let (inner, codec, sample_rate, layout, sample_format) = match params.codec { + CodecId::Pcm => { + let pcm = params.pcm.ok_or_else(|| AudioError::Malformed("PCM track without a sample format".into()))?; + let (kind, sample_format) = pcm_kind(pcm.bits, pcm.float, pcm.big_endian, !pcm.signed) + .ok_or_else(|| AudioError::Unsupported(format!("{}-bit PCM", pcm.bits)))?; + if channels == 0 || rate == 0 { + return Err(AudioError::Malformed("PCM track without channels or a sample rate".into())); + } + (Inner::Pcm { kind, channels }, CodecId::Pcm, rate, ChannelLayout::default_for(crate::layout::count(channels)), sample_format) + } + other => return Err(unsupported(other)), + }; + Ok(TrackDecoder { inner, codec, sample_rate, layout, sample_format, last_frames: 0 }) + } + + /// The codec. + pub fn codec(&self) -> CodecId { + self.codec + } + + /// The rate of the decoded audio. + pub fn sample_rate(&self) -> u32 { + self.sample_rate + } + + /// The speakers of the decoded channels; ordered layouts come in WAV order. + pub fn layout(&self) -> ChannelLayout { + self.layout + } + + /// The number of decoded channels. + pub fn channels(&self) -> u16 { + self.layout.channels() + } + + /// The stored sample format: the PCM depth, or [`SampleFormat::Compressed`]. + pub fn sample_format(&self) -> SampleFormat { + self.sample_format + } + + /// Decode one packet. A packet that fails is an error for every codec: call [`TrackDecoder::conceal`] for audio of its length, so what follows keeps + /// its timing. + pub fn decode(&mut self, packet: &[u8]) -> Result { + let channels = match &mut self.inner { + Inner::Pcm { kind, channels } => { + let frames = (packet.len() / (kind.bytes() * *channels).max(1)) as u64; + deinterleave(packet, *kind, *channels, frames)? + } + }; + if let Some(n) = channels.iter().map(Vec::len).min().filter(|&n| n > 0) { + self.last_frames = n; + if channels.len() != usize::from(self.layout.channels()) { + self.layout = ChannelLayout::unordered(crate::layout::count(channels.len())); + } + } + Ok(AudioBuffer { sample_rate: self.sample_rate, channels }) + } + + /// What stands in for a lost or undecodable packet (`packet` when its bytes are at hand): + /// silence as long as the packet would have been (from its size, or the last packet). Empty + /// when the length is unknown. Fails with [`AudioError::TooLarge`](crate::AudioError::TooLarge) + /// when that silence would pass the decode limit (2^30 samples, as [`decode`](Self::decode) + /// has) or cannot be allocated, rather than return less than the packet's length. + pub fn conceal(&mut self, packet: Option<&[u8]>) -> Result { + let data = packet.unwrap_or_default(); + let frames = match &mut self.inner { + Inner::Pcm { kind, channels } => Some(data.len() / (kind.bytes() * *channels).max(1)), + }; + let frames = frames.filter(|&n| n > 0).unwrap_or(self.last_frames); + Ok(AudioBuffer { sample_rate: self.sample_rate, channels: crate::pcm::silence(usize::from(self.layout.channels()), frames)? }) + } + + /// The frames every packet decodes to, for codecs whose packets all have one length; `None` + /// for PCM. + pub fn frames_per_packet(&self) -> Option { + match &self.inner { + Inner::Pcm { .. } => None, + } + } + + /// Frames to decode before a seek target and drop, so the decoder has settled when the target + /// is reached; none for PCM. + pub fn preroll(&self) -> usize { + match &self.inner { + Inner::Pcm { .. } => 0, + } + } + + /// Forget the decoder state, for a seek: the next packet decodes as if it were the first + /// (codecs with overlap need [`TrackDecoder::preroll`] frames to settle). + pub fn reset(&mut self) { + // the packet before a seek says nothing about the length of a packet lost after it + self.last_frames = 0; + match &mut self.inner { + Inner::Pcm { .. } => {} + } + } +} + +fn unsupported(codec: CodecId) -> AudioError { + if cfg!(feature = "symphonia") { + AudioError::Unsupported(format!("no native {} packet decoder yet (craft_audio::decode reads it through symphonia)", codec.name())) + } else { + AudioError::Unsupported(format!("no native {} packet decoder yet", codec.name())) + } +} + +/// The sample layout and stored format of PCM samples, `None` for a depth we do not read. +pub(crate) fn pcm_kind(bits: u16, float: bool, big_endian: bool, unsigned: bool) -> Option<(PcmKind, SampleFormat)> { + let bytes = usize::from(bits.div_ceil(8)); + match (float, bits) { + (true, 32) => Some((PcmKind::Float { bytes: 4, big_endian }, SampleFormat::Float32)), + (true, 64) => Some((PcmKind::Float { bytes: 8, big_endian }, SampleFormat::Float64)), + (false, 1..=32) => { + let sf = match bits { + 1..=8 => SampleFormat::Int8, + 9..=16 => SampleFormat::Int16, + 17..=24 => SampleFormat::Int24, + _ => SampleFormat::Int32, + }; + Some((PcmKind::Int { bytes, big_endian, unsigned }, sf)) + } + _ => None, + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::PcmFormat; + + #[test] + fn pcm_tracks_decode_in_wav_order() { + let params = CodecParams::new(CodecId::Pcm).with_sample_rate(48_000).with_channels(2).with_pcm(PcmFormat::int(16, true, false)); + let mut dec = TrackDecoder::new(¶ms, &[]).unwrap(); + assert_eq!((dec.codec(), dec.sample_rate(), dec.layout()), (CodecId::Pcm, 48_000, ChannelLayout::STEREO)); + let block = dec.decode(&[0x00, 0x40, 0x00, 0xC0]).unwrap(); + assert_eq!(block.channels, vec![vec![0.5], vec![-0.5]]); + assert_eq!(dec.conceal(None).unwrap().frames(), 1); + // a reset forgets the last packet's length + dec.reset(); + assert_eq!(dec.conceal(None).unwrap().frames(), 0); + // silence past the decode limit is an error, not shorter silence + dec.last_frames = usize::try_from(crate::pcm::MAX_DECODE_SAMPLES / 2 + 1).unwrap(); + assert!(matches!(dec.conceal(None), Err(AudioError::TooLarge(_)))); + } + + #[test] + fn codecs_without_a_packet_decoder_are_unsupported() { + for codec in [CodecId::Aac, CodecId::Flac, CodecId::Vorbis, CodecId::Alac, CodecId::Mp3, CodecId::Opus, CodecId::H264] { + assert!(matches!(TrackDecoder::new(&CodecParams::new(codec), &[]), Err(AudioError::Unsupported(_))), "{codec:?}"); + } + // PCM needs its sample format. + assert!(TrackDecoder::new(&CodecParams::new(CodecId::Pcm).with_channels(2).with_sample_rate(48_000), &[]).is_err()); + } +} diff --git a/crates/audio/src/wav.rs b/crates/audio/src/wav.rs new file mode 100644 index 00000000..61e32593 --- /dev/null +++ b/crates/audio/src/wav.rs @@ -0,0 +1,7 @@ +//! WAVE format constants shared by the reader and the writer (Microsoft multimedia specification). + +pub(crate) const WAVE_FORMAT_PCM: u16 = 1; +pub(crate) const WAVE_FORMAT_IEEE_FLOAT: u16 = 3; +pub(crate) const WAVE_FORMAT_EXTENSIBLE: u16 = 0xFFFE; +/// Tail of the KSDATAFORMAT_SUBTYPE_* GUIDs (bytes 2..16); bytes 0..2 hold the format tag. +pub(crate) const GUID_TAIL: [u8; 14] = [0x00, 0x00, 0x00, 0x00, 0x10, 0x00, 0x80, 0x00, 0x00, 0xAA, 0x00, 0x38, 0x9B, 0x71]; diff --git a/crates/audio/src/write/aiff.rs b/crates/audio/src/write/aiff.rs new file mode 100644 index 00000000..5e3da511 --- /dev/null +++ b/crates/audio/src/write/aiff.rs @@ -0,0 +1,154 @@ +//! Native AIFF / AIFF-C writer (Apple "Audio Interchange File Format" 1.3 and the AIFF-C +//! draft: big-endian IFF chunks, 80-bit IEEE extended sample rate). + +use super::pcm::{Quantizer, sample_format, validate_buffer, validate_layout, write_interleaved}; +use crate::{AudioBuffer, AudioError, BitDepth, Result}; +use std::io::{Cursor, Seek, SeekFrom, Write}; + +/// AIFF-C version 1 timestamp for the FVER chunk. +const AIFC_VERSION_1: u32 = 0xA280_5140; + +/// Encode a positive integer as an 80-bit IEEE 754 extended float (exact). +pub(crate) fn u32_to_extended(r: u32) -> [u8; 10] { + let mut out = [0u8; 10]; + if r == 0 { + return out; + } + let e = 31 - r.leading_zeros(); + let exp = (16383 + e) as u16; + let mant = u64::from(r) << (63 - e); + out[..2].copy_from_slice(&exp.to_be_bytes()); + out[2..].copy_from_slice(&mant.to_be_bytes()); + out +} + +fn push_chunk(out: &mut Vec, id: &[u8; 4], body: &[u8]) { + out.extend_from_slice(id); + out.extend_from_slice(&u32::try_from(body.len()).unwrap_or(u32::MAX).to_be_bytes()); + out.extend_from_slice(body); + if body.len() % 2 == 1 { + out.push(0); + } +} + +/// Encode AIFF (integer depths) or AIFF-C `fl32` (Float32). +pub(crate) fn encode(buf: &AudioBuffer, depth: BitDepth, dither: bool) -> Result> { + validate_buffer(buf)?; + let frames = buf.frames(); + let head_len = header(buf.channels.len(), buf.sample_rate, depth, frames as u64)?.len(); + let data_len = usize::from(sample_format(depth).0 / 8).saturating_mul(buf.channels.len()).saturating_mul(frames); + let total = head_len.saturating_add(data_len).saturating_add(1); + let mut out = Vec::new(); + out.try_reserve_exact(total).map_err(|_| AudioError::TooLarge(format!("cannot allocate {total} bytes for AIFF output")))?; + let mut w = Writer::new(Cursor::new(out), buf.channels.len(), buf.sample_rate, depth, dither)?; + w.write(&buf.channels, frames)?; + Ok(w.finish()?.into_inner()) +} + +/// An AIFF written block by block: a placeholder header first, then the real one (the same +/// length) written over it by [`Writer::finish`]. +pub(crate) struct Writer { + out: W, + start: u64, + channels: usize, + sample_rate: u32, + depth: BitDepth, + quantizers: Vec, + frames: u64, + bytes: Vec, + /// Set when a block write has failed: the sink may hold a prefix of that block while + /// `frames` does not count it, so the writer refuses every later call. + failed: bool, +} + +impl Writer { + pub(crate) fn new(mut out: W, channels: usize, sample_rate: u32, depth: BitDepth, dither: bool) -> Result { + validate_layout(channels, sample_rate)?; + let start = out.stream_position()?; + out.write_all(&header(channels, sample_rate, depth, 0)?)?; + let quantizers = Quantizer::per_channel(u32::from(sample_format(depth).0), dither, channels); + Ok(Writer { out, start, channels, sample_rate, depth, quantizers, frames: 0, bytes: Vec::new(), failed: false }) + } + + /// The frame count after `frames` more, or an error when AIFF can't hold that many (4 GiB). + pub(crate) fn check_room(&self, frames: u64) -> Result { + let total = self.frames.saturating_add(frames); + header(self.channels, self.sample_rate, self.depth, total)?; + Ok(total) + } + + /// Appends `frames` frames of planar audio (missing samples are silence), unless they don't + /// fit (see [`Writer::check_room`]). + pub(crate) fn write>(&mut self, block: &[C], frames: usize) -> Result<()> { + if self.failed { + return Err(super::failed_write_error()); + } + let total = self.check_room(frames as u64)?; + if let Err(e) = write_interleaved(&mut self.out, &mut self.bytes, block, frames, &mut self.quantizers, sample_format(self.depth), true) { + self.failed = true; + return Err(AudioError::from(e)); + } + self.frames = total; + Ok(()) + } + + pub(crate) fn finish(mut self) -> Result { + if self.failed { + return Err(super::failed_write_error()); + } + let data_len = u64::from(sample_format(self.depth).0 / 8).saturating_mul(self.channels as u64).saturating_mul(self.frames); + if data_len % 2 == 1 { + self.out.write_all(&[0])?; + } + let head = header(self.channels, self.sample_rate, self.depth, self.frames)?; + self.out.seek(SeekFrom::Start(self.start))?; + self.out.write_all(&head)?; + self.out.seek(SeekFrom::End(0))?; + self.out.flush()?; + Ok(self.out) + } +} + +/// Everything before the sample data of an AIFF holding `frames` frames; its length doesn't +/// depend on `frames`. +fn header(channels: usize, sample_rate: u32, depth: BitDepth, frames: u64) -> Result> { + let channels_i16 = i16::try_from(channels).map_err(|_| AudioError::Encode("too many channels for AIFF".into()))?; + let frames_u32 = u32::try_from(frames).map_err(|_| AudioError::TooLarge("too many frames for AIFF".into()))?; + let (bits, is_float) = sample_format(depth); + let data_len = u64::from(bits / 8).saturating_mul(channels as u64).saturating_mul(frames); + + let mut comm = Vec::with_capacity(40); + comm.extend_from_slice(&channels_i16.to_be_bytes()); + comm.extend_from_slice(&frames_u32.to_be_bytes()); + comm.extend_from_slice(&bits.to_be_bytes()); + comm.extend_from_slice(&u32_to_extended(sample_rate)); + if is_float { + comm.extend_from_slice(b"fl32"); + let name = b"32-bit float"; + comm.push(name.len() as u8); + comm.extend_from_slice(name); + if (name.len() + 1) % 2 == 1 { + comm.push(0); + } + } + + let mut head = Vec::with_capacity(128); + head.extend_from_slice(b"FORM"); + head.extend_from_slice(&[0, 0, 0, 0]); // patched below + head.extend_from_slice(if is_float { b"AIFC" } else { b"AIFF" }); + if is_float { + push_chunk(&mut head, b"FVER", &AIFC_VERSION_1.to_be_bytes()); + } + push_chunk(&mut head, b"COMM", &comm); + head.extend_from_slice(b"SSND"); + let ssnd_len = u32::try_from(data_len.saturating_add(8)).map_err(|_| AudioError::TooLarge("AIFF data exceeds 4 GiB".into()))?; + head.extend_from_slice(&ssnd_len.to_be_bytes()); + head.extend_from_slice(&[0u8; 8]); // offset, blockSize + + let total = (head.len() as u64).saturating_add(data_len).saturating_add(data_len & 1); + let form_len = u32::try_from(total - 8).map_err(|_| AudioError::TooLarge("AIFF file exceeds 4 GiB".into()))?; + if let Some(slot) = head.get_mut(4..8) { + slot.copy_from_slice(&form_len.to_be_bytes()); + } + Ok(head) +} diff --git a/crates/audio/src/write/mod.rs b/crates/audio/src/write/mod.rs new file mode 100644 index 00000000..8e845aa1 --- /dev/null +++ b/crates/audio/src/write/mod.rs @@ -0,0 +1,183 @@ +//! Encoding: WAV/BWF/RF64 and AIFF/AIFF-C writers (`pcm-encode`), whole-buffer and streaming. + +mod aiff; +mod pcm; +mod wav; + +use crate::{AudioBuffer, AudioError, ChannelLayout, EncodeFormat, EncodeOptions, Result}; +use std::io::{Seek, Write}; + +/// Encode `buf` as WAV/BWF or AIFF/AIFF-C. Integer depths clamp samples to [-1, 1]; float +/// keeps values above full scale. `opts.layout`, when set, must have `buf`'s channel count. +/// +/// ``` +/// use craft_audio::{AudioBuffer, BitDepth, EncodeFormat, EncodeOptions, encode}; +/// +/// let buf = AudioBuffer::new(48_000, 2, 48_000); // one second of stereo silence +/// let opts = EncodeOptions::new(EncodeFormat::Wav).with_bit_depth(BitDepth::Int24).with_dither(true); +/// let wav: Vec = encode(&buf, &opts).unwrap(); +/// assert_eq!(&wav[..4], b"RIFF"); +/// ``` +pub fn encode(buf: &AudioBuffer, opts: &EncodeOptions) -> Result> { + // Checked for every format; only a WAV records it. + let mask = opts.layout.map(|l| wav_mask(&l, buf.channels.len())).transpose()?; + match opts.format { + EncodeFormat::Wav => wav::encode(buf, opts.bit_depth, opts.dither, opts.bwf.as_ref(), opts.force_rf64, mask), + EncodeFormat::Aiff => aiff::encode(buf, opts.bit_depth, opts.dither), + } +} + +/// The file extensions of the formats [`encode`] can write. +pub const ENCODE_EXTENSIONS: &[&str] = &["wav", "wave", "bwf", "aif", "aiff", "aifc"]; + +/// The format [`encode`] writes for a file extension or format name (`wav`, `aiff`, case-insensitive, +/// with or without the dot), or `None` when it cannot write it. `aifc` is AIFF: float AIFF is +/// written as AIFF-C. +/// +/// ``` +/// use craft_audio::{EncodeFormat, encode_format_for}; +/// +/// assert_eq!(encode_format_for(".AIFF"), Some(EncodeFormat::Aiff)); +/// assert_eq!(encode_format_for("mp3"), None); +/// ``` +pub fn encode_format_for(name: &str) -> Option { + match name.trim_start_matches('.').to_ascii_lowercase().as_str() { + "wav" | "wave" | "bwf" => Some(EncodeFormat::Wav), + "aif" | "aiff" | "aifc" => Some(EncodeFormat::Aiff), + _ => None, + } +} + +/// The WAV speaker mask for `layout` over `channels` channels: its mask, or 0 (no speaker +/// assignment) for an unordered layout. A layout for another channel count is an error. +fn wav_mask(layout: &ChannelLayout, channels: usize) -> Result { + if usize::from(layout.channels()) != channels { + return Err(AudioError::Encode(format!("a {}-channel layout for {channels} channels", layout.channels()))); + } + Ok(layout.mask().unwrap_or(0)) +} + +/// The error a stream encoder returns for every call after one of its writes has failed: the +/// sink may already hold a prefix of the failed block, so the encoder's counters can no longer +/// describe the bytes on the sink, and later writes or `finish` must not claim success. +fn failed_write_error() -> AudioError { + AudioError::Encode("the stream encoder already failed on an earlier write; its output is incomplete".into()) +} + +/// Writes WAV/BWF (RF64 above 4 GiB) or AIFF / AIFF-C `fl32` block by block, producing +/// the same bytes [`encode`] would for the whole buffer: a file of any length never +/// has to fit in memory. The file starts at the sink's position, and nothing may follow that +/// position (a new file, or one truncated there): the encoder cannot truncate a `Write + Seek`, +/// so the bytes of a longer file already there would stay after the new one. +/// +/// ``` +/// use craft_audio::{EncodeFormat, EncodeOptions, FileEncoder}; +/// use std::io::Cursor; +/// +/// let opts = EncodeOptions::new(EncodeFormat::Aiff); +/// let mut enc = FileEncoder::new(Cursor::new(Vec::new()), &opts, 48_000, 2).unwrap(); +/// let block = [vec![0.25f32; 4096], vec![-0.25f32; 4096]]; // one Vec (or slice) per channel +/// for _ in 0..10 { +/// enc.write(&block, 4096).unwrap(); +/// } +/// let aiff = enc.finish().unwrap().into_inner(); +/// assert_eq!(&aiff[..4], b"FORM"); +/// ``` +pub struct FileEncoder { + writer: Writer, + channels: usize, +} + +impl std::fmt::Debug for FileEncoder { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + let format = match &self.writer { + Writer::Wav(_) => EncodeFormat::Wav, + Writer::Aiff(_) => EncodeFormat::Aiff, + }; + f.debug_struct("FileEncoder").field("format", &format).finish_non_exhaustive() + } +} + +enum Writer { + Wav(wav::Writer), + Aiff(aiff::Writer), +} + +impl FileEncoder { + /// Starts a file of `channels` channels at `sample_rate` at the current position of `out`. + /// `opts.layout`, when set, must have `channels` channels. + pub fn new(out: W, opts: &EncodeOptions, sample_rate: u32, channels: u16) -> Result { + let (depth, dither, channels) = (opts.bit_depth, opts.dither, usize::from(channels)); + let mask = opts.layout.map(|l| wav_mask(&l, channels)).transpose()?; + let writer = match opts.format { + EncodeFormat::Wav => Writer::Wav(wav::Writer::new( + out, + channels, + sample_rate, + depth, + dither, + wav::HeaderExtras { bwf: opts.bwf.as_ref(), mask, force_rf64: opts.force_rf64 }, + )?), + EncodeFormat::Aiff => Writer::Aiff(aiff::Writer::new(out, channels, sample_rate, depth, dither)?), + }; + Ok(FileEncoder { writer, channels }) + } + + /// Fails, without writing anything, when `frames` more frames would not fit the format (an + /// AIFF holds 4 GiB). + pub fn check_room(&self, frames: u64) -> Result<()> { + match &self.writer { + Writer::Wav(w) => w.check_room(frames), + Writer::Aiff(w) => w.check_room(frames), + } + .map(drop) + } + + /// Appends the first `frames` frames of `block`, one slice per channel (`&[Vec]`, + /// `&[&[f32]]`, …), unless they don't fit (see [`FileEncoder::check_room`]). `block` must + /// have one slice per channel, each at least `frames` long; otherwise nothing is written and + /// the error is [`AudioError::Encode`]. Once a write has failed for any other reason, the + /// encoder is unusable: the sink may hold a prefix of the failed block, so every later `write` + /// and `finish` fails instead of completing an inconsistent file. + pub fn write>(&mut self, block: &[C], frames: usize) -> Result<()> { + if block.len() != self.channels { + return Err(AudioError::Encode(format!("a block of {} channels for a {}-channel file", block.len(), self.channels))); + } + if let Some(short) = block.iter().map(|c| c.as_ref().len()).filter(|&n| n < frames).min() { + return Err(AudioError::Encode(format!("{frames} frames asked for, but a channel holds {short}"))); + } + match &mut self.writer { + Writer::Wav(w) => w.write(block, frames), + Writer::Aiff(w) => w.write(block, frames), + } + } + + /// Completes the headers and hands back `out`, positioned at the end of the file. + pub fn finish(self) -> Result { + match self.writer { + Writer::Wav(w) => w.finish(), + Writer::Aiff(w) => w.finish(), + } + } +} + +/// The header of a 32-bit float WAV holding `frames` frames (little-endian samples follow it). Its +/// length doesn't depend on `frames`, so a recorder can rewrite it in place as the file grows; it +/// switches to RF64 above 4 GiB. +/// +/// ``` +/// use craft_audio::wav_float_header; +/// +/// // A recorder writes the header, appends interleaved f32 samples, then rewrites the header. +/// let mut file = wav_float_header(48_000, 2, 0).unwrap(); +/// let samples = [0.25f32, -0.25, 0.5, -0.5]; +/// file.extend(samples.iter().flat_map(|s| s.to_le_bytes())); +/// let header = wav_float_header(48_000, 2, 2).unwrap(); +/// file[..header.len()].copy_from_slice(&header); +/// ``` +pub fn wav_float_header(sample_rate: u32, channels: u16, frames: u64) -> Result> { + if channels == 0 || sample_rate == 0 { + return Err(AudioError::Encode("a WAV needs channels and a sample rate".into())); + } + wav::header(usize::from(channels), sample_rate, (32, true), None, None, frames, false) +} diff --git a/crates/audio/src/write/pcm.rs b/crates/audio/src/write/pcm.rs new file mode 100644 index 00000000..834cd687 --- /dev/null +++ b/crates/audio/src/write/pcm.rs @@ -0,0 +1,124 @@ +//! PCM helpers shared by the WAV and AIFF writers: quantization (with optional TPDF +//! dither), interleaving and layout checks. + +use crate::{AudioBuffer, AudioError, Result}; +#[cfg(feature = "pcm-encode")] +use {crate::BitDepth, std::io::Write}; + +/// Converts `f32` samples to integers of a given bit depth, clamping to [-1, 1] and optionally +/// adding triangular (TPDF) dither of ±1 LSB. +pub(crate) struct Quantizer { + bits: u32, + dither: bool, + rng: u64, +} + +impl Quantizer { + /// A quantizer for one channel; each channel gets an independent dither sequence. + pub(crate) fn new(bits: u32, dither: bool, channel: usize) -> Self { + let rng = 0x9E37_79B9_7F4A_7C15u64 ^ (channel as u64).wrapping_add(1).wrapping_mul(0xD1B5_4A32_D192_ED03); + Self { bits: bits.clamp(2, 32), dither, rng: if rng == 0 { 1 } else { rng } } + } + + /// One quantizer per channel. + pub(crate) fn per_channel(bits: u32, dither: bool, channels: usize) -> Vec { + (0..channels).map(|c| Self::new(bits, dither, c)).collect() + } + + /// Uniform in [0, 1) from a xorshift64* generator (deterministic, no OS entropy needed). + fn uniform(&mut self) -> f64 { + let mut x = self.rng; + x ^= x >> 12; + x ^= x << 25; + x ^= x >> 27; + self.rng = x; + (x.wrapping_mul(0x2545_F491_4F6C_DD1D) >> 11) as f64 / (1u64 << 53) as f64 + } + + pub(crate) fn quantize(&mut self, x: f32) -> i32 { + let x = if x.is_nan() { 0.0 } else { x.clamp(-1.0, 1.0) }; + let scale = (1u64 << (self.bits - 1)) as f64; + let mut v = f64::from(x) * scale; + if self.dither { + v += self.uniform() - self.uniform(); + } + let max = scale - 1.0; + v.round().clamp(-scale, max) as i32 + } +} + +/// Bits per sample at `depth`, and whether they are float. +#[cfg(feature = "pcm-encode")] +pub(crate) fn sample_format(depth: BitDepth) -> (u16, bool) { + match depth { + BitDepth::Int16 => (16, false), + BitDepth::Int24 => (24, false), + BitDepth::Int32 => (32, false), + BitDepth::Float32 => (32, true), + } +} + +/// Interleaves `frames` frames of planar audio (missing samples are silence) as `bits`-bit +/// samples and writes them to `out`, a few thousand frames at a time through `scratch`. +#[cfg(feature = "pcm-encode")] +pub(crate) fn write_interleaved>( + out: &mut impl Write, + scratch: &mut Vec, + block: &[C], + frames: usize, + quantizers: &mut [Quantizer], + (bits, is_float): (u16, bool), + big_endian: bool, +) -> std::io::Result<()> { + let size = usize::from(bits / 8); + let mut start = 0; + while start < frames { + let end = frames.min(start + 4096); + scratch.clear(); + for f in start..end { + for (c, q) in quantizers.iter_mut().enumerate() { + let s = block.get(c).and_then(|ch| ch.as_ref().get(f)).copied().unwrap_or(0.0); + match (is_float, big_endian) { + (true, false) => scratch.extend_from_slice(&sanitize_float(s).to_le_bytes()), + (true, true) => scratch.extend_from_slice(&sanitize_float(s).to_be_bytes()), + (false, false) => scratch.extend_from_slice(q.quantize(s).to_le_bytes().get(..size).unwrap_or_default()), + (false, true) => scratch.extend_from_slice(q.quantize(s).to_be_bytes().get(4 - size..).unwrap_or_default()), + } + } + } + out.write_all(scratch)?; + start = end; + } + Ok(()) +} + +/// Preserve finite float samples, mapping NaN to 0 and infinities to their sign (±1). +#[cfg(feature = "pcm-encode")] +fn sanitize_float(x: f32) -> f32 { + if x.is_finite() { + x + } else if x.is_nan() { + 0.0 + } else { + x.signum() + } +} + +/// Check that a buffer is encodable: at least one channel and a non-zero sample rate. +pub(crate) fn validate_buffer(buf: &AudioBuffer) -> Result<()> { + validate_layout(buf.channels.len(), buf.sample_rate) +} + +/// Check that a stream is encodable: at least one channel and a non-zero sample rate. +pub(crate) fn validate_layout(channels: usize, sample_rate: u32) -> Result<()> { + if channels == 0 { + return Err(AudioError::Encode("buffer has no channels".into())); + } + if channels > usize::from(u16::MAX) { + return Err(AudioError::Encode("too many channels".into())); + } + if sample_rate == 0 { + return Err(AudioError::Encode("sample rate is 0".into())); + } + Ok(()) +} diff --git a/crates/audio/src/write/wav.rs b/crates/audio/src/write/wav.rs new file mode 100644 index 00000000..96ba5325 --- /dev/null +++ b/crates/audio/src/write/wav.rs @@ -0,0 +1,241 @@ +//! Native WAV / BWF / RF64 writer (RIFF chunk layout per the Microsoft multimedia +//! specification, `bext` per EBU Tech 3285, `ds64` per EBU Tech 3306). + +use super::pcm::{Quantizer, sample_format, validate_buffer, validate_layout, write_interleaved}; +use crate::wav::{GUID_TAIL, WAVE_FORMAT_EXTENSIBLE, WAVE_FORMAT_IEEE_FLOAT, WAVE_FORMAT_PCM}; +use crate::{AudioBuffer, AudioError, BitDepth, BwfInfo, Result}; +use std::io::{Cursor, Seek, SeekFrom, Write}; + +const BEXT_MIN_LEN: usize = 602; + +/// Default speaker mask for a channel count (callers that know the layout pass their own): the +/// usual layout's mask, or the first `n` positions for counts with no usual layout, up to 18. +fn channel_mask(channels: usize) -> u32 { + match crate::ChannelLayout::default_for(crate::layout::count(channels)).mask() { + Some(mask) => mask, + None if channels <= 18 => (1u32 << channels) - 1, + None => 0, + } +} + +/// `s` in a field of `len` bytes, cut at a character boundary so a reader finds valid UTF-8. +fn put_fixed_str(out: &mut Vec, s: &str, len: usize) { + let bytes = s.as_bytes(); + let n = (0..=bytes.len().min(len)).rev().find(|&n| s.is_char_boundary(n)).unwrap_or(0); + out.extend_from_slice(bytes.get(..n).unwrap_or_default()); + out.resize(out.len() + (len - n), 0); +} + +fn bext_chunk(bwf: &BwfInfo) -> Vec { + let mut c = Vec::with_capacity(BEXT_MIN_LEN); + put_fixed_str(&mut c, &bwf.description, 256); + put_fixed_str(&mut c, &bwf.originator, 32); + put_fixed_str(&mut c, &bwf.originator_reference, 32); + put_fixed_str(&mut c, &bwf.origination_date, 10); + put_fixed_str(&mut c, &bwf.origination_time, 8); + c.extend_from_slice(&((bwf.time_reference & 0xFFFF_FFFF) as u32).to_le_bytes()); + c.extend_from_slice(&((bwf.time_reference >> 32) as u32).to_le_bytes()); + c.extend_from_slice(&1u16.to_le_bytes()); // version 1 + c.resize(BEXT_MIN_LEN, 0); // UMID, loudness, reserved; no coding history + c +} + +fn push_chunk(out: &mut Vec, id: &[u8; 4], body: &[u8]) { + out.extend_from_slice(id); + out.extend_from_slice(&u32::try_from(body.len()).unwrap_or(u32::MAX).to_le_bytes()); + out.extend_from_slice(body); + if body.len() % 2 == 1 { + out.push(0); + } +} + +/// A WAV written block by block: a placeholder header first, then the real one (the same length, +/// RF64 above 4 GiB, or always when `force_rf64`) written over it by [`Writer::finish`]. +pub(crate) struct Writer { + out: W, + start: u64, + channels: usize, + sample_rate: u32, + format: (u16, bool), + bwf: Option, + mask: Option, + force_rf64: bool, + quantizers: Vec, + frames: u64, + bytes: Vec, + /// Set when a block write has failed: the sink may hold a prefix of that block while + /// `frames` does not count it, so the writer refuses every later call. + failed: bool, +} + +/// The optional header parts of a WAV file: BWF metadata, speaker mask, and whether to force RF64. +#[derive(Clone, Copy)] +pub(crate) struct HeaderExtras<'a> { + pub(crate) bwf: Option<&'a BwfInfo>, + pub(crate) mask: Option, + pub(crate) force_rf64: bool, +} + +impl Writer { + pub(crate) fn new(mut out: W, channels: usize, sample_rate: u32, depth: BitDepth, dither: bool, extras: HeaderExtras<'_>) -> Result { + let HeaderExtras { bwf, mask, force_rf64 } = extras; + validate_layout(channels, sample_rate)?; + let format = sample_format(depth); + let start = out.stream_position()?; + out.write_all(&header(channels, sample_rate, format, bwf, mask, 0, force_rf64)?)?; + let quantizers = Quantizer::per_channel(u32::from(format.0), dither, channels); + Ok(Writer { + out, + start, + channels, + sample_rate, + format, + bwf: bwf.cloned(), + mask, + force_rf64, + quantizers, + frames: 0, + bytes: Vec::new(), + failed: false, + }) + } + + fn header(&self, frames: u64) -> Result> { + header(self.channels, self.sample_rate, self.format, self.bwf.as_ref(), self.mask, frames, self.force_rf64) + } + + /// The frame count after `frames` more (RF64 has room for any bounce). + pub(crate) fn check_room(&self, frames: u64) -> Result { + let total = self.frames.saturating_add(frames); + self.header(total)?; + Ok(total) + } + + /// Appends `frames` frames of planar audio (missing samples are silence). + pub(crate) fn write>(&mut self, block: &[C], frames: usize) -> Result<()> { + if self.failed { + return Err(super::failed_write_error()); + } + let total = self.check_room(frames as u64)?; + if let Err(e) = write_interleaved(&mut self.out, &mut self.bytes, block, frames, &mut self.quantizers, self.format, false) { + self.failed = true; + return Err(AudioError::from(e)); + } + self.frames = total; + Ok(()) + } + + pub(crate) fn finish(mut self) -> Result { + if self.failed { + return Err(super::failed_write_error()); + } + let data_len = u64::from(self.format.0 / 8).saturating_mul(self.channels as u64).saturating_mul(self.frames); + if data_len % 2 == 1 { + self.out.write_all(&[0])?; + } + let head = self.header(self.frames)?; + self.out.seek(SeekFrom::Start(self.start))?; + self.out.write_all(&head)?; + self.out.seek(SeekFrom::End(0))?; + self.out.flush()?; + Ok(self.out) + } +} + +/// Encode a WAV file. `force_rf64` writes the RF64 layout even for small files (for tests); +/// it is used automatically when the file would exceed 4 GiB. +/// `mask` overrides the speaker mask derived from the channel count. +pub(crate) fn encode( + buf: &AudioBuffer, + depth: BitDepth, + dither: bool, + bwf: Option<&BwfInfo>, + force_rf64: bool, + mask: Option, +) -> Result> { + validate_buffer(buf)?; + let channels = buf.channels.len(); + let frames = buf.frames(); + let format = sample_format(depth); + let head = header(channels, buf.sample_rate, format, bwf, mask, frames as u64, force_rf64)?; + let data_len = (usize::from(format.0 / 8) * channels).checked_mul(frames).ok_or_else(|| AudioError::TooLarge("data size overflow".into()))?; + let mut out = Vec::new(); + out.try_reserve_exact(head.len().saturating_add(data_len).saturating_add(1)) + .map_err(|_| AudioError::TooLarge(format!("cannot allocate {data_len} bytes for WAV output")))?; + let mut w = Writer::new(Cursor::new(out), channels, buf.sample_rate, depth, dither, HeaderExtras { bwf, mask, force_rf64 })?; + w.write(&buf.channels, frames)?; + Ok(w.finish()?.into_inner()) +} + +/// Everything before the sample data of a WAV holding `frames` frames. +pub(crate) fn header( + channels: usize, + sample_rate: u32, + (bits, is_float): (u16, bool), + bwf: Option<&BwfInfo>, + mask: Option, + frames: u64, + force_rf64: bool, +) -> Result> { + let block = usize::from(bits / 8).checked_mul(channels).ok_or_else(|| AudioError::TooLarge("block size overflow".into()))?; + let block_align = u16::try_from(block).map_err(|_| AudioError::Encode("too many channels for WAV".into()))?; + let data_len = u64::from(block_align).checked_mul(frames).ok_or_else(|| AudioError::TooLarge("data size overflow".into()))?; + let extensible = channels > 2 || bits > 16 || mask.is_some_and(|m| m != channel_mask(channels)); + let fmt_tag = if is_float { WAVE_FORMAT_IEEE_FLOAT } else { WAVE_FORMAT_PCM }; + + let mut fmt = Vec::with_capacity(40); + fmt.extend_from_slice(&(if extensible { WAVE_FORMAT_EXTENSIBLE } else { fmt_tag }).to_le_bytes()); + fmt.extend_from_slice(&(channels as u16).to_le_bytes()); + fmt.extend_from_slice(&sample_rate.to_le_bytes()); + let byte_rate = sample_rate + .checked_mul(u32::from(block_align)) + .ok_or_else(|| AudioError::Encode("sample rate × block size overflows the WAV byte rate".into()))?; + fmt.extend_from_slice(&byte_rate.to_le_bytes()); + fmt.extend_from_slice(&block_align.to_le_bytes()); + fmt.extend_from_slice(&bits.to_le_bytes()); + if extensible { + fmt.extend_from_slice(&22u16.to_le_bytes()); + fmt.extend_from_slice(&bits.to_le_bytes()); + fmt.extend_from_slice(&mask.unwrap_or_else(|| channel_mask(channels)).to_le_bytes()); + fmt.extend_from_slice(&fmt_tag.to_le_bytes()); + fmt.extend_from_slice(&GUID_TAIL); + } + + let mut head = Vec::with_capacity(1024); + let rf64_body_len = 28usize; + // Reserve room for ds64 (as JUNK when not needed, so a later writer could upgrade in place). + let mut meta = Vec::new(); + if let Some(bwf) = bwf { + push_chunk(&mut meta, b"bext", &bext_chunk(bwf)); + } + push_chunk(&mut meta, b"fmt ", &fmt); + if is_float { + push_chunk(&mut meta, b"fact", &u32::try_from(frames).unwrap_or(u32::MAX).to_le_bytes()); + } + let total = ((4 + (8 + rf64_body_len) + meta.len() + 8) as u64) + .checked_add(data_len) + .and_then(|t| t.checked_add(data_len & 1)) + .ok_or_else(|| AudioError::TooLarge("WAV size overflow".into()))?; + let rf64 = force_rf64 || u32::try_from(total).is_err(); + + if rf64 { + head.extend_from_slice(b"RF64"); + head.extend_from_slice(&u32::MAX.to_le_bytes()); + head.extend_from_slice(b"WAVE"); + let mut ds64 = Vec::with_capacity(rf64_body_len); + ds64.extend_from_slice(&total.to_le_bytes()); + ds64.extend_from_slice(&data_len.to_le_bytes()); + ds64.extend_from_slice(&frames.to_le_bytes()); + ds64.extend_from_slice(&0u32.to_le_bytes()); + push_chunk(&mut head, b"ds64", &ds64); + } else { + head.extend_from_slice(b"RIFF"); + head.extend_from_slice(&(total as u32).to_le_bytes()); + head.extend_from_slice(b"WAVE"); + push_chunk(&mut head, b"JUNK", &[0u8; 28]); + } + head.extend_from_slice(&meta); + head.extend_from_slice(b"data"); + head.extend_from_slice(&(if rf64 { u32::MAX } else { data_len as u32 }).to_le_bytes()); + Ok(head) +} diff --git a/crates/audio/tests/audio/encode_format.rs b/crates/audio/tests/audio/encode_format.rs new file mode 100644 index 00000000..67c5da07 --- /dev/null +++ b/crates/audio/tests/audio/encode_format.rs @@ -0,0 +1,18 @@ +use craft_audio::{EncodeFormat, encode_format_for}; + +#[test] +fn only_formats_the_encoder_writes_are_named() { + assert_eq!(encode_format_for("wav"), Some(EncodeFormat::Wav)); + assert_eq!(encode_format_for("WAV"), Some(EncodeFormat::Wav)); + assert_eq!(encode_format_for(".aif"), Some(EncodeFormat::Aiff)); + assert_eq!(encode_format_for("aiff"), Some(EncodeFormat::Aiff)); + assert_eq!(encode_format_for("wave"), Some(EncodeFormat::Wav)); + assert_eq!(encode_format_for("BWF"), Some(EncodeFormat::Wav)); + assert_eq!(encode_format_for(".aifc"), Some(EncodeFormat::Aiff)); + for name in craft_audio::ENCODE_EXTENSIONS { + assert!(encode_format_for(name).is_some(), "{name}"); + } + for other in ["flac", "mp3", "ogg", "m4a", "xyz", ""] { + assert_eq!(encode_format_for(other), None, "{other}"); + } +} diff --git a/crates/audio/tests/audio/float_headroom.rs b/crates/audio/tests/audio/float_headroom.rs new file mode 100644 index 00000000..a16098b2 --- /dev/null +++ b/crates/audio/tests/audio/float_headroom.rs @@ -0,0 +1,101 @@ +use craft_audio::{AudioBuffer, BitDepth, EncodeFormat, EncodeOptions, FileEncoder, SampleFormat, decode, encode}; +use std::io::Cursor; + +fn encoded(audio: &AudioBuffer, options: &EncodeOptions, streaming: bool) -> Vec { + if !streaming { + return encode(audio, options).unwrap(); + } + let mut writer = FileEncoder::new(Cursor::new(Vec::new()), options, audio.sample_rate, audio.num_channels()).unwrap(); + // Cross the writer's 4096-frame scratch boundary, then write a short final block. + for start in (0..audio.frames()).step_by(4097) { + let end = (start + 4097).min(audio.frames()); + let block: Vec> = audio.channels.iter().map(|channel| channel[start..end].to_vec()).collect(); + writer.write(&block, end - start).unwrap(); + } + writer.finish().unwrap().into_inner() +} + +fn assert_float_headroom(streaming: bool) { + let samples = [ + -f32::MAX, + -2.0, + -1.25, + -1.0, + -0.5, + -f32::MIN_POSITIVE, + -f32::from_bits(1), + -0.0, + 0.0, + f32::from_bits(1), + f32::MIN_POSITIVE, + 0.5, + 1.0, + 1.25, + 2.0, + f32::MAX, + ]; + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + for dither in [false, true] { + for channels in [1, 2] { + let audio = AudioBuffer { + sample_rate: 48_000, + channels: (0..channels).map(|c| samples.iter().copied().cycle().skip(c).take(4103).collect()).collect(), + }; + let options = EncodeOptions::new(format).with_bit_depth(BitDepth::Float32).with_dither(dither); + let bytes = encoded(&audio, &options, streaming); + let pcm: Vec = + audio.interleaved().iter().flat_map(|v| if format == EncodeFormat::Wav { v.to_le_bytes() } else { v.to_be_bytes() }).collect(); + assert!(bytes.ends_with(&pcm), "{format:?}, dither={dither}: stored sample bytes changed"); + let (info, decoded) = decode(&bytes, None).unwrap(); + assert_eq!(info.sample_format, SampleFormat::Float32); + assert_eq!((decoded.sample_rate, decoded.num_channels(), decoded.frames()), (48_000, channels as u16, 4103)); + for (channel, (actual, expected)) in decoded.channels.iter().zip(&audio.channels).enumerate() { + for (frame, (actual, expected)) in actual.iter().zip(expected).enumerate() { + assert_eq!(actual.to_bits(), expected.to_bits(), "{format:?}, dither={dither}, channel={channel}, frame={frame}"); + } + } + } + } + } +} + +#[test] +fn whole_file_float32_preserves_finite_headroom() { + assert_float_headroom(false); +} + +#[test] +fn streaming_float32_preserves_finite_headroom() { + assert_float_headroom(true); +} + +#[test] +fn float32_keeps_non_finite_sample_handling() { + let audio = AudioBuffer { sample_rate: 48_000, channels: vec![vec![f32::NAN, f32::NEG_INFINITY, f32::INFINITY]] }; + let expected = [0.0_f32, -1.0, 1.0]; + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + for streaming in [false, true] { + let options = EncodeOptions::new(format).with_bit_depth(BitDepth::Float32).with_dither(false); + let bytes = encoded(&audio, &options, streaming); + // Inspect stored samples too: the decoder itself replaces non-finite values. + let pcm: Vec = expected.iter().flat_map(|v| if format == EncodeFormat::Wav { v.to_le_bytes() } else { v.to_be_bytes() }).collect(); + assert!(bytes.ends_with(&pcm), "{format:?}, streaming={streaming}"); + assert_eq!(decode(&bytes, None).unwrap().1.channels[0], expected); + } + } +} + +#[test] +fn integer_pcm_still_clips_at_full_scale() { + let audio = AudioBuffer { sample_rate: 48_000, channels: vec![vec![-2.0, -1.25, -1.0, -0.5, 0.0, 0.5, 1.0, 1.25, 2.0]] }; + for (depth, max) in [(BitDepth::Int16, 32767.0 / 32768.0), (BitDepth::Int24, 8_388_607.0 / 8_388_608.0), (BitDepth::Int32, 1.0)] { + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + for streaming in [false, true] { + let options = EncodeOptions::new(format).with_bit_depth(depth).with_dither(false); + let bytes = encoded(&audio, &options, streaming); + let (_, decoded) = decode(&bytes, None).unwrap(); + assert_eq!(decoded.channels[0], [-1.0, -1.0, -1.0, -0.5, 0.0, 0.5, max, max, max], "{format:?} {depth:?}, streaming={streaming}"); + } + } + } +} diff --git a/crates/audio/tests/audio/main.rs b/crates/audio/tests/audio/main.rs new file mode 100644 index 00000000..84b90aa1 --- /dev/null +++ b/crates/audio/tests/audio/main.rs @@ -0,0 +1,8 @@ +//! Integration tests, built as one test binary: each file is a module. + +mod encode_format; +mod float_headroom; +mod malformed; +mod roundtrip; +mod stream; +mod wav_empty_data; diff --git a/crates/audio/tests/audio/malformed.rs b/crates/audio/tests/audio/malformed.rs new file mode 100644 index 00000000..2486c232 --- /dev/null +++ b/crates/audio/tests/audio/malformed.rs @@ -0,0 +1,699 @@ +//! Hostile inputs must produce `Err` (or a best-effort `Ok`), never a panic. + +use craft_audio::{ + AudioBuffer, AudioError, BitDepth, CodecId, Container, EncodeFormat, EncodeOptions, SampleFormat, decode, detect_container, encode, probe, +}; +use proptest::prelude::*; + +fn small(format: EncodeFormat, depth: BitDepth, channels: usize) -> Vec { + let mut b = AudioBuffer::new(44_100, channels as u16, 300); + for ch in &mut b.channels { + for (i, s) in ch.iter_mut().enumerate() { + *s = ((i as f32) * 0.1).sin() * 0.5; + } + } + encode(&b, &EncodeOptions::new(format).with_bit_depth(depth).with_dither(false)).unwrap() +} + +/// An ID3v2.4 header whose syncsafe size field declares `size` bytes of tag body. +fn id3_header(size: usize) -> [u8; 10] { + let size = size as u32; + [b'I', b'D', b'3', 4, 0, 0, ((size >> 21) & 0x7F) as u8, ((size >> 14) & 0x7F) as u8, ((size >> 7) & 0x7F) as u8, (size & 0x7F) as u8] +} + +/// Wrap `body` in an ID3v2.4 tag. +fn id3_tag(body: &[u8]) -> Vec { + [&id3_header(body.len())[..], body].concat() +} + +/// Little helper for building RIFF files by hand. +fn riff(chunks: &[(&[u8; 4], Vec)]) -> Vec { + let mut body = b"WAVE".to_vec(); + for (id, data) in chunks { + body.extend_from_slice(*id); + body.extend_from_slice(&(data.len() as u32).to_le_bytes()); + body.extend_from_slice(data); + if data.len() % 2 == 1 { + body.push(0); + } + } + let mut out = b"RIFF".to_vec(); + out.extend_from_slice(&(body.len() as u32).to_le_bytes()); + out.extend_from_slice(&body); + out +} + +fn fmt(tag: u16, channels: u16, sr: u32, bits: u16) -> Vec { + let align = channels * bits.div_ceil(8); + let mut f = Vec::new(); + f.extend_from_slice(&tag.to_le_bytes()); + f.extend_from_slice(&channels.to_le_bytes()); + f.extend_from_slice(&sr.to_le_bytes()); + f.extend_from_slice(&(sr * u32::from(align)).to_le_bytes()); + f.extend_from_slice(&align.to_le_bytes()); + f.extend_from_slice(&bits.to_le_bytes()); + f +} + +#[test] +fn every_truncation_is_handled() { + let files = [ + small(EncodeFormat::Wav, BitDepth::Int16, 2), + small(EncodeFormat::Wav, BitDepth::Float32, 6), + small(EncodeFormat::Aiff, BitDepth::Int24, 1), + small(EncodeFormat::Aiff, BitDepth::Float32, 2), + ]; + for file in &files { + for n in 0..file.len() { + let cut = &file[..n]; + let _ = probe(cut, None); + if let Ok((info, buf)) = decode(cut, None) { + assert_eq!(buf.num_channels(), info.channels); + assert_eq!(buf.frames() as u64, info.frames); + } + } + } +} + +#[test] +fn truncated_data_chunk_yields_partial_audio() { + let file = small(EncodeFormat::Wav, BitDepth::Int16, 2); + let (info, buf) = decode(&file[..file.len() - 400], None).unwrap(); + assert_eq!(info.frames, 200); + assert_eq!(buf.frames(), 200); +} + +#[test] +fn every_single_byte_corruption_is_handled() { + for file in [small(EncodeFormat::Wav, BitDepth::Int24, 2), small(EncodeFormat::Aiff, BitDepth::Int16, 2)] { + for i in 0..file.len().min(200) { + for v in [0x00, 0xFF, 0x7F, 0x80] { + let mut f = file.clone(); + f[i] = v; + let _ = probe(&f, None); + let _ = decode(&f, None); + } + } + } +} + +#[test] +fn zero_channels_is_malformed() { + let wav = riff(&[(b"fmt ", fmt(1, 0, 44_100, 16)), (b"data", vec![0; 16])]); + assert!(matches!(decode(&wav, None), Err(AudioError::Malformed(_)))); + assert!(probe(&wav, None).is_err()); + let mut aiff = small(EncodeFormat::Aiff, BitDepth::Int16, 1); + let comm = aiff.windows(4).position(|w| w == b"COMM").unwrap(); + aiff[comm + 8] = 0; + aiff[comm + 9] = 0; + assert!(matches!(decode(&aiff, None), Err(AudioError::Malformed(_)))); +} + +#[test] +fn zero_sample_rate_is_malformed() { + let wav = riff(&[(b"fmt ", fmt(1, 2, 0, 16)), (b"data", vec![0; 16])]); + assert!(decode(&wav, None).is_err()); + let mut aiff = small(EncodeFormat::Aiff, BitDepth::Int16, 1); + let comm = aiff.windows(4).position(|w| w == b"COMM").unwrap(); + for b in &mut aiff[comm + 16..comm + 26] { + *b = 0; + } + assert!(decode(&aiff, None).is_err()); + // Infinite exponent. + aiff[comm + 16] = 0x7F; + aiff[comm + 17] = 0xFF; + assert!(decode(&aiff, None).is_err()); +} + +#[test] +fn absurd_chunk_sizes() { + // fmt chunk claims 4 GiB. + let mut wav = riff(&[(b"fmt ", fmt(1, 2, 44_100, 16)), (b"data", vec![0; 16])]); + wav[16..20].copy_from_slice(&u32::MAX.to_le_bytes()); + assert!(decode(&wav, None).is_err()); + // data chunk claims 4 GiB: lenient read of what's there. + let mut wav = riff(&[(b"fmt ", fmt(1, 2, 44_100, 16)), (b"data", vec![0; 16])]); + let d = wav.windows(4).position(|w| w == b"data").unwrap(); + wav[d + 4..d + 8].copy_from_slice(&u32::MAX.to_le_bytes()); + let (info, buf) = decode(&wav, None).unwrap(); + assert_eq!((info.frames, buf.frames()), (4, 4)); + // An unknown chunk with an absurd size before fmt. + let mut wav = riff(&[(b"junk", vec![0; 4]), (b"fmt ", fmt(1, 2, 44_100, 16)), (b"data", vec![0; 16])]); + wav[16..20].copy_from_slice(&0xFFFF_FFF0u32.to_le_bytes()); + assert!(decode(&wav, None).is_err()); + // RF64 with a ds64 claiming an exabyte of data. + let mut ds64 = Vec::new(); + ds64.extend_from_slice(&u64::MAX.to_le_bytes()); + ds64.extend_from_slice(&u64::MAX.to_le_bytes()); + ds64.extend_from_slice(&u64::MAX.to_le_bytes()); + ds64.extend_from_slice(&0u32.to_le_bytes()); + let mut rf = riff(&[(b"ds64", ds64), (b"fmt ", fmt(1, 1, 8_000, 8)), (b"data", vec![0x80; 10])]); + rf[..4].copy_from_slice(b"RF64"); + let d = rf.windows(4).rposition(|w| w == b"data").unwrap(); + rf[d + 4..d + 8].copy_from_slice(&u32::MAX.to_le_bytes()); + let (info, buf) = decode(&rf, None).unwrap(); + assert_eq!((info.frames, buf.frames()), (10, 10)); + // AIFF COMM claims more frames than present; SSND offset beyond the end. + let mut aiff = small(EncodeFormat::Aiff, BitDepth::Int16, 1); + let comm = aiff.windows(4).position(|w| w == b"COMM").unwrap(); + aiff[comm + 10..comm + 14].copy_from_slice(&u32::MAX.to_be_bytes()); + assert_eq!(decode(&aiff, None).unwrap().1.frames(), 300); + let ssnd = aiff.windows(4).position(|w| w == b"SSND").unwrap(); + aiff[ssnd + 8..ssnd + 12].copy_from_slice(&u32::MAX.to_be_bytes()); + assert_eq!(decode(&aiff, None).unwrap().1.frames(), 0); +} + +fn caf(chunks: &[(&[u8; 4], i64, Vec)]) -> Vec { + let mut out = b"caff\0\x01\0\0".to_vec(); + for (kind, size, body) in chunks { + out.extend_from_slice(*kind); + out.extend_from_slice(&size.to_be_bytes()); + out.extend_from_slice(body); + } + out +} + +fn caf_desc(format: &[u8; 4], bytes_per_packet: u32, frames_per_packet: u32, channels: u32, bits: u32) -> (&'static [u8; 4], i64, Vec) { + let mut d = 48_000f64.to_be_bytes().to_vec(); + d.extend_from_slice(format); + for v in [0, bytes_per_packet, frames_per_packet, channels, bits] { + d.extend_from_slice(&v.to_be_bytes()); + } + (b"desc", 32, d) +} + +fn packet_table(packets: i64, frames: i64, entries: &[u8]) -> Vec { + [&packets.to_be_bytes()[..], &frames.to_be_bytes(), &[0; 8], entries].concat() +} + +#[test] +fn caf_sizes_beyond_the_file_are_errors_not_aborts() { + let aac = || caf_desc(b"aac ", 0, 1024, 2, 0); + let huge = 1i64 << 50; + let cookie = caf(&[aac(), (b"kuki", huge, vec![0; 8])]); + let files = [ + cookie.clone(), + [vec![0; 100], cookie].concat(), + caf(&[aac(), (b"data", -1, [&[0; 4][..], b"kuki", &huge.to_be_bytes()].concat())]), + caf(&[aac(), (b"pakt", 24, packet_table(huge, 1024, &[])), (b"data", -1, vec![0; 16])]), + caf(&[aac(), (b"pakt", 32, packet_table(1, 1024, &[0x82, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0])), (b"data", -1, vec![0; 64])]), + caf(&[aac(), (b"pakt", 25, packet_table(1, 1024, &[0x80]))]), + ]; + for f in files { + assert!(matches!(probe(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + } + let constant = caf(&[caf_desc(b"lpcm", 2, 1, 1, 16), (b"pakt", 25, packet_table(1, 1, &[1])), (b"data", -1, vec![0; 16])]); + assert!(matches!(decode(&constant, None), Err(AudioError::Malformed(m)) if m.contains("lists packets"))); +} + +#[test] +fn well_formed_caf_still_reads() { + let pcm: Vec = (0..480i16).flat_map(|i| (i * 50).to_be_bytes()).collect(); + let chan = [((100u32 << 16) | 1).to_be_bytes(), [0; 4], [0; 4]].concat(); + let data = [vec![0; 4], pcm].concat(); + let lpcm = caf(&[caf_desc(b"lpcm", 2, 1, 1, 16), (b"chan", 12, chan), (b"free", 8, vec![0; 8]), (b"data", data.len() as i64, data)]); + let (info, buf) = decode(&lpcm, None).unwrap(); + assert_eq!((info.container, buf.frames()), (Container::Caf, 480)); + let table = packet_table(3, 3072, &[10, 0x81, 0x48, 5]); + let aac = caf(&[caf_desc(b"aac ", 0, 1024, 2, 0), (b"pakt", table.len() as i64, table), (b"data", 219, vec![0; 219])]); + assert_eq!(probe(&aac, None).unwrap().frames, 3072); +} + +fn atom(kind: &[u8; 4], body: &[u8]) -> Vec { + [&(8 + body.len() as u32).to_be_bytes()[..], kind, body].concat() +} + +/// A mono 16-bit, 48 kHz audio sample entry of `version`, its version fields and codec atoms. +fn sample_entry(kind: &[u8; 4], version: u8, fields: &[u8], codec: &[u8]) -> Vec { + let base = [0, 0, 0, 0, 0, 0, 0, 1, 0, version, 0, 0, 0, 0, 0, 0, 0, 1, 0, 16, 0, 0, 0, 0]; + atom(kind, &[&base[..], &(48_000u32 << 16).to_be_bytes(), fields, codec].concat()) +} + +/// An MP4 of `frames` 2-byte samples described by `entry`; `swap` may replace any version-0 atom. +fn mp4(frames: u32, entry: &[u8], swap: impl Fn(&[u8; 4], Vec) -> Vec) -> Vec { + let full = |kind: &[u8; 4], fields: &[u32]| { + let body: Vec = [0].iter().chain(fields).flat_map(|v: &u32| v.to_be_bytes()).collect(); + swap(kind, atom(kind, &body)) + }; + let moov = |offset: u32| { + let stsd = atom(b"stsd", &[&[0, 0, 0, 0, 0, 0, 0, 1][..], entry].concat()); + let stbl = + [stsd, full(b"stts", &[1, frames, 1]), full(b"stsc", &[1, 1, frames, 1]), full(b"stsz", &[2, frames]), full(b"stco", &[1, offset])]; + let minf = [full(b"smhd", &[0]), atom(b"stbl", &stbl.concat())].concat(); + let mdia = [full(b"mdhd", &[0, 0, 48_000, frames, 0]), full(b"hdlr", &[0, u32::from_be_bytes(*b"soun"), 0, 0, 0, 0]), atom(b"minf", &minf)]; + let trak = [full(b"tkhd", &[0; 20]), atom(b"mdia", &mdia.concat())].concat(); + atom(b"moov", &[full(b"mvhd", &[[0, 0, 48_000, frames, 1 << 16].as_slice(), &[0; 18], &[2]].concat()), atom(b"trak", &trak)].concat()) + }; + let ftyp = atom(b"ftyp", b"M4A \0\0\0\0M4A isom"); + let offset = (ftyp.len() + moov(0).len() + 8) as u32; + let pcm: Vec = (0..frames as i16).flat_map(|i| (i * 40).to_le_bytes()).collect(); + [ftyp, moov(offset), atom(b"mdat", &pcm)].concat() +} + +#[test] +fn mp4_sizes_beyond_the_file_are_errors_not_aborts() { + let pcm = sample_entry(b"sowt", 0, &[], &[]); + let (_, buf) = decode(&mp4(480, &pcm, |_, a| a), None).unwrap(); + assert_eq!(buf.frames(), 480); + let huge = |kind: &[u8; 4], a: Vec| [&1u32.to_be_bytes()[..], kind, &(1u64 << 60).to_be_bytes(), &a[8..]].concat(); + // The first field after version and flags: stts's entry count, stsz's sample size. + let first_field_max = |mut a: Vec| { + a[12..16].copy_from_slice(&u32::MAX.to_be_bytes()); + a + }; + let sizes: Vec = (0..480u32).flat_map(|i| if i == 479 { u32::MAX } else { 2 }.to_be_bytes()).collect(); + let per_sample = atom(b"stsz", &[&[0; 8][..], &480u32.to_be_bytes(), &sizes].concat()); + // A header that starts inside its container is read whole, past the container's end. + let mdia = atom(b"mdia", &1u32.to_be_bytes()); + let straddling = [&b"hdlr"[..], &(1u64 << 60).to_be_bytes(), &[0; 24]].concat(); + let files = [ + mp4(480, &pcm, |k, a| if k == b"hdlr" { huge(k, a) } else { a }), + mp4(480, &pcm, |k, a| if k == b"stts" { first_field_max(a) } else { a }), + mp4(480, &pcm, |k, a| if k == b"stsz" { first_field_max(a) } else { a }), + mp4(480, &pcm, |k, a| if k == b"stsz" { per_sample.clone() } else { a }), + [atom(b"ftyp", b"M4A \0\0\0\0M4A isom"), atom(b"moov", &atom(b"trak", &mdia)), straddling].concat(), + ]; + for f in files { + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + } +} + +#[test] +fn atoms_running_past_their_parent_are_walked_once() { + // Each moov holds a trak that runs to the end of the file; symphonia stops after the first. + let unit = atom(b"moov", &[&u32::MAX.to_be_bytes()[..], b"trak"].concat()); + let f = [atom(b"ftyp", b"M4A \0\0\0\0M4A isom"), unit.repeat(1 << 18)].concat(); + assert!(decode(&f, None).is_err()); +} + +#[test] +fn packets_longer_than_any_codec_uses_are_refused() { + let be = |v: &[u32]| v.iter().flat_map(|x| x.to_be_bytes()).collect::>(); + let v2 = |frames: u32| [&be(&[72])[..], &48_000f64.to_be_bytes(), &be(&[1, 0x7f00_0000, 16, 4, 2, frames])].concat(); + let lpcm = |frames: u32| sample_entry(b"lpcm", 2, &v2(frames), &[]); + let config = |frames: u32| [&be(&[0, frames])[..], &[0, 16, 40, 10, 14, 1, 0, 255], &be(&[0, 0, 48_000])].concat(); + let alac = |frames: u32| sample_entry(b"alac", 0, &[], &atom(b"alac", &config(frames))); + let (_, buf) = decode(&mp4(480, &lpcm(1), |_, a| a), None).unwrap(); + assert_eq!(buf.frames(), 480); + // PCM is read by craft-media's own demuxer, which reads version-2 sound description fields only + // in QuickTime files (ISO BMFF has no such version): the bogus frame count of this MP4 is never + // read, and nothing is allocated from it. symphonia read it, so SoundCraft refused the file. + let (_, buf) = decode(&mp4(480, &lpcm(1 << 21), |_, a| a), None).unwrap(); + assert_eq!(buf.frames(), 480); + // ALAC still goes to symphonia, behind the same guard. + assert!(matches!(decode(&mp4(480, &alac(1 << 21), |_, a| a), None), Err(AudioError::Malformed(m)) if m.contains("declares packets"))); +} + +/// Container magic inside an MP3 frame payload must not change detection or make probing reject the stream. +#[test] +fn a_valid_mp3_with_incidental_container_magic_still_probes() { + let mut frame = vec![0u8; 417]; + frame[..4].copy_from_slice(&[0xFF, 0xFB, 0x90, 0x00]); // MPEG-1 Layer III, 128 kbps, 44.1 kHz, stereo + let mut f: Vec = (0..8).flat_map(|_| frame.clone()).collect(); + let at = 417 * 2 + 10; // Inside frame 2's payload, leaving every frame header intact. + f[at..at + 8].copy_from_slice(b"caff\0\x01\0\0"); + f[at + 8..at + 12].copy_from_slice(b"kuki"); + f[at + 12..at + 20].copy_from_slice(&(1u64 << 50).to_be_bytes()); + assert_eq!(detect_container(&f, None), Some(Container::MpegAudio)); + let info = probe(&f, None).expect("valid MP3 with incidental container magic"); + assert_eq!((info.codec, info.sample_rate, info.channels), (CodecId::Mp3, 44_100, 2)); +} + +/// Symphonia 0.6's CAF/FLAC/MP4 `Scoreable` implementations are unconditional +/// (`Supported(255)`), so a container magic before the first audio frame *is* the reader the probe +/// selects; there is no API to ask the probe to skip it. The preflight must therefore keep +/// refusing an oversized CAF cookie instead of ignoring the marker and playing the MPEG frames +/// behind it (which would also let a real malformed container bypass the size guard). +#[test] +fn a_container_marker_before_the_first_audio_frame_is_the_probed_candidate() { + let mut frame = vec![0u8; 417]; + frame[..4].copy_from_slice(&[0xFF, 0xFB, 0x90, 0x00]); // MPEG-1 Layer III, 128 kbps, 44.1 kHz, stereo + let mp3: Vec = (0..8).flat_map(|_| frame.clone()).collect(); + let malformed = caf(&[caf_desc(b"aac ", 0, 1024, 2, 0), (b"kuki", 1i64 << 50, vec![0; 8])]); + for f in [[vec![0u8], malformed.clone(), mp3.clone()].concat(), [vec![0u8; 32], malformed.clone(), mp3.clone()].concat()] { + // The hint still names the raw stream, but the probe sees the CAF marker first. + assert_eq!(detect_container(&f, Some("mp3")), Some(Container::MpegAudio)); + assert!(matches!(probe(&f, Some("mp3")), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + assert!(matches!(decode(&f, Some("mp3")), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + } + // Plain leading junk without format magic is tolerated, as before. + let f = [vec![0u8; 32], mp3].concat(); + assert_eq!(detect_container(&f, Some("mp3")), Some(Container::MpegAudio)); + let info = probe(&f, Some("mp3")).expect("junk before a valid MP3"); + assert_eq!((info.codec, info.sample_rate, info.channels), (CodecId::Mp3, 44_100, 2)); +} + +/// More false `0xFFEx` pairs than the preflight's scoring budget must not make a later container +/// magic inside the valid audio payload authoritative. Past the cap the scan is unchecked and the +/// probe selects the MPEG reader at the first real frame. +#[test] +fn an_exhausted_audio_score_budget_does_not_reject_the_mp3_behind_it() { + let mut frame = vec![0u8; 417]; + frame[..4].copy_from_slice(&[0xFF, 0xFB, 0x90, 0x00]); + let mut audio: Vec = (0..8).flat_map(|_| frame.clone()).collect(); + let at = 417 * 2 + 10; // Inside frame 2's payload, leaving every frame header intact. + audio[at..at + 8].copy_from_slice(b"caff\0\x01\0\0"); + audio[at + 8..at + 12].copy_from_slice(b"kuki"); + audio[at + 12..at + 20].copy_from_slice(&(1u64 << 50).to_be_bytes()); + let false_syncs: Vec = (0..256).flat_map(|_| [0xFFu8, 0xE2]).collect(); + let f = [false_syncs, audio].concat(); + assert_eq!(detect_container(&f, Some("mp3")), Some(Container::MpegAudio)); + let info = probe(&f, Some("mp3")).expect("valid MP3 after an exhausted audio score budget"); + assert_eq!((info.codec, info.sample_rate, info.channels), (CodecId::Mp3, 44_100, 2)); + let (decoded, buf) = decode(&f, Some("mp3")).expect("decode the same MP3"); + assert_eq!((decoded.codec, decoded.sample_rate, decoded.channels), (CodecId::Mp3, 44_100, 2)); + assert_eq!(decoded.frames, buf.frames() as u64); +} + +/// Build one MPEG-4 AAC-LC ADTS frame at 48 kHz stereo around `payload`, with the 13-bit length covering the 7-byte +/// header. +fn adts_frame(payload: &[u8]) -> Vec { + // MPEG-4 AAC-LC, 48 kHz, stereo; the 13-bit frame length includes this 7-byte header. + let len = 7 + payload.len(); + let mut h = [0xFF, 0xF1, 0x4C, 0x00, 0x00, 0x1F, 0xFC]; + h[3] = 0x80 | ((len >> 11) & 0x03) as u8; // stereo channel config low bits + h[4] = ((len >> 3) & 0xFF) as u8; + h[5] = (((len & 0x07) as u8) << 5) | 0x1F; + [&h[..], payload].concat() +} + +/// A false MPEG sync prefix must not let malformed CAF or MP4 size fields bypass the container preflight. +#[test] +fn a_false_sync_does_not_bypass_container_checks() { + let bad_caf = caf(&[ + caf_desc(b"aac ", 0, 1024, 2, 0), + (b"pakt", 32, packet_table(1, 1024, &[0x82, 0x80, 0x80, 0x80, 0x80, 0x80, 0x80, 0])), + (b"data", -1, vec![0; 64]), + ]); + assert!(matches!(probe(&bad_caf, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + let f = [vec![0xFF, 0xE0, 0x00], bad_caf].concat(); + assert!(matches!(probe(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + + let pcm = sample_entry(b"sowt", 0, &[], &[]); + let bad_mp4 = mp4(480, &pcm, |kind, mut a| { + if kind == b"stsz" { + a[12..16].copy_from_slice(&u32::MAX.to_be_bytes()); + } + a + }); + assert!(probe(&bad_mp4, None).is_err()); + let f = [vec![0xFF, 0xE0, 0x00], bad_mp4].concat(); + assert!(matches!(probe(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); +} + +/// Container magic inside a valid ADTS payload must not override AAC detection or make probing fail. +#[test] +fn a_valid_adts_stream_with_incidental_container_magic_still_probes() { + let payload = [0u8; 400]; + let mut frames: Vec> = (0..8).map(|_| adts_frame(&payload)).collect(); + let at = 100; // Inside frame 2's payload, leaving every frame header intact. + frames[2][7 + at..7 + at + 12].copy_from_slice(b"caff\0\x01\0\0kuki"); + let f: Vec = frames.concat(); + assert_eq!(detect_container(&f, None), Some(Container::Adts)); + let info = probe(&f, None).expect("valid ADTS with incidental container magic"); + assert_eq!((info.codec, info.sample_rate, info.channels), (CodecId::Aac, 48_000, 2)); +} + +/// A later valid CAF must not excuse an earlier malformed container that the preflight or probe rejects. +#[test] +fn an_earlier_malformed_container_is_not_excused_by_a_later_valid_one() { + let malformed = caf(&[caf_desc(b"aac ", 0, 1024, 2, 0), (b"kuki", 1i64 << 50, vec![0; 8])]); + let pcm: Vec = (0..32i16).flat_map(|i| (i * 50).to_be_bytes()).collect(); + let data = [vec![0; 4], pcm].concat(); + let valid = caf(&[caf_desc(b"lpcm", 2, 1, 1, 16), (b"data", data.len() as i64, data)]); + let f = [vec![0u8; 32], malformed, valid].concat(); + assert!(matches!(probe(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); +} + +/// Truncated ID3 bodies, declared tag sizes beyond the file, and a complete tag with no audio after it are malformed +/// for both probe and decode. +#[test] +fn truncated_or_oversized_id3_tags_are_malformed() { + let cases = vec![[&id3_header(100)[..], b"xx"].concat(), id3_header(0x0FFF_FFFF).to_vec(), id3_tag(b"tag")]; + for f in cases { + assert!(matches!(probe(&f, None), Err(AudioError::Malformed(_)))); + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(_)))); + } +} + +#[test] +fn wav_info_entries_beyond_the_file_are_refused() { + let alaw = |len: u32| { + let info = [&b"INFO"[..], b"INAM", &len.to_le_bytes(), b"hi"].concat(); + riff(&[(b"fmt ", [fmt(6, 1, 8_000, 8), vec![0, 0]].concat()), (b"LIST", info), (b"data", vec![0xD5; 800])]) + }; + assert_eq!(decode(&alaw(2), None).unwrap().1.frames(), 800); + let mut f = alaw(0x7fff_fff0); + f[4..8].copy_from_slice(&u32::MAX.to_le_bytes()); + let list = f.windows(4).position(|w| w == b"LIST").unwrap(); + f[list + 4..list + 8].copy_from_slice(&0x7fff_fffcu32.to_le_bytes()); + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); +} + +#[test] +fn a_wave_behind_another_riff_form_is_checked() { + let info = [&b"INFO"[..], b"INAM", &0x7fff_fff0u32.to_le_bytes(), b"hi"].concat(); + let mut wav = riff(&[(b"fmt ", [fmt(6, 1, 8_000, 8), vec![0, 0]].concat()), (b"LIST", info), (b"data", vec![0xD5; 800])]); + wav[4..8].copy_from_slice(&u32::MAX.to_le_bytes()); + let list = wav.windows(4).position(|w| w == b"LIST").unwrap(); + wav[list + 4..list + 8].copy_from_slice(&0x7fff_fffcu32.to_le_bytes()); + // The probe skips the AVI form and reads the WAVE behind it. + let f = [&b"RIFF"[..], &4u32.to_le_bytes(), b"AVI ", &wav].concat(); + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); +} + +#[test] +fn wav_info_entries_are_found_where_symphonia_reads_them() { + let list = [&b"LIST"[..], &0x7fff_fffcu32.to_le_bytes(), b"INFO", b"INAM", &0x7fff_fff0u32.to_le_bytes(), b"hi"].concat(); + let alaw = |extra: u16| (b"fmt ", [fmt(6, 1, 8_000, 8), extra.to_le_bytes().to_vec()].concat()); + // A-law's extra bytes are read past its chunk; an odd list is padded twice. + let files = + [riff(&[alaw(12), (b"JUNK", [&[0; 4][..], &list].concat())]), [riff(&[alaw(0), (b"LIST", b"adtl\0".to_vec())]), vec![0], list].concat()]; + for mut f in files { + f[4..8].copy_from_slice(&u32::MAX.to_le_bytes()); + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(m)) if m.contains("larger than the file"))); + } +} + +#[test] +fn many_tiny_chunks_terminate() { + let mut chunks: Vec<(&[u8; 4], Vec)> = vec![(b"fmt ", fmt(1, 1, 8_000, 16))]; + for _ in 0..200_000 { + chunks.push((b"pad ", Vec::new())); + } + let wav = riff(&chunks); + assert!(decode(&wav, None).is_err()); // no data chunk within the scan limit +} + +#[test] +fn hand_built_wav_variants() { + // 8-bit unsigned. + let wav = riff(&[(b"fmt ", fmt(1, 1, 8_000, 8)), (b"data", vec![0x80, 0xFF, 0x00])]); + let (info, buf) = decode(&wav, None).unwrap(); + assert_eq!(info.sample_format, SampleFormat::Int8); + assert_eq!(buf.channels[0], vec![0.0, 127.0 / 128.0, -1.0]); + // 64-bit float, with NaN replaced by silence. + let mut data = Vec::new(); + for v in [0.25f64, -0.5, f64::NAN] { + data.extend_from_slice(&v.to_le_bytes()); + } + let wav = riff(&[(b"fmt ", fmt(3, 1, 8_000, 64)), (b"data", data)]); + let (info, buf) = decode(&wav, None).unwrap(); + assert_eq!(info.sample_format, SampleFormat::Float64); + assert_eq!(buf.channels[0], vec![0.25, -0.5, 0.0]); + // Unsupported PCM width. + let wav = riff(&[(b"fmt ", fmt(3, 1, 8_000, 16)), (b"data", vec![0; 4])]); + assert!(matches!(decode(&wav, None), Err(AudioError::Unsupported(_)))); + // No data chunk / no fmt chunk. + assert!(decode(&riff(&[(b"fmt ", fmt(1, 1, 8_000, 16))]), None).is_err()); + assert!(decode(&riff(&[(b"data", vec![0; 4])]), None).is_err()); + // Short fmt chunk. + assert!(decode(&riff(&[(b"fmt ", vec![1, 0, 1]), (b"data", vec![0; 4])]), None).is_err()); + // Unknown codec tag goes to symphonia and fails cleanly. + let wav = riff(&[(b"fmt ", fmt(0x1234, 1, 8_000, 16)), (b"data", vec![0; 4])]); + assert!(decode(&wav, None).is_err()); +} + +#[test] +fn aiff_sowt_little_endian() { + let be = small(EncodeFormat::Aiff, BitDepth::Int16, 2); + let (_, expect) = decode(&be, None).unwrap(); + // Rebuild as AIFC/sowt by hand. + let comm = be.windows(4).position(|w| w == b"COMM").unwrap(); + let ssnd = be.windows(4).position(|w| w == b"SSND").unwrap(); + let mut comm_body = be[comm + 8..comm + 26].to_vec(); + comm_body.extend_from_slice(b"sowt"); + comm_body.extend_from_slice(&[0, 0]); // empty pstring, padded + let mut ssnd_body = be[ssnd + 8..].to_vec(); + for pair in ssnd_body[8..].as_chunks_mut::<2>().0 { + pair.swap(0, 1); + } + let mut body = b"AIFC".to_vec(); + for (id, data) in [(b"COMM", &comm_body), (b"SSND", &ssnd_body)] { + body.extend_from_slice(id); + body.extend_from_slice(&(data.len() as u32).to_be_bytes()); + body.extend_from_slice(data); + } + let mut f = b"FORM".to_vec(); + f.extend_from_slice(&(body.len() as u32).to_be_bytes()); + f.extend_from_slice(&body); + let (info, got) = decode(&f, None).unwrap(); + assert_eq!(info.sample_format, SampleFormat::Int16); + assert_eq!(got, expect); +} + +#[test] +fn detect_container_by_magic_and_hint() { + assert_eq!(detect_container(&small(EncodeFormat::Wav, BitDepth::Int16, 1), Some("mp3")), Some(Container::Wav)); + assert_eq!(detect_container(&small(EncodeFormat::Aiff, BitDepth::Int16, 1), None), Some(Container::Aiff)); + assert_eq!(detect_container(b"OggS\0\x02rest", None), Some(Container::Ogg)); + assert_eq!(detect_container(b"caff\0\x01\0\0", None), Some(Container::Caf)); + assert_eq!(detect_container(b"ID3\x04\0\0\0\0\0\x00", None), Some(Container::MpegAudio)); + let mut id3_flac = b"ID3\x04\0\0\0\0\0\x02xx".to_vec(); + id3_flac.extend_from_slice(b"fLaC"); + assert_eq!(detect_container(&id3_flac, None), Some(Container::Flac)); + assert_eq!(detect_container(&[0xFF, 0xFB, 0x90, 0x00], None), Some(Container::MpegAudio)); + assert_eq!(detect_container(&[0xFF, 0xF1, 0x50, 0x80], None), Some(Container::Adts)); + assert_eq!(detect_container(&[0x0B, 0x77, 0x00, 0x00], None), Some(Container::Ac3)); + let mut m4a = b"\0\0\0\x20ftypM4A ".to_vec(); + m4a.extend_from_slice(b"....stsd\0\0\0\0\0\0\0\x01\0\0\0\x24alac"); + assert_eq!(detect_container(&m4a, None), Some(Container::Mp4)); + assert_eq!(detect_container(b"\0\0\0\x20ftypM4A ", None), Some(Container::Mp4)); + assert_eq!(detect_container(b"\0\0\0\x14ftypqt ", None), Some(Container::QuickTime)); + assert_eq!(detect_container(b"", Some(".WAV")), Some(Container::Wav)); + assert_eq!(detect_container(b"zz", Some("aif")), Some(Container::Aiff)); + assert_eq!(detect_container(b"zz", Some("m4a")), Some(Container::Mp4)); + assert_eq!(detect_container(b"zz", Some("M4B")), Some(Container::Mp4)); + assert_eq!(detect_container(b"zz", Some("webm")), Some(Container::WebM)); + assert_eq!(detect_container(b"zz", Some("flac")), Some(Container::Flac)); + assert_eq!(detect_container(b"zz", Some("xyz")), None); + assert_eq!(detect_container(b"zz", None), None); + assert_eq!(detect_container(b"", None), None); +} + +#[test] +fn garbage_in_compressed_containers_errors() { + let garbage: Vec = (0..4096u32).map(|i| (i.wrapping_mul(2_654_435_761) >> 13) as u8).collect(); + for hint in [None, Some("mp3"), Some("ogg"), Some("flac"), Some("m4a"), Some("caf"), Some("aac"), Some("mkv"), Some("wav"), Some("aiff")] { + assert!(decode(&garbage, hint).is_err(), "{hint:?}"); + assert!(probe(&garbage, hint).is_err(), "{hint:?}"); + } + for magic in [&b"fLaC"[..], b"OggS", b"caff", b"ID3\x03\0\0\0\0\0\0", &[0xFF, 0xFB, 0x90, 0x00], b"\0\0\0\x20ftypM4A "] { + let mut f = magic.to_vec(); + f.extend_from_slice(&garbage); + let _ = probe(&f, None); + assert!(decode(&f, None).map(|(_, b)| b.frames()).unwrap_or(0) < 1_000_000); + } + assert!(decode(&[], None).is_err()); +} + +proptest! { + #![proptest_config(ProptestConfig { cases: 256, ..ProptestConfig::default() })] + + #[test] + fn random_bytes_never_panic(bytes in proptest::collection::vec(any::(), 0..2048), hint in 0usize..6) { + let hint = [None, Some("wav"), Some("aiff"), Some("flac"), Some("mp3"), Some("ogg")][hint]; + let _ = detect_container(&bytes, hint); + let _ = probe(&bytes, hint); + let _ = decode(&bytes, hint); + } + + #[test] + fn random_bodies_behind_valid_headers_never_panic(body in proptest::collection::vec(any::(), 0..1024), which in 0usize..4) { + let mut f = match which { + 0 => b"RIFF\xff\xff\xff\xffWAVE".to_vec(), + 1 => b"RF64\xff\xff\xff\xffWAVE".to_vec(), + 2 => b"FORM\xff\xff\xff\xffAIFC".to_vec(), + _ => b"FORM\xff\xff\xff\xffAIFF".to_vec(), + }; + f.extend_from_slice(&body); + let _ = probe(&f, None); + let _ = decode(&f, None); + } +} + +/// An extensible `fmt ` chunk: cbSize, valid bits, mask, then the 16-byte subtype GUID. +fn extensible_fmt(cb_size: u16, valid_bits: u16, guid: &[u8]) -> Vec { + let mut f = fmt(0xFFFE, 1, 44_100, 16); + f.extend_from_slice(&cb_size.to_le_bytes()); + f.extend_from_slice(&valid_bits.to_le_bytes()); + f.extend_from_slice(&4u32.to_le_bytes()); + f.extend_from_slice(guid); + f +} + +const PCM_GUID: [u8; 16] = [1, 0, 0, 0, 0, 0, 0x10, 0, 0x80, 0, 0, 0xAA, 0, 0x38, 0x9B, 0x71]; + +#[test] +fn extensible_fmt_is_checked_in_full() { + let data = vec![0u8; 8]; + let good = riff(&[(b"fmt ", extensible_fmt(22, 16, &PCM_GUID)), (b"data", data.clone())]); + assert_eq!(probe(&good, None).unwrap().sample_format, SampleFormat::Int16); + // Subtype GUID cut short after the format code. + let cut = riff(&[(b"fmt ", extensible_fmt(22, 16, &PCM_GUID[..4])), (b"data", data.clone())]); + assert!(matches!(probe(&cut, None), Err(AudioError::Malformed(_)))); + assert!(matches!(decode(&cut, None), Err(AudioError::Malformed(_)))); + // cbSize too small for the extension. + let small_cb = riff(&[(b"fmt ", extensible_fmt(0, 16, &PCM_GUID)), (b"data", data.clone())]); + assert!(matches!(probe(&small_cb, None), Err(AudioError::Malformed(_)))); + // More valid bits than the 16-bit container holds. + let wide = riff(&[(b"fmt ", extensible_fmt(22, 24, &PCM_GUID)), (b"data", data.clone())]); + assert!(matches!(probe(&wide, None), Err(AudioError::Malformed(_)))); + // PCM-like prefix but a foreign GUID tail is not read as native PCM. + let mut foreign = PCM_GUID; + foreign[15] ^= 0xFF; + let foreign = riff(&[(b"fmt ", extensible_fmt(22, 16, &foreign)), (b"data", data)]); + assert!(!matches!(probe(&foreign, None), Ok(i) if i.sample_format == SampleFormat::Int16)); + // The Ambisonic PCM subtype is PCM. + let ambi = [1, 0, 0, 0, 0x21, 0x07, 0xD3, 0x11, 0x86, 0x44, 0xC8, 0xC1, 0xCA, 0, 0, 0]; + let ambi = riff(&[(b"fmt ", extensible_fmt(22, 16, &ambi)), (b"data", vec![0u8; 8])]); + assert_eq!(probe(&ambi, None).unwrap().sample_format, SampleFormat::Int16); +} + +#[test] +fn aiff_frames_without_sound_data_are_malformed() { + // COMM declares 300 frames, and the file ends before any SSND chunk + let full = small(EncodeFormat::Aiff, BitDepth::Int16, 1); + let ssnd = full.windows(4).position(|w| w == b"SSND").unwrap(); + let mut f = full[..ssnd].to_vec(); + let size = (f.len() - 8) as u32; + f[4..8].copy_from_slice(&size.to_be_bytes()); + assert!(matches!(probe(&f, None), Err(AudioError::Malformed(_)))); + // with no frames declared, no SSND is needed (AIFF 1.3) + let comm = f.windows(4).position(|w| w == b"COMM").unwrap(); + f[comm + 10..comm + 14].fill(0); + assert_eq!(probe(&f, None).unwrap().frames, 0); +} + +#[test] +fn an_empty_flac_has_one_track() { + use craft_audio::{DecodeOptions, decode_with}; + // STREAMINFO: 4096-frame blocks, 44.1 kHz, 2 channels, 16 bits, no samples + let mut f = b"fLaC".to_vec(); + f.extend_from_slice(&[0x80, 0, 0, 34]); + f.extend_from_slice(&[0x10, 0x00, 0x10, 0x00, 0, 0, 0, 0, 0, 0]); + f.extend_from_slice(&((44_100u64 << 44) | (1 << 41) | (15 << 36)).to_be_bytes()); + f.extend_from_slice(&[0; 16]); + assert_eq!(decode_with(&f, None, &DecodeOptions::default().with_track(0)).unwrap().1.frames(), 0); + let e = decode_with(&f, None, &DecodeOptions::default().with_track(1)).unwrap_err(); + assert!(matches!(e, AudioError::Unsupported(_)), "{e}"); +} + +/// A damaged MP4 or Matroska file is malformed when symphonia cannot read it either, as it is +/// without symphonia. +#[test] +fn damaged_mp4_and_matroska_are_malformed() { + // an `ftyp` box and nothing after it: no `moov` + let mp4 = [&16u32.to_be_bytes()[..], b"ftypisom", &[0, 0, 2, 0]].concat(); + // an EBML header naming Matroska and nothing after it: no Segment + let mkv = [&[0x1A, 0x45, 0xDF, 0xA3, 0x8B, 0x42, 0x82, 0x88][..], b"matroska"].concat(); + for f in [mp4, mkv] { + assert!(matches!(decode(&f, None), Err(AudioError::Malformed(_))), "{:?}", decode(&f, None)); + assert!(matches!(probe(&f, None), Err(AudioError::Malformed(_)))); + assert!(matches!(craft_audio::tracks(&f, None), Err(AudioError::Malformed(_)))); + } +} diff --git a/crates/audio/tests/audio/roundtrip.rs b/crates/audio/tests/audio/roundtrip.rs new file mode 100644 index 00000000..51198048 --- /dev/null +++ b/crates/audio/tests/audio/roundtrip.rs @@ -0,0 +1,274 @@ +use craft_audio::{ + AudioBuffer, BitDepth, BwfInfo, ChannelLayout, EncodeFormat, EncodeOptions, SampleFormat, decode, detect_container, encode, probe, + wav_float_header, +}; + +fn sine(sr: u32, channels: usize, frames: usize) -> AudioBuffer { + let mut b = AudioBuffer::new(sr, channels as u16, frames); + for (c, ch) in b.channels.iter_mut().enumerate() { + let f = 220.0 * (c as f32 + 1.0); + for (i, s) in ch.iter_mut().enumerate() { + *s = 0.8 * (2.0 * std::f32::consts::PI * f * i as f32 / sr as f32 + c as f32).sin(); + } + } + b +} + +fn tolerance(depth: BitDepth, dither: bool) -> f32 { + let lsb = match depth { + BitDepth::Int16 => 1.0 / 32768.0, + BitDepth::Int24 => 1.0 / 8_388_608.0, + BitDepth::Int32 => 1e-7, + BitDepth::Float32 => 0.0, + _ => unreachable!("a bit depth this test does not know"), + }; + lsb * if dither { 1.6 } else { 0.6 } + 1e-7 +} + +fn max_err(a: &AudioBuffer, b: &AudioBuffer) -> f32 { + assert_eq!(a.num_channels(), b.num_channels()); + assert_eq!(a.frames(), b.frames()); + a.channels.iter().zip(&b.channels).flat_map(|(x, y)| x.iter().zip(y).map(|(p, q)| (p - q).abs())).fold(0.0, f32::max) +} + +const DEPTHS: [BitDepth; 4] = [BitDepth::Int16, BitDepth::Int24, BitDepth::Int32, BitDepth::Float32]; + +fn expected_sf(d: BitDepth) -> SampleFormat { + match d { + BitDepth::Int16 => SampleFormat::Int16, + BitDepth::Int24 => SampleFormat::Int24, + BitDepth::Int32 => SampleFormat::Int32, + BitDepth::Float32 => SampleFormat::Float32, + _ => unreachable!("a bit depth this test does not know"), + } +} + +#[test] +fn wav_and_aiff_roundtrip_all_depths_and_layouts() { + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + for depth in DEPTHS { + for channels in [1, 2, 6] { + for dither in [false, true] { + let src = sine(48_000, channels, 4801); + let opts = EncodeOptions::new(format).with_bit_depth(depth).with_dither(dither); + let bytes = encode(&src, &opts).unwrap(); + assert_eq!(detect_container(&bytes, None), Some(format.container())); + let info = probe(&bytes, None).unwrap(); + assert_eq!(info.container, format.container()); + assert_eq!(info.sample_rate, 48_000); + assert_eq!(info.channels as usize, channels); + assert_eq!(info.frames, 4801); + assert_eq!(info.sample_format, expected_sf(depth)); + let (dinfo, out) = decode(&bytes, None).unwrap(); + assert_eq!(dinfo, info); + assert_eq!(out.sample_rate, 48_000); + let err = max_err(&src, &out); + assert!(err <= tolerance(depth, dither), "{format:?} {depth:?} {channels}ch dither={dither}: err {err}"); + } + } + } + } +} + +#[test] +fn bwf_roundtrip() { + let bwf = BwfInfo::default() + .with_description("Scene 12 take 3") + .with_originator("SoundCraft") + .with_originator_reference("SC-0001") + .with_origination("2026-10-06", "12:34:56") + .with_time_reference(0x1_2345_6789); // > 32 bits + let src = sine(96_000, 2, 1000); + // RF64 is a WAV layout: other formats ignore it. + let aiff = encode(&src, &EncodeOptions::new(EncodeFormat::Aiff).with_force_rf64(true)).unwrap(); + assert_eq!(&aiff[..4], b"FORM"); + for depth in DEPTHS { + let opts = EncodeOptions::new(EncodeFormat::Wav).with_bit_depth(depth).with_dither(false).with_bwf(bwf.clone()); + let bytes = encode(&src, &opts).unwrap(); + let info = probe(&bytes, None).unwrap(); + assert_eq!(info.bwf.as_ref(), Some(&bwf)); + let (dinfo, _) = decode(&bytes, None).unwrap(); + assert_eq!(dinfo.bwf, Some(bwf.clone())); + // RF64 layout reads back identically. + let rf64 = encode(&src, &opts.clone().with_force_rf64(true)).unwrap(); + assert_eq!(&rf64[..4], b"RF64"); + let (rinfo, rbuf) = decode(&rf64, None).unwrap(); + assert_eq!(rinfo.bwf, Some(bwf.clone())); + assert_eq!(rinfo.frames, 1000); + assert_eq!(rbuf, decode(&bytes, None).unwrap().1); + } + // Overlong strings are truncated to the field width, not an error. + let long = BwfInfo::default().with_description("x".repeat(400)); + let bytes = encode(&src, &EncodeOptions::default().with_bwf(long)).unwrap(); + assert_eq!(probe(&bytes, None).unwrap().bwf.unwrap().description.len(), 256); +} + +#[test] +fn wav_header_layout() { + let b = encode(&sine(44_100, 2, 10), &EncodeOptions::default().with_bit_depth(BitDepth::Int16)).unwrap(); + // Plain PCM for 16-bit stereo; extensible otherwise. + let fmt = b.windows(4).position(|w| w == b"fmt ").unwrap(); + assert_eq!(u16::from_le_bytes([b[fmt + 8], b[fmt + 9]]), 1); + let b = encode(&sine(44_100, 2, 10), &EncodeOptions::default()).unwrap(); + let fmt = b.windows(4).position(|w| w == b"fmt ").unwrap(); + assert_eq!(u16::from_le_bytes([b[fmt + 8], b[fmt + 9]]), 0xFFFE); + let riff = u32::from_le_bytes([b[4], b[5], b[6], b[7]]) as usize; + assert_eq!(riff + 8, b.len()); + // Odd data length (24-bit mono, odd frames) gets a pad byte. + let b = encode(&sine(44_100, 1, 3), &EncodeOptions::default()).unwrap(); + assert_eq!(b.len() % 2, 0); + assert_eq!(decode(&b, None).unwrap().1.frames(), 3); +} + +#[test] +fn aiff_float_is_aifc() { + let b = encode(&sine(44_100, 2, 10), &EncodeOptions::new(EncodeFormat::Aiff).with_bit_depth(BitDepth::Float32)).unwrap(); + assert_eq!(&b[8..12], b"AIFC"); + assert!(b.windows(4).any(|w| w == b"fl32")); + let b = encode(&sine(44_100, 2, 10), &EncodeOptions::new(EncodeFormat::Aiff)).unwrap(); + assert_eq!(&b[8..12], b"AIFF"); + let form = u32::from_be_bytes([b[4], b[5], b[6], b[7]]) as usize; + assert_eq!(form + 8, b.len()); +} + +#[test] +fn unusual_sample_rates_roundtrip() { + for sr in [1, 8_000, 11_025, 22_050, 44_100, 88_200, 176_400, 192_000, 352_800, 384_000, 768_000] { + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + let bytes = encode(&sine(sr, 1, 8), &EncodeOptions::new(format)).unwrap(); + assert_eq!(probe(&bytes, None).unwrap().sample_rate, sr, "{format:?}"); + } + } +} + +#[test] +fn encode_rejects_bad_buffers_and_formats() { + assert!(encode(&AudioBuffer::default(), &EncodeOptions::default()).is_err()); + assert!(encode(&AudioBuffer::new(0, 2, 10), &EncodeOptions::default()).is_err()); + // Formats `encode` cannot write are not `EncodeFormat` variants, so asking for one does not compile. + // Non-finite samples are sanitized; only integer output clips finite samples. + let mut b = AudioBuffer::new(48_000, 1, 4); + b.channels[0] = vec![f32::NAN, f32::INFINITY, -5.0, 0.5]; + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + for depth in DEPTHS { + let bytes = encode(&b, &EncodeOptions::new(format).with_bit_depth(depth)).unwrap(); + let (_, out) = decode(&bytes, None).unwrap(); + assert_eq!(out.channels[0][0], 0.0); + assert!(out.channels[0][1] > 0.99 && out.channels[0][1] <= 1.0); + assert_eq!(out.channels[0][2], if depth == BitDepth::Float32 { -5.0 } else { -1.0 }); + } + } + // Ragged buffers encode the common length. + let ragged = AudioBuffer { sample_rate: 48_000, channels: vec![vec![0.1; 10], vec![0.2; 7]] }; + let (_, out) = decode(&encode(&ragged, &EncodeOptions::default()).unwrap(), None).unwrap(); + assert_eq!(out.frames(), 7); +} + +#[test] +fn audio_buffer_helpers() { + let b = AudioBuffer::from_interleaved(48_000, 2, &[0.1, -0.2, 0.3, -0.4, 0.5]); + assert_eq!(b.frames(), 2); + assert_eq!(b.num_channels(), 2); + assert_eq!(b.interleaved(), vec![0.1, -0.2, 0.3, -0.4]); + assert!((b.peak() - 0.4).abs() < 1e-7); + assert!((b.duration_secs() - 2.0 / 48_000.0).abs() < 1e-12); + assert_eq!(AudioBuffer::from_interleaved(1, 0, &[1.0]).num_channels(), 0); + assert_eq!(AudioBuffer::new(0, 1, 5).duration_secs(), 0.0); + assert_eq!(AudioBuffer::default().frames(), 0); +} + +#[cfg(feature = "serde")] +#[test] +fn serde_types_roundtrip() { + let opts = EncodeOptions::default().with_bwf(BwfInfo::default().with_time_reference(42)); + let s = serde_json::to_string(&opts).unwrap(); + assert_eq!(serde_json::from_str::(&s).unwrap(), opts); + let info = probe(&encode(&sine(48_000, 1, 4), &opts).unwrap(), None).unwrap(); + let s = serde_json::to_string(&info).unwrap(); + assert_eq!(serde_json::from_str::(&s).unwrap(), info); + // A layout reads back only when its mask describes its channel count. + let l = ChannelLayout::SURROUND_5_1; + assert_eq!(serde_json::from_str::(&serde_json::to_string(&l).unwrap()).unwrap(), l); + assert_eq!(serde_json::from_str::(r#"{"channels":4,"mask":0}"#).unwrap(), ChannelLayout::unordered(4)); + assert!(serde_json::from_str::(r#"{"channels":2,"mask":63}"#).is_err()); + assert!(serde_json::from_str::(r#"{"channels":1,"mask":2147483648}"#).is_err()); +} + +#[test] +fn multichannel_wav_speaker_masks() { + let buf = |n: usize| AudioBuffer { sample_rate: 48_000, channels: (0..n).map(|c| vec![0.01 * c as f32; 64]).collect() }; + let opts = EncodeOptions::new(EncodeFormat::Wav).with_bit_depth(BitDepth::Int24).with_dither(false); + for (n, mask) in [(6usize, 0x3Fu32), (8, 0x63F), (12, 0x2_D63F)] { + let bytes = craft_audio::encode(&buf(n), &opts).unwrap(); + assert_eq!(probe(&bytes, None).unwrap().layout, ChannelLayout::from_mask(mask), "{n} channels"); + assert_eq!(craft_audio::decode(&bytes, Some("wav")).unwrap().1.channels.len(), n); + } + // Explicit masks: 7.0 (sides + rears, no LFE) and Ambisonics (no positions). + let bytes = craft_audio::encode(&buf(7), &opts.clone().with_layout(ChannelLayout::from_mask(0x637))).unwrap(); + assert_eq!(probe(&bytes, None).unwrap().layout, ChannelLayout::from_mask(0x637)); + let bytes = craft_audio::encode(&buf(4), &opts.clone().with_layout(ChannelLayout::unordered(4))).unwrap(); + assert_eq!(probe(&bytes, None).unwrap().layout, ChannelLayout::unordered(4)); + // Plain 16-bit stereo stays a plain PCM header (format tag 1), read back as stereo. + let o16 = opts.clone().with_bit_depth(BitDepth::Int16); + let bytes = craft_audio::encode(&buf(2), &o16.clone().with_layout(ChannelLayout::STEREO)).unwrap(); + let fmt = bytes.windows(4).position(|w| w == b"fmt ").unwrap(); + assert_eq!(bytes.get(fmt + 8..fmt + 10), Some(&[1u8, 0][..])); + assert_eq!(probe(&bytes, None).unwrap().layout, ChannelLayout::STEREO); + // A layout for another channel count is refused. + assert!(craft_audio::encode(&buf(2), &o16.clone().with_layout(ChannelLayout::SURROUND_5_1)).is_err()); +} + +#[test] +fn float_wav_header_is_rewritten_in_place_as_the_file_grows() { + let take = sine(48_000, 2, 1_000); + let mut file = wav_float_header(48_000, 2, 0).unwrap(); + let len = file.len(); + file.extend(take.interleaved().iter().flat_map(|s| s.to_le_bytes())); + assert_eq!(decode(&file, None).unwrap().1.frames(), 1_000); + file[..len].copy_from_slice(&wav_float_header(48_000, 2, 600).unwrap()); + assert_eq!(decode(&file, None).unwrap().1.frames(), 600); + file[..len].copy_from_slice(&wav_float_header(48_000, 2, 1_000).unwrap()); + assert_eq!(file, encode(&take, &EncodeOptions::default().with_bit_depth(BitDepth::Float32)).unwrap()); + let big = wav_float_header(48_000, 2, 600_000_000).unwrap(); + assert_eq!((&big[..4], big.len()), (&b"RF64"[..], len)); + assert!(wav_float_header(48_000, 0, 0).is_err()); +} + +#[test] +fn float_wav_header_rejects_a_frame_count_whose_total_size_overflows() { + // 4 bytes per frame: the data length fits in u64, but adding the header to it does not. + let frames = u64::MAX / 4; + assert!(matches!(wav_float_header(48_000, 1, frames), Err(craft_audio::AudioError::TooLarge(_)))); +} + +#[test] +fn dither_changes_integer_output() { + // a quiet signal whose samples fall between the integer levels + let mut src = AudioBuffer::new(48_000, 1, 4800); + for (i, s) in src.channels[0].iter_mut().enumerate() { + *s = 0.001 * (i as f32 * 0.01).sin(); + } + for depth in [BitDepth::Int16, BitDepth::Int24, BitDepth::Int32] { + let opts = EncodeOptions::new(EncodeFormat::Wav).with_bit_depth(depth); + let plain = encode(&src, &opts.clone().with_dither(false)).unwrap(); + let dithered = encode(&src, &opts.with_dither(true)).unwrap(); + assert_ne!(plain, dithered, "{depth:?}"); + } +} + +#[test] +fn bwf_text_is_cut_at_a_character_boundary() { + // 255 ASCII bytes then a 2-byte character: the 256-byte description field cannot hold it + let description = format!("{}é", "a".repeat(255)); + let bwf = BwfInfo::default().with_description(&description); + let opts = EncodeOptions::new(EncodeFormat::Wav).with_bwf(bwf); + let info = probe(&encode(&sine(48_000, 1, 100), &opts).unwrap(), None).unwrap(); + assert_eq!(info.bwf.unwrap().description, "a".repeat(255)); +} + +#[test] +fn a_byte_rate_past_32_bits_is_an_error() { + // 4 GHz × 8 bytes per frame does not fit the 32-bit nAvgBytesPerSec field + assert!(matches!(wav_float_header(4_000_000_000, 2, 0), Err(craft_audio::AudioError::Encode(_)))); + assert!(wav_float_header(4_000_000_000 / 8, 2, 0).is_ok()); +} diff --git a/crates/audio/tests/audio/stream.rs b/crates/audio/tests/audio/stream.rs new file mode 100644 index 00000000..a948ccf7 --- /dev/null +++ b/crates/audio/tests/audio/stream.rs @@ -0,0 +1,259 @@ +use craft_audio::{AudioBuffer, AudioError, BitDepth, BwfInfo, ChannelLayout, EncodeFormat, EncodeOptions, FileEncoder, encode}; +use std::io::{Cursor, Seek, SeekFrom, Write}; + +fn tone(channels: usize, frames: usize) -> AudioBuffer { + let channels = (0..channels).map(|c| (0..frames).map(|i| ((i * (c + 3)) as f32 * 0.013).sin() * 1.2).collect()).collect(); + AudioBuffer { sample_rate: 44_100, channels } +} + +fn streamed(buf: &AudioBuffer, opts: &EncodeOptions) -> Vec { + let mut out = Cursor::new(b"pre".to_vec()); + out.set_position(3); + let mut enc = FileEncoder::new(out, opts, buf.sample_rate, buf.channels.len() as u16).unwrap(); + let mut start = 0; + for len in [1, 7, 4095, 4097, 333].into_iter().cycle() { + if start >= buf.frames() { + break; + } + let len = len.min(buf.frames() - start); + // Borrowed slices, as a caller with its own buffers passes them. + let block: Vec<&[f32]> = buf.channels.iter().map(|c| &c[start..start + len]).collect(); + enc.write(&block, len).unwrap(); + start += len; + } + enc.finish().unwrap().into_inner() +} + +#[test] +fn streamed_files_match_whole_buffer_encodes() { + let bwf = BwfInfo::default().with_description("take").with_time_reference(123_456); + let cases = [ + (EncodeFormat::Wav, BitDepth::Int24, true, None, None), + (EncodeFormat::Wav, BitDepth::Float32, false, None, None), + (EncodeFormat::Wav, BitDepth::Int16, true, Some(bwf), Some(ChannelLayout::SURROUND_5_1_SIDE)), + (EncodeFormat::Aiff, BitDepth::Int24, true, None, None), + (EncodeFormat::Aiff, BitDepth::Float32, false, None, None), + ]; + for (format, bit_depth, dither, bwf, layout) in cases { + let base = EncodeOptions::new(format).with_bit_depth(bit_depth).with_dither(dither); + let base = match bwf { + Some(b) => base.with_bwf(b), + None => base, + }; + for channels in [1, 6] { + let buf = tone(channels, 20_001); + // A layout applies to the buffer with as many channels. + let opts = match layout.filter(|l: &ChannelLayout| usize::from(l.channels()) == channels) { + Some(l) => base.clone().with_layout(l), + None => base.clone(), + }; + let whole = encode(&buf, &opts).unwrap(); + assert_eq!(streamed(&buf, &opts), [b"pre".as_slice(), &whole].concat(), "{opts:?}, {channels} channels"); + } + } +} + +/// A `Write + Seek` sink that accepts `budget` bytes in total and then fails every write, so +/// a `write_all` on it appends a prefix of the block and only then reports the error. +struct FailAfter { + inner: Cursor>, + budget: usize, +} + +impl Write for FailAfter { + fn write(&mut self, buf: &[u8]) -> std::io::Result { + if self.budget == 0 { + return Err(std::io::Error::other("sink is full")); + } + let n = buf.len().min(self.budget); + self.budget -= n; + self.inner.write(&buf[..n]) + } + + fn flush(&mut self) -> std::io::Result<()> { + self.inner.flush() + } +} + +impl Seek for FailAfter { + fn seek(&mut self, pos: SeekFrom) -> std::io::Result { + self.inner.seek(pos) + } +} + +#[test] +fn a_failed_write_poisons_the_encoder() { + let buf = tone(2, 20_001); + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + let opts = EncodeOptions::new(format).with_bit_depth(BitDepth::Int16).with_dither(false); + let sink = FailAfter { inner: Cursor::new(Vec::new()), budget: 256 }; + let mut enc = FileEncoder::new(sink, &opts, buf.sample_rate, 2).unwrap(); + // The sink accepts a prefix of the first block, then fails. + assert!(enc.write(&buf.channels, buf.frames()).is_err(), "{format:?}"); + // Later calls must not claim success: the bytes on the sink no longer match the + // encoder's counters. + assert!(matches!(enc.write(&buf.channels, 16), Err(AudioError::Encode(_))), "{format:?}"); + assert!(matches!(enc.finish(), Err(AudioError::Encode(_))), "{format:?}"); + } +} + +#[test] +fn blocks_past_a_format_limit_are_refused_before_anything_is_written() { + let buf = tone(2, 100); + // AIFF sizes are 32-bit: 4 GiB of 16-bit stereo. + { + let (format, limit) = (EncodeFormat::Aiff, 1u64 << 30); + let opts = EncodeOptions::new(format).with_bit_depth(BitDepth::Int16).with_dither(false); + let mut enc = FileEncoder::new(Cursor::new(Vec::new()), &opts, buf.sample_rate, 2).unwrap(); + enc.write(&buf.channels, 100).unwrap(); + assert!(enc.check_room(limit / 2).is_ok(), "{format:?}"); + assert!(matches!(enc.check_room(limit), Err(AudioError::TooLarge(_))), "{format:?}"); + // A block that does not hold the frames it claims is refused before the limit is checked. + if let Ok(frames) = usize::try_from(limit) { + assert!(matches!(enc.write::>(&[], frames), Err(AudioError::Encode(_))), "{format:?}"); + } + assert_eq!(enc.finish().unwrap().into_inner(), encode(&buf, &opts).unwrap()); + } +} + +// A source that stops serving reads past `fail_from`, as a network-backed file might. +struct FlakySource { + bytes: Vec, + fail_from: u64, +} + +use craft_media::ByteSource; + +impl ByteSource for FlakySource { + fn len(&self) -> u64 { + self.bytes.len() as u64 + } + fn read_at(&self, offset: u64, buf: &mut [u8]) -> std::io::Result<()> { + if offset + buf.len() as u64 > self.fail_from { + return Err(std::io::Error::other("connection reset")); + } + self.bytes.as_slice().read_at(offset, buf) + } +} + +#[test] +fn decoder_reads_any_read_seek_source_and_reports_source_failures() { + use craft_audio::{DecodeOptions, Decoder}; + use craft_media::ReadSeekSource; + let wav = encode(&tone(2, 48_000), &EncodeOptions::new(EncodeFormat::Wav)).unwrap(); + + // `ReadSeekSource` (any `Read + Seek`) is accepted, so a `BufReader` works the same way. + let source = ReadSeekSource::new(Cursor::new(wav.clone())).unwrap(); + let info = Decoder::open(source, Some("wav"), &DecodeOptions::default()).unwrap().probe().unwrap(); + assert_eq!((info.frames, info.damaged), (48_000, 0)); + + // A source failing half way through gives an I/O error, not concealed silence. + let half = wav.len() as u64 / 2; + let mut dec = Decoder::open(FlakySource { bytes: wav, fail_from: half }, Some("wav"), &DecodeOptions::default()).unwrap(); + let err = loop { + match dec.next_block() { + Ok(Some(_)) => continue, + Ok(None) => panic!("the source failure was not reported"), + Err(e) => break e, + } + }; + assert!(matches!(err, AudioError::Io(_)), "{err}"); + assert_eq!(dec.info().damaged, 0); +} + +#[test] +fn seeking_to_or_past_the_end_ends_the_stream() { + use craft_audio::{DecodeOptions, Decoder}; + let buf = tone(2, 4800); + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + let file = encode(&buf, &EncodeOptions::new(format).with_bit_depth(BitDepth::Int16)).unwrap(); + for target in [4800, 4801, 1_000_000] { + let mut dec = Decoder::from_bytes(&file, None, &DecodeOptions::default()).unwrap(); + assert_eq!(dec.seek(target).unwrap(), 4800, "{format:?} seek({target})"); + assert!(dec.next_block().unwrap().is_none(), "{format:?} seek({target})"); + // Seeking back still works. + assert_eq!(dec.seek(4000).unwrap(), 4000, "{format:?}"); + let left: usize = std::iter::from_fn(|| dec.next_block().unwrap()).map(|b| b.frames()).sum(); + assert_eq!(left, 800, "{format:?}"); + } + } +} + +#[test] +fn single_track_files_refuse_other_track_indexes() { + use craft_audio::{DecodeOptions, decode_with}; + let buf = tone(1, 1000); + for format in [EncodeFormat::Wav, EncodeFormat::Aiff] { + let file = encode(&buf, &EncodeOptions::new(format)).unwrap(); + assert!(decode_with(&file, None, &DecodeOptions::default().with_track(0)).is_ok(), "{format:?}"); + let e = decode_with(&file, None, &DecodeOptions::default().with_track(1)).unwrap_err(); + assert!(matches!(e, AudioError::Unsupported(_)), "{format:?}: {e}"); + } +} + +#[test] +fn short_or_missing_channels_are_refused_without_writing() { + let opts = EncodeOptions::new(EncodeFormat::Wav).with_bit_depth(BitDepth::Int16); + let mut enc = FileEncoder::new(Cursor::new(Vec::new()), &opts, 48_000, 2).unwrap(); + let left = vec![0.1f32; 4]; + // More frames than the channels hold, or too few channels: refused, nothing written. + assert!(matches!(enc.write(&[&left, &left], 10), Err(AudioError::Encode(_)))); + assert!(matches!(enc.write(&[&left], 4), Err(AudioError::Encode(_)))); + // The encoder is still usable. + enc.write(&[&left, &left], 4).unwrap(); + let wav = enc.finish().unwrap().into_inner(); + assert_eq!(craft_audio::probe(&wav, None).unwrap().frames, 4); +} + +/// A 48 kHz mono float PCM Matroska file: ten 10 ms blocks from `first_ns`, with block durations +/// when `durations` is set (otherwise its length is unknown), and a `CodecDelay` of `delay_ns`. +fn pcm_mkv(first_ns: i64, durations: bool, delay_ns: u64) -> Vec { + use craft_media::matroska::{LacingMode, MkvWriter, MuxOptions, TrackKind, TrackSpec}; + let mut spec = TrackSpec::new(TrackKind::Audio, "A_PCM/FLOAT/IEEE"); + spec.audio = Some((48_000.0, 1, Some(32))); + spec.codec_delay_ns = delay_ns; + let opts = MuxOptions { lacing: LacingMode::None, ..MuxOptions::default() }; + let mut w = MkvWriter::new(Cursor::new(Vec::new()), vec![spec], opts).unwrap(); + for k in 0..10i64 { + let data: Vec = (0..480).flat_map(|i| ((k * 480 + i) as f32 * 0.001).sin().to_le_bytes()).collect(); + w.write_frame(0, first_ns + k * 10_000_000, true, &data, durations.then_some(10_000_000)).unwrap(); + } + w.finish().unwrap().into_inner() +} + +#[test] +fn a_matroska_track_that_starts_late_counts_only_its_own_frames() { + use craft_audio::{DecodeOptions, Decoder}; + let file = pcm_mkv(1_000_000_000, true, 0); + let dec = Decoder::from_bytes(&file, None, &DecodeOptions::default()).unwrap(); + assert!(dec.length_known()); + assert_eq!(dec.info().frames, 4800); + let decoded: usize = dec.probe().map(|i| i.frames as usize).unwrap(); + assert_eq!(decoded, 4800); +} + +#[test] +fn a_matroska_codec_delay_is_taken_off_the_length() { + use craft_audio::{decode, probe}; + // `probe` reports the length the container gives, before anything is decoded; `decode` + // reports what it presented. 100 ms are decoded and the 20 ms of codec delay dropped, wherever + // the first block is stored. + for first_ns in [-20_000_000, 0, 50_000_000] { + let file = pcm_mkv(first_ns, true, 20_000_000); + let (_, buf) = decode(&file, None).unwrap(); + assert_eq!((probe(&file, None).unwrap().frames, buf.frames()), (3840, 3840), "first block at {first_ns} ns"); + } +} + +#[test] +fn seeking_far_past_an_unknown_end_does_not_overflow() { + use craft_audio::{DecodeOptions, Decoder}; + let file = pcm_mkv(0, false, 1_000_000); + let mut dec = Decoder::from_bytes(&file, None, &DecodeOptions::default()).unwrap(); + assert!(!dec.length_known()); + dec.seek(u64::MAX).unwrap(); + assert!(dec.next_block().unwrap().is_none()); + // and back to the start + assert_eq!(dec.seek(0).unwrap(), 0); + assert!(dec.next_block().unwrap().is_some_and(|b| b.frames() > 0)); +} diff --git a/crates/audio/tests/audio/wav_empty_data.rs b/crates/audio/tests/audio/wav_empty_data.rs new file mode 100644 index 00000000..6a8a2c58 --- /dev/null +++ b/crates/audio/tests/audio/wav_empty_data.rs @@ -0,0 +1,87 @@ +//! A WAV whose data chunk is empty and followed by metadata decodes to no audio; a recording +//! that was never finalised (RIFF and data sizes left at 0) still yields what was written (#466). + +use craft_audio::{decode, probe}; + +fn fmt16(channels: u16, sr: u32) -> Vec { + let align = channels * 2; + let mut f = Vec::new(); + f.extend_from_slice(&1u16.to_le_bytes()); + f.extend_from_slice(&channels.to_le_bytes()); + f.extend_from_slice(&sr.to_le_bytes()); + f.extend_from_slice(&(sr * u32::from(align)).to_le_bytes()); + f.extend_from_slice(&align.to_le_bytes()); + f.extend_from_slice(&16u16.to_le_bytes()); + f +} + +fn chunk(id: &[u8; 4], data: &[u8]) -> Vec { + let mut c = id.to_vec(); + c.extend_from_slice(&(data.len() as u32).to_le_bytes()); + c.extend_from_slice(data); + if data.len() % 2 == 1 { + c.push(0); + } + c +} + +/// A finished file: correct RIFF size, chunks as given. +fn riff(chunks: &[Vec]) -> Vec { + let mut body = b"WAVE".to_vec(); + for c in chunks { + body.extend_from_slice(c); + } + let mut out = b"RIFF".to_vec(); + out.extend_from_slice(&(body.len() as u32).to_le_bytes()); + out.extend_from_slice(&body); + out +} + +fn list_info() -> Vec { + let mut l = b"INFO".to_vec(); + l.extend_from_slice(&chunk(b"INAM", b"Silence")); + chunk(b"LIST", &l) +} + +#[test] +fn an_empty_data_chunk_followed_by_metadata_has_no_frames() { + let file = riff(&[chunk(b"fmt ", &fmt16(2, 48_000)), chunk(b"data", &[]), list_info(), chunk(b"bext", &[0u8; 602])]); + let info = probe(&file, Some("wav")).unwrap(); + assert_eq!(info.frames, 0, "the LIST and bext chunks are not samples"); + let (info, buf) = decode(&file, Some("wav")).unwrap(); + assert_eq!((info.frames, buf.frames()), (0, 0)); + // The same with the metadata first. + let file = riff(&[chunk(b"fmt ", &fmt16(1, 44_100)), list_info(), chunk(b"data", &[])]); + assert_eq!(probe(&file, Some("wav")).unwrap().frames, 0); + // And with a single trailing chunk of odd size and no pad byte at the very end. + let mut file = riff(&[chunk(b"fmt ", &fmt16(1, 44_100)), chunk(b"data", &[]), chunk(b"note", b"abc")]); + file.pop(); + let riff_size = (file.len() - 8) as u32; + file[4..8].copy_from_slice(&riff_size.to_le_bytes()); + assert_eq!(probe(&file, Some("wav")).unwrap().frames, 0); +} + +#[test] +fn a_recording_that_was_never_finalised_still_gives_its_audio() { + let samples: Vec = (0..400i16).flat_map(|i| (i * 40).to_le_bytes()).collect(); + // RIFF size and data size both left at 0, as a recorder writes them until it finishes. + let mut file = riff(&[chunk(b"fmt ", &fmt16(2, 48_000)), chunk(b"data", &[])]); + file.extend_from_slice(&samples); + file[4..8].copy_from_slice(&0u32.to_le_bytes()); + let (info, buf) = decode(&file, Some("wav")).unwrap(); + assert_eq!(info.frames, 200); + assert_eq!(buf.frames(), 200); + // Even if the RIFF size is the file's, audio that is not a run of chunks is still audio. + let mut file = riff(&[chunk(b"fmt ", &fmt16(2, 48_000)), chunk(b"data", &[])]); + file.extend_from_slice(&samples); + let riff_size = (file.len() - 8) as u32; + file[4..8].copy_from_slice(&riff_size.to_le_bytes()); + assert_eq!(probe(&file, Some("wav")).unwrap().frames, 200); +} + +#[test] +fn a_data_chunk_with_audio_is_unchanged() { + let samples: Vec = (0..100i16).flat_map(|i| i.to_le_bytes()).collect(); + let file = riff(&[chunk(b"fmt ", &fmt16(1, 44_100)), chunk(b"data", &samples), list_info()]); + assert_eq!(probe(&file, Some("wav")).unwrap().frames, 100); +} diff --git a/crates/audio/tests/codecs/main.rs b/crates/audio/tests/codecs/main.rs new file mode 100644 index 00000000..5fddafb2 --- /dev/null +++ b/crates/audio/tests/codecs/main.rs @@ -0,0 +1,10 @@ +//! Tests of the codecs ported from FilmCraft, built as one test binary: each file is a module, as +//! in their original crates. +// Index loops follow the specifications, as in FilmCraft (which allows this lint workspace-wide). +#![allow(clippy::needless_range_loop)] + +use craft_media_testkit as common; +#[macro_use] +mod testkit; + +mod routing_oracle; diff --git a/crates/audio/tests/codecs/routing_oracle.rs b/crates/audio/tests/codecs/routing_oracle.rs new file mode 100644 index 00000000..6a01b9aa --- /dev/null +++ b/crates/audio/tests/codecs/routing_oracle.rs @@ -0,0 +1,320 @@ +//! `craft_audio::decode` routes each container and codec below to the home-grown demuxers and +//! decoders, or to symphonia (FLAC): each pair decodes, sample-aligned, to its presentation +//! length, within an SNR of ffmpeg's decode; truncated and corrupted copies never panic. Fixtures are written by ffmpeg +//! (skipped without it). + +use craft_audio::{ChannelLayout, CodecId, Container, decode, probe}; +use std::path::{Path, PathBuf}; +use std::process::Command; + +/// Five seconds of two tones and a little noise per channel. +const STEREO: &str = + "aevalsrc=0.4*sin(2*PI*220*t)+0.15*sin(2*PI*1375*t)+0.05*(random(0)-0.5)|0.35*sin(2*PI*330*t)+0.2*sin(2*PI*2750*t*(1+t/20))+0.05*(random(1)-0.5)"; +const SURROUND_EXTRA: &str = "|0.3*sin(2*PI*440*t)|0.1*sin(2*PI*60*t)|0.2*sin(2*PI*880*t)|0.2*sin(2*PI*990*t)"; +const MONO: &str = "aevalsrc=0.4*sin(2*PI*220*t)"; + +struct Case { + name: &'static str, + surround: bool, + mono: bool, + rate: u32, + args: &'static [&'static str], + container: Container, + codec: CodecId, + /// The speakers of the decoded channels (WAV order). + layout: ChannelLayout, + channels: usize, + /// Presented frames, or `None` for a raw stream (as many as ffmpeg decodes). + frames: Option, + /// Minimum SNR against ffmpeg. + min_snr: f64, +} + +const CASES: &[Case] = &[ + Case { + name: "flac.flac", + surround: false, + mono: false, + rate: 48_000, + args: &["-c:a", "flac", "-sample_fmt", "s16"], + container: Container::Flac, + codec: CodecId::Flac, + layout: ChannelLayout::STEREO, + channels: 2, + frames: Some(240_000), + min_snr: 140.0, + }, + Case { + name: "pcm-s24be.mov", + surround: false, + mono: false, + rate: 48_000, + args: &["-c:a", "pcm_s24be"], + container: Container::QuickTime, + codec: CodecId::Pcm, + layout: ChannelLayout::STEREO, + channels: 2, + frames: Some(240_000), + min_snr: 140.0, + }, + Case { + name: "pcm-f32.mkv", + surround: false, + mono: false, + rate: 48_000, + args: &["-c:a", "pcm_f32le"], + container: Container::Matroska, + codec: CodecId::Pcm, + layout: ChannelLayout::STEREO, + channels: 2, + frames: Some(240_000), + min_snr: 140.0, + }, +]; + +fn run(ff: &Path, args: &[&str]) -> bool { + Command::new(ff).args(["-v", "error", "-y"]).args(args).output().is_ok_and(|o| o.status.success()) +} + +/// The fixture for `c`, written by ffmpeg (cached), or `None` when this ffmpeg lacks an encoder. +fn fixture(ff: &Path, c: &Case) -> Option { + let out = crate::testkit::fixtures_dir("routing").join(c.name); + let source = match (c.mono, c.surround) { + (true, _) => MONO.to_string(), + (_, true) => format!("{STEREO}{SURROUND_EXTRA}"), + _ => STEREO.to_string(), + }; + let input = format!("{source}:s={}", c.rate); + crate::testkit::fixtures::generate(&out, |tmp| { + let tmp = tmp.to_string_lossy(); + let mut args = vec!["-f", "lavfi", "-i", &input, "-t", "5"]; + args.extend_from_slice(c.args); + args.push(&tmp); + run(ff, &args) + }) +} + +/// ffmpeg's decode of `file` as interleaved f32. +fn reference(ff: &Path, file: &Path) -> Vec { + let out = Command::new(ff).args(["-v", "error", "-i"]).arg(file).args(["-f", "f32le", "-c:a", "pcm_f32le", "-"]).output().unwrap(); + assert!(out.status.success(), "ffmpeg decode of {}", file.display()); + out.stdout.as_chunks::<4>().0.iter().map(|c| f32::from_le_bytes(*c)).collect() +} + +/// SNR (dB) of `ours` (planar) against `reference` (interleaved), sample-aligned, over the +/// frames both hold. +fn snr(ours: &[Vec], reference: &[f32]) -> f64 { + let ch = ours.len(); + let n = ours.iter().map(Vec::len).min().unwrap_or(0).min(reference.len() / ch); + let (mut s, mut e) = (0f64, 0f64); + for i in 0..n { + for (c, plane) in ours.iter().enumerate() { + let r = f64::from(reference[i * ch + c]); + let d = f64::from(plane[i]) - r; + s += r * r; + e += d * d; + } + } + if e == 0.0 { f64::INFINITY } else { 10.0 * (s / e).log10() } +} + +#[test] +fn every_container_and_codec_decodes_home_grown() { + let ff = require_ffmpeg!(); + for c in CASES { + let Some(path) = fixture(&ff, c) else { + eprintln!("SKIPPED {}: this ffmpeg cannot write it", c.name); + continue; + }; + let bytes = std::fs::read(&path).unwrap(); + let (info, buf) = decode(&bytes, None).unwrap_or_else(|e| panic!("{}: {e}", c.name)); + let reference = reference(&ff, &path); + assert_eq!((info.container, info.codec, info.sample_rate, buf.channels.len()), (c.container, c.codec, c.rate, c.channels), "{}", c.name); + assert_eq!(info.layout, c.layout, "{}: layout", c.name); + let expected = c.frames.unwrap_or(reference.len() / c.channels); + assert_eq!(buf.frames(), expected, "{}: presented frames", c.name); + let s = snr(&buf.channels, &reference); + assert!(s >= c.min_snr, "{}: SNR {s:.1} dB < {}", c.name, c.min_snr); + let p = probe(&bytes, None).unwrap(); + assert_eq!( + (p.container, p.codec, p.sample_rate, usize::from(p.channels), p.layout), + (c.container, c.codec, c.rate, c.channels, c.layout), + "{}: probe", + c.name + ); + } +} + +#[test] +fn truncated_and_corrupted_files_never_panic() { + let ff = require_ffmpeg!(); + for c in CASES { + let Some(path) = fixture(&ff, c) else { continue }; + let bytes = std::fs::read(&path).unwrap(); + for cut in [1, 7, 64, 500, bytes.len() / 3, bytes.len() / 2, bytes.len() - 1] { + let _ = decode(bytes.get(..cut).unwrap_or_default(), None); + let _ = probe(bytes.get(..cut).unwrap_or_default(), None); + } + let mut seed = 0x9E37_79B9_7F4A_7C15u64; + for _ in 0..6 { + let mut m = bytes.clone(); + for _ in 0..16 { + seed ^= seed << 13; + seed ^= seed >> 7; + seed ^= seed << 17; + let i = (seed % m.len() as u64) as usize; + m[i] ^= (seed >> 32) as u8 | 1; + } + let _ = decode(&m, None); + } + } +} + +/// SNR (dB) of `ours` against `reference`, both planar, over the frames both hold. +fn snr_planar(ours: &[Vec], reference: &[Vec]) -> f64 { + let (mut s, mut e) = (0f64, 0f64); + for (o, r) in ours.iter().zip(reference) { + for (&a, &b) in o.iter().zip(r) { + s += f64::from(b) * f64::from(b); + e += f64::from(a - b) * f64::from(a - b); + } + } + if e == 0.0 { f64::INFINITY } else { 10.0 * (s / e).log10() } +} + +/// The streaming decoder gives the same audio as `decode`, block by block, and a seek lands on the +/// exact frame: what follows matches `decode` from there (bit for bit for the deterministic +/// codecs; within the codec's bound where the decoder state is reconstructed from a pre-roll). +#[test] +fn streaming_and_seeking_match_the_whole_file_decode() { + use craft_audio::{DecodeOptions, Decoder}; + let ff = require_ffmpeg!(); + for c in CASES { + let Some(path) = fixture(&ff, c) else { continue }; + let bytes = std::fs::read(&path).unwrap(); + let (_, whole) = decode(&bytes, None).unwrap(); + let mut d = Decoder::from_bytes(&bytes, None, &DecodeOptions::default()).unwrap(); + let mut streamed: Vec> = vec![Vec::new(); whole.channels.len()]; + while let Some(b) = d.next_block().unwrap() { + for (s, ch) in streamed.iter_mut().zip(b.channels) { + s.extend(ch); + } + } + assert_eq!(streamed, whole.channels, "{}: streamed blocks", c.name); + + for target in [1, whole.frames() / 3, whole.frames() - 4800] { + assert_eq!(d.seek(target as u64).unwrap(), target as u64, "{}", c.name); + let mut after: Vec> = vec![Vec::new(); whole.channels.len()]; + while let Some(b) = d.next_block().unwrap() { + for (s, ch) in after.iter_mut().zip(b.channels) { + s.extend(ch); + } + } + assert_eq!(after[0].len(), whole.frames() - target, "{}: frames after a seek to {target}", c.name); + let expected: Vec> = whole.channels.iter().map(|ch| ch[target..].to_vec()).collect(); + let s = snr_planar(&after, &expected); + assert!(s == f64::INFINITY, "{}: SNR {s:.1} dB after a seek to {target}", c.name); + } + } +} + +/// `TrackDecoder` on the packets craft-media's demuxer reads gives the audio `decode` presents: +/// the same samples once the container's priming is skipped. +#[test] +fn track_decoder_on_demuxed_packets_matches_decode() { + use craft_audio::TrackDecoder; + use craft_media::isobmff; + let ff = require_ffmpeg!(); + for name in ["pcm-s24be.mov"] { + let c = CASES.iter().find(|c| c.name == name).unwrap(); + let Some(path) = fixture(&ff, c) else { continue }; + let bytes = std::fs::read(&path).unwrap(); + let (info, whole) = decode(&bytes, None).unwrap(); + let file = isobmff::open(bytes.as_slice()).unwrap(); + let track = file.track_of_kind(isobmff::TrackKind::Audio).unwrap(); + let t = &file.tracks[track]; + let params = t.codec_params(); + let packets: Vec> = (0..t.samples.len()).map(|i| file.read_sample(bytes.as_slice(), track, i).unwrap()).collect(); + let lookahead: Vec<&[u8]> = packets.iter().take(8).map(Vec::as_slice).collect(); + let mut dec = TrackDecoder::new(¶ms, &lookahead).unwrap(); + assert_eq!((dec.codec(), dec.sample_rate(), dec.layout()), (info.codec, info.sample_rate, info.layout), "{name}"); + let mut out: Vec> = vec![Vec::new(); usize::from(dec.channels())]; + for p in &packets { + for (o, ch) in out.iter_mut().zip(dec.decode(p).unwrap().channels) { + o.extend(ch); + } + } + // The edit list's media time is the priming `decode` skips. + let skip = t.edits.first().map_or(0, |e| (e.media_time.max(0) as u64 * u64::from(info.sample_rate) / u64::from(t.timescale)) as usize); + let presented: Vec> = out.iter().map(|ch| ch[skip..skip + whole.frames()].to_vec()).collect(); + assert_eq!(presented, whole.channels, "{name}"); + } +} + +/// `tracks` lists every audio track, `DecodeOptions::track` picks one, and `max_samples` refuses +/// a longer decode. +#[test] +fn track_choice_and_the_sample_limit() { + use craft_audio::{AudioError, DecodeOptions, decode_with, tracks}; + let ff = require_ffmpeg!(); + for ext in ["mkv", "mov"] { + let out = crate::testkit::fixtures_dir("routing").join(format!("two-tracks-pcm.{ext}")); + let Some(path) = crate::testkit::fixtures::generate(&out, |tmp| { + let tmp = tmp.to_string_lossy(); + run( + &ff, + &[ + "-f", + "lavfi", + "-i", + "sine=f=440:r=48000:d=2", + "-f", + "lavfi", + "-i", + "aevalsrc=0.3*sin(2*PI*660*t)|0.3*sin(2*PI*880*t):s=44100:d=2", + "-map", + "0:a", + "-map", + "1:a", + "-c:a", + "pcm_s16le", + &tmp, + ], + ) + }) else { + continue; + }; + let bytes = std::fs::read(&path).unwrap(); + let list = tracks(&bytes, None).unwrap(); + assert_eq!(list.len(), 2, "{ext}"); + assert_eq!((list[0].codec, list[0].sample_rate, list[0].channels), (CodecId::Pcm, Some(48_000), Some(1)), "{ext}"); + assert_eq!((list[1].sample_rate, list[1].channels), (Some(44_100), Some(2)), "{ext}"); + let (first, _) = decode_with(&bytes, None, &DecodeOptions::default()).unwrap(); + let (second, buf) = decode_with(&bytes, None, &DecodeOptions::default().with_track(1)).unwrap(); + assert_eq!((first.channels, first.sample_rate), (1, 48_000), "{ext}"); + assert_eq!((second.channels, second.sample_rate, buf.frames()), (2, 44_100, 88_200), "{ext}"); + let limited = decode_with(&bytes, None, &DecodeOptions::default().with_max_samples(1000)); + assert!(matches!(limited, Err(AudioError::TooLarge(_))), "{ext}"); + // Refused from the track list, without reading the file through the fallback. + let missing = decode_with(&bytes, None, &DecodeOptions::default().with_track(2)); + assert!(matches!(&missing, Err(AudioError::Unsupported(m)) if m.contains("no audio track 2")), "{ext}: {missing:?}"); + } +} + +/// `Decoder::open` reads from a file without loading it. +#[test] +fn decoder_reads_from_a_file() { + use craft_audio::{DecodeOptions, Decoder}; + let ff = require_ffmpeg!(); + let c = CASES.iter().find(|c| c.name == "pcm-s24be.mov").unwrap(); + let Some(path) = fixture(&ff, c) else { return }; + let (_, whole) = decode(&std::fs::read(&path).unwrap(), None).unwrap(); + let mut d = Decoder::open(std::fs::File::open(&path).unwrap(), None, &DecodeOptions::default()).unwrap(); + assert!(d.length_known()); + assert_eq!(d.info().frames, whole.frames() as u64); + let mut frames = 0; + while let Some(b) = d.next_block().unwrap() { + frames += b.frames(); + } + assert_eq!(frames, whole.frames()); +} diff --git a/crates/audio/tests/codecs/testkit.rs b/crates/audio/tests/codecs/testkit.rs new file mode 100644 index 00000000..91b7f2ae --- /dev/null +++ b/crates/audio/tests/codecs/testkit.rs @@ -0,0 +1,61 @@ +//! The parts of FilmCraft's `filmcraft-testkit` that the ported codec tests use, on craft-media's +//! oracle rules (`tests/common`): `CRAFT_MEDIA_FFMPEG`, then `PATH`; a missing tool skips unless +//! `CRAFT_MEDIA_REQUIRE_ORACLES=1`; fixtures go to `$CRAFT_MEDIA_FIXTURES_DIR` or +//! `target/fixtures`, never committed. + +use crate::common; +use std::path::{Path, PathBuf}; + +/// `let ff = require_ffmpeg!();`: the ffmpeg path, or return from the test after printing +/// `SKIPPED` (a failure with `CRAFT_MEDIA_REQUIRE_ORACLES=1`). +macro_rules! require_ffmpeg { + () => { + match $crate::testkit::ffmpeg_or_skip(concat!(module_path!(), ":", line!())) { + Some(p) => p, + None => return, + } + }; + ($ret:expr) => { + match $crate::testkit::ffmpeg_or_skip(concat!(module_path!(), ":", line!())) { + Some(p) => p, + None => return $ret, + } + }; +} + +/// ffmpeg, or `None` after printing `SKIPPED` for `context`. +pub fn ffmpeg_or_skip(context: &str) -> Option { + common::tool("ffmpeg").or_else(|| common::skip(&format!("{context}: ffmpeg not found (set CRAFT_MEDIA_FFMPEG or put ffmpeg on PATH)"))) +} + +/// `/` (`sub` may hold `/`), created if missing. +pub fn fixtures_dir(sub: &str) -> PathBuf { + let mut d = common::fixture_subdir("codecs"); + for part in sub.split('/').filter(|p| !p.is_empty()) { + d.push(part); + } + std::fs::create_dir_all(&d).unwrap_or_else(|e| panic!("create {}: {e}", d.display())); + d +} + +pub mod fixtures { + use super::*; + + /// `out` if it already exists (non-empty); otherwise `make(tmp)` on a temporary sibling, renamed + /// into place. `None` when `make` fails. + pub fn generate(out: &Path, make: impl FnOnce(&Path) -> bool) -> Option { + if std::fs::metadata(out).is_ok_and(|m| m.len() > 0) { + return Some(out.to_path_buf()); + } + if let Some(dir) = out.parent() { + let _ = std::fs::create_dir_all(dir); + } + let tmp = common::temp_path(out); + if !make(&tmp) || !tmp.exists() { + let _ = std::fs::remove_file(&tmp); + return None; + } + std::fs::rename(&tmp, out).ok()?; + Some(out.to_path_buf()) + } +} diff --git a/crates/media/README.md b/crates/media/README.md index 2785debb..fcf2db62 100644 --- a/crates/media/README.md +++ b/crates/media/README.md @@ -5,8 +5,8 @@ clean-room ISO BMFF (MP4), QuickTime (MOV) and Matroska (MKV, WebM) demuxers and demuxer, written from the public specifications. No `unsafe`, no panics on malformed input, builds for `wasm32-unknown-unknown`. Dependencies: `thiserror` (for `isobmff`), `serde` (optional). -Codecs go in separate crates that depend on this one: `craft-audio` for audio (in a follow-up), -and a video codec crate later. What both need lives here: the containers, the +Codecs are in separate crates that depend on this one: [`craft-audio`](../audio/README.md) for +audio; video codecs will go in a separate crate. What both need lives here: the containers, the bit I/O, one `ByteSource` trait, and the [shared identifiers](#containers-codecs-and-codec-parameters) that codec crates use to describe a file and configure a decoder.