pauljd/craft-libs
1
0
Fork 0
mirror of https://github.com/storytold/craft-libs.git synced 2026-11-12 00:42:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

craft-audio: AAC encoder (6/10) (#444)

Browse Source 
* craft-audio: AAC decoder

FilmCraft's `filmcraft-aac` decoder as `craft_audio::aac` behind
`aac-decode`, kept over symphonia's: the same quality (about 135 dB
SNR against ffmpeg), and it decodes 5.1, which symphonia refuses.
HE-AAC decodes as its AAC-LC core, upsampled 2x to the stream's rate.

- `TrackDecoder` decodes AAC, including ADTS-framed packets without
  extradata (MPEG-TS, raw `.aac`).
- The native read path takes AAC in MP4 and Matroska and raw ADTS
  streams (frame sync over the file, with resync after damage).
- AAC errors convert into `AudioError`.

The AAC module's shared parts (Huffman, MDCT, TNS, ICS) arrive whole
as ported; their encoder half is unused until the encoder PR.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 1 (#443)

Raw ADTS and TrackDecoder:
- Channel configuration 0 (a PCE in the stream, as ffmpeg's -aac_pce and
  AudioToolbox's 7.x write) decodes; it was refused.
- Frames with several raw data blocks are refused; only their first block
  was decoded, silently losing the rest.
- A frame that changes the sample rate, profile or channel configuration is
  concealed; it played at the wrong speed.
- A source that fails during the frame scan is AudioError::Io; it looked
  like the end of the file.
- A stream after an ID3 tag longer than the file head is found.
- conceal counts all the ADTS frames of a packet, and a packet of more than
  1024 frames is refused before it is decoded into memory.

AAC decoder:
- Indexed channel configurations place elements by arrival order, not tag,
  as ffmpeg does; tagged differently, channels were swapped.
- A PCE in an access unit that fails no longer changes the layout, which
  left every later packet failing.
- Non-finite samples are zeroed before the output and the overlap.

MP4: probe and decode agree: the samples' durations end the track when no
single edit does, and an edit cannot outlast the media.

The routing oracle fails on a fixture ffmpeg cannot write when oracles are
required. The README's Limits names HE-AAC v2 as mono, multi-block ADTS,
iTunSMPB, Matroska's tick precision and AAC in CAF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 2 (#443)

Corrects round 1:
- A multi-block ADTS frame after the stream has started is concealed at its
  whole length; round 1's filter skipped it, shifting the audio after it.
- Sample durations only trim an MP4's last packet: wrong ones (some count a
  HE-AAC packet's core frames) were cutting audio. Durations of 0 with an edit
  use the edit.

Raw ADTS packets now keep their headers, so TrackDecoder alone handles format
changes, multi-block frames and concealment, for apps as for files, and nested
ADTS payloads are not split again. Headers come from the frame scan's reads,
not one read per frame. The stream's format is the one most of its first frames
share, so one damaged header does not silence the file. The ID3 walk and a
packet's ADTS split are bounded.

AAC:
- HE-AAC is delayed by the SBR filterbanks' 481 core samples; it played 19.5 ms
  early against ffmpeg in every container.
- A failed access unit leaves the decoder's state as it was, so the next packet
  decodes as if it had not been there.
- reset restarts the noise generator, so a seek into PNS repeats.
- A signalled rate that is not twice the core's plays the core; it reported the
  signalled rate with the core's samples.

The decoder oracle's tolerance tightens from 1e-3 to 1e-5. Tests cover HE-AAC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 3 (#443)

Corrects round 2:
- ADTS headers are compared with the stream's format only when the first header
  gave it. With extradata the format comes from there, so ADTS in MP4 whose
  headers say something else decodes again; every packet had been concealed.
- A stream's only program config element is found in its first frames even when
  that frame's header is damaged.
- The test of wrong sample durations patched the second run's count, not the
  delta, so "durations of one" changed nothing. It now patches the delta, and
  checks a file was changed and that seeks land on the same audio.

MP4 seeks place a packet by its index when every packet decodes to the same
length, as decoding does; they used the durations round 2 had stopped trusting
for the length, and returned the wrong audio. A last delta of 0 is overridden too.

Docs: what TrackInfo says for raw ADTS, that opening one reads the whole file, and
which codecs symphonia reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 4 (#443)

- TrackDecoder takes an ADTS stream's format from the one most of the lookahead
  shares, as the file path did, so one damaged first header does not fail every
  packet of an MPEG-TS stream. open_adts now just asks it.
- MP4 samples of several ADTS frames keep the sample table's timing; a packet
  count of one frame each truncated them silently.
- Sample durations past what the packets decode to no longer over-report the
  length.
- split_adts_upto is not behind aac-decode, as split_adts is not: an aac-encode
  build in the next PR did not compile.

Removes an unreachable ID_END check and open_adts's unused core rate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 5 (#443)

No change in decoding. An aac-encode build in the next PR needs the
split_adts_upto re-export behind aac-decode, and ID_END only in tests.

Docs say the lookahead's headers give an ADTS stream's configuration, that AAC in
CAF still goes through symphonia, and that ProgramConfig is not #[non_exhaustive].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 6 (#443)

Two tests from round 1 failed on ffmpeg before 9, including CI's 8.0. Neither was
a decoding fault:
- ffmpeg 9 ends an MP4 with no edit list at its sample durations and earlier
  versions present the whole last packet, so the length is checked against the
  file's known duration, not ffmpeg's decode.
- The order ffmpeg gives channels with -aac_pce differs by version, so each decoded
  channel must match a different one of ffmpeg's, whatever the order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 7 (#443)

Reading:
- an element of another type or a tag a program config does not list is
  refused, not written to another channel; only an LFE coded as an SCE
  is still allowed, as ffmpeg allows it
- an explicit sampling rate of 0 is refused
- a packet's later ADTS frames are searched for SBR
- a multi-frame ADTS packet that fails leaves the decoder as it was, and
  one whose frames change the channel count is an error
- the whole ADTS frames before a damaged end of a packet decode
- a failed read of a raw stream's first frames is an I/O error
- an MP4 sample of several ADTS frames past the lookahead keeps its length
- an Ogg file with leading bytes keeps its track order and mask comments

Writing:
- ProgramConfig::write, AudioSpecificConfig::to_bytes and
  AdtsHeader::write refuse what their fields cannot hold

Tests:
- the routing test checks one alignment for the ffmpeg version under test
- a default decode past an Ogg stream symphonia has no track for

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 8 (#443)

From wanikhawar's review:
- AudioSpecificConfig::parse reads backward-compatible SBR and PS signalling
  (a 0x2b7 sync extension after the config), which craft-media's Matroska
  demuxer writes for legacy HE-AAC IDs. Without lookahead those streams decoded
  as their core.
- An MP4 media edit after one empty edit is applied: its priming skip and its
  end. The empty edit's delay is not added as silence.
- A packet lost whole is concealed as long as the last one decoded, so a stream
  of multi-frame ADTS packets keeps its timing.
- ADTS-framed MP4 samples keep their sample table's timing, instead of every
  sample being read at open to look for several frames in one.
- AdtsHeader::write refuses an object type, sampling frequency index or channel
  configuration its fields cannot hold; it masked them into other values.
- AudioSpecificConfig::lc no longer panics for a sampling frequency index with no
  rate, and the decoder refuses such a config.
- AudioSpecificConfig::to_bytes refuses channel configuration 0 without a program
  config element, which would not parse back.
- Upsample2x starts each channel's history afresh when the channel count
  changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 9 (#443)

From CodeRabbit's review of round 8:
- AudioSpecificConfig::parse consumes extensionFlag3 after the config when
  extensionFlag is set, so the 0x2b7 sync extension after it is found.
- AudioSpecificConfig::to_bytes refuses a sampling rate of 0, which would not
  parse back.
- ADTS-framed HE-AAC in MP4 whose sample durations count core frames is not cut
  to the durations' half length: durations short of one frame per sample no
  longer end an ADTS-framed track, and its length is unknown until decoded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 10 (#443)

ADTS-framed AAC in MP4: durations short of the frames in the samples read
ahead no longer end the track. With HE-AAC samples of two ADTS frames each,
timed as two core frames, the durations matched one output frame per sample,
so the undercount check passed and the track stopped at half its length,
which probe reported as known. The frames each read-ahead sample holds are
now counted, and durations short of them leave the length unknown until
decoded, as durations short of one frame per sample already did. Correctly
timed AAC-LC samples of several frames keep their end trim.

Reported by FredApps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 11 (#443)

Corrects round 10. A short ADTS-framed MP4 track, whose last sample is in
the read-ahead, lost that sample's trim when it cut more than one frame:
the trim counted as durations falling short, so the length was left to
decoding and its padding was presented. The track's last sample is now left
out of the read-ahead comparison.

Reported by CodeRabbit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: document the ADTS-in-MP4 seek and edit limits (#443)

AAC in MP4 stored as ADTS frames, with sample durations that count fewer
frames than the samples hold, decodes in full from the start, but seeks
place samples by those durations and land later than asked, and an edit
list measured in them can end the track early. The README's Limits section
now says so, and the seek example points to it.

Reported by FredApps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: say why the ADTS-in-MP4 seek limit is kept (#443)

The Limits entry from 79f5839 said what goes wrong but not why it is left.
It now says that such a file is outside the MP4 specification: an MP4 AAC
sample is one raw access unit with the configuration in the sample
description (ISO/IEC 14496-14, 14496-3), so ADTS headers do not belong in
it, and a sample's duration is the time it decodes to (ISO/IEC 14496-12).
It says why files within the specification are not affected, what still
works, and why seeks and edits in such files are not corrected: finding
where each sample's audio starts means counting the ADTS frames in every
sample before it, one read per sample, which would slow every ADTS-framed
MP4 at open to work around files outside the specification.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: AAC encoder

FilmCraft's AAC-LC encoder behind `aac-encode`: psychoacoustic model,
window switching, M/S, TNS, bit reservoir, CBR and VBR, 1-8 channels,
optional ADTS framing. `rayon` encodes channels in parallel.
`aac::aac_order` maps WAV order to the encoder's input order. Settings
the encoder cannot use are `aac::Error::InvalidSettings`, which
converts to `AudioError::Encode`. `encode` now turns on `aac-encode`.

Tests that need AAC streams written by our own encoder come with it:
the encoder oracle and robustness tests, packet concealment, and
`TrackDecoder` on ADTS without extradata.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 1 (#444)

From CodeRabbit's first review of the AAC encoder:

- CBR bitrate limits use the exact bits per frame (bitrate * 1024
  against channels * sample_rate) instead of the truncated quotient, so
  288001 b/s at 48 kHz mono is rejected.
- Rates below 48 bits per channel per frame are rejected. That is the
  largest silent frame per channel measured across 1-8 channels; below
  it the encoder overshot the requested rate (96 kHz mono at 1000 b/s
  came out near 4500 b/s).
- Fill elements are capped so the byte-aligned access unit stays within
  6144 bits per channel. write_fill can write 8 bits over its request,
  which gave 769-byte mono units at the maximum rate and when the
  reservoir was flushed on the last frame.
- encode() buffers one frame of input at a time. Appending the whole
  slice and draining 1024 samples per frame copied the rest of the
  input once per frame, quadratic in the slice length. Output is
  byte-identical to feeding the input in chunks.
- README: 8 channels are taken in channel configuration 7's element
  order; a WAV 7.1 buffer has no equivalent there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 2 (#444)

From CodeRabbit's review of round 1:

- The CBR minimum adds the program config's bits when ADTS is on. The
  7-channel layout repeats it in every ADTS frame, and at full
  bandwidth a silent frame took 344 bits against the 336 bits that
  31,500 b/s gives at 96 kHz. Bandwidth alone stays within 48 bits
  per channel.
- write_fill never writes more than it is asked for: each element's
  size comes from the bits left, so the total ends 0-6 bits short.
  The old version could pass the request by 8 bits and could also
  stop early (asked for 14, it wrote 7).
- The encoder fills to the unit's target end, capped at
  max_frame_bits, instead of keeping round 1's 8-bit margin. Frames at
  the maximum rate were 767 bytes when there was little to code; they
  are now exactly 768 bytes per channel.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 3 (#444)

Encoder::new rejects a NaN VBR quality, the known issue ported from
FilmCraft and listed in the PR description.

NaN passes f32::clamp, so the rate-distortion multiplier became NaN.
The frame-size loop compares it with `<` and never ran, and no band
was skipped, so every frame was coded at full precision: mono units
reached 2465 bytes against the 768-byte limit, and 8-channel units of
up to 17024 bytes did not fit the ADTS length field, so they went out
without headers. Infinities and other out-of-range values are clamped
and stay within the limit; the test checks both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* craft-audio: review fixes, round 4 (#444)

From CodeRabbit's review of round 3:

- The CBR fill target is rounded down to a whole byte. Round 2 filled
  to an unaligned target, so after byte alignment a unit could end up
  to 7 bits past the bits it had: in a sweep of silent CBR encodes,
  1188 of 13200 frames overspent, all of them frames with fill, hidden
  by the level clamp at 0. The unit now ends exactly at the rounded
  target. The 0-7 bits rounded off stay in the reservoir and are left
  unspent after the last frame; the comment says so instead of
  claiming an exact total.
- The NaN-quality test checks that infinities and out-of-range values
  encode exactly as the nearest end of the 0.5-6.0 range.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Aleksander Helgaker
2026-11-11 07:21:26 -05:00
committed by GitHub
co-authored by Claude Opus 5.5
parent 139a39e94c
commit ca99f91a18
18 changed files with 2354 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/audio.yml
+1 -1
View File
@@ -39,7 +39,7 @@ jobs:
# Each feature on its own, so code that needs another feature fails here.
- name: Clippy, each feature alone
run: |
for f in "" decode decode-native symphonia encode lossless-encode pcm-encode flac-encode peaks aac-decode serde; do
for f in "" decode decode-native symphonia encode lossless-encode pcm-encode flac-encode peaks aac-decode aac-encode "aac-encode,rayon" serde; do
echo "::group::features: ${f:-none}"
cargo clippy -p craft-audio --all-targets --locked --no-default-features ${f:+--features "$f"} -- -D warnings
echo "::endgroup::"
Cargo.lock
Generated
+1
View File
@@ -818,6 +818,7 @@ dependencies = [
"craft-media-testkit",
"log",
"proptest",
"rayon",
"serde",
"serde_json",
"symphonia",
crates/audio/Cargo.toml
+8 -4
View File
@@ -1,6 +1,6 @@
[package]
name = "craft-audio"
description = "Audio file I/O and codecs: WAV/AIFF read and write, FLAC encoder, AAC decoder."
description = "Audio file I/O and codecs: WAV/AIFF read and write, FLAC encoder, AAC decoder and encoder."
version.workspace = true
edition.workspace = true
license.workspace = true
@@ -22,8 +22,8 @@ decode = ["decode-native", "symphonia"]
decode-native = ["aac-decode", "craft-media/isobmff", "craft-media/matroska", "craft-media/ogg", "dep:log"]
# The symphonia fallback for the formats with no native decoder yet.
symphonia = ["decode-native", "dep:symphonia"]
# Every encoder: the lossless file writers.
encode = ["lossless-encode"]
# Every encoder: the lossless file writers and the AAC encoder.
encode = ["lossless-encode", "aac-encode"]
# Encode WAV/BWF/RF64, AIFF/AIFF-C and FLAC files.
lossless-encode = ["pcm-encode", "flac-encode"]
# Encode WAV/BWF/RF64 and AIFF/AIFF-C files.
@@ -32,8 +32,11 @@ pcm-encode = []
flac-encode = ["craft-media/bitstream"]
# Codec modules on their own, for packet-level use.
aac-decode = ["craft-media/bitstream"]
aac-encode = ["craft-media/bitstream"]
# Multi-resolution waveform overviews (`peaks`).
peaks = []
# Encode AAC channels in parallel (with `aac-encode`).
rayon = ["dep:rayon"]
# Serialize and Deserialize for the shared types.
serde = ["dep:serde", "craft-media/serde"]
@@ -42,6 +45,7 @@ craft-media = { workspace = true }
thiserror = { workspace = true }
serde = { workspace = true, optional = true }
log = { version = "0.4", optional = true }
rayon = { workspace = true, optional = true }
# symphonia keeps its AAC decoder for AAC in CAF, a container craft-media does not read.
symphonia = { version = "0.6.1", optional = true, default-features = false, features = ["mp1", "mp2", "mp3", "flac", "vorbis", "ogg", "wav", "pcm", "aiff", "isomp4", "aac", "alac", "adpcm", "caf", "mkv"] }
@@ -72,4 +76,4 @@ required-features = ["decode"]
[[example]]
name = "encode"
required-features = ["lossless-encode"]
required-features = ["lossless-encode", "aac-encode"]
crates/audio/README.md
+41 -15
View File
@@ -1,7 +1,7 @@
# craft-audio
Audio file I/O and codecs for the Craft apps. Native implementations: WAV and AIFF read/write,
FLAC encoder, AAC decoder.
FLAC encoder, AAC decoder/encoder.
Formats without a native decoder are decoded by [symphonia](#symphonia), which is to be replaced
by native decoders.
@@ -26,12 +26,13 @@ No default features. The shared types (`AudioBuffer`, `AudioInfo`, `ChannelLayou
| `decode` | `decode`, `probe`, `Decoder`, `tracks`, `TrackDecoder`; every format in [Format support](#format-support) | `decode-native`, `symphonia` |
| `decode-native` | the same, native formats only (others are `Unsupported`) | `aac-decode`, craft-media `isobmff`/`matroska`/`ogg`, `log` |
| `symphonia` | the fallback for formats with no native decoder; turning it on is the same as `decode` | `decode-native`, symphonia |
| `encode` | every encoder: the WAV, AIFF and FLAC file writers | `lossless-encode` |
| `encode` | every encoder: the WAV, AIFF and FLAC file writers, and the AAC packet encoder | `lossless-encode`, `aac-encode` |
| `lossless-encode` | WAV, AIFF and FLAC writing | `pcm-encode`, `flac-encode` |
| `pcm-encode` | `encode`, `FileEncoder` for WAV and AIFF; `wav_float_header` | none |
| `flac-encode` | `encode`, `FileEncoder` for FLAC | craft-media `bitstream` |
| `aac-decode` | `aac::Decoder` | craft-media `bitstream` |
| `aac-decode`, `aac-encode` | `aac::Decoder`, `aac::Encoder` | craft-media `bitstream` |
| `peaks` | `peaks::Peaks` | none |
| `rayon` | parallel AAC encoding (with `aac-encode`) | `rayon` |
| `serde` | `Serialize`/`Deserialize` on the shared types | `serde` |
`aac-decode` also gives `TrackDecoder`, for AAC and PCM.
@@ -51,10 +52,10 @@ Decode, every encoder and waveform overviews:
craft-audio = { git = "https://github.com/storytold/craft-libs", rev = "<commit>", features = ["decode", "encode", "peaks"] }
```
Packets from an app's own demuxer, AAC:
Packets from an app's own demuxer, AAC both ways:
```toml
craft-audio = { git = "https://github.com/storytold/craft-libs", rev = "<commit>", features = ["aac-decode"] }
craft-audio = { git = "https://github.com/storytold/craft-libs", rev = "<commit>", features = ["aac-decode", "aac-encode"] }
```
craft-audio re-exports the craft-media types its API uses (`ByteSource`, `ReadSeekSource`,
@@ -73,7 +74,7 @@ cargo run -p craft-audio --example info --features decode -- take.m4a
```
```bash
cargo run -p craft-audio --example encode --features lossless-encode -- out
cargo run -p craft-audio --example encode --features lossless-encode,aac-encode -- out
```
```bash
@@ -216,6 +217,31 @@ features can write. `write` refuses a block without one slice per channel, or wi
shorter than `frames`, and writes nothing. After a write fails for another reason, every later
`write` and `finish` returns an error.
### Encode AAC
```rust
use craft_audio::AudioError;
use craft_audio::aac::{Encoder, EncoderConfig, aac_order};
let mut enc = Encoder::new(EncoderConfig::cbr(48_000, 6, 384_000))?;
let order = aac_order(6).ok_or_else(|| AudioError::Unsupported("no AAC order for 6 channels".into()))?;
let input: Vec<&[f32]> = order.iter().map(|&c| wav_5_1[c].as_slice()).collect();
let mut units = enc.encode(&input); // zero or more access units
units.extend(enc.flush());
let asc = enc.audio_specific_config(); // for the MP4 `esds` box
let priming = enc.priming_samples(); // for the edit list (1024)
```
The sample rate must be one of the 13 AAC rates (7350–96000 Hz). `aac_order` returns `None` for
mono and stereo, which are already in order, and for 8 channels. Eight channels are written as AAC
channel configuration 7 and taken in its element order: C, Lc Rc, L R, Ls Rs, LFE (front centre,
front left and right of centre, front left and right, surrounds). A WAV 7.1 buffer (L R C LFE Lb Rb
Ls Rs) has no equivalent in that layout; passed unchanged, its channels go to the wrong speakers.
`EncoderConfig::cbr(..).with_adts(true)` makes every access unit an ADTS frame; concatenated, they
are a playable `.aac` file. To write MP4 or Matroska, pass the access units to craft-media's
muxers ([MP4](../media/README.md#muxing), [Matroska](../media/README.md#muxing-1)).
### Waveform overviews
```rust
@@ -240,7 +266,7 @@ convert into `AudioError` with `?`.
| `Unsupported` | valid input using an unsupported codec, feature or option, or a format whose feature is off |
| `Malformed` | corrupt or truncated input, or not the claimed format |
| `TooLarge` | over a size limit (`DecodeOptions::max_samples`, an allocation) |
| `Encode` | an encoder rejected the input or its settings (a buffer with no channels or a 0 Hz rate, a block shorter than its frame count, a wrong layout) |
| `Encode` | an encoder rejected the input or its settings (a buffer with no channels or a 0 Hz rate, a block shorter than its frame count, a wrong layout, a rate AAC has no index for) |
| `Io` | the source failed a read, or the `FileEncoder` writer failed |
`AudioError` is `#[non_exhaustive]`, so a `match` needs a `_` arm:
@@ -260,7 +286,7 @@ match craft_audio::decode(&bytes, None) {
| WAV, BWF, RF64/BW64 | native | native |
| AIFF, AIFF-C | native | native |
| FLAC | symphonia | native |
| AAC-LC | native | — |
| AAC-LC | native | native |
| HE-AAC v1/v2 | native, core only (see [Limits](#limits)) | — |
| MP1, MP2, MP3 (MPEG audio Layer I/II/III) | symphonia | — |
| ALAC, Vorbis, ADPCM, A-law, µ-law | symphonia | — |
@@ -343,7 +369,7 @@ Each codec module's rustdoc lists the coding tools it implements.
|---|---|---|---|
| WAV, AIFF | Microsoft RIFF, Apple AIFF/AIFF-C | read: integer 8–32-bit, float 32/64-bit, `WAVE_FORMAT_EXTENSIBLE` masks, RF64/BW64, BWF `bext`, truncated files; write: 16/24/32-bit integer and 32-bit float, RF64 above 4 GiB, any layout, TPDF dither | round trips; hostile-input tests |
| FLAC encoder | RFC 9639 | 16, 24, 32 bits; 1–8 channels, layouts decoders would not assume for the count as a `WAVEFORMATEXTENSIBLE_CHANNEL_MASK` comment; up to 655,350 Hz; LPC, stereo decorrelation, wasted bits, STREAMINFO MD5 | bit-exact through ffmpeg, STREAMINFO MD5 checked against ffmpeg |
| AAC (`aac`) | ISO/IEC 14496-3 | AAC-LC decode, 1–8 channels; HE-AAC: see [Limits](#limits) | ffmpeg: ~135 dB SNR; PNS noise differs per sample at equal band energy |
| AAC (`aac`) | ISO/IEC 14496-3 | AAC-LC decode and encode (CBR, VBR, ADTS), 1–8 channels; HE-AAC: see [Limits](#limits) | ffmpeg: ~135 dB SNR; PNS noise differs per sample at equal band energy |
## symphonia
@@ -414,16 +440,15 @@ For the apps switching from their own code; to be removed once all three have sw
The codec modules' public items keep their names, so apps can re-export them
(`pub use craft_audio::aac::*;`). Names kept only for that are `#[doc(hidden)]`:
`aac::Upsample2x`, `aac::ProgramConfig::parse`, `aac::config_layout` and `aac::sample_rate_index`.
What else an app adjusts:
`aac::Upsample2x`, `aac::ProgramConfig::parse`, `aac::config_layout` and `aac::sample_rate_index`. What else an app adjusts:
- `AudioInfo` has `container`, `codec`, `layout` and `damaged` instead of SoundCraft's `format`.
`detect_container` replaces `detect_format`. `probe` reports the layout instead of
`wav_channel_mask`.
- `EncodeOptions` is built with `EncodeOptions::new(format)` and `with_*` methods. It carries the
speaker layout and RF64, which replace `encode_with_layout`, `encode_wav_rf64` and
`FileEncoder::new`'s layout argument. `BwfInfo` is built the same way. `ENCODE_EXTENSIONS` is a
list.
`FileEncoder::new`'s layout argument. `BwfInfo` and `aac::EncoderConfig` are built the same way
(`with_adts`, …). `ENCODE_EXTENSIONS` is a list.
- Arguments go sample rate, then channels, and channel counts are `u16`:
`FileEncoder::new(out, &opts, sample_rate, channels)`, `wav_float_header(sample_rate,
channels, frames)`, `AudioBuffer::new(sample_rate, channels, frames)`,
@@ -435,8 +460,9 @@ What else an app adjusts:
`Speaker::ALL` is a slice.
- `TrackInfo::sample_rate` and `channels` are `Option`s.
- Features are per codec. SoundCraft's `encode` (WAV, AIFF, FLAC) is now `lossless-encode`, made
of `pcm-encode` (WAV, AIFF) and `flac-encode`. `encode` turns on every encoder. Writing a format
whose feature is off returns `AudioError::Unsupported`.
of `pcm-encode` (WAV, AIFF) and `flac-encode`. `encode` turns on every encoder, AAC
included. Writing a format whose feature is off returns `AudioError::Unsupported`.
- AAC encoder settings it cannot use are `aac::Error::InvalidSettings` (`AudioError::Encode`).
- `FileEncoder::write` takes any per-channel slices, but refuses a block with missing channels or
short slices instead of filling them with silence.
- `DecodeOptions` is not `Copy`.
crates/audio/examples/encode.rs
+8 -2
View File
@@ -1,6 +1,6 @@
//! Write a tone as WAV and FLAC, whole and streamed.
//! Write a tone as WAV and FLAC, whole and streamed, and as AAC packets.
//!
//! `cargo run -p craft-audio --example encode --features lossless-encode -- out`
//! `cargo run -p craft-audio --example encode --features lossless-encode,aac-encode -- out`
use craft_audio::{AudioBuffer, BitDepth, ChannelLayout, EncodeFormat, EncodeOptions, FileEncoder, encode};
@@ -27,5 +27,11 @@ fn main() -> Result<(), Box<dyn std::error::Error>> {
enc.write(&[chunk, chunk], chunk.len())?;
}
enc.finish()?;
// AAC packets for an MP4 or Matroska muxer, with the configuration its sample entry needs.
let mut aac = craft_audio::aac::Encoder::new(craft_audio::aac::EncoderConfig::cbr(rate as u32, 2, 128_000))?;
let mut units = aac.encode(&[&tone, &tone]);
units.extend(aac.flush());
println!("{} AAC access units, AudioSpecificConfig {:02x?}, priming {}", units.len(), aac.audio_specific_config(), aac.priming_samples());
Ok(())
}
crates/audio/src/aac/config.rs
+1
View File
@@ -505,6 +505,7 @@ mod tests {
for sf_index in 13..=15 {
let asc = AudioSpecificConfig::lc(sf_index, 2);
assert_eq!(asc.sample_rate, 0);
#[cfg(feature = "aac-decode")]
assert!(crate::aac::Decoder::from_config(asc).is_err());
}
}
crates/audio/src/aac/encoder/mod.rs
+1067
View File
File diff suppressed because it is too large Load Diff
crates/audio/src/aac/encoder/psy.rs
+176
View File
@@ -0,0 +1,176 @@
//! Psychoacoustic model operating on MDCT coefficients.
//!
//! Per scalefactor band: energy; tonality from the spectral flatness measure (SFM) of a
//! neighbourhood of ≥ 16 lines; masking-threshold PSD by spreading the band PSDs with Schroeder's
//! spreading function in the Bark domain; a tonality-dependent masking offset (≈18 dB for tonal,
//! ≈6 dB for noise-like maskers); the absolute threshold of hearing (Terhardt) as a floor; and
//! perceptual entropy (PE) as an estimate of the bits the frame needs.
use crate::aac::tables::{swb_offsets_long, swb_offsets_short};
/// Precomputed per-band data for one window length.
pub struct BandModel {
pub offsets: &'static [u16],
/// Linear spreading factor `[maskee][masker]` (power).
spread: Vec<Vec<f32>>,
/// ATH as band energy in encoder spectrum units.
ath: Vec<f32>,
/// Neighbourhood `[lo, hi)` in lines used for the tonality estimate.
tonal_span: Vec<(usize, usize)>,
}
fn bark(f: f64) -> f64 {
13.0 * (0.00076 * f).atan() + 3.5 * (f / 7500.0).powi(2).atan()
}
/// Terhardt's threshold in quiet (dB SPL), capped at 40 dB so content near the band edge is not
/// discarded wholesale (the bandwidth setting limits the coded range instead).
fn ath_db(f: f64) -> f64 {
let f = (f / 1000.0).max(0.02);
(3.64 * f.powf(-0.8) - 6.5 * (-0.6 * (f - 3.3).powi(2)).exp() + 1e-3 * f.powi(4)).min(40.0)
}
impl BandModel {
fn new(sample_rate: u32, offsets: &'static [u16], m: usize) -> BandModel {
let nb = offsets.len() - 1;
let hz_per_line = sample_rate as f64 / (2.0 * m as f64);
let centre: Vec<f64> = (0..nb).map(|b| bark((offsets[b] as f64 + offsets[b + 1] as f64) * 0.5 * hz_per_line)).collect();
let spread = (0..nb)
.map(|i| {
(0..nb)
.map(|j| {
let dz = centre[i] - centre[j] + 0.474;
let db = 15.81 + 7.5 * dz - 17.5 * (1.0 + dz * dz).sqrt();
if db < -60.0 { 0.0 } else { 10f64.powf(db / 10.0) as f32 }
})
.collect()
})
.collect();
// A full-scale (16-bit) sine of power P_ref = 32768²/2 has MDCT energy ≈ 2·M²·P per frame
// with the encoder's scaling; it is taken as 96 dB SPL. 3 dB of safety margin.
let p_ref = 32768f64 * 32768.0 / 2.0;
let ath = (0..nb)
.map(|b| {
let mut lo = f64::MAX;
for k in offsets[b]..offsets[b + 1] {
lo = lo.min(ath_db((k as f64 + 0.5) * hz_per_line));
}
(2.0 * (m * m) as f64 * p_ref * 10f64.powf((lo - 96.0 - 3.0) / 10.0)) as f32
})
.collect();
let tonal_span = (0..nb)
.map(|b| {
let (lo, hi) = (offsets[b] as usize, offsets[b + 1] as usize);
if hi - lo >= 16 {
(lo, hi)
} else {
let c = (lo + hi) / 2;
let lo = c.saturating_sub(8);
let hi = (lo + 16).min(m);
(hi.saturating_sub(16), hi)
}
})
.collect();
BandModel { offsets, spread, ath, tonal_span }
}
pub fn num_bands(&self) -> usize {
self.offsets.len() - 1
}
}
pub struct Psy {
pub long: BandModel,
pub short: BandModel,
}
/// Per-window, per-band analysis of one channel.
#[derive(Clone, Default)]
pub struct PsyResult {
/// `[window][band]` energy.
pub en: Vec<Vec<f32>>,
/// `[window][band]` masking threshold (allowed noise energy).
pub thr: Vec<Vec<f32>>,
pub pe: f32,
}
impl Psy {
pub fn new(sample_rate: u32, sf_index: u8) -> Psy {
Psy {
long: BandModel::new(sample_rate, swb_offsets_long(sf_index), 1024),
short: BandModel::new(sample_rate, swb_offsets_short(sf_index), 128),
}
}
/// Analyse a spectrum: 1024 lines (long) or 8×128 (short).
pub fn analyse(&self, spec: &[f32], short: bool) -> PsyResult {
let (model, nwin, wlen) = if short { (&self.short, 8, 128) } else { (&self.long, 1, 1024) };
let nb = model.num_bands();
let mut res = PsyResult { en: Vec::with_capacity(nwin), thr: Vec::with_capacity(nwin), pe: 0.0 };
let mut psd = vec![0f32; nb];
let mut tonality = vec![0f32; nb];
for w in 0..nwin {
let s = &spec[w * wlen..(w + 1) * wlen];
let mut en = vec![0f32; nb];
for b in 0..nb {
let (lo, hi) = (model.offsets[b] as usize, model.offsets[b + 1] as usize);
en[b] = s[lo..hi].iter().map(|v| v * v).sum();
psd[b] = en[b] / (hi - lo) as f32;
// spectral flatness over the neighbourhood
let (tlo, thi) = model.tonal_span[b];
let n = (thi - tlo) as f64;
let mut sum = 0f64;
let mut log_sum = 0f64;
for &v in &s[tlo..thi] {
let p = v as f64 * v as f64 + 1e-3;
sum += p;
log_sum += p.ln();
}
let sfm_db = 10.0 * ((log_sum / n).exp() / (sum / n)).log10();
// real-valued Gaussian noise gives ≈ -5.5 dB; a pure tone ≤ -25 dB
tonality[b] = ((-sfm_db - 6.0) / 19.0).clamp(0.0, 1.0) as f32;
}
let mut thr = vec![0f32; nb];
for b in 0..nb {
let row = &model.spread[b];
let mut sp = 0f32;
for j in 0..nb {
sp += psd[j] * row[j];
}
let width = (model.offsets[b + 1] - model.offsets[b]) as f32;
let offset_db = tonality[b] * 30.0 + (1.0 - tonality[b]) * 6.0;
let t = sp * width * 10f32.powf(-offset_db / 10.0);
thr[b] = t.max(model.ath[b]);
if en[b] > thr[b] {
res.pe += width * 0.5 * (en[b] / thr[b]).log2();
}
}
res.en.push(en);
res.thr.push(thr);
}
res
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn tone_is_masked_less_than_noise() {
let psy = Psy::new(44100, 4);
let tone: Vec<f32> = (0..1024).map(|k| if k == 100 { 1e6 } else { 0.0 }).collect();
let mut seed = 1u32;
let noise: Vec<f32> = (0..1024)
.map(|_| {
seed = seed.wrapping_mul(1664525).wrapping_add(1013904223);
((seed >> 8) as f32 / (1 << 24) as f32 - 0.5) * 1e4
})
.collect();
let rt = psy.analyse(&tone, false);
let rn = psy.analyse(&noise, false);
let b = psy.long.offsets.iter().position(|&o| o > 100).unwrap() - 1;
assert!(rt.thr[0][b] < rt.en[0][b] / 30.0);
assert!(rn.thr[0][b] > rn.en[0][b] / 10.0);
}
}
crates/audio/src/aac/encoder/quant.rs
+383
View File
@@ -0,0 +1,383 @@
//! Quantisation, scalefactor selection, codebook/section selection, bit counting and writing of one
//! `individual_channel_stream`.
//!
//! Coefficients are kept in bitstream order ("interleaved": group → band → window → line), so every
//! (group, band) is a contiguous slice and short-window grouping needs no special casing.
use craft_media::bitstream::BitWriter;
use crate::aac::huffman::{self, ZERO_HCB};
use crate::aac::ics::IcsInfo;
use crate::aac::tables::pow43;
use crate::aac::tns::TnsData;
/// Magic rounding offset for AAC quantisation (`0.5 - 0.0946`), minimising the error for the
/// `x^(3/4)` companding.
const ROUND: f32 = 0.4054;
const MAX_Q: i32 = 8191;
/// One (group, band) of a channel, in bitstream order.
#[derive(Clone, Copy, Debug)]
pub struct Band {
/// Range in the interleaved coefficient array.
pub start: usize,
pub end: usize,
/// Allowed noise energy at λ = 1.
pub thr: f32,
/// Coefficient energy.
pub en: f32,
/// Largest `|x|^(3/4)`.
pub max34: f32,
}
/// Everything the rate loop needs for one channel.
pub struct ChannelInput {
pub info: IcsInfo,
/// Interleaved (bitstream-order) coefficients.
pub x: Vec<f32>,
pub xabs: Vec<f32>,
pub x34: Vec<f32>,
pub bands: Vec<Band>,
pub tns: Option<TnsData>,
}
impl ChannelInput {
/// `spec` is window-major (short windows at stride 128); `thr[g][sfb]` and the grouping come from
/// `info`.
pub fn new(info: IcsInfo, swb: &[u16], spec: &[f32], thr: &[Vec<f32>], tns: Option<TnsData>) -> ChannelInput {
let mut x = Vec::with_capacity(1024);
let mut bands = Vec::new();
for g in 0..info.num_groups {
let w0 = info.group_start(g);
for sfb in 0..info.max_sfb {
let start = x.len();
for w in w0..w0 + info.group_len[g] as usize {
x.extend_from_slice(&spec[w * 128 + swb[sfb] as usize..w * 128 + swb[sfb + 1] as usize]);
}
bands.push(Band { start, end: x.len(), thr: thr[g][sfb], en: 0.0, max34: 0.0 });
}
}
let xabs: Vec<f32> = x.iter().map(|v| v.abs()).collect();
let x34: Vec<f32> = xabs.iter().map(|v| v.powf(0.75)).collect();
for b in &mut bands {
b.en = x[b.start..b.end].iter().map(|v| v * v).sum();
b.max34 = x34[b.start..b.end].iter().cloned().fold(0.0, f32::max);
}
ChannelInput { info, x, xabs, x34, bands, tns }
}
}
/// A quantised channel ready to be written.
#[derive(Clone, Default)]
pub struct QuantChannel {
pub q: Vec<i32>,
/// Scalefactor per band (only meaningful where `cb != 0`).
pub sf: Vec<i32>,
pub cb: Vec<u8>,
/// Per group: (codebook, number of bands).
pub sections: Vec<Vec<(u8, usize)>>,
pub global_gain: i32,
/// Bits of the ICS excluding `ics_info`.
pub bits: u32,
}
#[inline]
fn gain34(sf: i32) -> f32 {
(-0.1875 * (sf as f32 - 100.0)).exp2()
}
#[inline]
fn gain(sf: i32) -> f32 {
(0.25 * (sf as f32 - 100.0)).exp2()
}
/// Quantise a slice at `sf`, returning the distortion; writes the (signed) values to `out` if given.
fn quantise(x: &[f32], xabs: &[f32], x34: &[f32], sf: i32, out: Option<&mut [i32]>) -> f32 {
let g34 = gain34(sf);
let g = gain(sf);
let mut d = 0f32;
match out {
Some(out) => {
for i in 0..x.len() {
let q = ((x34[i] * g34 + ROUND) as i32).min(MAX_Q);
let e = xabs[i] - pow43(q as u32) * g;
d += e * e;
out[i] = if x[i] < 0.0 { -q } else { q };
}
}
None => {
for i in 0..x.len() {
let q = ((x34[i] * g34 + ROUND) as i32).min(MAX_Q);
let e = xabs[i] - pow43(q as u32) * g;
d += e * e;
}
}
}
d
}
/// Largest scalefactor whose distortion stays within `allowed`.
fn search_sf(ch: &ChannelInput, b: &Band, allowed: f32) -> i32 {
let r = b.start..b.end;
let (x, xa, x34) = (&ch.x[r.clone()], &ch.xabs[r.clone()], &ch.x34[r]);
let l2 = b.max34.log2();
let lo = ((100.0 - ((MAX_Q as f32 + 0.5).log2() - l2) / 0.1875).ceil() as i32).clamp(0, 255);
let zero = ((100.0 - ((1.0 - ROUND).log2() - l2) / 0.1875).floor() as i32 + 1).clamp(0, 256);
let mut a = lo;
let mut hi = (zero - 1).clamp(lo, 255);
if quantise(x, xa, x34, a, None) > allowed {
return a;
}
while a < hi {
let mid = (a + hi + 1) / 2;
if quantise(x, xa, x34, mid, None) <= allowed {
a = mid;
} else {
hi = mid - 1;
}
}
a
}
/// Candidate codebooks for a band whose largest magnitude is `m` (0 for all-zero bands).
fn candidates(m: i32) -> &'static [u8] {
match m {
0 => &[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
1 => &[1, 2, 3, 4, 5, 6],
2 => &[3, 4, 5, 6, 7, 8],
3..=4 => &[5, 6, 7, 8, 9, 10],
5..=7 => &[7, 8, 9, 10, 11],
8..=12 => &[9, 10, 11],
_ => &[11],
}
}
/// Quantise one channel at rate-distortion multiplier `lambda` and select sections.
#[allow(clippy::cognitive_complexity, reason = "the AAC rate/distortion loop, ported as-is from FilmCraft")]
pub fn quantise_channel(ch: &ChannelInput, lambda: f32) -> QuantChannel {
let nb = ch.bands.len();
let mut sf = vec![-1i32; nb];
for (i, b) in ch.bands.iter().enumerate() {
let allowed = lambda * b.thr;
if b.max34 <= 0.0 || b.en <= allowed {
continue;
}
sf[i] = search_sf(ch, b, allowed);
}
// Scalefactor differences between coded bands must stay within ±60: lower (refine) offenders,
// then quantise; repeat if a band unexpectedly quantised to all zeros (it leaves the chain).
let mut q = vec![0i32; ch.x.len()];
let mut maxq = vec![0i32; nb];
loop {
for _ in 0..8 {
let mut changed = false;
let coded: Vec<usize> = (0..nb).filter(|&i| sf[i] >= 0).collect();
for w in coded.windows(2) {
if sf[w[1]] > sf[w[0]] + 60 {
sf[w[1]] = sf[w[0]] + 60;
changed = true;
}
}
for w in coded.windows(2).rev() {
if sf[w[0]] > sf[w[1]] + 60 {
sf[w[0]] = sf[w[1]] + 60;
changed = true;
}
}
if !changed {
break;
}
}
let mut removed = false;
for (i, b) in ch.bands.iter().enumerate() {
let r = b.start..b.end;
if sf[i] < 0 {
q[r].fill(0);
maxq[i] = 0;
continue;
}
quantise(&ch.x[r.clone()], &ch.xabs[r.clone()], &ch.x34[r.clone()], sf[i], Some(&mut q[r.clone()]));
maxq[i] = q[r].iter().map(|v| v.abs()).max().unwrap_or(0);
if maxq[i] == 0 {
sf[i] = -1;
removed = true;
}
}
if !removed {
break;
}
}
// Section selection: dynamic programming per group over codebooks.
let info = &ch.info;
let (sect_bits, esc) = if info.is_short() { (3u32, 7usize) } else { (5u32, 31usize) };
let mut cb = vec![ZERO_HCB; nb];
let mut sections = Vec::with_capacity(info.num_groups);
let mut spectral_bits = 0u32;
let mut section_bits = 0u32;
let per_group = info.max_sfb;
for g in 0..info.num_groups {
let base = g * per_group;
let n = per_group;
if n == 0 {
sections.push(Vec::new());
continue;
}
// cost[i][c]
const INF: u32 = u32::MAX / 4;
let mut cost = vec![[INF; 12]; n];
for i in 0..n {
let bi = base + i;
let b = &ch.bands[bi];
let vals = &q[b.start..b.end];
for &c in candidates(maxq[bi]) {
cost[i][c as usize] = if c == 0 {
0
} else {
// all-zero band inside a coded section also needs a (zero) scalefactor difference
huffman::count_bits(c, vals) + if maxq[bi] == 0 { 1 } else { 0 }
};
}
}
let new_sec = 4 + sect_bits;
let mut best = [INF; 12];
let mut back = vec![[0u8; 12]; n];
for c in 0..12 {
if cost[0][c] < INF {
best[c] = new_sec + cost[0][c];
}
}
for i in 1..n {
let (mut min_c, mut min_v) = (0usize, INF);
for c in 0..12 {
if best[c] < min_v {
min_v = best[c];
min_c = c;
}
}
let mut nb_ = [INF; 12];
for c in 0..12 {
if cost[i][c] >= INF {
continue;
}
let stay = best[c];
let switch = min_v + new_sec;
if stay <= switch {
nb_[c] = stay + cost[i][c];
back[i][c] = c as u8;
} else {
nb_[c] = switch + cost[i][c];
back[i][c] = min_c as u8;
}
}
best = nb_;
}
let mut c = (0..12).min_by_key(|&c| best[c]).unwrap_or(0);
for i in (0..n).rev() {
cb[base + i] = c as u8;
c = back[i][c] as usize;
}
// run-length sections and exact costs
let mut secs: Vec<(u8, usize)> = Vec::new();
for i in 0..n {
let c = cb[base + i];
match secs.last_mut() {
Some((lc, len)) if *lc == c => *len += 1,
_ => secs.push((c, 1)),
}
if c != ZERO_HCB {
let b = &ch.bands[base + i];
spectral_bits += huffman::count_bits(c, &q[b.start..b.end]);
}
}
for &(_, len) in &secs {
section_bits += 4 + sect_bits * (len / esc + 1) as u32;
}
sections.push(secs);
}
// Scalefactors: global gain is the first coded band's value; bands coded with a non-zero
// codebook but all-zero values reuse the previous value (difference 0).
let first = (0..nb).find(|&i| sf[i] >= 0).map(|i| sf[i]).unwrap_or(100);
let mut prev = first;
let mut sf_bits = 0u32;
for i in 0..nb {
if cb[i] == ZERO_HCB {
continue;
}
if sf[i] < 0 {
sf[i] = prev;
}
sf_bits += huffman::sf_bits(sf[i] - prev);
prev = sf[i];
}
let tns_bits = 1 + ch.tns.as_ref().map(|t| t.bit_count(info)).unwrap_or(0);
let bits = 8 + section_bits + sf_bits + 1 + tns_bits + 1 + spectral_bits;
QuantChannel { q, sf, cb, sections, global_gain: first, bits }
}
/// Write an `individual_channel_stream`; `write_info` is false inside a common-window CPE.
pub fn write_ics(bw: &mut BitWriter, ch: &ChannelInput, qc: &QuantChannel, write_info: bool) {
let info = &ch.info;
bw.write_bits(qc.global_gain as u32, 8);
if write_info {
info.write(bw);
}
let (sect_bits, esc) = if info.is_short() { (3u32, 7usize) } else { (5u32, 31usize) };
for secs in &qc.sections {
for &(c, len) in secs {
bw.write_bits(c as u32, 4);
let mut l = len;
while l >= esc {
bw.write_bits(esc as u32, sect_bits);
l -= esc;
}
bw.write_bits(l as u32, sect_bits);
}
}
let mut prev = qc.global_gain;
for i in 0..ch.bands.len() {
if qc.cb[i] == ZERO_HCB {
continue;
}
huffman::write_sf(bw, qc.sf[i] - prev);
prev = qc.sf[i];
}
bw.write_bits(0, 1); // pulse_data_present
match &ch.tns {
Some(t) => {
bw.write_bits(1, 1);
t.write(bw, info);
}
None => bw.write_bits(0, 1),
}
bw.write_bits(0, 1); // gain_control_data_present
for (i, b) in ch.bands.iter().enumerate() {
if qc.cb[i] != ZERO_HCB {
huffman::write_spectral(bw, qc.cb[i], &qc.q[b.start..b.end]);
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::aac::ics::WindowSequence;
use crate::aac::mdct::WindowShape;
use crate::aac::tables::swb_offsets_long;
use craft_media::bitstream::BitWriter;
#[test]
fn bit_count_matches_written_bits() {
let swb = swb_offsets_long(4);
let info = IcsInfo { window_sequence: WindowSequence::OnlyLong, window_shape: WindowShape::Sine, max_sfb: 40, ..Default::default() };
let spec: Vec<f32> = (0..1024).map(|i| ((i * 7919 % 1000) as f32 - 500.0) * (1000.0 / (i as f32 + 10.0))).collect();
let thr = vec![vec![1000.0f32; 64]];
let ch = ChannelInput::new(info, swb, &spec, &thr, None);
for lambda in [0.01f32, 1.0, 100.0, 1e5] {
let qc = quantise_channel(&ch, lambda);
let mut bw = BitWriter::new();
write_ics(&mut bw, &ch, &qc, false);
assert_eq!(bw.bit_len() as u32, qc.bits, "lambda {lambda}");
}
}
}
crates/audio/src/aac/mod.rs
+82 -8
View File
@@ -1,8 +1,11 @@
//! Clean-room AAC-LC decoder (ISO/IEC 14496-3, MPEG-4 Audio, Low Complexity profile).
//! Clean-room AAC-LC encoder and decoder (ISO/IEC 14496-3, MPEG-4 Audio, Low Complexity profile).
//!
//! [`Decoder`] (feature `aac-decode`): `AudioSpecificConfig` + raw access units → planar f32.
//! Supports M/S, intensity stereo, PNS, TNS, pulse data, both window shapes and every window
//! sequence.
//! - [`Encoder`] (feature `aac-encode`): planar f32 → raw access units (+
//! [`Encoder::audio_specific_config`] for MP4 `esds`, optional ADTS framing). Psychoacoustic
//! model, window switching, M/S, TNS, bit reservoir, CBR/VBR, 1–8 channels.
//! - [`Decoder`] (feature `aac-decode`): `AudioSpecificConfig` + raw access units → planar f32.
//! Supports M/S, intensity stereo, PNS, TNS, pulse data, both window shapes and every window
//! sequence.
//!
//! HE-AAC (SBR/PS), Main/SSR/LTP profiles, coupling channels and 960-sample frames are not
//! supported; for HE-AAC only the AAC-LC core is decoded (at the core rate; [`Decoder::sbr`] says
@@ -10,15 +13,38 @@
//!
//! From FilmCraft's `filmcraft-aac` (storytold/filmcraft@8fda2f2). Public items keep their names,
//! so FilmCraft can re-export this module.
//!
//! ```
//! # #[cfg(all(feature = "aac-encode", feature = "aac-decode"))] {
//! use craft_audio::aac::{Decoder, Encoder, EncoderConfig};
//!
//! let mut enc = Encoder::new(EncoderConfig::cbr(48_000, 2, 128_000)).unwrap();
//! let left: Vec<f32> = (0..4800).map(|i| (i as f32 * 0.05).sin() * 0.5).collect();
//! let right = left.clone();
//! let mut aus = enc.encode(&[&left, &right]); // 0..n access units
//! aus.extend(enc.flush());
//! let asc = enc.audio_specific_config(); // for the MP4 `esds`
//! assert_eq!(enc.priming_samples(), 1024); // MP4 edit list media_time
//!
//! let mut dec = Decoder::new(&asc).unwrap();
//! for au in &aus {
//! let pcm = dec.decode(au).unwrap(); // planar, 1024 samples per channel
//! assert_eq!(pcm.len(), 2);
//! }
//! # }
//! ```
// Index loops follow the specification's pseudo-code (FilmCraft allows this lint workspace-wide).
#![allow(clippy::needless_range_loop)]
// Tables, Huffman coding, MDCT, window and TNS code shared by the decoder and the encoder, as
// ported; the encoder's half is unused until the encoder is added.
#![allow(dead_code)]
// Tables, Huffman coding, MDCT, window and TNS code shared by the decoder and the encoder; each
// side uses only part of it.
#![cfg_attr(not(all(feature = "aac-decode", feature = "aac-encode")), allow(dead_code))]
mod config;
#[cfg(feature = "aac-decode")]
mod decoder;
#[cfg(feature = "aac-encode")]
mod encoder;
mod huffman;
mod huffman_tables;
mod ics;
@@ -29,20 +55,27 @@ mod tns;
#[cfg(feature = "aac-decode")]
pub(crate) use config::split_adts_upto;
pub use config::{AdtsHeader, AudioSpecificConfig, ElementType, ProgramConfig, config_layout, split_adts};
#[cfg(feature = "aac-decode")]
pub use decoder::Decoder;
#[cfg(feature = "aac-encode")]
pub use encoder::{BitrateMode, Encoder, EncoderConfig, EncoderStats};
pub use mdct::WindowShape;
pub use tables::{SAMPLE_RATES, sample_rate_index};
/// Samples per AAC-LC frame (per channel).
pub const FRAME_LEN: usize = 1024;
/// Errors from configuration parsing and decoding.
/// Errors from configuration parsing, encoding and decoding.
#[non_exhaustive]
#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)]
pub enum Error {
/// The configuration is malformed or outside the supported range.
#[error("invalid configuration: {0}")]
InvalidConfig(&'static str),
/// [`EncoderConfig`] asks for something the encoder cannot do (a rate that is not an AAC
/// rate, too many channels, a bitrate out of range).
#[error("invalid encoder settings: {0}")]
InvalidSettings(&'static str),
/// A valid stream uses a feature this crate does not implement.
#[error("unsupported feature: {0}")]
Unsupported(&'static str),
@@ -80,6 +113,28 @@ pub fn wav_order(channel_config: u8) -> Option<&'static [usize]> {
})
}
/// The order the [`Encoder`] takes its input in, from WAV order: for each encoder input channel
/// (AAC element order, see [`EncoderConfig::channels`]), the WAV-order channel to feed it. The
/// inverse of [`wav_order`] for 3 to 7 channels; `None` for mono and stereo (already in order) and
/// for 8 channels, whose AAC layout (configuration 7) has no WAV equivalent.
///
/// ```
/// let wav: Vec<Vec<f32>> = vec![vec![0.0; 1024]; 6]; // L R C LFE Ls Rs
/// let order = craft_audio::aac::aac_order(6).unwrap();
/// let input: Vec<&[f32]> = order.iter().map(|&c| wav[c].as_slice()).collect(); // C L R Ls Rs LFE
/// assert_eq!(order, &[2, 0, 1, 4, 5, 3]);
/// ```
pub fn aac_order(channels: usize) -> Option<&'static [usize]> {
Some(match channels {
3 => &[2, 0, 1],
4 => &[2, 0, 1, 3],
5 => &[2, 0, 1, 3, 4],
6 => &[2, 0, 1, 4, 5, 3],
7 => &[2, 0, 1, 5, 6, 4, 3],
_ => return None,
})
}
/// The output rate of an AAC stream: twice the core rate for HE-AAC (ISO/IEC 14496-3 §1.6.5).
/// Explicit signalling names the rate in the `AudioSpecificConfig`; with implicit signalling (an
/// AAC-LC config at 24 kHz or less) SBR is only visible as fill-element data in the access units,
@@ -219,3 +274,22 @@ mod tests {
assert!(up.process(vec![vec![0.0; 600]]).remove(0).iter().all(|&s| s == 0.0));
}
}
#[cfg(test)]
mod order_tests {
use super::*;
/// `aac_order` undoes `wav_order` for each channel count the encoder maps to a configuration
/// (3–6 channels: configurations 3–6; 7: configuration 11).
#[test]
fn aac_order_inverts_wav_order() {
for (channels, config) in [(3, 3), (4, 4), (5, 5), (6, 6), (7, 11)] {
let (to_wav, to_aac) = (wav_order(config).unwrap(), aac_order(channels).unwrap());
for (wav_channel, &aac_channel) in to_wav.iter().enumerate() {
assert_eq!(to_aac[aac_channel], wav_channel, "{channels} channels");
}
}
assert_eq!(aac_order(2), None);
assert_eq!(aac_order(8), None);
}
}
crates/audio/src/lib.rs
+22 -5
View File
@@ -17,14 +17,15 @@
//! `f32` in WAV channel order.
//! - [`encode`], [`FileEncoder`]: WAV/BWF/RF64 and AIFF/AIFF-C (feature `pcm-encode`, which also
//! gives [`wav_float_header`]) and FLAC (feature `flac-encode`), whole-buffer or incremental into
//! any `Write + Seek`. Feature `lossless-encode` turns on both; feature `encode` is
//! the same as `lossless-encode`.
//! any `Write + Seek`. Feature `lossless-encode` turns on both; feature `encode` turns on
//! `lossless-encode` and `aac-encode`.
//! - [`peaks`] (feature `peaks`): min/max/RMS pyramid for waveform drawing.
//! - Codec modules for packet-level use, each with its own features: [`aac`] (`aac-decode`).
//! - Codec modules for packet-level use, each with its own features: [`aac`] (`aac-decode`,
//! `aac-encode`).
//!
//! The shared types below build with no features.
#[cfg(feature = "aac-decode")]
#[cfg(any(feature = "aac-decode", feature = "aac-encode"))]
pub mod aac;
mod buffer;
mod layout;
@@ -87,11 +88,12 @@ pub type Result<T> = std::result::Result<T, AudioError>;
// use `?` throughout: a codec's "unsupported" stays `Unsupported`, anything else in the input is
// `Malformed`, and an encoder's refusal is `Encode`.
#[cfg(feature = "aac-decode")]
#[cfg(any(feature = "aac-decode", feature = "aac-encode"))]
impl From<aac::Error> for AudioError {
fn from(e: aac::Error) -> Self {
match e {
aac::Error::Unsupported(_) => AudioError::Unsupported(e.to_string()),
aac::Error::InvalidSettings(_) => AudioError::Encode(e.to_string()),
_ => AudioError::Malformed(e.to_string()),
}
}
@@ -345,3 +347,18 @@ impl Default for EncodeOptions {
Self::new(EncodeFormat::Wav)
}
}
// An encoder refusing its settings is an encode error, not damaged input.
#[cfg(all(test, feature = "aac-encode"))]
mod tests {
use super::*;
#[test]
fn aac_encoder_settings_are_encode_errors() {
let e = match aac::Encoder::new(aac::EncoderConfig::cbr(44_000, 2, 128_000)) {
Err(e) => AudioError::from(e),
Ok(_) => panic!("44 kHz is not an AAC rate"),
};
assert!(matches!(e, AudioError::Encode(_)), "{e}");
}
}
crates/audio/tests/codecs/aac/common.rs
+40 -1
View File
@@ -3,7 +3,7 @@
use std::path::{Path, PathBuf};
use std::process::Command;
use craft_audio::aac::{Decoder, split_adts};
use craft_audio::aac::{Decoder, Encoder, EncoderConfig, EncoderStats, split_adts};
/// ffmpeg (see `crate::testkit::oracle`); `None` after printing `SKIPPED`.
pub fn ffmpeg() -> Option<PathBuf> {
@@ -119,6 +119,45 @@ pub fn clicks(rate: u32, n: usize) -> Vec<f32> {
.collect()
}
pub struct Encoded {
pub adts: Vec<u8>,
pub raw_bytes: usize,
pub frames: usize,
pub stats: EncoderStats,
pub asc: Vec<u8>,
pub bandwidth: u32,
}
pub fn encode(cfg: EncoderConfig, planar: &[Vec<f32>]) -> Encoded {
let mut enc = Encoder::new(cfg.with_adts(true)).unwrap();
let mut adts = Vec::new();
// feed in uneven chunks
let n = planar[0].len();
let mut pos = 0;
let mut step = 700;
while pos < n {
let end = (pos + step).min(n);
let chunk: Vec<&[f32]> = planar.iter().map(|c| &c[pos..end]).collect();
for au in enc.encode(&chunk) {
adts.extend(au);
}
pos = end;
step = step * 7 % 3001 + 100;
}
for au in enc.flush() {
adts.extend(au);
}
let stats = enc.stats();
Encoded {
raw_bytes: stats.bytes as usize,
frames: stats.frames as usize,
stats,
asc: enc.audio_specific_config(),
bandwidth: enc.bandwidth(),
adts,
}
}
/// Decode with ffmpeg → planar f32 (ffmpeg's own channel order), plus stderr.
pub fn ffmpeg_decode(ff: &Path, file: &Path, channels: usize) -> (Vec<Vec<f32>>, String) {
let out = Command::new(ff).args(["-v", "error", "-i"]).arg(file).args(["-f", "f32le", "-"]).output().unwrap();
crates/audio/tests/codecs/aac/encoder_oracle.rs
+273
View File
@@ -0,0 +1,273 @@
//! Encoder quality/bitrate tests using ffmpeg as an external decoding oracle (skipped if absent).
use super::common;
use common::*;
use craft_audio::aac::{BitrateMode, EncoderConfig, WindowShape};
struct Case {
name: &'static str,
rate: u32,
signal: fn(u32, usize, usize) -> Vec<Vec<f32>>,
channels: usize,
}
fn mono(v: Vec<f32>, ch: usize) -> Vec<Vec<f32>> {
(0..ch).map(|c| if c == 0 { v.clone() } else { v.iter().map(|x| x * (0.8 - 0.1 * c as f32)).collect() }).collect()
}
fn sig_sweep(r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
mono(sweep(r, n, 0.5), ch)
}
fn sig_noise(_r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
(0..ch).map(|c| noise(n, 0.3, 7 + c as u64)).collect()
}
fn sig_speech(r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
mono(speech(r, n, 3), ch)
}
fn sig_chords(r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
(0..ch).map(|c| chords(r, n, 0.002 * c as f32)).collect()
}
fn sig_clicks(r: u32, n: usize, ch: usize) -> Vec<Vec<f32>> {
mono(clicks(r, n), ch)
}
struct Row {
label: String,
target: u32,
actual: f64,
snr: f64,
bw_snr: f64,
bw: u32,
seg: f64,
peak: f32,
shorts: u64,
ours_vs_ff: f32,
}
fn run(case: &Case, bitrate: u32, cfg_mod: impl Fn(&mut EncoderConfig)) -> Option<Row> {
let ff = ffmpeg()?;
let secs = 2.0;
let n = (case.rate as f32 * secs) as usize + 333;
let input = (case.signal)(case.rate, n, case.channels);
let mut cfg = EncoderConfig::cbr(case.rate, case.channels, bitrate);
cfg_mod(&mut cfg);
let enc = encode(cfg.clone(), &input);
let tag = match cfg.mode {
BitrateMode::Cbr(b) => format!("{b}"),
BitrateMode::Vbr(q) => format!("q{q}"),
};
let path = fixtures().join(format!("enc_{}_{}ch_{}_{}.aac", case.name, case.channels, case.rate, tag));
std::fs::write(&path, &enc.adts).unwrap();
let (dec, err) = ffmpeg_decode(&ff, &path, case.channels);
assert!(err.trim().is_empty(), "ffmpeg reported errors for {}: {err}", path.display());
assert_eq!(dec[0].len(), enc.frames * 1024);
assert!(dec[0].len() >= n + 1024, "decoded stream too short");
// ffmpeg reorders channels to its native layouts: map each of our (AAC element order) decoded
// channels to the matching ffmpeg channel.
let ours = our_decode(&enc.adts);
let mut taken = vec![false; case.channels];
let mut snr_sum = 0.0;
let mut seg_sum = 0.0;
let mut bw_sum = 0.0;
let mut peak = 0f32;
let mut ours_vs_ff = 0f32;
let mut counted = 0;
for c in 0..case.channels {
let (j, err) =
(0..case.channels).filter(|&j| !taken[j]).map(|j| (j, max_abs_diff(&ours[c], &dec[j]))).min_by(|a, b| a.1.total_cmp(&b.1)).unwrap();
taken[j] = true;
ours_vs_ff = ours_vs_ff.max(err);
let d = &dec[j][1024..1024 + n];
peak = peak.max(d.iter().fold(0f32, |m, v| m.max(v.abs())));
if case.channels >= 6 && c == case.channels - 1 {
continue; // LFE is band-limited to 240 Hz by design
}
snr_sum += snr(&input[c], d);
let fc = enc.bandwidth as f32 * 0.97;
bw_sum += snr(&lowpass(&input[c], fc, case.rate as f32), &lowpass(d, fc, case.rate as f32));
seg_sum += seg_snr(&input[c], d);
counted += 1;
}
let actual = enc.stats.bitrate(case.rate);
Some(Row {
label: format!("{:<7} {}ch {:>5}", case.name, case.channels, case.rate),
target: if matches!(cfg.mode, BitrateMode::Vbr(_)) { 0 } else { bitrate },
actual,
snr: snr_sum / counted as f64,
bw_snr: bw_sum / counted as f64,
bw: enc.bandwidth,
seg: seg_sum / counted as f64,
peak,
shorts: enc.stats.short_blocks,
ours_vs_ff,
})
}
fn print(rows: &[Row]) {
println!(
"{:<22} {:>8} {:>8} {:>7} {:>6} {:>7} {:>7} {:>7} {:>6} {:>6} {:>9}",
"signal", "target", "actual", "dev%", "bw", "SNR", "bwSNR", "segSNR", "peak", "short", "us-vs-ff"
);
for r in rows {
let (target, dev) = if r.target == 0 {
("vbr".to_string(), "-".to_string())
} else {
(r.target.to_string(), format!("{:+.2}", (r.actual / r.target as f64 - 1.0) * 100.0))
};
println!(
"{:<22} {:>8} {:>8.0} {:>7} {:>6} {:>7.2} {:>7.2} {:>7.2} {:>6.3} {:>6} {:>9.2e}",
r.label, target, r.actual, dev, r.bw, r.snr, r.bw_snr, r.seg, r.peak, r.shorts, r.ours_vs_ff
);
}
}
fn check(r: &Row, min_seg: f64) {
let dev = (r.actual / r.target as f64 - 1.0).abs();
assert!(dev < 0.05, "{}: bitrate {} vs target {}", r.label, r.actual, r.target);
assert!(r.peak < 1.0, "{}: clipping (peak {})", r.label, r.peak);
assert!(r.seg > min_seg, "{}: segmental SNR {} too low", r.label, r.seg);
assert!(r.ours_vs_ff < 1e-3, "{}: our decoder differs from ffmpeg by {}", r.label, r.ours_vs_ff);
}
#[test]
fn stereo_matrix_44k_48k() {
if ffmpeg().is_none() {
eprintln!("ffmpeg not found; skipping");
return;
}
let cases = [
Case { name: "sweep", rate: 44100, signal: sig_sweep, channels: 2 },
Case { name: "noise", rate: 44100, signal: sig_noise, channels: 2 },
Case { name: "speech", rate: 48000, signal: sig_speech, channels: 2 },
Case { name: "chords", rate: 44100, signal: sig_chords, channels: 2 },
Case { name: "clicks", rate: 48000, signal: sig_clicks, channels: 2 },
];
let mut rows = Vec::new();
for c in &cases {
for br in [64_000, 128_000, 192_000, 320_000] {
let r = run(c, br, |_| {}).unwrap();
rows.push(r);
}
}
print(&rows);
for r in &rows {
check(
r,
if r.label.starts_with("noise") {
0.5
} else if r.target >= 128_000 {
8.0
} else {
3.0
},
);
}
}
#[test]
fn mono_and_surround() {
if ffmpeg().is_none() {
eprintln!("ffmpeg not found; skipping");
return;
}
let mut rows = Vec::new();
for (c, br) in [
(Case { name: "speech", rate: 44100, signal: sig_speech, channels: 1 }, 64_000),
(Case { name: "chords", rate: 48000, signal: sig_chords, channels: 1 }, 96_000),
(Case { name: "clicks", rate: 44100, signal: sig_clicks, channels: 1 }, 128_000),
(Case { name: "chords", rate: 48000, signal: sig_chords, channels: 6 }, 320_000),
(Case { name: "noise", rate: 48000, signal: sig_noise, channels: 6 }, 384_000),
] {
rows.push(run(&c, br, |_| {}).unwrap());
}
print(&rows);
for r in &rows {
check(r, 3.0);
}
}
#[test]
fn kbd_windows_no_tns_no_ms() {
if ffmpeg().is_none() {
return;
}
let c = Case { name: "kbd", rate: 44100, signal: sig_clicks, channels: 2 };
let r = run(&c, 128_000, |cfg| {
cfg.window_shape = WindowShape::Kbd;
cfg.tns = false;
cfg.ms_stereo = false;
})
.unwrap();
print(std::slice::from_ref(&r));
check(&r, 5.0);
}
#[test]
fn sample_rates_8k_to_96k() {
if ffmpeg().is_none() {
return;
}
let mut rows = Vec::new();
for (rate, br) in [
(8000u32, 24_000u32),
(11025, 32_000),
(12000, 32_000),
(16000, 48_000),
(22050, 64_000),
(24000, 64_000),
(32000, 96_000),
(64000, 192_000),
(88200, 256_000),
(96000, 256_000),
] {
let c = Case { name: "chords", rate, signal: sig_chords, channels: 2 };
rows.push(run(&c, br, |_| {}).unwrap());
}
print(&rows);
for r in &rows {
check(r, 5.0);
}
}
#[test]
fn channel_layouts_3_to_8() {
if ffmpeg().is_none() {
return;
}
let mut rows = Vec::new();
for (ch, br) in [(3usize, 192_000u32), (4, 256_000), (5, 320_000), (7, 384_000), (8, 448_000)] {
let c = Case { name: "chords", rate: 48000, signal: sig_chords, channels: ch };
rows.push(run(&c, br, |_| {}).unwrap());
}
print(&rows);
for r in &rows {
check(r, 5.0);
}
}
#[test]
fn vbr_quality_scale() {
if ffmpeg().is_none() {
return;
}
let mut rows = Vec::new();
for sig in [sig_chords as fn(u32, usize, usize) -> Vec<Vec<f32>>, sig_speech] {
for q in [1.0f32, 2.0, 3.0, 4.0, 5.0] {
let c = Case { name: "vbr", rate: 44100, signal: sig, channels: 2 };
let mut r = run(&c, 1, |cfg| cfg.mode = BitrateMode::Vbr(q)).unwrap();
r.label = format!("vbr q{q} 2ch 44100");
rows.push(r);
}
}
print(&rows);
for pair in rows.chunks(5) {
for w in pair.windows(2) {
assert!(w[1].actual > w[0].actual, "VBR bitrate must grow with quality");
assert!(w[1].seg > w[0].seg - 0.5, "VBR quality must grow with quality");
}
for r in pair {
assert!(r.peak < 1.0 && r.ours_vs_ff < 1e-3);
}
}
}
crates/audio/tests/codecs/aac/mod.rs
+2
View File
@@ -2,3 +2,5 @@
mod common;
mod decoder_oracle;
mod encoder_oracle;
mod robustness;
crates/audio/tests/codecs/aac/robustness.rs
+157
View File
@@ -0,0 +1,157 @@
//! Fuzz-style robustness: mutated access units, configs and ADTS streams must never panic, and the
//! encoder must accept hostile input (NaN, ±inf, > full scale, silence, odd chunk sizes).
use super::common;
use common::*;
use craft_audio::aac::{AudioSpecificConfig, Decoder, Encoder, EncoderConfig, split_adts};
fn corpus() -> Vec<(Vec<u8>, Vec<Vec<u8>>)> {
let mut out = Vec::new();
for (ch, rate, br) in [(1usize, 44100u32, 96_000u32), (2, 48000, 128_000), (6, 48000, 256_000), (7, 48000, 256_000)] {
let n = rate as usize / 2;
let sig: Vec<Vec<f32>> = (0..ch).map(|c| if c % 2 == 0 { clicks(rate, n) } else { chords(rate, n, 0.01) }).collect();
let mut enc = Encoder::new(EncoderConfig::cbr(rate, ch, br)).unwrap();
let refs: Vec<&[f32]> = sig.iter().map(|v| v.as_slice()).collect();
let mut aus = enc.encode(&refs);
aus.extend(enc.flush());
out.push((enc.audio_specific_config(), aus));
}
out
}
#[test]
fn mutated_access_units_never_panic() {
let mut rng = Rng(0x9E37_79B9_7F4A_7C15);
let mut rand = move |n: usize| ((rng.next() * 0.5 + 0.5) * n as f32) as usize % n.max(1);
let mut decoded = 0usize;
let mut errors = 0usize;
for (asc, aus) in corpus() {
let mut dec = Decoder::new(&asc).unwrap();
for round in 0..3000 {
let mut au = aus[rand(aus.len())].clone();
match round % 6 {
0 => {
for _ in 0..1 + rand(8) {
let i = rand(au.len());
au[i] ^= 1 << rand(8);
}
}
1 => {
let i = rand(au.len());
au[i] = rand(256) as u8;
}
2 => au.truncate(rand(au.len() + 1)),
3 => au.extend((0..rand(64)).map(|_| rand(256) as u8)),
4 => au = (0..rand(2000)).map(|_| rand(256) as u8).collect(),
_ => {
// splice two access units
let other = &aus[rand(aus.len())];
let cut = rand(au.len());
au.truncate(cut);
au.extend_from_slice(&other[rand(other.len())..]);
}
}
match dec.decode(&au) {
Ok(pcm) => {
decoded += 1;
assert!(pcm.iter().all(|c| c.len() == 1024));
}
Err(_) => errors += 1,
}
}
}
println!("mutated AUs: {decoded} decoded, {errors} rejected, 0 panics");
}
#[test]
fn random_configs_and_adts_never_panic() {
let mut rng = Rng(12345);
let mut byte = move || ((rng.next() * 0.5 + 0.5) * 256.0) as u8;
for _ in 0..20000 {
let len = (byte() % 12) as usize;
let asc: Vec<u8> = (0..len).map(|_| byte()).collect();
if let Ok(cfg) = AudioSpecificConfig::parse(&asc) {
let _ = cfg.channels();
if let Ok(mut d) = Decoder::from_config(cfg) {
let au: Vec<u8> = (0..(byte() as usize * 3)).map(|_| byte()).collect();
let _ = d.decode(&au);
}
}
}
for _ in 0..2000 {
let mut s = vec![0xFF, 0xF1];
s.extend((0..(byte() as usize * 8)).map(|_| byte()));
if let Ok(frames) = split_adts(&s)
&& let Ok(mut d) = Decoder::from_adts(&frames[0].0)
{
for (_, au) in frames {
let _ = d.decode(au);
}
}
}
}
#[test]
fn encoder_handles_hostile_input() {
for ch in [1usize, 2, 6] {
let mut enc = Encoder::new(EncoderConfig::cbr(44100, ch, 64_000 * ch as u32)).unwrap();
let n = 5000;
let evil: Vec<f32> = (0..n)
.map(|i| match i % 7 {
0 => f32::NAN,
1 => f32::INFINITY,
2 => -f32::INFINITY,
3 => 40.0,
4 => -1e30,
_ => ((i as f32) * 0.37).sin(),
})
.collect();
let chans: Vec<&[f32]> = (0..ch).map(|_| evil.as_slice()).collect();
let mut aus = Vec::new();
for chunk in [0usize, 1, 7, 1023, 1024, 1025, 3000] {
let c: Vec<&[f32]> = chans.iter().map(|s| &s[..chunk.min(s.len())]).collect();
aus.extend(enc.encode(&c));
}
// fewer channels than configured: treated as silence
aus.extend(enc.encode(&[&evil[..100]]));
aus.extend(enc.flush());
let mut dec = Decoder::new(&enc.audio_specific_config()).unwrap();
for au in &aus {
let pcm = dec.decode(au).unwrap();
assert!(pcm.iter().flatten().all(|v| v.is_finite()));
}
let expected = (1 + 7 + 1023 + 1024 + 1025 + 3000 + 100usize).div_ceil(1024) + 1;
assert_eq!(aus.len(), expected);
}
// empty stream: flush yields nothing
let mut enc = Encoder::new(EncoderConfig::cbr(48000, 2, 128_000)).unwrap();
assert!(enc.flush().is_empty());
// bad configs are rejected
assert!(Encoder::new(EncoderConfig::cbr(44000, 2, 128_000)).is_err());
assert!(Encoder::new(EncoderConfig::cbr(44100, 9, 128_000)).is_err());
assert!(Encoder::new(EncoderConfig::cbr(8000, 2, 2_000_000)).is_err());
}
#[test]
fn roundtrip_and_delay() {
// Own encoder → own decoder: priming is exactly 1024 samples and the stream covers the input.
let rate = 48000;
let n = 10_000;
let sig = chords(rate, n, 0.0);
let mut enc = Encoder::new(EncoderConfig::cbr(rate, 1, 128_000)).unwrap();
assert_eq!(enc.priming_samples(), 1024);
let mut aus = enc.encode(&[&sig]);
aus.extend(enc.flush());
let mut dec = Decoder::new(&enc.audio_specific_config()).unwrap();
let mut out = Vec::new();
for au in &aus {
out.extend(dec.decode(au).unwrap().remove(0));
}
assert!(out.len() >= n + 1024);
let s = snr(&sig, &out[1024..1024 + n]);
assert!(s > 15.0, "roundtrip SNR {s}");
// no energy in the priming region beyond pre-echo of the first attack
let e: f32 = out[..512].iter().map(|v| v * v).sum();
assert!(e < 1e-3);
}
crates/audio/tests/codecs/concealment.rs
+56
View File
@@ -0,0 +1,56 @@
//! A packet that fails to decode is replaced by silence of its length, so the audio after it keeps
//! its timing; a stream where nothing decodes is still an error. The streams are made with
//! craft-audio's own AAC encoder (no ffmpeg needed).
use craft_audio::aac::{AdtsHeader, Encoder, EncoderConfig};
use craft_audio::{AudioError, decode};
/// One second of a 440 Hz stereo tone as ADTS AAC, and the offset and length of each frame.
fn adts_stream() -> (Vec<u8>, Vec<(usize, usize)>) {
let rate = 48_000;
let tone: Vec<f32> = (0..rate).map(|i| (i as f32 * 440.0 * std::f32::consts::TAU / rate as f32).sin() * 0.5).collect();
let mut enc = Encoder::new(EncoderConfig::cbr(rate as u32, 2, 128_000).with_adts(true)).unwrap();
let mut units = enc.encode(&[&tone, &tone]);
units.extend(enc.flush());
let mut frames = Vec::new();
let mut bytes = Vec::new();
for u in units {
frames.push((bytes.len(), u.len()));
bytes.extend(u);
}
(bytes, frames)
}
#[test]
fn garbled_frames_keep_the_timeline() {
let (clean, frames) = adts_stream();
let (_, reference) = decode(&clean, Some("aac")).unwrap();
let mut garbled = clean.clone();
// Overwrite the payload (not the 7-byte header) of ten frames in the middle.
for &(at, len) in &frames[10..20] {
let header = AdtsHeader::parse(&garbled[at..]).unwrap().header_length;
for b in &mut garbled[at + header..at + len] {
*b = 0xFF;
}
}
let (_, damaged) = decode(&garbled, Some("aac")).unwrap();
assert_eq!(damaged.frames(), reference.frames(), "a garbled frame must not shorten the stream");
// Audio after the damage stays where it was.
let at = reference.frames() - 4800;
assert_eq!(&damaged.channels[0][at..], &reference.channels[0][at..]);
}
#[test]
fn a_stream_where_nothing_decodes_is_an_error() {
let (mut bytes, frames) = adts_stream();
for &(at, len) in &frames {
let header = AdtsHeader::parse(&bytes[at..]).unwrap().header_length;
for b in &mut bytes[at + header..at + len] {
*b = 0xFF;
}
}
match decode(&bytes, Some("aac")) {
Err(AudioError::Malformed(_) | AudioError::Unsupported(_)) => {}
other => panic!("expected an error, not silence: {:?}", other.map(|(i, b)| (i.frames, b.frames()))),
}
}
crates/audio/tests/codecs/main.rs
+2
View File
@@ -8,4 +8,6 @@ use craft_media_testkit as common;
mod testkit;
mod aac;
mod concealment;
mod routing_oracle;
mod track_decoder;
crates/audio/tests/codecs/track_decoder.rs
+34
View File
@@ -0,0 +1,34 @@
//! `TrackDecoder` on streams an app's own demuxer hands over without codec configuration:
//! ADTS-framed AAC (MPEG-TS).
use craft_audio::aac::{Encoder as AacEncoder, EncoderConfig};
use craft_audio::{CodecId, CodecParams, TrackDecoder};
fn tone(n: usize) -> Vec<f32> {
(0..n).map(|i| (i as f32 * 0.03).sin() * 0.4).collect()
}
#[test]
fn adts_aac_without_extradata() {
let left = tone(48_000);
let right = tone(48_000);
let mut enc = AacEncoder::new(EncoderConfig::cbr(48_000, 2, 128_000).with_adts(true)).unwrap();
let mut units = enc.encode(&[left.as_slice(), right.as_slice()]);
units.extend(enc.flush());
// No AudioSpecificConfig: the first ADTS header gives it, and every header is stripped.
let lookahead: Vec<&[u8]> = units.iter().take(craft_audio::LOOKAHEAD_PACKETS).map(Vec::as_slice).collect();
let mut dec = TrackDecoder::new(&CodecParams::new(CodecId::Aac), &lookahead).unwrap();
assert_eq!((dec.codec(), dec.sample_rate(), dec.channels()), (CodecId::Aac, 48_000, 2));
let frames: usize = units.iter().map(|u| dec.decode(u).unwrap().frames()).sum();
assert_eq!(frames, units.len() * 1024);
// Two ADTS frames in one packet, as a PES payload often holds, decode as both.
let mut dec = TrackDecoder::new(&CodecParams::new(CodecId::Aac), &lookahead).unwrap();
let pair = [units[0].as_slice(), units[1].as_slice()].concat();
assert_eq!(dec.decode(&pair).unwrap().frames(), 2048);
// Raw access units still need their configuration.
let raw = &units[0][7..];
assert!(TrackDecoder::new(&CodecParams::new(CodecId::Aac), &[raw]).is_err());
}