OrbitRIR

NeurIPS 2026 · ESR Workshop

OrbitRIRFrame-Averaged Few-Shot RIR Synthesis
for 3D Sound Source Localization

A geometry-conditioned generative model that uses forward acoustic knowledge to solve inverse 3D sound source localization problems.

Yixun HuZhixuan ZhaoTao ZhongChristine Allen-Blanchette

Princeton University

arXiv Coming soonCode Coming soon

Abstract

Room impulse responses (RIRs) capture how sound interacts with the geometry of a room, making acoustics a rich signal for spatial reasoning. Few-shot acoustic models can predict RIRs from geometry and sparse room observations, but they do not necessarily respect known physical symmetries.

We introduce OrbitRIR, a geometry-conditioned generative acoustic model that incorporates receiver-yaw invariance through Frame Averaging. By averaging conditioning features over equivalent coordinate frames, OrbitRIR builds this symmetry into the model. The trained generator can then be frozen and inverted for 3D sound source localization through analysis-by-synthesis, without localization-specific training or fine-tuning. Experiments on AcousticRooms show improved few-shot RIR synthesis and downstream localization relative to the unconstrained FLAC baseline.

Method

Learn a room’s acoustic response from panoramic geometry and a few context RIRs. Then use the same forward model to infer where an observed sound originated.

OrbitRIR training and localization pipeline. Frame-averaged geometry conditions RIR generation; at inference, candidate source positions are scored against an observed response.
One model, two uses. Train for few-shot acoustic synthesis, then freeze the generator for source localization.
01

Condition on the room

A panoramic depth map, source coordinates, and sparse acoustic context describe the scene. A latent flow-matching model generates plausible RIRs for a queried source position.

02

Average over frames

Transform geometry and source coordinates consistently across a finite receiver-yaw orbit. A shared encoder processes each frame, and averaging its features yields invariant conditioning.

03

Localize by synthesis

Query the frozen generator at candidate 3D positions. Compare synthesized and observed RIRs in the frozen AGREE audio embedding space, and select the most compatible position.

Frame Averaging: different frames, the same scene

For an omnidirectional monaural RIR, re-expressing the same scene in a different receiver-yaw coordinate frame should not change its acoustics. Frame Averaging makes the geometry representation invariant to the chosen finite yaw group: a frame change simply reorders the features being averaged.

Explore the symmetry · conceptual

A rotation only reorders the sum.

Choose a C4 frame change. The four encoded features are permuted; their average stays the same.

F̄ = ¼ (F₀ + F₁ + F₂ + F₃)
Same set → same average
The average is unchanged under a C4 frame change.
Four yaw-aligned panoramic geometry and source-coordinate frames pass through a shared ViT and projection. Their encoded features are averaged into one conditioning representation.
Frame-averaged conditioning. The same encoder and projection are applied across the yaw orbit before feature averaging.

The invariance is defined over a finite yaw group, illustrated here with C4. It concerns coordinate changes of the same acoustic scene, rather than arbitrary physical rotations of directional sound sources.

Results

We evaluate both the forward task—few-shot RIR synthesis—and the inverse task—3D sound source localization in unseen rooms.

Few-shot acoustic synthesis

OrbitRIR improves all reported synthesis metrics over FLAC in both sparse and dense context settings. Its synthesis quality also stays stable across the evaluated receiver-yaw frames, while FLAC varies with the coordinate frame. Compared with yaw augmentation, the methods exhibit metric-dependent trade-offs: OrbitRIR improves clarity and early-decay errors, while yaw augmentation achieves lower reverberation-time error.

Six synthesis metrics across four yaw frames, comparing FLAC and OrbitRIR with sparse and dense context. OrbitRIR's curves remain nearly constant across frames.
Stable across receiver-yaw frames. Synthesis errors and retrieval recalls on unseen AcousticRooms. Error bars show the standard deviation over evaluation seeds.

3D sound source localization

The frozen acoustic generator supports localization without task-specific training. OrbitRIR improves localization over FLAC and achieves the strongest resolution-aware success rate among the compared methods. Results vary by metric: yaw-augmented FLAC has a lower median localization error.

Compare localization predictions

Choose any two methods and explore the room examples from the paper.

OrbitRIR localization examples in six rooms from Figure 10.
OrbitRIR predictionFull paper figure ↗
FLAC localization examples in six rooms from Figure 12.
FLAC predictionFull paper figure ↗
Ground-truth source Receiver Method prediction

Original visualizations from manuscript Figures 9–13. Select “All six rooms” or open a full paper figure to see each original panel in context.

Stay tuned

arXiv and code will be released soon.

Links to the paper and research code will appear here when available.