UE4.27.2 安卓音频库集成的调试分析
2025-06-01 10:52:45 #Cpp #Unreal Engine 4 #OpenSL ES

AudioTrack 初始化

创建音频后端 Device

FAudioDeviceManager::LoadDefaultAudioDeviceModule 实现了读取配置文件，创建音频后端 Device

1
2
3
4
5
6
7
8
9
10
11
12
* frame #0: 0x0000007bd4ae1ba0 libUE4.so`FAudioDeviceManager::LoadDefaultAudioDeviceModule(this=0x0000007c7983e5c0) at AudioDeviceManager.cpp:575:40
  frame #1: 0x0000007bd4ae1a9c libUE4.so`FAudioDeviceManager::InitializeManager(this=0x0000007c7983e5c0) at AudioDeviceManager.cpp:512:6
  frame #2: 0x0000007bd4ae451c libUE4.so`FAudioDeviceManager::Initialize() at AudioDeviceManager.cpp:824:19
  frame #3: 0x0000007bd5889fb0 libUE4.so`UEngine::InitializeAudioDeviceManager(this=0x0000007c780040c0) at UnrealEngine.cpp:3107:2
  frame #4: 0x0000007bd5877f30 libUE4.so`UEngine::Init(this=<unavailable>, InEngineLoop=<unavailable>) at UnrealEngine.cpp:1643:2
  frame #5: 0x0000007bd4e62e34 libUE4.so`UGameEngine::Init(this=0x0000007c780040c0, InEngineLoop=<unavailable>) at GameEngine.cpp:1072:11
  frame #6: 0x0000007bd01c3158 libUE4.so`FEngineLoop::Init(this=<unavailable>) at LaunchEngineLoop.cpp:4017:12
  frame #7: 0x0000007bd01c2458 libUE4.so`AndroidMain(state=<unavailable>) at LaunchAndroid.cpp:501:14
  frame #8: 0x0000007bd01d20fc libUE4.so`android_main(state=0x0000007c9d910d00) at LaunchAndroid.cpp:777:2
  frame #9: 0x0000007bd02015b8 libUE4.so`android_app_entry(param=0x0000007c9d910d00) at android_native_app_glue.c:233:5
  frame #10: 0x0000007d25d1e2a8 libc.so`__pthread_start(void*) + 200
  frame #11: 0x0000007d25d0f624 libc.so`__start_thread + 68

默认的逻辑是先根据 AudioMixerModuleName 加载，如果加载失败，再根据 AudioDeviceModuleName 加载

在 <工程根目录>\Saved\Temp\Android\Engine\Config\Android\AndroidEngine.ini 可以看到

1
2
3
[Audio]
AudioDeviceModuleName=AndroidAudio
AudioMixerModuleName=AudioMixerAndroid

于是这里它先读取 AudioMixerAndroid 模块，成功了之后就不再读取 AndroidAudio 模块了

注册 BufferQueue 回调

BufferQueue 的回调为 FMixerPlatformAndroid::OpenSLBufferQueueCallback

1
2
3
4
5
6
7
8
void FMixerPlatformAndroid::OpenSLBufferQueueCallback(SLAndroidSimpleBufferQueueItf InQueueInterface, void* pContext)
{
    FMixerPlatformAndroid* MixerPlatformAndroid = (FMixerPlatformAndroid*)pContext;
    if (MixerPlatformAndroid != nullptr)
    {
        MixerPlatformAndroid->ReadNextBuffer();
    }
}

他是在这里注册的

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
* frame #0: 0x0000007bd5e94d08 libUE4.so`Audio::FMixerPlatformAndroid::OpenAudioStream(this=0x0000007b08ed1c80, Params=<unavailable>) at AudioMixerPlatformAndroid.cpp:291:14
  frame #1: 0x0000007bd3ed27a4 libUE4.so`Audio::FMixerDevice::InitializeHardware(this=0x0000007b086b20e0) at AudioMixerDevice.cpp:403:28
  frame #2: 0x0000007bd446695c libUE4.so`FAudioDevice::Init(this=0x0000007b086b20e0, InDeviceID=<unavailable>, InMaxSources=<unavailable>) at AudioDevice.cpp:513:7
  frame #3: 0x0000007bd44bb224 libUE4.so`FAudioDeviceManager::FAudioDeviceContainer::FAudioDeviceContainer(this=0x0000007be27f32d0, InParams=<unavailable>, InDeviceID=1, DeviceManager=<unavailable>) at AudioDeviceManager.cpp:1672:14
  frame #4: 0x0000007bd44b307c libUE4.so`FAudioDeviceManager::CreateNewDevice(this=0x0000007b0a3cbf40, InParams=0x0000007be27f3418) at AudioDeviceManager.cpp:651:28
  frame #5: 0x0000007bd44b2f40 libUE4.so`FAudioDeviceManager::RequestAudioDevice(this=0x0000007b0a3cbf40, InParams=0x0000007be27f3418) at AudioDeviceManager.cpp:465:10
  frame #6: 0x0000007bd44b5068 libUE4.so`FAudioDeviceManager::CreateMainAudioDevice(this=0x0000007b0a3cbf40) at AudioDeviceManager.cpp:552:27
  frame #7: 0x0000007bd525bfc0 libUE4.so`UEngine::InitializeAudioDeviceManager(this=0x0000007b086b40c0) at UnrealEngine.cpp:3111:23
  frame #8: 0x0000007bd5249f30 libUE4.so`UEngine::Init(this=<unavailable>, InEngineLoop=<unavailable>) at UnrealEngine.cpp:1643:2
  frame #9: 0x0000007bd4834e34 libUE4.so`UGameEngine::Init(this=0x0000007b086b40c0, InEngineLoop=<unavailable>) at GameEngine.cpp:1072:11
  frame #10: 0x0000007bcfb95158 libUE4.so`FEngineLoop::Init(this=<unavailable>) at LaunchEngineLoop.cpp:4017:12
  frame #11: 0x0000007bcfb94458 libUE4.so`AndroidMain(state=<unavailable>) at LaunchAndroid.cpp:501:14
  frame #12: 0x0000007bcfba40fc libUE4.so`android_main(state=0x0000007bdf2881c0) at LaunchAndroid.cpp:777:2
  frame #13: 0x0000007bcfbd35b8 libUE4.so`android_app_entry(param=0x0000007bdf2881c0) at android_native_app_glue.c:233:5
  frame #14: 0x0000007d25d1e2a8 libc.so`__pthread_start(void*) + 200
  frame #15: 0x0000007d25d0f624 libc.so`__start_thread + 68

搜索 )->RegisterCallback(，只有七条结果，可以看到，整个 UE 引擎中对 BufferQueue Callback 的注册都是很简单的

首次播放音频

FMixerDevice::InitializeHardware 初始化的时候会播放一次

首次提交数据

1
2
3
4
5
6
7
* frame #0: 0x0000007bd64c49bc libUE4.so`Audio::FMixerPlatformAndroid::SubmitBuffer(this=0x0000007c78041280, Buffer="") at AudioMixerPlatformAndroid.cpp:410:3
  frame #1: 0x0000007bd44e3a20 libUE4.so`Audio::IAudioMixerPlatformInterface::RunInternal(this=0x0000007c78041280) at AudioMixer.cpp:662:3
  frame #2: 0x0000007bd44e3bdc libUE4.so`Audio::IAudioMixerPlatformInterface::Run(this=0x0000007c78041280) at AudioMixer.cpp:0
  frame #3: 0x0000007bd152b698 libUE4.so`FRunnableThreadPThread::Run(this=0x0000007c780b6c40) at PThreadRunnableThread.cpp:25:24
  frame #4: 0x0000007bd1408078 libUE4.so`FRunnableThreadPThread::_ThreadProc(pThis=0x0000007c780b6c40) at PThreadRunnableThread.h:185:15
  frame #5: 0x0000007d25d1e2a8 libc.so`__pthread_start(void*) + 200
  frame #6: 0x0000007d25d0f624 libc.so`__start_thread + 68

IAudioMixerPlatformInterface::RunInternal 表示，音频线程首次运行的时候提交一次 buffer

IAudioMixerPlatformInterface 派生自 FRunnable，是一个可以运行的线程

初始化混音数据

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
frame #0: 0x0000007a9f10b1b8 libUE4.so`Audio::FMixerDevice::LoadMasterSoundSubmix(this=0x00000079d3e630d0, InType=Master, InDefaultName=string=u"MasterSubmixDefault", bInDefaultMuteWhenBackgrounded=<unavailable>, InObjectPath=0x00000079d4fd1630) at AudioMixerDevice.cpp:803
frame #1: 0x0000007a9f10da38 libUE4.so`Audio::FMixerDevice::InitSoundSubmixes(this=0x00000079d3e630d0) at AudioMixerDevice.cpp:957:4
frame #2: 0x0000007a9f103930 libUE4.so`Audio::FMixerDevice::InitializeHardware(this=0x00000079d3e630d0) at AudioMixerDevice.cpp:450:5
frame #3: 0x0000007a9f69795c libUE4.so`FAudioDevice::Init(this=0x00000079d3e630d0, InDeviceID=<unavailable>, InMaxSources=<unavailable>) at AudioDevice.cpp:513:7
frame #4: 0x0000007a9f6ec224 libUE4.so`FAudioDeviceManager::FAudioDeviceContainer::FAudioDeviceContainer(this=0x0000007b009f32d0, InParams=<unavailable>, InDeviceID=1, DeviceManager=<unavailable>) at AudioDeviceManager.cpp:1672:14
frame #5: 0x0000007a9f6e407c libUE4.so`FAudioDeviceManager::CreateNewDevice(this=0x00000079d4fdba00, InParams=0x0000007b009f3418) at AudioDeviceManager.cpp:651:28
frame #6: 0x0000007a9f6e3f40 libUE4.so`FAudioDeviceManager::RequestAudioDevice(this=0x00000079d4fdba00, InParams=0x0000007b009f3418) at AudioDeviceManager.cpp:465:10
frame #7: 0x0000007a9f6e6068 libUE4.so`FAudioDeviceManager::CreateMainAudioDevice(this=0x00000079d4fdba00) at AudioDeviceManager.cpp:552:27
frame #8: 0x0000007aa048cfc0 libUE4.so`UEngine::InitializeAudioDeviceManager(this=0x00000079d3e650b0) at UnrealEngine.cpp:3111:23
frame #9: 0x0000007aa047af30 libUE4.so`UEngine::Init(this=<unavailable>, InEngineLoop=<unavailable>) at UnrealEngine.cpp:1643:2
frame #10: 0x0000007a9fa65e34 libUE4.so`UGameEngine::Init(this=0x00000079d3e650b0, InEngineLoop=<unavailable>) at GameEngine.cpp:1072:11
frame #11: 0x0000007a9adc6158 libUE4.so`FEngineLoop::Init(this=<unavailable>) at LaunchEngineLoop.cpp:4017:12
frame #12: 0x0000007a9adc5458 libUE4.so`AndroidMain(state=<unavailable>) at LaunchAndroid.cpp:501:14
frame #13: 0x0000007a9add50fc libUE4.so`android_main(state=0x0000007ab0383300) at LaunchAndroid.cpp:777:2
frame #14: 0x0000007a9ae045b8 libUE4.so`android_app_entry(param=0x0000007ab0383300) at android_native_app_glue.c:233:5
frame #15: 0x0000007c068ab2a8 libc.so`__pthread_start(void*) + 200
frame #16: 0x0000007c0689c624 libc.so`__start_thread + 68

BufferQueue 回调在 Tick 中被调用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
void FMixerPlatformAndroid::SubmitBuffer(const uint8* Buffer)
{
    check(DeviceBuffer.Num() == NumSamplesPerDeviceCallback);

    int32 PushResult = CircularOutputBuffer.Push((const int16*)Buffer, NumSamplesPerRenderCallback);
    check(PushResult == NumSamplesPerRenderCallback)

    while (CircularOutputBuffer.Num() >= NumSamplesPerDeviceCallback)
    {
        int32 PopResult = CircularOutputBuffer.Pop(DeviceBuffer.GetData(), NumSamplesPerDeviceCallback);
        check(PopResult == NumSamplesPerDeviceCallback);

        const auto BufferSize = NumSamplesPerDeviceCallback * sizeof(int16);
        SLresult Result = (*SL_PlayerBufferQueue)->Enqueue(SL_PlayerBufferQueue, Buffer, BufferSize);
        OPENSLES_LOG_ON_FAIL(Result);
    }
}

回调堆栈如下

1
2
3
4
5
6
7
8
9
* frame #0: 0x0000007bcf8c49bc libUE4.so`Audio::FMixerPlatformAndroid::SubmitBuffer(this=0x0000007b03b25b00, Buffer="") at AudioMixerPlatformAndroid.cpp:410:3
  frame #1: 0x0000007bcd8e328c libUE4.so`Audio::IAudioMixerPlatformInterface::ReadNextBuffer(this=0x0000007b03b25b00) at AudioMixer.cpp:542:3
  frame #2: 0x0000007d43b68aa0 libwilhelm.so`android::AudioTrackCallback::onMoreData(android::AudioTrack::Buffer const&) - 18446743535702603103
  frame #3: 0x0000007d25997398 libaudioclient.so`android::AudioTrack::processAudioBuffer() + 3304
  frame #4: 0x0000007d25996354 libaudioclient.so`android::AudioTrack::AudioTrackThread::threadLoop() + 340
  frame #5: 0x0000007d49c5f5cc libutils.so`android::Thread::_threadLoop(void*) + 252
  frame #6: 0x0000007d33313490 libandroid_runtime.so`android::AndroidRuntime::javaThreadShell(void*) + 144
  frame #7: 0x0000007d25d1e2a8 libc.so`__pthread_start(void*) + 200
  frame #8: 0x0000007d25d0f624 libc.so`__start_thread + 68

也就是

FMixerPlatformAndroid::OpenSLBufferQueueCallback -> IAudioMixerPlatformInterface::ReadNextBuffer -> FMixerPlatformAndroid::SubmitBuffer

BufferQueue 回调是在 Tick 中被调用的

如果当前没有音频播放，那么 Buffer 指向的内存是全 0

如果当前有音频播放，那么 Buffer 指向的内存是非 0 值，应该就是音频数据

FAudioDevice 控制播放与暂停

我是在蓝图里面调用 UAudioComponent::Play 来播放音频的

它最终调用 UAudioComponent::PlayInternal

UAudioComponent::PlayInternal 中核心的部分还是 AudioDevice->AddNewActiveSound(NewActiveSound);

调用到 FAudioDevice::AddNewActiveSoundInternal

首先，把函数自己转到音频线程

1
2
3
4
5
6
7
8
9
10
11
12
if (!IsInAudioThread())
{
    DECLARE_CYCLE_STAT(TEXT("FAudioThreadTask.AddNewActiveSound"), STAT_AudioAddNewActiveSound, STATGROUP_AudioThreadCommands);

    FAudioDevice* AudioDevice = this;
    FAudioThread::RunCommandOnAudioThread([AudioDevice, NewActiveSound, VirtualLoopToRetrigger]()
    {
        AudioDevice->AddNewActiveSoundInternal(NewActiveSound, VirtualLoopToRetrigger);
    }, GET_STATID(STAT_AudioAddNewActiveSound));

    return;
}

然后程序化生成、虚拟 loop 等就不说了

最终添加播放音频

1
2
3
4
5
ActiveSounds.Add(ActiveSound);
if (ActiveSound->GetAudioComponentID() > 0)
{
    AudioComponentIDToActiveSoundMap.Add(ActiveSound->GetAudioComponentID(), ActiveSound);
}

可以看到 FAudioDevice 中并没有名称很直接的 Play 方法

Exec, Update 这两个方法看上去像一点

不过看了 Exec，它是处理命令行的

于是看 FAudioDevice::Update

1
2
3
4
5
6
7
8
9
10
11
* frame #0: 0x0000007bd707402c libUE4.so`FAudioDevice::Update(this=0x0000007b06cb20e0, bGameTicking=<unavailable>) at AudioDevice.cpp:4201:6
  frame #1: 0x0000007bd70aa538 libUE4.so`FAudioDeviceManager::IterateOverAllDevices(TUniqueFunction<void (unsigned int, FAudioDevice*)>) [inlined] UE4Function_Private::TFunctionRefBase<UE4Function_Private::TFunctionStorage<true>, void (unsigned int, FAudioDevice*)>::operator()(this=0x0000007be3ee3c20, Params=1, Params=0x0000007b06cb20e0) const at Function.h:676:11
  frame #2: 0x0000007bd70aa50c libUE4.so`FAudioDeviceManager::IterateOverAllDevices(this=0x0000007b081ebbc0, ForEachDevice=TUniqueFunction<void (unsigned int, FAudioDevice *)> @ 0x0000007be3ee3c20) at AudioDeviceManager.cpp:924
  frame #3: 0x0000007bd70a9a24 libUE4.so`FAudioDeviceManager::UpdateActiveAudioDevices(this=0x0000007b081ebbc0, bGameTicking=<unavailable>) at AudioDeviceManager.cpp:906:2
  frame #4: 0x0000007bd742dc50 libUE4.so`UGameEngine::Tick(this=<unavailable>, DeltaSeconds=<unavailable>, bIdleMode=false) at GameEngine.cpp:1951:27
  frame #5: 0x0000007bd278ffdc libUE4.so`FEngineLoop::Tick(this=<unavailable>) at LaunchEngineLoop.cpp:4915:12
  frame #6: 0x0000007bd2787634 libUE4.so`AndroidMain(state=<unavailable>) at LaunchAndroid.cpp:534:16
  frame #7: 0x0000007bd27970fc libUE4.so`android_main(state=0x0000007c9d910f80) at LaunchAndroid.cpp:777:2
  frame #8: 0x0000007bd27c65b8 libUE4.so`android_app_entry(param=0x0000007c9d910f80) at android_native_app_glue.c:233:5
  frame #9: 0x0000007d25d1e2a8 libc.so`__pthread_start(void*) + 200
  frame #10: 0x0000007d25d0f624 libc.so`__start_thread + 68

这个调用堆栈也很清晰，可以看到，可能存在有多个 Device

对于 Update 的内容

这里面有一个 FAudioDevice::HandlePause，但是他不是用来设置单个 Sound 的暂停的，它是根据游戏全局暂停状态来设置所有的声音的暂停或者恢复的

然后这里面看到一个 FAudioDevice::ProcessingPendingActiveSoundStops 应该是比较像的

于是我在蓝图用 AudioComp 暂停音频的时候，断点打到

1
2
3
4
5
6
7
8
9
10
if (bDeleteActiveSound)
{
    if (ActiveSound->bIsPreviewSound && bModulationInterfaceEnabled && ModulationInterface.IsValid())
    {
        ModulationInterface->OnAuditionEnd();
    }
    ActiveSound->bAsyncOcclusionPending = false;
    PendingSoundsToDelete.RemoveAtSwap(i, 1, false);
    delete ActiveSound;
}

确实是删除了 Active Sound

这也说明 FAudioDevice 是 Active Sound 的持有者。毕竟只有持有它的人才有权力销毁他。

其实回来看 AudioComp，跟播放同理，UAudioComponent::Stop -> FAudioDevice::StopActiveSound -> FAudioDevice::AddSoundToStop。最终 FAudioDevice::AddSoundToStop 中把某个要暂停的 Active Sound 放入 pending 队列 PendingSoundsToStop

双缓冲实现无锁化

前面还没有看完 FAudioDevice::Update

FAudioDevice::Update 最后一段

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
if (Sources.Num())
{
    // Kill any sources that have finished
    for (int32 SourceIndex = 0; SourceIndex < Sources.Num(); SourceIndex++)
    {
        // Source has finished playing (it's one shot)
        if (Sources[ SourceIndex ]->IsFinished())
        {
            Sources[ SourceIndex ]->Stop();
        }
    }

    // Poll audio components for active wave instances (== paths in node tree that end in a USoundWave)
    ActiveWaveInstances.Reset();
    FirstActiveIndex = GetSortedActiveWaveInstances(ActiveWaveInstances, (bGameTicking ? ESortedActiveWaveGetType::FullUpdate : ESortedActiveWaveGetType::PausedUpdate));

    // Stop sources that need to be stopped, and touch the ones that need to be kept alive
    StopSources(ActiveWaveInstances, FirstActiveIndex);

    // Start and/or update any sources that have a high enough priority to play
    StartSources(ActiveWaveInstances, FirstActiveIndex, bGameTicking);

    // Check which sounds are active from these wave instances and update passive SoundMixes
    UpdatePassiveSoundMixModifiers(ActiveWaveInstances, FirstActiveIndex);

    // If not paused, update the playback time of the active sounds after we've processed passive mix modifiers
    // Note that for sounds which play while paused, this will result in longer active sound playback times, which will be ok. If we update the
    // active sound is updated while paused (for a long time), most sounds will be stopped when unpaused.
    UpdateActiveSoundPlaybackTime(bGameTicking);
}

应该是为了控制高优先级的音频先播放

然后是一个虚函数 UpdateHardware，这里就可以跳转到 FMixerDevice::UpdateHardware

感觉其中重要的还是这两句

1
2
3
SourceManager->Update();

AudioMixerPlatform->OnHardwareUpdate();

于是看 FMixerSourceManager::Update 有段双缓冲实现无锁化的代码，很酷，我还是第一次接触无锁化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
// If the command was triggered, then we want to do a swap of command buffers
if (CommandsProcessedEvent->Wait(0))
{
    int32 CurrentGameIndex = !RenderThreadCommandBufferIndex.GetValue();

    // This flags the audio render thread to be able to pump the next batch of commands
    // And will allow the audio thread to write to a new command slot
    const int32 NextIndex = (CurrentGameIndex + 1) & 1;

    FCommands& NextCommandBuffer = CommandBuffers[NextIndex];

    // Make sure we've actually emptied the command queue from the render thread before writing to it
    if (FlushCommandBufferOnTimeoutCvar && NextCommandBuffer.SourceCommandQueue.Num() != 0)
    {
        UE_LOG(LogAudioMixer, Warning, TEXT("Audio render callback stopped. Flushing %d commands."), NextCommandBuffer.SourceCommandQueue.Num());

        // Pop and execute all the commands that came since last update tick
        for (int32 Id = 0; Id < NextCommandBuffer.SourceCommandQueue.Num(); ++Id)
        {
            TFunction<void()>& CommandFunction = NextCommandBuffer.SourceCommandQueue[Id];
            CommandFunction();
            NumCommands.Decrement();
        }

        NextCommandBuffer.SourceCommandQueue.Reset();
    }

    // Here we ensure that we block for any pending calls to AudioMixerThreadCommand.
    FScopeLock ScopeLock(&CommandBufferIndexCriticalSection);
    RenderThreadCommandBufferIndex.Set(CurrentGameIndex);

    CommandsProcessedEvent->Reset();
}

数据结构设计

命令缓冲区结构

1
2
3
4
5
6
struct FCommands
{
    TArray<TFunction<void()>> SourceCommandQueue; // 存储命令函数的动态数组
};

FCommands CommandBuffers[2]; // 双缓冲结构

同步控制变量

1
2
3
FThreadSafeCounter RenderThreadCommandBufferIndex; // 原子计数器，表示当前渲染线程使用的缓冲区索引
FEvent* CommandsProcessedEvent; // 事件对象，用于线程同步
FCriticalSection CommandBufferIndexCriticalSection; // 临界区，保护索引交换

双缓冲工作机制

缓冲区索引管理

使用 RenderThreadCommandBufferIndex 原子变量(取值0或1)标识当前渲染线程正在处理的缓冲区

游戏线程总是向另一个缓冲区(!RenderThreadCommandBufferIndex)写入命令

工作流程

  1. 游戏线程写入阶段：

    游戏线程向非活动缓冲区(与渲染线程当前使用的不同)添加命令

    命令以lambda函数形式存储在SourceCommandQueue数组中

  2. 缓冲区交换阶段：

    当需要提交命令时(如每帧更新)，游戏线程检查CommandsProcessedEvent

    确认渲染线程已完成前一帧命令处理后，交换缓冲区索引

  3. 渲染线程处理阶段：

    渲染线程持续检查RenderThreadCommandBufferIndex获取当前命令缓冲区

    按顺序执行缓冲区中的所有命令函数

    处理完成后触发CommandsProcessedEvent

关键同步逻辑

缓冲区交换代码

1
2
3
4
5
6
7
8
9
10
// 获取当前渲染线程使用的缓冲区索引
int32 CurrentGameIndex = !RenderThreadCommandBufferIndex.GetValue(); 

// 准备下一个缓冲区索引
const int32 NextIndex = (CurrentGameIndex + 1) & 1; // 0↔1切换

// 保护索引交换操作
FScopeLock ScopeLock(&CommandBufferIndexCriticalSection);
RenderThreadCommandBufferIndex.Set(CurrentGameIndex);
CommandsProcessedEvent->Reset();

超时处理机制

1
2
3
4
5
6
7
8
9
10
11
if (FlushCommandBufferOnTimeoutCvar && 
    NextCommandBuffer.SourceCommandQueue.Num() != 0)
{
    // 强制处理积压的命令
    for (int32 Id = 0; Id < NextCommandBuffer.SourceCommandQueue.Num(); ++Id)
    {
        NextCommandBuffer.SourceCommandQueue[Id](); // 执行命令
        NumCommands.Decrement();
    }
    NextCommandBuffer.SourceCommandQueue.Reset();
}

FAudioDevice 的 Update 控制 FMixerPlatformAndroid 环形缓冲

还是没有找到是谁控制了 FMixerPlatformAndroid 环形缓冲

于是再回来看 FMixerPlatformAndroid::SubmitBuffer

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
void FMixerPlatformAndroid::SubmitBuffer(const uint8* Buffer)
{
    check(DeviceBuffer.Num() == NumSamplesPerDeviceCallback);

    int32 PushResult = CircularOutputBuffer.Push((const int16*)Buffer, NumSamplesPerRenderCallback);
    check(PushResult == NumSamplesPerRenderCallback)

    while (CircularOutputBuffer.Num() >= NumSamplesPerDeviceCallback)
    {
        int32 PopResult = CircularOutputBuffer.Pop(DeviceBuffer.GetData(), NumSamplesPerDeviceCallback);
        check(PopResult == NumSamplesPerDeviceCallback);

        const auto BufferSize = NumSamplesPerDeviceCallback * sizeof(int16);
        SLresult Result = (*SL_PlayerBufferQueue)->Enqueue(SL_PlayerBufferQueue, Buffer, BufferSize);
        OPENSLES_LOG_ON_FAIL(Result);
    }
}

我现在才看懂他的意思。先向环形缓冲拷贝数据，拷贝数据的大小为 NumSamplesPerRenderCallback。然后从环形缓冲拷出数据，拷到 DeviceBuffer.GetData()，拷贝数据的大小为 NumSamplesPerDeviceCallback。内部都是 memcpy。

奇怪的是，它 EnQueue 还是传输的传入的参数 Buffer 而不是 DeviceBuffer.GetData()？

看看他 Tick 中回调中调用 FMixerPlatformAndroid::SubmitBuffer 的 IAudioMixerPlatformInterface::ReadNextBuffer 中是怎么传入 SubmitBuffer 的参数的

1
2
3
int32 NumSamplesPopped = 0;
TArrayView<const uint8> PoppedAudio = OutputBuffer.PopBufferData(NumSamplesPopped);
SubmitBuffer(PoppedAudio.GetData());

ok，这是一个存储要 EnQueue 实际数据的缓冲

1
2
/** List of generated output buffers. */
Audio::FOutputBuffer OutputBuffer;

看看他的使用，应该是在 FOutputBuffer::MixNextBuffer

但是有两个地方都在 while 中调用了它，打个断点看看到底是哪个循环

IAudioMixerPlatformInterface::RunInternal 在 while 循环外调用了一次，这是首次加载时的工作

然后就一直在 IAudioMixerPlatformInterface::RunInternal 的 while 中被调用

于是看 FOutputBuffer::MixNextBuffer 做了什么

他就是调用 Mixer 填充 RenderBuffer，然后把数据从一个 RenderBuffer 拷贝到 CircularBuffer 中

IAudioMixerPlatformInterface 有一个 FOutputBuffer 成员，FOutputBuffer 有一个 CircularBuffer 成员

class FMixerPlatformAndroid : public IAudioMixerPlatformInterface 是派生自 IAudioMixerPlatformInterface

而 FMixerPlatformAndroid 还有一个 CircularOutputBuffer 成员，无敌了，为什么要这么设计

1
2
3
// This buffer is pushed to and popped from in the SubmitBuffer callback. 
// This is required for devices that require frame counts per callback that are not powers of two.
Audio::TCircularAudioBuffer<int16> CircularOutputBuffer;

FMixerDevice 是持有 AudioMixerPlatform 的

1
2
/** Ptr to the platform interface, which handles streaming audio to the hardware device. */
IAudioMixerPlatformInterface* AudioMixerPlatform;

这是实现了多态的行为

最终发现这个 IAudioMixerPlatformInterface 是运行在一个独立的线程，这个线程专门负责生成音频数据

那么，这个线程是在哪里被发起的呢？谁调用了他的 Run()？暂时没找到。

Submit 路径

1
2
3
4
5
6
7
8
* frame #0: 0x0000007aa0ebdb0c libUE4.so`Audio::FMixerPlatformAndroid::SubmitBuffer(this=0x00000079d2e86780, Buffer="") at AudioMixerPlatformAndroid.cpp:418:4
  frame #1: 0x0000007a9eedc28c libUE4.so`Audio::IAudioMixerPlatformInterface::ReadNextBuffer(this=0x00000079d2e86780) at AudioMixer.cpp:542:3
  frame #2: 0x0000007bf7ceaaa0 libwilhelm.so`android::AudioTrackCallback::onMoreData(android::AudioTrack::Buffer const&) - 18446743541271057759
  frame #3: 0x0000007c01f1c398 libaudioclient.so`android::AudioTrack::processAudioBuffer() + 3304
  frame #4: 0x0000007c01f1b354 libaudioclient.so`android::AudioTrack::AudioTrackThread::threadLoop() + 340
  frame #5: 0x0000007be492b5cc libutils.so`android::Thread::_threadLoop(void*) + 252
  frame #6: 0x0000007bf99a2490 libandroid_runtime.so`android::AndroidRuntime::javaThreadShell(void*) + 144
  frame #7: 0x0000007c068ab2a8 libc.so`__pthread_start(void*) + 200

AudioMixerAndroid 总结

概述

音频数据封装成 ActiveSound 类，由 AudioDevice 类维护一个 ActiveSound 的列表。

有一个专门的音频混合线程，输出混合后的数据

如果当前设备没有 ActiveSound，那么就填充全 0，有则填充有效数据

不论有无 ActiveSound，都会向 OutputBuffer 填充数据。

OpenSL 处理完 BufferQueue 之后，触发混合器注册的回调。该回调从 OutputBuffer 中拷贝数据到环形缓冲，然后把环形缓冲的数据通过 OpenSL 的 EnQueue 方法提交给 OpenSL。这样，OpenSL 又可以开始处理 BufferQueue 了。

路径

注册 BufferQueue 回调到 OpenSL 的路径：

FAudioDevice::Init -> FMixerDevice::InitializeHardware -> FMixerPlatformAndroid::OpenAudioStream

OpenSL 调用的 BufferQueue 回调的工作内容：

FMixerPlatformAndroid::OpenSLBufferQueueCallback -> IAudioMixerPlatformInterface::ReadNextBuffer -> FMixerPlatformAndroid::SubmitBuffer

SubmitBuffer 的工作内容是，从环形缓冲拷贝数据，然后把该数据再次 Enqueue 到 OpenSL

开始音乐播放的路径：

FAudioDevice::AddNewActiveSound -> FAudioDevice::AddNewActiveSoundInternal

它将要播放的 FActiveSound 放入一个待播放的队列

暂停音乐播放的路径：

FAudioDevice::StopActiveSound -> FAudioDevice::AddSoundToStop

它将要暂停的 FActiveSound 放入一个 pending 队列

填充音频缓冲的路径：

IAudioMixerPlatformInterface::RunInternal -> FOutputBuffer::MixNextBuffer -> FMixerDevice::OnProcessAudioStream

最终是对所有音频混音得到缓冲数据

UE Staff 对 AudioMixerAndroid 的介绍

https://forums.unrealengine.com/t/android-new-audiomixerandroid-in-4-17p1-results-in-crackling-audio/392002

他这里提到，如果出现噼啪声，是因为 OpenSL 缓冲区数据不够

为了解决这个问题，Staff 通过把一批缓冲入队来缓解（这算合批吗？）

这会有效果。但是安卓 CPU 性能欠佳，音频线程可能会被别的工作挤占，所以在负载很高的情况下，合批缓冲可能仍然不够。

更换音频后端

直接改 ini 是不够的。打包的时候，ini 会被刷新

看了下 DefaultEngine.ini，这里存了默认地图啥的，也没有相关的

1
2
3
4
5
6
7
8
9
10
bool FAudioDeviceManager::LoadDefaultAudioDeviceModule()
{
	check(!AudioDeviceModule);

	// Check if we're going to try to force loading the audio mixer from the command line
	bool bForceAudioMixer = FParse::Param(FCommandLine::Get(), TEXT("AudioMixer"));

	bool bForceNoAudioMixer = FParse::Param(FCommandLine::Get(), TEXT("NoAudioMixer"));

	bool bForceNonRealtimeRenderer = FParse::Param(FCommandLine::Get(), TEXT("DeterministicAudio"));

如果在 Configuration 的 Launch Option - Launch Flags 里面添加 -NoAudioMixer

会导致出现

1
2
Error running 'app'
Process ID com.YourCompany.<项目名> was not found. Aborting session

的错误

问了 AI，说是需要改成 --es CommandLine "-NoAudioMixer" 也没有效果

于是我在引擎源码直接改为 bForceNoAudioMixer 为 true

构建出来的引擎可以跑，日志中都有一些 Error 了

1
Error        LogAudio                  Submix buffer listener only works with the audio mixer. Please run with audio mixer enabled.

不过应该不影响

然后在这里打断点

1
2
3
if (!AudioDeviceModule && AudioDeviceModuleName.Len() > 0)
{
    AudioDeviceModule = FModuleManager::LoadModulePtr<IAudioDeviceModule>(*AudioDeviceModuleName);

可见他最终加载到 FSLESAudioDeviceModule

创建 FSLESAudioDevice

1
2
3
4
5
6
7
8
9
10
11
12
13
14
* frame #0: 0x0000007aa1887230 libUE4.so`FSLESAudioDeviceModule::CreateAudioDevice(this=0x00000079edc682b0) at AndroidAudioDevice.cpp:23:10
  frame #1: 0x0000007a9fc7b010 libUE4.so`FAudioDeviceManager::FAudioDeviceContainer::FAudioDeviceContainer(this=0x0000007b547e42d0, InParams=<unavailable>, InDeviceID=1, DeviceManager=0x00000079d49abd80) at AudioDeviceManager.cpp:1657:46
  frame #2: 0x0000007a9fc7537c libUE4.so`FAudioDeviceManager::CreateNewDevice(this=0x00000079d49abd80, InParams=0x0000007b547e4418) at AudioDeviceManager.cpp:651:28
  frame #3: 0x0000007a9fc75240 libUE4.so`FAudioDeviceManager::RequestAudioDevice(this=0x00000079d49abd80, InParams=0x0000007b547e4418) at AudioDeviceManager.cpp:465:10
  frame #4: 0x0000007a9fc76650 libUE4.so`FAudioDeviceManager::CreateMainAudioDevice(this=0x00000079d49abd80) at AudioDeviceManager.cpp:552:27
  frame #5: 0x0000007aa0c53b3c libUE4.so`UEngine::InitializeAudioDeviceManager(this=0x00000079d30850b0) at UnrealEngine.cpp:3111:23
  frame #6: 0x0000007aa0c41aac libUE4.so`UEngine::Init(this=<unavailable>, InEngineLoop=<unavailable>) at UnrealEngine.cpp:1643:2
  frame #7: 0x0000007aa022c9b0 libUE4.so`UGameEngine::Init(this=0x00000079d30850b0, InEngineLoop=<unavailable>) at GameEngine.cpp:1072:11
  frame #8: 0x0000007a9b58d158 libUE4.so`FEngineLoop::Init(this=<unavailable>) at LaunchEngineLoop.cpp:4017:12
  frame #9: 0x0000007a9b58c458 libUE4.so`AndroidMain(state=<unavailable>) at LaunchAndroid.cpp:501:14
  frame #10: 0x0000007a9b59c0fc libUE4.so`android_main(state=0x0000007ab106db40) at LaunchAndroid.cpp:777:2
  frame #11: 0x0000007a9b5cb5b8 libUE4.so`android_app_entry(param=0x0000007ab106db40) at android_native_app_glue.c:233:5
  frame #12: 0x0000007c068ab2a8 libc.so`__pthread_start(void*) + 200
  frame #13: 0x0000007c0689c624 libc.so`__start_thread + 68

初始化

1
2
3
4
5
6
7
8
9
10
11
* frame #0: 0x0000007aa1881cd4 libUE4.so`FSLESAudioDevice::InitializeHardware(this=0x00000079d4162b80) at AndroidAudioDevice.cpp:77:2
  frame #1: 0x0000007a9fe6c000 libUE4.so`FAudioDevice::Init(this=0x00000079d4162b80, InDeviceID=<unavailable>, InMaxSources=<unavailable>) at AudioDevice.cpp:513:7
  frame #2: 0x0000007a9fc7b054 libUE4.so`FAudioDeviceManager::FAudioDeviceContainer::FAudioDeviceContainer(this=0x0000007b547e42d0, InParams=<unavailable>, InDeviceID=1, DeviceManager=<unavailable>) at AudioDeviceManager.cpp:1672:14
  frame #3: 0x0000007a9fc7537c libUE4.so`FAudioDeviceManager::CreateNewDevice(this=0x00000079d49abd80, InParams=0x0000007b547e4418) at AudioDeviceManager.cpp:651:28
  frame #4: 0x0000007a9fc75240 libUE4.so`FAudioDeviceManager::RequestAudioDevice(this=0x00000079d49abd80, InParams=0x0000007b547e4418) at AudioDeviceManager.cpp:465:10
  frame #5: 0x0000007a9fc76650 libUE4.so`FAudioDeviceManager::CreateMainAudioDevice(this=0x00000079d49abd80) at AudioDeviceManager.cpp:552:27
  frame #6: 0x0000007aa0c53b3c libUE4.so`UEngine::InitializeAudioDeviceManager(this=0x00000079d30850b0) at UnrealEngine.cpp:3111:23
  frame #7: 0x0000007aa0c41aac libUE4.so`UEngine::Init(this=<unavailable>, InEngineLoop=<unavailable>) at UnrealEngine.cpp:1643:2
  frame #8: 0x0000007aa022c9b0 libUE4.so`UGameEngine::Init(this=0x00000079d30850b0, InEngineLoop=<unavailable>) at GameEngine.cpp:1072:11
  frame #9: 0x0000007a9b58d158 libUE4.so`FEngineLoop::Init(this=<unavailable>) at LaunchEngineLoop.cpp:4017:12
  frame #10: 0x0000007a9b58c458 libUE4.so`AndroidMain(state=<unavailable>) at LaunchAndroid.cpp:501:14

在 FSLESAudioDevice::InitializeHardware 这里其实也是有 Mix 的，他创建了 SL_OutputMixObject 对象

FSLESAudioDevice 播放与暂停

但是在 <引擎根目录>\Engine\Source\Runtime\Android\AndroidAudio\Private\AndroidAudioDevice.cpp 看不到其他的 Play 方法之类的

于是蹲守 UAudioComponent::PlayInternal 的 AudioDevice->AddNewActiveSound(NewActiveSound);

这个时候进入的 FAudioDevice 其实就是 FSLESAudioDevice

哦……仍然是管理 ActiveSound 的逻辑，很合理，FAudioDevice 是基类

搜索 )->RegisterCallback( 看看 FSLESAudioDevice 怎么绑定回调的？

哦，于是看到了 Engine/Source/Runtime/Android/AndroidAudio/Private/AndroidAudioSource.cpp 底下有播放相关的代码和 RegisterCallback 相关

他这个就是属于 AndroidAudio 模块的，所以确实是这里

于是在 FSLESSoundSource::Play 打断点，看到堆栈

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
* frame #0: 0x0000007aa1886614 libUE4.so`FSLESSoundSource::Play(this=0x00000079d39c5500) at AndroidAudioSource.cpp:537:24
  frame #1: 0x0000007a9fe85f94 libUE4.so`FAudioDevice::StartSources(this=0x00000079d4162b80, WaveInstances=size=1, FirstActiveIndex=<unavailable>, bGameTicking=true) at AudioDevice.cpp:4080:17
  frame #2: 0x0000007a9fe871e0 libUE4.so`FAudioDevice::Update(this=0x00000079d4162b80, bGameTicking=<unavailable>) at AudioDevice.cpp:4347:3
  frame #3: 0x0000007a9b5afedc libUE4.so`TGraphTask<TFunctionGraphTaskImpl<void (), (ESubsequentsMode::Type)0>>::ExecuteTask(TArray<FBaseGraphTask*, TSizedDefaultAllocator<32>>&, ENamedThreads::Type) [inlined] UE4Function_Private::TFunctionRefBase<UE4Function_Private::TFunctionStorage<true>, void ()>::operator()(this=<unavailable>) const at Function.h:676:11
  frame #4: 0x0000007a9b5afeb4 libUE4.so`TGraphTask<TFunctionGraphTaskImpl<void (), (ESubsequentsMode::Type)0>>::ExecuteTask(TArray<FBaseGraphTask*, TSizedDefaultAllocator<32>>&, ENamedThreads::Type) [inlined] TFunctionGraphTaskImpl<void (), (ESubsequentsMode::Type)0>::DoTaskImpl(Function=<unavailable>, CurrentThread=AudioThread, MyCompletionGraphEvent=0x00000079eddfc178) at TaskGraphInterfaces.h:1368
  frame #5: 0x0000007a9b5afeb4 libUE4.so`TGraphTask<TFunctionGraphTaskImpl<void (), (ESubsequentsMode::Type)0>>::ExecuteTask(TArray<FBaseGraphTask*, TSizedDefaultAllocator<32>>&, ENamedThreads::Type) [inlined] TFunctionGraphTaskImpl<void (), (ESubsequentsMode::Type)0>::DoTask(this=<unavailable>, CurrentThread=AudioThread, MyCompletionGraphEvent=0x00000079eddfc178) at TaskGraphInterfaces.h:1361
  frame #6: 0x0000007a9b5afeb0 libUE4.so`TGraphTask<TFunctionGraphTaskImpl<void (), (ESubsequentsMode::Type)0>>::ExecuteTask(this=0x00000079eddfc0e0, NewTasks=<unavailable>, CurrentThread=<unavailable>) at TaskGraphInterfaces.h:886
  frame #7: 0x0000007a9c7d8888 libUE4.so`FNamedTaskThread::ProcessTasksNamedThread(int, bool) [inlined] FBaseGraphTask::Execute(this=<unavailable>, NewTasks=<unavailable>, CurrentThread=AudioThread) at TaskGraphInterfaces.h:524:3
  frame #8: 0x0000007a9c7d8858 libUE4.so`FNamedTaskThread::ProcessTasksNamedThread(this=<unavailable>, QueueIndex=<unavailable>, bAllowStall=<unavailable>) at TaskGraph.cpp:710
  frame #9: 0x0000007a9c7d6f84 libUE4.so`FNamedTaskThread::ProcessTasksUntilQuit(this=0x0000007ab52d9ba0, QueueIndex=0) at TaskGraph.cpp:601:4
  frame #10: 0x0000007a9fec8eb8 libUE4.so`FAudioThread::Run() [inlined] AudioThreadMain(TaskGraphBoundSyncEvent=0x00000079d31715e0) at AudioThread.cpp:104:29
  frame #11: 0x0000007a9fec8e78 libUE4.so`FAudioThread::Run(this=<unavailable>) at AudioThread.cpp:210
  frame #12: 0x0000007a9c8f5698 libUE4.so`FRunnableThreadPThread::Run(this=0x00000079d3113fa0) at PThreadRunnableThread.cpp:25:24
  frame #13: 0x0000007a9c7d2078 libUE4.so`FRunnableThreadPThread::_ThreadProc(pThis=0x00000079d3113fa0) at PThreadRunnableThread.h:185:15
  frame #14: 0x0000007c068ab2a8 libc.so`__pthread_start(void*) + 200
  frame #15: 0x0000007c0689c624 libc.so`__start_thread + 68

似乎涉及到的就是

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
bool FSLESSoundSource::EnqueuePCMBuffer( bool bLoop)
{
	SLresult result;
	// If looping, register a callback to requeue the buffer
	if( bLoop ) 
	{
		result = (*SL_PlayerBufferQueue)->RegisterCallback(SL_PlayerBufferQueue, OpenSLBufferQueueCallback, (void*)this);
		if (result != SL_RESULT_SUCCESS) { UE_LOG(LogAndroidAudio, Warning, TEXT("FAILED OPENSL BUFFER QUEUE RegisterCallback 0x%x "), result); return false; }
	}
	
	result = (*SL_PlayerBufferQueue)->Enqueue(SL_PlayerBufferQueue, SLESBuffer->AudioData, SLESBuffer->GetSize() );
	if (result != SL_RESULT_SUCCESS) {
		UE_LOG(LogAndroidAudio, Warning, TEXT("FAILED OPENSL BUFFER Enqueue SL_PlayerBufferQueue 0x%x params( %p, %d)"), result, SLESBuffer->AudioData, int32(SLESBuffer->GetSize()));
		if (bLoop)
		{
			result = (*SL_PlayerBufferQueue)->RegisterCallback(SL_PlayerBufferQueue, NULL, NULL);
		}
		return false;
	}

还有一个 EnqueuePCMRTBuffer 似乎是变体，就跳过了

于是去看 FSLESSoundSource::OnRequeueBufferCallback

分为流式传输或者非流式传输

如果是非流式传输，那么直接 EnQueue 同一个缓冲，那就是循环播放本音频

如果是流式传输，那么从解码任务（可能异步）那边获得一个缓冲区来 EnQueue，然后再去解码

在我的 case 中，我啥也没配置，直接是 AudioComp 播放，调试居然也进入流式传输

1
2
3
4
5
6
7
8
* frame #0: 0x0000007aa1882d24 libUE4.so`FSLESSoundSource::OnRequeueBufferCallback(this=0x00000079d39c5500, InQueueInterface=<unavailable>) at AndroidAudioSource.cpp:34:7
  frame #1: 0x0000007bf7ceaaa0 libwilhelm.so`android::AudioTrackCallback::onMoreData(android::AudioTrack::Buffer const&) - 18446743541271057759
  frame #2: 0x0000007c01f1c398 libaudioclient.so`android::AudioTrack::processAudioBuffer() + 3304
  frame #3: 0x0000007c01f1b354 libaudioclient.so`android::AudioTrack::AudioTrackThread::threadLoop() + 340
  frame #4: 0x0000007be492b5cc libutils.so`android::Thread::_threadLoop(void*) + 252
  frame #5: 0x0000007bf99a2490 libandroid_runtime.so`android::AndroidRuntime::javaThreadShell(void*) + 144
  frame #6: 0x0000007c068ab2a8 libc.so`__pthread_start(void*) + 200
  frame #7: 0x0000007c0689c624 libc.so`__start_thread + 68

它读取数据的方法 FSLESSoundSource::ReadMorePCMData 我是调试跳到 FSLESSoundBuffer::ReadCompressedData -> FVorbisAudioInfo::ReadCompressedData。这里面用了 ov_read 来实现数据读取

Stop 的时候就把 OpenSL BufferQueue 回调注销了

Prev
2025-06-01 10:52:45 #Cpp #Unreal Engine 4 #OpenSL ES
Next