parallel pcm audio streams minecraft 1.8.9 problem

[DevTastisch]

Hey,

I am currently working on an Opus codec based voicechat modification for minecraft 1.8.9.
While preparing an audio stream for a player I do the following:

this.sndSystem.rawDataStream(audioFormat, true, audioSourceName, 0f, 0f, 0f, SoundSystemConfig.ATTENUATION_LINEAR, this.voiceChat.getSurroundRange());


When receiving audio data from another client I try to play it with this:

this.sndSystem.feedRawAudioData(audioSourceName, latencyBuffer.get());
this.sndSystem.play(audioSourceName);

And so far this works like a charm, while the client only receives one single audio stream.
But as soon as I start streaming a second audio from another client, that system fails.

It feels like this method call is synchronized. It is clearly to hear, that one audio chunk from stream1 is played, and then one from stream2 and from there it goes in a loop.

I captured my test. At first you can hear a 100hz sin wave for a few seconds. This sounds very clear.
After that you can hear a 1000hz sin wave. This is very clear too. In the End both sounds are combined.

Both sounds do have different source names and are called from different threads. But it is clear to hear that the streams are not merged or something like that. It is more like a race condition between them who can playback the audio chunk first.

I know minecraft 1.8.9 is quite old and there are probably some library updates since release, but I couldn't find any documentation to that so I thought I ask here for some help.

[Paul]

This seems like what would happen if both streams were trying to play on the same channel.  Do you know how many streaming channels you have configured?  I think by default it is set at 4, so should be that if you haven't changed it.  I suppose another problem could be the particular mixer on that system.

Couple of tests that might provide more information about the problem:

1) If you play two different streams from the same thread, do you get the same behavior?

2) Do you experience similar behavior if you play two normal (non-streaming) sources at the same time?

[DevTastisch]

Hey,

thanks for your fast reply,
I dont know how many channels I have configured. I didn't change that. So it should be the default.

1) I tried playing streams from the same thread and the same problem occurs exactly the way it does while multithreading.

2) I only experience that behavior while using streaming sources

[DevTastisch]

okay I noticed something very very weird. I am not sure if I am doing something wrong here or if it is your library.
I created a little dummy project to experiment with the sound system and it looks like this atm:

- Note that it is not the final usecase to play raw pcm files, but little pcm frames after one and another. -
But the problem is the same

Code: newbielink:javascript:void(0); [nonactive]

    File pcmData = ...;
    ScheduledExecutorService executorService = Executors.newScheduledThreadPool(0);

    SoundSystemConfig.addLibrary(LibraryJavaSound.class);
    SoundSystemConfig.setCodec("wav", CodecWav.class);
    SoundSystemConfig.setNumberStreamingChannels(5);
    SoundSystemConfig.setNumberNormalChannels(27);

    SoundSystem mySoundSystem = new SoundSystem();

    mySoundSystem.rawDataStream(
        new AudioFormat(AudioFormat.Encoding.PCM_SIGNED, 48000, 16, 1, 2, 48000, false),
        false,
        "test1",
        0,
        0,
        0,
        SoundSystemConfig.ATTENUATION_ROLLOFF,
        SoundSystemConfig.getDefaultRolloff());

    mySoundSystem.rawDataStream(
        new AudioFormat(AudioFormat.Encoding.PCM_SIGNED, 48000, 16, 1, 2, 48000, false),
        false,
        "test2",
        0,
        0,
        0,
        SoundSystemConfig.ATTENUATION_ROLLOFF,
        SoundSystemConfig.getDefaultRolloff());

    executorService.submit(
        () -> {
          try {
            mySoundSystem.feedRawAudioData("test1", new byte[] {});
            mySoundSystem.feedRawAudioData("test1", Files.toByteArray(pcmData));
            mySoundSystem.play("test1");
          } catch (IOException e) {
            e.printStackTrace();
          }
        });

    executorService.schedule(
        () -> {
          try {
            mySoundSystem.feedRawAudioData("test2", new byte[] {});
            mySoundSystem.feedRawAudioData("test2", Files.toByteArray(pcmData));
            mySoundSystem.play("test2");
          } catch (Exception ex) {
            ex.printStackTrace();
          }
        },
        2,
        TimeUnit.SECONDS);

While doing this, I would expect the file to be played once at the beginning and a second time starting 2 seconds after the first in parallel. But it does not. It does not even cut off the other source, but rather waits until the first is finished until it finally plays the second time. I have also notices the feedRawAudioData method is blocking when trying to play the second sound until the first one has finished.

This behavior is exactly the same I have noticed previously with pcm audio chunks in my voicechat.

Oh and I started debugging if both sources are playing on the same channel: They absolutely don't. They all have their own channel how it should be.
It looks like the whole command queue thread of the sound system is interrupted while playing some audio there.

I really dont think this was done on purpose.

[Paul]

Could you email me the full program (with the source of pcmData).  My address is "paul" at this domain.  I'll see if I experience the same behavior (if so I should be able to debug it more easily, and if not that also is still useful to know)

[DevTastisch]

Done, should be in your inbox.