PaulsCode Forum

General => Other Support => Topic started by: CreativeMD on May 02, 2017, 04:50:20 PM

Title: SoundEngine loop stream sound
Post by: CreativeMD on May 02, 2017, 04:50:20 PM
Hey,
I'm having issues to loop a streaming source using the sound engine. I'm currently programming a mod for minecraft. Somehow the sound will not be repeat, but instead end after playing once. I'm not sure if that's an issue from minecraft or a bug of the sound engine. I tried to find out the cause of it, but unfortunately couldn't find anything.
So maybe you can help out?

In Regards
CreativeMD
Title: Re: SoundEngine loop stream sound
Post by: Paul on May 02, 2017, 09:24:15 PM
I can take a look to see if there is a bug.  Is the audio file available somewhere for download?
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 03, 2017, 04:05:31 AM
Here is one example: https://www.dropbox.com/s/d6ww6niprdfqzfi/light-wind.ogg?dl=1
Title: Re: SoundEngine loop stream sound
Post by: Paul on May 03, 2017, 08:25:28 AM
Thanks, got it.  Which library and codec plugins are you using?
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 03, 2017, 04:30:07 PM
Found it:
Code: [Select]
SoundSystemConfig.addLibrary(LibraryLWJGLOpenAL.class);
SoundSystemConfig.setCodec("ogg", CodecJOrbis.class);
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 09, 2017, 07:57:00 AM
Could find anything out?
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 15, 2017, 06:56:50 AM
Ok, it looks like it's a weird Minecraft issue. Thanks for your help anyway. Wish you all the best.
Title: Re: SoundEngine loop stream sound
Post by: Paul on May 15, 2017, 07:23:55 AM
No problem.  Sorry for the delayed response.  I have been busy with work and didn't get a chance to look into it yet.

On a side note, I have started teaching my son programming, and we are writing a simple Java game.  As a result, I'll probably get back to updating elements of the SoundSystem library that have become outdated.
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 16, 2017, 04:59:07 PM
Sounds great :D

Unfortunately there are lot of issue with Minecraft's implementation. They use com.google.common.collect.HashBiMap to store all playing sound. Basically those are two HashMap connected together, one for finding the sound for a given source-name and one to get the source-name from a sound (reversed). Theoretically those should be equal, but eventually i found out that those HashMap's are not synchronized directly, but over time. So there are "threading" issues without using threads.

Example: If you play a sound and check if it is playing, it will return false for about 1-2 secs (depends on FPS) until the second hashmap will be updated. No idea who wrote that code ... but that person didn't know what those HashBiMap are used for.
Title: Re: SoundEngine loop stream sound
Post by: Paul on May 16, 2017, 09:06:51 PM
The issue with "playing()" method not returning true immediately, that is code I wrote.  This came as a result of the "command queue" framework which defers commands to a separate thread rather than blocking the current thread.  I believe my thought there was that the source isn't actually playing until the command thread executes the play command.  This gives you the opportunity to potentially poll and determine when the source actually starts playing.

I'm not sure this is the best behavior though.  In your opinion, would you rather this function return "true" immediately after calling one of the play functions?  Maybe a callback would be a better solution for anyone interested in when the play command actually gets executed.  Note that I was new to Java when I wrote the SoundSystem library (there are several things I might have done differently if I were to rewrite it) :)
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 17, 2017, 04:38:39 AM
No, i was not talking about your sound engine. I mean your engine needs to be thread safe so it's a good idea to use a command queue for that.

It's about the SoundManager class:
Minecraft has an interface (ISound) for all sounds. It contains the location, pitch, volume etc. The SoundManager connects the minecraft way of using sounds with your sound engine. Once a sound will be played, minecraft generates a uuid for it and places it together with the ISound object in a hashmap (Map<String,ISound>). Through this way they can link a given uuid to a sound, but most commonly you would need it the other way round, that's why they are using a reversed hashmap (Map<ISound, string>) a so called HashBiMap (library from google). Those two hashmaps should be equal, but reversed. Sadly this is not the cause, since they will not be synchronized immediately, which causes a tone amount of issues.

I'm currently rewriting my mod AmbientSounds (https://minecraft.curseforge.com/projects/ambientsounds (https://minecraft.curseforge.com/projects/ambientsounds)), which adds some more atmospheric sounds to minecraft. Theoretically it should play a sound, loop it until it will be stopped at some point, but that does not really work since minecraft acts quite strange (stopping sounds for no reason, continues to play the sound from the start every 2 seconds). :(

So all i have to do know, is to create my own connector to your sound engine and everything should work fine :D
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 22, 2017, 10:18:12 AM
Hey,

unfortunately there seem to be some more issues with the sound engine itself. Since the minecraft implementation has lot of bugs, i decided to write my own, but as i said before ran into some problems.

There seems to be one big issue and that is to tell whether a sound is playing or not. As you told me already, playing() will not return true immediately, which isn't that big kind of a problem. I could solve it by waiting until playing() is true for the first time.

To my concerns that did not fix it
- sometimes a sound does not event start to play. This occurs randomly (no idea why)
- playing() returns false although the sound is still playing (somehow depends on how many sounds are playing. It's difficult to find a pattern, but sometimes all the sounds return false for 1-4 seconds, eventually the seems to recover from that). I could record a video to demonstrate you what is happening

I also tried to use library directly (get the source), but that does not change anything. I tested it for hours tried to find a solution, but sadly without any success.

Maybe you have an idea? What i would need is a 100% safe method which tells me if a sound is playing or not, everything else works just fine.
Title: Re: SoundEngine loop stream sound
Post by: Paul on May 22, 2017, 12:28:54 PM
Do you have some sample code for the simplest test case you can simulate the behavior?  In particular playing() returning false when the source is actually playing.

Off hand I'm thinking one issue might be the library running out of available channels.  Typically the sum of streaming and normal channels adds up to 32.  By default I think 4 are reserved for streaming channels and the rest for normal channels if I recall correctly.  If you play more sources than the limit, it will stop a playing channel and replace the source that was playing on it with the newest source.  The old one would show playing() false and the new one true.

Sounds like there may be other issues though, since you mentioned playing() returning false for 1-4 seconds, which is much higher than I have experienced.  Maybe something is broken in newer Java versions (the library is quite out dated at this point).
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 23, 2017, 11:11:41 AM
Ok ... it looks like things get a little bit more complicated. It might be a good idea if i tell you exactly what is happening and show you my code.

So AmbientSounds (the mod i'm working on) should play sounds which should fit to the current area the player is. An area has about 3 sounds which play contentiously and a few others which play from time to time.

I recorded two videos, which should show you what is going. I'm trying to bring the system to it's limits by switching the areas rather quickly, through that way more sounds will be played simultaneously. The console (in eclipse) shows which sounds are currently playing.

In this video a random crash of the sound engine occurs:
https://www.youtube.com/watch?v=v9Nfe8RYxeA

Code: [Select]
[17:36:04] [Thread-11/INFO]: [STDERR]: java.lang.NullPointerException
[17:36:04] [Thread-11/INFO]: [STDERR]: at com.jcraft.jogg.StreamState.packetout(StreamState.java:186)
[17:36:04] [Thread-11/INFO]: [STDERR]: at paulscode.sound.codecs.CodecJOrbis.readBytes(CodecJOrbis.java:575)
[17:36:04] [Thread-11/INFO]: [STDERR]: at paulscode.sound.codecs.CodecJOrbis.read(CodecJOrbis.java:354)
[17:36:04] [Thread-11/INFO]: [STDERR]: at paulscode.sound.Source.stream(Source.java:953)
[17:36:04] [Thread-11/INFO]: [STDERR]: at paulscode.sound.StreamThread.run(StreamThread.java:129)

The second video shows the problem i was talking about before: https://www.youtube.com/watch?v=9TmNl-es50k at around 0:40 the engine unexpectedly ends some of the sounds.

Code: [Select]
[17:40:56] [Client thread/INFO]: [STDOUT]: ================Playing================
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:animals.bird.bird-jungle-night, 0.023/0.0
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:animals.cricket.cricket-jungle-night, 0.023/0.0
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:animals.cricket.cricket-forest-night, 0.023/0.0
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:animals.cricket.cricket-warm-night, 0.5029971/0.0
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:animals.cricket.cricket, 0.0019999743/0.0
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:animals.wolf.wolf, 0.057999957/0.0 pause: 136
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:animals.seagull.seagull, 0.10300003/0.75 pause: 12
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:animals.seagull.seagull-long, 0.0/0.75 pause: 737
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:water.beach, 0.09900002/0.4
[17:40:56] [Client thread/INFO]: [STDOUT]: ambientsounds:wind.light-wind, 0.09900002/1.0
[17:40:56] [Client thread/INFO]: [STDOUT]: Unexpected ending sound ambientsounds:animals.cricket.cricket af7699ba-c5c7-4bf5-ae2d-298220803624
[17:40:56] [Client thread/INFO]: [STDOUT]: Unexpected ending sound ambientsounds:animals.cricket.cricket-jungle-night 5806f294-3cc4-43ac-8702-73a543136fd1

This seems to persist until those sounds fade out (since i left the area).

The code i'm currently using is a bit chaotic. I tried a lot of things to found out if a sound has stopped or not:
https://github.com/CreativeMD/AmbientSounds/blob/1.11/src/main/java/com/creativemd/ambientsounds/AmbientSoundEngine.java#L92. I hope i could explain my situation, so you can understand it.

Not having enough streaming channels might be the issue here, is there a way to increase it?


Title: Re: SoundEngine loop stream sound
Post by: Paul on May 23, 2017, 12:27:07 PM
Thanks, I''ll look into this when I get a chance.

Yes, you can increase the number of streaming channels using something like:

Code: [Select]
SoundSystemConfig.setNumberStreamingChannels( 11 );
SoundSystemConfig.setNumberNormalChannels( 21 );

Just have the total number add up to 32.  You'll need to do this before you instantiate the SoundSystem.  The logic for streaming channels has to be a bit more responsive than for normal channels, so depending on the speed of your system and what all else you have going on, you might start hearing gaps in the streams if it can't keep up with the number of channels you have set.
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on May 24, 2017, 05:32:43 AM
Yes!!! Thank you sooo much. It seems to be fixed, everything works just fine: https://youtu.be/Z7GmncqhGpI

Now there is only this random crash, but i don't know if there is a way to fix it.

Again thank you so much for all your spent time and effort! :D You really helped me, i have been trying to solve this issue for weeks. Now i can finally release! I wish you all the best for your future projects. Thank you.
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on June 19, 2017, 10:07:28 AM
Hey there, I'm sorry to bother you again, but unfortunately this random crash (which i mentioned earlier) has become quiet annoying.

Just as a reminder I'm talking about this one:
Code: [Select]
[17:36:04] [Thread-11/INFO]: [STDERR]: java.lang.NullPointerException
[17:36:04] [Thread-11/INFO]: [STDERR]: at com.jcraft.jogg.StreamState.packetout(StreamState.java:186)
[17:36:04] [Thread-11/INFO]: [STDERR]: at paulscode.sound.codecs.CodecJOrbis.readBytes(CodecJOrbis.java:575)
[17:36:04] [Thread-11/INFO]: [STDERR]: at paulscode.sound.codecs.CodecJOrbis.read(CodecJOrbis.java:354)
[17:36:04] [Thread-11/INFO]: [STDERR]: at paulscode.sound.Source.stream(Source.java:953)
[17:36:04] [Thread-11/INFO]: [STDERR]: at paulscode.sound.StreamThread.run(StreamThread.java:129)

It causes the sound engine to crash from time to time, so all sounds will be stopped. The sound engine has to be restarted in order to make it working again.

Other mod authors are aware of this issue too, but none of us knows how to fix it. My guess is, that this is a weird thread issue, which is rather unlikely to occur, but will eventually happen if you play the game long enough. Of course the more streaming sounds will be played the more likely it is to happen.

Maybe you can help us out here? Do you have an idea what could cause this crash and how to fix it?

If not, I'm searching for a proper way to catch the exception and try to end the stream properly without causing the engine to crash as well, but unfortunately it does not work out, at least as far as i can tell, since it's rather difficult to reproduce this crash:
Code: [Select]
private byte[] readBytes()
    {
        if( !initialized( GET, XXX ) )
            return null;

        if( endOfStream( GET, XXX ) )
            return null;

        if( convertedBuffer == null )
            convertedBuffer = new byte[ convertedBufferSize ];
        byte[] returnBuffer = null;

        float[][] pcmf;
        int samples, bout, ptr, mono, val, i, j;

        switch( joggSyncState.pageout( joggPage ) )
        {
            case( 0 ):
            case( -1 ):
                break;
            default:
            {
                joggStreamState.pagein( joggPage );
                if( joggPage.granulepos() == 0 )
                {
                    endOfStream( SET, true );
                    return null;
                }
               
                try{
                processPackets: while( true )
                {
                    switch( joggStreamState.packetout( joggPacket ) )
                    {
                        case( 0 ):
                            break processPackets;
                        case( -1 ):
                            break;
                        default:
                        {
                            if( jorbisBlock.synthesis( joggPacket ) == 0 )
                                jorbisDspState.synthesis_blockin( jorbisBlock );

                            while( ( samples=jorbisDspState.synthesis_pcmout(
                                                     pcmInfo, pcmIndex ) ) > 0 )
                            {
                                pcmf = pcmInfo[0];
                                bout = ( samples < convertedBufferSize ?
                                                samples : convertedBufferSize );
                                for( i = 0; i < jorbisInfo.channels; i++ )
                                {
                                    ptr = i * 2;
                                    mono = pcmIndex[i];
                                    for( j = 0; j < bout; j++ )
                                    {
                                        val = (int) ( pcmf[i][mono + j] *
                                                                       32767. );
                                        if( val > 32767 )
                                            val = 32767;
                                        if( val < -32768 )
                                            val = -32768;
                                        if( val < 0 )
                                            val = val | 0x8000;
                                        convertedBuffer[ptr] = (byte) (val);
                                        convertedBuffer[ptr+1] =
                                                               (byte) (val>>>8);
                                        ptr += 2 * (jorbisInfo.channels);
                                    }
                                }
                                jorbisDspState.synthesis_read( bout );

                                returnBuffer = appendByteArrays( returnBuffer,
                                               convertedBuffer,
                                               2 * jorbisInfo.channels * bout );
                            }
                        }
                    }
                }
               
                }catch(Exception e){
                e.printStackTrace();
                endOfStream( SET, true );
                return null;
                }

                if( joggPage.eos() != 0 )
                    endOfStream( SET, true );
            }
        }

Is "endOfStream( SET, true );" a proper way of stopping the sound or will it cause future issues?
Title: Re: SoundEngine loop stream sound
Post by: Paul on June 19, 2017, 11:05:21 PM
Do you know what specifically is null when that exception is thrown?  If lucky, a null check might fix the problem.  Besides a thread issue, possibly other causes might be an empty buffer or index out of bounds problem (i.e. some corner case in the read process).  Maybe try updating JOrbis if there is a more recent version.

If you can't solve it, setting end of stream should be fine I think.
Title: Re: SoundEngine loop stream sound
Post by: CreativeMD on June 21, 2017, 02:53:48 AM
The newest version of JOrbis is 0.0.17 and from 2008, i'm pretty sure you are already using the newest version.

The crash itself takes place here:
Code: [Select]
public int packetout(Packet op){

    /* The last part of decode. We have the stream broken into packet
       segments.  Now we need to group them into packets (or return the
       out of sync markers) */

    int ptr=lacing_returned;

    if(lacing_packet<=ptr){
      return (0);
    }

    if((lacing_vals[ptr]&0x400)!=0){ //<- this line is causing the exception
      /* We lost sync here; let the app know */
      lacing_returned++;

      /* we need to tell the codec there's a gap; it might need to
         handle previous packet dependencies. */
      packetno++;
      return (-1);
    }

    /* Gather the whole packet. We'll have no holes or a partial packet */
    {
      int size=lacing_vals[ptr]&0xff;
      int bytes=0;

      op.packet_base=body_data;
      op.packet=body_returned;
      op.e_o_s=lacing_vals[ptr]&0x200; /* last packet of the stream? */
      op.b_o_s=lacing_vals[ptr]&0x100; /* first packet of the stream? */
      bytes+=size;

      while(size==255){
        int val=lacing_vals[++ptr];
        size=val&0xff;
        if((val&0x200)!=0)
          op.e_o_s=0x200;
        bytes+=size;
      }

      op.packetno=packetno;
      op.granulepos=granule_vals[ptr];
      op.bytes=bytes;

      body_returned+=bytes;

      lacing_returned=ptr+1;
    }
    packetno++;
    return (1);
  }

I think catching the exception is the best way to do it right know. Future tests will tell if that fixes the crash of the sound engine or not.

As i said earlier, my guess it's a threading issue. The codec will cleanup everything, but still tries to continue to stream. (maybe once he runs out of streaming channels).

Thanks again for your help, if you ever run across a solution for this problem just let me know :D . I really appreciate your work and your spent time. Have a good day.

EDIT
Am I allowed to upload a modified version of your CodecJOrbis on github (open source)? I would also mark all changes i made.
Title: Re: SoundEngine loop stream sound
Post by: Paul on June 21, 2017, 08:15:41 AM
No problem with uploading modified code.. the license is basically a "do whatever you want" license (only restriction really is that if you happen to give me attribution, be sure to mention the website here).