Author Topic: Having trouble creating a rawDataStream through the 3D Sound System  (Read 7947 times)

Offline lukesomething

  • bit
  • Posts: 7
    • View Profile
I've been tinkering with the 3D sound system for a little while now, and I'm trying to stream data to a 3D source. I've tried lots of things, like emulating a saw wave, streaming (what I think is) PCM data from a file, and even just filling up a byte array with random bytes in the hope of simply squeezing out a wee bit of sound. No luck what so ever. Note that I'm working off of an existing codebase (namely minecraft, if that's relevant), so that might have something to do with it.

This is what I've been doing, mostly, as that's what makes the most sense to me. I was unable to find full examples of how to use the rawDataStream however, so I might be completely off on this.

Code: [Select] [nofollow]
system.rawDataStream(new AudioFormat(sampleRate, sampleSize, channels, false, true), true, "hue", x, y, z, 0, 10000);
system.setVolume("hue", 1.0f);
system.feedRawAudioData("hue", data); // The data is simply a 1024^2 byte long byte array filled with random data in the range of 0..255
system.play("hue");

My main question is, am I missing something crucial here?

Offline lukesomething

  • bit
  • Posts: 7
    • View Profile
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #1 on: July 31, 2013, 01:39:27 PM »
Bump? I would like some help/clarification on this. If it helps, all other sounds play like normal, and no errors show up in the console. Only error I managed to get out of it was when I mistakenly fed it data in the range of 0..256 (something along the lines of "invalid parameter").

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #2 on: July 31, 2013, 10:26:17 PM »
Sorry, I was going to do some testing with random data before answering, but haven't had any time.  My guess is that based on whatever audio format you are specifying, the random data is either at a pitch or volume that isn't audible, or otherwise invalid.  But that is only a guess.  Could you run a test where you feed actual audio data to the stream, in its proper format?  For example, something like this (use a short duration audio file that you know works with normal sources):

Code: [Select]
        try
        {
            SoundSystemConfig.addLibrary( LibraryLWJGLOpenAL.class );
            SoundSystemConfig.setCodec( "wav", CodecWav.class );
        }
        catch( SoundSystemException e )
        {}
        soundSystem = new SoundSystem();
        FilenameURL filenameURL = new FilenameURL( "tada.wav" );
        ICodec codec = SoundSystemConfig.getCodec( filenameURL.getFilename() );
        codec.reverseByteOrder( true );  // required by OpenAL, but not JavaSound
        URL url = filenameURL.getURL();
        codec.initialize( url );
        AudioFormat audioFormat = codec.getAudioFormat();
        SoundBuffer buffer = codec.readAll();
        codec.cleanup();
        soundSystem.rawDataStream( audioFormat, false, "source_1", 0, 0, 0,
                               SoundSystemConfig.ATTENUATION_ROLLOFF,
                               SoundSystemConfig.getDefaultRolloff() );
        soundSystem.feedRawAudioData( "source_1", buffer.audioData );
        soundSystem.feedRawAudioData( "source_1", buffer.audioData );
        soundSystem.feedRawAudioData( "source_1", buffer.audioData );
        soundSystem.play( "source_1" );

The above should play the sound three times.  If it works, then it means there is either a problem with your audio data or with the audio format you specified.  If it doesn't work, then it means there is something wrong with raw data streams.
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline lukesomething

  • bit
  • Posts: 7
    • View Profile
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #3 on: August 01, 2013, 09:59:50 AM »
The above example worked splendid. No issues. I still find myself unable to produce any kind of raw "PCM compatible" data. I took the above example, stripped out the bits where it read form the file, added some of my own magic, and ended up with this:
Code: [Select] [nofollow]
Random r = new Random();
        int sampleRate = 16000;
        int sampleSize = 8;
        int time = 8; // in seconds
        AudioFormat format = new AudioFormat(sampleRate, sampleSize, 1, false, false); // unsigned PCM little endian
        int sampledDataSize = sampleRate * time * (sampleSize / 8); // total length of the data

        SoundBuffer bfr = new SoundBuffer(new byte[sampledDataSize], format);
        for (int i = 0; i < sampledDataSize; i++ ) {
            //bfr.audioData[i] = (byte)((i % 5) * 63);
            bfr.audioData[i] = (byte)r.nextInt(255);
        }

        system.rawDataStream(format, false, "source_1", 0, 0, 0,
                               SoundSystemConfig.ATTENUATION_ROLLOFF,
                               SoundSystemConfig.getDefaultRolloff() );
        system.feedRawAudioData( "source_1", bfr.audioData );
        system.play( "source_1" );
The purpose of the code above is to simply generate white noise. Should work, right? Or am I missing something obvious? Also in the loop in which it generates the data you'll find I commented out a line, that one's purpose is to generate a saw wave at the speed of sampleRate / 5 (in this case 3200, meaning it's speed was 3200 Hz). That did not play either. What am I doing wrong here?

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #4 on: August 01, 2013, 01:15:52 PM »
I can't say for sure, but I think you are simply producing a sound that is too high-pitched to hear.  Let me explain why.

The way PCM works, is it uses samples to describe a wave.  So to represent a sine wave, you use values something like this.  Now imagine that same picture where every column had a random height (in that example, a random int between 0 to 15).  Then "connect the dots" to draw the wave that those values would represent.  What you end up with is a very thin wave that bounces up and down more-or-less every column.  The horizontal distance between the peaks represents pitch (the closer together, the higher the pitch).  I believe what you are ending up with is a sound of varying amplitude at near-maximum pitch.

So what is the maximum pitch?  Well that should be the sample size multiplied by the sample rate (so in your code example, that comes to 128 Khz).  According to wikipedia, the human ear can hear on average up to a maximum 20 Khz (although I am skeptical even of that -- listen to what the top-right cell here sounds like, and that is only 16 Khz)

I can't verify the math behind your saw wave formula, though, so I could be completely wrong in my analysis of the problem.  You might try using some simpler math to generate a basic sine wave at a reasonable frequency (taking into account the sample rate and sample size you've specified).
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline lukesomething

  • bit
  • Posts: 7
    • View Profile
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #5 on: August 01, 2013, 02:03:42 PM »
Okay, I took your advice, and for testing purposes tried the random noise approach again with a samplerate of 1000. That will leave me with white noise within a range of 0Hz - 8KHz, in theory. And, since the data buffer is 8000 bytes long, that noise should play for 8 seconds. Nothing plays though. I also tried feeding it a sinewave, however I think i screwed up somewhere as my math isn't exactly great, but something along the lines of below should work, right?
Code: [Select] [nofollow]
bfr.audioData[i] = (byte)(Math.floor(Math.sin((double)i / 8.0D) * 127.0D) + 128.0D);
I know this might be a long shot, but is there some kind of utility that will allow me to take a look at the data being played? Like an oscilloscope or something?

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #6 on: August 01, 2013, 04:47:55 PM »
No utilities in SoundSystem for that.  I'll have to play around with random data when I have some time.  I honestly have never had any use for raw data streams myself (this was a feature request by someone a while back), so I do not have any experience playing around with programatically generated sounds.  Another thought might be that the audio formats you are specifying are not "good" according to the underlying library.  You might try setting up your raw data stream with a known-to-work audio format (use the values you get from a file that is known to work).  Get that raw data stream playing your synthesized audio first before tinkering with other formats.  I'm just grasping at straws here..
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline lukesomething

  • bit
  • Posts: 7
    • View Profile
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #7 on: August 01, 2013, 05:41:08 PM »
The format is simply PCM (signed/unsigned specified by the 4th parameter in AudioFormat's constructor). I could try to use your example, but instead of feeding it the datafrom the file, something programmatically generated? I don't know how that would work exactly.

Offline littleguy

  • Developer
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #8 on: August 02, 2013, 09:33:08 AM »
Don't mean to be a smarty-pants, but the OP's algorithm for generating white noise is correct.  The waveform of white noise may appear to contain only high frequencies, but that's just a visual illusion.  There is actually just as much energy in the low frequencies as the high.  My point is that the issues are probably stemming from somewhere else (e.g. misunderstanding the SoundSystem API, coding bug, etc.).
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #9 on: August 02, 2013, 09:52:25 AM »
He is using the API correctly, so that leaves the most likely culprit a problem with the AudioFormat instance.  I'll try and make some time this weekend to experiment with feeding generated data into raw data streams to see what I can see, rather than continuing to speculate.

instead of feeding it the datafrom the file, something programmatically generated? I don't know how that would work exactly.

Code: [Select]
        codec.initialize( url );
        AudioFormat audioFormat = codec.getAudioFormat();
        soundSystem.rawDataStream( audioFormat, false, "source_1", 0, 0, 0,
                               SoundSystemConfig.ATTENUATION_ROLLOFF,
                               SoundSystemConfig.getDefaultRolloff() );
        // TODO: programmatically create a buffer of audio data
        soundSystem.feedRawAudioData( "source_1", buffer.audioData );
        soundSystem.play( "source_1" );
« Last Edit: August 02, 2013, 10:10:17 AM by Paul »
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #10 on: August 02, 2013, 10:16:32 AM »
Another likely culprit is an unreported bug in SoundSystem (I tend to start blaming other code first, but considering raw data streams are not well tested and have had other bugs pointed out recently, this is a strong possibility).

You could try playing your data at a lower level API -- set up a source or source data line (depending on if you are using OpenAL or Java Sound) and see if that can play the data.  Another thing I thought of is you could try feeding more than one copy of the buffer to the raw data stream, in case there is some issue with only trying to play one buffer (maybe a bug in the initialization code I wrote, perhaps)
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline lukesomething

  • bit
  • Posts: 7
    • View Profile
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #11 on: August 02, 2013, 07:08:31 PM »
Quick update, just tried filling the SoundBuffer's data with random data AFTER the file was loaded (so all loading of Codecs, Audioformats etc. are taken care of), and yes, finally, I managed to squeeze some sound out of the sucker. And hurt my ear in the process. Regardless, this is progress. What I found weird however, is that, according to the audioformat read from the codec, it is using signed PCM with a samplerate of 44100 Hz. Stereo. 16 bit samplesize and 4 bytes per frame. Admittedly, this is very different form what I've been using. Could that be what have been causing this problem all along? Conflict of Audioformat details? I'll try to start something up from scratch using that format and get back to you.

Update: Weirdness all around (at first glance anyway). I managed to "synthesize sound" that is, play back random data. And I got sound. In fact, I think I know why I couldn't get anything to play in the first place. You know, I never removed the 2 extra calls to feedRawAudioData (that I thought were redundant) from your sample. It struck me a bit odd that it only played 2 times considering data got fed 3 times, but I went on without thinking much of it. Now, in my own code I've only fed data once before playing. I know realize that was a big mistake. I tried this with both mono and stereo, in both cases no sound was played when data was fed once, but both played when data was fed twice.

This is the code I've been using, nothing plays on my end. But, if you uncomment the 2nd last line you'll hear white noise (I should warn you, it could get pretty loud)
Code: [Select] [nofollow]
int sampleRate = 44100;
        int sampleSize = 16;
        int channels = 2;
        int frameSize = channels * sampleSize / 8;
        int frameRate = sampleRate;
        int time = 2;
        boolean bigEndian = false;

        AudioFormat audioFormat = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED, sampleRate, sampleSize, channels, frameSize, frameRate, bigEndian);
        System.out.println(audioFormat);

        SoundBuffer buffer = new SoundBuffer(new byte[frameSize * frameRate * time], audioFormat);

        Random r = new Random();
        r.nextBytes(buffer.audioData);

        system.rawDataStream(audioFormat, false, "source_1", 0, 0, 0, SoundSystemConfig.ATTENUATION_ROLLOFF, SoundSystemConfig.getDefaultRolloff() );
        system.feedRawAudioData( "source_1", buffer.audioData );
        //system.feedRawAudioData( "source_1", buffer.audioData );
        system.play( "source_1" );
« Last Edit: August 02, 2013, 08:19:54 PM by lukesomething »

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #12 on: August 02, 2013, 08:27:47 PM »
Great, sounds like my theory was right about there being a problem in the initialization code.  I'll try and debug it along with the other problems.In the mean time I guess just be sure to feed in at least two buffers of data.
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline lukesomething

  • bit
  • Posts: 7
    • View Profile
Re: Having trouble creating a rawDataStream through the 3D Sound System
« Reply #13 on: August 02, 2013, 08:30:06 PM »
Will do, hopefully it wont affect too much. Thanks for all the help guy, much appreciated.