PaulsCode Forum

General => Your Projects => Topic started by: xperia64 on December 26, 2011, 04:32:36 PM

Title: TiMidity++ AE
Post by: xperia64 on December 26, 2011, 04:32:36 PM
If i can ever figure out how SDL works, i plan on making a simple non stock based midi player for android. I do however know that i have to use SDL_mixer as that is what pelya used in his port of OpenTTD. I'm going to look at the mupen64plus source to see where to start  :P
Title: Re: TiMidity++ AE
Post by: Paul on December 26, 2011, 05:15:42 PM
It might be easier to start with the sample app that ships with SDL, just because I made a few "customizations" in the code specific to my app (mainly with the keyboard input sections and the SDLActivity and SDLSurface Java classes) which wouldn't really apply to most other projects (it's kind of a hybrid between Pelya's port and the official port with some emulation and GUI interfaces thrown in).  On the other hand, it might give you some ideas about things that can be customized.  Let me know if you run into any problems understanding the port (I've become pretty familiar with the code and where various components are located)
Title: Re: TiMidity++ AE
Post by: xperia64 on December 26, 2011, 06:23:56 PM
I'm starting with the easy stuff first such as making the icon:
http://db.tt/LLT96qVh
I have re-sized it per android's directions for the 4 standard screen resolutions.
Title: Re: TiMidity++ AE
Post by: Epic_bubble on December 27, 2011, 08:05:15 AM
Hey there xperia64, is this your first time making an android app? :)
Title: Re: TiMidity++ AE
Post by: xperia64 on December 27, 2011, 04:42:01 PM
The first one involving actual coding. I should probably start with something easier
Title: Re: TiMidity++ AE
Post by: Paul on December 27, 2011, 05:01:09 PM
I started with some simple jPCT apps in pure Java/SDK, because I was familiar with the PC version of the library.  I then ported DOSBox to android using Pelya's SDL port (pretty much plug-and-play) as a way of getting my feet wet with the NDK and SDL on Android.  Something like that's really all the practice you'd need (the rest is not much different than any other project, since you clearly have some development experience to draw on).  The SDK is actually the hardest part to figure out IMO (very different than traditional swing)
Title: Re: TiMidity++ AE
Post by: xperia64 on January 21, 2012, 02:33:41 PM
Ok so i have learned that I actually don't need sdl, just the compiled libTiMidity as specified in the readme :D
Code: [Select]
libTiMidity -- MIDI to WAVE converter library.

This library is based on the TiMidity decoder from SDL_sound library.
Purpose to create this library is to avoid unnecessary dependences.
SDL_sound requires SDL and some other libraries, that not needed to
process MIDI files. In addition libTiMidity provides more suitable
API to work with MIDI songs, it enables to specify full path to the
timidity configuration file, and have function to retrieve meta data
from MIDI song.

If you have propositions regarding development of this library please
mail to me <lostclus@ua.fm>
this should make things a lot easier
Title: Re: TiMidity++ AE
Post by: xperia64 on February 05, 2012, 05:18:18 PM
Easy part done. Library is compiled. Now for the hard part of actually making the app
Title: Re: TiMidity++ AE
Post by: Paul on February 06, 2012, 03:59:54 PM
Here are those functions, from the chat earlier:

Code: [Select]
void JNI_Init_Varriables( JNIEnv* env, jclass cls )
{
    mEnv = env;
    mActivityClass = cls;

    midGetSongName = (*mEnv)->GetStaticMethodID( mEnv, mActivityClass,
                                "getSongName","()Ljava/lang/String;" );
    midGetConfigFile = (*mEnv)->GetStaticMethodID( mEnv, mActivityClass,
                                "getConfigFile","()Ljava/lang/String;" );
}
Code: [Select]
static char strBuff[1024];
char* JNI_GetSongName()
{
    jstring javaS = (*mEnv)->CallStaticStringMethod( mEnv, mActivityClass, midGetSongName );
    const char *nativeS = (*mEnv)->GetStringUTFChars( mEnv, javaS, 0 );
    strcpy( strBuff, nativeS );
    (*mEnv)->ReleaseStringUTFChars( mEnv, javaS, nativeS );
    return strBuff;
}
Code: [Select]
char* JNI_GetConfigFile()
{
    jstring javaS = (*mEnv)->CallStaticStringMethod( mEnv, mActivityClass, midGetConfigFile );
    const char *nativeS = (*mEnv)->GetStringUTFChars( mEnv, javaS, 0 );
    strcpy( strBuff, nativeS );
    (*mEnv)->ReleaseStringUTFChars( mEnv, javaS, nativeS );
    return strBuff;
}

The basic idea for linking them to your Activity class (Java side) is to add the following definition:

Code: [Select]
public static native void JNI_Init_Varriables();
Call it in the onCreate method:

Code: [Select]
JNI_Init_Varriables();
And implement the following static methods:

Code: [Select]
public static String getSongName()
{
   // TODO: get the song filename somehow, and return it
}
public static String getConfigFile()
{
   // TODO: get the config filename somehow, and return it
}
Title: Re: TiMidity++ AE
Post by: xperia64 on February 06, 2012, 06:51:47 PM
2/6/12 0.1: http://db.tt/V9atcm0T 
2/7/12 0.2: http://db.tt/l10gtvo8 Fixed scrambled folder lists. changed dp to 30
2/9/12 0.3: http://db.tt/kl91TwkN Added a filter which only displays .mid, .midi, or .smf files.
2/12/12 0.4: http://db.tt/yz08Ouxp Fixed rotation app restart, made back button go up a directory, changed filter to display 32 different music file types. Preparation to actually hook up the native code.
2/16/12 0.4.1: http://db.tt/Ni9eIJzw Minor fix, made it so when keypad slides out, it does not reset the app.
2/19/12 0.5: http://db.tt/mz3xsRGh Added command that causes app to exit while pressing back and the file browser is in the root of the sd card.
just a folder viewer, native timidity library is compiled but not implemented.
Title: Re: TiMidity++ AE
Post by: xperia64 on February 07, 2012, 07:50:29 PM
fixed the file filter by filtering the objects as they were being transferred from the array to the arraylist.
and here is what should be the main file as far as i know.
Title: Re: TiMidity++ AE
Post by: Paul on February 09, 2012, 07:08:34 PM
Ok, start by adding including jni.h at the top of one of your .c files (if you create a new .c file for this, which you should eventually, then be sure to also include timidity.h at the top as well).  Then add the following static globals and function:

Code: [Select]
static char cFilename[1024];  // increase the size if paths could be super-long
static MidSong *midiSong;  // TODO: figure out where you want this (assuming static global for now)
void JNI_Play_Song( JNIEnv* env, jclass cls, jstring jFilename )
{
    const char *nativeS = (*mEnv)->GetStringUTFChars( mEnv, jFilename, 0 );
    strcpy( cFilename, nativeS );
    (*mEnv)->ReleaseStringUTFChars( mEnv, jFilename, nativeS );

    // ok, the filename is now contained in 'cFilename'.

    MidIStream *midiStream;
    midiStream = mid_istream_open_file ( cFilename );
    MidSongOptions *midiOptions;
    // TODO: this could be a memory leak here, call free() somewhere:
    midiOptions = (MidSongOptions *) malloc( sizeof( MidSongOptions ) );
    // TODO: some kind of error checking, to avoid segfault if null pointers
   
    midiSong = mid_song_load( midiStream, midiOptions );  // TODO: error checking
    mid_song_start( midiSong );
}

Then, in your .java file (the file browser for now), add the following prototype:

Code: [Select]
public static native void JNI_Play_Song( String filename );
And finally, in the part of the code where the filename was chosen (you'll want the full path, BTW), place the following line:
Code: [Select]
JNI_Play_Song( filename );
Once you get it so it compiles and runs (and there are no simple errors showing up in the logcat), then you will be ready to move on to creating the Android audio context through JNI (I expect that it won't actually play any audio right now).  There might be some error about there not being an audio device or something, and it might segfault, or it might just be silent (the logcat output should give you an idea what it is complaining about).
Title: Re: TiMidity++ AE
Post by: xperia64 on February 09, 2012, 08:00:57 PM
Code: [Select]
public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        setContentView(R.layout.main);
        {
        System.loadLibrary("timidity"); // or whatever the name is (take of the "lib" and ".so")
        }
        catch(  UnsatisfiedLinkError, e )
        {
        Log.e( "Oops!", "Unable to load native library 'timidity'" );
        }
        myPath = (TextView) findViewById(R.id.path);

        getDir(rootsd);

    }
Title: Re: TiMidity++ AE
Post by: Paul on February 09, 2012, 08:05:21 PM
You are missing the "try" and open-bracket.  Should look like this:

Code: [Select]
public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        setContentView(R.layout.main);
                try
                {
            System.loadLibrary("timidity"); // or whatever the name is (take of the "lib" and ".so")
        }
        catch(  UnsatisfiedLinkError, e )
        {
            Log.e( "Oops!", "Unable to load native library 'timidity'" );
        }
        myPath = (TextView) findViewById(R.id.path);

        getDir(rootsd);

    }

--EDIT--
And there was a stray bracket in there.  Fixed.
Title: Re: TiMidity++ AE
Post by: xperia64 on February 10, 2012, 01:39:32 PM
i added the try and catch thing. it does not give the "unable to load" message so that is good, but still crashes when selecting a song. here is the logcat.
Code: [Select]
02-10 14:36:00.938: D/dalvikvm(2135): No JNI_OnLoad found in /data/data/com.xperia64.timidityae/lib/libtimidity.so 0x2afc8960, skipping init
02-10 14:36:15.548: E/AndroidRuntime(2135): FATAL EXCEPTION: main
02-10 14:36:15.548: E/AndroidRuntime(2135): java.lang.UnsatisfiedLinkError: JNI_Play_Song
02-10 14:36:15.548: E/AndroidRuntime(2135): at com.xperia64.timidityae.TimidityAEActivity.JNI_Play_Song(Native Method)
02-10 14:36:15.548: E/AndroidRuntime(2135): at com.xperia64.timidityae.TimidityAEActivity.onListItemClick(TimidityAEActivity.java:215)
02-10 14:36:15.548: E/AndroidRuntime(2135): at android.app.ListActivity$2.onItemClick(ListActivity.java:319)
02-10 14:36:15.548: E/AndroidRuntime(2135): at android.widget.AdapterView.performItemClick(AdapterView.java:284)
02-10 14:36:15.548: E/AndroidRuntime(2135): at android.widget.ListView.performItemClick(ListView.java:3518)
02-10 14:36:15.548: E/AndroidRuntime(2135): at android.widget.AbsListView$PerformClick.run(AbsListView.java:1902)
02-10 14:36:15.548: E/AndroidRuntime(2135): at android.os.Handler.handleCallback(Handler.java:587)
02-10 14:36:15.548: E/AndroidRuntime(2135): at android.os.Handler.dispatchMessage(Handler.java:92)
02-10 14:36:15.548: E/AndroidRuntime(2135): at android.os.Looper.loop(Looper.java:123)
02-10 14:36:15.548: E/AndroidRuntime(2135): at android.app.ActivityThread.main(ActivityThread.java:3701)
02-10 14:36:15.548: E/AndroidRuntime(2135): at java.lang.reflect.Method.invokeNative(Native Method)
02-10 14:36:15.548: E/AndroidRuntime(2135): at java.lang.reflect.Method.invoke(Method.java:507)
02-10 14:36:15.548: E/AndroidRuntime(2135): at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:862)
02-10 14:36:15.548: E/AndroidRuntime(2135): at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:620)
02-10 14:36:15.548: E/AndroidRuntime(2135): at dalvik.system.NativeStart.main(Native Method)
Title: Re: TiMidity++ AE
Post by: Paul on February 10, 2012, 03:27:20 PM
Oops, my bad.  That JNI_PlaySong method should start like this instead:

Code: [Select]
JNIEXPORT void JNICALL Java_com_xperia64_timidityae_MyActivity_JNI_Play_Song( JNIEnv* env, jclass cls, jstring jFilename )
{
...etc
Title: Re: TiMidity++ AE
Post by: xperia64 on February 10, 2012, 07:04:19 PM
I have decided it will be easier to add the "400 pound gorilla" to my project than to figure out how the stand alone timidity lib works. However, this will allow me to support 32 formats!
Title: Re: TiMidity++ AE
Post by: xperia64 on February 11, 2012, 03:57:42 PM
check this makefile if you could. thanks
Title: Re: TiMidity++ AE
Post by: xperia64 on February 19, 2012, 01:54:53 PM
Version 0.5a is out. Next step is to (finally) get the native components hooked up.  :)
Title: Re: TiMidity++ AE
Post by: xperia64 on March 05, 2012, 07:13:18 PM
Log.
Title: Re: TiMidity++ AE
Post by: xperia64 on March 07, 2012, 06:13:36 PM
Status Update: SDL is hooked up, now i just need to figure out how to prevent it from hanging on SDL_Android_Init()
Title: Re: TiMidity++ AE
Post by: xperia64 on March 10, 2012, 11:23:20 AM
Update: Audio!  Ogg, wav, flac, and midi files work with it.
So far mp3 doens't want to work but its a start :D
Title: Re: TiMidity++ AE
Post by: xperia64 on March 10, 2012, 06:03:16 PM
Current Music format compatibility list:
Code: [Select]
Completely working:
*.mid
*.wav
*.669
*.amf
*.dsm
*.far
*.gdm
*.imf
*.it
*.m15
*.med
*.mod
*.mtm
*.okt
*.s3m
*.stm
*.stx
*.ult
*.uni
*.umx
*.psm
*.mt2
*.xm
Partially working:
*.mp3
*.flac
*.ogg
*.aif
All require the audio rate to be set perfectly
Not working:
*.raw
*.pcm
Unknown:
*.spx
*.shn
*.mpg
*.mpeg
*.voc
I plan on writing the frontend and including the formats that work perfectly so i can release it. After release, i will try to write a sample rate reader that will automatically choose the right sample rate
Title: Re: TiMidity++ AE
Post by: xperia64 on March 18, 2012, 12:30:27 PM
Market Link: https://play.google.com/store/apps/details?id=com.xperia64.timidityae
Title: Re: TiMidity++ AE
Post by: Tom.K on March 24, 2012, 05:54:03 AM
Any chance to use this as an Icon?

(http://i.imgur.com/M0l1P.png)
Title: Re: TiMidity++ AE
Post by: xperia64 on March 24, 2012, 09:30:37 AM
ill think about it
Title: Re: TiMidity++ AE
Post by: xperia64 on August 15, 2013, 09:14:36 PM
For quite a while now I've been wanting to completely redo timidity actually using TiMidity++. However, I can't get it to even compile&output a wav file let alone stream audio to the speakers. If anyone would be willing to help get timidity running let me know.. Once it actually works, it will be fully open source to comply with the license, but I will still charge $0.99.
What needs to be done (that I can't figure out):
Compile timidity as a dynamic or static library.
Write an interface that either uses opensles or preferably write bytes/shorts using android.media.AudioTrack for API<9 compatibility.
Avoid SDL at all costs.

Here are some possible resources I've found so far:
First, obviously the official TiMidity++ library (preferably compile this): http://sourceforge.net/projects/timidity/files/TiMidity%2B%2B/TiMidity%2B%2B-2.14.0/
The original devs usually show up once a year or less so they probably wont be much help. I got it to compile as a dynamic lib but it always crashed in resample.c, even if I tried to force it to not resample.
Second, someone managed to compile timidity as a static lib for the iphone: https://github.com/yoyofr/modizer/tree/master/libtim
Can't find what/where the equivalent of a makefile for xcode is. I asked the dev how he managed to compile it and am waiting for a response.
Third and least preferable option: ftp://ling.lll.hawaii.edu/pub/greg/gt-0.4.tar.gz
Contains an older/simplified version of timidity, designed mainly as a binary file with interfaces missing that existed in TiMidity and full TiMidity++

This has been driving me insane for ages.
Other random notes:
Fluidsynth works and has been incorporated into another app as well, but is a huge processor hog and doesn't support certain soundfonts properly (nails on a blackboard).
The TiMidity++ binary compiled on my pandora will run once all of the libs are copied to the right place and can successfully output wav files. ALSA!=TinyAlsa. Once I get my nexus 7 back, I may be able to pipe the wav file to the tinyplay binary (only available on some devices) for dynamic playback instead of actually saving the file.
Title: Re: TiMidity++ AE
Post by: xperia64 on October 27, 2013, 07:15:28 PM
Given up on TiMidity++ again for now, but I've encountered a terribly annoying issue with the internal android media player in general:
Code: [Select]
10-27 20:01:57.990: E/MP3Extractor(1805): Unable to resync. Signalling end of stream.It calls setOnCompletitionListener rather than setOnErrorListener making the app think it just played the file properly and should now skip to the next one. Seem to have it mostly under control with a lot of redundancy and Thread.sleep's. I also apparently have to keep the FileInputStream open forever. If anyone has any ideas on how I could fix this, please let me know. My current set up is load file->call proper playback method for that file->change values of variables accordingly->start a thread to constantly check if the player isn't currently loading, was playing, isn't paused, and check the player type (ndk or android mediaplayer) and whether that method is still playing->Change to next song if that is true. For the ndk method I poll whether it is currently playing or not and for the android mediaplayer method I set an onCompletitionListener. For some reason the file skipping occasionally happens with the ndk method too.
Title: Re: TiMidity++ AE
Post by: Paul on October 28, 2013, 08:33:15 AM
Do you know the cause of the "Unable to resync" error?  I would track that one down first if it is getting your app into a loop where it is skipping track after track and not playing anything (I gathered that was the behavior you were experiencing, correct me if I'm wrong).

As for the process logic, it seems like it should work on a high-level.  That said, personally I would try to hook into callback methods and do the state checks there if at all possible, rather than setting up a thread to poll the states constantly in a loop (it would make the code easier to follow, and you wouldn't have to worry about thread management or a zombie process draining the user's battery when it isn't needed).  I'd have to look at the actual code though, since I am not very familiar with the APIs involved.
Title: Re: TiMidity++ AE
Post by: xperia64 on October 28, 2013, 02:00:34 PM
MP3Extractor Unable to resync is just a weird bug with the system android mediaplayer. I'll try to log more.
Title: Re: TiMidity++ AE
Post by: xperia64 on October 28, 2013, 09:44:10 PM
Except when using gigantic sound banks in fluidsynth, I managed to control the skipping problem by increasing the delay between each song finished check to 1 second from 10 milliseconds and loading android supported song types into MediaPlayer directly from a path instead of a fileinputstream.
Title: Re: TiMidity++ AE
Post by: xperia64 on January 12, 2014, 05:34:57 PM
Good news! I got TiMidity++ compiled into a working library with wave file output. It seems to be a lot faster than fluidsynth and sounds better than both fluidsynth and timidity. I still need to write a streaming output driver for android. Perhaps I'll also integrate libxmp as well to cover the subpar timidity++ amiga playback as I plan to completely ditch SDL and related libraries.
Fluidsynth vs TiMidity++ on a GBA midi file+soundfont on android (do I even need to say why TiMidity++ is better?):
Fluidsynth: https://db.tt/tgUJYd1A
TiMidity++: https://db.tt/gjz7GlRN
Title: Re: TiMidity++ AE
Post by: xperia64 on January 13, 2014, 06:57:52 PM
We now have audio streaming, the hardest part was calling java from c (still not sure if I'm doing it properly) to pass byte arrays. After nearly 2 years trying to achieve this, I managed to do it in 2 days :P.
Title: Re: TiMidity++ AE
Post by: Paul on January 13, 2014, 09:48:16 PM
the hardest part was calling java from c (still not sure if I'm doing it properly) to pass byte arrays.

Working on the SDL2 upgrade (and updating the target for FrodoC64), I picked up a little experience with some of the pitfalls of JNI.  If you have the code posted somewhere, I'll take a look and see if I spot any potential snags.
Title: Re: TiMidity++ AE
Post by: xperia64 on January 14, 2014, 03:33:02 PM
http://pastebin.com/44ahc97z
Title: Re: TiMidity++ AE
Post by: xperia64 on January 14, 2014, 07:38:05 PM
Sorry to rant again, but the NDK is being very strange and obnoxious. For whatever reason, yesterday I compiled timidity++ and it had excellent performance. I made a backup of all of the JNI stuff, I think after I built the most recent test apk. The test apk has excellent performance when playing midi files, however all builds after have terrible performance. I can't figure out why. The md5sums of the libs as they are now and the libs in the apk are different, but have 10 different bytes (maybe packing into an apk changed a couple of things?). Perhaps I should just start over from git...
As TiMidity++ is GPL, I may as well post the source now if anyone wants to check it. https://db.tt/9ewNy9H3
Example of ndk weirdness: I'd change sample rate to 44100 on the NDK side but it still played at 48000. The only real changes I made from regular git is decreasing the voice count from 256 to 128 and changing resampling to newton. Unless newton finally kicked in with these newer builds and is producing the lag, I don't know what to make of it.
Title: Re: TiMidity++ AE
Post by: littleguy on January 14, 2014, 09:07:07 PM
Did you change NDK versions in the last 48 hours?
Title: Re: TiMidity++ AE
Post by: xperia64 on January 14, 2014, 09:13:04 PM
Nope. Sticking with r8d, hasn't let me down before.
Title: Re: TiMidity++ AE
Post by: xperia64 on January 15, 2014, 06:15:08 PM
Hmm, decided to re-port my changes to a fresh tar ball, and its still not great. However, I found out how to use the note-cutoff feature (like frameskip for audio) which greatly improves performance. Still, I wonder what I did the first time that made it run better... Perhaps it was the audio filtering...
Anyway, it doesn't sound as nasty, and works pretty well on my N7. It crashes on my S2 for some reason with error "jit unchain for all threadid=XX".
Title: Re: TiMidity++ AE
Post by: xperia64 on March 18, 2014, 07:05:10 PM
Well well. Apparently I need a huge audio buffer for this and 1KB was not cutting it. 96000 bytes works great even on lower clockspeeds with my N7 for intense midi processing.
Title: Re: TiMidity++ AE
Post by: xperia64 on March 22, 2014, 10:22:15 AM
Ok, got it working fine on my S2. Now I'm trying to tackle a problem with the volume. TiMidity++ was designed to:
Load an unchanging array of songs all at once
Play all of them with internal next and previous commands
Completely quit and repeat.

As most music players don't want the entire app to quit after finishing some songs, I'm passing an array of a single song to the timidity_play_main function, letting it play and "quit", then load the next song as an array and repeat. It works decently, however when I change songs, the volume of the new song is too loud and peaks on one or more track. It doesn't peak when I load two songs "properly" in a single array. I could do something with an interface I guess, but I'd rather be able to call timidity_play_main multiple times.
Source (JNI only): https://db.tt/m3JzGLqD
Title: Re: TiMidity++ AE
Post by: xperia64 on April 02, 2014, 06:50:02 PM
I suppose I answered my own question. Interfaces are the way to go. timidity_play_main should never be called more than once, it never ends well.. Now that the clipping issue is taking care of and I have a much nicer way to interface with TiMidity++'s core, I can start the polishing phase.