PaulsCode Forum

Mupen64Plus AE => General Discussion => Topic started by: littleguy on September 17, 2012, 10:56:10 AM

Title: Source code location to add native HID joystick support?
Post by: littleguy on September 17, 2012, 10:56:10 AM
Awesome app!  I'm actually building my own Arduino-based HID joystick & case for my Nexus 7 to enhance the gaming experience further.

I'd love to have native HID analog support for this app (Android 3+), but I know your attention is (understandably) diverted from this project for now.  I'd like to try adding the functionality myself, so I downloaded the source code from git.  However I was quickly overwhelmed with the amount of code!  Would you mind pointing me to the particular source files/classes pertinent to adding native HID/analog support, so I can give it a try?  If you have any cursory thoughts on the matter I'm all ears as well.  Of course if I'm successful, I will be happily obliged to contribute my mods back into the next release.

As an aside,  I am using USB/BT Joystick Center app with a Logitech gamepad, everything works fine.  Unfortunately my custom HID joystick (which is plug-n-play in windows) isn't working with Joystick Center and I've been in contact with that app's developer.  So between that app and this, I hope to have a solution to  Mupen64Plus + analog controls +my controller.

Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 17, 2012, 12:47:57 PM
This is something I was planning to look into at some point, but haven't gotten around to.  Unfortunately, that means I don't have a lot of useful information with regards to getting data from Bluetooth and interpreting it.  I think that will have to be your first step.  I'd start by reading the Android documentation for Bluetooth (http://developer.android.com/guide/topics/connectivity/bluetooth.html) to see what you have to work with on the Java side.  You may end up needing to interface from the native side if the Java API isn't sufficient.

I'd recommend putting together a list of specific things you need to know, and searching on Stack Overflow (http://stackoverflow.com/) for people who have asked similar questions.  If your questions have not already been answered, then post new questions there.

Once you know how to receive and interpret data from your joystick in Android, then processing it into a form that can be used by Mupen64Plus AE will be pretty simple.  If you have access to the data on the Java side, then the easiest way would be to call the method updateVirtualGamePadStates (in main/src/paulscode/android/mupen64plusae/GameActivityCommon.java) whenever you have new input from the joystick.  (Ignore the name.. it is used by the Virtual Gamepad, but can be used anywhere you have input to send to the emulator):
Code: [Select]
/* From the N64 func ref: The 3D Stick data is of type signed char and in the range between
80 and -80. (32768 / 409 = ~80.1) */
    // Sends virtual gamepad states to the input plug-in:
    public static native void updateVirtualGamePadStates( int controllerNum, boolean[] buttons, int axisX, int axisY );

If you have access to the data on the native side rather than from Java, then simply update the following array values in main/jni/input-sdl/src/plugin.c (see the function right below these lines in the code for an example):
Code: [Select]
//// paulscode, for the virtual gamepad:
unsigned char virtualGamePadButtonState[4][14];
signed char virtualGamePadAxisState[4][2];

Sorry I can't help you much with actually getting input data from the joytick, but I'm happy to help if you run into any problems linking up with the emulator once you get to that point.  You'd also need to add some interface to the GUI, and I can help with that as well.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 17, 2012, 06:57:58 PM
Awesome!  Actually, I'm not using a bluetooth joystick.  It's just a plain usb joystick - generic HID driver.

Android programming is still new to me, but I've been programming hardware for quite a while.  I'll see what I can do.  Thanks for the pointers for the important files.  I'll take a look tonight.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 17, 2012, 07:20:49 PM
Ah, gotchya (not sure why I assumed it was a Bluetooth joystick).  Probably Android's API documentation for USB Host (http://developer.android.com/guide/topics/connectivity/usb/host.html) will be a good place to start researching.  Basic process will be the same -- figure out how to get the input from the joystick, and then link that up with the emulator either on the Java side or the native c/ c++ side.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 17, 2012, 09:54:48 PM
Ok, doing some more digging, this is really helpful.  A few more questions:

1.  So it looks like you sprinkled a few lines here and there into the original C++ source for Mupen64Plus, then do the bulk of your work in Java, using Java wrappers to invoke the functions on the C++ side?

2.  Where does the code reside that bridges your app with BT/USB Joystick Center app?  In other words, is the translation code a part of Mupen64PlusAE source or is it part of the Joystick Center source?  If the former, what file contains that code?  If the latter, then it sounds like I need to write a little Android app that reads the USB and invokes Mupen64 code?  Or am I way off the mark here?

Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 17, 2012, 11:07:54 PM
1.  So it looks like you sprinkled a few lines here and there into the original C++ source for Mupen64Plus, then do the bulk of your work in Java, using Java wrappers to invoke the functions on the C++ side?
Right, the native c/ c++ code for the input plug-in is as close to the code that ships with the PC version of Mupen64Plus as possible.  My reasoning behind this is that I want to eventually merge this project back to the original source, while in the mean time being able to parallel updates as Richard42 releases them.  So it makes sense to change as little as possible on the native side, and to code the Android pieces on the Java side of the house, and interact with the emulator through the config files wherever possible, rather than through JNI to the native c/ c++ code.

2.  Where does the code reside that bridges your app with BT/USB Joystick Center app?  In other words, is the translation code a part of Mupen64PlusAE source or is it part of the Joystick Center source?  If the former, what file contains that code?
Actually translations of the input data happen in both apps.  Note that the BT/USB Joystick Center app is an IME designed to run separately from the apps using it.  It achieves this by translating the joystick data into keyboard events with a specific pattern, which can be received by any app listening for input from the keyboard.  I have my code interpret this pattern and translate that into the form used by the emulator.  This happens in the method onKey (in main/src/paulscode/android/mupen64plusae/SDLSurface.java):

Code: [Select]
        if( keyCode > 255 && Globals.analog_100_64 )
        {
            key = ( keyCode / 100 );
            if( action == KeyEvent.ACTION_DOWN )
                str = ( (float) keyCode - ( (float) key * 100.0f ) );
        }

Notice that I have a boolean that tells the emulator whether or not to do the translation.  At the moment, key event translation is only done for input from the USB/BT Joystick Center and BT Controller IMEs.  Both produce the same (x * 100 + [0-64]) key code pattern for analog input (thanks to my recommendations to DroidBean when he was adding analog support to his app), so I have this boolean hard-coded to true.  When I add support for other IMEs using a different pattern in the future, this will have to be controlled from the GUI in the settings menu instead.

In your case, you probably won't be writing an IME (unless that is your goal), so there won't be any need for the two-step translation process.  You could simply translate the joystick input data directly into the form used by the emulator, bypassing the key code thing all together.  Of course, if you do all the work of figuring out how to get the data, you may decide to write an IME anyway so you can use your joystick with your other apps, not just Mupen64Plus AE.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 18, 2012, 07:21:28 AM
Thanks!  This is very clear, and thanks for the quick feedback.  I'll start tinkering with the Android usb host api until I get the mechanics of the joystick data all figured out.  Hope to make some progress over the next week or so.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 23, 2012, 10:40:41 PM
Ok, I had some time this weekend to start working on this.  I wrote a trivial app that just displays native joystick axes in a TextView object in the main activity.  Isn't much work at all to get the data, just override the onGenericMotionEvent method and make a few calls to the API.

Code: [Select]
public class MainActivity extends Activity {

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
    }

    @TargetApi(12)
    @Override
    public boolean onGenericMotionEvent(MotionEvent event) {
   
        // Get the device (not used below, but maybe useful later)
    InputDevice device = event.getDevice();
   
        // Spit out some debug info
    String message = "MotionEvent:";
    message += "\r\nAction: " + event.getAction();
    message += "\r\nActionIndex: " + event.getActionIndex();
    message += "\r\nActionMasked: " + event.getActionMasked();

        // These are the axes used by most XBox360 (XInput) gamepads, including Logitech F710
    message += "\r\n  AXIS_X : " + event.getAxisValue(MotionEvent.AXIS_X);           // Left-stick X
    message += "\r\n  AXIS_Y : " + event.getAxisValue(MotionEvent.AXIS_Y);           // Left-stick Y
    message += "\r\n  AXIS_Z : " + event.getAxisValue(MotionEvent.AXIS_Z);           // Left analog trigger
    message += "\r\n  AXIS_RX: " + event.getAxisValue(MotionEvent.AXIS_RX);          // Right-stick X
    message += "\r\n  AXIS_RY: " + event.getAxisValue(MotionEvent.AXIS_RY);          // Right-stick Y
    message += "\r\n  AXIS_RZ: " + event.getAxisValue(MotionEvent.AXIS_RZ);          // Right analog trigger
    message += "\r\n  AXIS_HAT_X: " + event.getAxisValue(MotionEvent.AXIS_HAT_X);    // D-pad left/right
    message += "\r\n  AXIS_HAT_Y: " + event.getAxisValue(MotionEvent.AXIS_HAT_Y);    // D-pad up/down
   
        // But not all gamepads use those particular axes...
        // ...some may put their data in other axes (esp. older gamepads)
        // So print out any other active axes
    for (int i = 0; i < 50; i++) {
    double value = event.getAxisValue(i);
    if (value != 0)
        message += "\r\n  Axis " + i + ": " + value;   
    }

        // Write the info onto the GUI
    TextView view = (TextView) findViewById(R.id.text_info);
    view.setText(message);   
    return true;   
    }   
}

Axis values are floats ranging from -1.0 to 1.0.  Note that the d-pad is also registered as an axis, even though it only ever reports the values {-1.0, 0.0, 1.0}.  Positive is to the right and down (not up) for all axes.  Analog triggers are 1.0 when fully depressed, -1.0 when released.  Handling digital buttons as key presses will continue to work, although there are a few buttons (e.g. clicking the joysticks) that aren't natively mapped as a keypress.

I just noticed that you are handling input using the listener pattern rather that overrides.  That should be fine too.  You would just need to implement View.OnGenericMotionEventListener (http://developer.android.com/reference/android/view/View.OnGenericMotionListener.html).

At this point, I think the biggest task will just be extending the user settings interface.  We probably need a way for Android 3.1+ users to remap axes if they're not using an Xbox controller or don't like the default arrangement.  Perhaps a way to disable native support too (just revert to the current approach), although I can't think of a reason they'd want to do that.

So I thought I would take a crack at that part too, just to identify any roadblocks.  I successfully built Mupen64Plus-ae from the source on github.  I'm able to start the app and navigate menus, but I get a force close as soon as I try to start a ROM.  I might have a build error or something, so I'm digging into that.  I'm doing this in Eclipse with the Android SDK and NDK, on Windows.  Maybe I should try building from Linux.  What O/S are you using for development?
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 23, 2012, 11:56:53 PM
For building, I am using Linux, but others have built successfully in Windows and Mac.  Lioncash has been doing a bunch of core updates from the main repository, so something may be broken.  I'll check when I have some time tomorrow.  In the mean time, is there anything useful in the logcat output after the crash?

I was planning to implement View.OnGenericMotionEventListener since that is used by the Xperia Play with ICS as I recall (the existing NativeActivity method is great for pre-ICS Xperia Play's, but doesn't work in ICS very well, because the action bar input is blocked).  It will have to be done in a way that doesn't break the emulator for older devices though, because that interface is only available in API level 12 and higher.  This may not apply to your case anyway, if you are just building a custom version of the emulator for yourself, since it is for your own custom joystick you are building.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 24, 2012, 06:31:34 AM
Yeah, I'll check the logcat.  This is my first Android programming experience.  I'd like to get this into the main branch if it's not too hard, since I know many others would like the functionality.  The joystick I'm building is just a standard Xbox controller clone, so I don't view this as a personal customized version of Mupen just for myself.

We could probably implement View.OnGenericMotionEventListener in a separate class, analogous to the listeners in SDLSurface.java.  We would only instantiate and register the listener if the user's Android version was >=12.  I definitely don't want to break compatibility with older Android versions.

I'll do some more digging, hopefully get the source running properly on my device.  Thanks for the help.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 24, 2012, 10:12:16 AM
Sounds good - that ought to work while still keeping it compatible on older devices.

I just built the latest commit (ac71f08) from Eclipse, and it isn't crashing on my device.  Was there anything useful in the logcat?
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 26, 2012, 07:00:14 PM
Ok, after a few wrong turns I got everything to build properly on windows.  I posted some updated instructions on another thread (http://www.paulscode.com/forum/index.php?topic=599.0) for total noobs like me.

So I'll start integrating my additions into the source code, hopefully should have something to share in the next week or so!
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 27, 2012, 04:39:40 PM
Ok, that wasn't too hard at all once I climbed the learning curve!  Patch files attached below. Works like butter, and no longer dependent on a $5 IME.  I may be imagining things, but I feel like I have a lot more control sensitivity now making mario walk slowly.  With USB/BT Joystick Center app, it never felt this smooth.

Right now I have all the analog axis mappings hard-coded to an Xbox controller's axes.  I'm using the right analog stick for the c-pad buttons, and the left analog trigger for the z-button.  This lets me map the R and L buttons back to the left and right shoulder buttons of the controller.  I added some lines to the Updater.java code so that IF you're running Android 3.1 or greater, the defaults are mapped properly to an xbox controller out of the box.  Otherwise it keeps the legacy default key mappings.

The real work will involve the user interface, to permit remapping of the analog stick, and to permit analog axes to be mapped to buttons (e.g. left-analog-trigger --> Z, right-stick --> C-buttons).  This will probably require someone with a little more Android gui programming experience than me.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 27, 2012, 04:45:07 PM
Thanks, I'll work on incorporating this into the code and updating the GUI to handle mapping the axes.  Thanks for working on this!  It is definitely something I've been wanting to add at some point.

--EDIT-- committed the changes to the repository, and added a note that there needs to be integration into the GUI button mapping section, and that the app needs to be tested on a device with Gingerbread to ensure it is not broken for pre-Honeycomb users.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 27, 2012, 05:03:56 PM
One question:  Do you know if the following line will limit the SDLSurface class to API level 12 (in other words it will not be available on pre-Honeycomb devices)?  I don't have a device running Gingerbread to test it on at the moment.

Code: [Select]
@TargetApi(12)
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 27, 2012, 05:16:08 PM
From what I understand, this just removes Lint errors.  I think as long as you target 12 or higher for the release build (sounds like you're targeting 14?) it will compile just fine.  The 12+ code is excluded at runtime by the conditional checks.  I'm still new to this, but it seems like the standard approach when I looked at examples on the web.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 27, 2012, 05:34:41 PM
Ok, great.  I'll post a test build for folks to run just to be sure, once I fix a different (unrelated) piece of code that is limiting the app to API 11 right now.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 27, 2012, 05:37:56 PM
Hmmm I thought I had this figured out but now I'm not sure after reading this (http://tools.android.com/recent/lintapicheck).  If I comment that line out, I get Lint errors all over the place in SDLSurface.java, not just in the new stuff I added.  Some of the calls are using API 5, others API 11, and some of the new stuff I added uses API 12.  Are you explicitly doing run-time hardware checks before calling these methods?

onTouch uses some API 5 methods:

onKey uses some API 11 methods:

The native gamepad additions use some API 12 methods:

Edit: Attached an updated diff file that is more conservative in the lint annotations.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 27, 2012, 05:47:45 PM
For the ActionBar stuff, I just put in checks like this:

Code: [Select]
if( Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB )
That's a good catch on the MotionEvent stuff.  I probably should change the minimum API level to 5 instead of 4.

It's odd you are getting error messages, though.  Generally it will let you build when you have the target API level high enough, but it will crash on the device if you call something that isn't there.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 27, 2012, 05:51:11 PM
It's odd you are getting error messages, though.  Generally it will let you build when you have the target API level high enough, but it will crash on the device if you call something that isn't there.
I probably have my Lint settings set conservatively, to error on compile rather than warn.

Edit: Here's the officially sanctioned idiom (http://developer.android.com/training/basics/supporting-devices/platforms.html) for supporting multiple platforms.  It recommends that you always target the highest available API for the release build.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 27, 2012, 05:57:21 PM
BTW, if it isn't a huge hassle, could you post the diff from the previous SDLSurface.java you had to this one (rather than from the original), since I've already committed the first one?  I can apply the changes by hand if it's too much trouble.

Also, if you'd like write access to the github repository, let me know your username and I can add you.  I don't mind uploading the changes myself, just thought I'd offer in case you are interested.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 27, 2012, 06:00:26 PM
Sorry about that.  It's probably just as easy to do it by hand.  The only changes from the previous are to add/remove the @TargetsAPI annotations.  Just do a text search for "@TargetsAPI" in the latest diff.

Edit: and remove the @TargetsAPI(12) at the class declaration at the top.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 27, 2012, 06:18:59 PM
Weird.. when I started playing around with TargetApi, I began getting all kind of API-related errors that weren't there before.  I was able to do all of the changes you had except this one:

Code: [Select]
@@ -393,6 +398,7 @@ class SDLSurface extends SurfaceView implements SurfaceHolder.Callback,
                     GameActivityCommon.mSingleton.runOnUiThread(
                         new Runnable()
                         {
+                            @TargetApi(11)
                             public void run()
                             {
                                 if( GameActivityCommon.mSingleton.getActionBar

I went ahead and just left that one out, since that code is already inside a block that will only be run if the device is running Honeycomb or higher, due to this line:

Code: [Select]
            else if( key == KeyEvent.KEYCODE_BACK && Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB )
And to shut up the other errors (from the Action Bar stuff), I just added @SuppressLint("NewApi") to the SDLSurface class.  There is definitely something screwy going on with the compiler, since it never had this problem before.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 27, 2012, 06:53:31 PM
Yeah, I was noticing the same.  It built out of the box with no problems, then I added my new code, got the Lint errors, then soon all sorts of stuff was giving compile errors where it hadn't before.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 28, 2012, 08:45:06 AM
I forked the latest build on github (username littleguy77) so my inputs should be easier to integrate going forward.

Bear with me, I'm a git noob as well.  Been programming for in C++ for 15 years and C# for the past 6, using Visual Studio and Subversion.  I had a little Java and Eclipse experience going into this, but the Android SDK and Git are brand new.  Thanks for your patience as I have fun learning all this.  ;D
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on September 28, 2012, 08:52:16 AM
No problem.  I gave you write access to the main repository too, in case you need to use it.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on September 28, 2012, 09:00:10 AM
Thanks a lot ;D

Edit: Started a new branch, deleted my fork.  Better integration now.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on October 01, 2012, 11:39:14 AM
Paul - I think I'll try my hand at updating the key/axis mapping GUI over the next week or two.  I'll make a separate branch for this until it stabilizes, then you guys can see what you think.

Question: Is anyone already updating the keymapping UI right now?  I thought I saw a few commits by lioncash recently regarding the "scancode" dialog UI.  Don't want to duplicate efforts...
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on October 01, 2012, 12:01:30 PM
I haven't started on it, so you are welcome to take it on if you like.  I'm slowly hacking away at getting the overall theme to match and fix the "forgetful checkboxes" bugs, and that will take a while to complete.  I'll just leave the controller section for last so I don't step on your toes.

About the "scancode" dialog UI, lioncash just changed the xml layout on it to simplify the code.  The functionality wasn't changed.  As long as you branch from a commit sometime after that update, you should be good.

If you have any problems let me know.  I'm pretty familiar with that part of the code.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on October 01, 2012, 12:32:22 PM
Will do.  Thanks much.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on October 03, 2012, 04:15:00 PM
Turns out adding "implements View.OnGenericMotionListener" causes the app to crash on older versions of Android, with the following error:

Code: [Select]
E/AndroidRuntime(14234): java.lang.RuntimeException: Unable to start activity
ComponentInfo{paulscode.android.mupen64plusae.zeldamm/paulscode.android.mupen64plusae.GameActivityXperiaPlay}:
android.view.InflateException: Binary XML file line #7: Error inflating class paulscode.android.mupen64plusae.SDLSurface

We'll need to come up with another way to work around this problem.  Probably creating a separate OnGenericMotionListener that is attached only when the proper version of Android is running.
Title: Re: Source code location to add native HID joystick support?
Post by: Paul on October 03, 2012, 05:01:21 PM
Fixed the problem.  I just moved the listener to a separate class (JoystickListener.java), and only link with it if the Android build is high enough.
Title: Re: Source code location to add native HID joystick support?
Post by: littleguy on October 03, 2012, 05:15:41 PM
Cool, I was wondering about that and was thinking the same solution.  Thanks for testing.