Author Topic: Source code location to add native HID joystick support?  (Read 13620 times)

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Source code location to add native HID joystick support?
« on: September 17, 2012, 10:56:10 AM »
Awesome app!  I'm actually building my own Arduino-based HID joystick & case for my Nexus 7 to enhance the gaming experience further.

I'd love to have native HID analog support for this app (Android 3+), but I know your attention is (understandably) diverted from this project for now.  I'd like to try adding the functionality myself, so I downloaded the source code from git.  However I was quickly overwhelmed with the amount of code!  Would you mind pointing me to the particular source files/classes pertinent to adding native HID/analog support, so I can give it a try?  If you have any cursory thoughts on the matter I'm all ears as well.  Of course if I'm successful, I will be happily obliged to contribute my mods back into the next release.

As an aside,  I am using USB/BT Joystick Center app with a Logitech gamepad, everything works fine.  Unfortunately my custom HID joystick (which is plug-n-play in windows) isn't working with Joystick Center and I've been in contact with that app's developer.  So between that app and this, I hope to have a solution to  Mupen64Plus + analog controls +my controller.

2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3496
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Source code location to add native HID joystick support?
« Reply #1 on: September 17, 2012, 12:47:57 PM »
This is something I was planning to look into at some point, but haven't gotten around to.  Unfortunately, that means I don't have a lot of useful information with regards to getting data from Bluetooth and interpreting it.  I think that will have to be your first step.  I'd start by reading the Android documentation for Bluetooth to see what you have to work with on the Java side.  You may end up needing to interface from the native side if the Java API isn't sufficient.

I'd recommend putting together a list of specific things you need to know, and searching on Stack Overflow for people who have asked similar questions.  If your questions have not already been answered, then post new questions there.

Once you know how to receive and interpret data from your joystick in Android, then processing it into a form that can be used by Mupen64Plus AE will be pretty simple.  If you have access to the data on the Java side, then the easiest way would be to call the method updateVirtualGamePadStates (in main/src/paulscode/android/mupen64plusae/GameActivityCommon.java) whenever you have new input from the joystick.  (Ignore the name.. it is used by the Virtual Gamepad, but can be used anywhere you have input to send to the emulator):
Code: [Select]
/* From the N64 func ref: The 3D Stick data is of type signed char and in the range between
80 and -80. (32768 / 409 = ~80.1) */
    // Sends virtual gamepad states to the input plug-in:
    public static native void updateVirtualGamePadStates( int controllerNum, boolean[] buttons, int axisX, int axisY );

If you have access to the data on the native side rather than from Java, then simply update the following array values in main/jni/input-sdl/src/plugin.c (see the function right below these lines in the code for an example):
Code: [Select]
//// paulscode, for the virtual gamepad:
unsigned char virtualGamePadButtonState[4][14];
signed char virtualGamePadAxisState[4][2];

Sorry I can't help you much with actually getting input data from the joytick, but I'm happy to help if you run into any problems linking up with the emulator once you get to that point.  You'd also need to add some interface to the GUI, and I can help with that as well.
« Last Edit: September 17, 2012, 12:52:41 PM by Paul »
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Source code location to add native HID joystick support?
« Reply #2 on: September 17, 2012, 06:57:58 PM »
Awesome!  Actually, I'm not using a bluetooth joystick.  It's just a plain usb joystick - generic HID driver.

Android programming is still new to me, but I've been programming hardware for quite a while.  I'll see what I can do.  Thanks for the pointers for the important files.  I'll take a look tonight.
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3496
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Source code location to add native HID joystick support?
« Reply #3 on: September 17, 2012, 07:20:49 PM »
Ah, gotchya (not sure why I assumed it was a Bluetooth joystick).  Probably Android's API documentation for USB Host will be a good place to start researching.  Basic process will be the same -- figure out how to get the input from the joystick, and then link that up with the emulator either on the Java side or the native c/ c++ side.
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Source code location to add native HID joystick support?
« Reply #4 on: September 17, 2012, 09:54:48 PM »
Ok, doing some more digging, this is really helpful.  A few more questions:

1.  So it looks like you sprinkled a few lines here and there into the original C++ source for Mupen64Plus, then do the bulk of your work in Java, using Java wrappers to invoke the functions on the C++ side?

2.  Where does the code reside that bridges your app with BT/USB Joystick Center app?  In other words, is the translation code a part of Mupen64PlusAE source or is it part of the Joystick Center source?  If the former, what file contains that code?  If the latter, then it sounds like I need to write a little Android app that reads the USB and invokes Mupen64 code?  Or am I way off the mark here?

2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3496
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Source code location to add native HID joystick support?
« Reply #5 on: September 17, 2012, 11:07:54 PM »
1.  So it looks like you sprinkled a few lines here and there into the original C++ source for Mupen64Plus, then do the bulk of your work in Java, using Java wrappers to invoke the functions on the C++ side?
Right, the native c/ c++ code for the input plug-in is as close to the code that ships with the PC version of Mupen64Plus as possible.  My reasoning behind this is that I want to eventually merge this project back to the original source, while in the mean time being able to parallel updates as Richard42 releases them.  So it makes sense to change as little as possible on the native side, and to code the Android pieces on the Java side of the house, and interact with the emulator through the config files wherever possible, rather than through JNI to the native c/ c++ code.

2.  Where does the code reside that bridges your app with BT/USB Joystick Center app?  In other words, is the translation code a part of Mupen64PlusAE source or is it part of the Joystick Center source?  If the former, what file contains that code?
Actually translations of the input data happen in both apps.  Note that the BT/USB Joystick Center app is an IME designed to run separately from the apps using it.  It achieves this by translating the joystick data into keyboard events with a specific pattern, which can be received by any app listening for input from the keyboard.  I have my code interpret this pattern and translate that into the form used by the emulator.  This happens in the method onKey (in main/src/paulscode/android/mupen64plusae/SDLSurface.java):

Code: [Select]
        if( keyCode > 255 && Globals.analog_100_64 )
        {
            key = ( keyCode / 100 );
            if( action == KeyEvent.ACTION_DOWN )
                str = ( (float) keyCode - ( (float) key * 100.0f ) );
        }

Notice that I have a boolean that tells the emulator whether or not to do the translation.  At the moment, key event translation is only done for input from the USB/BT Joystick Center and BT Controller IMEs.  Both produce the same (x * 100 + [0-64]) key code pattern for analog input (thanks to my recommendations to DroidBean when he was adding analog support to his app), so I have this boolean hard-coded to true.  When I add support for other IMEs using a different pattern in the future, this will have to be controlled from the GUI in the settings menu instead.

In your case, you probably won't be writing an IME (unless that is your goal), so there won't be any need for the two-step translation process.  You could simply translate the joystick input data directly into the form used by the emulator, bypassing the key code thing all together.  Of course, if you do all the work of figuring out how to get the data, you may decide to write an IME anyway so you can use your joystick with your other apps, not just Mupen64Plus AE.
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Source code location to add native HID joystick support?
« Reply #6 on: September 18, 2012, 07:21:28 AM »
Thanks!  This is very clear, and thanks for the quick feedback.  I'll start tinkering with the Android usb host api until I get the mechanics of the joystick data all figured out.  Hope to make some progress over the next week or so.
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Source code location to add native HID joystick support?
« Reply #7 on: September 23, 2012, 10:40:41 PM »
Ok, I had some time this weekend to start working on this.  I wrote a trivial app that just displays native joystick axes in a TextView object in the main activity.  Isn't much work at all to get the data, just override the onGenericMotionEvent method and make a few calls to the API.

Code: [Select]
public class MainActivity extends Activity {

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
    }

    @TargetApi(12)
    @Override
    public boolean onGenericMotionEvent(MotionEvent event) {
   
        // Get the device (not used below, but maybe useful later)
    InputDevice device = event.getDevice();
   
        // Spit out some debug info
    String message = "MotionEvent:";
    message += "\r\nAction: " + event.getAction();
    message += "\r\nActionIndex: " + event.getActionIndex();
    message += "\r\nActionMasked: " + event.getActionMasked();

        // These are the axes used by most XBox360 (XInput) gamepads, including Logitech F710
    message += "\r\n  AXIS_X : " + event.getAxisValue(MotionEvent.AXIS_X);           // Left-stick X
    message += "\r\n  AXIS_Y : " + event.getAxisValue(MotionEvent.AXIS_Y);           // Left-stick Y
    message += "\r\n  AXIS_Z : " + event.getAxisValue(MotionEvent.AXIS_Z);           // Left analog trigger
    message += "\r\n  AXIS_RX: " + event.getAxisValue(MotionEvent.AXIS_RX);          // Right-stick X
    message += "\r\n  AXIS_RY: " + event.getAxisValue(MotionEvent.AXIS_RY);          // Right-stick Y
    message += "\r\n  AXIS_RZ: " + event.getAxisValue(MotionEvent.AXIS_RZ);          // Right analog trigger
    message += "\r\n  AXIS_HAT_X: " + event.getAxisValue(MotionEvent.AXIS_HAT_X);    // D-pad left/right
    message += "\r\n  AXIS_HAT_Y: " + event.getAxisValue(MotionEvent.AXIS_HAT_Y);    // D-pad up/down
   
        // But not all gamepads use those particular axes...
        // ...some may put their data in other axes (esp. older gamepads)
        // So print out any other active axes
    for (int i = 0; i < 50; i++) {
    double value = event.getAxisValue(i);
    if (value != 0)
        message += "\r\n  Axis " + i + ": " + value;   
    }

        // Write the info onto the GUI
    TextView view = (TextView) findViewById(R.id.text_info);
    view.setText(message);   
    return true;   
    }   
}

Axis values are floats ranging from -1.0 to 1.0.  Note that the d-pad is also registered as an axis, even though it only ever reports the values {-1.0, 0.0, 1.0}.  Positive is to the right and down (not up) for all axes.  Analog triggers are 1.0 when fully depressed, -1.0 when released.  Handling digital buttons as key presses will continue to work, although there are a few buttons (e.g. clicking the joysticks) that aren't natively mapped as a keypress.

I just noticed that you are handling input using the listener pattern rather that overrides.  That should be fine too.  You would just need to implement View.OnGenericMotionEventListener.

At this point, I think the biggest task will just be extending the user settings interface.  We probably need a way for Android 3.1+ users to remap axes if they're not using an Xbox controller or don't like the default arrangement.  Perhaps a way to disable native support too (just revert to the current approach), although I can't think of a reason they'd want to do that.

So I thought I would take a crack at that part too, just to identify any roadblocks.  I successfully built Mupen64Plus-ae from the source on github.  I'm able to start the app and navigate menus, but I get a force close as soon as I try to start a ROM.  I might have a build error or something, so I'm digging into that.  I'm doing this in Eclipse with the Android SDK and NDK, on Windows.  Maybe I should try building from Linux.  What O/S are you using for development?
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3496
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Source code location to add native HID joystick support?
« Reply #8 on: September 23, 2012, 11:56:53 PM »
For building, I am using Linux, but others have built successfully in Windows and Mac.  Lioncash has been doing a bunch of core updates from the main repository, so something may be broken.  I'll check when I have some time tomorrow.  In the mean time, is there anything useful in the logcat output after the crash?

I was planning to implement View.OnGenericMotionEventListener since that is used by the Xperia Play with ICS as I recall (the existing NativeActivity method is great for pre-ICS Xperia Play's, but doesn't work in ICS very well, because the action bar input is blocked).  It will have to be done in a way that doesn't break the emulator for older devices though, because that interface is only available in API level 12 and higher.  This may not apply to your case anyway, if you are just building a custom version of the emulator for yourself, since it is for your own custom joystick you are building.
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Source code location to add native HID joystick support?
« Reply #9 on: September 24, 2012, 06:31:34 AM »
Yeah, I'll check the logcat.  This is my first Android programming experience.  I'd like to get this into the main branch if it's not too hard, since I know many others would like the functionality.  The joystick I'm building is just a standard Xbox controller clone, so I don't view this as a personal customized version of Mupen just for myself.

We could probably implement View.OnGenericMotionEventListener in a separate class, analogous to the listeners in SDLSurface.java.  We would only instantiate and register the listener if the user's Android version was >=12.  I definitely don't want to break compatibility with older Android versions.

I'll do some more digging, hopefully get the source running properly on my device.  Thanks for the help.
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3496
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Source code location to add native HID joystick support?
« Reply #10 on: September 24, 2012, 10:12:16 AM »
Sounds good - that ought to work while still keeping it compatible on older devices.

I just built the latest commit (ac71f08) from Eclipse, and it isn't crashing on my device.  Was there anything useful in the logcat?
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Source code location to add native HID joystick support?
« Reply #11 on: September 26, 2012, 07:00:14 PM »
Ok, after a few wrong turns I got everything to build properly on windows.  I posted some updated instructions on another thread for total noobs like me.

So I'll start integrating my additions into the source code, hopefully should have something to share in the next week or so!
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Source code location to add native HID joystick support?
« Reply #12 on: September 27, 2012, 04:39:40 PM »
Ok, that wasn't too hard at all once I climbed the learning curve!  Patch files attached below. Works like butter, and no longer dependent on a $5 IME.  I may be imagining things, but I feel like I have a lot more control sensitivity now making mario walk slowly.  With USB/BT Joystick Center app, it never felt this smooth.

Right now I have all the analog axis mappings hard-coded to an Xbox controller's axes.  I'm using the right analog stick for the c-pad buttons, and the left analog trigger for the z-button.  This lets me map the R and L buttons back to the left and right shoulder buttons of the controller.  I added some lines to the Updater.java code so that IF you're running Android 3.1 or greater, the defaults are mapped properly to an xbox controller out of the box.  Otherwise it keeps the legacy default key mappings.

The real work will involve the user interface, to permit remapping of the analog stick, and to permit analog axes to be mapped to buttons (e.g. left-analog-trigger --> Z, right-stick --> C-buttons).  This will probably require someone with a little more Android gui programming experience than me.
« Last Edit: September 27, 2012, 04:46:00 PM by littleguy »
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3496
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Source code location to add native HID joystick support?
« Reply #13 on: September 27, 2012, 04:45:07 PM »
Thanks, I'll work on incorporating this into the code and updating the GUI to handle mapping the axes.  Thanks for working on this!  It is definitely something I've been wanting to add at some point.

--EDIT-- committed the changes to the repository, and added a note that there needs to be integration into the GUI button mapping section, and that the app needs to be tested on a device with Gingerbread to ensure it is not broken for pre-Honeycomb users.
« Last Edit: September 27, 2012, 05:19:05 PM by Paul »
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3496
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Source code location to add native HID joystick support?
« Reply #14 on: September 27, 2012, 05:03:56 PM »
One question:  Do you know if the following line will limit the SDLSurface class to API level 12 (in other words it will not be available on pre-Honeycomb devices)?  I don't have a device running Gingerbread to test it on at the moment.

Code: [Select]
@TargetApi(12)
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted