Author Topic: Ideas needed: Improving input mapping and listening  (Read 1866 times)

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Ideas needed: Improving input mapping and listening
« on: January 22, 2013, 11:10:41 PM »
Ok, been thinking about this awhile now so I think I'll try to communicate the current input handling process for analog data and the issues we're facing.  As I've said in other posts, the current implementation is kind of brittle and it would be nice to have something a little more robust.  In one sense, the most reliable solution to the problems outlined below is to make the user go through a detailed controller setup process.  However, I expect this would annoy or confuse many users.  We could stuff more options into "advanced" menus and the like, but that may just shift where the confusion happens.  I'm still holding out hope that we can devise a more automatic solution that "just works" like it generally does on a PC or console.

So in a nutshell, here's what's going on...:
  • What you see in the "Controller diagnostics" screen are the raw, unadulterated key and motion events passed from the Android framework.  If controller signals do not show up on this screen, then it will not be a quick-fix to get that particular controller working.  The notable exception is the Xperia PLAY touchpad, which required considerable effort to integrate into Mupen (thanks Paul!).  On the other hand, if user inputs are visible here, then we should be able to map them to Mupen commands without a lot of changes to the existing code. Currently, the app just listens to KeyEvents (digital) and MotionEvents (analog) and uses the user-defined map to convert them to Mupen commands. For digital buttons, this is very easy and the signals are used in raw form.  Analog signals should be equally easy to interpret and convert, but...
  • Xbox 360 controllers have a little quirk.  Unlike most controllers, their analog triggers produce values in the range [-1,1] with a resting value of -1 (most controller triggers use [0,1] and rest at 0).  So when you go to map, you'll instantly map everything to one of the triggers, since it looks like it's fully pressed in the negative direction.  Solutions I've considered include:
        (a) Communicate to users that they may have to press their triggers half-way to remove the bias during mapping.  Obviously a bad user experience, but including it for the sake of completeness.
        (b) Offset the trigger axes (AXIS_Z and AXIS_RZ) so that their resting value is 0.  Or ignore negative values on these axes.  Problem is that most other controllers (e.g. PS3) use those axes for right stick.  So fixing it for Xbox breaks it for most others.
        (c) Hard-code the fix only for xbox controllers.  First problem is that there are so many clones on the market that it would be a real maintenance chore, and would require a user to alert and work with us to put a fix in.  Second problem is that you might not be able to reliably distinguish a clone from another controller.  Might just give you a name like "Generic gamepad" or something from the SDK.
        (d) Add a user option to select between default and Xbox controller styles, then use the solution above.  Or auto-detect but provide an override option to the user to force Xbox or standard mode.
        (e) Somehow infer the bias at run-time based on the analog signals, and let the user be blissfully unaware.  This is the approach that is currently implemented.  Unfortunately it's brittle, as I'll describe later.
  • Assuming we somehow get the bias issues fixed, we're not out of the woods yet.  Some controllers do just the opposite: they map a stick to [0,1] with a rest position of 0.5.  I've only encountered this with one controller (old Logitech Wingman Rumblepad) so maybe we can just ignore it and leave good enough alone.  Or we could try solving it by
        (a) Normalizing all analog values to [-1,1].  That is, newValue = -1 + 2 * (rawValue - minValue) / (maxValue - minValue) where the min and max values are easily obtained through the SDK.  This solves problem (3) but now introduces problem (2) to most controllers on the market.  For example a PS3 trigger with resting value 0 and range [0,1] now behaves just like the xbox controllers.  My hope was that I could come up with a reliable solution to (2) so that this normalizing trick wouldn't create new issues.
  • Assuming (2) and (3) are solvable, there are still other quirks to be solved.  For example, some controllers send out a digital signal in addition to the analog signal when the analog crosses the 0.5 threshold.  For example, the PS3 controller sends out KEYCODE_L1 whenever AXIS_LTRIGGER crosses 0.5.  If want to map left trigger to an analog signal, we must guarantee that AXIS_LTRIGGER is heard first.  The obvious solution is just to reduce the threshold on analog signals during the mapping process.  Simple solution but brittle, as I'll discuss later.  There are also more confusing cases, such as the Nyko PlayPad (and Pro) controllers, which send out AXIS_X and AXIS_HAT_X simultaneously for the left stick, but rounds the value on AXIS_HAT_X, i.e. AXIS_HAT_X = math.round(AXIS_X)
  • Some controllers combine multiple modalities in a single device.  E.g. many of the new "Android" controllers send two AXIS_X signals - one coming from a "mouse" and another coming from a "joystick" whenever you press one stick.  Another example is the OUYA controller which has both a touchpad and a left stick that deliver AXIS_X and AXIS_Y signals.  This is actually easy to test for, but I only discovered the issue (and solution) this week.

So, all told, issues 3, 4, and 5 are not too big a deal. Issue 2 has multiple possible solutions (chime in if you have other ideas) but none without flaws.  Option (2e) is currently implemented but is still brittle.  Here's the pseudocode:
Code: [Select]
on analog input (AXIS_ARRAY):

    strongest_val := 0
    AXIS_strongest := null

    foreach (AXIS in AXIS_ARRAY)
        // normalize
        AXIS_val := -1 + 2 * (AXIS_val - AXIS_min) / (AXIS_max - AXIS_min)
        // identify bias
        if (AXIS_bias not defined)
            AXIS_bias := round(AXIS_val)
        // remove bias
        AXIS_val := AXIS_val - AXIS_bias
        // record the strongest axis
        if (abs(AXIS_val) > strongest)
            strongest_val := abs(AXIS_val)
            AXIS_strongest := AXIS
    // map the strongest axis if appropriate
    if (strongest_val > threshold)
        map.put(AXIS_strongest, MUPEN_COMMAND)
Threshold can be set to a value less than 0.5 to address issue (4). In everything I've seen, the biases are always in the set {-1, 0, 1}, hence the rounding function. Though it's not strictly necessary and I've tinkered with not rounding it.

Now here's where the brittleness comes in.  It all has to do with when the bias is defined.  If the user ramps up an analog input very quickly (easy to do, especially with stiff nubs) and their cpu is already burdened, the first analog value heard might be large enough to be rounded to the wrong value.  Removing the rounding process may help but is not a panacea.  For example, true bias = 0, first value heard (and inferred bias) is 0.7 and threshold is 0.4... this axis will become unmappable regardless of whether rounding is used or not.  Other options are to allow the user to undefine the biases on command so they can be re-inferred.  Or let the user manually specify the biases.  Also, it helps to limit the bias inference only to AXES that truly might be biased.  For example, AXIS_X and AXIS_Y are always unbiased for joysticks; inferring the bias for these axes is pointless and can only introduce errors.  So the current implementation limits the bias inference process to AXIS_Z and AXIS_RZ (for Xbox); AXIS_LTRIGGER and AXIS_RTRIGGER (for PS3); AXIS_GAS and AXIS_BRAKE (for Nyko PlayPad); and AXIS_GENERIC_1 (for OUYA).  But since the inference process is not perfect right now, the bias on any of these axes may be erroneous, leading to mapping difficulties.

The next approach that I may implement soon would allow the user to define which axes are triggers and which are sticks.  This could take the form of some sort of list and radio buttons, or it could be more seamless where the user is asked to wiggle all sticks, then wiggle all triggers.  This would remove the inference (and normalization) requirements and might be the simplest and least annoying solution in the grand scheme of things.  It could also auto-detect based on ranges, so many users would never even need to use the wizard.

Whew.  Just talking through this has helped me.  If you've stuck with me through all that, congratulations.  If you can offer any ideas or concerns, I'd love to hear them.
« Last Edit: January 22, 2013, 11:22:49 PM by littleguy »
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version