Author Topic: Brainstorming Version 3.0  (Read 123778 times)

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Brainstorming Version 3.0
« Reply #45 on: September 24, 2013, 09:20:01 PM »
I mentioned this briefly in chat, and on the OpenPandora boards.  For some time, I've been wanting to restructure the folders to better match Richard's project.  The 3.0 update might be a good time to take a serious look at this.

I did a little thinking, and I believe the basic structure would look like this:

The root folder will contain folders for all the core modules, using the same names as upstream.  (Recommend removing rsp-hle-nosound.. Does this opimization really have a noticeable speed difference?)
mupen64plus-audio-sdl/
mupen64plus-core/
mupen64plus-input-sdl/
mupen64plus-rsp-hle/
mupen64plus-ui-console/

-- EDIT -- (and of course the video plugins)

Each of those will have in them:
projects/android/
This folder will contain an Android project to allow building the native .so plug-in stand-alone if desired

In addition to the core folders, the root folder will contain:
mupen64plus-ae/
This folder will contain the "master" android project.  Jni folder will have sub-folders for Xperia Play and ae-bridge, as well as a single Android.mk file that builds all the core modules (will use relative paths when referencing the source files: ../mupen64plus-core/core/src, etc).  Anyone who wanted to build a smaller APK without a particular plugin (or to add their own custom plugin) could just edit the relevant block in this Android.mk file.  This folder will also contain all the java, resources, etc for building the app.

This structure would work well for command-line compiling via ndk-build followed by ant.  However, I will need to play around with Eclipse to figure out how to set up an Eclipse project for this.  Obviously having the Eclipse project reside in the "master" android project folder would mean the native C/C++ files would be outside of the project, which would not be ideal.  Would be better if the Eclipse project could be set up from the root folder.  Alternately, could use separate individual Eclipse projects for the "master" android project and each plug-in ("master" project would build everything, and other projects would build the plug-ins individually).
« Last Edit: September 24, 2013, 09:28:19 PM by Paul »
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #46 on: September 24, 2013, 09:41:47 PM »
(Recommend removing rsp-hle-nosound.. Does this opimization really have a noticeable speed difference?)

I agree, let's remove it.

This folder will contain the "master" android project.  Jni folder will have sub-folders for Xperia Play and ae-bridge, as well as a single Android.mk file that builds all the core modules (will use relative paths when referencing the source files: ../mupen64plus-core/core/src, etc).  Anyone who wanted to build a smaller APK without a particular plugin (or to add their own custom plugin) could just edit the relevant block in this Android.mk file.  This folder will also contain all the java, resources, etc for building the app.

Sounds good to me.  One thing though, we don't need mupen64plus-input-sdl.   I'm guessing we put SDL2, gles2glide64, gles2n64, png, and libsamplerate outside of mupen64plus-ae as well?  I'd say keep input-android in mupen64plus-ae for the time being since it's not upstream and is still very specific to android.

Obviously having the Eclipse project reside in the "master" android project folder would mean the native C/C++ files would be outside of the project, which would not be ideal.

I like the folder organization you laid out, and agree that it would be much tidier to keep android stuff isolated in mupen64plus-ae.  My guess is that the project would build just fine from eclipse, but you wouldn't be able to browse into the native stuff easily in the IDE.  Will have to think about that one a bit...
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Brainstorming Version 3.0
« Reply #47 on: September 24, 2013, 09:58:14 PM »
Sounds good to me.  One thing though, we don't need mupen64plus-input-sdl.
It will be used by the Open Pandora port, so will need to keep it for that (no Android project for this one).  input-android would make sense to keep under the mupen64plus-ae/jni folder, since it is Android-specific.

I'm guessing we put SDL2, gles2glide64, gles2n64, png, and libsamplerate outside of mupen64plus-ae as well?
Yes, forgot to bring up the other libraries besides the core ones, but yes they would also go into the root folder.  Should be named to match the corresponding upstream project (glide64mk2, for example), to promote synchronization.

My guess is that the project would build just fine from eclipse, but you wouldn't be able to browse into the native stuff easily in the IDE.  Will have to think about that one a bit...
If all else fails, it might not be too big of a deal to just allow creating separate Eclipse projects for each module that is outside of the main Android project.  Wouldn't need to create an Eclipse project for every module, just the ones being edited.
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #48 on: September 24, 2013, 10:11:01 PM »
Oh right, openpandora.

Yes, if the goal is upstream sync, then it may be just as well if the external libs aren't visible in the eclipse project.  We should be treating them as black boxes ideally.
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #49 on: September 25, 2013, 08:14:20 AM »
One other thing, before we move/rename the libraries, we should let Sven know.  I'm not entirely sure the process he goes through to merge the upstream back into master.

Just did a quick test, moved the upstream libs outside the ae project.  From eclipse it compiles from just fine but I can't link yet.  Probably not too hard to track down.  Here's the makefile I used:

Code: [Select]
JNI_LOCAL_PATH := $(call my-dir)
NATIVE_ROOT_PATH := $(JNI_LOCAL_PATH)/../..

AE_BRIDGE_INCLUDES := $(JNI_LOCAL_PATH)/ae-bridge/
M64P_API_INCLUDES := $(NATIVE_ROOT_PATH)/mupen64plus-core/src/api/
SDL_INCLUDES := $(NATIVE_ROOT_PATH)/SDL2/include/
PNG_INCLUDES := $(NATIVE_ROOT_PATH)/png/include/
SAMPLERATE_INCLUDES := $(NATIVE_ROOT_PATH)/libsamplerate/

COMMON_CFLAGS :=                    \
    -O3                             \
    -ffast-math                     \
    -fno-strict-aliasing            \
    -fomit-frame-pointer            \
    -frename-registers              \
    -fsingle-precision-constant     \
    -fvisibility=hidden             \

COMMON_CPPFLAGS :=                  \
    -fvisibility-inlines-hidden     \

include $(call all-subdir-makefiles)
include $(wildcard $(NATIVE_ROOT_PATH)/*/projects/android/Android.mk)

2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #50 on: September 25, 2013, 09:05:56 AM »
On a different note, it might be a good idea to consider using a new package name for version 3.  There are some obvious drawbacks, but the most important benefit would be the use of the sharedUserId attribute in the manifest:
http://developer.android.com/guide/topics/manifest/manifest-element.html#uid

The sharedUserId would make it much easier to create multiple apps on the store that interact with each other.  I.e. we could more easily produce add-on apps for debugging/expert settings/data migration, etc.  I think it would also allow us to package native libs as separate "apps", e.g. if we wanted to distribute experimental video plugins but not have them installed by default.  We wouldn't have to publish every add-on on the google play either, so if it were a really niche add-on (e.g. very specific debugging tools) we could just make them available on the forum only and point specific users to them.

The thing about using sharedUserId is that you can't suddenly apply it to a package that's already been published (users will get upgrade errors).  So you have to make the decision up front.  See here for more: http://java-hamster.blogspot.com/2010/05/androids-shareduserid.html

Some other benefits of a new package name is that it would allow users to keep v2 and v3 installed on the same device.  We could make a third app that safely migrates the datafiles from v2 to v3.  So upgrade wouldn't be an all-or-nothing event.  I was thinking we could borrow FilePwn's implementation for a near bullet-proof file migration app.  (OUYA developer logs show zero crashes after the fourth release (Aug 20), with an install base of 20k users.)  For now, I'd prefer that source to be closed (I might put it up on google play someday), which would be a good reason to make the migration a separate app.

Of course the problems with changing the package name are:
 - Users need to be aware of the new package.  This could probably be addressed by adding a check in v2 to see if v3 is installed, and if not pop up a message.
 - We start over with google play statistics.  This might not be entirely bad.  If we have good conversion from v2 to v3, we'll hopefully have our high install count back soon.  And if we do a good job with version 3, it may be a good thing to shed all the old obsolete 1-star ratings from v2.

Finally, if we change the package name, I'd suggest com.paulscode.mupen64plus (or org or net or whatever) just to keep with the recommended naming conventions.  We could even consider just calling the app Mupen64Plus-Android.

Anyhow, just wanted to throw those out there.  I'm sure there some other drawbacks I hadn't thought of, so all discussion welcome.

« Last Edit: September 25, 2013, 09:12:20 AM by littleguy »
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Brainstorming Version 3.0
« Reply #51 on: September 25, 2013, 10:05:09 AM »
A couple other big drawbacks that I can think of are:

1) Folks who donated would have to donate again, or be stuck with the free version for 3.0 (not that they are any different).  This could be misconstrued as a money-grab by some users who have already donated.

2) A new user who is looking for a N64 emulator would suddenly be met with 3 or 4 instances of Mupen64Plus AE published on Google Play, which could be confusing.  A lot of folks already have trouble wrapping their head around the paid version being no different than the free version.

3) In the case of another bogus DMCA take-down, the number of flagged apps would be greater, which would increase the likelyhood of a subsequent ban of my developer account.


Perhaps a better solution would be to keep the same package name, and just warn people in the previous version before the change that they are going to experience an upgrade error and will need to uninstall the app and reinstall it.  We can even make it nag them every time they launch the app, rather than just once (so there is no chance of them missing it).  Release it a couple weeks at least before the change, so that majority of users have updated and received the nag dialog.
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #52 on: September 25, 2013, 10:06:28 AM »
Good points...
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #53 on: September 25, 2013, 10:13:20 AM »
I am still concerned about the data upgrade and all-or-nothing-ness of an in-place upgrade.  Keeping two separate apps would vastly simplify the transition.

For 2) maybe you just rename the v2's "Mupen64Plus AE (obsolete)" and change the icon?

But 1) and 3) are definitely serious...
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #54 on: September 25, 2013, 10:21:30 AM »
For 1) here's an alternative, but it requires a fair bit of work.  Move donations for v3 to an in-app payment, rather than a separate app.  Also helps with 2).  Then when v2-donate users install v3, we do some sort of account check and apply the donation entitlement to their v3 app.  As if they already donated in-app.

For 3).... don't know.  I wish google was more opaque on this stuff.  I wonder if you can give any sort of warning notice to google to pre-empt false dmca takedowns?  Has the previous dmca takedown been wiped off your "permanent record"?  I.e. it sounds like you are concerned with an accumulation of bad marks against your account.
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Brainstorming Version 3.0
« Reply #55 on: September 25, 2013, 10:33:58 AM »
Going back to the folder restructure topic for a second, one drawback I thought of is that moving files around will cause them to lose history.  Might talk with Sven to see how he was able to bring in the upstream history into our repository (like some way of merging the file from the old location into the new location)
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #56 on: September 25, 2013, 10:36:08 AM »
Agreed, that's what I was attempting to say a few posts back.  Need to get in touch with Sven...
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #57 on: November 20, 2013, 03:51:04 PM »
Ok, the rails are finally greased for me to start working on this.  Whew.  I'll start laying stuff out on some concept branches over the next few weeks.

Regarding restructuring the folders, I'll do a proof-of-concept to make sure all the build stuff works correctly and put that on a little branch of its own.  I recommend we continue to keep a copy of each plugin/library in our own single github repo.  Makes it much easier to go back in time to fix regressions.  If I get it working properly, I'll put it on ice and we can do the restructuring at the very end just before we release 3.0.
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version

Offline Paul

  • Administrator
  • double
  • *****
  • Posts: 3499
  • Developer
    • View Profile
    • PaulsCode.Com
Re: Brainstorming Version 3.0
« Reply #58 on: November 20, 2013, 05:54:19 PM »
Excellent.  That sounds like a good plan.  I may be switching gears for the next couple weeks to work on the SoundSystem library.  There are a few long-standing bugs I need to address, and I want to add a software mixer to LibraryJavaSound plugin.  I'm also going to finish the Android port with an SLES plugin, which will hopefully get me more comfortable with SLES and apply the lessons learned to the new audio plugin for Mupen64Plus AE.  I definitely want that to make it into 3.0.
Device: Samsung Galaxy Nexus i515
CPU: TI OMAP4460, 1.2 GHz (dual core, ARM Cortex-A9)
GPU: PowerVR SGX540, 307 MHz
RAM: 1 GB
Resolution: 720 x 1280
Rom: omni-4.4.4-20141014-toro-FML KitKat 4.4.4, rooted

Device: Eee PC 1015PEM
CPU: Intel Atom N550, 1.5 GHz (dual core, x86)
GPU: Intel GMA 3150, 200 MHz (dual core)
RAM: 2GB
Resolution: 1024 x 600
Rom: android-x86-4.3-20130725 Jelly Bean 4.3, rooted

Offline littleguy

  • Moderator
  • double
  • *****
  • Posts: 1945
    • View Profile
Re: Brainstorming Version 3.0
« Reply #59 on: November 20, 2013, 06:50:39 PM »
SLES probably holds the key to removing latency.  RetroArch has no noticeable latency in their mupen branch, and it looks like they're using a SLES plugin on Android.  So there's an existence proof.  I'm still not entirely confident that the Android portion of SDL2 is up to snuff.  Entirely possible it's just a bug in their glue code.  Might be possible to temporarily swap out the Android for the Linux audio driver in SDL...  I may try that tonight, see if there's a quick fix.
2012 Nexus 7, rooted stock Lollipop
Samsung Galaxy Victory, rooted stock Jelly Bean
Xperia PLAY, stock Gingerbread
OUYA, retail version