PaulsCode Forum

Mupen64Plus AE => General Discussion => Topic started by: Paul on January 01, 2016, 10:50:54 AM

Title: Bandwidth Limit Exceeded
Post by: Paul on January 01, 2016, 10:50:54 AM
Some of you may have noticed yesterday, that on the last day of December, the monthly bandwidth was exceeded for PaulKnows.Com (which currently only really consists of subdomain, which this website PaulsCode.Com is an alias for).  We are receiving enough in Mupen64Plus AE donations to order a higher tier for increased bandwidth, and I have done that (double the previous tier), and we can go even higher if needed.  That said, it might be wise to take the time and analyze the hotspots to determine where the most bandwidth is being consumed, and see if there are ways to reduce bandwidth consumption in those areas.

Here is a breakdown of bandwidth consumption by folder (this is probably the best metric to determine where users are consuming the most data).  I've highlighted some of the interesting areas that are not part of the forum which are consuming bandwidth in the GB's:


Obviously, the AutoBuilds are consuming the highest bandwidth.  Over 19K users in December, consuming over 600 GB of bandwidth.  This could be an area where we can look at reducing bandwidth.  Some options that come to mind are 7-zipping to squeeze a little more compression out of the files (although APKs are already ZIP files, so this wouldn't likely have much effect), or purchasing a separate, cheaper file storage solution.  Another option (which I don't like, but should put it out there for discussion) would be to stream the AutoBuilds (versus posting open links) and make the routes to stream them only available to logged in users on the forum.

Another one of potential concern is the CoverArt folder.  This isn't consuming nearly as much bandwidth as the auto-builds, but I am surprised to see over 37K hits on this folder in one month.  My thought is that there are probably Mupen64Plus AE clones on the market which are using the new gallery UIs, and pulling cover art from our servers.  Another possibility is that various emulation websites or applications are pulling from us simply because we have made the files open to the wide web.  I'm not necessarily opposed to this, but if it becomes such a bandwidth hog that users of the official app are being negatively impacted, we might want to think up a validation system of some type to ensure only our app can pull cover art from our website.  As with the AutoBuilds, a couple other immediate solutions that come to mind are 7-zip and purchasing a separate file storage solution.

The 19 Jan build isn't too surprising, since that is the latest published build of Mupen64Plus AE.  I mainly highlighted it for comparison with the AutoBuilds usage (looks like folks are primarily pulling the latest builds, which is good -- indicates there is still a large community of interest in recent work on the project).

The 23 Dec and 7 May builds are more interesting to me.  I am not sure what makes these two special.  23 Dec is the next-to-last published build, so some of those numbers could be due to regressions, but the numbers are still rather high considering how long it has been since the last published update.  And I really don't see anything unique at all about the 7 May build.  My thought is that maybe there are some outdated links on other websites that are linking back to those files.  It may be possible to free up a little bandwidth by changing the links for those two builds.

I'll be looking at the options some more over the next few days, and let everyone know what I have decided.  If anyone has input on any of my suggestions, or some other ideas for addressing bandwidth consumption, feel free to post.
Title: Re: Bandwidth Limit Exceeded
Post by: retroben on January 01, 2016, 04:06:07 PM
In hindsight,probably not a good idea to have put autobuilds into the menu for the bandwidth reason.
However,links to both the latest and a confirmed stable build could be a better idea while keeping the full set back on the index page.
Another impact cause was probably the holiday season where many new devices are activated and internet use is dramatically increased.

I wish Google would make their fiber internet more globally accessible and make a cheaper option with decent speeds.
(less than gigabit but more speed than cable and at a lower price would be great)
Title: Re: Bandwidth Limit Exceeded
Post by: Paul on January 01, 2016, 05:06:54 PM
Another thought I had for the auto builds, since the bandwidth is mainly consumed by lots of individual users downloading the same files, would be to host torrent files instead.  This would allow the bandwidth usage to be spread out and shared by the users, with less impact on the server as more users access them.  I would need to provide clear instructions though if I go that route, since I know a lot of folks are not familiar with torrents.
Title: Re: Bandwidth Limit Exceeded
Post by: williansti on January 02, 2016, 06:49:36 AM
I found a clone of the emulator, based on the latest revisions, a copy shameless