Project

General

Profile

Actions

Issue #1867

closed

Download app not working when on USB network or RepWifi connection

Added by Fil Bergamo about 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
RepWifi
Target version:
Start date:
01/20/2018
Due date:
06/01/2018
% Done:

100%

Estimated time:
60.00 h
Resolution:
fixed
Device:
Grant:
Type of work:

Description

When no SIM is present, or mobile data are turned off, the Download app is pausing as soon as any download starts, and it never resumes.
This is almost certainly related to Issue #1792.

While #1792 was solved by patching the application framework, I suppose the Download app calls a different function in the framework and not the one that got patched. This could be a possible reason why the issue is not solved for it, while it is solved for other "regular" apps.

I will investigate and probably find a solution.


Related issues

Related to Replicant - Issue #1792: WiFi connection established via RepWifi not correctly reported by the Android Framework to user-space appsResolvedFil Bergamo04/12/201709/01/2017

Actions
Actions #1

Updated by Fil Bergamo about 6 years ago

  • Related to Issue #1792: WiFi connection established via RepWifi not correctly reported by the Android Framework to user-space apps added
Actions #2

Updated by Fil Bergamo about 6 years ago

  • Due date changed from 04/20/2018 to 05/20/2018
  • % Done changed from 0 to 10
  • Estimated time changed from 24.00 h to 60.00 h

TL;DR: it's a long way to the top..

After a first debugging overview, things seem harder than I thought.

Indeed, the Downloads app follows a much more complex scheme in retrieving network information, as opposed to other "regular" apps that simply ask for what kind of network is available.

Downloads app calls multiple functions in the framework, to retrieve various pieces of information about the network connection.
Most of that information is currently NOT available for the connection established via RepWifi.
The same goes with the "reverse tethering" connection. They happen to be kind-of "ghost connections".

I'm still trying to figure out WHERE and HOW the missing pieces of information are generated, stored, retrieved and reported to calling apps.

So, things will be harder and will take longer than expected.
But I'm still confident something can be done to mitigate this.

Actions #3

Updated by Kurtis Hanna about 6 years ago

Just a note for anyone that really needs to download something using RepWifi, one really horrible work around is to use the extremely old Zirco Browser to download things.

Actions #4

Updated by Fil Bergamo almost 6 years ago

Ok, I think I have it solved, somehow!

Everything is still experimental, and things need to be cleaned up, but the POC is working.

I painfully and boringly scanned a relevant portion of the framework's source code, and I found the method that is meant to do exactly what we need:

om.android.server.ConnectivityService.registerNetworkAgent()

which is the method that is used by the internal WiFi framework to register all the details for a network connection (interface, routes, dns...)
Unfortunately, that method is conceived to only be used internally, and is hidden to regular apps.
The same goes with the classes used as input parameters.
This means we can't just use them directly; we need to use java reflection instead. This is why I put up a set of "proxy classes" that wrap up reflection methods for using hidden stuff.

Also, a special permission is needed, in order to be able to call methods on the internal framework:

android.permission.CONNECTIVITY_INTERNAL

so the only way for RepWifi to obtain that permission is to be a system app.

RepWifi already is a system app, when it comes pre-packed with the Replicant zip, but this prevents updating it from F-Droid, because of different signatures. As a result, my intention was (and still is) to stop building RepWifi together with Replicant, and shipping F-droid's prebuilt APK in future releases of Replicant. This would allow people to update RepWifi normally from F-droid, which is much quicker and more convenient than having to wait for the next Replicant release.

So the next step will be to either find a way to obtain the needed privilege without being a system app (not very plausible),
or to programmatically make RepWifi a system app after it has been installed from F-Droid (plausible).

Another point which I'm still investigating is how to notify the system when we disconnect (currently, I only found out how to notify when we connect).
But this should be an easier achievement..

So, to recap..
Stay tuned for RepWifi updates during the next month, as next release will solve this issue.
Also, this new solution is much more elegant than the hackish one that I used with Issue #1792, because it doesn't involve any modification to the application framework itself.
This means that, if everything goes well, we could revert the previous patch on #1792, and remove the hackish part from the framework.

A lot of work still needs to be done, but the solution exists and can be used just fine.

Actions #5

Updated by Fil Bergamo almost 6 years ago

  • % Done changed from 10 to 40
Actions #6

Updated by Fil Bergamo almost 6 years ago

  • Category changed from Framework to RepWifi
  • % Done changed from 40 to 70

Ok, I made it.
I have a working proof of concept for RepWifi to correctly signal both "connection" and "disconnection" to the system.

I tested my work with the Download app, and it definitely works.
The final fix should also be beneficial to other apps that currently can't use the WiFi connection made by RepWifi.

The general concept is this:
by leveraging java reflection, RepWifi is instantiating its own "NetworkAgent", derived from

frameworks/base/core/java/android/net/NetworkAgent.java

it is registering its NetworkAgent to the system's connectivity framework, by calling

android.net.ConnectivityManager.registerNetworkAgent()

From that moment, RepWifi is able to pass "connectivity change" events to the system's ConnectivityService, just like the inner WiFi compoments would do.
All the rest is done by the default, built-in mechanisms: the connection is made available to regular apps through the canonical channels exposed by the frameworks, and, most importantly, it is entirely managed by the framework, including network access permissions and other stuff related to the Android security model.
Everybody is happy with that.

The last part remains to provide a painless way to switch RepWifi from being a regular app to a system app, and then back to regular when it needs to be updated through F-droid.
But this seems like an easy task to accomplish.

So, the issue is solved.
I just need some other time to clean up the code, and package the whole thing into RepWifi.
I will provide next version of RepWifi both on F-droid and on this very page.

Stay tuned.

Actions #7

Updated by Kurtis Hanna almost 6 years ago

Great job, Fil! I greatly look forward to running the next RepWifi release!

How is the code that switches RepWifi from being a regular app to a system app coming along?

Actions #8

Updated by Fil Bergamo almost 6 years ago

  • % Done changed from 70 to 80

Kurtis Hanna wrote:

How is the code that switches RepWifi from being a regular app to a system app coming along?

This one turned out to be pretty simple to implement:
every time RepWifi opens, it checks to be a system app. If it's not, it prompts the user for a confirmation.
If the user agrees, an automated script starts, makes the app a system one, and reboots the phone.
From that moment, it's also possible to update from F-droid without even removing RepWifi from being a system app (provided that it first was installed from F-droid).

As for the rest, I'm doing a massive restyle to the codebase, so next one will be a completely new version of RepWifi. As such, things are going a bit slowly, but I'm pretty confident I'll be able to release before June.
Please folks, keep patience for just a little more.. Next release is worth waiting, I promise!

Actions #9

Updated by Fil Bergamo almost 6 years ago

  • Due date changed from 05/20/2018 to 06/01/2018

Got my HDD fried last week.. :(
This means next release will take even longer than expected.
Sorry about that.

Actions #10

Updated by Kurtis Hanna over 5 years ago

That stinks that your HDD got fried!

Actions #11

Updated by Fil Bergamo over 5 years ago

  • % Done changed from 80 to 100
  • Resolution set to fixed

With considerable delay...
I'm happy to announce that this issue is solved inside RepWifi!

For the moment, there won't be no official release (no f-droid release), as this new version has a whole lot of new features that need to be tested for a while before getting to a stable release.

Nonetheless, I'm distributing an "unofficial" release, signed with my own GPG key, for users' convenience and to allow the community to test the new features.

The compiled APK is released in the original RepWifi forum thread
This new version is v0.9-beta
Please, make sure you verify the attached gpg signature.

The key fingerprint for the signature is:

2D36 719C A4B8 3C7A DD9D  33AE A02A 2F68 CD07 CEAD

As usual, source code is published here

Please, test this beta release, especially in combination with the Download app, and please report any bugs!

Happy Hacking!

Fil

Actions #12

Updated by Kurtis Hanna over 5 years ago

Great work Fil! I've tested a number of the new features for a number of days now and haven't found any bugs in them as of yet. I personally am in favor of moving it from beta to stable.

Actions #13

Updated by Fil Bergamo over 5 years ago

I've just pushed a new git tag on the source repo, so that F-Droid should recognize it as a new release.
The new release is still marked as "beta" in the tag name, so that people are aware that it needs testing.

Nonetheless, F-droid should be publishing the new version of RepWifi in some days (maybe something more that just some days.. it depends on them).

Actions #14

Updated by Kurtis Hanna over 5 years ago

Should we close this issue? It seems to be resolved.

Actions #15

Updated by Fil Bergamo over 5 years ago

  • Status changed from New to Resolved
Actions

Also available in: Atom PDF