https://redmine.replicant.us/https://redmine.replicant.us/favicon.ico?15984615062018-01-20T20:35:49ZReplicantReplicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=69882018-01-20T20:35:49ZFil Bergamofil.bergamo@riseup.net
<ul><li><strong>Related to</strong> <i><a class="issue tracker-3 status-15 priority-21 priority-default closed behind-schedule" href="/issues/1792">Issue #1792</a>: WiFi connection established via RepWifi not correctly reported by the Android Framework to user-space apps</i> added</li></ul> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70032018-01-24T18:29:47ZFil Bergamofil.bergamo@riseup.net
<ul><li><strong>Due date</strong> changed from <i>04/20/2018</i> to <i>05/20/2018</i></li><li><strong>% Done</strong> changed from <i>0</i> to <i>10</i></li><li><strong>Estimated time</strong> changed from <i>24.00 h</i> to <i>60.00 h</i></li></ul><p>TL;DR: it's a long way to the top..</p>
<p>After a first debugging overview, things seem harder than I thought.</p>
<p>Indeed, the Downloads app follows a much more complex scheme in retrieving network information, as opposed to other "regular" apps that simply ask for what kind of network is available.</p>
<p>Downloads app calls multiple functions in the framework, to retrieve various pieces of information about the network connection.<br />Most of that information is currently NOT available for the connection established via RepWifi.<br />The same goes with the "reverse tethering" connection. They happen to be kind-of "ghost connections".</p>
<p>I'm still trying to figure out WHERE and HOW the missing pieces of information are generated, stored, retrieved and reported to calling apps.</p>
<p>So, things will be harder and will take longer than expected.<br />But I'm still confident something can be done to mitigate this.</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70282018-02-24T04:13:54ZKurtis HannaKurtis@riseup.net
<ul></ul><p>Just a note for anyone that really needs to download something using RepWifi, one really horrible work around is to use the extremely old Zirco Browser to download things.</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70392018-04-07T16:11:26ZFil Bergamofil.bergamo@riseup.net
<ul></ul><a name="Ok-I-think-I-have-it-solved-somehow"></a>
<h2 >Ok, I think I have it solved, somehow!<a href="#Ok-I-think-I-have-it-solved-somehow" class="wiki-anchor">¶</a></h2>
<p>Everything is still experimental, and things need to be cleaned up, but the POC is working.</p>
<p>I painfully and boringly scanned a relevant portion of the framework's source code, and I found <strong>the</strong> method that is meant to do exactly what we need:</p>
<blockquote>
<p>om.android.server.ConnectivityService.registerNetworkAgent()</p>
</blockquote>
<p>which is the method that is used by the internal WiFi framework to register all the details for a network connection (interface, routes, dns...)<br />Unfortunately, that method is conceived to <strong>only</strong> be used internally, and is hidden to regular apps.<br />The same goes with the classes used as input parameters.<br />This means we can't just use them directly; we need to use java reflection instead. This is why I put up a set of "proxy classes" that wrap up reflection methods for using hidden stuff.</p>
<p>Also, a special permission is needed, in order to be able to call methods on the internal framework:</p>
<blockquote>
<p>android.permission.CONNECTIVITY_INTERNAL</p>
</blockquote>
<p>so the only way for RepWifi to obtain that permission is to be a system app.</p>
<p>RepWifi already <strong>is</strong> a system app, when it comes pre-packed with the Replicant zip, but this prevents updating it from F-Droid, because of different signatures. As a result, my intention was (and still is) to stop building RepWifi together with Replicant, and shipping F-droid's prebuilt APK in future releases of Replicant. This would allow people to update RepWifi normally from F-droid, which is much quicker and more convenient than having to wait for the next Replicant release.</p>
<p>So the next step will be to either find a way to obtain the needed privilege without being a system app (not very plausible),<br /><strong>or</strong> to programmatically make RepWifi a system app <strong>after</strong> it has been installed from F-Droid (plausible).</p>
<p>Another point which I'm still investigating is how to notify the system when we <strong>disconnect</strong> (currently, I only found out how to notify when we <strong>connect</strong>).<br />But this should be an easier achievement..</p>
<p>So, to recap..<br />Stay tuned for RepWifi updates during the next month, as next release will solve this issue.<br />Also, this new solution is much more elegant than the hackish one that I used with Issue <a class="issue tracker-3 status-15 priority-21 priority-default closed behind-schedule" title="Issue: WiFi connection established via RepWifi not correctly reported by the Android Framework to user-s... (Resolved)" href="https://redmine.replicant.us/issues/1792">#1792</a>, because it doesn't involve any modification to the application framework itself.<br />This means that, if everything goes well, we could revert the previous patch on <a class="issue tracker-3 status-15 priority-21 priority-default closed behind-schedule" title="Issue: WiFi connection established via RepWifi not correctly reported by the Android Framework to user-s... (Resolved)" href="https://redmine.replicant.us/issues/1792">#1792</a>, and remove the hackish part from the framework.</p>
<p>A lot of work still needs to be done, but the solution exists and can be used just fine.</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70402018-04-09T18:44:06ZFil Bergamofil.bergamo@riseup.net
<ul><li><strong>% Done</strong> changed from <i>10</i> to <i>40</i></li></ul> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70412018-04-14T15:46:48ZFil Bergamofil.bergamo@riseup.net
<ul><li><strong>Category</strong> changed from <i>Framework</i> to <i>RepWifi</i></li><li><strong>% Done</strong> changed from <i>40</i> to <i>70</i></li></ul><p>Ok, I made it.<br />I have a working proof of concept for RepWifi to correctly signal both "connection" and "disconnection" to the system.</p>
<p>I tested my work with the Download app, and it definitely works. <br />The final fix should also be beneficial to other apps that currently can't use the WiFi connection made by RepWifi.</p>
<p>The general concept is this:<br />by leveraging java reflection, RepWifi is instantiating its own "NetworkAgent", derived from<br /><pre>frameworks/base/core/java/android/net/NetworkAgent.java</pre></p>
<p>it is registering its NetworkAgent to the system's connectivity framework, by calling<br /><pre>android.net.ConnectivityManager.registerNetworkAgent()</pre></p>
<p>From that moment, RepWifi is able to pass "connectivity change" events to the system's ConnectivityService, just like the inner WiFi compoments would do.<br />All the rest is done by the default, built-in mechanisms: the connection is made available to regular apps through the canonical channels exposed by the frameworks, and, most importantly, it is entirely managed by the framework, including network access permissions and other stuff related to the Android security model.<br />Everybody is happy with that.</p>
<p>The last part remains to provide a painless way to switch RepWifi from being a regular app to a system app, and then back to regular when it needs to be updated through F-droid.<br />But this seems like an easy task to accomplish.</p>
<p>So, the issue <strong>is</strong> solved.<br />I just need some other time to clean up the code, and package the whole thing into RepWifi.<br />I will provide next version of RepWifi both on F-droid and on this very page.</p>
<p>Stay tuned.</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70582018-05-09T20:50:51ZKurtis HannaKurtis@riseup.net
<ul></ul><p>Great job, Fil! I greatly look forward to running the next RepWifi release!</p>
<p>How is the code that switches RepWifi from being a regular app to a system app coming along?</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70592018-05-10T20:58:27ZFil Bergamofil.bergamo@riseup.net
<ul><li><strong>% Done</strong> changed from <i>70</i> to <i>80</i></li></ul><p>Kurtis Hanna wrote:</p>
<blockquote>
<p>How is the code that switches RepWifi from being a regular app to a system app coming along?</p>
</blockquote>
<p>This one turned out to be pretty simple to implement:<br />every time RepWifi opens, it checks to be a system app. If it's not, it prompts the user for a confirmation.<br />If the user agrees, an automated script starts, makes the app a system one, and reboots the phone.<br />From that moment, it's also possible to update from F-droid without even removing RepWifi from being a system app (provided that it first was installed from F-droid).</p>
<p>As for the rest, I'm doing a <strong>massive</strong> restyle to the codebase, so next one will be a completely new version of RepWifi. As such, things are going a bit slowly, but I'm pretty confident I'll be able to release before June.<br />Please folks, keep patience for just a little more.. Next release is worth waiting, I promise!</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70602018-05-20T12:30:46ZFil Bergamofil.bergamo@riseup.net
<ul><li><strong>Due date</strong> changed from <i>05/20/2018</i> to <i>06/01/2018</i></li></ul><p>Got my HDD fried last week.. :(<br />This means next release will take even longer than expected.<br />Sorry about that.</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=70972018-06-29T20:17:45ZKurtis HannaKurtis@riseup.net
<ul></ul><p>That stinks that your HDD got fried!</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=71002018-07-05T20:34:20ZFil Bergamofil.bergamo@riseup.net
<ul><li><strong>% Done</strong> changed from <i>80</i> to <i>100</i></li><li><strong>Resolution</strong> set to <i>fixed</i></li></ul><p>With considerable delay...<br />I'm happy to announce that this issue is solved inside RepWifi!</p>
<p>For the moment, there won't be no official release (no f-droid release), as this new version has a whole lot of new features that need to be tested for a while before getting to a stable release.</p>
<p>Nonetheless, I'm distributing an "unofficial" release, signed with my own GPG key, for users' convenience and to allow the community to test the new features.</p>
<p>The compiled APK is released in <a href="https://redmine.replicant.us/boards/9/topics/14079" class="external">the original RepWifi forum thread</a><br />This new version is v0.9-beta<br />Please, make sure you verify the attached gpg signature.</p>
<p>The key fingerprint for the signature is:<br /><pre>2D36 719C A4B8 3C7A DD9D 33AE A02A 2F68 CD07 CEAD</pre></p>
<p>As usual, source code is published <a href="https://git.replicant.us/contrib/Fil/RepWifiApp/tree/?h=testing" class="external">here</a></p>
<p>Please, test this beta release, especially in combination with the Download app, and please report any bugs!</p>
<p>Happy Hacking!</p>
<p>Fil</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=71022018-07-15T19:14:58ZKurtis HannaKurtis@riseup.net
<ul></ul><p>Great work Fil! I've tested a number of the new features for a number of days now and haven't found any bugs in them as of yet. I personally am in favor of moving it from beta to stable.</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=71032018-07-19T20:59:26ZFil Bergamofil.bergamo@riseup.net
<ul></ul><p>I've just pushed a new git tag on the source repo, so that F-Droid should recognize it as a new release.<br />The new release is still marked as "beta" in the tag name, so that people are aware that it needs testing.</p>
<p>Nonetheless, F-droid should be publishing the new version of RepWifi in some days (maybe something more that just some days.. it depends on them).</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=71712018-08-30T17:30:16ZKurtis HannaKurtis@riseup.net
<ul></ul><p>Should we close this issue? It seems to be resolved.</p> Replicant - Issue #1867: Download app not working when on USB network or RepWifi connectionhttps://redmine.replicant.us/issues/1867?journal_id=71782018-08-30T19:26:26ZFil Bergamofil.bergamo@riseup.net
<ul><li><strong>Status</strong> changed from <i>New</i> to <i>Resolved</i></li></ul>