Issue #1827
closedPrivacy issue with phone number lookup priovider. Was: WhitePagesPeopleLookup privacy leaks?
0%
Description
Hi, I noticed the following in my logs (i9300):
08-18 09:05:20.935 8892 8923 V LookupProvider: Future called for FilterThread 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: People lookup failed 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: java.io.FileNotFoundException: http://whitepages.com/search/FindPerson?who=XX%20XX%20XX%20XX%20XX 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:238) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.LookupUtils.httpFetch(LookupUtils.java:61) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.LookupUtils.httpGet(LookupUtils.java:99) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.whitepages.WhitePagesApi.httpGet(WhitePagesApi.java:252) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.whitepages.WhitePagesApi.peopleLookup(WhitePagesApi.java:67) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.whitepages.WhitePagesPeopleLookup.lookup(WhitePagesPeopleLookup.java:45) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.LookupProvider.handleFilter(LookupProvider.java:313) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.LookupProvider.-wrap0(LookupProvider.java) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.LookupProvider$1.call(LookupProvider.java:171) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.LookupProvider$1.call(LookupProvider.java:170) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at com.android.dialer.lookup.LookupProvider$FutureCallable.call(LookupProvider.java:103) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at java.util.concurrent.FutureTask.run(FutureTask.java:237) 08-18 09:05:21.315 8892 8923 E WhitePagesPeopleLookup: at java.lang.Thread.run(Thread.java:818) 08-18 09:05:23.700 2356 2416 D lights : set_light_buttons: 1 08-18 09:05:23.890 2356 2440 D AudioService: Stream muted, skip playback 08-18 09:05:23.925 8892 8892 I SearchFragment: onItemClick: shortcutType=1 08-18 09:05:24.060 8892 8892 W Settings: Setting airplane_mode_on has moved from android.provider.Settings.System to android.provider.Settings.Global, returning read-only value.
XX%20XX%20XX%20XX%20XX is a phone number I probably typed by hand in the dialer. I replaced the actual number with some X in the log for privacy reasons.
Denis.
Files
Updated by Fil Bergamo over 7 years ago
- Assignee set to Fil Bergamo
- Estimated time set to 20.00 h
Denis 'GNUtoo' Carikli wrote:
Hi, I noticed the following in my logs (i9300):
[...]
XX%20XX%20XX%20XX%20XX is a phone number I probably typed by hand in the dialer. I replaced the actual number with some X in the log for privacy reasons.Denis.
Mmmm.. That seems suspicious indeed!
The logs show that the exception is raised by java code, so I think I will be able to have a look at the issue.
I'm beginning to get familiar with the android framework code, so I'll look into the com.android.dialer.lookup.* namespace to see if there is something scaring in this regard.
Thank you very much for pointing out!
Updates coming soon.
Fil
Updated by Wolfgang Wiedmeyer over 7 years ago
- Device added
- Device deleted (
Galaxy S 3 (I9300))
Thanks for reporting this, Denis!
The lookup providers can be configured in the dialer settings ("Phone number lookup"). The "People lookup" is enabled by default which very likely causes this privacy leak. So this setting needs to be disabled by default which is an easy fix.
There are some other settings in that menu that would need more sensible defaults. The Forward Lookup maybe needs to be disabled as well by default. At least the provider should be set to OpenStreetMap instead of Google by default. And the Reverse Lookup should be disabled as well by default for privacy concerns. The dialer app isn't yet forked, so all of this requires a fork.
Updated by Denis 'GNUtoo' Carikli over 6 years ago
I've not yet succeeded at getting a network trace of the issue but I've probably found where it comes from:
There is a stock application called 'phone' (probably also called dialer). In it there is a field with "Enter a name or phone number" and 3 dots on the right.
The 3 dots opens a menu with:- Call History
- Import/export
[...] - Settings
- Display options
[...] - Phone number lookup "Loopup of unknown phone numbers"
[...]
We find:
- "WhitePages (US)" as People lookup provider
- Google as Forward lookup provider
- "OpenCnam (US)" as reverse lookup provider
And All are activated!
It would be wise to:- Find a way to trigger them to be able to test if they are disabled later
- Disable them by default and force the disabled defaults when settings are wiped for a reason or another
- Verify that they are really disabled by network traces and/or looking at the source code
Updated by Denis 'GNUtoo' Carikli over 6 years ago
- Subject changed from WhitePagesPeopleLookup privacy leaks? to Privacy issue with phone number lookup priovider. Was: WhitePagesPeopleLookup privacy leaks?
- Priority changed from Normal to High
Updated by Denis 'GNUtoo' Carikli over 6 years ago
Simply typing 0102030405 in the dialer form result in DNS querries to whitepages.com
I obtained the log by disabling orbot (force stop) on the smartphone and connecting it to a WiFi access point I made with hostapd on my laptop with a simple script to share the connection (https://framagit.org/GNUtoo/simple-nat), and the following extra dnsmasq configuration (added to dnsmasq.conf.local) to trace DNS querries:
log-queries=extra
This gave the following log (other queries removed):
dnsmasq: 13 192.168.10.246/27181 query[A] whitepages.com from 192.168.10.246 dnsmasq: 13 192.168.10.246/27181 forwarded whitepages.com to 192.168.10.3 dnsmasq: 13 192.168.10.246/27181 forwarded whitepages.com to 192.168.10.1 dnsmasq: 16 192.168.10.246/27181 query[A] whitepages.com from 192.168.10.246 dnsmasq: 16 192.168.10.246/27181 forwarded whitepages.com to 192.168.10.3 dnsmasq: 16 192.168.10.246/27181 forwarded whitepages.com to 192.168.10.1 dnsmasq: 18 192.168.10.246/28925 query[A] whitepages.com from 192.168.10.246 dnsmasq: 18 192.168.10.246/28925 forwarded whitepages.com to 192.168.10.3 dnsmasq: 18 192.168.10.246/28925 forwarded whitepages.com to 192.168.10.1 dnsmasq: 19 192.168.10.246/8626 query[A] whitepages.com from 192.168.10.246 dnsmasq: 19 192.168.10.246/8626 forwarded whitepages.com to 192.168.10.3 dnsmasq: 19 192.168.10.246/8626 forwarded whitepages.com to 192.168.10.1 dnsmasq: 20 192.168.10.246/30252 query[A] whitepages.com from 192.168.10.246 dnsmasq: 20 192.168.10.246/30252 forwarded whitepages.com to 192.168.10.3 dnsmasq: 20 192.168.10.246/30252 forwarded whitepages.com to 192.168.10.1 dnsmasq: 21 192.168.10.246/30431 query[A] whitepages.com from 192.168.10.246 dnsmasq: 21 192.168.10.246/30431 forwarded whitepages.com to 192.168.10.3 dnsmasq: 21 192.168.10.246/30431 forwarded whitepages.com to 192.168.10.1 dnsmasq: 22 192.168.10.246/32041 query[A] whitepages.com from 192.168.10.246 dnsmasq: 22 192.168.10.246/32041 forwarded whitepages.com to 192.168.10.3 dnsmasq: 22 192.168.10.246/32041 forwarded whitepages.com to 192.168.10.1 dnsmasq: 24 192.168.10.246/30431 query[A] whitepages.com from 192.168.10.246 dnsmasq: 24 192.168.10.246/30431 forwarded whitepages.com to 192.168.10.3 dnsmasq: 24 192.168.10.246/30431 forwarded whitepages.com to 192.168.10.1 dnsmasq: 25 192.168.10.246/28925 query[A] whitepages.com from 192.168.10.246 dnsmasq: 25 192.168.10.246/28925 forwarded whitepages.com to 192.168.10.3 dnsmasq: 25 192.168.10.246/28925 forwarded whitepages.com to 192.168.10.1 dnsmasq: 26 192.168.10.246/8626 query[A] whitepages.com from 192.168.10.246 dnsmasq: 26 192.168.10.246/8626 forwarded whitepages.com to 192.168.10.3 dnsmasq: 26 192.168.10.246/8626 forwarded whitepages.com to 192.168.10.1 dnsmasq: 27 192.168.10.246/30252 query[A] whitepages.com from 192.168.10.246 dnsmasq: 27 192.168.10.246/30252 forwarded whitepages.com to 192.168.10.3 dnsmasq: 27 192.168.10.246/30252 forwarded whitepages.com to 192.168.10.1 dnsmasq: 28 192.168.10.246/32041 query[A] whitepages.com from 192.168.10.246 dnsmasq: 28 192.168.10.246/32041 forwarded whitepages.com to 192.168.10.3 dnsmasq: 28 192.168.10.246/32041 forwarded whitepages.com to 192.168.10.1
Updated by Kurtis Hanna over 6 years ago
- Target version changed from Replicant 6.0 to Replicant 6.0 0005
Updated by Kurtis Hanna over 6 years ago
Just curious, could this have anything to do with the fact that the USSD caller ID thing referenced here hasn't been implimented? https://redmine.replicant.us/issues/1011#note-10
Updated by Denis 'GNUtoo' Carikli over 6 years ago
I've no idea:
- I didn't look deep enough at the related dialer code
- I didn't have time to get (and analyze) more conclusive traces about the communication with such third party services.
Updated by Fil Bergamo almost 6 years ago
Having a look at
packages/apps/Dialer/res/xml/lookup_settings.xml
shows that indeed the default value of all three settings is set to "false" (attribute android:defaultValue="false")
e.g.:
<SwitchPreference android:key="enable_reverse_lookup" android:title="@string/enable_reverse_lookup_title" android:summary="@string/enable_reverse_lookup_summary" android:defaultValue="false" android:persistent="false" />
But I can confirm that, in a fresh clean Replicant 6.0 install, all three settings are switched on by default in the mentioned menu (Phone-> [searchbar] -> Settings -> Phone number lookup)
It will be interesting to find where the default "off" value is overridden in the code or in the build/installation process..
I'm having a look into it right now..
Updated by Fil Bergamo almost 6 years ago
The underlying database file related to phone number lookup settings is
/data/data/org.cyanogenmod.cmsettings/databases/cmsettings.db
Effectively, lookup settings are set to "true" in the database:
This is the result of running a query on a fresly-installed Replicant 6.0 image, after clearing data and cache.
1|root@i9300:/data/data/org.cyanogenmod.cmsettings/databases # sqlite3 cmsettings.db SQLite version 3.8.10.2 2017-07-21 04:08:41 Enter ".help" for usage hints. sqlite> .tables android_metadata global secure system sqlite> select * from system ...> ; 1|qs_quick_pulldown|1 2|notification_light_brightness_level|255 3|notification_light_multiple_leds_enable|0 5|enable_forward_lookup|1 6|enable_people_lookup|1 7|enable_reverse_lookup|1 8|notification_light_pulse_custom_enable|0 9|swap_volume_keys_on_rotation|0 10|status_bar_battery_style|2 11|volume_adjust_sounds_enabled|1 12|system_profiles_enabled|1 13|display_color_adjustment|1.0 1.0 1.0 14|live_display_hinted|-2 15|edge_service_for_gestures|0
Relevant values are "enable_forward_lookup", "enable_people_lookup", "enable_reverse_lookup".
I can confirm that setting the values to "0" in the SQLite database results in the settings being switched off in the Phone GUI.
It remains to be tested whether disabling this settings actually prevents lookup queries (this should be trivial using the testing setup proposed by Denis).
Also, we need to know exactly WHEN and WHERE the default database file is generated during the build/installation/initial-configuration process.
Changing the default values should be trivial after that.
Does anybody have any clue about that???
Updated by Fil Bergamo almost 6 years ago
Default values seem to be defined in
vendor/cmsdk/packages/CMSettingsProvider/res/values/defaults.xml
and they are indeed set to "True" by default:
<!-- Default value of CMSettings.System.ENABLE_FORWARD_LOOKUP --> <integer name="def_forward_lookup">1</integer> <!-- Default value of CMSettings.System.ENABLE_PEOPLE_LOOKUP --> <integer name="def_people_lookup">1</integer> <!-- Default value of CMSettings.System.ENABLE_REVERSE_LOOKUP --> <integer name="def_reverse_lookup">1</integer>
This means that if we set them to "0" by default, the lookup "anti-features" should be disabled by default on first boot and until the user decides to switch them on. Furthermore, a factory reset is expected to restore these default values, and switch off lookup again.
I think I can come up with a patch in a few days.
Updated by Fil Bergamo almost 6 years ago
- File 0001-Disable-phone-number-lookup-by-default-for-privacy.patch 0001-Disable-phone-number-lookup-by-default-for-privacy.patch added
Denis 'GNUtoo' Carikli wrote:
It would be wise to:
- Find a way to trigger them to be able to test if they are disabled later
Your suggestion seemed like a good way of probing.
I tested it myself and I can confirm that those DNS queries are not present when the settings are disabled and they come back up again when lookup settings are enabled again.
- Disable them by default and force the disabled defaults when settings are wiped for a reason or another
Default settings are part of the CyanogenMod SDK, not part of AOSP.
I created a patch to disable all lookup anti-features by default. The patch is attached here and works like this:
- A fresh new replicant installation with my patch has lookup services disabled by default (and I detected no dns query to lookup services).
- Enabling the settings manually in the dialer app makes the DNS queries come back.
- Doing a "factory reset" when lookup is enabled results in lookup services being disabled again as the default configuration expects.
- Verify that they are really disabled by network traces and/or looking at the source code
I didn't go into any detailed analysis of the source code to verify that disabling the lookup features actually stops unwanted network activity, but I think it's worth doing so.
For the moment, I think it's wise to disable lookup services by default, so here's my patch: https://redmine.replicant.us/attachments/download/1586/0001-Disable-phone-number-lookup-by-default-for-privacy.patch
Updated by Fil Bergamo almost 6 years ago
- Status changed from New to Feedback
- % Done changed from 0 to 90
Updated by Fil Bergamo almost 6 years ago
I've conducted some deeper testing, both by debugging and by collecting network traces with wireshark and I can confirm that connections to third parties are only present when the lookup features are enabled in the settings.
This is how I tested it:
(Warning: long read ahead. Who's not interested in the full technical analysis can jump to Conclusions)
Debugging the source code:¶
It is quite difficult to manually perform static analysis to track down the precise path of code execution in the Dialer app, as in many other parts of the AOSP tree, because of all the asynchronous "callbacks", "providers", "handles" involved in the android-ish programming model and because of the lack of specific documentation for the mechanisms that are implemented in the miriad of classes, subclasses, fragments...
TL;DR: Android is a mess.
By the way, debugging the running code is not easy either, but it's a lot quicker to track the execution path this way..
All relevant code is at *packages/apps/Dialer/src/com/android/dialer/lookup*
Relevant classes are the generic *ForwardLookup*, *PeopleLookup*, *ReverseLookup* and all the children classes, specific to each provider (Google, OSM, WhitePages, Auskunft, etc.)
The settings about lookup services are managed by *LookupSettings*, and specifically these getters are accessed to retrieve the status of what service is enabled/disabled:
LookupSettings::isForwardLookupEnabled() LookupSettings::isPeopleLookupEnabled() LookupSettings::isReverseLookupEnabled()
When the Dialer's searchbar gets focus, a *PhoneNumberListAdapter* is instantiated which in turn initializes contacts lookup "directories", both "local" and "remote" (local directories are the ones in the contacts database, while remote ones are provided by the remote lookup services we're investigating).
PhoneNumberListAdapter.java:161 calls:
mExtendedDirectories = manager.getExtendedDirectories(mContext);
which is implemented at ExtendedLookupDirectories.java:40 and reads as follows:
ArrayList<DirectoryPartition> list = new ArrayList<DirectoryPartition>(); // The directories are shown in reverse order, so insert forward lookup // last to make it show up at the top if (LookupSettings.isPeopleLookupEnabled(context)) { DirectoryPartition dp = new DirectoryPartition(false, true); dp.setContentUri(LookupProvider.PEOPLE_LOOKUP_URI.toString()); dp.setLabel(context.getString(R.string.people)); dp.setPriorityDirectory(false); dp.setPhotoSupported(true); dp.setDisplayNumber(false); dp.setResultLimit(3); list.add(dp); } else { Log.i(TAG, "Forward lookup (people) is disabled"); } if (LookupSettings.isForwardLookupEnabled(context)) { DirectoryPartition dp = new DirectoryPartition(false, true); dp.setContentUri(LookupProvider.NEARBY_LOOKUP_URI.toString()); dp.setLabel(context.getString(R.string.nearby_places)); dp.setPriorityDirectory(false); dp.setPhotoSupported(true); dp.setDisplayNumber(false); dp.setResultLimit(3); list.add(dp); } else { Log.i(TAG, "Forward lookup (nearby places) is disabled"); } return list;
by which, it is evident that when remote lookup services are disabled in the settings no "remote lookup directory" is added to the list, so that subsequent searches are only performed on the local contacts database.
When lookup services are enabled, the actual remote queries are performed via HTTP requests to specifically created URLs, that are crafted based on the text the user types in the search bar. These requests originate from each specific implementation of the following abstract methods:
ForwardLookup.lookup(Context, String, Location) ReverseLookup.lookupNumber(Context, String, String) PeopleLookup.lookup(Context, String);
To confirm this, I put breakpoints on all specific implementations of the lookup methods and tried different settings.
1) Reverse Lookup only works when receiving incoming calls, so I didn't have a quick way to actively test it.
But the following lines at ReverseLookupService.java:60 indicate pretty clearly that no query is performed when the service is disabled:
if (!LookupSettings.isReverseLookupEnabled(mContext)) { LookupCache.deleteCachedContacts(mContext); return; }
2) When only Forward Lookup is enabled queries are only performed if Location services are enabled, and if the location of the device is known.
3) When People Lookup is enabled queries are always performed, regardless of any other setting; when any string is typed in the Dialer's search bar, the breakpoint at the specific lookup method implementation is always hit.
For example, when using Auskunft as a People lookup provider, this lines in *AuskunftPeopleLookup* are always executed:
public ContactInfo[] lookup(Context context, String filter) { List<ContactInfo> infos = null; try { infos = AuskunftApi.query(filter, ContactBuilder.PEOPLE_LOOKUP, null, null); } catch (IOException e) { Log.e(TAG, "People lookup failed", e); } return (infos != null && !infos.isEmpty()) ? infos.toArray(new ContactInfo[infos.size()]) : null; }
which results into calling *AuskunftApi*, that finally performs the actual HTTP request:
// get all search entry sections List<String> entries = LookupUtils.allRegexResults(LookupUtils.httpGet(uri.toString(), null), SEARCH_RESULTS_REGEX, true);
4) When both People Lookup and Forward Lookup are disabled, the execution of the Dialer app does never reach any of the aforementioned instructions, which is pretty much a guarantee that no remote connection is made when all lookup services are disabled in the settings.
To further confirm, I went even further and captured some network traces:
Analysis of Network activity:¶
I put up the testing environment described by GNUtoo, connecting my i9300 to my laptop's WiFi hotspot.
I then captured the network traffic of my laptop's wlan interface using Wireshark, while using the Dialer's app search bar on the i9300, with different combinations of Lookup Settings.
This is what I registered:
1) When lookup services are all disabled in the settings, the only network activity present consist of NTP queries, and no TCP connection is made to any host whatsoever.
2) When People Lookup is enabled and uses WhitePages as provider, some TCP traffic is generated towards this IP address: 54.201.217.74, which resolves to "whitepages.com"
3) When People Lookup is enabled and uses Auskunft as provider, some TCP traffic is generated towards this IP address: 91.195.199.133, which resolves to "auskunft.at"
I did not test Forward and Reverse lookup, for the reasons mentioned above, but it seems reasonable to suppose they would show analogous behaviour.
Conclusions:¶
All the tests I performed clearly indicate that no network activity is generated to any third party host when all lookup anti-features are disabled in the settings.
It is otherwise proved that enabling phone number lookup results in silent background connections to online services, exposing users' searches in their own contact list to third parties.
All this being said, my personal opinion is that simply disabling those services via the settings is to be considered safe enough for the users' privacy, and that there's no strong motivation to completely remove them from Replicant.
Updated by Fil Bergamo almost 6 years ago
- Status changed from Feedback to Closed
- Resolution set to fixed
Patch pushed; closing the issue.
Updated by Fil Bergamo almost 6 years ago
- Status changed from Closed to Resolved
Updated by Kurtis Hanna almost 6 years ago
- Target version changed from Replicant 6.0 0005 to Replicant 6.0 0004