Replicant

Is there any reason why we shouldn't add installation instructions where TWRP is used? A user reported that it can be used to install Replicant without a PC. https://redmine.replicant.us/boards/3/topics/14715

It is my understanding that TWRP is free software, but they have an android app in Google Play that isn't free software.

This can be used as a starting point to complete this issue: https://ftp.osuosl.org/pub/replicant/images/replicant-6.0/0004-rc1/README_VERY_IMPORTANT_INFORMATION.txt

Between Replicant 6.0 0003 to Replicant 6.0 0004 RC1 we may have had some changes affecting data/ like the fix of the bug #1827, however between the 0004 RC1 and 0004 RC2 we have no such changes.

However, since Replicant 0004 RC1, instead of having replicant-6.0 in Settings->About phone->Replicant version, we now have something like replicant-6.0-0004-rc1 or replicant-6.0-0004-rc2. So I wonder if it's related or not.

In any case that change is desirable as it helps users understand if they need to upgrade or not, so the best way is to fix it somehow.

• Debug this issue
• Making a TWRP version that is fully free software (so with a deblobbed kernel). Thanks a lot for the pointer on that.
• Making sure that the future Replicant 10 recovery doesn't do that check and enable to safely install older versions of Replicant as well (for instance by checking gpg signatures and displaying versions in a generic way).

$ repo grep "on top of incompatible data" returns only one entry.

In build/tools/releasetools/edify_generator.py we have:

  def ValidateSignatures(self, command):
    self.script.append('package_extract_file("META-INF/org/replicant/releasekey", "/tmp/releasekey");')
    # Exit code 124 == abort. run_program returns raw, so left-shift 8bit
    self.script.append('run_program("/tmp/install/bin/otasigcheck.sh") != "31744" || abort("Can\'t install this package on top of incompatible data. Please try another package or run a factory reset");')

In turn otasigcheck.sh seem to be in vendor/replicant/prebuilt/common/bin/otasigcheck.sh .

$ repo grep "incompatible data" returns more than one entry but only the entry above is code, the rest are only comments.

edit1: added $ repo grep "incompatible data"

Also, using

-CM_DISPLAY_VERSION := $(REPLICANT_VERSION)
+CM_DISPLAY_VERSION := replicant-6.0-0004-rc3

-RELEASE=replicant-6.0
+RELEASE=replicant-6.0-0004-rc2

I don't know why the return code is to be shifted left by 8, but 124 << 8 31744, and we have the following comment as well in the code described above: "Exit code 124 abort. run_program returns raw, so left-shift 8bit", so this is the issue in otasigcheck.sh:

  if [ "$pkgkey" != "$relkey" ]; then
     echo "You have an installed system that isn't signed with this build's key, aborting..." 
     exit 124
  fi

Now we need to understand more that check not to create any side issues if we remove it.

Would following these directions help us fix this issue? https://gist.github.com/ttimasdf/0cdd4b958dd00ff2bcdb125415aa1269

Thanks, I've used that information to look for something similar in LineageOS but I then figured out that it was better to look for that in TWRP since you pointed to me that a user got it working with TWRP.

In fact they have scripts to handle that case, and this one might be something like the script we are looking for:
https://raw.githubusercontent.com/omnirom/android_bootable_recovery/android-9.0/prebuilt/fix_permissions.sh

There are also other scripts in prebuilt:
https://github.com/omnirom/android_bootable_recovery/blob/android-9.0/prebuilt
/fix_permissions.sh

Note that TWRP is GPLv3 as per their COPYING so we won't have a license issue here even if that file lacks licensing information. We'd just create the header with the proper info instead.

edit1: modified link from prebuilt to the actual file and re-added to prebuilt
edit2: fixed edit1 message, added COPYING

This LineageOS patchset seems to have never been added to any of their branches: https://github.com/LineageOS/android_vendor_cm/commit/2f7c7decc4cd5b42f044a7841a74468e4cacd694#diff-a5ab82d48d569fdffe41cbf3b31a5767

Is it compatible with Replicant 6?

Kurtis Hanna wrote in #note-15:

This LineageOS patchset seems to have never been added to any of their branches: https://github.com/LineageOS/android_vendor_cm/commit/2f7c7decc4cd5b42f044a7841a74468e4cacd694#diff-a5ab82d48d569fdffe41cbf3b31a5767

Is it compatible with Replicant 6?

Thanks for the findings.

It's not compatible as-is but replacing the old and new keys and certificates inside result in something that works.

To do that it's possible to use the script that I made to automatize it a bit.

• The python script I pushed I wrote needs to be checked out, and it's also a good idea to get all all the files inside that same directory (templates/, etc) to avoid any dependency issues.
• A device that is not used and can completely wiped is required. It needs to be compatible with the old and new images.
• Then a full factory reset needs to be done in the Replicant recovery
• After that the old images needs to be installed on the device and the device has to complete its boot
• Then /data/system/applications.xml needs to be extracted from it and renamed to something meaningful like old_applications.xml
• Then a full factory reset needs to be done in the Replicant recovery again
• Then the new images needs to be installed on the device and the device has to complete its boot
• Then the new /data/system/applications.xml needs to be extracted from it and renamed to something meaningful like new_applications.xml

Then the script can be used like that:

$ ./generate_key_migration_script.py path-to-replicant-root-tree/vendor/replicant/prebuilt/common/bin/keys-migration.sh old_applications.xml new_applications.xml

Limitations¶

It's probably not a big issue to require to extract the applications.xml from the phone, however for the new one you need to build the target image first to then be able to run the python script to then build the final image.

So the next step would be to find a way to use the newly generated certificate and encode them in the format expected by applications.xml and re-use it in the generation of keys-migration.sh

Further developments¶

We could also add the old certificates in a repo and/or download them from the FTP to not have to boot a device to build an image.

Key policies¶

We also need to define policies as for what to do with the generated keys.

• Paul Kocialkowski did all the Replicant 4.2 releases
• Wolfgang Wiedmeyer did all the Replicant 6.0 releases until Replicant 6.0 0003

As users wiped their data between major Replicant version and that a change of maintainer didn't occur within a minor version they didn't have the issue.

Worlfgang most probably kept the generated keys between the Replicant 6.0 releases.

• Having to keep these keys and/or passing them around is not ideal.
• Having to make new migration scripts each time the developer making the release changes is not ideal either unless it's almost completely automated.

update1: fix grammar
update2: fix typo

$ openssl x509 -in vendor/replicant-security/platform.x509.pem -inform PEM -pubkey  | grep -B 100 "\-----END PUBLIC KEY-----" | grep -v "^--" | xargs echo -n  | sed 's# ##g' ;echo
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1OieLpOFLahyvUGDoJhB0GnLNfJ7qf+VEg59vW50yhFZAj0HseFBzXSy+PVtkIAAuqIyXfPwgg0lAfnPh5oW2gvBqW6dTZC1DPH84Fg3FCqu2T+fEXpdRafpWjLG2oWfgupWWg9Yg/QH9Ft9LbTMLGiXa/OpmVfx3aJrW/EzPdUMR87xuZmJeSoTC1VCgv/lkqnft8xxJrt+43sXz5OKM56Ia3ardBIdBxrlwvDCP46Fp351rx1+RTpSBg7zDBiRaJLbRQJtcA6bEpZriylslndnACh45jXKDrYV55k0m5Dh+Zj+BRZK+/eyPk2JCZp/pgdWt6HaQvz9/wlWXhF9lQIDAQAB
$ grep 'MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1OieLpOFLahyvUGDoJhB0GnLNfJ7qf+VEg59vW50yhFZAj0HseFBzXSy+PVtkIAAuqIyXfPwgg0lAfnPh5oW2gvBqW6dTZC1DPH84Fg3FCqu2T+fEXpdRafpWjLG2oWfgupWWg9Yg/QH9Ft9LbTMLGiXa/OpmVfx3aJrW/EzPdUMR87xuZmJeSoTC1VCgv/lkqnft8xxJrt+43sXz5OKM56Ia3ardBIdBxrlwvDCP46Fp351rx1+RTpSBg7zDBiRaJLbRQJtcA6bEpZriylslndnACh45jXKDrYV55k0m5Dh+Zj+BRZK+/eyPk2JCZp/pgdWt6HaQvz9/wlWXhF9lQIDAQAB' packages.xml 
            <public-key identifier="1" value="MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA1OieLpOFLahyvUGDoJhB0GnLNfJ7qf+VEg59vW50yhFZAj0HseFBzXSy+PVtkIAAuqIyXfPwgg0lAfnPh5oW2gvBqW6dTZC1DPH84Fg3FCqu2T+fEXpdRafpWjLG2oWfgupWWg9Yg/QH9Ft9LbTMLGiXa/OpmVfx3aJrW/EzPdUMR87xuZmJeSoTC1VCgv/lkqnft8xxJrt+43sXz5OKM56Ia3ardBIdBxrlwvDCP46Fp351rx1+RTpSBg7zDBiRaJLbRQJtcA6bEpZriylslndnACh45jXKDrYV55k0m5Dh+Zj+BRZK+/eyPk2JCZp/pgdWt6HaQvz9/wlWXhF9lQIDAQAB" />

• Update the python script to use vendor/replicant-security certs
• Add support for multiple old certificates/key sets
• We can take the keys/certs from the releases on the wiki image page and add them to git in the data repository for instance
• We add the data repository to the manifest
• We then update the python script to sed all known keys/certs
  => We have something that always works for upgrades (assuming we add new certs in the data repo at each new release)

• We also add support for creating a script to downgrade the images
• We add instructions to use that script in the recovery. Most of the instructions can be reused from backuping the EFS.

Update1: Added the next steps
Update2: Status update + change Steps I
Update3: Status update