Project

General

Profile

Issue #2061

Do git snapshots to protect mirror branches against upstream changing revision or rewriting history

Added by Denis 'GNUtoo' Carikli 3 months ago. Updated 3 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
06/24/2020
Due date:
% Done:

0%

Estimated time:
Hosting infrastructure:
VM @ FSF

Files

git-snapshot (2.19 KB) git-snapshot Script to keep Git mirrors up-to-date incl. snapshots of references doak complex, 06/26/2020 06:16 PM
git-snapshot (2.36 KB) git-snapshot doak complex, 06/26/2020 06:45 PM
#1

Updated by Denis 'GNUtoo' Carikli 3 months ago

  • Hosting infrastructure VM @ FSF added

Background information

We use mirrors for various projects (LineageOS, AOSP, etc) which are tagged at each release. So corruption aside, we know we have the right history because it's behind a tag, regardless of what upstream does and on if we fetch new upstream revisions or not.

However this is not the case for branches like replicant-6.0.

Here's the advise I got on IRC:
< doak[m]> You could for example do something like this during the regular update:
< doak[m]> `git fetch "refs/heads/*:refs/snapshot/<date>/heads/*"`

#2

Updated by Denis 'GNUtoo' Carikli 3 months ago

And:

DATE="$(date -Imin)" 
git fetch "refs/heads/*:refs/snapshots/$DATE/heads/*" 
git fetch "refs/tags/*:refs/snapshots/$DATE/tags/*" 

#3

Updated by doak complex 3 months ago

With the assumed used refspec +refs/*:refs/* to fetch everything from the remote into own mirror, every remote reference, which had been rewritten, will also be rewritten in the mirror. This includes remote branches and tags.
Though this is usually intended, creating snapshots of them just in case something will be changed maliciously, seems to be a good idea.

#4

Updated by doak complex 3 months ago

I made an conceptional mistake: `git fetch <remote> --prune "+refs/*:refs/*"` would also prune any snapshots below `refs/snapshots` :/

I played around with different approaches, but all the simple ones had some flaw.
Attached is a small script which does the job pretty well:
  • It fetches all references available on the remote and prune local ones if necessary.
  • It keeps a snapshot of all current references (which are never pruned) below refs/snapshots/<date>.
  • It removes the default fetch configuration to ensure, that the snapshots can't be pruned with a plain git fetch --prune by accident. This also reminds the user that she should use this script to update the mirror.
#5

Updated by doak complex 3 months ago

Did not fetch like expected. Here is the fixed version.

Also available in: Atom PDF