Issue #2061


Do git snapshots to protect mirror branches against upstream changing revision or rewriting history

Added by Denis 'GNUtoo' Carikli almost 4 years ago. Updated almost 4 years ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
Hosting infrastructure:
Type of work:


git-snapshot (2.19 KB) git-snapshot Script to keep Git mirrors up-to-date incl. snapshots of references doak complex, 06/26/2020 06:16 PM
git-snapshot (2.36 KB) git-snapshot doak complex, 06/26/2020 06:45 PM
Actions #1

Updated by Denis 'GNUtoo' Carikli almost 4 years ago

  • Hosting infrastructure VM @ FSF added

Background information

We use mirrors for various projects (LineageOS, AOSP, etc) which are tagged at each release. So corruption aside, we know we have the right history because it's behind a tag, regardless of what upstream does and on if we fetch new upstream revisions or not.

However this is not the case for branches like replicant-6.0.

Here's the advise I got on IRC:
< doak[m]> You could for example do something like this during the regular update:
< doak[m]> `git fetch "refs/heads/*:refs/snapshot/<date>/heads/*"`

Actions #2

Updated by Denis 'GNUtoo' Carikli almost 4 years ago


DATE="$(date -Imin)" 
git fetch "refs/heads/*:refs/snapshots/$DATE/heads/*" 
git fetch "refs/tags/*:refs/snapshots/$DATE/tags/*" 

Actions #3

Updated by doak complex almost 4 years ago

With the assumed used refspec +refs/*:refs/* to fetch everything from the remote into own mirror, every remote reference, which had been rewritten, will also be rewritten in the mirror. This includes remote branches and tags.
Though this is usually intended, creating snapshots of them just in case something will be changed maliciously, seems to be a good idea.

Actions #4

Updated by doak complex almost 4 years ago

I made an conceptional mistake: `git fetch <remote> --prune "+refs/*:refs/*"` would also prune any snapshots below `refs/snapshots` :/

I played around with different approaches, but all the simple ones had some flaw.
Attached is a small script which does the job pretty well:
  • It fetches all references available on the remote and prune local ones if necessary.
  • It keeps a snapshot of all current references (which are never pruned) below refs/snapshots/<date>.
  • It removes the default fetch configuration to ensure, that the snapshots can't be pruned with a plain git fetch --prune by accident. This also reminds the user that she should use this script to update the mirror.
Actions #5

Updated by doak complex almost 4 years ago

Did not fetch like expected. Here is the fixed version.


Also available in: Atom PDF