Reduce disk writes in post-compile triggers

We have a system with thousands of repositories located on disks where:

1. write operations are much slower than read operations
2. IO is burstable but is throttled after a certain limit

In such conditions, writing out 10-15 git config settings per each of the
15,000 repositories resulted in around 200,000 calls to git-config and
200,000 disk writes to 15,000 files on every gitolite-admin push, even
if there were no changes to any git configs.

With this change, we call git-config once to read the entirety of the
git config for the repo, and issue writes only if there are changes to
any of the settings.

Similarly, we only touch git-daemon-export-ok if such file doesn't
already exist to avoid another few thousand unnecessary writes.

As a result, our gitolite-admin push times went from 10-12 minutes to 20
seconds.

Signed-off-by: Konstantin Ryabitsev <konstantin@linuxfoundation.org>

2 files changed, 5 insertions, 2 deletions

diff --git a/src/triggers/post-compile/update-git-configs b/src/triggers/post-compile/update-git-configs
index b048992..6eb2f46 100755
--- a/src/triggers/post-compile/update-git-configs
+++ b/src/triggers/post-compile/update-git-configs
@@ -39,12 +39,15 @@ sub fixup_config {
     my $creator = creator($pr);
 
     my $gc = git_config( $pr, '.', 1 );
+    my $ac = `git config --file $RB/$pr.git/config -l`;
     while ( my ( $key, $value ) = each( %{$gc} ) ) {
         next if $key =~ /^gitolite-options\./;
         $value =~ s/(@\w+)/expand_group($1)/ge if $rc{EXPAND_GROUPS_IN_CONFIG};
+        my $lkey = lc $key;
+        next if $ac =~ /^\Q$lkey\E=\Q$value\E$/m;
         if ( $value ne "" ) {
             system( "git", "config", "--file", "$RB/$pr.git/config", $key, $value );
-        } else {
+        } elsif ( $ac =~ /^\Q$lkey\E=/m ) {
             system( "git", "config", "--file", "$RB/$pr.git/config", "--unset-all", $key );
         }
     }
diff --git a/src/triggers/post-compile/update-git-daemon-access-list b/src/triggers/post-compile/update-git-daemon-access-list
index b1cf460..ade97a8 100755
--- a/src/triggers/post-compile/update-git-daemon-access-list
+++ b/src/triggers/post-compile/update-git-daemon-access-list
@@ -24,7 +24,7 @@ for my $d (`$cmd | gitolite access % daemon R any`) {
     my @F = split "\t", $d;
     if ($F[2] =~ /DENIED/) {
         unlink "$RB/$F[0].git/$EO";
-    } else {
+    } elsif (! -f "$RB/$F[0].git/$EO") {
         textfile( file => $EO, repo => $F[0], text => "" );
     }
 }