diff options
author | Robin H. Johnson <robbat2@gentoo.org> | 2015-08-08 13:49:04 -0700 |
---|---|---|
committer | Robin H. Johnson <robbat2@gentoo.org> | 2015-08-08 17:38:18 -0700 |
commit | 56bd759df1d0c750a065b8c845e93d5dfa6b549d (patch) | |
tree | 3f91093cdb475e565ae857f1c5a7fd339e2d781e /app-admin/gentoo-rsync-mirror | |
download | gentoo-56bd759df1d0c750a065b8c845e93d5dfa6b549d.tar.gz gentoo-56bd759df1d0c750a065b8c845e93d5dfa6b549d.tar.bz2 gentoo-56bd759df1d0c750a065b8c845e93d5dfa6b549d.zip |
proj/gentoo: Initial commit
This commit represents a new era for Gentoo:
Storing the gentoo-x86 tree in Git, as converted from CVS.
This commit is the start of the NEW history.
Any historical data is intended to be grafted onto this point.
Creation process:
1. Take final CVS checkout snapshot
2. Remove ALL ChangeLog* files
3. Transform all Manifests to thin
4. Remove empty Manifests
5. Convert all stale $Header$/$Id$ CVS keywords to non-expanded Git $Id$
5.1. Do not touch files with -kb/-ko keyword flags.
Signed-off-by: Robin H. Johnson <robbat2@gentoo.org>
X-Thanks: Alec Warner <antarus@gentoo.org> - did the GSoC 2006 migration tests
X-Thanks: Robin H. Johnson <robbat2@gentoo.org> - infra guy, herding this project
X-Thanks: Nguyen Thai Ngoc Duy <pclouds@gentoo.org> - Former Gentoo developer, wrote Git features for the migration
X-Thanks: Brian Harring <ferringb@gentoo.org> - wrote much python to improve cvs2svn
X-Thanks: Rich Freeman <rich0@gentoo.org> - validation scripts
X-Thanks: Patrick Lauer <patrick@gentoo.org> - Gentoo dev, running new 2014 work in migration
X-Thanks: Michał Górny <mgorny@gentoo.org> - scripts, QA, nagging
X-Thanks: All of other Gentoo developers - many ideas and lots of paint on the bikeshed
Diffstat (limited to 'app-admin/gentoo-rsync-mirror')
8 files changed, 288 insertions, 0 deletions
diff --git a/app-admin/gentoo-rsync-mirror/files/gentoo-mirror.conf b/app-admin/gentoo-rsync-mirror/files/gentoo-mirror.conf new file mode 100644 index 000000000000..d848c3b91d7f --- /dev/null +++ b/app-admin/gentoo-rsync-mirror/files/gentoo-mirror.conf @@ -0,0 +1,9 @@ +# Gentoo rsync mirror config + +RSYNC="/usr/bin/rsync" +OPTS="--quiet --recursive --links --perms --times --devices --delete --timeout=300" +#Uncomment the following line only if you have been granted access to rsync1.us.gentoo.org +#SRC="rsync://rsync1.us.gentoo.org/gentoo-portage" +#If you are waiting for access to our master mirror, select one of our mirrors to mirror from: +SRC="rsync://rsync.de.gentoo.org/gentoo-portage" +DST="/opt/gentoo-rsync/portage/" diff --git a/app-admin/gentoo-rsync-mirror/files/rsync-gentoo-portage.sh b/app-admin/gentoo-rsync-mirror/files/rsync-gentoo-portage.sh new file mode 100644 index 000000000000..e053fadab1ed --- /dev/null +++ b/app-admin/gentoo-rsync-mirror/files/rsync-gentoo-portage.sh @@ -0,0 +1,11 @@ +#!/bin/bash + +source /etc/rsync/gentoo-mirror.conf + +echo "Started update at" `date` >> $0.log 2>&1 +logger -t rsync "re-rsyncing the gentoo-portage tree" +${RSYNC} ${OPTS} ${SRC} ${DST} >> $0.log 2>&1 +logger -t rsync "deleting spurious Changelog files" +find ${DST} -iname ".ChangeLog*" | xargs rm -rf + +echo "End: "`date` >> $0.log 2>&1
\ No newline at end of file diff --git a/app-admin/gentoo-rsync-mirror/files/rsyncd.conf b/app-admin/gentoo-rsync-mirror/files/rsyncd.conf new file mode 100644 index 000000000000..349b723b50d9 --- /dev/null +++ b/app-admin/gentoo-rsync-mirror/files/rsyncd.conf @@ -0,0 +1,26 @@ +# Copyright 1999-2004 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 +# $Id$ + +uid = nobody +gid = nobody +use chroot = yes +max connections = 20 +pid file = /var/run/rsyncd.pid +motd file = /etc/rsync/rsyncd.motd +transfer logging = no +log format = %t %a %m %f %b +syslog facility = local3 +timeout = 300 + +[gentoo-x86-portage] +#this entry is for compatibility +path = /opt/gentoo-rsync/portage +comment = Gentoo Linux Portage tree + +[gentoo-portage] +#modern versions of portage use this entry +path = /opt/gentoo-rsync/portage +comment = Gentoo Linux Portage tree mirror +exclude = distfiles + diff --git a/app-admin/gentoo-rsync-mirror/files/rsyncd.motd b/app-admin/gentoo-rsync-mirror/files/rsyncd.motd new file mode 100644 index 000000000000..35950dd898dd --- /dev/null +++ b/app-admin/gentoo-rsync-mirror/files/rsyncd.motd @@ -0,0 +1 @@ +This is rsync[number].[country].gentoo.org. diff --git a/app-admin/gentoo-rsync-mirror/files/rsynclogparse-extended.pl b/app-admin/gentoo-rsync-mirror/files/rsynclogparse-extended.pl new file mode 100644 index 000000000000..b0befc31c7ab --- /dev/null +++ b/app-admin/gentoo-rsync-mirror/files/rsynclogparse-extended.pl @@ -0,0 +1,128 @@ +#!/usr/bin/env perl + +# +# rsynclogparse-extended.pl, version 1.0 +# Script for producing daily or hourly stats from an rsync.log, in +# plain text or XML output formats +# +# (C) Tim Haynes <gentoo@stirfried.vegetable.org.uk>, February 2003 +# Redistributable under the terms of the BSD licence +# <http://www.opensource.org/licenses/bsd-license.php> +# + +$|=1; + +#Determine whether we have a commandline option or not +$arg=""; +$arg=shift + if $ARGV[0]=~/^-/; + +#Hash of variables to be output and descriptions +%outputVars=( + "mirrorid" => "which mirror name this box is", + "contact" => "email address to contact the server administrator", + "read" => "total bytes read", + "wrote" => "total bytes served", + "total" => "total bytes both directions", + "count" => "number of connections", + "meanxfer" => "mean transfer size", + "biggestXfer" => "biggest individual transfer", + "speedupavg" => "mean speedup-a-like ratio for all conns", + "avgbandwidth" => "mean bandwith reequirement over 1d", + "interval" => "most recent n-seconds' worth of data", + "maxconns" => "number of times max-conns reached", + "percmaxconns" => "percentage of connections rejected", + "configmaxconns" => "max concurrent connections configured", + "timestamp" => "Current time these stats were generated" + ); + +#Initialise all the above to 0 +map { $$_ =0 ; } keys %outputVars; + +#Set fields for this specific server +$mirrorid="rsync1.uk.gentoo.org"; +$contact="gentoo\@stirfried.vegetable.org.uk"; +$configmaxconns=5; + +$timestamp=time(); + +#Determine if we're doing a daily or hourly thing +$interval=3600; +$interval=2600*24 + if $arg=~/d/; + +($sec,$min,$hour,$mday,$mon,$year,$wday,$yday) = gmtime(time); +$now=dateToTstamp(sprintf("%04d/%02d/%02d %02d:%02d:%02d", + $year+1900, $mon+1, $mday, $hour, $min, $sec)); + + +#Read in all remaining files on commandline and stdin +while (<>) { + chomp; + + if (m#^((\d+)/(\d+)/(\d+) (\d+):(\d+):(\d+))#) { + $tstamp=dateToTstamp($1); + } + + next # skip too-old log entries + if $tstamp < ($now-$interval); + + $maxconns++ + if /max connections .\d+. reached/oi; + + /wrote (\d+) bytes.*read (\d+) bytes.*size (\d+)/oi + or next; + + $wrote+=$1; #running total of outgoing + $read+=$2; #running total of incoming + $volumesize=$3; #total size of the volume, serversize + $localtotal=$1 + $2; + $speedupsum+=$volumesize/$localtotal; #running total of "speedup" ratios + $count++; + $biggestXfer=($localtotal>$biggestXfer)?$localtotal:$biggestXfer; +} + +#Compute a few things +$total=$read+$wrote; +$speedupavg=$speedupsum/$count; #average speedup ratio +$meanxfer=$total/$count; #mean-size xfer per connection +$avgbandwidth=$total/$interval; #mean bandwith consumed over this interval +$percmaxconns=100*$maxconns/($count+$maxconns); + +#Choice of output format +$arg =~/xml/ ? &outputXML : &outputText; + +1; + +################ + +sub outputText { + foreach $i ( keys %outputVars ) { + printf("%-20s: $$i\n", $i); + } +} + + +sub outputXML { + print "<xml>\n <rsyncstats>\n"; + foreach $i ( keys %outputVars ) { + if ($arg=~/v/o) { + print " <$i desc=\"$outputVars{$i}\">$$i</$i>\n"; + } else { + print " <$i>$$i</$i>\n"; + } + } + print " </rsyncstats>\n</xml>\n"; +} + + +sub dateToTstamp { + my $str=shift; + + $str =~ m#^(\d+)/(\d+)/(\d+) (\d+):(\d+):(\d+)#; + + $tstamp=$6 + 60*$5 + 3600*$4 + 3600*24*$3 + + 3600*24*31*$2 + 3600*24*365*($1-1975); + + return $tstamp; +} diff --git a/app-admin/gentoo-rsync-mirror/gentoo-rsync-mirror-1.0-r4.ebuild b/app-admin/gentoo-rsync-mirror/gentoo-rsync-mirror-1.0-r4.ebuild new file mode 100644 index 000000000000..d98bf1c2d799 --- /dev/null +++ b/app-admin/gentoo-rsync-mirror/gentoo-rsync-mirror-1.0-r4.ebuild @@ -0,0 +1,50 @@ +# Copyright 1999-2015 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 +# $Id$ + +DESCRIPTION="Ebuild for setting up a Gentoo rsync mirror" +HOMEPAGE="https://wiki.gentoo.org/wiki/Project:Infrastructure/Rsync" +SRC_URI="" + +LICENSE="GPL-2" +SLOT="0" +KEYWORDS="alpha amd64 hppa ppc ppc64 sparc x86" +IUSE="" + +src_install() { + dodir /opt/gentoo-rsync + cp "${FILESDIR}"/rsync-gentoo-portage.sh "${D}"/opt/gentoo-rsync + cp "${FILESDIR}"/rsynclogparse-extended.pl "${D}"/opt/gentoo-rsync + insinto etc/rsync + doins "${FILESDIR}"/rsyncd.conf + doins "${FILESDIR}"/rsyncd.motd + doins "${FILESDIR}"/gentoo-mirror.conf + dodir /opt/gentoo-rsync/portage +} + +pkg_postinst() { + elog "The rsync-mirror is now installed into /opt/gentoo-rsync" + elog "The local portage copy resides in /opt/gentoo-rsync/portage" + elog "Please change /opt/gentoo-rsync/rsync-gentoo-portage.sh for" + elog "configuration of your main rsync server and use it so sync." + elog "Change /etc/rsync/rsyncd.motd to display your correct alias." + elog + elog "RSYNC_OPTS="--config=/etc/rsync/rsyncd.conf" needs" + elog "to be set in /etc/conf.d/rsyncd to make allow syncing." + elog + elog "The service can be started using /etc/init.d/rsyncd start" + elog "If you are setting up an official mirror, don't forget to add" + elog "00,30 * * * * root /opt/gentoo-rsync/rsync-gentoo-portage.sh" + elog "to your /etc/crontab to sync your tree every 30 minutes." + elog + elog "If you are setting up a private (unofficial) mirror, you can add" + elog "0 3 * * * root /opt/gentoo-rsync/rsync-gentoo-portage.sh" + elog "to your /etc/crontab to sync your tree once per day." + elog + elog "****IMPORTANT****" + elog "If you are setting up a private mirror, DO NOT sync against the" + elog "gentoo.org official rotations more than once a day. Doing so puts" + elog "you at risk of having your IP address banned from the rotations." + elog + elog "For more information visit: https://wiki.gentoo.org/wiki/Project:Infrastructure/Rsync" +} diff --git a/app-admin/gentoo-rsync-mirror/gentoo-rsync-mirror-1.0-r5.ebuild b/app-admin/gentoo-rsync-mirror/gentoo-rsync-mirror-1.0-r5.ebuild new file mode 100644 index 000000000000..6efa9c1b5c5f --- /dev/null +++ b/app-admin/gentoo-rsync-mirror/gentoo-rsync-mirror-1.0-r5.ebuild @@ -0,0 +1,54 @@ +# Copyright 1999-2015 Gentoo Foundation +# Distributed under the terms of the GNU General Public License v2 +# $Id$ + +EAPI=5 + +DESCRIPTION="Ebuild for setting up a Gentoo rsync mirror" +HOMEPAGE="https://wiki.gentoo.org/wiki/Project:Infrastructure/Rsync" +SRC_URI="" + +LICENSE="GPL-2" +SLOT="0" +KEYWORDS="~amd64 ~alpha ~ppc ~sparc ~x86 ~hppa ~ppc64" +IUSE="" + +S="${WORKDIR}" + +src_install() { + exeinto /opt/gentoo-rsync + doexe "${FILESDIR}"/rsync-gentoo-portage.sh + doexe "${FILESDIR}"/rsynclogparse-extended.pl + insinto etc/rsync + doins "${FILESDIR}"/rsyncd.conf + doins "${FILESDIR}"/rsyncd.motd + doins "${FILESDIR}"/gentoo-mirror.conf + dodir /opt/gentoo-rsync/portage +} + +pkg_postinst() { + elog "The rsync-mirror is now installed into /opt/gentoo-rsync" + elog "The local portage copy resides in /opt/gentoo-rsync/portage" + elog "Please change /opt/gentoo-rsync/rsync-gentoo-portage.sh for" + elog "configuration of your main rsync server and use it to sync." + elog "Change /etc/rsync/rsyncd.motd to display your correct alias." + elog + elog "RSYNC_OPTS="--config=/etc/rsync/rsyncd.conf" needs" + elog "to be set in /etc/conf.d/rsyncd to make allow syncing." + elog + elog "The service can be started using /etc/init.d/rsyncd start" + elog "If you are setting up an official mirror, don't forget to add" + elog "00,30 * * * * root /opt/gentoo-rsync/rsync-gentoo-portage.sh" + elog "to your /etc/crontab to sync your tree every 30 minutes." + elog + elog "If you are setting up a private (unofficial) mirror, you can add" + elog "0 3 * * * root /opt/gentoo-rsync/rsync-gentoo-portage.sh" + elog "to your /etc/crontab to sync your tree once per day." + elog + elog "****IMPORTANT****" + elog "If you are setting up a private mirror, DO NOT sync against the" + elog "gentoo.org official rotations more than once a day. Doing so puts" + elog "you at risk of having your IP address banned from the rotations." + elog + elog "For more information visit: https://wiki.gentoo.org/wiki/Project:Infrastructure/Rsync" +} diff --git a/app-admin/gentoo-rsync-mirror/metadata.xml b/app-admin/gentoo-rsync-mirror/metadata.xml new file mode 100644 index 000000000000..2a2586cc30ca --- /dev/null +++ b/app-admin/gentoo-rsync-mirror/metadata.xml @@ -0,0 +1,9 @@ +<?xml version="1.0" encoding="UTF-8"?> +<!DOCTYPE pkgmetadata SYSTEM "http://www.gentoo.org/dtd/metadata.dtd"> +<pkgmetadata> +<herd>proxy-maintainers</herd> +<maintainer> + <email>rgkmg25@gmail.com</email> + <name>Robert Kowalski</name> +</maintainer> +</pkgmetadata> |