I wanted to make a git repository for config files on all my servers. Here's a timeline of the config files I've changed:
Original files from backups:
0baseline
Additions or changes made on these dates:
2024-0708
2024-0720
2024-1021
2024-1130
2025-0125
2025-0218
2025-0227
I can copy these files in stages into a new directory "repo", create the git repository, and do repeated checkins.
I use GNU (https://git.savannah.gnu.org/cgit/config.git/plain/config.guess) to come up with a generic system identifier. It's in the form ARCHITECTURE-OS, like this:
x86_64-freebsd-11.3-rel x86_64-centos6-linux i386-pc-solaris2.11
Here's a (slightly anonymized) directory tree for some boxes I've maintained:
/dist | +--servers | | +--freebsd | | | +--x86_64-freebsd-11.3-rel | | | | | | | | +--ORIGINAL [Existing freebsd-11 backups] | | | | | +--0baseline | | | | | | +--etc | | | | | | | +--hosts | | | | | | | +--mail | | | | | | | +--make.conf | | | | | | | +--passwd | | | | | | | +--rc.conf | | | | | | | +--rc.local | | | | | | | +--resolv.conf | | | | | | | +--services | | | | | | | +--shells | | | | | | | +--sysctl.conf | | | | | | +--usr | | | | | | | +--local | | | | | | | | +--etc | | | | | | | | | +--moduli | | | | | | | | | | +--2024-0708 [Changed files as of 8 July 2024] | | | | | | +--usr | | | | | | | +--local | | | | | | | | +--etc | | | | | | | | | | +--2024-0720 [Changed files as of 20 July 2024] | | | | | | +--usr | | | | | | | +--local | | | | | | | | +--etc | | | | | | | | | +--sysinfo.conf | | | | | | | | | | +--... [Changed...] | | | | +--furbag.home.arpa [Test git repo created here] | | | | | | +--x86_64-freebsd-13.2-rel | | | | +--hairball.home.arpa | | | | +--linux | | | +--x86_64-centos6-linux | | | | +--backups.work.com [Holds backups for fileserver] | | | | +--solaris | | | +--i386-pc-solaris2.11 | | | | +--testora.work.com [Test box for Oracle DB] | | | | | | +--sparc-sun-solaris2.10 | | | | +--filesvr.work.com [Original fileserver]
Copy config files to their home:
root# cd /dist/servers/freebsd/x86_64-freebsd-11.3-rel/furbag.home.arpa root# mkdir repo [copy /etc, /usr/local/etc files from initial install to repo] root# cd repo root# ls -l repo/etc -rw-r--r-- 1 root wheel 611 28-Oct-2023 05:12:43 hosts drwxr-xr-x 2 root wheel 5 24-Nov-2020 02:24:43 mail/ -rw-r--r-- 1 root wheel 2344 10-Nov-2022 18:13:28 make.conf -rw-r--r-- 1 root wheel 2607 10-Aug-2023 17:25:19 passwd drwxr-xr-x 6 root wheel 6 16-May-2024 00:57:58 periodic/ -rw-r--r-- 1 root wheel 1879 11-Oct-2023 21:52:38 rc.conf [...]
Create a bare repository:
root# git version git version 2.43.3 root# git init -b main --bare .git Initialized empty Git repository in /dist/servers/freebsd/x86_64-freebsd-11.3-rel/furbag.home.arpa/repo/.git/ root# cat .git/HEAD ref: refs/heads/main
Don't nag me about untracked files, and mark this as no longer being a bare repository:
root# cat .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = true
root# git config --local status.showUntrackedFiles no
root# git config --local core.bare false
root# cat .git/config
[core]
repositoryformatversion = 0
filemode = true
bare = false
[status]
showUntrackedFiles = no
Here are some config files I'd like to track:
root# git status
On branch main
No commits yet
nothing to commit (create/copy files and use "git add" to track)
root# ls -lF
drwxr-xr-x 5 root wheel 14 16-May-2024 00:57:58 etc/
drwxr-sr-x 3 root sys 3 19-Jun-2024 05:17:33 usr/
root# git add etc
root# git add usr
root# git status
On branch main
No commits yet
Changes to be committed:
(use "git rm --cached <file>..." to unstage)
new file: etc/hosts
new file: etc/mail/aliases
new file: etc/mail/local-host-names
new file: etc/mail/mailertable.sample
new file: etc/make.conf
new file: etc/passwd
new file: etc/periodic/dayend/100.dmesg
new file: etc/periodic/dayend/200.ets
new file: etc/periodic/hourly/100.swap
new file: etc/periodic/locate/SETUP
new file: etc/periodic/snapshot/100.zsnap-inc
new file: etc/rc.conf
new file: etc/rc.local
new file: etc/resolv.conf
new file: etc/services
new file: etc/shells
new file: etc/ssl/dovecot-openssl.cnf
new file: etc/sysctl.conf
new file: usr/local/etc/man.d/heirloom-doctools.conf
new file: usr/local/etc/moduli
new file: usr/local/etc/pf.conf
new file: usr/local/etc/ssh_config
new file: usr/local/etc/sshd_config
new file: usr/local/etc/sysinfo.conf
new file: usr/local/etc/xdays
new file: usr/local/etc/xweeks
Untracked files not listed (use -u option to show untracked files)
First commit:
root# git commit -m 'Baseline commit.'
[main (root-commit) c4a4caf] Baseline commit.
Committer: Charlie Root <root@furbag.home.arpa>
Your name and email address were configured automatically based
on your username and hostname. Please check that they are accurate.
You can suppress this message by setting them explicitly:
git config --global user.name "Your Name"
git config --global user.email you@example.com
After doing this, you may fix the identity used for this commit with:
git commit --amend --reset-author
26 files changed, 62728 insertions(+)
create mode 100644 etc/hosts
create mode 100644 etc/mail/aliases
create mode 100644 etc/mail/local-host-names
create mode 100644 etc/mail/mailertable.sample
create mode 100644 etc/make.conf
create mode 100644 etc/passwd
create mode 100755 etc/periodic/dayend/100.dmesg
create mode 100755 etc/periodic/dayend/200.ets
create mode 100755 etc/periodic/hourly/100.swap
create mode 100644 etc/periodic/locate/SETUP
create mode 100755 etc/periodic/snapshot/100.zsnap-inc
create mode 100644 etc/rc.conf
create mode 100755 etc/rc.local
create mode 100644 etc/resolv.conf
create mode 100644 etc/services
create mode 100644 etc/shells
create mode 100644 etc/ssl/dovecot-openssl.cnf
create mode 100644 etc/sysctl.conf
create mode 100644 usr/local/etc/man.d/heirloom-doctools.conf
create mode 100644 usr/local/etc/moduli
create mode 100644 usr/local/etc/pf.conf
create mode 100644 usr/local/etc/ssh_config
create mode 100644 usr/local/etc/sshd_config
create mode 100644 usr/local/etc/sysinfo.conf
create mode 100644 usr/local/etc/xdays
create mode 100644 usr/local/etc/xweeks
Here's my name and email:
root# git config --local user.name 'Karl Vogel' root# git config --local user.email 'vogelke@furbag.home.arpa' root# git commit --amend --reset-author
Here are some other Git settings I've found useful:
root# cat .git/config
# Basics.
[core]
editor = vim
pager =
repositoryformatversion = 0
filemode = true
logallrefupdates = true
bare = false
# Treat all tag names as version numbers and sort them correctly.
[tag]
sort = v:refname
[status]
showUntrackedFiles = no
[user]
name = Karl Vogel
email = vogelke@furbag.home.arpa
# Instead of typing "git commit", just type "git ci".
[alias]
br = branch
ci = commit
co = checkout
last = log -1 HEAD
st = status
unstage = reset HEAD --
# https://git-scm.com/docs/pretty-formats
# FORMATTING
# Opt Output (reldate = relative date)
# -------------------------------------------------------
# %ad Author date (format respects the ?date= option)
# %aD Author date, RFC2822 style
# %ar Author date, relative
# %at Author date, UNIX timestamp
# %ai Author date, ISO 8601-like format
# %aI Author date, strict ISO 8601 format
# %as Author date, short format (YYYY-MM-DD)
# %ah Author date, human style
# -------------------------------------------------------
# %ae Author e-mail %h Abbreviated commit hash
# %an Author name %H Commit hash
# %ar Author reldate %n Newline
# %cd Committer date %p Abbreviated parent hashes
# %ce Committer email %P Parent hashes
# %cn Committer name %s Subject
# %cr Committer reldate %t Abbreviated tree hash
# %d Ref names %T Tree hash
# -------------------------------------------------------
[format]
pretty = %h %ai %an - %s
Now we have a basic setup. To see the files being tracked:
root# git ls-files etc/hosts etc/mail/aliases etc/mail/local-host-names etc/mail/mailertable.sample etc/make.conf [...] usr/local/etc/pf.conf usr/local/etc/ssh_config usr/local/etc/sshd_config usr/local/etc/sysinfo.conf usr/local/etc/xdays usr/local/etc/xweeks root# git ls-files | xargs ls -ld -rw-r--r-- 1 root wheel 611 28-Oct-2023 05:12:43 etc/hosts -rw-r--r-- 1 root wheel 1556 24-Nov-2020 02:24:43 etc/mail/aliases ...
We haven't changed anything since the first commit:
root# git ls-files --modified [no output]
Add files that were updated 7/8/2024:
root# git ls-files --modified usr/local/etc/xdays usr/local/etc/xweeks root# git ls-files --modified | xargs ls -ld -r--r--r-- 1 root bin 1701155 08-Jul-2024 04:33:56 usr/local/etc/xdays -r--r--r-- 1 root bin 1234806 08-Jul-2024 04:33:56 usr/local/etc/xweeks
Commit them:
root# git commit -a -m "commit 2024-07-08" [main 315f8bf] commit 2024-07-08 2 files changed, 52029 insertions(+), 6739 deletions(-) root# git log 315f8bf 2025-03-16 18:59:10 -0400 Karl Vogel - commit 2024-07-08 dc6db9d 2025-03-16 18:42:58 -0400 Karl Vogel - Baseline commit.
Do the same for files that were updated 7/20/2024:
[copy /usr/local/etc files from later snapshot] root# git ls-files --modified usr/local/etc/sysinfo.conf root# git ls-files --modified | xargs ls -ld -rw-r--r-- 1 root wheel 531 20-Jul-2024 23:27:24 usr/local/etc/sysinfo.conf root# git commit -a -m "commit 2024-07-20" [main 1adc4ce] commit 2024-07-20 1 file changed, 1 insertion(+), 1 deletion(-) root# git log 1adc4ce 2025-03-16 19:05:44 -0400 Karl Vogel - commit 2024-07-20 315f8bf 2025-03-16 18:59:10 -0400 Karl Vogel - commit 2024-07-08 dc6db9d 2025-03-16 18:42:58 -0400 Karl Vogel - Baseline commit.
"sysinfo" is a utility used to gather system config information. Here's what changed in that file between the baseline commit and 7/20:
root# git diff dc6db9d 1adc4ce usr/local/etc/sysinfo.conf
diff --git a/usr/local/etc/sysinfo.conf b/usr/local/etc/sysinfo.conf
index d3e1344..0ab3b8b 100644
--- a/usr/local/etc/sysinfo.conf
+++ b/usr/local/etc/sysinfo.conf
@@ -11,7 +11,7 @@ MODPATH=${APPDIR}/modules
ALLMODS="system bios cpu mem os storage network user packages services misc"
# Colorful messages?
-COLORS="YES"
+COLORS="NO"
# Include informative messages in the output?
INFO_MSGS="YES"
Instead of having to use git hashes to identify commits, use tags:
root# git log 1adc4ce 2025-03-16 19:05:44 -0400 Karl Vogel - commit 2024-07-20 315f8bf 2025-03-16 18:59:10 -0400 Karl Vogel - commit 2024-07-08 dc6db9d 2025-03-16 18:42:58 -0400 Karl Vogel - Baseline commit. root# git tag -a 2024.0619 -m "" dc6db9d root# git tag -a 2024.0708 -m "" 315f8bf root# git tag -a 2024.0720 -m "" 1adc4ce
Running git tag with no arguments displays the current tags:
root# git tag 2024.0619 2024.0708 2024.0720
I'd like to display tags in log messages.
You can create a script called "git-whatever", put it in your path, and run "git whatever" to use it like any other git command. You can't override builtin commands that way, so I alias "lo" to "logtag":
root# git config --local alias.lo logtag root# git config -l | grep alias alias.br=branch alias.ci=commit alias.co=checkout alias.last=log -1 HEAD alias.st=status alias.unstage=reset HEAD -- alias.lo=logtag
Here's the git-logtag script:
#!/bin/ksh
#<git-logtag: put tags in log output
export PATH=/usr/local/bin:/bin:/usr/bin
set -o nounset
tag=${0##*/}
# Get userid associated with author name.
author=$(git log --pretty='%an' | head -n1)
userid=$(getent passwd | grep "$author" | cut -f1 -d:)
git log "--format=format:%h %d %ai $userid - %s" "$@" |
sed -e 's/(.*tag: \([0-9\.]*\))/[\1]/'
exit 0
Results:
root# git lo 1adc4ce [2024.0720] 2025-03-16 19:05:44 -0400 vogelke - commit 2024-07-20 315f8bf [2024.0708] 2025-03-16 18:59:10 -0400 vogelke - commit 2024-07-08 dc6db9d [2024.0619] 2025-03-16 18:42:58 -0400 vogelke - Baseline commit.
Tags can make comparisons easier:
root# git diff --name-only 2024.0619 2024.0720 usr/local/etc/sysinfo.conf usr/local/etc/xdays usr/local/etc/xweeks
For more specific info:
root# git diff --name-status 2024.0619 2024.0720 M usr/local/etc/sysinfo.conf M usr/local/etc/xdays M usr/local/etc/xweeks
Where:
A = Added D = Deleted M = Modified R = Renamed
I committed some additional changes. If you mis-tag something, you can always remove and replace it:
root# git log a50016c 2025-03-16 19:34:46 -0400 Karl Vogel - commit 2025-02-27 ac53a74 2025-03-16 19:34:10 -0400 Karl Vogel - commit 2025-02-18 66eaa96 2025-03-16 19:27:07 -0400 Karl Vogel - commit 2024-11-30 d40def9 2025-03-16 19:26:24 -0400 Karl Vogel - commit 2024-10-21 1adc4ce 2025-03-16 19:05:44 -0400 Karl Vogel - commit 2024-07-20 315f8bf 2025-03-16 18:59:10 -0400 Karl Vogel - commit 2024-07-08 dc6db9d 2025-03-16 18:42:58 -0400 Karl Vogel - Baseline commit. root# git tag -a 2024-10-21 -m "" d40def9 root# git tag -d 2024-10-21 Deleted tag '2024-10-21' (was 0fd2bb9) root# git tag -a 2024.1021 -m "" d40def9 root# git tag -a 2024.1130 -m "" 66eaa96 root# git tag -a 2025.0218 -m "" ac53a74 root# git tag -a 2025.0227 -m "" a50016c root# git lo a50016c [2025.0227] 2025-03-16 19:34:46 -0400 vogelke - commit 2025-02-27 ac53a74 [2025.0218] 2025-03-16 19:34:10 -0400 vogelke - commit 2025-02-18 66eaa96 [2024.1130] 2025-03-16 19:27:07 -0400 vogelke - commit 2024-11-30 d40def9 [2024.1021] 2025-03-16 19:26:24 -0400 vogelke - commit 2024-10-21 1adc4ce [2024.0720] 2025-03-16 19:05:44 -0400 vogelke - commit 2024-07-20 315f8bf [2024.0708] 2025-03-16 18:59:10 -0400 vogelke - commit 2024-07-08 dc6db9d [2024.0619] 2025-03-16 18:42:58 -0400 vogelke - Baseline commit.
If you want to see what was changed in each commit:
root# git tag 2024.0619 2024.0708 2024.0720 2024.1021 2024.1130 2025.0218 2025.0227 root# for x in $(git tag); do > printf "\n==========================================================\n" > git show --stat --oneline $x > done ======================================================== tag 2024.0619 dc6db9d (tag: 2024.0619) Baseline commit. etc/hosts | 19 + etc/mail/aliases | 69 + etc/mail/local-host-names | 3 + etc/mail/mailertable.sample | 7 + etc/make.conf | 67 + etc/passwd | 43 + ... usr/local/etc/sysinfo.conf | 23 + 26 files changed, 62728 insertions(+) ======================================================== tag 2024.0708 315f8bf (tag: 2024.0708) commit 2024-07-08 usr/local/etc/xdays | 36117 +++++++++++++++++++++++++++++++++--------- usr/local/etc/xweeks | 22651 ++++++++++++++++++++++++++++++- 2 files changed, 52029 insertions(+), 6739 deletions(-) ======================================================== tag 2024.0720 1adc4ce (tag: 2024.0720) commit 2024-07-20 usr/local/etc/sysinfo.conf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ======================================================== tag 2024.1021 d40def9 (tag: 2024.1021) commit 2024-10-21 usr/local/etc/moduli | 266 +++++++++++++++++++---------------- usr/local/etc/ssh_config | 16 ++- usr/local/etc/sshd_config | 15 ++- 3 files changed, 174 insertions(+), 123 deletions(-) ======================================================== tag 2024.1130 66eaa96 (tag: 2024.1130) commit 2024-11-30 usr/local/etc/pf.conf | 140 +++------------------------------------ 1 file changed, 7 insertions(+), 133 deletions(-) ======================================================== tag 2025.0218 ac53a74 (tag: 2025.0218) commit 2025-02-18 etc/shells | 2 -- 1 file changed, 2 deletions(-) ======================================================== tag 2025.0227 a50016c (HEAD -> main, tag: 2025.0227) commit 2025-02-27 etc/hosts | 12 ++++++------ etc/mail/aliases | 2 +- etc/mail/local-host-names | 2 +- etc/mail/mailertable.sample | 4 ++-- etc/periodic/dayend/100.dmesg | 2 +- etc/periodic/dayend/200.ets | 2 +- etc/periodic/hourly/100.swap | 2 +- etc/periodic/locate/SETUP | 9 +++++---- etc/periodic/snapshot/100.zsnap-inc | 2 +- etc/rc.conf | 27 ++++++++++++++++++++++----- etc/rc.local | 2 +- etc/resolv.conf | 2 +- etc/services | 2 +- etc/ssl/dovecot-openssl.cnf | 4 ++-- etc/sysctl.conf | 2 +- 15 files changed, 47 insertions(+), 29 deletions(-)
You don't have to run hash commands to check for changed files; that's where a good VCS really shines. The modtime on /etc/passwd changed on 25 Jan 2025 but the contents didn't, so the file appeared in the backups. Git didn't do anything because it stores files based on the hash of their contents:
root# git commit -a -m 'commit 2025-01-25' On branch main nothing to commit (use -u to show untracked files)
Git doesn't preserve ownership and permissions well enough to be trusted with things like /etc files, so you should track that yourself.
I've included two scripts (*getperm* and *setperm*) to save and restore metadata. The scripts are in Perl; you can use something like *mtree* (BSD) or the Linux port if you prefer something else. Integrity checkers like *aide* can save metadata, but you'd have to write soemthing to restore from it.
*getperm* saves user, group, filemode and modification time (Unix epoch) for files you provide:
me% cat etc-files /etc/login.conf /etc/make.conf /etc/nsswitch.conf /etc/ntp.conf /etc/periodic.conf /etc/rc.conf /etc/resolv.conf /etc/sysctl.conf /etc/syslog.conf me% getperm < etc-files > /tmp/PERM me% cat /tmp/PERM root|wheel|0644|1602146383|/etc/login.conf root|wheel|0644|1740961763|/etc/make.conf root|wheel|0644|1562302062|/etc/nsswitch.conf root|wheel|0644|1604374088|/etc/ntp.conf root|wheel|0664|1746132227|/etc/periodic.conf root|wheel|0644|1750916712|/etc/rc.conf root|wheel|0644|1740692215|/etc/resolv.conf root|wheel|0644|1740692263|/etc/sysctl.conf root|wheel|0644|1694070209|/etc/syslog.conf
You can restore these settings as root:
root# setperm < /tmp/PERM
You can download getperm and setperm here.
Getting files from the servers: on the server in question, have root create a tarball of the files you want to track and leave it in an agreed-on directory. Use the "-p" flag to preserve permissions; this should be the default if running as the superuser.
Unpack the files in a scratch directory next to the repository and run something to save metadata in a separate file.
Then move all unpacked files to the repository, along with the stored metadata file -- I use "PERM" so it overwrites the previous metadata and is stored with the current files.
To get your filelist, you can provide your own or start with something like this: copy everything under the system and local "etc" directories that isn't data or a symlink:
root# find /etc /usr/local/etc -print0 |
xargs -0 file -N --mime-type |
grep -E -v 'application/octet-stream|inode/symlink|\.sample' |
sed -e 's/: .*//' -e 's/^\///' | sort > /tmp/tocopy
Results:
root# cat /tmp/tocopy etc etc/X11 etc/amd.map etc/apmd.conf etc/auto_master ... etc/sysctl.conf etc/syslog.conf etc/syslog.conf.orig etc/syslog.d usr/local/etc usr/local/etc/Muttrc usr/local/etc/Muttrc.dist usr/local/etc/UPower usr/local/etc/UPower/UPower.conf ... usr/local/etc/wgetrc usr/local/etc/xpdfrc
Create a tarball named after the host (assumes GNU tar):
root# host="$(hostname)"
root# cd /
root# tar --no-recursion -b 2560 --files-from=/tmp/tocopy \
--preserve-permissions -czf /tmp/${host}-cfg.tgz
Use scp or whatever you want to bring /tmp/${host}-cfg.tgz back to your central server. Unpack it in the repository and get the metadata:
root# cd /dist/servers/repo
[scp remote /tmp/${host}-cfg.tgz here]
[ssh remove /tmp/${host}-cfg.tgz on remote system]
root# tar -tzf ${host}-cfg.tgz > new.toc
root# tar -xzf ${host}-cfg.tgz
root# /path/to/getperm < new.toc > PERM
root# rm ${host}-cfg.tgz new.toc
Now you're ready to check your files in. If you're not comfortable running the git commands as root, change the ownership to an unprivileged user and run as that instead; you already have the metadata so you can restore it any time.
I create a regular user "bkup" to copy via SCP between systems; this way I can use root to create a tarball and copy it elsewhere safely.
Remove the entire fileset (output from git ls-files) before unpacking the new set. This lets us accurately track deleted files. Example:
root# rm etc/ssl/dovecot-openssl.cnf root# git diff --name-status D etc/ssl/dovecot-openssl.cnf root# git ls-files --modified etc/ssl/dovecot-openssl.cnf
Feel free to send comments.
Generated from article.t2t by
txt2tags
$Revision: 1.4 $
$UUID: 60591950-34ac-3b45-95d4-d3797805ebb3 $