This is written from a Unix/Linux perspective, but Mac users running a recent version of the operating system should be able to make sense of it.
Some terms for non-Unix types:
The problem: the filesystem on my Unix workstation was a mess. I couldn't find anything without grepping all over creation. About half the time, I'd actually find something useful. Usually, I'd get no hits at all, or I'd match something like a compiled binary and end up hosing my display beyond belief.
This is what didn't do the trick:
I tried about six different categorization schemes, all of which went nowhere. Poking around in the IBM and Red Hat distribution webpages gave me setups that looked like this:
Making the directory tree was easy. Figuring out where to put things under it, not so much.
For example, I'd learn how to use a program like tcpdump, which is great for examining network traffic to find out why two of your systems aren't working and playing nicely together.
Where does it go? Should it be Networks? Well, only systems people can use the program because it's intended for troubleshooting and you can see things like passwords and the contents of email messages in transit, so how about Admin? I'm not storing the software as such, just instructions on how to use it, so how about Documentation?
Notice the Misc category? Where do you suppose most stuff ended up?
A friend of mine had a well-organized system, and I happened to notice that he had a directory called projects under his home directory. Sounded good, so I made one, too.
OK, do I use the project name for each separate project? Oops, some projects have either duplicate or very close names. How about using the name of the person requesting the work? Does a two-minute job count as a project?
In short, all the same defects as making up my own categories, plus some brand new ones.
I tried this during my "to hell with it" phase, and wound up with over 1,000 files in my home directory. Take a wild guess how well that worked out.
OK, let's try putting text files in one area, PDF files in a second area, web-pages used for presentations in a third area, etc.
All this got me was a bunch of strangely-named files without much in common, in no particular order. At least I could grep through the text files without accidentally getting a match from a PDF file and having gibberish all over my screen.
I noticed someone else storing most of his daily work as email. Anything having to do with a given project all ended up in a mail folder, and he could just use the mail-reader of his choice to navigate around in it. Sounded cool.
658 email folders later, I noticed a tiny problem. Where did I put that security notice about Solaris operating systems? Was it in the security folder or the solaris folder? And why did I put it in this one when it clearly should have gone into the other one? (Pick whatever values you like for "this" or "the other".)
All these methods had one thing in common: putting a bunch of semi-structured information somewhere and expecting it to magically organize itself. Trying to impose additional order on stuff like this is a waste of time when you consider how much new semi-structured information we send and receive every day.
A few things I tried that almost worked...
I ran into something called CyberDewey, written by David Mundie. (Unfortunately, all the links seem to be dead -- you might find something on archive.org). This sounded like the neatest thing since sliced bread, especially since he seemed to have the same problems I did when trying to organize files.
I went so far as to buy a copy of the Abridged Dewey Decimal Catalog, which is actually pretty nifty; if you're looking to organize your paper files, you could do a lot worse than use an existing classification scheme like this.
For example, let's say I have an article on "hurricane relief" and I want to know where to file it. I flip to the Relative Index portion of the catalog (about 200 pages out of 1,000 total), look for "hurricanes", and see something like this:
Hurricanes: 551.55 Weather forecasting: 551.64 See also: Disasters
"Disasters" sounds promising, so I try that:
Disasters: 904 ... Social services: 363.34 Public administration: 353.9
This gives me some category numbers to check in the front portion of the catalog:
363.34 Disasters, including floods and war
353.9 Safety administration
...
Disaster and emergency planning
904 Collected accounts of events
551.55 Atmospheric disturbances, including cyclones, hurricanes, ...
At this point, I have some choices about how to file my article: make a folder called 904 if it's an interview with a Katrina survivor, or 353.9 if it's about FEMA or government response, etc. It's nice because the hard stuff (figuring out the categories and where they go) has all been done for you.
Unfortunately, the stuff available in the Abridged version is a little too general for my job. I also didn't feel like either typing in the categories by hand or forking over $275 for a copy of WebDewey. You can get some of the DDC headings in digital form, but not enough to solve my problem.
See also:
The Open Directory Project (aka DMOZ) wanted to be the largest, most comprehensive human-edited directory of the Web. Among other things, they had a nice set of categories that were well-organized, free, and already in digital form. Unfortunately, most of the links have gone away; the best one I know of is here.
It's not hard to take their category list and turn it into a directory tree suitable for a webpage:
+--Code | +--C | | +--Compilers | | +--Libraries | | +--Mail | | +--Security | | +--Utilities | +--Perl | | +--Admin | | +--CGI | | +--Mail | | +--Misc | +--Shell | | +--Bourne | | +--Korn | +--Tcl | | +--Expect | +--Configuration_Management | +--System_Files | +--Languages | +--Awk | +--C | +--Perl | +--Python | +--Shell | | +--Bourne | | +--Korn | | +--Tcsh | | +--Zsh | +--OS | +--Unix | | +--BSD | | +--Linux | | +--Solaris | +--Security | +--Log-Analysis
The 00hierarchy link holds the computer-related topics from the DMOZ category list that I thought would be most useful.
This is OK for a setup with a small number of files and reasonably clean delineation between topics, but it didn't quite do the trick for my daily work.
"But minister, it isn't like this film is the first troublesome thing to come out of Canada. Let us not forget Bryan Adams."
"No, no. The Canadian government has apologized for Bryan Adams on several occasions." --"South Park, Bigger, Longer, and Uncut"
Canada took a stab at making a set of consistent categories for government records management called ARCS. It's a block numeric records classification system based on function and subject. Each functional or subject grouping of records is assigned a unique three- or four-digit number; this is called a primary number, and it's the main building block for the system. This number is used to classify all information related to a subject or function, regardless of physical format.
Most government offices deal with a similar set of administrative requirements, and the ARCS setup is a pretty nice representation. The documentation comes in PDF form and is very thorough.
The plain-text categories file could be used to make a directory tree suitable for web access:
Unfortunately, I can't seem to find it elsewhere any more. This is intended more for administrative work than something like my job.
I looked at several trouble-ticket/user-request systems, and they're dandy for collaboration between helpdesk people, but they don't really do the trick for organizing my day-to-day information. Here are some of the more interesting packages and papers I saw:
The idea of doing a complete brain-dump and getting everything stored in a trusted location was probably the most useful thing I got from the "Getting Things Done" book.
My job as a system administrator doesn't change every day, but it's much easier to keep track of things via date rather than via subject. I tend to remember things in time-order, so I finally stopped trying to change the way I work to fit some hierarchy. Instead, I made a directory structure on the machine to match my work habits.
I have a top directory cleverly named notebook with subdirectories in the form yyyy/mmdd, so every day has its own folder:
me% dtree ~/notebook notebook: +--2000 | +--1230 | | +--solaris_dev_random.txt +--2001 +--2002 ... +--2020 | +--0204 | | +--Maildir | | | +--cur | | | +--new | | | +--tmp | | | +--awk-tutorial | | | +--index.htm | | | +--iptables-redhat-howto | | | +--sudo-replacement | | | +--doas-mastery | | | +--doas-multiplexd-a307339.zip | | | +--doas-slicer69-6.2p4.tgz
Someone might ask me, "Remember that thing we broke last Friday, and then we fixed it on Monday before anyone noticed?"
me% date
Thu Feb 6 16:45:31 EST 2020
me% cal 2 2020
February 2020
Su Mo Tu We Th Fr Sa
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
me% cd ~/notebook/2020/0131
me% ls -lF --time-style='+%d-%b-%Y %T'
-rw-r--r--. 1 vogelke mis 124 31-Jan-2020 14:03:03 that-thing-we-broke
me% cat that-thing-we-broke
Some blarney about whatever's broke. Could be an email message,
plain text, etc.
me% cd ~/notebook/2020/0203
me% ls -lF --time-style='+%d-%b-%Y %T'
-rw-r--r--. 1 vogelke mis 92 03-Feb-2020 08:05:11 how-we-fixed-it
me% cat how-we-fixed-it
Face-saving crap about how it needed replacing anyways.
The most noticeable improvement was finally being able to find the things that happened on a given date.
Here's what convinced me that I was on the right track: I'd think of a problem, and a fix would occur to me a moment later that was consistent and doable. Most of the scripts and utilities below were either already available or didn't take more than 20 minutes to write.
About 95% of the time, I'm dealing with stuff for yesterday, today, and the coming week. A few symbolic links under my home directory made navigation pretty easy.
Links for 6 Feb 2020:
lrwxrwxrwx. 1 vogelke Feb 6 00:03 yesterday -> notebook/2020/0205 lrwxrwxrwx. 1 vogelke Feb 6 00:03 today -> notebook/2020/0206 lrwxrwxrwx. 1 vogelke Feb 6 00:03 tomorrow -> notebook/2020/0207 lrwxrwxrwx. 1 vogelke Feb 6 00:03 friday -> notebook/2020/0207 lrwxrwxrwx. 1 vogelke Feb 6 00:03 saturday -> notebook/2020/0208 lrwxrwxrwx. 1 vogelke Feb 6 00:03 sunday -> notebook/2020/0209 lrwxrwxrwx. 1 vogelke Feb 6 00:03 monday -> notebook/2020/0210 lrwxrwxrwx. 1 vogelke Feb 6 00:03 tuesday -> notebook/2020/0211 lrwxrwxrwx. 1 vogelke Feb 6 00:03 wednesday -> notebook/2020/0212 lrwxrwxrwx. 1 vogelke Feb 6 00:03 thursday -> notebook/2020/0206
I can get to next Monday's folder by typing cd ~/monday at the prompt, no matter where I am in the filesystem. Since I use the Z-shell, I can also get there by just typing monday. Here are the shell settings:
cdpath=(.. ~ ) # specify a search path for the cd command. setopt autocd # cd to a directory if it's the first word on the command line.
Remind is a calendar and reminder program for Linux and most UNIX Systems, and it's made to order for this sort of thing. Here's an article from the author:
The source can be found at https://dianne.skoll.ca/projects/remind/ or on Fossies. I really like the Fossies setup -- you can download a tarball, see the source tree, or go to the project homepage:
Contents of remind-03.03.00.tar.gz (31 Jan 17:19, 390762 Bytes) About: remind a sophisticated calendar/reminder and alarm program. Fossies downloads: /linux/misc/ remind-03.03.00.tar.gz (tar.bz2|tar.xz|zip) Fossies services: Doxygen docs | Diffs report | CLOC analysis Original URL: https://dianne.skoll.ca/projects/remind/download/remind-03.03.00.tar.gz Home page: https://dianne.skoll.ca/projects/remind/
I use remind to generate the symlinks above from a cron job that runs just after midnight every day:
#!/bin/ksh
#<mknbdir: make notebook directory for the day; run just after midnight.
# make Maildirs for today and tomorrow, plus symlinks for the week.
export PATH=/usr/local/bin:/usr/gnu/bin:/bin:/usr/bin
umask 022
tag=${0##*/}
DATE='date'
# General logging.
logmsg () {
test -t 2
case "$?" in
0) echo "$(date '+%F %T') $tag: $@" ;;
*) logger -t $tag "$@" ;;
esac
}
warn () { logmsg "WARNING: $@"; }
die () { logmsg "FATAL: $@"; exit 1; }
# Make notebooks and incoming mail directories for today and tomorrow
# if they don't already exist.
# NOTE: "date" must have "-d" option for this to work.
cd $HOME
top='notebook'
test -d "$top" || mkdir "$top" || die "$HOME/$top: cannot create"
for day in today tomorrow
do
cur=$($DATE -d "$day 03:00" +%Y/%m%d)
nb="$HOME/$top/$cur"
test -d "$nb" || mkdir -p $nb || die "$nb: cannot create"
chgrp mis $nb
chmod 2755 $nb
# Disable warnings about same age or newer files; if anything else
# didn't work, I'll find out soon enough.
if test -d "$HOME/etc/skel"; then
(
cd $HOME/etc/skel
find . -depth -print | cpio -pdm --quiet $nb 2> /dev/null
logmsg "copied skel to $nb"
)
else
(
cd $nb
mkdir -p Maildir/{cur,new,tmp} || die "$nb: cannot make Maildir"
chmod 700 Maildir/{cur,new,tmp}
logmsg "created $nb/Maildir"
)
fi
done
# Handle symlinked directories.
# NOTE: "date" must have "-d" option for this to work.
other='today yesterday tomorrow sunday monday tuesday
wednesday thursday friday saturday'
for day in $other
do
cur=$($DATE -d "$day 03:00" +%Y/%m%d)
nb="$HOME/$top/$cur"
test -d "$nb" || mkdir -p "$nb" || die "$nb: cannot create"
test -L $day && rm $day
ln -s $top/$cur $day
done
# Any files from yesterday with an extension of .keep should be
# moved to today's directory.
if cd $HOME/yesterday; then
set X $(ls *.keep 2> /dev/null)
shift
case "$#" in
0) ;;
*) logmsg "handling keep files"; mv $* $HOME/today ;;
esac
else
warn "cannot cd to yesterday"
fi
# Keep a year's worth of directories in advance.
# NOTE: must have "remind" installed for this to work.
cd $HOME/$top
echo 'REM MSG daily' |
remind -s+53 - |
sed -e 's!^\(....\)/\(..\)/\(..\).*!\1/\2\3!' |
while read dir
do
test -d $dir || mkdir -p $dir || warn "cannot mkdir $dir"
done
exit 0
This script also illustrates using remind to generate dates without having to worry about leap years, etc. To print this week plus the next 52 weeks:
me% date
Thu Feb 6 16:45:31 EST 2020
me% cal 2 2020
February 2020
Su Mo Tu We Th Fr Sa
1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
me% echo REM MSG daily | remind -s+53 -
2020/02/02 * * * * daily
2020/02/03 * * * * daily
2020/02/04 * * * * daily
2020/02/05 * * * * daily
...
2020/02/27 * * * * daily
2020/02/28 * * * * daily
2020/02/29 * * * * daily <== Leap years are handled correctly
2020/03/01 * * * * daily
2020/03/02 * * * * daily
...
2020/12/29 * * * * daily
2020/12/30 * * * * daily
2020/12/31 * * * * daily
2021/01/01 * * * * daily
...
2021/02/05 * * * * daily
2021/02/06 * * * * daily
Anything on my todo list for today goes in a file called agenda under today's notebook directory. The a command simply runs remind on today's agenda file.
me% echo 'REM MSG Finish bzip upgrade' > ~/today/agenda me% a Reminders for Thursday, 6th February, 2020 (today): * Finish bzip upgrade
A slightly more advanced script to display your agenda for the next few days can be found here.
Let's say I have a meeting at 2pm in room 205, and I'd like a reminder to show up on my screen 5 minutes before it starts.
me% echo REM MSG Meeting in 205 > ~/today/1355.rem
is all it takes. A small program called showcal runs once every minute looking for a file called hhmm.rem in today's notebook directory. If it finds one, the contents are run through remind and zenity to handle the screen popups.
What if I want a reminder every day at a certain time? I just put the same type of file in the ~/.calendar directory instead:
me% cat ~/.calendar/1845.rem REM Tue MSG Trash night
This reminder will show up at 6:45 pm every Tuesday to remind me about trash night. It won't show up on any other days.
me% locate charleston | grep notebook /home/vogelke/notebook/2014/0426/charleston-login /home/vogelke/notebook/2015/0629/charleston-webpage
Alias /home /home/your-userid <Directory /home/your-userid> Options Indexes Order deny,allow deny from all allow from 127.0.0.1 allow from ...whatever... </Directory>
It's a lot easier to break up my workstation text files by frequency of updates, and then set up searching appropriately:
grep is fine for doing a quick-and-dirty search through a few files, but it doesn't work nearly as well for anything larger. Pick something like Rekoll or Xapian that meets your needs.
Feel free to send comments.
Generated from organizing.t2t by
txt2tags
$Revision: 2.5 $
$Date: 2022-01-23 18:05:44-05 $
$UUID: 0a4d682a-c991-474c-89de-7406d4b6df1d $