         fffff       lll
        fff  ff       lll
 iii    fff fff iii   lll
 iii    fff     iii   lll
      fffffff         lll   eeeeee           000    3333    2222
iiii    fff    iiii   lll  eee   ee         0   0       3       2
 iii    fff     iii   lll  eeeeeeee     v v 0   0    333     222
 iii    fff     iii   lll  eee          v v 0   0       3   2
iiiii  fffff   iiiii lllll  eeeeee       v   000  * 3333  * 22222

Tue Apr 22 17:38:00 EDT 1997

If you try to use ifile at all, I would definitely like to hear from you.
I would greatly appreciate hearing about your installation and/or the actual
operation of the system.
Please send me e-mail at "jr6b+@andrew.cmu.edu".

NOTE FOR THOSE UPGRADING: Please read the UPGRADE file.

NOTE FOR ALL USERS: Please read the CHANGES file.

Current version can be found under "ftp://syrinx.res.cmu.edu/pub/ifile/".

What is ifile?

ifile is an intelligent mail filtering program which uses framework provided by
the MH mailing system and the slocal filtering system.  It uses the text-based
learning algorithm, Naive Bayesian, to determine what type of mail you like
to have filtered into your different folders.  It learns by observing any
actions you take to move (refile) messages to different folders than they were
originally filtered to.  The great benefit of ifile is that once ifile is in
place, ifile will update its knowlege about your preferences with every
message that you refile.  Normally, with other filters, you must write
down a strict set of rules and then change those rules as your preferences
change.  ifile does not deal with keywords, but rather with the word frequency
of the messages you receive, thus putting a much stronger emphasis on message
content and the actual meaning of the message.  As it is well integrated in
the EXMH package, it requires little maintainance after the initial
installation.

Does it actually work?

Yes.  I've had a number of users tell me that ifile does a great job of
filtering mail.  In my own experiences, it has also done a nice job.
Since I've added code to keep track of the number of filters vs. refiles,
ifile has correctly filtered over 3/4 of my mail.  What is impressive about
that number is that I have over 30 folders into which I filter mail.  Getting
such a high accuracy for such a difficult task isn't easy.

What exactly does it do?

The ifile mail filtering system replaces two pieces of the system commonly
found with MH.  The first of those is the MH refile program, which is
completely replaced by irefile.  Instead of simply refiling the message which
it is given, irefile reads the entire message and keeps track of the content
of the message in your ~/.idata file.  To ensure complete compatibility with
MH, the message is passed on to the MH refile program in order to be placed in
its rightful mailbox.

ifile also replaces the filtering.  Better put, it uses the slocal filtering
system to call its own mail filtering program.  The idea is simple enough that
it could be called from any filtering program, or even exmh itself (idea for
future version :), but the simplicity of running through slocal won out for
early versions.  The filtering program reads in one message from standard
input, lexes it and then uses the information in the ifile knowledge
database, .idata, to decide which folder to filter the message into.

How do I know if I can actually use this system?

To be able to use this system, you must have the entire MH mailing system
installed on your machine.  You will need to have root access
on your machine to install the new refile program.  You must have the slocal
mail filtering program installed on your machine.  You must have perl v4.036 or
later installed on your system (I have tested it with perl v5.003).  You will
also need a C compiler (preferably gcc - I have used gcc v2.7.2 for testing).

As of this v0.3.0, I believe that I have full support for MH.  I have
performed limited testing which has shown ifile to work with command line MH,
but I give no guarantees.  ifile was originally designed to work with EXMH and
thus is more likely to work with EXMH.

So, how do I install this system?

Unzip this package in the directory of your choice (I would suggest
/usr/local/ - it will create a subdirectory for its own files) and then
try to compile the binaries with a 'make clean; make all' command.  The
install program will also perform this command, but you should do it
beforehand to make sure the programs compile.  If you receive any warnings
or errors, you will most likely have difficulties running ifile.  Before
reporting these problems, please read the CHANGES file.  If after trying the
suggestions in CHANGES, report these problems to "jr6b+@andrew.cmu.edu".  Make
sure to include such information as operating system name and version,
compiler name and version and ifile version.

Once having seen a successful compile, run the install script (if you do not
have perl installed in /usr/bin, you will need to edit 'install.perl' and
change the first line to the proper directory).  Note that you must run the
install script as root.

The script will search for certain binaries, prompting you if it cannot find
them.  If you are prompted and aren't sure of their location, stop the
installation (Ctrl-C) and take the time to find them.

Once it has the directory information, the installation program will modify
your ~/.maildelivery file (creating it if necessary) so that slocal will call
the ifilter program.  It will then modify the programs to reflect the binary
locations and compile the necessary code to create executable ifilter and
irefile programs.  After having done that, it will backup the current MH
refile program (renaming it as refile.bak) and make a link to the location
of irefile.  It is important that refile.bak is not moved or deleted as ifile
relies on it to perform refile actions.

If you are using EXMH, you will need to change the [Preferences]
"Incorporate Mail -> Ways to Inc" option to "presort".

If you are using MH, you will need to have your mail sent to slocal (which
is what the above option will do for EXMH).  One possible way to do this is
to create a .forward file which pipes your mail to slocal.  If that
doesn't work, just about anything you do which will properly execute
slocal will allow ifile to filter your mail.

With the install program run, ifile is ready to be used by your user account.
Unfortunately, if you do not have a single-user system, this WILL effect the
other users on your system who use MH or EXMH.  The two major effects will be
a noticable slowdown when refiling mail and the creation of file in their
home directory, ".idata".  In the near future, I will be doing testing to see
if I can make it possible to have a local installation (i.e. levying no
ill-effect on other users of the system).

To allow other users to use the ifile filtering system, each user must
have slocal filter their mail and must add the following line to their
.maildelivery file:

* - ^ ? /ifile/install/directory/ifilter

The ifilter line acts as any other entry in the .maildelivery file.  If you
wish, you may add entries before the ifilter line if there are messages with
certain keywords which you always want handled in a certain way.

With this version of ifile, I have included a program which will search through
your currently stored messages and create your .idata file accordingly.  I
have found that this system of knowlege accumulation allows ifile much better
knowlege of your preferences and does a much better job of filtering messages.
To use this program, run "knowledge_base.perl" from your user account (i.e.
NOT as root).  If there are any folders which you do not want ifile to
attempt to filter messages into (like a 'drafts' folder), create a ".skip_me"
file in that folder.

Is that it?

Yup.  Once you have completed those tasks, ifile will be installed.  If you
choose to run the knowledge base script, ifile should have a good idea of
how you like your mail organized and you should be able to see the beauty
of ifile after the first few messages you receive.

In general, mailboxes with a specific theme, or those which only contain
messages from a certain person should be easy for ifile to learn.  Those with
varying topics or which contain mail from a wide spread of people may be more
difficult for ifile to learn.

Why is this version 0.3.2?

The program is still under heavy development.  The first two numbers indicate
the major version number (0.3).  The last number indicates the minor version
number (2).  Each new major version is likely to contain major developments
and lots of new code unseen in previous new versions.  Each new minor version
within a major version will comprise primarily of bug fixes and other less
major changes.

Why is there an ifile.jpg file in the distribution?

I felt like ifile could use a logo of some sort, so I decided to create
one.  It's hardly a work of art, but it's kind of neat.  Btw, for those
of you interested, it was created with the GIMP v0.99.3.

Who wrote this?

Why, I did!  'I' am Jason Rennie, a sophomore majoring in Computer Science at
Carnegie Mellon University.  I've been programming for most of my life,
starting with BASIC as a youth, moving through Pascal during high school and
then widening my horizons in college with such languages as C/C++, perl, Java
and ML.  Since installing it in December of '95, I have become a big-time linux
fan, running it exclusively.
I've been doing quite a bit of work with learning algorithms, especially
text-based learning,  and I decided it would be cool to apply
one of the text learning algorithms to mail filtering.  So, I did.

You'll also notice credits to Diego Zamboni for the ifilter.c code.  He began
the port to C, translating the perl code to C and adding a few additions to
up the efficiency.  I definitely have to thank him for his help.

How do I contact this computer geek?

By e-mail, of course!  jr6b+@andrew.cmu.edu  You can also check out my web
page at "http://syrinx.res.cmu.edu/".  One of these days I'll even have a
web page for ifile...


Credits
-------
Tom Mitchell - Professor who was willing to hire me for the summer during
	       which I was given a great boost into the field of machine
               learning and text learning.  Also a nice guy to work with. :)
Dave Robinson - Alpha tester, pointed out some major bugs in version 0.1.x.
                Good source of encouragement :)
Diego Zamboni - Wrote first port of ifilter to C; also wrote the first
                server/client version of ifilter (in the search of efficiency).
Chris Browne - Proposed numerous suggestions and ideas for increasing the
               efficiency of the ifile C code.  Wrote a perl/DBM version of
               ifile.

Thanks also go to such people as Colin McCormack, Alfred Werner,
Mikael Nilsson, Jim Grinnell and Ken Wallich who have made their own little
contributions to the success of ifile, either by helping me work out the bugs
of the early 0.1.x and 0.2.x versions, making good suggestions for
improvement or by conducting an internet interviews with me about ifile :)

If you do find this program helpful and would like to support it and see
it grow into a powerful, full-featured, robust mail filtering system (gotta
make sure to include all th buzzwords) please fill out the following short
survey and mail it to jr6b@andrew.cmu.edu:

------------------------->8--------------------------

ifile v0.3.2

What operating system do you run? :

What version of exmh/mh do you have? :

What version of perl is on your system? :

Did you have any problems installing ifile?  If so, please describe them here:








Did you encounter any problems while running ifile (lost mail - poor job of
filtering)?  If so, please describe them here (copies of /tmp/irefile.info and
/tmp/ifilter.info would be really helpful!) :








Do you have any suggestions on how to improve ifile?  (this would be most
helpful if you have specific code changes)








----------------------8<---------------------------

Thank you for your interest!!!! :)