         fffff       lll
        fff  ff       lll
 iii    fff fff iii   lll
 iii    fff     iii   lll
      fffffff         lll   eeeeee           000    3333    3333
iiii    fff    iiii   lll  eee   ee         0   0       3       3
 iii    fff     iii   lll  eeeeeeee     v v 0   0    333     333
 iii    fff     iii   lll  eee          v v 0   0       3       3
iiiii  fffff   iiiii lllll  eeeeee       v   000  * 3333  * 3333

Thu May 15 05:49:54 EDT 1997

If you try to use ifile at all, I would definitely like to hear from you.
I would greatly appreciate hearing about your installation and/or the actual
operation of the system.
Please send me e-mail at "jr6b+@andrew.cmu.edu".

NOTE FOR THOSE UPGRADING: Please read the UPGRADE file.

NOTE FOR ALL USERS: Please read the CHANGES file.

Official ifile web site --> "http://www.cs.cmu.edu/~jr6b/ifile/" <--

There is now an ifile mailing list! (two, in fact)  The first,
ifile-announce, is meant for all users of ifile.  Posts to this list will
be limited to new version and major developmental announcements.
The second, exmh-ifile, is meant for all those who wish to help in the
developmental process of ifile by making suggestions, submitting bug
reports and discussing various aspects of ifile.  All those who are
subscribed to exmh-ifile will automatically be subscribed to ifile-announce.

To subscribe to exmh-ifile, send mail to jr6b+@andrew.cmu.edu with the
subject header "subscribe exmh-ifile".  To subscribe to ifile-announce, send
mail to jr6b+@andrew.cmu.edu with the subject header
"subscribe ifile-announce".

What is ifile?

ifile is an intelligent mail filtering program which uses framework provided by
the MH mailing system and the slocal filtering system.  It uses the text-based
learning algorithm, Naive Bayesian, to determine what type of mail you like
to have filtered into your different folders.  It learns by observing any
actions you take to move (refile) messages to different folders than they were
originally filtered to.  The great benefit of ifile is that once ifile is in
place, ifile will update its knowlege about your preferences with every
message that you refile.  Normally, with other filters, you must write
down a strict set of rules and then change those rules as your preferences
change.  ifile does not deal with keywords, but rather with the word frequency
of the messages you receive, thus putting a much stronger emphasis on message
content and the actual meaning of the message.  As it is well integrated in
the EXMH package, it requires little maintainance after the initial
installation.

Does it actually work?

Yes.  I've had a number of users tell me that ifile does a great job of
filtering mail.  In my own experiences, it has also done a nice job.
Since I've added code to keep track of the number of filters vs. refiles,
ifile has correctly filtered over 3/4 of my mail.  What is impressive about
that number is that I have over 30 folders into which I filter mail.  Getting
such a high accuracy for such a difficult task isn't easy.

What exactly does it do?

The ifile mail filtering system adds two pieces to the system commonly
found with MH.  The first of those is a wrapper for the MH refile program.
When a refile request is made, it is first sent to irefile, where information
about the message(s) is gathered.  Once the necessary information is
recorded and stored in the ifile data file, ~/.idata, control is passed to
the MH refile to deal with the messages like normal.

ifile also adds an extension to the slocal filtering program.  A line is
added to your ~/.maildelivery file so that whenever slocal is executed to
filter your mail, control is immediately passed to ifilter.  ifilter reads
the message to be filter, determines which mailbox the message should be
filed into and then calls rcvstore to do the actual filing.  One benefit of
this system is that lines may be added to your .maildelivery file which will
catch mail before it gets to ifilter.  Certain mail which is filtered
with little concern to message content will be very difficult for ifilter
to handle and thus it may be advantagous to bypass ifile and use normal
slocal filtering.

How do I know if I can actually use this system?

To be able to use this system, you must have the entire MH mailing system
installed on your machine.  You will need to have root access
on your machine to install the new refile program.  You must have the slocal
mail filtering program installed on your machine.  You must have perl v4.036 or
later installed on your system (I have tested it with perl v5.003).  You will
also need a C compiler (preferably gcc - I have used gcc v2.7.2 for testing).

As of v0.3.0, I believe that I have full support for MH.  I have performed
limited testing which has shown ifile to work well with command line MH,
but I give no guarantees on its performance.

So, how do I install this system?

(the following is duplicated in the INSTALL file)

Unzip this package in the directory of your choice (I would suggest
/usr/local/ - it will create a subdirectory for its own files) and then
try to compile the binaries with a 'make clean; make all' command.  The
install program will also perform this command, but you should do it
beforehand to make sure the programs compile.  If you receive any warnings
or errors, you will most likely have difficulties running ifile.  Before
reporting these problems, please read the CHANGES file.  If after trying any
suggestions in CHANGES, report these problems to "jr6b+@andrew.cmu.edu".  Make
sure to include such information as operating system name and version,
compiler name and version and ifile version.

Once having seen a successful compile, run the install script (if you do not
have perl installed in /usr/bin, you will need to edit 'install.perl' and
change the first line to the proper directory).  Note that you MUST run the
install script as root (unless your account has read/write access to the
MH bin/ directory).

The script will search for certain binaries, prompting you if it cannot find
them.  If you are prompted and aren't sure of their location, stop the
installation (Ctrl-C) and take the time to find them.

Once it has the directory information, the installation program will modify
your ~/.maildelivery file (creating it if necessary) so that slocal will call
the ifilter program.  It will then modify the programs to reflect the binary
locations and compile the necessary code to create executable ifilter and
irefile programs.  After having done that, it will backup the current MH
refile program (renaming it as refile.bak) and make a link to the location
of irefile.  It is important that refile.bak is not moved or deleted as ifile
relies on it to perform refile actions.

If you are using EXMH, you will need to change the [Preferences]
"Incorporate Mail -> Ways to Inc" option to "presort".

If you are using MH, you will need to have your mail sent to slocal (which
is what the above option will do for EXMH).  One possible way to do this is
to create a .forward file which pipes your mail to slocal.  If that
doesn't work, just about anything you do which will properly execute
slocal will allow ifile to filter your mail.

With the install program run, ifile is ready to be used by your user account.
Unfortunately, if you do not have a single-user system, this WILL effect the
other users on your system who use MH or EXMH.  The two major effects will be
a noticable slowdown when refiling mail and the creation of a file in their
home directory, ".idata".  In the near future, I will be doing testing to see
if I can make it possible to have a local installation (i.e. levying no
ill-effect on other users of the system).

To allow other users to use the ifile filtering system, each user must
have slocal filter their mail and must add the following line to their
~/.maildelivery file:

* - ^ ? /ifile/install/directory/ifilter

(note that ~/.maildelivery must have permissions -rw-r--r--/0644 for slocal to
 properly execute)

The ifilter line acts as any other entry in the .maildelivery file.  If you
wish, you may add entries before the ifilter line if there are messages with
certain keywords which you always want handled in a certain way.

With this version of ifile, I have included a program which will search through
your currently stored messages and create your .idata file accordingly.  I
have found that this system of knowlege accumulation allows ifile much better
knowlege of your preferences and does a much better job of filtering messages.
To use this program, run "knowledge_base.perl" from your user account (i.e.
NOT as root).  If there are any folders which you do not want ifile to
attempt to filter messages into (like a 'drafts' folder), create a ".skip_me"
file in that folder.

Is that it?

Yup.  Once you have completed those tasks, ifile will be installed.  If you
choose to run the knowledge base script, ifile should have a good idea of
how you like your mail organized and you should be able to see the beauty
of ifile after the first few messages you receive.

In general, mailboxes with a specific theme, or those which only contain
messages from a certain person should be easy for ifile to learn.  Those with
varying topics or which contain mail from a wide spread of people may be more
difficult for ifile to learn.

Why is this version 0.3.3?

The program is still under fairly heavy development.  The first two numbers
indicate
the major version number (0.3).  The last number indicates the minor version
number (3).  Each new major version is likely to contain major developments
and lots of new code unseen in previous new versions.  Each new minor version
within a major version will comprise primarily of bug fixes and other less
major changes.

Why are there ifile.jpg and Ifile.jpg files in the distribution?

I felt like ifile could use pictures of some sort, so I decided to create
some ifile images.  ifile.jpg is more of a logo, where Ifile.jpg might serve
as a nice web page image link, indiating that you filter your mail with ifile.
They are hardly works of art, but I think they are kind of neat. :)  Btw,
for those of you interested, they were created with the GIMP v0.99.3.

What's up with the new files, error.c and ifile.c?

I'm starting to put some serious work into modularizing ifile.  error.c is
the library which is being used for message handling and error reporting,
while ifile.c is the library being used for general ifile functions.  I would
like to strip ifilter.c and irefile.c to nothing more than a few function
calls from the ifile.c library.

v0.3.3 brings in a new system of message/error reporting.  I do all message
reporting through a new function, ifile_verbosify().  It allows each message
to be prioritized so that the output of ifile programs can be regulated
with the change of a single variable.  The variable which regulates the
output (ifile_verbosity_level) can be found in error.c and the definitions
for its possible values (ifile_verbosity_levels) can be found in ifile.h
The normal distribution setting will be ifile_quiet, which will only report
to standard error warnings and errors, yet will keep a log of progress
information in the /tmp/*.info file.

Soon there will be functions for database reading, message lexing, message
evaluation and database writing.

Who wrote this?

Why, I did!  'I' am Jason Rennie, a sophomore majoring in Computer Science at
Carnegie Mellon University.  I've been programming for most of my life,
starting with BASIC as a youth, moving through Pascal during high school and
then widening my horizons in college with such languages as C/C++, perl, Java
and ML.  Since installing it in December of '95, I have become a big-time linux
fan, running it exclusively.
I've been doing quite a bit of work with learning algorithms, especially
text-based learning,  and I decided it would be cool to apply
one of the text learning algorithms to mail filtering.  So, I did.

You'll also notice credits to Diego Zamboni for the ifilter.c code.  He began
the port to C, translating the perl code to C and adding a few additions to
up the efficiency.  I definitely have to thank him for his help.

How do I contact this computer geek?

By e-mail, of course!  jr6b+@andrew.cmu.edu  You can also check out my web
page at "http://syrinx.res.cmu.edu/" (may be down during the summer
months)


Credits
-------
Tom Mitchell - Professor who was willing to hire me for the summer during
	       which I was given a great boost into the field of machine
               learning and text learning.  Also a nice guy to work with. :)
Dave Robinson - Alpha tester, pointed out some major bugs in version 0.1.x.
                Good source of encouragement :)
Diego Zamboni - Wrote first port of ifilter to C; also wrote the first
                server/client version of ifilter (in the search of efficiency).
Chris Browne - Proposed numerous suggestions and ideas for increasing the
               efficiency of the ifile C code.  Wrote a perl/DBM version of
               ifile.

Thanks also go to such people as Colin McCormack, Alfred Werner,
Mikael Nilsson, Jim Grinnell and Ken Wallich who have made their own little
contributions to the success of ifile, either by helping me work out the bugs
of the early versions of ifile, making good suggestions for improvement or
by conducting an internet interview with me about ifile :)

If you do find this program helpful and would like to support it and see
it grow into a powerful, full-featured, robust mail filtering system (gotta
make sure to include all the buzzwords) please fill out the following short
survey and mail it to jr6b+@andrew.cmu.edu:

------------------------->8--------------------------

ifile v0.3.3

What operating system do you run? :

What is your default C compiler? :

What versions of exmh and mh do you run? :

What version of perl is on your system? :

Did you have any problems installing ifile?  If so, please describe them here:








Did you encounter any problems while running ifile (lost mail - poor job of
filtering)?  If so, please describe them here (copies of /tmp/irefile.info and
/tmp/ifilter.info would be really helpful!) :








Do you have any suggestions on how to improve ifile?  (this would be most
helpful if you have specific code changes)








----------------------8<---------------------------

Thank you for your interest!!!! :)