- => achieved
+ => not yet achieved
[] => post-release comments

Known Bugs  (i.e. the ToDo list)

+ ifile does not handle foreign character sets
+ C code looks better, but still needs some fixing and modularization
+ ifile may cause problems (corrupt ~/.idata file or mishandle incoming
  mail) if 2 irefiles are called simultaneously or ifilter is called while
  running irefile (this is only a problem with the MH command line system -
  EXMH does not try to multitask these programs)
+ irefile does not handle 'cur' as a message name, but will refile the
  'cur'rent message if no message is given on the command line
+ irefile is still quite slow and inefficient

Suggested Improvements  (possible long-term goals)

+ ifile doesn't allow user to override decision made by ifile
  (user must refile to correct ifile's mistake)
+ separate install programs for system/individual user
+ allow local installation of irefile (i.e. eliminate the need for
  being root on system)
+ cause the frequency counts in the .idata file to decay at some rate so that
  more recent training is more important to ifile

Notes

- The 'make' program distrubted with SunOS v4.1.3 is buggy.  There are two
  solutions to this problem.  1) Get GNU make and compile ifile using that
  or 2) hand link the executables.  The make included w/ SunOS will compile
  the source files, but it will not link the object files to create the
  executables.  You can hand link the executables with the following commands:

    gcc ifilter.o error.o ifile.o assoc_array_int.o -o ifilter -lm
    gcc irefile.o error.o ifile.o assoc_array_int.o -o irefile -lm

Successful compiles (v0.4.2):

- RedHat Linux v4.2/gcc v2.7.2.1
- SunOS v4.1.3/gcc v2.7.2 (using GNU make)
- HP-UX A.09.05/gcc v2.7.2
- SunOS v5.5.1/gcc v2.7.2

v0.4.5

- ifile.c: parsing of .mh_profile file made slightly more robust

v0.4.4

- irefile.c: changed to look for .skip_me file in correct place.

v0.4.3

- knowledge_base.perl: keep separate nuking counts for entire run and
  each aging call
- .idata file is now explicitly closed - fclose() call did not exist
  in earlier versions
- minor changes to extendable array code
- ifile_rate_categories now adds "inbox" to the mailbox listing of the
  mailbox listing is empty (leaving it empty would cause a seg fault in
  some cases)
- Makefile now contains explicit linking commands (some 'make' programs
  will not link executables without explicit commands)
- added to distribution descriptions of how to better integrate ifile into
  EXMH (see exmh_integration, exmh_integration2 and filter_button)

v0.4.2

- minor changes to install.perl script
- news2mail.perl added to distribution (search for 'news2mail' in README file
  for some explaination)
- .idata file locking introduced into system.  ifile programs are synched
  via lock files so that none will ever read or write a corrupted .idata file
  (not implemented in all executables yet)
- new version of "knowledge_base.perl".  Does age updates periodically
  to lessen processor overhead.  Includes commented out lines for attaching
  associative arrays to disk (primarily useful if you have a very large
  corpus of e-mail and a large number of mailbox folders).
- irefile no longer learns on messages which are refiled to folders which
  contain a .skip_me file (synchronizes behavior with knowledge_base.perl)
- primitive code aiming toward alternate style refile included in
  distribution.  Search for 'irefile queue' in README file for more info.
- cutoff frequency for eliminating words from .idata now done using a log
  scale (previously used a constant).
[- modularized idata writing]

v0.4.1

- moved opening of log file to top of ifilter.c to prevent seg fault when
  argc > 1
- previouslly while reading a message, ifile would reset as_nk value of
  any word/category it came across (essentially drilling holes in the
  .idata file).  This has been fixed.
- to allow a smoother transition between lexing styles, each Subject:, From:
  word is now lexed into two words, one with the prefix and one without.
  (the duplication will be eliminated in the near future)

v0.4.0

- removed 'make clean' command from install.perl so that the user can make
  his/her own executables and then use install.perl to install them
- fixed problem with opening of log file.  Program previously tried to use
  entire argv[0] as part of the log file path.  Now uses only executable
  name.
- because a portion of argv[0] is used to name the .info files, the irefile
  .info file will most likely appear as "/tmp/refile.info"
- renaming of some variables
- main ifilter program modularized. <cbbrowne>
- corresponding parts of irefile modularized.
- ifilter now only reads sections of .idata which it needs to classify the
  message it is given (greatly increases efficiency) <cbbrowne>
  ^^^ Good stuff - ifilter is quite a bit faster and less of a memory hog :)
- invariants removed from "category rating" loop to promote efficiency
  <cbbrowne>
- the UPGRADE file has been removed from distribution (due to lack of utility)
- the convert.perl file has been removed from the distribution (you should
  instead use knowledge_base.perl to recreate your .idata file)
- message headers, "From:" and "Subject:", are now treated specially during
  lexing.
[- machine names and e-mail addresses are now lexed as one word and are no
   longer broken to pieces.]

v0.3.3

- loop variable initialization bugfixes
- rewrite of mh_refile function (hopefully eliminating strange string
  bugs/random seg faults) --> thanks to Colin McCormack for the suggestion!
- fixed logical AND (&&) to bitwise AND (&) in refile_exe function (doh!)
- some clean-up of code - code is more modular
- added ifile_sprintf() and ifile_cats() to aid in string handling
  Any programmers who have deal with strings in C should love these :)
- added error.c for message printing and error handling
- modularized printing of progress and error messages
- added ifile.c to hold library of ifile functions
- converted readline() to a function (from a #define) and in doing so
  introduced (but shortly thereafter fixed) a bug
- ifile now adds a header to filtered messages "X-filter: ifile v0.3.3"

v0.3.2

- included errno.h in irefile.c (required by SunOS v5.5.1/gcc-2.7.2)
- eliminated strerror() function from code.  Function is unnecessary
  and SunOS 4.1.3/gcc-2.7.2 produces undefined references to 'strerror'
  during compilation.  This seemed to be the only easy fix.
- eliminated problem with 'make' putting -lm (math library) option at the
  beginning of the line

v0.3.1

- install.perl links to 'irefile' instead of 'refile', like it should

v0.3.0

- C implementation of ifilter.  Incredible speed improvements (see chart below)
- C implementation of irefile.  Significant increase in speed (see chart
  below).  Not quite as extreme as ifilter.  irefile heavily depends on (slow)
  associative arrays for writing the .idata file
- Both programs are much more memory conscious (see chart below).
- Full support for all MH refile options.  I think I finally have full
  support for all the options.  Please tell me if I don't!
- irefile will use 'cur' message and current folder if no message/source
  folder is given
- message headers (subject, from) NOT given extra weight (this will probably be
  reversed in the future)
- install program does not affect location of ifilter entry in .maildelivery
  filter, simply changes ifilter path

My own experiences on my i586 100Mhz Linux box (48 megs ram)
202k .idata file, 39 folders, 5061 words

time/memory comsumption to filter/refile a single 2k message

                processor time   memory usage
perl irefile        20 sec         5.1 Meg
C    irefile       7.5 sec         2.2 Meg
perl ifilter        10 sec         4.7 Meg
C    ifilter       1.5 sec         2.0 Meg

v0.2.5

- ifile now properly addresses the '-file' option [mostly]
- knowledge_base.perl is more memory conscious
- resets .idata_accuracy file if accuracy goes negative
- irefile would previously refile all messages after it read in the first of
  any batch.  It now waits to refile after all messages are read in.
- uses context file to get the current-folder when no source file
  is indicated in the calling of irefile [this is actually incorrect]
- ifilter will not filter messages into a folder with a .skip_me file
- finding PWD was made slightly more robust

v0.2.4

- irefile no longer mistakes MH refile options for file names
- allows for more flexibility in reading of .mh_profile
- passing '-help' to ifilter or irefile will print usage

v0.2.3

- script included to create knowledge database according to where mail is
  currently located (knowledge_base.perl).
- message headers (subject, from) given extra weight

v0.2.2

- if original MH refile program is available as refile.bak, ifile will use MH
  refile program to move the message (increases compatibility)
- keeps track of number of filters and number of refilings for approximate
- /tmp/ifilter.info now contains lots of info about ifile's filtering decisions
  
v0.2.1

- handling of multi-level folders (sub-folders)
- refiled messages no longer appear as new messages
- refiling to folder which message comes from does not destroy message
- protections set up to restore original message if refile is not performed
  correctly

v0.2.0

- fixes problems concerning strange folder names
- more concise data file - provides quicker filtering and refiling
- keeps track of number of messages filtered and number of messages refiled
- looks at .folders to determine folder names, rather than directory structure
- keeps data file streamlined by eliminating infrequent words
- records 'words' of length 3 (previously required length of 4 or greater)

v0.1.2

- Fixed problem of not allowing user to indicate directory of the rmm binary
- Installation program changed to search for binaries in common directories
  before asking user (provides easier installation)

v0.1.1

- Coordinated naming of main data file - '.idata'
- Installation program allows customization of binary directories (previously,
  program made assumptions as to locations of mh binaries)


Ditched Suggestions (things that DON'T work well):

+ Write programs in perl but use DBM files to store database
+ Client/server version of ifile (data file is usually put in swap space)