これは jfilter.info、jfilter.texi より makeinfo バージョン 4.1
によって作成されました。

Japanese Character Code Conversion Filter procedures in Scheme
**************************************************************

Cover
*****

  This collection contains Scheme procedures converting ISO 2022
Japanese character codes (JIS, EUC) and Shift-JIS, as well as
converting a certain number of 'ZENKAKU' characters into 'HANKAKU'
characters among them.  They were first conceived to be compiled with
HOBBIT to help SCM to speed up the letter by letter text handling,
which no interpreters are good at.  I tried this collection to be
compliant with R5RS.  This is free software distributed under the terms
of GNU General Public License.

           May 2002 By Dai INUKAI (inukai.d@jeans.ocn.ne.jp)

About this collection of procedures
***********************************

  This procedure collection deals with Japanese characters.  The
primitives' codes can be extended or reduced by the user according to
the objectives to which they are applied. The collection might also be
rewritten for the character code conversion between ISO 2022 compliant
GB 2312 or KS C5601 and EUC because the code conversion algorithm would
not be different.

  The Japanese characters are made of 2 phonetics, 'katakana' and
'hiragana', and of ideographical 'kanji' characters (Chinese
characters).  Alphabets are also used.

  'hiragana' and 'kanji' are expressed in 2 bytes and there are 2
versions of 'katakana': single byte 'hankanu katakana' and 2-byte
'zenkaku katakana'.  This collection converts any 'hankanu katakana'
into 'zenkaku katakana'.

  There are 3 ascii derived code systems in Japanese: JIS, EUC and
Shift-JIS.

JIS:

  The Japanese character basics are defined by ISO 2022 or JIS (Japanese
Industrial Standards):

  - JISX0201 character sets define control characters, ascii characters
(#x01 through #x7e) and 'hankaku' characters (#xa1 through #xdf).

  - JISX0208 character sets (appeared in 1978 and revised in 1983)
define 2 byte characters which are composed of 7-bit 1st byte (#x21
through #x7e) and 7-bit 2nd byte (#x21 through #x7e).

  The character sets are specified by escape sequences, Escape Character
code (#x1b) with specification characters, after which follow JIS
character codes.  This method seems to be common in Chinese GB 2312 and
Korean KS C5601.

  The control and ascii characters are common in all 3 character sets.
The differences are in multi-byte characters.

EUC:

  Every character is expressed in 8 bits (single byte or multi byte).
If the MSBs of JIS code becomes 1, then it becoms EUC code.

  The 'hankaku katakana' characters are the same as in JIS but preceded
by #x8e.

Shift JIS:

  This code system is widely used in the MS-DOS derived operating
systems and the text is often structured with both CARRIAGE RETURN and
NEWLINE.  Every character is expressed in 8 bits and the ascii and
'hankaku katakana' characters are the same as in JIS.

  The Shift JIS 2 byte characters are made from JIS as follows:

   * JIS 2 byte codes are divided into 2 groups: odd 1st byte codes and
     even 1st byte codes.

   * For the codes of which the 1st byte is even, adding #x7e to the
     2nd byte makes the Shift JIS 2nd byte.

   * For the codes of which the 1st byte is odd:

        * If the 2nd byte is less than #x60, adding #x1f to it makes
          the Shift JIS 2nd byte.

        * If the 2nd byte is equal or superior to #x60, adding #x20 to
          it makes the Shift JIS 2nd byte.

   * Finally the 1st byte is shifted as follows so that it fits into
     the area not used by ascii code nor by 'hankaku katakana':

        * If the 1st byte is less than #x5f, it is shifted to fit
          between #x81 and #x9f.

        * If the 1st byte is equal or superior to #x5f, it is shifted
          to fit between #xe0 and #xfc.

  There are character code conversion programs such as `nkf' or `qkc'
already.  To add a new similar program, there is no other reason than
that it is written in Scheme.  But because it is written in Scheme, you
can modify or add the codes, and it is easy to link with other programs
written in Scheme.

Constants
*********

  There are constants used for various purposes.

  These are character codes not clearly defined in R5RS but used in the
proceduress frequenty:

     CHAR:ESCAPE: Escape character corresponding to hexadecimal #X1B
     CHAR:RETURN: Carriage return  corresponding to hexadecimal #X0D

  The CHAR:RETURN is used in MS-DOS derived operating systems to
structure the text with lines in combination with NEWLINE code.

  These are symbols used to denote the character sets:

     'jis   : denotes JIS character sets, 1978  or 1983.
     'eucj  : denotes Japanese EUC character sets.
     'sjis  : denotes Shift JIS character sets.
     'ascii : denotes ascii character sets.
     'binary: This is not a character set but denotes binary codes.

  Thess are ISO 2022 Escape Sequences, list of Escape Sequence
characters, denoting the following character property:

     JCCCF:ASCII:        #x1B #x28 #x42
                       Following characters are ascii.
     JCCCF:ROMAN:        #x1B #x28 #x4a
                       Following characters are roman.
     JCCCF:X0201:        #x1B #x28 #x49
                       Following characters are 'hankaku katakana'.
     JCCCF:LATIN1:       #x1B #x2D #x41
                       latin-1 character set is following.
     JCCCF:X0208-1978:   #x1B #x24 #x40
                       JISX0208-1978 2 byte characters follow.
     JCCCF:X0208-1983:   #x1B #x24 #x42
                       JISX0208-1983 2 byte characters follow.
     JCCCF:X0208-1978-2: #x1B #x24 #x28 #x40
                       Another way of denoting JISX0208-1978.
     JCCCF:X0208-1983-2: #x1B #x24 #x28 #x42
                       Another way of denoting JISX0208-1983.

  There are other escape sequences in ISO 2022 for Chinese and Korean
but this collection does not use them.

Code Conversion predicates
**************************

 - procedure: eucj1? c
     Returns #t if the char C is the 1st byte of Japanese euc 2-byte
     character, otherwise #f.

 - procedure: eucj2? c
     Returns #t if the char C is the 2nd byte of Japanese euc 2-byte
     character, otherwise #f.

 - procedure: sjis1? c
     Returns #t if the char C is the 1st byte of shift-jis 2-byte
     character, otherwise #f.

 - procedure: sjis2? c
     Returns #t if the char C is the 2nd byte of shift-jis 2-byte
     character, otherwise #f.

 - procedure: hankana? c
     Returns #t if the char C is a 'hankaku katakana' character.

Low level procedures for Jfilter
********************************

  The procedures in this section are the primitives and service
procedures to construct the Japanese character code conversion Filter
procedures in the next section.

 - procedure: upper-byte w
     Returns upper byte of the 2 bytes integer W.

 - procedure: lower-byte w
     Returns lower byte of the 2 bytes integer W.

 - procedure: byte->word uc lc
     Returns 2 bytes integer whose upper char code is UC and lower char
     code LC.

  The above 3 procedures are 'helpers' used internally in the following
procedures.

 - procedure: han->zen c to-code
     Returns 2 byte 'zenkaku katakana' character list in TO-CODE,
     corresponding to 1 byte 'hankaku katakana' character list C.
     Specify 'jis, 'sjis or 'eucj to TO-CODE. This procedure changes
     the argument C into 'zenkaku katakana' character list, too.

 - procedure: jis->sjis s
     Converts jis character list S into shift-jis character list and
     returns it.  This procedure rewrites the argument S.

 - procedure: sjis->jis s
     Converts shift-jis character list S into jis character list and
     returns it.  This procedure rewrites the argument S.

 - procedure: jis->eucj s
     Converts jis character list S into Japanese euc character list and
     returns it.  This procedure rewrites the argument S.

 - procedure: eucj->jis s
     Converts Japanese euc character list S into jis character list and
     returns it.  This procedure rewrites the argument S.

 - procedure: sjis->eucj s
     Converts sjis character list S into Japanese euc character list
     and returns it.  This procedure rewrites the argument S.

 - procedure: eucj->sjis s
     Converts Japanese euc character list S into jis character list and
     returns it.  This procedure rewrites the argument S.

 - procedure: jcccf:convert s from-code to-code prev-sequence
          cur-sequence add-cr zen2han?
     Converts character list S in FROM-CODE into TO-CODE and returns
     it. Specify different variables for PREV-SEQUENCE and CUR-SEQUENCE
     so that you can check if the character currently being read
     changes the type from PREV-SEQUENCE _because Scheme reads files
     always SEQUENTIALLY_.  See the source code of JFILTER:CV for the
     use of these variables and the procedure itself.  The boolean
     ADD-CR is used to add the Carriage Return character before
     #\newline or not to when you are converting into Shift JIS. Set #t
     to add and #f not to.  This procedure rewrites the argument S.

     ZEN2HAN? is a boolean argument which controls whether to convert a
     certain number of 'zenkaku' (2 byte) characters in EUCJP,
     SHIFT_JIS or JIS(ISO-2022-JP) into 'hankaku' (sigle byte)
     characteurs.

     The local procedure zen->han to jcccf:convert searches the
     'zenkaku' character given to the character list CHAR-LIST from the
     'zenkaku' vectors and returns the corresponding 'hankaku'
     character having the same index in the 'hankaku' character vector.

     I made an arbitrary choice of 'zenkaku' characters corresponding
     to the following 'hankaku' characters:

     SP,.:;?!"^~_-/|`'()[]{}<>+-=$%#&*  and

     [0-9A-Za-z]*.

 - procedure: jcccf:set-sequence! cur-sequence prev-sequence sequence
     Sets the newly encountered character property specified in
     SEQUENCE to the current property CUR-SEQUENCE.  The property which
     has been valid and become obsolete is set to the PREV-SEQUENCE.
     Specify to SEQUENCE Escape Sequences as described at the top of
     this document.  This procedure is provided to change the passed
     arguments themselves, rather than the copy of them.  The lists
     CUR-SEQUENCE and PREV-SEQUENCE must be different ones.

 - procedure: jcccf:write-list s port
     Writes character list S to the given port.

Jfilter Main procedures
***********************

 - procedure: cv-file input output from-code to-code remove-cr add-cr
          check-length zen2han?
     Converts one entire INPUT in FROM-CODE into OUTPUT in TO-CODE. The
     INPUT and OUTPUT may be either Scheme ports, filename strings or
     #f. The TO-CODE and FROM-CODE must be one of 'jis, 'eucj, 'sjis,
     or #f. The ascii characters are recognized and passed through
     without any change.  The last 4 argguments, REMOVE-CR, ADD-CR,
     CHECK-LENGTH and HAN2ZEN? are optional and can be omitted.  The
     REMOVE-CR and ADD-CR, boolean arguments, serves for the conversion
     from or to Shift-Jis and specifies, if they exists, whether the
     Carriage Return must be removed or added. The CHECK-LENGTH, number
     argument, specifies the number of characters to be read by the
     program in order to establish the code of input.  The HAN2ZEN?,
     boolean arguement, specifies whether to allow conversions of
     'zenkaku' characters into 'hankaku', whatever the character code
     is.  In order to specify HAN2ZEN?, all of the optional arguments
     must be specified.  In order to specify the CHECK-LENGTH you must
     specify both REMOVE-CR and ADD-CR, and REMOVE-CR can not be
     ommitted to specify ADD-CR.  You can specify #f to all the
     mandatory arguments (cv-file #f #f #f #f), in which case INPUT
     defaults to (current-input-port), OUTPUT to (current-output-port),
     FROM-CODE to <judged automatically> calling JUDGE-FILE, TO-CODE to
     'eucj, REMOVE-CR to #t, ADD-CR to #f, CEHCK-LENGTH to 5000 and
     HAN2ZEN? to #f.

 - procedure: cv-string string from-code to-code
     Converts the Scheme string STRING in FROM-CODE into TO-CODE and
     returns the converted newly allocated string.  This procedure uses
     the SLIB library package 'string-port.  The TO-CODE and FROM-CODE
     must be one of 'jis, 'eucj or 'sjis. The ascii characters are
     passed through without any change.  The procedure may be useful
     when used from other Scheme programs converting text lines portion
     by portion.  When compiled with HOBBIT, the code seems not
     clonable and slower than with the SCM interpreter exectution.  For
     the users of SCM and HOBBIT, I recommed to use the code from your
     ScmInit.scm.

          ;an example
          (require 'line-i/o)

          (define (convert-string file from-code to-code)
            (call-with-input-file file
              (lambda (inport)
                (do ((s (read-line inport)(read-line inport)))
                    ((eof-object? s))
                  (write-line (cv-string s from-code to-code))))))

     The above procedure converts all the lines of a text file and
     displays them.  The procedure cv-file is more efficient for this
     purpose.  Though no limit in string length, the CV-STRING is
     better used for converting a line portion of a text file.

 - procedure: jfilter:cv inport outport from-code to-code remove-cr
          add-cr
     The <body> of CV-FILE and CV-STRING, which accepts ports and other
     parameters from the upper procedures and makes the character code
     conversion.  Converts the contents of the INPORT and the result is
     output to the OUTPORT.

 - procedure: judge-file input c-length
     C-LENGTH is optional. Judges the code of INPUT reading INPUT upto
     C-LENGTH characters if it exists, upto 5000 characters if it
     doesn't. Returns a symbol from 'jis, 'sjis, 'eucj, 'ascii or
     'binary according to the file specified to INPUT. Specification to
     INPUT may be a filename string or a Scheme port.  Beware this
     procedure is far from perfect when there are plenty of character
     codes common to all 3 character code sets in the file INPUT,
     especially 'hankaku katakana' characters.