As mentioned in Chapter 4, the hook variable after-change-functions is special because the functions in it take three arguments (whereas normal hook functions take no arguments). The three arguments refer to the change that took place in the buffer before after-change-functions was executed.

The numbers start and end refer to positions in the buffer after the change. The length len refers to the text before the change. After an insertion, len is zero (because no previously existing text in the buffer was affected), and the newly inserted text lies between start and end. After a deletion, len is the length of the deleted text, now gone, and start and end are the same number, since deleting the text closed the gap, so to speak, between the two ends of the deletion.

Now that we know what the parameters to refill have to be, we can make an artless first attempt at defining it:

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (fill-paragraph nil))

This is a totally inadequate solution because fill-paragraph is far too expensive a function to invoke on every keystroke! It also has the problem that each time you try to add a space to the end of a line, fill-paragraph immediately deletes it—it cleans up trailing whitespace when it fills a paragraph—and since, while you’re typing, the cursor spends most of its time at the end of a line, the only way to get a space between words is to type the two words together, likethis, then go back and put a space between them. But this first try does prove the concept, and gives us a starting point for refinement.^[29]

To optimize refill, let’s analyze the problem a bit. First of all, does the entire paragraph have to be filled every time?

No. When text is inserted or deleted, only the affected line and subsequent lines in the paragraph need to be refilled. Prior lines needn’t be. If text is inserted, the line may become too long, which may cause some text to spill over onto the next line (which may become too long in turn, at which point the process is repeated). If text is deleted, the line may become too short, which may call for some text being slurped up from the following line (which may become too short in turn, and the process is repeated). So changes can’t affect any lines prior to the one in which they occur.

Actually, there’s one case where changes can affect at most one prior line. Consider the following paragraph:

Glitzy features like on-the-fly filling of paragraphs are
needed only to hide the program's many inadequacies
compared to Emacs

Suppose we delete the word “compared” from the beginning of the third line:

Glitzy features like on-the-fly filling of paragraphs are
needed only to hide the program's many inadequacies
to Emacs

The word “to” can now be filled onto the end of the prior line, like so:

Glitzy features like on-the-fly filling of paragraphs are
needed only to hide the program's many inadequacies to
Emacs

A moment’s reflection should convince you that at most one prior line needs to be refilled—and then only when the first word on the current line is shortened or removed.

So we can constrain the paragraph-filling operation to the affected line, perhaps the line before it, and the subsequent lines in the current paragraph. Instead of using fill-paragraph, which determines the paragraph boundaries itself, we’ll choose our own “paragraph boundaries” and use fill-region.

The boundaries we choose for fill-region should enclose the entire affected portion of the paragraph. For an insertion, the “left” boundary is simply start, the point of insertion, and the “right” boundary is the end of the current paragraph. For a deletion, the left boundary is the beginning of the previous line (that is, the line prior to the one containing start), and the right boundary is again the end of the paragraph. So here’s the outline of the new refill:

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (let ((left (if this is an insertion
                  start
                beginning of previous line))
        (right end of paragraph))
    (fill-region left right ...)))

Filling in this is an insertion is easy. Recall that when refill is called, a zero value for len means insertion and a non-zero len means deletion.

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (let ((left (if (zerop len)      ;is len zero?
                  start
                beginning of previous line))
        (right end of paragraph))
    (fill-region left right ...)))

To compute beginning of previous line, we first move the cursor to start, then move the cursor to the end of the previous line (oddly, this can be done with (beginning-of-line 0)), then take the value of (point), all inside a save-excursion:

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (let ((left (if (zerop len)
                  start
                (save-excursion
                  (goto-char start)
                  (beginning-of-line 0)
                  (point))))
        (right end of paragraph))
    (fill-region left right ...)))

We could do something similar for end of paragraph, but instead we’ll use a convenient feature of fill-region: it’ll find the end of the paragraph for us. The fifth argument to fill-region (there are two mandatory arguments and three optional ones), if non-nil, tells fill-region to keep filling through the end of the region until the next paragraph boundary. So there’s no need to compute right.

Our new version of refill is not complete. We must first solve the problem of fill-region positioning the cursor at the end of the affected region. Naturally, it is unacceptable for the cursor to jump to the end of the paragraph on every keystroke! Wrapping the call to fill-region in a call to save-excursion solves the problem.

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (let ((left (if (zerop len)
                  start
                (save-excursion
                  (goto-char start)
                  (beginning-of-line 0)
                  (point))))
    (save-excursion
      (fill-region left end nil nil t)))))

(The second argument to fill-region is ignored because we’re using the feature that finds the end of the paragraph. We pass in end just because it’s handy and not entirely meaningless to a human reader.)

Well, that’s the basic idea, but there’s still plenty to do. For one thing, when computing left, we shouldn’t back up to the previous line if the previous line is not in the same paragraph. So we should locate the beginning of the paragraph and the beginning of the previous line, then use whichever position is greater.

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (let ((left (if (zerop len)
                  start
                (max (save-excursion
                       (goto-char start)
                       (beginning-of-line 0)
                       (point))
                     (save-excursion
                       (goto-char start)
                       (backward-paragraph 1)
                       (point))))))
    (save-excursion
      (fill-region left end nil nil t))))

(The function max returns the larger of its arguments.)

We now have three calls to save-excursion, which is a moderately expensive function. It might be better to combine two of them into one and compute both values inside it.

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (let ((left (if (zerop len)
                  start
                (save-excursion
                  (max (progn
                         (goto-char start)
                         (beginning-of-line 0)
                         (point))
                       (progn
                         (goto-char start)
                         (backward-paragraph 1)
                         (point)))))))
  (save-excursion
    (fill-region left end nil nil t))))

Next, recall our earlier observation about filling the prior line: “at most one prior line needs to be refilled—and then only when the first word on the current line is shortened or removed.” But in the code we’ve written, we’re backing up to the previous line on every deletion. Let’s see if we can avoid that in the case where the deletion occurred in or beyond the second word of a line.

We’ll do this by changing this

(if (zerop len)
    start
  find previous line)

to

(if (or (zerop len)
        (not (before-2nd-word-p start)))
    start
 find previous line)

where before-2nd-word-p is a function that tells whether its argument, a buffer position, lies before the second word on a line.

Now we must write before-2nd-word-p. It should locate the second word on the line, then compare its position with its argument.

How shall we locate the second word on a line?

We could go to the beginning of the line, then call forward-word to skip over the first word. The problem with that solution is that it puts us at the end of the first word, not at the beginning of the second word, which may follow after much whitespace.

We could go to the beginning of the line, then call forward-word twice (actually, we’d call forward-word once, with an argument of 2), then call backward-word, which will put us at the beginning of the second word. That’s fine, but now we realize that the way forward-word and backward-word define a “word” isn’t the same as the definition we need. According to those functions, punctuation (such as a hyphen) separates words, so that (for example) “forward-word” is actually two words. That’s bad for us, because our function needs to count words as separate only when they’re separated by whitespace.

We could go to the beginning of the line, then skip over all non-whitespace characters (the first word), then skip over all whitespace characters (the whitespace after the first word), which will leave us positioned at the second word. That sounds promising; let’s give it a try.

(defun before-2nd-word-p (pos)
  "Does POS lie before the second word on the line?"
  (save-excursion
    (goto-char pos)
    (beginning-of-line)
    (skip-chars-forward "^ ")
    (skip-chars-forward " ")
    (< pos (point))))

The function skip-chars-forward is very useful. It moves the cursor forward until encountering a character either in or not in a set of characters you specify. The set of characters works exactly like the inside of a square-bracketed regular expression (see regular expression rule 3 in the section Regular Expressions in Chapter 4). So

(skip-chars-forward "^ ")

means “skip over characters that aren’t a space,” while

(skip-chars-forward " ")

means “skip over spaces.”

One problem with this approach is that if the line has no spaces,

(skip-chars-forward "^ ")

will skip right on to the next line! We don’t want that. So let’s make sure we don’t skip too far by adding a newline (written " " in strings) to the first skip-chars-forward:

(skip-chars-forward "^ 
")      ;skip to first space or newline

The next problem is that a tab character (" " in strings) may be used to separate words just like spaces. So we must modify our two skip-chars-forward calls like so:

(skip-chars-forward "^ 	
")
(skip-chars-forward " 	")

Are there other characters like space and tab that are considered whitespace? Possibly. The formfeed character (ASCII 12) is usually considered to be whitespace. And if the buffer is using some character set other than ASCII, there may be other characters that are word-separating whitespace. For example, in the 8-bit character set known as Latin-1, character numbers 32 and 160 are both spaces—though 160 is a “non-breaking space” which means lines should not be broken there.

Rather than worry about these details, why not let Emacs worry about them? This is where syntax tables come in handy. A syntax table is a mode-specific mapping from characters to “syntax classes.” Classes include “word constituent” (usually letters and apostrophes and sometimes digits), “balanced brackets” (usually pairs like (), [], {}, and sometimes <>), “comment delimiters” (which are ; and newline for Lisp mode, /* and */ for C mode), “punctuation,” and of course, “whitespace.”

The syntax table is used by commands like forward-word and backward-word to figure out just what a word is. Because different buffers can have different syntax tables, the definition of a word can vary from one buffer to another. We’re going to use the syntax table to figure out which characters are to be considered whitespace in the current buffer.

All we need to do is replace our two calls to skip-chars-forward with two calls to skip-syntax-forward like so:

(skip-syntax-forward "^ ")
(skip-syntax-forward " ")

For each syntax class, there’s a code letter.^[30] Space is the code letter meaning “whitespace,” so the two lines above mean “skip all non-whitespace” and “skip all whitespace.”

Unfortunately, we again have the problem that our first call to skip-syntax-forward might traverse to the next line. Worse, this time we can’t simply add to skip-syntax-forward’s argument, because isn’t the code letter for the syntax of newline characters. In fact, the code letter for the syntax of newline characters will be different in different buffers.

What we can do is ask Emacs to tell us the code letter for the syntax of newline characters, then use that result to construct the argument to skip-syntax-forward:

(skip-syntax-forward (concat "^ "
                             (char-to-string
                              (char-syntax ?
))))

The function char-syntax returns a character’s syntax code as another character. That’s then converted to a string with char-to-string and appended to "^ ".

Here’s the final form of before-2nd-word-p:

(defun before-2nd-word-p (pos)
  "Does POS lie before the second word on the line?"
  (save-excursion
    (goto-char pos)
    (beginning-of-line)
    (skip-syntax-forward (concat "^ "
                                 (char-to-string
                                  (char-syntax ?
))))
    (skip-syntax-forward " ")
    (< pos (point))))

Bear in mind that the cost of computing before-2nd-word-p might outweigh the benefit it’s meant to provide (i.e., avoiding the calls to end-of-line and backward-paragraph in refill). If you’re interested, you can try using the profiler (see Appendix C) to see which version of refill is faster, the one with a call to before-2nd-word-p or the one without.

We needn’t refill the paragraph every time an insertion occurs. A small insertion that doesn’t push any text beyond the right margin doesn’t affect any line but its own, so if the current change is an insertion, and start and end are on the same line, and the end of the line isn’t beyond the right margin, let’s not call fill-region at all.

This means we must surround our call to fill-region with an if that looks something like this:

(if (and (zerop len)                        ;if it's an insertion...
         (same-line-p start end)            ;...that doesn't span lines...
         (short-line-p end))                ;...and the line's still short
    nil                                     ;then do nothing
  (save-excursion
    (fill-region ...)))                     ;otherwise, refill

We must now define same-line-p and short-line-p.

Writing same-line-p should be easy by now. We simply test whether end falls between start and the end of the line.

(defun same-line-p (start end)
  "Are START and END on the same line?"
  (save-excursion
    (goto-char start)
    (end-of-line)
    (<= end (point))))

Writing short-line-p is similarly straightforward. The variable controlling the “right margin” is called fill-column, and current-column returns the horizontal position of point.

(defun short-line-p (pos)
  "Does line containing POS stay within 'fill-column'?"
  (save-excursion
    (goto-char pos)
    (end-of-line)
    (<= (current-column) fill-column)))

Here’s the new definition of refill:

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (let ((left (if (or (zerop len)
                      (not (before-2nd-word-p start)))
                  start
                (save-excursion
                  (max (progn
                         (goto-char start)
                         (beginning-of-line 0)
                         (point))
                       (progn
                         (goto-char start)
                         (backward-paragraph 1)
                         (point)))))))
    (if (and (zerop len)
             (same-line-p start end)
             (short-line-p end))
        nil
      (save-excursion
        (fill-region left end nil nil t)))))

We still haven’t dealt with the problem that fill-region deletes trailing whitespace from each line, particularly the one you’re editing, requiring you to type words likethis, then back up and insert a space!

Our strategy will be to avoid refilling altogether whenever the cursor follows whitespace at the end of a line, or if the cursor is in whitespace at the end of a line. This condition can be expressed by

(and (eq (char-syntax (preceding-char))
         ? )
     (looking-at "\s *$"))

which is true when the character preceding the cursor is whitespace and when nothing but whitespace follows the cursor on the line. Let’s take a closer look at this.

First we compute (char-syntax (preceding-char)), which gives the syntax class of the character preceding the cursor, and compare it with '?'. That strange construct—question mark, backslash, space—is the Emacs Lisp way of writing a space character. Recall that the space character is the code letter for the “whitespace” syntax class, so this test tells whether the preceding character is whitespace.

Next we call looking-at, a function that tells whether the text following the cursor matches a given regular expression. The regexp in this case is s *$ (remember, backslashes get doubled in Lisp strings). In Emacs Lisp regexps, s introduces a syntax class based on the current buffer’s syntax table. The character following s tells which syntax class to use. In this case, it’s space, meaning “whitespace.” So 's ' is a regexp meaning “match a character of whitespace,” and s *$ means “match zero or more whitespace characters, followed by end of line.”

Our final version of refill includes this new test.

(defun refill (start end len)
  "After a text change, refill the current paragraph."
  (let ((left (if (or (zerop len)
                      (not (before-2nd-word-p start)))
                  start
                (save-excursion
                  (max (progn
                         (goto-char start)
                         (beginning-of-line 0)
                         (point))
                       (progn
                         (goto-char start)
                         (backward-paragraph 1)
                         (point)))))))
    (if (or (and (zerop len)
                 (same-line-p start end)
                 (short-line-p end))
            (and (eq (char-syntax (preceding-char))
                     ? )
                 (looking-at "\s *$")))
        nil
      (save-excursion
        (fill-region left end nil nil t)))))

For performance reasons, it’s normally a good idea to avoid putting functions, especially complicated ones like refill, in after-change-hooks. If your computer is fast enough, you may not notice the cost of executing this function on every keypress; otherwise, you might find it makes Emacs unusably sluggish. In the next chapter, we’ll examine a way to speed it up.