Reading Word Lists

For the exercises in this chapter we need a list of English words. There are lots of word lists available on the web, but the one most suitable for our purpose is one of the word lists collected and contributed to the public domain by Grady Ward as part of the Moby lexicon project. It is a list of 113,809 official crosswords; that is, words that are considered valid in crossword puzzles and other word games. In the Moby collection, the filename is 113809of.fic; you can download a copy, with the simpler name words.txt, from this book’s GitHub repository.

This file is in plain text, so you can open it with a text editor, but you can also read it from Julia. The built-in function open takes a name of the file as a parameter and returns a file stream you can use to read the file:

julia> fin = open("words.txt")
IOStream(<file words.txt>)

fin is a file stream used for input. When it is no longer needed, it has to be closed with close(fin).

Julia provides several function for reading, including readline, which reads characters from the file until it gets to a NEWLINE and returns the result as a string:

julia> readline(fin)
"aa"

The first word in this particular list is “aa,” which is a kind of lava.

The file stream keeps track of where it is in the file, so if you call readline again, you get the next word:

julia> readline(fin)
"aah"

The next word is “aah,” which is a perfectly legitimate word, so stop looking at me like that.

You can also use a file as part of a for loop. This program reads words.txt and prints each word, one per line:

for line in eachline("words.txt")
    println(line)
end

Exercises

Search

All of the exercises in the previous section have something in common; they can be solved with the search pattern. The simplest example is:

function hasno_e(word)
    for letter in word
        if letter == 'e'
            return false
        end
    end
    true
end

The for loop traverses the characters in word. If we find the letter e, we can immediately return false; otherwise, we have to go to the next letter. If we exit the loop normally, that means we didn’t find an e, so we return true.

You could write this function more concisely using the ∉ ( otin TAB) operator, but I started with this version because it demonstrates the logic of the search pattern.

avoids is a more general version of hasno_e, but it has the same structure:

function avoids(word, forbidden)
    for letter in word
        if letter ∈ forbidden
            return false
        end
    end
    true
end

We can return false as soon as we find a forbidden letter; if we get to the end of the loop, we return true.

usesonly is similar except that the sense of the condition is reversed:

function usesonly(word, available)
    for letter in word
        if letter ∉ available
            return false
        end
    end
    true
end

Instead of an array of forbidden letters, we have an array of available letters. If we find a letter in word that is not in available, we can return false.

usesall is similar except that we reverse the role of the word and the string of letters:

function usesall(word, required)
    for letter in required
        if letter ∉ word
            return false
        end
    end
    true
end

Instead of traversing the letters in word, the loop traverses the required letters. If any of the required letters do not appear in word, we can return false.

If you were really thinking like a computer scientist, you would have recognized that usesall was an instance of a previously solved problem, and you would have written:

function usesall(word, required)
    usesonly(required, word)
end

This is an example of a program development plan called reduction to a previously solved problem, which means that you recognize the problem you are working on as an instance of a solved problem and apply an existing solution.

Looping with Indices

I wrote the functions in the previous section with for loops because I only needed the characters in the strings; I didn’t have to do anything with the indices.

For isabecedarian we have to compare adjacent letters, which is a little tricky with a for loop:

function isabecedarian(word)
    i = firstindex(word)
    previous = word[i]
    j = nextind(word, i)
    for c in word[j:end]
        if c < previous
            return false
        end
        previous = c
    end
    true
end

An alternative is to use recursion:

function isabecedarian(word)
    if length(word) <= 1
        return true
    end
    i = firstindex(word)
    j = nextind(word, i)
    if word[i] > word[j]
        return false
    end
    isabecedarian(word[j:end])
end

Another option is to use a while loop:

function isabecedarian(word)
    i = firstindex(word)
    j = nextind(word, 1)
    while j <= sizeof(word)
        if word[j] < word[i]
            return false
        end
        i = j
        j = nextind(word, i)
    end
    true
end

The loop starts at i=1 and j=nextind(word, 1) and ends when j>sizeof(word). Each time through the loop, it compares the ith character (which you can think of as the current character) to the jth character (which you can think of as the next).

If the next character is less than (alphabetically before) the current one, then we have discovered a break in the abecedarian trend, and we return false.

If we get to the end of the loop without finding a fault, then the word passes the test. To convince yourself that the loop ends correctly, consider an example like "flossy".

Here is a version of ispalindrome that uses two indices; one starts at the beginning and goes up, and the other starts at the end and goes down:

function ispalindrome(word)
    i = firstindex(word)
    j = lastindex(word)
    while i<j
        if word[i] != word[j]
            return false
        end
        i = nextind(word, i)
        j = prevind(word, j)
    end
    true
end

Or we could reduce to a previously solved problem and write

function ispalindrome(word)
    isreverse(word, word)
end

using isreverse from “Debugging”.

Debugging

Testing programs is hard. The functions in this chapter are relatively easy to test because you can check the results by hand. Even so, it is somewhere between difficult and impossible to choose a set of words that tests for all possible errors.

Taking hasno_e as an example, there are two obvious cases to check: words that have an e should return false, and words that don’t should return true. You should have no trouble coming up with one of each.

Within each case, there are some less obvious subcases. Among the words that have an e, you should test words with an e at the beginning, the end, and somewhere in the middle. You should test long words, short words, and very short words, like the empty string. The empty string is an example of a special case, which is one of the nonobvious cases where errors often lurk.

In addition to the test cases you generate, you can also test your program with a word list like words.txt. By scanning the output, you might be able to catch errors, but be careful: you might catch one kind of error (words that should not be included, but are) and not another (words that should be included, but aren’t).

In general, testing can help you find bugs, but it is not easy to generate a good set of test cases, and even if you do, you can’t be sure your program is correct. According to a legendary computer scientist:

Program testing can be used to show the presence of bugs, but never to show their absence!

Edsger W. Dijkstra

Glossary

file stream

A value that represents an open file.

reduction to a previously solved problem

A way of solving a problem by expressing it as an instance of a previously solved problem.

special case

A test case that is atypical or nonobvious (and less likely to be handled correctly).

Exercises