Chapter 4: Talking with and Writing to Your Macintosh

In This Chapter

Using handwriting recognition to control OS X

Speaking to your Mac

Having your Mac speak back

Exploring assistive capabilities and devices

Using VoiceOver to provide feedback in Mountain Lion

If you’re a hunt-and-peck typist — leaving you certain that there must be a better way to get information into your computer — Apple has you in mind. Ever since the very first Mac rolled off the assembly line, Apple has had a keen interest in alternative modes of interaction between human and machine. OS X continues this tradition by offering three options for controlling OS X without the keyboard:

Handwriting: By using a stylus and computer tablet, you can enter text into your Mac by simply writing as you would on a sheet of paper.

Speech: Talk to your Mac to make it obey your commands. It even talks back!

Assistive devices: Use a number of third-party devices that can help disabled Mac owners operate their computers, such as a head-mounted eye tracker.

This chapter guides you through the various options for controlling your Macintosh without using the keyboard. First, I cover the OS X Ink feature (also called Inkwell), which you use to write on a tablet to enter data into your computer. (Although it sounds a bit ironic, think of Ink as your “digital paper” for the new millennium.) Whatever you write on the tablet appears onscreen as text.

I also take a look at the more space-age speech capabilities available in OS X. With your voice, you can command a Macintosh to perform all sorts of interesting feats. And just so you don’t get lonely, the Mac even talks back to you. (Now you can control your computer just like Spock from Star Trek!) And Mountain Lion’s VoiceOver feature makes it easy for your Mac to read aloud all sorts of text, including web pages, Mail messages, and word processing documents.

So scoot away from your computer, lean that chair back, and let your Mac take care of the rest.

Using Ink with a Tablet

Typing on a keyboard can be tedious and error-prone for even the best typists. To help out, Apple includes some useful handwriting features in OS X. Based in part on some of Apple’s handheld software for its ill-fated Newton (one of the first personal digital assistants, released before its time), the handwriting recognition in OS X gives you the ability to write text on a compatible tablet in your favorite applications.

The basic process of working with handwriting in OS X goes like this:

1. Attach a tablet to your Mac.

Most tablets use a Universal Serial Bus (USB) connection, so connecting one to your computer is as simple as plugging in the cable from your tablet to the USB port on your Mac.

2. Open the desired document on your Mac.

3. Write on the tablet with the stylus that accompanies it.

A stylus is the “pen” that accompanies most tablets. A stylus doesn’t have any ink in it: It’s just a pen-shaped tool with a plastic tip meant for writing on a tablet.

Your Mac interprets your handwriting and places that text (at the cursor, where you would typically type with the keyboard) in the active application. You’re spared the whole training bit, too.

You aren’t restricted to writing just text on the tablet, naturally. You can use your tablet to control the interface of your Mac as you would a mouse. A tablet also works great for graphics applications, such as Corel Painter, Adobe Illustrator, and Photoshop and Photoshop Elements from Adobe. Many artists are frustrated when drawing with a mouse; when you use a tablet, though, you can feel right at home with natural pen or brush movements.

Mountain Lion also offers a few settings in the Ink pane within System Preferences (accessible from the Dock). From there, click the Ink icon to adjust settings for your tablet.

If you don’t have a tablet connected to your Macintosh, you can’t view the System Preferences pane for Ink. OS X is smart enough to show you only the settings for your current hardware setup.

Computer, Can You Hear Me?

Remember that classic scene from the movie Star Trek IV: The Voyage Home in which Scotty picks up the mouse on a Macintosh and tries to talk directly to the computer? (We’re not to that point yet, Scotty, but we’re working on it.) Since the early days of the Mac Quadra computer line, Apple has included some form of speech recognition in its computers. Mountain Lion continues to improve on speech recognition by offering a host of tools that let you get more work done in a shorter amount of time as well as provide voice control for those Mac owners who have difficulty or are unable to use a traditional keyboard and mouse.

Speak a word, phrase, or sentence, and your Mac translates what you said — and if it understands the phrase, it then performs an action associated with that phrase. The great part about this system is that you can say any phrase in continuous speech and have your Mac perform any action that you can imagine. In fact, you aren’t limited to just one action: You can perform dozens of actions upon speaking a particular phrase.

Speakable Items are different from Mountain Lion’s Dictation feature, which simply types what you speak into an application. (I discuss Dictation later in the sidebar, “Mac, take a memo!”)

Before you start using Speakable Items, make sure that you have a microphone with which to import sound into your Mac. Many current Macintosh models have a built-in microphone. An Intel iMac sports a microphone built into the monitor. MacBooks have a similar microphone built into the screen. If your Mac doesn’t have a microphone, though, you can easily connect one via the microphone jack. (Apple’s line of Cinema LED displays also includes a built-in camera and microphone.)

If you’re looking for the best quality audio input from your microphone for use with speech recognition (as well as Messages, which I cover in Book V, Chapter 3), check out a microphone with a USB connection. You’ll get far better sound quality than afforded by either your Mac’s built-in microphone or a microphone that connects to your audio jack.

The Speakable Items controls

To get started with speech recognition in OS X, open the System Preferences window by clicking its icon on the Dock and then clicking the Accessibility icon. Click the Speakable Items entry in the list to the left to display the settings shown in Figure 4-1.

Two different panes in System Preferences make up the speech settings of OS X:

Speakable Items (appears in the Accessibility pane)

Text to Speech (appears in the Dictation & Speech pane)

In this section, I’m concerned only with the Speakable Items settings. Later, in the “Your Mac Talks Back!” section, I explore the Text to Speech settings.

The Speakable Items display in the Accessibility pane consists of three subtabs:

Settings: Here you find a number of settings that control how your Mac listens to Its Master’s Voice (meaning you, friendly reader). From here, you can toggle Speakable Items on and off, set the sound input, change microphone settings, and specify the action that your Mac should take when a command is recognized.

Listening Key: On this tab, you can adjust the key on the keyboard that toggles speech recognition on and off (the listening key), specify when your Mac should listen for commands, and name your computer with a keyword. (You do want to call your computer by name as any techno-wizard does, don’t you?)

Commands: When Speakable Items are active, your Mac can understand any number of commands. From the Commands tab, you tell the Mac what type of command it should expect you to give. You can configure any number of specific applications and menus with speakable items, such as contact names within Mountain Lion’s Contacts application.

The Settings tab

Crowning the Settings tab are the Speakable Items On and Off radio buttons. You’ve probably already guessed how to use ’em to switch the Speakable Items feature on and off.

When you select the On radio button, the small circular Speech Recognition Feedback window appears on your screen, floating above all other windows. Know this face well because the Feedback window (shown in Figure 4-1) is your friend and partner. If you use Speakable Items often, it’ll become a constant companion on your Desktop (more on it in the next section).

You can select the microphone you want to use from the Microphone pop-up menu — a great feature if you have more than one microphone connected to your Mac. Click the Calibrate button to adjust the sound volume for better recognition.

At the bottom of the Settings tab is the Upon Recognition section. When your Mac comprehends one of your stentorian commands, you can set it to respond by playing a sound, speaking an acknowledgement, or both. This is helpful when you’re not sure whether your Mac understands you. Like handwriting recognition, 100 percent speech recognition isn’t a reality on any computer at this point, so sometimes it helps to have feedback. Otherwise, you might feel silly shouting at your machine while it sits there doing nothing. (Or perhaps not if you’re into inexpensive anger management.)

The Listening Key tab

You can choose between two styles of listening with the Listening Method options:

Listen Only While Key Is Pressed: Speakable Items works only while the designated key is held down.

Listen Continuously with Keyword: When you speak the keyword, listening turns on and remains on.

To change what key must be toggled or held down, click the Change Key button.

Why change the keyword? Instead of saying, “Computer, empty the Trash!” you might prefer, “Elrond, empty the Trash!” This adds a little bit of personality to the interaction and also gives your computer a slightly longer time to react to your command. (As a general rule, the longer the spoken phrase, the more likely your Mac will understand it.) If you select the Listen Continuously with Keyword feature, you can change your computer’s name via the Keyword text box.

The Commands tab

When Speakable Items are active, your Mac listens for whatever phrases appear in your Speakable Items folder (a directory on your hard drive that holds a number of scripts). The Commands tab — shown in Figure 4-2 — allows you to view the contents of this folder. When you speak a phrase that matches one of these filenames, your Mac automatically executes that script. The script can perform any number of actions, which is what makes Speakable Items so powerful. Apple includes a large number of scripts with OS X, but you’re free to create your own, too.

To make something speakable, select the item and then speak the command, “Make this speakable.” The new speakable command is based upon the item’s name.

To view the contents of the Speakable Items folder, click the Open Speakable Items Folder button on the Commands tab. The Finder comes to the foreground and navigates to the folder that holds the scripts.

To the right of the Open Speakable Items Folder button is the Helpful Tips button. Click it to get some pointers on how to get the best performance from your microphone.

The Speakable Items feature isn’t restricted to items in the Speakable Items folder. Any application that supports Speakable Items is also fair game for your verbal manipulation. To control commands within other applications, use the Commands tab (refer to Figure 4-2). Here you can select the following check boxes:

Contacts

Global Speakable Items

Application Specific Items

Application Switching

Front Window (requires that you activate assistive devices at the bottom of the Accessibility pane)

Menu Bar (requires that you activate assistive devices at the bottom of the Accessibility pane)

Select any of these options to allow your Mac to listen to those kinds of commands.

The Feedback window

After you activate Speakable Items, you instantly see the Feedback window. You can click and drag the edge of the window to position it anywhere on your Desktop.

The Feedback window includes controls and displays of its own:

Microphone Level Meter: View indicators to let you know how loud the input level to your microphone is.

Visual Indicator: See visual feedback to let you know the active mode: idle, listening, or hearing a command. When the microphone isn’t grayed out but there are no arrows on either side of the microphone, you’re in listening mode. When the microphone is flanked by animated arrows, your computer is hearing a command spoken. When Speech Recognition is idle, no arrows are present, and the microphone is grayed out.

Quick Access Menu: You can quickly view the Speech Commands window. Just click the downward-pointing arrow at the bottom of the Feedback window, and a menu appears.

As soon as you disable Speakable Items, the Feedback window disappears.

The Speech Commands window

Because Speakable Items might be listening for different sets of commands from the Finder or many other applications, OS X provides you with a single listing of all commands that you might speak at any given time: the Speech Commands window. To open the Speech Commands window, click the triangle at the bottom of the Feedback window and choose Open Speech Commands Window from the menu that appears.

The Speech Commands window is a simple one, but it serves an important purpose: to let you know what commands OS X understands. The Speech Commands pane, as shown in Figure 4-3, organizes commands into categories that match the settings in the Commands tab.

If you launch another application that supports Speakable Items, OS X adds that application’s commands to the Speech Commands window. Speak any of these commands to make your Mac execute that function. For example, OS X ships with speech commands for Contacts, such as Mail To and Meet With.

Apple might be a big, serious, computer company — yeah, right — but it isn’t without a humorous side! With Speakable Items enabled, say the phrase, “Tell me a joke.” Your Mac replies with a random joke. Say it again, and your Mac tells you another joke. (Brace yourself, these jokes were likely written by preschoolers . . . they’re really, really bad.) Oh, and if you get a “Knock, Knock” joke, remember that you have to actually say, “Who’s there?”

Your Mac Talks Back!

OS X is great at listening to your speech, but the fun doesn’t stop there. Your Mac can talk to you, too! Choose from one of many available voices — including the default Mountain Lion voice, Alex — you can make your computer talk or even sing (not as well as Sinatra, but better than Bob Dylan). And whereas Speakable Items lets you speak to your Macintosh, the VoiceOver feature enables your Mac to speak text. This is an especially useful feature for listening to your e-mails, web pages, or even your homework — sometimes the eyes need a break. The text-to-speech capability gives you the opportunity to lean back in your chair or even get up and walk around while still using your Macintosh.

Text-to-speech settings appear in three places within System Preferences: the Dictation & Speech pane, the Date & Time pane, and the VoiceOver settings within the Accessibility pane.

Setting Text to Speech options

The text-to-speech engine that comes with Mountain Lion has a collection of many different voices to choose among, including male, female, and (um) nonhuman. To select your Mac’s voice in OS X, follow these steps:

1. Click the System Preferences icon on the Dock.

2. Click the Dictation & Speech icon.

3. Click the Text to Speech tab, as shown in Figure 4-4.

4. Open the System Voice pop-up menu to choose a voice for your Mac.

To hear the voice, click Play; Mountain Lion speaks a sentence as a demonstration. To the left of the Play button is a slider for adjusting the speed of the speech. Move the slider right to increase the speed and left to slow it down.

Naturally, Alex receives the most attention these days because he’s the default voice for Mountain Lion, and Alex definitely provides the most natural-sounding tone and the best pacing of the bunch. However, I also think that the old-school voices, like Bruce and Vicki, are still quite intelligible. (I’m a Vicki kinda guy, myself.)

With your voice selected, turn your attention to enabling talking alerts and having your Mac speak selected text at the press of a key.

Talking alerts

You have these options for setting talking alerts:

Announce when an Alert dialog pops up

Announce when an application needs your attention

Announce the time (more on that in an upcoming section)

You enable either of the first two check box options (or both) on the Text to Speech tab; refer to Figure 4-4.

Alert dialogs come in two levels of importance, but the key word here is “important”: You don’t want to miss these messages:

Stop sign: Something particularly important requires your attention — usually an error or a dire warning.

Yield sign: Not as severe as the Stop sign, but important nonetheless. Proceed cautiously.

Note that these two traditional types of alerts are different from Notification Center alerts, which are new in Mountain Lion. Notification Center alerts are configured within the Notification pane in System Preferences.

If you select the Announce When Alerts Are Displayed check box, then click the Set Alert Options button to set alert-specific options:

Voice: Choose the voice that should speak alerts. (By default, it’s any voice you opted for earlier.)

Phrase: Click this pop-up menu to choose a “prefix” phrase that’s spoken before the actual alert text. By default, Mountain Lion speaks the name of the application that displayed the alert, but you can also choose a phrase from the list or choose to mix up the phrases for a little variety. To add or remove phrases, click the Edit Phrases item.

Delay: Drag this slider to control the time that passes before Mountain Lion speaks.

To hear what your spoken alert settings sound like, click the Play button.

If you click the Play button on the Set Alert Options sheet and don’t hear anything right away, remember that it doesn’t begin speaking until the time you set with the Delay slider has elapsed. To hear the spoken alerts speak as soon as you click the Play button, move the slider to 0 (zero).

In addition to spoken alerts, you can allow your Mac to speak when other actions occur. Again, these settings are found on the Text to Speech tab. Your Mac can speak in the following circumstances:

When an application requires your attention: If your Dock is hidden, you might not know when an application needs your attention, but you can tell Mountain Lion to grab your attention via speech. Enable this by selecting the Announce When an Application Requires Your Attention check box.

Announce the time: Are you, like me, constantly getting lost in time while immersed in your work? (Not to mention your favorite game.) I’ve set my MacBook Pro to announce the hour, which always keeps my time sense firmly planted. More on this in “The Date & Time pane.”

Have your Mac read to you

Ready for your Mac to read text aloud? Although perhaps not as soothing as a parent’s gentle voice, your Mac can indeed read any text you can select with the cursor, in any application — that includes Safari, Preview, Notes and even third-party applications like Microsoft Word.

If you have a child who’s learning to read, the Mac can help her by reading a selection of text. (Often, kids can figure out how to drag and select text more quickly than they can read the text itself.) And for those with less-than-perfect eyesight, selecting a large amount of text and having it read aloud can save both headaches and eye strain.

To read a selection of text when you press a particular key, enable the Speak Selected Text When the Key Is Pressed check box. By default, the keyboard shortcut to read text is Option+Esc, but you can click the Change Key button to assign your own shortcut.

The Date & Time pane

There you are, deep in concentration as you finish the final chapter of your Great American Novel, when you glance at the clock in the Finder menu bar and realize that you were supposed to pick up your kids at soccer practice an hour ago!

In the future, you can avoid this shameful lapse of parental responsibility by turning on Mountain Lion’s automatic spoken time feature, which you control from the Date & Time pane (in System Preferences). Click the Open Date & Time Preferences button on the Text to Speech tab, and Mountain Lion immediately switches to display the Date & Time pane. (Or, click the Clock on the Finder menu bar and then click Open Date & Time. Again, it’s all about the convenience when it comes to the geniuses at Apple.)

From the Date & Time pane, click the Clock tab. Select the Announce the Time check box to enable your Mac to speak the time; use the Period pop-up menu to choose spoken time at the quarter, half, or full hour. You can also customize the voice for spoken time as well.

After you’re done, zip back to the Speech pane by clicking the Back button at the top-left corner of the System Preferences window.

Configuring VoiceOver within the Accessibility pane

With Mountain Lion’s VoiceOver utility, your Mac can give you all sorts of verbal feedback, creating a spoken English interface with Mountain Lion — a valuable addition to the OS for the physically impaired. The feedback includes

Announcing when certain keys are pressed: Tells you when a modifier key (such as Control, Option, or ) is pressed or when the Caps Lock key is pressed.

Announcing cursor movements: You hear an audible alert when your mouse cursor switches among windows or when you click a menu.

Announcing the position of the VoiceOver cursor: VoiceOver can audibly identify all OS controls (such as buttons, sliders, and list boxes) by using a special onscreen cursor.

Reading documents, web pages, and Mail messages: VoiceOver can read aloud the contents of all sorts of documents and application windows.

Speaking the characters you type: You can set VoiceOver to speak every character or each word you type.

Here is how to enable VoiceOver (or to launch the VoiceOver utility, as shown in Figure 4-5):

1. Click the Open Accessibility Preferences button on the Text to Speech panel (refer to Figure 4-4).

2. Click the VoiceOver item in the list at the left.

3. Select the On radio button (or press +F5) to enable VoiceOver. (If you’re using one of the latest Apple keyboards, press +Fn+F5 instead.)

To customize how VoiceOver operates, click the Open VoiceOver Utility button.

Speaking text through applications

Although VoiceOver provides a comprehensive text-to-speech interface for OS X, it might be more than you need. If you simply want to hear text spoken within your applications, a number of alternative methods are available in Mountain Lion that don’t require VoiceOver. As I mention earlier, your Mac can read selected text: Enable the Speak Selected Text When the Key Is Pressed check box on the Text to Speech tab, select the desired text, and press the keyboard shortcut.

Ah, but what if you don’t want to select a huge text document? Another way to hear spoken text in OS X is by using TextEdit, the simple text processor that accompanies every copy of OS X. Besides its handy word processing features, TextEdit can also speak text, which can be a helpful document review tool. To hear spoken text with TextEdit, follow these steps:

1. Click the Launchpad icon on the Dock and click the TextEdit icon.

2. Select the desired text document and click Open.

3. Choose Edit⇒Speech⇒Start Speaking.

Your Mac begins speaking the text from the document. The speech engine has some intelligence, so you can enter dollar amounts (such as $25,423.12) or Roman numerals (such as Chapter XIV), and the speech engine reads them back in plain English. The result of these two strings would be “twenty-five thousand, four-hundred twenty-three dollars, and twelve cents” and “chapter fourteen.”

4. Choose Edit⇒Speech⇒Stop Speaking.

Your Mac stops speaking. It also stops speaking when it reaches the end of the text.