Topics in This Chapter L1
This chapter covers in detail implementing your own operators—methods with the same syntax as the familiar mathematical operators. Operators are often used to build domain-specific languages—minilanguages embedded inside Scala. Implicit conversions (type conversion functions that are applied automatically) are another tool facilitating the creation of domain-specific languages. This chapter also discusses the special methods apply
, update
, and unapply
. We end the chapter with a discussion of dynamic invocations—method calls that can be intercepted at runtime, so that arbitrary actions can occur depending on the method names and arguments.
The key points of this chapter are:
Identifiers contain either alphanumeric or operator characters.
Unary and binary operators are method calls.
Operator precedence depends on the first character, associativity on the last.
The apply
and update
methods are called when evaluating expr(args)
.
Extractors extract tuples or sequences of values from an input.
Types extending the Dynamic
trait can inspect the names of methods and arguments at runtime. L2
The names of variables, functions, classes, and so on are collectively called identifiers. In Scala, you have more choices for forming identifiers than in most other programming languages. Of course, you can follow the time-honored pattern: sequences of alphanumeric characters, starting with an alphabetic character or an underscore, such as input1
or next_token
.
Unicode characters are allowed. For example, quantité
or ποσό are valid identifiers.
In addition, you can use operator characters in identifiers:
The ASCII characters ! # % & * + - / : < = > ? @ ^ | ~
that are not letters, digits, underscore, the . , ;
punctuation marks, parentheses () [] {}
, or quotation marks ’ ` "
.
Unicode mathematical symbols or other symbols from the Unicode categories Sm and So.
For example, **
and √
are valid identifiers. With the definition
val √ = scala.math.sqrt
you can write √(2)
to compute a square root. This may be a good idea, provided one’s programming environment makes it easy to type the symbol.
Note
The identifiers @ # : = _ => <- <: <% >:
⇒ ← are reserved in the specification, and you cannot redefine them.
You can also form identifiers from alphanumerical characters, followed by an underscore, and then a sequence of operator characters, such as
val happy_birthday_!!! = "Bonne anniversaire!!!"
This is probably not a good idea.
Finally, you can include just about any sequence of characters in backquotes. For example,
val `val` = 42
That example is silly, but backquotes can sometimes be an “escape hatch.” For example, in Scala, yield
is a reserved word, but you may need to access a Java method of the same name. Backquotes to the rescue: Thread.`yield`()
.
You can write
a identifier b
where identifier denotes a method with two parameters (one implicit, one explicit). For example, the expression
1 to 10
is actually a method call
1.to(10)
This is called an infix expression because the operator is between the arguments. The operator can contain letters, as in to
, or it can contain operator characters—for example,
1 -> 10
is a method call
1.->(10)
To define an operator in your own class, simply define a method whose name is that of the desired operator. For example, here is a Fraction
class that multiplies two fractions according to the law
(n1 / d1) × (n2 / d2) = (n1n2 / d1d2)
class Fraction(n: Int, d: Int) :
private val num = ...
private val den = ...
...
def *(other: Fraction) = Fraction(num * other.num, den * other.den)
If you want to call a symbolic operator from Java, use the @targetName
annotation to give it an alphanumeric name:
@targetName("multiply") def *(other: Fraction) =
Fraction(num * other.num, den * other.den)
In Scala code, you use f * g
, but in Java, you use f.multiply(g)
.
In order to use a method with an alphanumeric name as an infix operator, use the infix
modifier:
infix def times(other: Fraction) = Fraction(num * other.num, den * other.den)
Now you can call f times g
in Scala.
Note
You can use a method with an alphanumeric name with infix syntax whenever it is followed by an opening brace.
f repeat { "Hello" }
The method need not be declared as infix
.
Caution
It is possible to declare an infix operator with multiple arguments. The +=
operator for mutable collections has such a form:
val smallPrimes = ArrayBuffer[Int]()
smallPrimes += 2 //
Binary infix operator, invokes +=(Int)smallPrimes += (3, 5) //
Multiple arguments, invokes +=(Int, Int, Int*)
This sounded a good idea at the time, but it gives grief with tuples. Consider:
val twinPrimes = ArrayBuffer[(Int, Int)]()
twinPrimes += (11, 13) //
Error
Surely this should add the tuple (11, 13)
to the buffer of tuples, but it triggers the multi-argument infix syntax and fails, since 11
and 13
are not tuples.
At some point, infix operators with multiple arguments may be removed. In the meantime, you have to call
twinPrimes += ((11, 13))
Infix operators are binary operators—they have two parameters. An operator with one parameter is called a unary operator.
The four operators +
, -
, !
, ~
are allowed as prefix operators, appearing before their arguments. They are converted into calls to methods with the name unary_operator
. For example,
-a
means the same as a.unary_-
.
If a unary operator follows its argument, it is a postfix operator. For example, the expression 42 toString
is the same as 42.toString
.
However, postfix operators can lead to parsing errors. For example, the code
val result = 42 toString
println(result)
yields the error message “Recursive value result needs type.” Since parsing precedes type inference and overload resolution, the compiler does not yet know that toString
is a unary method. Instead, the code is parsed as val result = 42.toString(println(result))
.
For that reason, Scala now discourages the use of postfix operators. If you really want to use them, you must use the compiler option -language:postfixOps
or add the clause
import scala.language.postfixOps
An assignment operator has the form operator=
, and the expression
a operator= b
means the same as
a = a operator b
For example, a += b
is equivalent to a = a + b
.
There are a few technical details.
<=
, >=
, and !=
are not assignment operators.
An operator starting with an =
is never an assignment operator (==
, ===
, =/=
, and so on).
If a
has a method called operator=
, then that method is called directly.
When you have two or more operators in a row without parentheses, the ones with higher precedence are executed first. For example, in the expression
1 + 2 * 3
the *
operator is evaluated first.
In most languages, there is a fixed set of operators, and the language standard decrees which have precedence over which. Scala can have arbitrary operators, so it uses a scheme that works for all operators, while also giving the familiar precedence order to the standard ones.
Except for assignment operators, the precedence is determined by the first character of the operator (see Table 11–1).
Table 11–1 Infix Operator Precedence from First Character
Highest precedence: An operator character other than those below |
|
|
|
|
|
|
|
|
A character that is not an operator character |
Lowest precedence: Assignment operators |
Characters in the same row yield operators with the same precedence. For example, +
and ->
have the same precedence.
Postfix operators have lower precedence than infix operators:
a infixOp bpostfixOp
is the same as
(a infixOp b)postfixOp
When you have a sequence of operators of the same precedence, the associativity determines whether they are evaluated left-to-right or right-to-left. For example, in the expression 17 − 2 − 9, one computes (17 − 2) − 9. The − operator is left-associative.
In Scala, all operators are left-associative except for
operators that end in a colon (:
)
assignment operators
In particular, the ::
operator for constructing lists is right-associative. For example,
1 :: 2 :: Nil
means
1 :: (2 :: Nil)
This is as it should be—we first need to form the list containing 2
, and that list becomes the tail of the list whose head is 1
.
A right-associative binary operator is a method of its second argument. For example,
2 :: Nil
means
Nil.::(2)
apply
and update
MethodsScala lets you extend the function call syntax
f(arg1, arg2, ...)
to values other than functions. If f
is not a function or method, then this expression is equivalent to the call
f.apply(arg1, arg2, ...)
unless it occurs to the left of an assignment. The expression
f(arg1, arg2, ...) = value
corresponds to the call
f.update(arg1, arg2, ..., value)
This mechanism is used in arrays and maps. For example,
val scores = scala.collection.mutable.HashMap[String, Int]()
scores("Bob") = 100 //
Calls scores.update("Bob", 100)val bobsScore = scores("Bob") //
Calls scores.apply("Bob")
Note
As you have already seen in Chapter 5, the companion object of every class has an apply
method that calls the primary constructor. For example, Fraction(3, 4)
calls the Fraction.apply
method, which returns new Fraction(3, 4)
.
unapply
Method L2An apply
method takes construction parameters and turns them into an object. An unapply
method does the opposite. It takes an object and extracts values from it—usually the values from which the object was, or could be, constructed.
Consider the Fraction
class from Section 11.2, “Infix Operators,” on page 151. A call such as Fraction(3, 4)
calls the Fraction.apply
method which makes a fraction from a numerator and denominator. An unapply
method does the opposite and extracts the numerator and denominator from a fraction.
One way to invoke an extractor is in a variable declaration. Here is an example:
val Fraction(n, d) = Fraction(3, 4) * Fraction(2, 5)
//
n, d are initialized with the numerator and denominator of the result
This statement declares two variables n
and d
, both of type Int
, and not a Fraction
. The variables are initialized with the values that are extracted from the right-hand side.
More commonly, extractors are invoked in pattern matches such as the following:
val value = f match
case Fraction(a, b) => a.toDouble / b
// a, b are bound to the numerator and denominator
case _ => Double.NaN
The details of implementing unapply
are somewhat tedious. You may want to skip them until after reading Chapter 14.
Since a pattern match can fail. the unapply
method returns an Option
. Upon success, the Option
contains a tuple holding the extracted values. In our case, we return an Option[(Int, Int)]
.
object Fraction :
def unapply(input: Fraction) =
if input.den == 0 then None else Some((input.num, input.den))
This method returns None
when the fraction is malformed (with a zero denominator), indicating no match.
A statement
val Fraction(a, b) = f
leads to the method call
Fraction.unapply(f)
If the method returns None
, a MatchError
is thrown. Otherwise, the variables a
and b
are set to the components of the returned tuple.
Note that neither the Fraction.apply
method nor the Fraction
constructor are called. However, the intent is to initialize a
and b
so that they would yield f
if they were passed to Fraction.apply
. This is sometimes called destructuring. In that sense, unapply
is the inverse of apply
.
It is not a requirement for the apply
and unapply
methods to be inverses of one another. You can use extractors to extract information from an object of any type.
For example, suppose you want to extract first and last names from a string:
val author = "Cay Horstmann"
val Name(first, last) = author //
Calls Name.unapply(author)
Provide an object Name
with an unapply
method that returns an Option[(String, String)]
. If the match succeeds, return a pair with the first and last name. Otherwise, return None
.
object Name :
def unapply(input: String) =
val pos = input.indexOf(" ")
if pos >= 0 then Some((input.substring(0, pos), input.substring(pos + 1)))
else None
Note
In this example, there is no Name
class. The Name
object is an extractor for String
objects.
Note
The unapply
methods in this section return an Option
of a tuple. It is possible to return other types. See Chapter 14 for the details.
unapplySeq
Method L2The unapply
method extracts a fixed number of values. To extract an arbitrary number of values, the method needs to be called unapplySeq
. In the simplest case, the method returns an Option[Seq[T]]
, where T
is the type of the extracted values. For example, a Name
extractor can produce a sequence of the name’s components:
object Name :
def unapplySeq(input: String): Option[Seq[String]] =
if input.strip == "" then None else Some(input.strip.split(",?\s+").toSeq)
Now you can extract any number name components:
val Name(first, middle, last, rest*) = "John D. Rockefeller IV, B.A."
The rest
variable is set to a Seq[String]
.
Caution
Do not supply both an unapply
and an unapplySeq
method with the same parameter types.
unapply
and unapplySeq
Methods L3In Section 11.8, “The unapply
Method,” on page 155, you saw how to implement an unapply
method that returns an Option
of a tuple:
object Fraction :
def unapply(input: Fraction) =
if input.den == 0 then None else Some((input.num, input.den))
But the return type of unapply
is quite a bit more flexible.
You don’t need an Option
if the match never fails.
Instead of an Option
, you can use any type with methods isEmpty
and get
.
Instead of a tuple, you can use any subtype of Product
with methods _1
, _2
, ..., _n
.
To extract a single value, you don’t need a tuple or Product
.
Return a Boolean
to have the match succeed or fail without extracting a value.
In the preceding example, the Fraction.unapply
method has return type Option[(Int, Int)]
. Fractions with zero denominators return None
. To have the match succeed in all cases, simply return a tuple without wrapping it into an Option
:
object Fraction :
def unapply(input: Fraction) = (input.num, input.den)
Here is an example of an extractor that produces a single value:
object Number :
def unapply(input: String): Option[Int] =
try
Some(input.strip.toInt)
catch
case ex: NumberFormatException => None
With this extractor, you can extract a number from a string:
val Number(n) = "1729"
An extractor returning Boolean
tests its input without extracting any value. Here is such a test extactor:
object IsCompound :
def unapply(input: String) = input.contains(" ")
You can use this extractor to add a test to a pattern:
author match
case Name(first, last @ IsCompound()) => ...
// Matches if the last name is compound, such as van der Linden
case Name(first, last) => ...
Finally, the return type of unapplySeq
can be more general than an Option[Seq[T]]
:
An Option
isn’t needed if the match never fails.
Any type with methods isEmpty
and get
can be used instead of Option
.
Any type with methods apply
, drop
, toSeq
, and either length
or lengthCompare
can be used instead of Seq
.
Scala is a strongly typed language that reports type errors at compile time rather than at runtime. If you have an expression x.f(args)
, and your program compiles, then you know for sure that x
has a method f
that can accept the given arguments. However, there are situations where it is desirable to define methods in a running program. This is common with object-relational mappers in dynamic languages such as Ruby or JavaScript. Objects that represent database tables have methods findByName
, findById
, and so on, with the method names matching the table columns. For database entities, the column names can be used to get and set fields, such as person.lastName = "Doe"
.
In Scala, you can do this too. If a type extends the trait scala.Dynamic
, then method calls, getters, and setters are rewritten as calls to special methods that can inspect the name of the original call and the parameters, and then take arbitrary actions.
Note
Dynamic types are an “exotic” feature, and the compiler wants your explicit consent when you implement such a type. You do that by adding the import
statement
import scala.language.dynamics
Users of such types do not need to provide the import
statement.
Here are the details of the rewriting. Consider obj.name
, where obj
belongs to a class that’s a subtype of Dynamic
. Here is what the Scala compiler does with it.
If name
is a known method or field of obj
, it is processed in the usual way.
If obj.name
is followed by (arg1, arg2, ...)
,
If none of the arguments are named (of the form name=arg
), pass the arguments on to applyDynamic
:
obj.applyDynamic("name")(arg1, arg2, ...)
If at least one of the arguments is named, pass the name/value pairs on to applyDynamicNamed:
obj.applyDynamicNamed("name")((name1, arg1), (name2, arg2), ...)
Here, name1
, name2
, and so on are strings with the argument names, or ""
for unnamed arguments.
If obj.name
is to the left of an =
, call
obj.updateDynamic("name")(rightHandSide)
Otherwise call
obj.selectDynamic("sel")
Note
The calls to updateDynamic
, applyDynamic
, and applyDynamicNamed
are “curried”—they have two sets of parentheses, one for the selector name and one for the arguments. This construct is explained in Chapter 12.
Let’s look at a few examples. Suppose person
is an instance of a type extending Dynamic
. A statement
person.lastName = "Doe"
is replaced with a call
person.updateDynamic("lastName")("Doe")
The Person
class must have such a method:
class Person :
...
def updateDynamic(field: String)(newValue: String) = ...
It is then up to you to implement the updateDynamic
method. For example, if you are implementing an object-relational mapper, you might update the cached entity and mark it as changed, so that it can be persisted in the database.
Conversely, a statement
val name = person.lastName
turns into
val name = name.selectDynamic("lastName")
The selectDynamic
method would simply look up the field value.
Method calls are translated to calls of the applyDynamic
or applyDynamicNamed
method. The latter is used for calls with named parameters. For example,
val does = people.findByLastName("Doe")
becomes
val does = people.applyDynamic("findByLastName")("Doe")
and
val johnDoes = people.find(lastName = "Doe", firstName = "John")
becomes
val johnDoes =
people.applyDynamicNamed("find")(("lastName", "Doe"), ("firstName", "John"))
It is then up to you to implement applyDynamic
and applyDynamicNamed
as calls that retrieve the matching objects.
Here is a concrete example. Suppose we want to be able to dynamically look up and set elements of a java.util.Properties
instance, using the dot notation:
val sysProps = DynamicProps(System.getProperties)
sysProps.username = "Fred" //
Sets the "username" property to "Fred"val home = sysProps.java_home //
Gets the "java.home" property
For simplicity, we replace periods in the property name with underscores. (Exercise 13 on page 165 shows how to keep the periods.)
The DynamicProps
class extends the Dynamic
trait and implements the updateDynamic
and selectDynamic
methods:
class DynamicProps(val props: java.util.Properties) extends Dynamic :
def updateDynamic(name: String)(value: String) =
props.setProperty(name.replaceAll("_", "."), value)
def selectDynamic(name: String) =
props.getProperty(name.replaceAll("_", "."))
As an additional enhancement, let us use the add
method to add key/value pairs in bulk, using named arguments:
sysProps.add(username="Fred", password="Secret")
Then we need to supply the applyDynamicNamed
method in the DynamicProps
class. Note that the name of the method is fixed. We are only interested in arbitrary parameter names.
def applyDynamicNamed(name: String)(args: (String, String)*) =
if name != "add" then throw IllegalArgumentException()
for (k, v) <- args do
props.setProperty(k.replaceAll("_", "."), v)
These examples are only meant to illustrate the mechanism. Is it really that useful to use the dot notation for map access? Like operator overloading, dynamic invocation is a feature that is best used with restraint.
In the preceding section, you saw how to resolve selections obj.selector
and method calls obj.method(args)
dynamically. However, that approach is not typesafe. If selector
or method
are not appropriate, a runtime error occurs. In this section, you will learn how to detect invalid selections and invocations at compile time.
Instead of the Dynamic
trait, you use the Selectable
trait. It has methods
def selectDynamic(name: String): Any
def applyDynamic(name: String)(args: Any*): Any
The key difference is that you specify the selectors and methods that can be applied.
Let us first look at selection. Suppose we have objects with properties from a cache of database records, or JSON values, or, to keep the example simple, from a map. Then we can select a property with this class:
class Props(props: Map[String, Any]) extends Selectable :
def selectDynamic(name: String) = props(name)
Now let’s move on to the typesafe part. Suppose we want to work with invoice items. Define a type with the valid properties, like this:
type Item = Props {
val description: String
val price: Double
}
This is an example of a structural type—see Chapter 18 for details.
Construct instances as follows:
val toaster = Props(
Map("description" -> "Blackwell Toaster", "price" -> 29.95)).asInstanceOf[Item]
When you call
toaster.price
the selectDynamic
method is invoked. This is no different from that in the preceding section.
However, a call toaster.brand
is a compile-time error since brand
is not one of the listed selectors in the Item
type.
Next, let’s turn to methods. We want to write a library for making REST calls with Scala syntax. Calls such as
val buyer = myShoppingService.customer(id)
shoppingCart += myShoppingService.item(id)
should, behind the scenes, invoke REST requests http://myserver.com/customer/id
and http://myserver.com/item/id
, yielding the JSON responses.
And we want this to be typesafe. A call to a nonexistent REST service should fail at compile time.
First, make a generic class
class Request(baseURL: String) extends Selectable :
def applyDynamic(name: String)(args: Any*): Any =
val url = s"$baseURL/$name/${args(0)}"
scala.io.Source.fromURL(url).mkString
Then constrain to the services that you know to be actually supported.
I don’t have access to a shopping service, but I have a service that produces random nouns and adjectives:
type RandomService = Request {
def nouns(qty: Int) : String
def adjectives(qty: Int) : String
}
Construct an instance:
val myRandomService =
new Request("https://horstmann.com/random").asInstanceOf[RandomService]
Now a call
myRandomService.nouns(5)
is translated to a call
myRandomService.applyDynamic("nouns")(5)
However, if the method name is neither nouns
nor adjectives
, a compiler error occurs.
1. According to the precedence rules, how are 3 + 4 -> 5
and 3 -> 4 + 5
evaluated?
2. The BigInt
class has a pow
method, not an operator. Why didn’t the Scala library designers choose **
(as in Fortran) or ^
(as in Pascal) for a power operator?
3. Implement the Fraction
class with operations + - * /
. Normalize fractions, for example, turning 15/−6 into −5/2. Divide by the greatest common divisor, like this:
class Fraction(n: Int, d: Int) :
private val num: Int = if d == 0 then 1 else n * sign(d) / gcd(n, d);
private val den: Int = if d == 0 then 0 else d * sign(d) / gcd(n, d);
override def toString = s"$num/$den"
def sign(a: Int) = if a > 0 then 1 else if a < 0 then -1 else 0
def gcd(a: Int, b: Int): Int = if b == 0 then abs(a) else gcd(b, a % b)
...
4. Implement a class Money
with fields for dollars and cents. Supply +
, -
operators as well as comparison operators ==
and <
. For example, Money(1, 75) + Money(0, 50) == Money(2, 25)
should be true
. Should you also supply *
and /
operators? Why or why not?
5. Provide operators that construct an HTML table. For example,
Table() | "Java" | "Scala" || "Gosling" | "Odersky" || "JVM" | "JVM, .NET"
should produce
<table><tr><td>Java</td><td>Scala</td></tr><tr><td>Gosling...
6. Provide a class ASCIIArt
whose objects contain figures such as
/\_/
( ' ' )
( - )
| | |
(__|__)
Supply operators for combining two ASCIIArt
figures horizontally
/\_/ -----
( ' ' ) / Hello
( - ) < Scala |
| | | Coder /
(__|__) -----
or vertically. Choose operators with appropriate precedence.
7. Implement a class BitSequence
that stores a sequence of 64 bits packed in a Long
value. Supply apply
and update
operators to get and set an individual bit.
8. Provide a class Matrix
. Choose whether you want to implement 2 × 2 matrices, square matrices of any size, or m × n matrices. Supply operations +
and *
. The latter should also work with scalars, for example, mat * 2
. A single element should be accessible as mat(row, col)
.
9. Define an object PathComponents
with an unapply
operation class that extracts the directory path and file name from an java.nio.file.Path
. For example, the file /home/cay/readme.txt
has directory path /home/cay
and file name readme.txt
.
10. Modify the PathComponents
object of the preceding exercise to instead define an unapplySeq
operation that extracts all path segments. For example, for the file /home/cay/readme.txt
, you should produce a sequence of three segments: home
, cay
, and readme.txt
.
11. Show that the return type of unapply
can be an arbitrary subtype of Product
. Make your own concrete type MyProduct
that extends Product
and defines methods _1
, _2
. Then define an unapply
method returning a MyProduct
instance. What happens if you don’t define _1
?
12. Provide an extractor for strings where the first component is a title from an appropriate enumeration, and the remainder is a sequence of name components. For example, when called with "Dr. Peter van der Linden"
, the result would be Some(Title.DR, Seq("Peter", "van", "der", "Linden")
. Show how the values can be obtained through destructuring or in a match
expression.
13. Improve the dynamic property selector in Section 11.11, “Dynamic Invocation,” on page 159 so that one doesn’t have to use underscores. For example, sysProps.java.home
should select the property with key "java.home"
. Use a helper class, also extending Dynamic
, that contains partially completed paths.
14. Define a class XMLElement
that models an XML element with a name, attributes, and child elements. Using dynamic selection and method calls, make it possible to select paths such as rootElement.html.body.ul(id="42").li
, which should return all li
elements inside ul
with id
attribute 42
inside body
inside html
.
15. Provide an XMLBuilder
class for dynamically building XML elements, as builder.ul(id="42", style="list-style: lower-alpha;")
, where the method name becomes the element name and the named arguments become the attributes. Come up with a convenient way of building nested elements.
16. Section 11.12, “Typesafe Selection and Application,” on page 162 describes a Request
class that makes requests to https://$server/$name/$arg
. However, the URLs of real REST APIs aren’t always that regular. Consider https://www.thecocktaildb.com/api.php
. Change the Request
class so that it receives a map from method names to template strings, where the $
character is replaced with the argument value.
For this service, the following should work:
val cocktailServiceTemplate = Map(
"cocktail" -> "https://www.thecocktaildb.com/api/json/v1/1/search.php?s=$",
"ingredient" -> "https://www.thecocktaildb.com/api/json/v1/1/search.php?i=$")
val cocktailService =
Request(cocktailServiceTemplate).asInstanceOf[CocktailService]
Now the call
cocktailService.cocktail("Negroni")
should invoke
https://www.thecocktaildb.com/api/json/v1/1/search.php?s=Negroni