Chapter 22

Stream I/O

In chapter 9 we saw how to use scala.io.Source and java.util.Scanner to read from files. We also saw how we could use java.io.PrintWriter to write out files. In this chapter we will explore the overall structure of the java.io package and see how streams can be used to represent a general approach to input and output. This will set us up in the following chapter to use streams for network communication.

22.1 The java.io Package

The original library for doing input and output in Java was the java.io package. While there is now a java.nio package that can provide higher performance, the original library is still used as a foundation for many other parts of the Java libraries, which makes it worth knowing and understanding.

The majority of the classes in the java.io package are subtypes of one of four different abstract types: InputStream, OutputStream, Reader, and Writer. The InputStream and OutputStream have basic operations that work with bytes. The Reader and Writer have almost the same set of methods, but they work with characters. Here are the signatures of the method in InputStream.

available():Int

close():Unit

mark(readlimit:Int):Unit

markSupported():Boolean

read():Int // This method is abstract

read(b:Array[Byte]):Int

read(b:Array[Byte], off:Int, len:Int):Int

reset():Unit

skip(long n):Long

The basic read() method is abstract. The other read methods have default implementations that depend on this one, though they can be overridden to provide more efficient implementations as well. The read() method returns the value of the next byte in the stream giving an Int in the range of 0-255 if the read succeeds. If the end of the stream has been reached, it returns ‒1. The methods for Reader are very similar. One major difference is that the read methods take an Array[Char] instead of an Array[Byte]. We will be focusing on the streams in this chapter. Refer to the Java Applications Programming Interface (API) for full details on Reader and Writer.

The term stream comes from the way that data is accessed. The data is in a flow and you can see what is there, but you can not really jump around to random places. Imagine water flowing through a stream. You can see the water as it goes by or you can look away and skip it. However, once it has passed you do not get the option to repeat seeing that same water. That is the idea of a stream. Note that the InputStream has methods called markSupported, mark, and reset. For certain streams, these methods will allow you to effectively cheat a bit and jump backward in a stream. However, not all streams will support this behavior and even in those that do, you are constrained to only being able to return to the point that was previously marked.

The OutputStream has a slightly shorter set of methods. Like the InputStream, it works with bytes. In this case, writing data to the stream is like throwing something into flowing water. It gets whisked away and you do not have the option of making alterations. Here are the methods for OutputStream.

close():Unit

flush():Unit

write(b:Array[Byte]):Unit

write(b:Array[Byte], off:Int, len:Int):Unit

write(b:Int):Unit // This method is abstract

Once again, there is one critical method, in this case it is write(b:Int), that is abstract. While it takes an Int, only the lowest eight bits (the lowest byte) will be written out. The other 24 bits are ignored.

22.2 Streams for Files

The InputStream and OutputStream do not actually have any code to read from/write to anything. This is why they are abstract. Their sole purpose is to serve as base classes for inheritance. The subtypes are more specific about what source/sink of data they are attached to. The most obvious things to attach streams to are files. The FileInputStream and FileOutputStream are subtypes that implement the read/write methods so that they work with files. You instantiate these types using new and passing in the file you want either as a string or as an instance of java.io.File. Here is a simple example that illustrates using a FileInputStream to read a file. It prints each byte as a numeric value.

import java.io._

val fis = new FileInputStream(args(0))

var byte = fis.read()

while(byte>=0) {

 print(byte+" ")

 byte = fis.read()

}

println()

fis.close()

Running this program on itself produces the following output.

> scala ReadBytes.scala ReadBytes.scala

105 109 112 111 114 116 32 106 97 118 97 46 105 111 46 95 10 10 118 97 108 32 102

105 115 32 61 32 110 101 119 32 70 105 108 101 73 110 112 117 116 83 116 114 101

97 109 40 97 114 103 115 40 48 41 41 10 118 97 114 32 98 121 116 101 32 61 32 102

105 115 46 114 101 97 100 40 41 10 119 104 105 108 101 40 98 121 116 101 62 61 48

41 32 123 10 32 32 112 114 105 110 116 40 98 121 116 101 43 34 32 34 41 10 32 32

98 121 116 101 32 61 32 102 105 115 46 114 101 97 100 40 41 10 125 10 112 114 105

110 116 108 110 40 41 10 102 105 115 46 99 108 111 115 101 40 41 10

A slight modification will produce a script that can be used to copy files.

import java.io._

val fis = new FileInputStream(args(0))

val fos = new FileOutputStream(args(1))

var byte = fis.read()

while(byte>=0) {

  fos.write(byte)

  byte = fis.read()

}

fis.close()

fos.close()

This script takes two arguments, just like the Linux cp command. It uses the first as the file name for the FileInputStream and the second for the name of the FileOutputStream. Instead of printing the bytes as numbers to standard output, they are written to the output stream. When everything is done, both files are closed.

22.3 Exceptions

If you run one of the scripts and provide a name for the input file that is not valid you will get something that looks somewhat like the following.

java.io.FileNotFoundException: notthere.txt (No such file or directory)

  at java.io.FileInputStream.open(Native Method)

  at java.io.FileInputStream.<init>(FileInputStream.java:138)

  at java.io.FileInputStream.<init>(FileInputStream.java:97)

  at Main$$anon$1.<init>(ReadBytes.scala:3)

  at Main.main(ReadBytes.scala:1)

  at Main.main(ReadBytes.scala)

  ...

You have probably seen quite a few outputs like this in working with Scala. This is a stack trace and it was printed out because something bad happened that caused the program to throw an exception. To this point we have generally ignored exceptions and expected the user or the computer to behave in a way that would allow the program to run without errors. It is time to learn how to deal with these situations instead of ignoring them and hoping they will not occur.

Exceptions are meant to occur in exceptional circumstances, such as when there is an error. In the example above, you can see that it was a java.io.FileNotFoundException that occurred. This should make sense given that we tried to open a file that was not there. Real applications need to be able to deal with this. For example, if a user specified a file that did not exist, the program should tell the user that and perhaps ask for an alternate file. It should not crash. In order to prevent the crash, we need a construct that has the ability to execute some alternate code when an exception occurs. The construct for doing this is the try/catch.

To understand this construct, it helps to have some terminology. The act of raising an exception is described by the term "throw". You can use the throw keyword to indicate when an exception occurs in your own code. Here we demonstrate this in the REPL.

scala> throw new Exception("Testing")

java.lang.Exception: Testing

  at .<init>(<console>:8)

  at .<clinit>(<console>)

  at .<init>(<console>:11)

  at .<clinit>(<console>)

  ...

You can throw any subtype of the java.lang.Throwable class. The two immediate subtypes of this are java.lang.Error and java.lang.Exception. Typically an Error represents something that you will not be able to recover from. An Exception is something that you should consider dealing with. While this example throws an instance of Exception itself, that is not something you should do normally. Instead, you should throw a subtype that matches what has happened. If such a subtype exists in the libraries, use it. If not, create your own subtype.

You can also see here that the Exception can be passed a string argument. One of the main advantages of exceptions over other ways of dealing with errors is that exceptions can be informative. The ideal exception gives the programmer all the information they need to fix the error. All exceptions give you a stack trace so that you can see what line caused the exception and how that was reached. Often you need additional information to know what went wrong. In the case of the FileNotFoundException, the name of the file is critical information. If you try to access a value that is out of bounds on an Array, knowing what index was being accessed and how big the Array really was are critical pieces of information. So when you throw an exception or make a new subtype of Exception, you should make sure that it has all the information that will be needed to fix it.

22.3.1 try-catch-finally

Any time you open a file that was provided to you by the user, you know that something might go wrong and it might throw an exception. To stop that exception from crashing the thread that it is running in, you tell Scala that you are going to "try" to execute some code. If an exception is thrown that you want to deal with, you should "catch" it. The try keyword is followed by the block of code that you want to try to run. After the block of code you put the catch keyword followed by a block of cases that include the handlers for different exceptions that might arise. In the case of opening a file and printing the contents, code might look like this.

import java.io._

try {

  val fis = new FileInputStream(args(0))

  var byte = fis.read()

  while(byte>=0) {

  print(byte+" ")

  byte = fis.read()

  }

  println()

  fis.close()

} catch {

  case ex:FileNotFoundException =>

  println("The file "+args(0)+" could not be opened.")

  case ex:IOException =>

  println("There was a problem working with the file.")

}

This has two different cases in the catch. The first deals with the exception we saw before where the file could not be opened. This can happen even if the file exists if you do not have permission to read that file. The second case catches the more general IOException. This is a supertype of FileNotFoundException and represents any type of error that can occur with input and output. In this case, it would imply that something goes wrong with the reading of the file. Other than hardware errors it is hard to picture anything going wrong with this particular code because it makes no assumptions about the structure of the data in the file. It simply reads bytes until it runs out of data to read.

This code has a problem with it that would be even more significant if we were doing more interesting things with the reading. The problem arises if the file opens, but an exception occurs while it is being read. Imagine that an IOException is thrown by the fis.read()call in the while loop because something happens to the file. The exception causes the try block to stop and control jumps down to the proper case in the catch. After the code in that case is executed, the program continues with whatever follows the catch block. This means that the code in the try block after the location where the exception occurs is never executed. This is a problem, because that code included the closing of the file. Failure to close files can be a significant problem for large, long-running programs as any given program is only allowed to have so many files open at a given time.

What we need to fix this problem is a way where we can specify a block of code that will run after a try block regardless of what happens in the try block. This is done with a finally block. You can put a finally block after a catch block or directly after a try block. To do our file reading problem correctly we could use the following code.

import java.io._

try {

  val fis = new FileInputStream(args(0))

  try {

 var byte = fis.read()

 while(byte>=0) {

   print(byte+" ")

   byte = fis.read()

  }

  println()

 } catch {

  case ex:IOException =>

   println("There was a problem working with the file.")

 } finally {

  fis.close()

 }

} catch {

 case ex:FileNotFoundException =>

 println("The file "+args(0)+" could not be opened.")

}

This has one try block that contains everything, and inside of that is another try block that reads the file with a finally for closing the file. This is a bit verbose. We will see shortly how we can set things up so that we do not have to repeat this.

As with nearly every other construct in Scala, the try-catch-finally has a value and can be used as an expression. If the try block executes without throwing an exception then the value will be that of the try block. If an exception is thrown and caught, the value of the whole expression is the value of the catch for that case. Any value for finally is ignored as finally is typically used for clean-up, not part of the normal computation.

There is also the possibility that the try block executes and throws an exception that is not caught. In this situation the value of the full try expression is Nothing. You might remember this type from figure 19.1. It is at the very bottom of the type diagram. The Nothing type is a subtype of everything else. There are no instances of Nothing. It is mostly used by the type inference system. When trying to figure out the type of a complex expression with multiple possible return values, Scala picks the lowest type in the hierarchy that is a supertype of all the possibilities. Since a try can throw an exception that is not caught, it is almost always possible for it to produce a Nothing. The fact that Nothing is at the base of the type hierarchy means that it does not impact the inferred type.

If you use a try-catch-finally as an expression, you have to be careful of the types of the last expressions in each of the different cases of the catch. In particular, you probably do not want the last thing in any of the cases to be a print statement. Calls like println have a return type of Unit. Unless all the options are Unit, the inferred type will almost certainly be AnyVal or Any. Those are types that do not provide you with significant functionality. The catch cases of a try that is used as an expression probably should not print anything. If it should be an expression, then it should communicate results with a return value instead of through side effects.

22.3.2 Effect of Exceptions

So far we have treated an uncaught exception as causing the thread to terminate. This is not really the case. When an exception is thrown in a method/function, it causes execution of the current block to stop and control jumps either to the catch of an enclosing try block, or out of that method/function all together. If there is not an enclosing try, or if the catch on that try does not have a case for the exception that was thrown, the exception will pop up the stack to the method that called the current function. If the call point in that function is in a try block, the cases for that try block are checked against the exception. This process continues up the call stack until either a case is found that matches the exception or the top of the stack is reached and the thread terminates.

The fact that exceptions propagate up the call stack means that you often do not catch them at the place where they originate. Instead, you should catch exceptions at a place in the code where you know how to deal with them. Often this is several calls above the one that caused the exception to occur.

22.3.3 Loan Pattern

The need to double nest try expressions with a finally to properly handle closing a file is enough of a pain that it would prevent most programmers from putting in the effort to do it. This same problem existed in Java so they altered the language with an enhanced try-catch in Java 7. The syntax of Scala lets you streamline this code without altering the language by making it easy to implement the loan pattern [9]. Here is code that does the reading we have been doing in this way.

import java.io._

def useFileInputStream[A](fileName:String)(body:FileInputStream=>A):A = {

  val fis = new FileInputStream(fileName)

  try {

  body(fis)

  } finally {

  fis.close()

  }

}

try {

  useFileInputStream(args(0))(fis => {

  var byte = fis.read()

  while(byte>=0) {

   print(byte+" ")

   byte = fis.read()

  }

  println()

  })

} catch {

 case ex:FileNotFoundException =>

   println("The file "+args(0)+" could not be opened.")

  case ex:IOException =>

   println("There was a problem working with the file.")

}

Here the try with the finally has been moved into a separate function called useFileInputStream. This function takes a file name in one argument list and a function in a second. The function is used after that with a try that is only responsible for handling the errors. There are still two try statements, but the one in the useFileInputStream only has to be written once. The rest of the code can call that function and safely include only a single try-catch without a finally.

While it is not used in this example, the useFileInputStream function takes a type argument that allows it to be used as an expression. The type is inferred from the code in the body function.

22.4 Decorating Streams

The InputStream and OutputStream types only allow reading/writing of single bytes or Arrays of bytes. This is technically sufficient to do anything you want as all the data on a computer is stored as sequence of bytes when you get down to a certain level. Being sufficient though does not mean that it is ideal. What do we do if we are reading a stream that is supposed to contain a mixture of different data that includes things like Int or Double and not just Byte. You can build an Int from four bytes, but you do not want to have to write the code to do so yourself, especially not every time you need that functionality. This limitation of the basic streams, and others including the file streams, are addressed by decorating streams.1

Types like FileInputStream know what they are pulling data from, but they are not very flexible or efficient. This shortcoming is addressed by having stream types that are constructed by wrapping them around other streams. This wrapping of one stream around another to provide additional functionality is the decorating.

22.4.1 Buffering

One of the shortcomings of FileInputStream and FileOutputStream deals with performance. Accessing files is a slow process, it is one of the slower things that you can do on a computer. Reading one byte at a time is particularly slow. It is much better to read a large block of data than to read all the bytes, one at a time. This is the reason why BufferedInputStream and BufferedOutputStream were created. These types have the same methods as the standard InputStream and OutputStream. The only difference is how the read/write method is implemented.

Both types keep a buffer in memory. In the BufferedInputBuffer if that buffer has unread bytes in it, the read pulls from the buffer. When the buffer runs out of data, a full buffer is read at once. For the BufferedOutputStream, the bytes that are written go into the buffer until it is full, then all of them are dumped out at once. If you need to force things to go out before the buffer is full you can call the flush method.

The buffered types do not actually read from or write to any particular source or sink. In order to use one of the buffered types you need to give it a stream to pull data from or push data to. The syntax for doing that could look like this.

val bis = new BufferedInputStream(new FileInputStream(fileName))

The fact that disk access has really high latency means that you should always wrap file streams with buffered streams. The speed boost of doing so for large files can be remarkable.

22.4.2 Binary Data

The other drawback of the standard InputStream and OutputStream is that they only work with bytes. Pretty much all applications will need to work with data in some more complex format. We have seen how to do this using flat text files and using XML. When we introduced XML in chapter 14, we looked at several of the shortcomings of flat text files. The XML format put more information in the file so that anyone could pick it up and have a chance of figuring out what was in it. It also increased flexibility. XML did not address the problems of speed and using a significant amount of memory. If anything, it made these worse. In order to get speed and to use a small amount of space we have to sacrifice the benefits of XML and write the data in a format that matches how it is stored in the memory of the computer. We need to store the data in binary format.

The first way to read/write data in binary format is with the DataInputStream and DataOutputStream. These types get wrapped around other streams like the BufferedInputStream and BufferedOutputStream. What they give us is a set of additional methods. For the DataInputStream these methods include the following.

readBoolean():Boolean

readByte():Byte

readChar():Char

readDouble():Double

readFloat():Float

readInt():Int

readLong():Long

readShort():Short

readUTF():String

Each of these reads the specified data value from the stream that is wrapped. The DataOutputStream has matching methods for writing data. It also has a few extras that can write strings in other ways.

writeBoolean(v:Boolean):Unit

writeByte(v:Int):Unit

writeBytes(s:String):Unit

writeChar(v:Int):Unit

writeChars(s:String):Unit

writeDouble(v:Double):Unit

writeFloat(v:Float):Unit

writeInt(v:Int):Unit

writeLong(v:Long):Unit

writeShort(v:Int):Unit

writeUTF(str:String):Unit

The combination of these methods gives you the ability the write data to a file and read it back in.

If you are working with files for your binary data you still really need to have buffering for performance reasons. The beauty of the way the java.io streams library works is that you can decorate streams however you want. In this situation you want to wrap the data stream around the buffered stream, which is wrapped around the file stream. Code for that looks like the following.

val dis = new DataInputStream(new BufferedInputStream(new FileInputStream(file)))

The order is significant here, mainly because the methods we want to be able to call are part of the DataInputStream. The general rule is that the outermost type needs to have the methods that you want to call. The ones between that and the actual source/sink stream should implement the basic methods in an altered fashion, such as buffering. Those should be stacked in an order that makes sense for the application.

The challenge in working with binary data files is that they can not be easily edited with any standard programs. To understand this, consider the following code.

import java.io._

def withDOS[A](fileName:String)(body:DataOutputStream=>A):A = {

 val dos = new DataOutputStream(new BufferedOutputStream(new

   FileOutputStream(fileName)))

 try {

  body(dos)

 } finally {

  dos.close()

 }

}

def withDIS[A](fileName:String)(body:DataInputStream=>A):A = {

  val dis = new DataInputStream(new BufferedInputStream(new

  FileInputStream(fileName)))

  try {

  body(dis)

 } finally {

  dis.close()

 }

}

def writeDoubleArray(fileName:String,data:Array[Double]) {

  withDOS(fileName)(dos => {

  dos.writeInt(data.size)

  data.foreach(x => dos.writeDouble(x))

 })

}

def readDoubleArray(fileName:String):Array[Double] = {

 withDIS(fileName)(dis => {

  Array.fill(dis.readInt)(dis.readDouble)

 })

}

This code contains two functions that can be used to generally work with DataOutputStreams and DataInputStreams. In an application, these should probably be methods in an object that contains a number of such utility methods. The other two methods use those first two and write out or read back in an Array of Doubles. If you load that into the REPL, you can test it with the following.

scala> writeDoubleArray("data.bin",Array.fill(10)(math.random))

scala> readDoubleArray("data.bin")

res3: Array[Double] = Array(0.6609985904587437, 0.49319578039338174,

 0.5859229163803784, 0.965200016272522, 0.4832664731158809, 0.7969665973550756,

 0.7572733105097633, 0.1578906904549643, 0.08416227543386434,

 0.6824199253206102)

After you have done this you should go look at the contents of the "data.txt" file. What you will find using less, cat, or vi is that the contents look like random garbage characters. If you were to edit the file using vi then try to read it in with the readDoubleArray the data will almost certainly be messed up if it manages to read at all.

The problem with looking at a file like "data.bin" is that normal characters only account for a fairly small fraction of the possible values that each byte can take. Binary data tends to use all possible values, including many that do not print well. There are command-line tools like hexdump and xxd that can be used to view binary files. The following shows the output of xxd.

> xxd data.bin

0000000: 0000 000a 3fe5 26e6 8417 1dea 3fdf 9085 ....?.&.....?...

0000010: 08d4 253c 3fe2 bfe1 6a7a 94d5 3fee e2eb ..%<?...jz..?...

0000020: 24ff a71e 3fde edd6 8052 4d5c 3fe9 8024 $...?....RM?..$

0000030: 17f4 9f55 3fe8 3b95 3cd8 bd20 3fc4 35c3 ...U?.;.<.. ?.5.

0000040: 1bec 683c 3fb5 8ba8 ac8b 9ec8 3fe5 d662 ..h<?.......?..b

0000050: 4fac 814c    O..L

The first column shows the position in the file. The next eight columns show hexadecimal values for the contents of the file. There are two characters for each byte because 256 = 162. So each line shows 16 bytes. The last column shows the ASCII characters with any non-printable characters appearing as a dot. The xxd tool also does a reverse encoding. You can have it output to a file, edit the hex section of the file, and run the reverse encoding to get back a binary file. Doing so is a somewhat delicate operation because it is easy to mess up.

22.4.3 Serialization

Often when you are saving data to a file, you want to write out whole objects. The process of converting an object to some format that you can write out and read back in later is called serialization. Java and a number of other modern platforms have built in serialization methods. The Java serialization method uses a binary output format. You can also write your own code to serialize objects in other formats. XML happens to be a useful serialization format. We will look at each of these separately.

22.4.3.1 Binary Serialization

The Java platform has a rather powerful form of serialization that is built into the system. Other systems have different ways of supporting this. To make a type that can be serialized in Scala have it inherit from Serializable.3 The reason you would want to have things that are serializable is that they can be used with ObjectInputStream and ObjectOutputStream. These types have the same methods as DataInputStream and DataOutputStream plus readObject():AnyRef and writeObject(obj:AnyRef):Unit, respectively.

To help illustrate this, consider the following little application that includes a serializable class and some code to either write an instance of it to a file or read one in from a file and print it.

import java.io._

class Student(val name:String,val grades:Array[Int]) extends Serializable

object Main {

 def main(args:Array[String]) {

  args(0) match {

   case "-r" =>

  val ois = new ObjectInputStream(new BufferedInputStream(new

     FileInputStream(args(1))))

  ois.readObject() match {

   case s:Student => println(s.name+" "+s.grades.mkString(", "))

   case _ => println("Unidentified type.")

  }

  ois.close()

   case "-w" =>

  val oos = new ObjectOutputStream(new BufferedOutputStream(new

    FileOutputStream(args(1))))

  val s = new Student(args(2),args.drop(3).map(_.toInt))

  oos.writeObject(s)

  oos.close()

  case _ =>

  println("Usage: -r filename | -w filename name g1 g2 ...")

 }

 }

}

Put this in a file and compile it with scalac. From the command-line you can invoke this first step using this command.4

scala Main -w obj.bin John 98 78 88 93 100 83

After running this there will be a file called "obj.bin". You can look at it with cat or xxd. You will see that this is clearly a binary file, but there are some parts that are human readable strings. One of these is the name itself, but there are others that gives type names like Student and java/lang/String. These have to be in the file because when you deserialize the file it has to know what types of objects to create. You can verify that this process works with the following command.

scala Main -r obj.bin

This will read back in the file you just created and print the Student object that was read.

One critical aspect to note about the code for the ‒r option is that it includes a match. If you leave out the match and assign the result of ois.readObject() to a variable, the variable will have type AnyRef. That is because readObject has a return type of AnyRef. You will not be able to get name or grades from an AnyRef because that type does not have those. The match allows you to check the type of the object that was read in and do the print statement we want if it is a Student or print an error message if it is not.5

When an object is serialized, some indication of its type is written out, followed by a serialization of its contents. This only works if all the contents are serializable. If you try running this code here, you will find that it throws an exception.

import java.io._

class OtherData(val id:String,val course:String)

class Student(val name:String,val grades:Array[Int],val od:OtherData) extends

  Serializable

object Main {

  def main(args:Array[String]) {

  val oos = new ObjectOutputStream(new FileOutputStream("fail.bin"))

  val s = new Student("John",Array(98,90),new OtherData("0123","CS2"))

  oos.writeObject(s)

  oos.close()

 }

}

The details of the exception are shown here.

java.io.NotSerializableException: OtherData

The problem with this code is that od:OtherData is a member of Student, but it is not serializable. So when the serialization process gets to the od member, it fails. In this case it is simple enough to fix that by making it so that OtherData extends Serializable. In other situations you will not have that type of control because the type that you are dealing with might have been written by someone else and not be Serializable.

One way to deal with information that is not serializable is to simply not write it out. There are other times when this approach is valid as well. For example, the DrawRectangle type keeps the propPanel variable so that it does not make a new Graphical User Interface (GUI) component every time the user looks at the settings. However, if you save off a DrawRectangle, there is no reason to save all the information associated with the GUI component. That could be easily recreated from the other information. In order to do this in Scala we use an annotation.

An annotation is specified by a normal name that is preceded by an @. Annotations provide meta-information about a program. That is information that is used by higher-level tools and is not really part of the normal program code. There are a number of standard annotations that are part of the Scala compiler. You can identify them in the API because they start with lowercase letters. The two associated with serialization are @transient and @SerialVersionUID. If you have a member in a class that you do not want to have serialized, simply annotate it with @transient. Here we have code where this has been done with od.

import java.io._

class OtherData(val id:String,val course:String)

class Student(val name:String,

  val grades:Array[Int],

  @transient val od:OtherData) extends Serializable

object Main {

  def main(args:Array[String]) {

  val oos = new ObjectOutputStream(new FileOutputStream("pass.bin"))

  val s = new Student("John",Array(98,90),new OtherData("0123","CS2"))

  oos.writeObject(s)

  oos.close()

  val ois = new ObjectInputStream(new FileInputStream("pass.bin"))

  ois.readObject() match {

   case s2:Student => println(s2.name+" "+s2.grades+" "+s2.od)

   case _ => println("Unknown type read.")

  }

  ois.close()

 }

}

This code also reads the object back in and prints the different fields. This is done to illustrate what happens when you read a serialized object that that has a transient field. That field is not part of the serialization, so when the object is read back in, that field is given a default value. For any subtype of AnyRef, the default value is null. Running this code will show you that the last value printed is indeed null.

The other annotation, @SerialVersionUID, is used to attach a version number to a type. This is done so that when you load back in an object, the version number of the saved version can be compared to that of the loaded version to make sure they are the same. Java will automatically generate these for you, but they change nearly every time the code for the type changes. This will cause saved files to break, even if they would still work fine. To prevent this, you might consider putting this annotation, followed by an argument of a numeric ID, before the type declaration.

The implication this has for your programming is that anything that is transient generally must be a var instead of a val. This is due to the fact that when you deserialize such an object, that member will have a default value that almost certainly needs to be changed. The other side of this is that places that use any transient members likely need to have conditional code that will initialize the values if they are not set properly. This is exactly what was done with the propPanel in the Drawable types that we have made. If the field is accessed a lot, you should consider having a private local var that is only accessed through a method. Here is an alternate version of Student which follows that rule.

class Student(val name:String,

 val grades:Array[Int],

 @transient private var lod:OtherData) extends Serializable {

  assert(lod!=null)

  def od : OtherData = {

  if(lod==null) {

   lod = new OtherData("012345","Default")

  }

  lod

 }

}

You can substitute this version in the code above and you will no longer get null in the print out. This is because now the reference to s2.od is a call to the method that will create a default value if the member lod has not been assigned.

class Student(val name:String,

 val grades:Array[Int],

 @transient private var lod:OtherData) extends Serializable {

 assert(lod!=null)

 def od = lod

 private def writeObject(oos:ObjectOutputStream) {

 oos.defaultWriteObject()

 oos.writeUTF(od.id)

 oos.writeUTF(od.course)

 }

 private def readObject(ois:ObjectInputStream) {

 ois.defaultReadObject()

 lod = new OtherData(ois.readUTF,ois.readUTF)

 }

}

class Painting(val name:String,

  // ...

  @transient private var lImg:BufferedImage) extends Serializable {

  assert(lImg!=null)

  def img = lImg

  private def writeObject(oos:ObjectOutputStream) {

  oos.defaultWriteObject()

  oos.writeInt(img.getWidth)

  oos.writeInt(img.getHeight)

  oos.writeInt(img.getType)

  for(i <- 0 until img.getWidth; j <- 0 until img.getHeight) {

   oos.writeInt(img.getRGB(i,j))

  }

  }

  private def readObject(ois:ObjectInputStream) {

  ois.defaultReadObject()

  val (w,h,t) = (ois.readInt(),ois.readInt(),ois.readInt())

  lImg = new BufferedImage(w,h,t)

  for(i <- 0 until w; j <- 0 until h) {

   lImg.setRGB(i,j,ois.readInt())

  }

 }

}

22.4.3.2 XML Serialization

Java’s default serialization comes with the standard advantages and limitations of a binary format. In addition, it only works for programs running under the Java Virtual Machine (JVM). That includes not only Java and Scala, but a number of other languages as well. It does not include all languages. C and C++ stand out as languages that do not have a JVM implementation. There are also some situations where you do not need the benefits of binary format and you would rather have the portability and readability of XML.

There is not a built-in method of converting objects to XML or getting them back, but you can add this type of functionality to your own code fairly easily. To do this we will follow three fairly simple rules with each class we want to be able to serialize.

• Put a toXML : scala.xml.Node method in the class that returns an XML element with all the information you want to save for those objects.
• Include a companion object with an apply(node:scala.xml.Node) : Type method that deserializes the XML node and returns an object of the type you have created.
• Make all the fields that are serialized be arguments to the class.

If you do this consistently across your types, when one type includes a reference to another, you can simply call toXML on that type.6 You can use the normal methods in scala.xml.XML to read the XML from a file or write it back out to a file.

22.5 Saving Drawings (Project Integration)

Now it is time to integrate these different concepts into our project. We will do this by giving our program the ability to save drawings. For completeness, there will be three different save options: default serialized binary, zipped serialized binary, and XML. These three options will allow us to cover nearly everything that was introduced in the chapter in a manner that is more significant than the small samples shown with each topic.

There are three separate pieces to adding this code. We need to have GUI elements for the user to interact with. Those GUI elements need to call code that deals with the files. Then changes have to be made to the Drawing and Drawables so that they can be serialized and deserialized using both the default method and XML. We will start with the GUI code for two reasons. First, it is fairly easy to add menu items. Second, in order to test the other parts we need to have the ability to make the program call that code and that is what the GUI elements do. This is as easy to do as adding the following four lines to DrawingMain.

  contents += new MenuItem(Action("Save Binary")(saveBinary))

  contents += new MenuItem(Action("Save Zip")(saveZip))

  contents += new MenuItem(Action("Save XML")(saveXML))

  contents += new MenuItem(Action("Load Drawing")(load))

If you also add empty stubs for the methods called in the actions into DrawingMain, the code will compile and let you run it to see the menus.

At this point things get more interesting because we have to implement those four methods. The saveBinary and saveZip methods have a fair bit of code in common. They both let the user select a file to write to, open a FileInputStream, wrap it in some way, and then write the currently selected drawing out to the stream. The saveXML method is going to have the user select a file, then use scala.xml.XML.save to write a Node to disk. The node needs to come from the currently selected drawing and given what was said above, this will probably be done by adding toXML methods to Drawing and other places. The load method needs to allow the user to select a file, then identify the file type by the extension and execute the appropriate deserialization code.

We will start with saveBinary and saveZip and pull any code that would be duplicated out into other methods. This is what they look like.

 private def saveBinary {

  withSaveFile(file => {

   val oos = new ObjectOutputStream(new BufferedOutputStream(new

    FileOutputStream(file)))

   withOutputStream(oos)(strm => {

  serializeDrawingToStream(strm,file.getName())

  })

 })

 }

 private def saveZip {

  withSaveFile(file => {

  val zos = new ZipOutputStream(new BufferedOutputStream(new

   FileOutputStream(file)))

  zos.putNextEntry(new ZipEntry(file.getName().dropRight(3)+"bin"))

  val oos = new ObjectOutputStream(zos)

  withOutputStream(oos)(strm => {

   serializeDrawingToStream(strm,file.getName())

   })

  })

 }

The withSaveFile method opens a dialog box and allows the user to select a file, then executes the function that is passed on that file. The withOutputStream separates out the try/finally code for making sure the streams are closed off.7 Lastly, the serializeDrawingToStream method takes an ObjectOutputStream and a name for the file then serializes the current drawing to that stream and changes the text on the tab to the file name.

We can reuse the withSaveFile method in the saveXML method using the following code.

 private def saveXML {

 withSaveFile(file => {

  val drawing = openDrawings(tabbedPane.selection.index)

  xml.XML.save(file.getAbsolutePath(),drawing.toXML)

  tabbedPane.selection.page.title = file.getName()

 })

 }

In order to get this code to compile, you have to add a toXML : xml.Node method to the Drawing class. At this point, that function can just return <drawing></drawing>.

The load method is a bit longer because it can handle any of the three types of files based on the extension. It would be a better overall design for an application to be symmetric in the load and save options, but mixing them in this situation allows us to demonstrate how you might do it with one menu option or several. This sacrifices proper UI design for other pedagogical purposes.

 private def load {

 val chooser = new FileChooser()

 if(chooser.showSaveDialog(tabbedPane)==FileChooser.Result.Approve) {

  if(chooser.selectedFile.getName().endsWith(".bin")) {

  val ois = new ObjectInputStream(new BufferedInputStream(new

   FileInputStream(chooser.selectedFile)))

  withInputStream(ois)(strm => {

   deserializeDrawingFromStream(strm,chooser.selectedFile.getName())

  })

  } else if(chooser.selectedFile.getName().endsWith(".zip")) {

  val zis = new ZipInputStream(new BufferedInputStream(new

    FileInputStream(chooser.selectedFile)))

  zis.getNextEntry

  val ois = new ObjectInputStream(zis)

  withInputStream(ois)(strm => {

   deserializeDrawingFromStream(strm,chooser.selectedFile.getName())

  })

  } else if(chooser.selectedFile.getName().endsWith(".xml")) {

  val nd = Drawing(xml.XML.loadFile(chooser.selectedFile))

  openDrawings += nd

  tabbedPane.pages += new

   TabbedPane.Page(chooser.selectedFile.getName(),nd.propertiesPanel)

  }

 }

 }

Similar helper files are used here to those that were used for saving. For the XML option to work, a companion object must be added for Drawing that has an apply method that takes an xml.Node and returns a Drawing. For now this can simply return a new Drawing.

A complete listing of the revised DrawingMain is shown here.

package scalabook.drawing

import scala.swing._

import scala.collection.mutable

import java.io._

import java.util.zip._

object DrawingMain {

  private val tabbedPane = new TabbedPane

  private val openDrawings = mutable.Buffer[Drawing]()

  private def newDrawing {

 val nd = Drawing()

 openDrawings += nd

 tabbedPane.pages += new TabbedPane.Page("Unnamed",nd.propertiesPanel)

  }

  private def withOutputStream[A,B <: OutputStream](os:B)(body:B=>A):A = {

 try {

  body(os)

 } finally {

  os.close()

 }

  }

  private def serializeDrawingToStream(oos:ObjectOutputStream,name:String) {

 val drawing = openDrawings(tabbedPane.selection.index)

 oos.writeObject(drawing)

 tabbedPane.selection.page.title = name

  }

  private def withSaveFile(body: File =>Unit) {

 val chooser = new FileChooser

 if(chooser.showSaveDialog(tabbedPane)==FileChooser.Result.Approve) {

  try {

  body(chooser.selectedFile)

 } catch {

  case ex:FileNotFoundException => ex.printStackTrace()

  case ex:IOException => ex.printStackTrace()

  }

 }

 }

 private def saveBinary {

 withSaveFile(file => {

  val oos = new ObjectOutputStream(new BufferedOutputStream(new

   FileOutputStream(file)))

  withOutputStream(oos)(strm => {

   serializeDrawingToStream(strm,file.getName())

  })

 })

 }

 private def saveZip {

  withSaveFile(file => {

   val zos = new ZipOutputStream(new BufferedOutputStream(new

  FileOutputStream(file)))

   zos.putNextEntry(new ZipEntry(file.getName().dropRight(3)+"bin"))

   val oos = new ObjectOutputStream(zos)

   withOutputStream(oos)(strm => {

  serializeDrawingToStream(strm,file.getName())

   })

 })

 }

 private def saveXML {

  withSaveFile(file => {

   val drawing = openDrawings(tabbedPane.selection.index)

   xml.XML.save(file.getAbsolutePath(),drawing.toXML)

   tabbedPane.selection.page.title = file.getName()

 })

 }

 private def withInputStream[A,B <: InputStream](is:B)(body:B=>A):A = {

 try {

  body(is)

 } finally {

  is.close()

 }

 }

 private def deserializeDrawingFromStream(ois:ObjectInputStream,name:String) {

 val obj = ois.readObject()

 obj match {

  case nd : Drawing =>

  openDrawings += nd

  tabbedPane.pages += new TabbedPane.Page(name,nd.propertiesPanel)

  case _ =>

 }

 }

 private def load {

 val chooser = new FileChooser()

 if(chooser.showSaveDialog(tabbedPane)==FileChooser.Result.Approve) {

  if(chooser.selectedFile.getName().endsWith(".bin")) {

   val ois = new ObjectInputStream(new BufferedInputStream(new

   FileInputStream(chooser.selectedFile)))

   withInputStream(ois)(strm => {

  deserializeDrawingFromStream(strm,chooser.selectedFile.getName())

   })

 } else if(chooser.selectedFile.getName().endsWith(".zip")) {

  val zis = new ZipInputStream(new BufferedInputStream(new

    FileInputStream(chooser.selectedFile)))

  zis.getNextEntry

  val ois = new ObjectInputStream(zis)

  withInputStream(ois)(strm => {

  deserializeDrawingFromStream(strm,chooser.selectedFile.getName())

  })

 } else if(chooser.selectedFile.getName().endsWith(".xml")) {

  val nd = Drawing(xml.XML.loadFile(chooser.selectedFile))

  openDrawings += nd

  tabbedPane.pages += new

    TabbedPane.Page(chooser.selectedFile.getName(),nd.propertiesPanel)

 }

  }

 }

 private val frame = new MainFrame {

 contents = tabbedPane

 menuBar = new MenuBar {

  contents += new Menu("File") {

   contents += new MenuItem(Action("New")(newDrawing))

   contents += new MenuItem(Action("Save Binary")(saveBinary))

   contents += new MenuItem(Action("Save Zip")(saveZip))

   contents += new MenuItem(Action("Save XML")(saveXML))

   contents += new MenuItem(Action("Load Drawing")(load))

   contents += new Separator()

   contents += new MenuItem(Action("Exit")(sys.exit(0)))

 }

 }

 size = new Dimension(800,600)

 }

 def main(args : Array[String]) : Unit = {

  frame.visible=true

 }

}

The withInputStream and withOutputStream methods are curried to facilitate the type inference. Without the currying, it would be necessary to put a type on the strm argument for the functions that are passed in.

The error handling in this code is minimal, at best. Having a print of a stack trace in the code can be very helpful for programmers. However, it tells the user nothing. For a GUI based application like this one, it is likely that the user will not even see a console where the stack trace would print out. This code should pop up a window letting the user know what has happened. It has been left out here to not bloat the code and is left as an exercise for the reader.

If you try to run this code at this point, none of the options will really work. To make them work we have to alter Drawing and the Drawable hierarchy to support default and XML serialization. We will start with Drawing because that is what we are actually serializing. For the default serialization this means that the Drawing class needs to extend Serializable and that members that are not saved should be made transient. In addition, code needs to be altered so that transient fields do not cause problems when they are deserialized with a value of null.

For the XML-based serialization, that means implementing the toXML method to put any values we want saved into an XML element and implementing a deserializing apply method in a companion object. To make this work, values that are saved should also be moved up to be arguments of the class. After doing this to Drawing, we get code that looks like the following.

package scalabook.drawing

import swing._

import event._

import javax.swing.event._

import actors.Actor

/∗∗

 ∗ This type represents a Drawing that you can have open in the program.

 ∗/

class Drawing(private val root:DrawTransform) extends Serializable {

  @transient private var lDrawPanel = new DrawPanel

  @transient private var propPanel:Component = null

  @transient private var lTree = new javax.swing.JTree(root)

  root.drawing=this

  private def drawPanel = {

 if(lDrawPanel==null) lDrawPanel = new DrawPanel

 lDrawPanel

  }

  private def tree = {

 if(lTree==null) lTree = new javax.swing.JTree(root)

 lTree

  }

  private[drawing] def refresh {drawPanel.repaint()}

  private class ResultEvent(val text:String,val result:Any) extends Event

  /∗∗

 ∗ Returns a panel that displays the properties for this drawing.

 ∗

 ∗ @return a Component that can be put into a GUI.

 ∗/

  def propertiesPanel = {

  if(propPanel == null) {

   val commandArea = new TextArea()

   commandArea.editable = false

   val commandField = new TextField() {

   listenTo(this)

   reactions += {

   case e:EditDone =>

     if(!text.isEmpty) {

    val t = text

    Actor.actor {

     publish(new ResultEvent(t,Commands(t,Drawing.this)))

     }

     text=""

   }

   case e:ResultEvent =>

    commandArea append "> "+e.text+"
"+e.result+"
"

  }

  }

  val commandPanel = new BorderPanel {

  layout += commandField -> BorderPanel.Position.North

  layout += new ScrollPane(commandArea) -> BorderPanel.Position.Center

  preferredSize = new Dimension(500,200)

  }

  val drawProps = new GridPanel(1,1)

  propPanel = new DrawingPane(new SplitPane(Orientation.Vertical,

  new GridPanel(2,1) {

   contents += new BorderPanel {

   layout += new GridPanel(1,2) {

     contents += Button("Add"){

     executeOnSelection(d => addTo(d),addTo(root))

     }

     contents += Button("Remove"){

    executeOnSelection(d => remove(d))

     }

   } -> BorderPanel.Position.North

   layout+= new ScrollPane(new Component {

    override lazy val peer = tree

    tree.addTreeSelectionListener(new TreeSelectionListener {

    def valueChanged(e:TreeSelectionEvent) {

     executeOnSelection(d => {

      drawProps.contents.clear()

      drawProps.contents += d.propertiesPanel

      drawProps.revalidate

      drawProps.repaint

     })

    }

   })

  }) -> BorderPanel.Position.Center

  }

  contents += new ScrollPane(drawProps)

 },new SplitPane(Orientation.Horizontal,

   new ScrollPane(drawPanel),

   commandPanel))

  )

 }

 propPanel

 }

 def toXML : xml.Node =

 <drawing>

  {root.toXML}

 </drawing>

 private def executeOnSelection(f: Drawable=>Unit,default: =>Unit = {}) {

  val path = tree.getSelectionPath

  if(path!=null) {

  val last = path.getLastPathComponent match {

  case drawable:Drawable if(drawable!=null)=>

    f(drawable)

  case _ =>

   }

  } else default

 }

 private def addTo(d:Drawable) = {

  assert(d!=null)

  val parent = d match {

   case dt:DrawTransform => dt

   case _ => d.getParent

  }

  val options = Array[(String, ()=>Drawable)](

   ("Bouncing Balls", () => DrawBouncingBalls(parent)),

   ("Ellipse", () => new DrawEllipse(parent)),

   ("Mandebrot", () => new DrawMandelbrot(parent)),

   ("Rectangle", () => new DrawRectangle(parent)),

   ("Transform", () => new DrawTransform(parent))

  ).toMap

  val choice = Dialog.showInput(propPanel,

   "What do you want to add?","Draw Type",Dialog.Message.Question,

   null,options.keys.toSeq,options.keys.head)

  choice match {

   case Some(ch) =>

   val newChild = options(ch)()

   parent.addChild(newChild)

   drawPanel.repaint

   tree.getModel match {

    case m:javax.swing.tree.DefaultTreeModel => m.reload

    case _ =>

   }

   case _ =>

  }

  }

  private def remove(d:Drawable) {

  assert(d!=null)

  if(d.getParent!=null) {

   d.getParent.removeChild(d)

   drawPanel.repaint

   tree.getModel match {

  case m:javax.swing.tree.DefaultTreeModel => m.reload

  case _ =>

   }

  }

 }

 private class DrawPanel extends Panel {

  override def paint(g:Graphics2D) {

   g.setPaint(java.awt.Color.white)

   g.fillRect(0,0,size.width,size.height)

   root.draw(g)

  }

 }

 class DrawingPane(c:Component) extends GridPanel(1,1) {

  contents += c

 def drawing = Drawing.this

 }

}

object Drawing {

 def apply(data:xml.Node):Drawing = {

 new Drawing(DrawTransform(null,(data  "drawTransform")(0)))

 }

 def apply():Drawing = {

 new Drawing(new DrawTransform(null))

 }

}

The primary alterations are at the beginning and the end. In order for this code to compile, you must add a toXML method and a companion object with an apply method to DrawTransform. There are some other changes in here that are required because of alterations that we will run into later.

The fact that DrawTransform is part of an inheritance hierarchy, combined with the fact that default serialization is done through inheritance means that we will want to start at the top. So before looking at DrawTransform, we will look at Drawable. Here is the code for that trait.

package scalabook.drawing

import java.awt.Graphics2D

import javax.swing.tree.TreeNode

/∗∗

 ∗ This represents the supertype for all the different types that can appear in our drawing.

 ∗/

trait Drawable extends TreeNode with Serializable {

/∗∗

 ∗ Stores the Drawing this Drawable is part of.

 ∗/

@transient protected var lDrawing:Drawing = null

/∗∗

 ∗ Causes this object to be drawn to g.

 ∗

 ∗ @param g a Graphics2D object to draw to.

 ∗/

def draw(g : Graphics2D) : Unit

/∗∗

 ∗ Gives back a GUI component that allows the user to change drawing properties.

 ∗

 ∗ @return A component that should be put in the GUI so the user can edit this

   object.

 ∗/

def propertiesPanel() : scala.swing.Component

/∗∗

 ∗ Return an XML serialization of this object.

 ∗/

def toXML : xml.Node

/∗∗

 ∗ Make it so that the getParent inherited from TreeNode returns a DrawTransform.

 ∗/

override def getParent : DrawTransform

/∗∗

 ∗ Returns the drawing this Drawable is part of.

 ∗/

def drawing = lDrawing

/∗∗

 ∗ Sets the Drawing this is part of. Implementations in subtypes

 ∗ should recursively descend through Drawables.

 ∗/

 def drawing_=(d:Drawing):Unit

}

We have added Serializable to the list of types being inherited from and also put in an abstract toXML method. The former now makes all subtypes serializable. The latter will force us to put in concrete toXML methods in the subtypes. For the subtypes then we need to make the proper values transient, write toXML, and add a companion object with the appropriate apply method. This code also has a protected member called lDrawing with public methods that can be used to access or set it. This is included to deal with a problem that we will encounter with DrawBouncingBalls. Having it there will also allow us to do some things that we did not previously.

For the subtypes, we will start with DrawRectangle because it is fairly simple. Here is code for the revision to that class, leaving out the draw, propertiesPanel, and toString methods which are not changed.

class DrawRectangle(p:DrawTransform,

  private var width:Double = 100.0,

  private var height:Double = 100.0,

  private var color:Color = Color.black) extends DrawLeaf {

 val parent = p

 @transient private var propPanel:Component = null

 ...

 def toXML : xml.Node = {

  val colStr = color.getRGB.toHexString

  <drawRectangle width={width.toString} height={height.toString}

   color={"0"∗(6-colStr.length)+colStr}/>

 }

}

object DrawRectangle {

 def apply(p:DrawTransform,data:xml.Node) = {

  new

   DrawRectangle(p,(data"@width").text.toDouble,(data"@height").text.toDouble,

  new Color(Integer.parseInt((data"@color").text,16)))

  }

}

The simplicity of DrawRectangle means that the XML element can be written with the short form and given three properties. To make it easy to store the color, the RGB value is converted to hexadecimal padded to eight characters. The eight characters represent AARRGGBB for alpha, red, green, and blue respectively. To get that back to an Int, we use the Integer.parseInt method from the Java libraries because it can take a second parameter that specifies the base to use. The changes to DrawEllipse are the same as those to DrawRectangle.

You might note that the code for DrawRectangle is lacking the drawing_= method. That has been added to DrawLeaf so that it does not have to be added to all the subtypes. The method in DrawLeaf looks like this.

 def drawing_=(d:Drawing) {lDrawing = d}

The fact that the leaf elements do not have children means that all they need to do is set their locally stored value of lDrawing.

The other Drawable subtypes each represent a more significant challenge. In the case of DrawMandelbrot there are significantly more data members that need to be moved around to be arguments with default values. You can see the changes here.

package scalabook.drawing

import java.awt.{Graphics2D,Color}

import java.awt.image.BufferedImage

import swing._

import event._

class DrawMandelbrot(p:DrawTransform,

  private var rmin:Double = -1.5,

  private var rmax:Double = 0.5,

  private var imin:Double = -1.0,

  private var imax:Double = 1.0,

  private var width:Int = 600,

  private var height:Int = 600,

  private var maxCount:Int = 100) extends DrawLeaf {

  val parent = p

  @transient private var propPanel:Component = null

  @transient private var img:BufferedImage = null

  @transient private var changed = false

  private def properties:Seq[(String,() => Any,String => Unit)] = Seq(

  ("Real Min", () => rmin, s => rmin = s.toDouble),

  ("Real Max", () => rmax, s => rmax = s.toDouble),

  ("Imaginary Min", () => imin, s => imin = s.toDouble),

  ("Imaginary Max", () => imax, s => imax = s.toDouble),

  ("Width", () => width, s => width = s.toInt),

  ("Height", () => height, s => height = s.toInt),

  ("Max Count", () => maxCount, s => maxCount = s.toInt)

  )

  def draw(g:Graphics2D) {

  if(img==null || changed) {

  if(img==null || img.getWidth!=width || img.getHeight!=height) {

    img = new BufferedImage(width,height,BufferedImage.TYPE_INT_ARGB)

  }

  for(i <- 0 until width par) {

  val cr = rmin + i∗(rmax-rmin)/width

  for(j <- 0 until height) {

   val ci = imax - j∗(imax-imin)/width

   val cnt = mandelCount(cr,ci)

   img.setRGB(i,j,if(cnt == maxCount) Color.black.getRGB else

    new Color(1.0f,0.0f,0.0f,cnt.toFloat/maxCount).getRGB)

   }

  }

  }

  g.drawImage(img,0,0,null)

 }

 def propertiesPanel() : Component = {

  if(propPanel==null) {

  propPanel = new BorderPanel {

   val props = properties

   layout += new GridPanel(props.length,1) {

  for((propName,value,setter) <- props) {

   contents += new BorderPanel {

    layout += new Label(propName) -> BorderPanel.Position.West

    layout += new TextField(value().toString) {

    listenTo(this)

    reactions += {case e:EditDone => setter(text); changed = true}

    } -> BorderPanel.Position.Center

   }

   }

  } -> BorderPanel.Position.North

  }

 }

 propPanel

 }

  override def toString() = "Mandelbrot"

  def toXML : xml.Node = {

   <drawMandelbrot rmin={rmin.toString} rmax={rmax.toString}

  imin={imin.toString} imax={imax.toString}

  width={width.toString} height={height.toString} maxCount={maxCount.toString}/>

 }

 private def mandelIter(zr:Double,zi:Double,cr:Double,ci:Double) =

   (zr∗zr-zi∗zi+cr, 2∗zr∗zi+ci)

 private def mandelCount(cr:Double,ci:Double):Int = {

  var ret = 0

  var (zr, zi) = (0.0, 0.0)

  while(ret<maxCount && zr∗zr+zi∗zi<4) {

  val (tr,ti) = mandelIter(zr,zi,cr,ci) zr = tr

  zi = ti

  ret += 1

  }

  ret

 }

}

object DrawMandelbrot {

  def apply(p:DrawTransform,data:xml.Node) = {

  new DrawMandelbrot(p,

   (data"@rmin").text.toDouble,

   (data"@rmax").text.toDouble,

   (data"@imin").text.toDouble,

   (data"@imax").text.toDouble,

   (data"@width").text.toInt,

   (data"@height").text.toInt,

   (data"@maxCount").text.toInt)

  }

}

In addition to adding the code that handles XML, the other significant change is that properties has been converted from a val to a def and the point where it is used in the propertiesPanel method now introduces a temporary variable called props. This is done because data members interact with serialization while methods do not. The properties data member does not need to be serialized. It can be rebuilt whenever it is needed. Previously a val was slightly easier to use. With serialization, that is no longer the case.

The only other subtype of DrawLeaf that we have created so far is DrawBouncingBalls. The challenges here center around the fact that it contains a full collection so the XML has to have the ability to deal with that.

package scalabook.drawing

import java.awt.{Graphics2D,Color}

import java.awt.geom._

import swing._

import event._

import javax.swing.Timer

class DrawBouncingBalls(p:DrawTransform,

  private var balls:Vector[DrawBouncingBalls.Ball]) extends DrawLeaf {

 val parent = p

 @transient private var propPanel:Component = null

 @transient private var lTimer:Timer = null

 private def timer = {

  if(lTimer==null) {

   lTimer = new Timer(100,Swing.ActionListener(e => {

   balls = (for(DrawBouncingBalls.Ball(x,y,vx,vy,s) <- balls par) yield {

    var nvy = vy + 0.01 // gravity

    var nvx = vx

    var nx = x+nvx

    var ny = y+nvy

    if(nx-s<0.0) {

     nx = 2∗s-nx

     nvx = nvx.abs

    } else if(nx+s>1.0) {

     nx = 2.0-(nx+2∗s)

     nvx = -nvx.abs

    }

    if(ny-s<0.0) {

     ny = 2∗s-ny

     nvy = nvy.abs

    } else if(ny+s>1.0) {

     ny = 2.0-(ny+2∗s)

     nvy = -nvy.abs

    }

    DrawBouncingBalls.Ball(nx,ny,nvx,nvy,s)

   }).seq

   drawing.refresh

  }))

 }

 lTimer

 }

 def draw(g:Graphics2D) {

 g.setPaint(Color.black)

 g.draw(new Rectangle2D.Double(0,0,100,100))

 g.setPaint(Color.green)

 for(DrawBouncingBalls.Ball(x,y,_,_,s) <- balls) {

  g.fill(new Ellipse2D.Double((x-s)∗100,(y-s)∗100,s∗200,s∗200))

 }

 }

 def propertiesPanel() : Component = {

 if(propPanel==null) {

  propPanel = new BorderPanel {

   layout += new GridPanel(1,1) {

  val button = new Button("Start")

  button.action = Action("Start"){

   if(button.text == "Start") {

    timer.start()

    button.text="Stop"

   } else {

    timer.stop()

    button.text="Start"

   }

  }

  contents += button

  } -> BorderPanel.Position.North

  }

 }

 propPanel

 }

 override def toString = "Bouncing Balls"

 def toXML : xml.Node =

  <drawBouncingBalls>

   {balls.map(_.toXML)}

  </drawBouncingBalls>

}

object DrawBouncingBalls {

 def apply(p:DrawTransform,data:xml.Node) = {

  new DrawBouncingBalls(p,Vector((data"ball").map(bxml => {

   val x=(bxml"@x").text.toDouble

   val y=(bxml"@y").text.toDouble

   val vx=(bxml"@vx").text.toDouble

   val vy=(bxml"@vy").text.toDouble

   val size=(bxml"@size").text.toDouble

   Ball(x,y,vx,vy,size)

  }):_∗))

 }

 def apply(p:DrawTransform,minSize:Double = 0.01,maxSize:Double = 0.05) = {

   new DrawBouncingBalls(p,Vector.fill(20) {

   val size = minSize+math.random∗(maxSize-minSize)

  Ball(size+math.random∗(1-2∗size),size+math.random∗(1-2∗size),

   (math.random-0.5)∗0.02,(math.random-0.5)∗0.02,size)

  })

 }

 case class Ball(x:Double,y:Double,vx:Double,vy:Double,size:Double) {

  def toXML = <ball x={x.toString} y={y.toString}

    vx={vx.toString} vy={vy.toString} size={size.toString}/>

 }

}

The need for the collection to be properly serialized in both manners requires moving the code that does the creation of the balls down into apply methods of the companion object. In addition, the timer needs to be transient so that the button on the properties panel will continue to work after a file is loaded in.

The only type that remains to convert is the DrawTransform. The modifications to this type to support default serialization are fairly minimal. Those for the XML serialization are a bit more significant.

class DrawTransform(val parent:DrawTransform,

 private var transformType:DrawTransform.TransformType.Value =

  DrawTransform.TransformType.Translate,

  private val transformValue:Array[Double] = Array.fill(3)(0.0)

  ) extends Drawable {

 private val subnodes = mutable.Buffer[Drawable]()

 @transient private var propPanel:Component = null

 import DrawTransform.TransformType._

 ...

 def toXML : xml.Node =

  <drawTransform type={transformType.id.toString}

   values={transformValue.mkString(":")}>

   {subnodes.map(_.toXML)}

  </drawTransform>

 private def buildTransform:AffineTransform = transformType match {

  case Translate =>

   AffineTransform.getTranslateInstance(transformValue(0),transformValue(1))

  case Rotate =>

   AffineTransform.getRotateInstance(transformValue(0),transformValue(1),

      transformValue(2))

  case Scale =>

   AffineTransform.getScaleInstance(transformValue(0),transformValue(1))

  case Shear =>

   AffineTransform.getShearInstance(transformValue(0),transformValue(1))

 }

 ...

 def drawing_=(d:Drawing) {

  lDrawing = d

  subnodes.foreach(_.drawing=d)

 }

 ...

}

object DrawTransform {

 object TransformType extends Enumeration {

  val Translate,Rotate,Scale,Shear = Value

 }

 def makeDrawable(p:DrawTransform,cxml:xml.Node):Drawable = {

  cxml.label match {

  case "drawTransform" => DrawTransform(p,cxml)

  case "drawRectangle" => DrawRectangle(p,cxml)

  case "drawEllipse" => DrawEllipse(p,cxml)

  case "drawMandelbrot" => DrawMandelbrot(p,cxml)

  case "drawBouncingBalls" => DrawBouncingBalls(p,cxml)

  case _ => throw new IllegalArgumentException("XML contains unknown type:

   "+cxml)

  }

 }

 def apply(p:DrawTransform,data:xml.Node) = {

  val ret = new DrawTransform(p, TransformType((data"@type").text.toInt),

  (data"@values").text.split(":").map(_.toDouble))

  data.child.filter(cxml => cxml match {

   case e:xml.Elem => true

   case _ => false

  }).map(cxml => makeDrawable(ret,cxml)).foreach(ret.addChild)

  ret

  }

}

The Enumeration for the type of transformation was moved into the companion object to eliminate problems it might cause with serialization. The subnodes member is left out of the argument list because when we deserialize a transform, the Drawables under it need to have it as their parent. For this reason, we have to instantiate a complete transform that does not yet have children, then make the children and add them to the transform. This is all handled by methods in the companion object. Lastly, the drawing_= method implemented here not only sets the local value, it also recursively sets the values on all the subnodes.

With these changes in place, you now have the ability to make whatever drawings you wish and save them off in one of three styles, then load them back in. In practice you would likely choose between the binary serialization and XML serialization methods as few applications would need to support both.

22.6 End of Chapter Material

22.6.1 Summary of Concepts

• The java.io package contains quite a few useful types. There are four extended type hierarchies rooted in InputStream, OutputStream, Reader, and Writer. The first two have operations for reading and writing bytes. The second two operate on characters. We previously saw how to use scala.io.Source and java.io.PrintWriter for text access so this chapter focuses mainly on the two stream hierarchies.
• The InputStream and OutputStream classes are both abstract. Their subtypes come in two general forms, those that actually specify a source or sink for reading or writing, and those that modify the manner or functionality of reading and writing. The FileInputStream and FileOutputStream are of the former types. They provide stream behaviors attached to files.
• Exceptions are a way of dealing with unexpected conditions or other errors in code. When something goes wrong, an exception can be thrown. It is then up to the code that called the function where the error occurred to figure out what to do about it.
  • Exception handling is done through the try-catch expression. Code that might fail is put in a try block. The catch partial function has cases for the different exceptions the coder knows how to handle at that point.
  • When an exception is thrown, it immediately begins to pop up the call stack until it comes to a catch that can handle it. If none exists, it will go all the way to the top of the call stack and crash that thread.
  • The loan pattern is a handy approach to use in Scala to make it easy to write code that can throw exceptions.
• There are other subtypes of the primary stream types that do not inherently attach to a source or sink. These classes alter behavior of the basic methods or add additional methods to provide new functionality. You use them to "decorate" an existing stream.
  • One of the most common forms of decoration is buffering. This makes sure that slow operations like disk access are done in large chunks.
  • The DataInputStream and DataOutputStream provide additional methods that allow you to read or write data in binary. This is generally faster and more compact than text, but it loses the convenience of human readability.
  • The ObjectInputStream and ObjectOutputStream go a step beyond providing basic binary data reading and writing capabilities, they allow you to serialize whole objects assuming that the objects adhere to certain rules.

22.6.2 Exercises

1. Make the save for a drawing be a single option, and give the dialog different file types and extensions.
2. Add code to what was written in the chapter where you pop up error messages using Dialog.showMessage if something goes wrong.
3. The code shown for saving drawings has a significant bug that occurs if you try to save when there are no open drawings. Figure out what this is, and edit the code to fix it.
4. Make a class called Matrix that has inside of it an Array[Array[Double]]. Write code so that your class can be serialized and deserialized using Java serialization, XML, and custom binary formats.
5. Make a binary file with 1,000,000 integer values written with a DataOutputStream using writeInt. Try reading that back in with and without a BufferedInputStream between the DataInputStream and the FileInputStream. Time the results to see how much it matters on your hardware.
6. Write a binary file of integers with ten values in it. Make the values all different, perhaps random. Using xxd attempt to increment every value by one and put the result in a different file. Write Scala code to read back in that file and see if what you did worked.
7. If you did the last exercise and you are looking for a significant challenge, try to do the same thing with Doubles instead of Ints.
8. Write code to save some XML data out to a file. The challenge here is that it should be written to a ZIP file, not as plane text. Use an unzip utility to see if what you did worked. Also try to read it back in using Scala.

22.6.3 Projects

The projects for this chapter involve saving and loading data through serializing objects. You can decide whether you want to take the route of default binary serialization, XML based serialization, or a custom binary serialization. Which is best depends a lot on the project you are doing.

Something to note in general about the default binary serialization is that if you go this route, you almost certainly want to label your serializable classes is @SerialVersionUID so that the saved files do not break quite so often. They will still break, just not as often. The extra control you have over the XML gives it something of an advantage in this regard.

1. The main thing that will need to be saved in a MUD is the players. Over time they will collect different items and should probably gain in abilities. At this point, you can only have one player playing the game at a time. However, you can add the ability for a player to log in and log back out. That way the game can shift from one player to another. When the player logs out, their information should be saved. You can decide the format and method you want to use for saving the players.
2. If you are building the web spider, you inevitably want to store the data that you are interested locally in a format that you can easily work with. You might not want to store images locally, but at least keep a local copy of the URL. For text data that you process, storing it in the processed form will make it much easier to work with.

   So for this project you need to start building an internal representation of the data you care about, then work on serializing that out. Using the default binary serialization is probably ideal for numeric or tabular data. Things that are really text and images could work with XML. Choose the format that matches what you are interested in.

3. Like the MUD, a networked, multiplayer game will inevitably need a way to store off information about how various players are doing. Implement that for this project along with menu options for saving.
4. Even games that are not networked likely have a state that needs to be remembered. If you are working on that type of project, add menu options for saving and loading games.
5. If you are working on the math worksheet, that needs to be saved. You can pick between default serialization and XML for that.
6. Obviously, a Photoshop® type application needs to be able to save what you are drawing. This is one option where the default serialization is probably the easiest route. You will have to do some customized serialization code for images, but images do not store well in XML either.
7. Storing simulation data in plain text can make it easy to load in with simple plotting tools. However, large simulations need the space efficiency and speed of binary. For this, the ideal is neither XML, nor default serialization. Instead, you want to write custom binary serialization. For example, if you have N particles with x, y, z, υx, υy , and υz coordinates, as well as a mass, the efficient binary serialization starts with writing N as a binary Int, then following it with 7N binary Doubles for the data values of the particles.

1This terminology comes about because the java.io library uses the decorator pattern.[6]

2These are short for megabits per second and gigabits per second.

3Case classes automatically inherit from Serializable.

4You can do this in Eclipse as well. You need to open the Run settings and change the arguments. Using command-line arguments is easier in the terminal. If you want to test this in Eclipse you might consider changing from using args to standard input and output.

5This same type of operation can be done with isInstanceOf[A] and asInstanceOf[A] methods. However, the use of those methods is strongly frowned upon in Scala. Using a match is the appropriate Scala style for determining the type of an object and getting a reference to that object of the proper type.

6The only shortcoming of this approach is that if there are two or more references to any given object, it will be duplicated. The Java serialization code caches objects so this does not happen.

7The way that object streams work makes it so you need to close the object stream, not one of the streams it is wrapped around, like the file stream.