In this chapter, we continue our exploration of the Java API by looking at many of the classes in the java.io
and java.nio
packages. These packages offer a rich set of tools for basic I/O (input/output) and also provide the framework on which all file and network communication in Java is built. Figure 11-1 shows the class hierarchy of these packages. We’ll only cover a selection of this hierarchy, but you can see that it is quite broad. Once you have a handle on local file I/O, we’ll add the java.net
package and look at some basic networking concepts. (We’ll tackle the most popular of networking environments—the web—in Chapter 12.)
We’ll start by looking at the stream classes in java.io
, which are subclasses of the basic InputStream
, OutputStream
, Reader
, and Writer
classes. Then we’ll examine the File
class and discuss how you can read and write files using classes in java.io
. We also take a quick look at data compression and serialization. Along the way, we’ll also introduce the java.nio
package. The NIO package (or “new” I/O) adds significant functionality tailored for building high-performance services and in some cases simply provides newer, better APIs that can be used in place of some java.io
features.1
Most fundamental I/O in Java is based on streams. A stream represents a flow of data with (at least conceptually) a writer at one end and a reader at the other. When you are working with the java.io
package to perform terminal input and output, reading or writing files, or communicating through sockets in Java, you are using various types of streams. Later in this chapter, we’ll look at the NIO package, which introduces a similar concept called a channel. One difference betwen the two is that streams are oriented around bytes or characters while channels are oriented around “buffers” containing those data types—yet they perform roughly the same job. Let’s start by summarizing the available types of streams:
InputStream
, OutputStream
Abstract classes that define the basic functionality for reading or writing an unstructured sequence of bytes. All other byte streams in Java are built on top of the basic InputStream
and OutputStream
.
Reader
, Writer
Abstract classes that define the basic functionality for reading or writing a sequence of character data, with support for Unicode. All other character streams in Java are built on top of Reader
and Writer
.
InputStreamReader
, OutputStreamWriter
Classes that bridge byte and character streams by converting according to a specific character encoding scheme. (Remember: in Unicode, a character is not necessarily one byte!)
DataInputStream
, DataOutputStream
Specialized stream filters that add the ability to read and write multibyte data types, such as numeric primitives and String
objects in a universal format.
ObjectInputStream
, ObjectOutputStream
Specialized stream filters that are capable of writing whole groups of serialized Java objects and reconstructing them.
BufferedInputStream
, BufferedOutputStream
, BufferedReader
, BufferedWriter
Specialized stream filters that add buffering for additional efficiency. For real-world I/O, a buffer is almost always used.
PrintStream
, PrintWriter
Specialized streams that simplify printing text.
PipedInputStream
, PipedOutputStream
, PipedReader
, PipedWriter
“Loopback” streams that can be used in pairs to move data within an application. Data written into a PipedOutputStream
or PipedWriter
is read from its corresponding PipedInputStream
or PipedReader
.
FileInputStream
, FileOutputStream
, FileReader
, FileWriter
Implementations of InputStream
, OutputStream
, Reader
, and Writer
that read from and write to files on the local filesystem.
Streams in Java are one-way streets. The java.io
input and output classes represent the ends of a simple stream, as shown in Figure 11-1. For bidirectional conversations, you’ll use one of each type of stream.
InputStream
and OutputStream
are abstract classes that define the lowest-level interface for all byte streams. They contain methods for reading or writing an unstructured flow of byte-level data. Because these classes are abstract, you can’t create a generic input or output stream. Java implements subclasses of these for activities such as reading from and writing to files and communicating with sockets. Because all byte streams inherit the structure of InputStream
or OutputStream
, the various kinds of byte streams can be used interchangeably. A method specifying an InputStream
as an argument can accept any subclass of InputStream
. Specialized types of streams can also be layered or wrapped around basic streams to add features such as buffering, filtering, or handling higher-level data types.
Reader
and Writer
are very much like InputStream
and OutputStream
, except that they deal with characters instead of bytes. As true character streams, these classes correctly handle Unicode characters, which is not always the case with byte streams. Often, a bridge is needed between these character streams and the byte streams of physical devices, such as disks and networks. InputStreamReader
and OutputStreamWriter
are special classes that use a character-encoding scheme to translate between character and byte streams.
This section describes all the interesting stream types with the exception of FileInputStream
, FileOutputStream
, FileReader
, and FileWriter
. We postpone the discussion of file streams until the next section, where we cover issues involved with accessing the filesystem in Java.
The prototypical example of an InputStream
object is the standard input of a Java application. Like stdin
in C or cin
in C++, this is the source of input to a command-line (non-GUI) program. It is an input stream from the environment—usually a terminal window or possibly the output of another command. The java.lang.System
class, a general repository for system-related resources, provides a reference to the standard input stream in the static variable System.in
. It also provides a standard output stream and a standard error stream in the out
and err
variables, respectively.2 The following example shows the correspondence:
InputStream
stdin
=
System
.
in
;
OutputStream
stdout
=
System
.
out
;
OutputStream
stderr
=
System
.
err
;
This snippet hides the fact that System.out
and System.err
aren’t just OutputStream
objects, but more specialized and useful PrintStream
objects. We’ll explain these later in “PrintWriter and PrintStream”, but for now we can reference out
and err
as OutputStream
objects because they are derived from OutputStream
.
We can read a single byte at a time from standard input with the InputStream
’s read()
method. If you look closely at the API, you’ll see that the read()
method of the base InputStream
class is an abstract
method. What lies behind System.in
is a particular implementation of InputStream
that provides the real implementation of the read()
method:
try
{
int
val
=
System
.
in
.
read
();
}
catch
(
IOException
e
)
{
...
}
Although we said that the read()
method reads a byte value, the return type in the example is int
, not byte
. That’s because the read()
method of basic input streams in Java uses a convention carried over from the C language to indicate the end of a stream with a special value. Data byte values are returned as unsigned integers in the range 0 to 255 and the special value of -1
is used to indicate that end of stream has been reached. You’ll need to test for this condition when using the simple read()
method. You can then cast the value to a byte if needed. The following example reads each byte from an input stream and prints its value:
try
{
int
val
;
while
(
(
val
=
System
.
in
.
read
())
!=
-
1
)
System
.
out
.
println
((
byte
)
val
);
}
catch
(
IOException
e
)
{
...
}
As we’ve shown in the examples, the read()
method can also throw an IOException
if there is an error reading from the underlying stream source. Various subclasses of IOException
may indicate that a source such as a file or network connection has had an error. Additionally, higher-level streams that read data types more complex than a single byte may throw EOFException
(“end of file”), which indicates an unexpected or premature end of stream.
An overloaded form of read()
fills a byte array with as much data as possible up to the capacity of the array and returns the number of bytes read:
byte
[]
buff
=
new
byte
[
1024
];
int
got
=
System
.
in
.
read
(
buff
);
In theory, we can also check the number of bytes available for reading at a given time on an InputStream
using the available()
method. With that information, we could create an array of exactly the right size:
int
waiting
=
System
.
in
.
available
();
if
(
waiting
>
0
)
{
byte
[]
data
=
new
byte
[
waiting
];
System
.
in
.
read
(
data
);
...
}
However, the reliability of this technique depends on the ability of the underlying stream implementation to detect how much data can be retrieved. It generally works for files but should not be relied upon for all types of streams.
These read()
methods block until at least some data is read (at least one byte). You must, in general, check the returned value to determine how much data you got and if you need to read more. (We look at nonblocking I/O later in this chapter.) The skip()
method of InputStream
provides a way of jumping over a number of bytes. Depending on the implementation of the stream, skipping bytes may be more efficient than reading them.
The close()
method shuts down the stream and frees up any associated system resources. It’s important for performance to remember to close most types of streams when you are finished using them. In some cases, streams may be closed automatically when objects are garbage-collected, but it is not a good idea to rely on this behavior. In Java 7, the try
-with-resources language feature was added to make automatically closing streams and other closeable entities easier. We’ll see some examples of that in “File Streams”. The flag interface java.io.Closeable
identifies all types of stream, channel, and related utility classes that can be closed.
Finally, we should mention that in addition to the System.in
and System.out
standard streams, Java provides the java.io.Console
API through System.console()
. You can use the Console
to read passwords without echoing them to the screen.
In early versions of Java, some InputStream
and OutputStream
types included methods for reading and writing strings, but most of them operated by naively assuming that a 16-bit Unicode character was equivalent to an 8-bit byte in the stream. This works only for Latin-1 (ISO 8859-1) characters and not for the world of other encodings that are used with different languages. In Chapter 8, we saw that the java.lang.String
class has a byte array constructor and a corresponding getBytes()
method that each accept character encoding as an argument. In theory, we could use these as tools to transform arrays of bytes to and from Unicode characters so that we could work with byte streams that represent character data in any encoding format. Fortunately, however, we don’t have to rely on this because Java has streams that handle this for us.
The java.io Reader
and Writer
character stream classes were introduced as streams that handle character data only. When you use these classes, you think only in terms of characters and string data and allow the underlying implementation to handle the conversion of bytes to a specific character encoding. As we’ll see, some direct implementations of Reader
and Writer
exist, for example, for reading and writing files. But more generally, two special classes, InputStreamReader
and OutputStreamWriter
, bridge the gap between the world of character streams and the world of byte streams. These are, respectively, a Reader
and a Writer
that can be wrapped around any underlying byte stream to make it a character stream. An encoding scheme is used to convert between possible multibyte encoded values and Java Unicode characters. An encoding scheme can be specified by name in the constructor of InputStreamReader
or OutputStreamWriter
. For convenience, the default constructor uses the system’s default encoding scheme.
For example, let’s parse a human-readable string from the standard input into an integer. We’ll assume that the bytes coming from System.in
use the system’s default encoding scheme:
try
{
InputStream
in
=
System
.
in
;
InputStreamReader
charsIn
=
new
InputStreamReader
(
in
);
BufferedReader
bufferedCharsIn
=
new
BufferedReader
(
inReader
);
String
line
=
bufferedCharsIn
.
readLine
();
int
i
=
NumberFormat
.
getInstance
().
parse
(
line
).
intValue
();
}
catch
(
IOException
e
)
{
}
catch
(
ParseException
pe
)
{
}
First, we wrap an InputStreamReader
around System.in
. This reader converts the incoming bytes of System.in
to characters using the default encoding scheme. Then, we wrap a BufferedReader
around the InputStreamReader
. BufferedReader
adds the readLine()
method, which we can use to grab a full line of text (up to a platform-specific, line-terminator character combination) into a String
. The string is then parsed into an integer using the techniques described in Chapter 8.
The important thing to note is that we have taken a byte-oriented input stream, System.in
, and safely converted it to a Reader
for reading characters. If we wished to use an encoding other than the system default, we could have specified it in the InputStreamReader
’s constructor like so:
InputStreamReader
reader
=
new
InputStreamReader
(
System
.
in
,
"UTF-8"
);
For each character that is read from the reader, the InputStreamReader
reads one or more bytes and performs the necessary conversion to Unicode.
We return to the topic of character encodings when we discuss the java.nio.charset
API, which allows you to query for and use encoders and decoders explicitly on buffers of characters and bytes. Both InputStreamReader
and OutputStreamWriter
can accept a Charset
codec object as well as a character encoding name.
What if we want to do more than read and write a sequence of bytes or characters? We can use a “filter” stream, which is a type of InputStream
, OutputStream
, Reader
, or Writer
that wraps another stream and adds new features. A filter stream takes the target stream as an argument in its constructor and delegates calls to it after doing some additional processing of its own. For example, we can construct a BufferedInputStream
to wrap the system standard input:
InputStream
bufferedIn
=
new
BufferedInputStream
(
System
.
in
);
The BufferedInputStream
is a type of filter stream that reads ahead and buffers a certain amount of data. The BufferedInputStream
wraps an additional layer of functionality around the underlying stream. Figure 11-2 shows this arrangement for a DataInputStream
, which is a type of stream that can read higher-level data types, such as Java primitives and strings.
As you can see from the previous code snippet, the BufferedInputStream
filter is a type of InputStream
. Because filter streams are themselves subclasses of the basic stream types, they can be used as arguments to the construction of other filter streams. This allows filter streams to be layered on top of one another to provide different combinations of features. For example, we could first wrap our System.in
with a BufferedInputStream
and then wrap the BufferedInputStream
with a DataInputStream
for reading special data types with buffering.
Java provides base classes for creating new types of filter streams: FilterInputStream
, FilterOutputStream
, FilterReader
, and FilterWriter
. These superclasses provide the basic machinery for a “no op” filter (a filter that doesn’t do anything) by delegating all their method calls to their underlying stream. Real filter streams subclass these and override various methods to add their additional processing. We’ll make an example filter stream later in this chapter.
DataInputStream
and DataOutputStream
are filter streams that let you read or write strings and primitive data types composed of more than a single byte. DataInputStream
and DataOutputStream
implement the DataInput
and DataOutput
interfaces, respectively. These interfaces define methods for reading or writing strings and all of the Java primitive types, including numbers and Boolean values. DataOutputStream
encodes these values in a machine-independent manner and then writes them to its underlying byte stream. DataInputStream
does the converse.
You can construct a DataInputStream
from an InputStream
and then use a method such as readDouble()
to read a primitive data type:
DataInputStream
dis
=
new
DataInputStream
(
System
.
in
);
double
d
=
dis
.
readDouble
();
This example wraps the standard input stream in a DataInputStream
and uses it to read a double
value. The readDouble()
method reads bytes from the stream and constructs a double
from them. The DataInputStream
methods expect the bytes of numeric data types to be in network byte order, a standard that specifies that the high-order bytes are sent first (also known as “big endian,” as we discuss later).
The DataOutputStream
class provides write methods that correspond to the read methods in DataInputStream
. For example, writeInt()
writes an integer in binary format to the underlying output stream.
The readUTF()
and writeUTF()
methods of DataInputStream
and DataOutputStream
read and write a Java String
of Unicode characters using the UTF-8 “transformation format” character encoding. UTF-8 is an ASCII-compatible encoding of Unicode characters that is very widely used. Not all encodings are guaranteed to preserve all Unicode characters, but UTF-8 does. You can also use UTF-8 with Reader
and Writer
streams by specifying it as the encoding name.
The BufferedInputStream
, BufferedOutputStream
, BufferedReader
, and BufferedWriter
classes add a data buffer of a specified size to the stream path. A buffer can increase efficiency by reducing the number of physical read or write operations that correspond to read()
or write()
method calls. You create a buffered stream with an appropriate input or output stream and a buffer size. (You can also wrap another stream around a buffered stream so that it benefits from the buffering.) Here’s a simple buffered input stream called bis
:
BufferedInputStream
bis
=
new
BufferedInputStream
(
myInputStream
,
32768
);
...
bis
.
read
();
In this example, we specify a buffer size of 32 KB. If we leave off the size of the buffer in the constructor, a reasonably sized one is chosen for us. (Currently the default is 8 KB.) On our first call to read()
, bis
tries to fill our entire 32 KB buffer with data, if it’s available. Thereafter, calls to read()
retrieve data from the buffer, which is refilled as necessary.
A BufferedOutputStream
works in a similar way. Calls to write()
store the data in a buffer; data is actually written only when the buffer fills up. You can also use the flush()
method to wring out the contents of a BufferedOutputStream
at any time. The flush()
method is actually a method of the OutputStream
class itself. It’s important because it allows you to be sure that all data in any underlying streams and filter streams has been sent (before, for example, you wait for a response).
Some input streams such as BufferedInputStream
support the ability to mark a location in the data and later reset the stream to that position. The mark()
method sets the return point in the stream. It takes an integer value that specifies the number of bytes that can be read before the stream gives up and forgets about the mark. The reset()
method returns the stream to the marked point; any data read after the call to mark()
is read again.
This functionality could be useful when you are reading the stream in a parser. You may occasionally fail to parse a structure and so must try something else. In this situation, you can have your parser generate an error and then reset the stream to the point before it began parsing the structure:
BufferedInputStream
input
;
...
try
{
input
.
mark
(
MAX_DATA_STRUCTURE_SIZE
);
return
(
parseDataStructure
(
input
)
);
}
catch
(
ParseException
e
)
{
input
.
reset
();
...
}
The BufferedReader
and BufferedWriter
classes work just like their byte-based counterparts, except that they operate on characters instead of bytes.
Another useful wrapper stream is java.io.PrintWriter
. This class provides a suite of overloaded print()
methods that turn their arguments into strings and push them out the stream. A complementary set of println()
convenience methods appends a new line to the end of the strings. For formatted text output, printf()
and the identical format()
methods allow you to write printf
-style formatted text to the stream.
PrintWriter
is an unusual character stream because it can wrap either an OutputStream
or another Writer
. PrintWriter
is the more capable big brother of the legacy PrintStream
byte stream. The System.out
and System.err
streams are PrintStream
objects; you have already seen such streams strewn throughout this book:
System
.
out
.
(
"Hello, world... "
);
System
.
out
.
println
(
"Hello, world..."
);
System
.
out
.
printf
(
"The answer is %d"
,
17
);
System
.
out
.
println
(
3.14
);
Early versions of Java did not have the Reader
and Writer
classes and used PrintStream
, which converted bytes to characters by simply making assumptions about the character encoding. You should use a PrintWriter
for all new development.
When you create a PrintWriter
object, you can pass an additional Boolean value to the constructor, specifying whether it should “auto-flush.” If this value is true
, the PrintWriter
automatically performs a flush()
on the underlying OutputStream
or Writer
each time it sends a newline:
PrintWriter
pw
=
new
PrintWriter
(
myOutputStream
,
true
/*autoFlush*/
);
pw
.
println
(
"Hello!"
);
// Stream is automatically flushed by the newline.
When this technique is used with a buffered output stream, it corresponds to the behavior of terminals that send data line by line.
The other big advantage that PrintStream
and PrintWriter
have over regular character streams is that they shield you from exceptions thrown by the underlying streams. Unlike methods in other stream classes, the methods of PrintWriter
and PrintStream
do not throw IOException
s. Instead, they provide a method to explicitly check for errors if required. This makes life a lot easier for printing text, which is a very common operation. You can check for errors with the checkError()
method:
System
.
out
.
println
(
reallyLongString
);
if
(
System
.
out
.
checkError
()
){
...
// uh oh
The java.io.File
class encapsulates access to information about a file or directory. It can be used to get attribute information about a file, list the entries in a directory, and perform basic filesystem operations, such as removing a file or making a directory. While the File
object handles these “meta” operations, it doesn’t provide the API for reading and writing file data; there are file streams for that purpose.
You can create an instance of ++File++ from a ++String++ pathname:
File
fooFile
=
new
File
(
"/tmp/foo.txt"
);
File
barDir
=
new
File
(
"/tmp/bar"
);
You can also create a file with a relative path:
File
f
=
new
File
(
"foo"
);
In this case, Java works relative to the “current working directory” of the Java interpreter. You can determine the current working directory by reading the user.dir
property in the System Properties
list:
System
.
getProperty
(
"user.dir"
);
// e.g.,"/Users/pat"
An overloaded version of the File
constructor lets you specify the directory path and filename as separate String
objects:
File
fooFile
=
new
File
(
"/tmp"
,
"foo.txt"
);
With yet another variation, you can specify the directory with a File
object and the filename with a String
:
File
tmpDir
=
new
File
(
"/tmp"
);
// File for directory /tmp
File
fooFile
=
new
File
(
tmpDir
,
"foo.txt"
);
None of these File
constructors actually creates a file or directory, and it is not an error to create a File
object for a nonexistent file. The File
object is just a handle for a file or directory whose properties you may wish to read, write, or test. For example, you can use the exists()
instance method to learn whether the file or directory exists.
One issue with working with files in Java is that pathnames are expected to follow the conventions of the local filesystem. Two differences are that the Windows filesystem uses “roots” or drive letters (for example, C:) and a backslash () instead of the forward slash (/) path separator that is used in other systems.
Java tries to compensate for the differences. For example, on Windows platforms, Java accepts paths with either forward slashes or backslashes. (On others, however, it only accepts forward slashes.)
Your best bet is to make sure you follow the filename conventions of the host filesystem. If your application has a GUI that is opening and saving files at the user’s request, you should be able to handle that functionality with the Swing JFileChooser
class. This class encapsulates a graphical file-selection dialog box. The methods of the JFileChooser
take care of system-dependent filename features for you.
If your application needs to deal with files on its own behalf, however, things get a little more complicated. The File
class contains a few static
variables to make this task possible. File.separator
defines a String
that specifies the file separator on the local host (e.g., /
on Unix and Macintosh systems and on Windows systems);
File.separatorChar
provides the same information as a char
.
You can use this system-dependent information in several ways. Probably the simplest way to localize pathnames is to pick a convention that you use internally, such as the forward slash (/), and do a String
replace to substitute for the localized separator character:
// we'll use forward slash as our standard
String
path
=
"mail/2004/june/merle"
;
path
=
path
.
replace
(
'/'
,
File
.
separatorChar
);
File
mailbox
=
new
File
(
path
);
Alternatively, you could work with the components of a pathname and build the local pathname when you need it:
String
[]
path
=
{
"mail"
,
"2004"
,
"june"
,
"merle"
};
StringBuffer
sb
=
new
StringBuffer
(
path
[
0
]);
for
(
int
i
=
1
;
i
<
path
.
length
;
i
++)
{
sb
.
append
(
File
.
separator
+
path
[
i
]
);
}
File
mailbox
=
new
File
(
sb
.
toString
()
);
One thing to remember is that Java interprets a literal backslash character () in source code as an escape character when used in a
String
. To get a backslash in a String
, you have to use \
.
To grapple with the issue of filesystems with multiple “roots” (for example, C:
on Windows), the File
class provides the static method listRoots()
, which returns an array of File
objects corresponding to the filesystem root directories. Again, in a GUI application, a graphical file chooser dialog shields you from this problem entirely.
Once we have a File
object, we can use it to ask for information about and perform standard operations on the file or directory it represents. A number of methods let us ask questions about the File
. For example, isFile()
returns true
if the File
represents a regular file, while isDirectory()
returns true
if it’s a directory. isAbsolute()
indicates whether the File
encapsulates an absolute path or relative path specification. An absolute path is a system-dependent notion that means that the path doesn’t depend on the application’s working directory or any concept of a working root or drive (e.g., in Windows, it is a full path including the drive letter: c:\Userspatfoo.txt).
Components of the File
pathname are available through the following methods: getName()
, getPath()
, getAbsolutePath()
, and getParent()
. getName()
returns a String
for the filename without any directory information. If the File
has an absolute path specification, getAbsolutePath()
returns that path. Otherwise, it returns the relative path appended to the current working directory (attempting to make it an absolute path). getParent()
returns the parent directory of the file or directory.
The string returned by getPath()
or getAbsolutePath()
may not follow the same case conventions as the underlying filesystem. You can retrieve the filesystem’s own or “canonical” version of the file’s path by using the method getCanonicalPath()
. In Windows, for example, you can create a File
object whose getAbsolutePath()
is C:Autoexec.bat but whose getCanonicalPath()
is C:AUTOEXEC.BAT; both actually point to the same file. This is useful for comparing filenames that may have been supplied with different case conventions or for showing them to the user.
You can get or set the modification time of a file or directory with lastModified()
and setLastModified()
methods. The value is a long
that is the number of milliseconds since the epoch (Jan 1, 1970, 00:00:00 GMT). We can also get the size of the file in bytes with length()
.
Here’s a fragment of code that prints some information about a file:
File
fooFile
=
new
File
(
"/tmp/boofa"
);
String
type
=
fooFile
.
isFile
()
?
"File "
:
"Directory "
;
String
name
=
fooFile
.
getName
();
long
len
=
fooFile
.
length
();
System
.
out
.
println
(
type
+
name
+
", "
+
len
+
" bytes "
);
If the File
object corresponds to a directory, we can list the files in the directory with the list()
method or the listFiles()
method:
File
tmpDir
=
new
File
(
"/tmp"
);
String
[]
fileNames
=
tmpDir
.
list
();
File
[]
files
=
tmpDir
.
listFiles
();
list()
returns an array of String
objects that contains filenames. listFiles()
returns an array of File
objects. Note that in neither case are the files guaranteed to be in any kind of order (alphabetical, for example). You can use the Collections API to sort strings alphabetically like so:
List
list
=
Arrays
.
asList
(
fileNames
);
Collections
.
sort
(
list
);
If the File
refers to a nonexistent directory, we can create the directory with mkdir()
or mkdirs()
. The mkdir()
method creates at most a single directory level, so any intervening directories in the path must already exist. mkdirs()
creates all directory levels necessary to create the full path of the File
specification. In either case, if the directory cannot be created, the method returns false
. Use renameTo()
to rename a file or directory and delete()
to delete a file or directory.
Although we can create a directory using the File
object, this isn’t the most common way to create a file; that’s normally done implicitly when we intend to write data to it with a FileOutputStream
or FileWriter
, as we’ll discuss in a moment. The exception is the createNewFile()
method, which can be used to attempt to create a new zero-length file at the location pointed to by the File
object. The useful thing about this method is that the operation is guaranteed to be “atomic” with respect to all other file creation in the filesystem. createNewFile()
returns a Boolean value that tells you whether the file was created or not. This is sometimes used as a primitive locking feature—whoever creates the file first “wins.” (The NIO package supports true file locks, as we’ll see later.) This is useful in combination deleteOnExit()
, which flags the file to be automatically removed when the Java VM exits. This combination allows you to guard resources or make an application that can only be run in a single instance at a time. Another file creation method that is related to the File
class itself is the static method createTempFile()
, which creates a file in a specified location using an automatically generated unique name. This, too, is useful in combination with deleteOnExit()
.
The toURL()
method converts a file path to a file:
URL object. URLs are an abstraction that allows you to point to any kind of object anywhere on the Net. Converting a File
reference to a URL may be useful for consistency with more general utilities that deal with URLs. File URLs also come into greater use with the NIO File API where they can be used to reference new types of filesystems that are implemented directly in Java code.
Table 11-1 summarizes the methods provided by the File
class.
Method | Return type | Description |
---|---|---|
|
|
Is the file executable? |
|
|
Is the file (or directory) readable? |
|
|
Is the file (or directory) writable? |
|
|
Creates a new file. |
|
|
Static method to create a new file, with the specified prefix and suffix, in the default temp file directory. |
|
|
Deletes the file (or directory). |
|
|
When it exits, Java runtime system deletes the file. |
|
|
Does the file (or directory) exist? |
|
|
Returns the absolute path of the file (or directory). |
|
|
Returns the absolute, case-correct and relative-element-resolved path of the file (or directory). |
|
|
Get the number of bytes of unallocated space on the partition holding this path or 0 if the path is invalid. |
|
|
Returns the name of the file (or directory). |
|
|
Returns the name of the parent directory of the file (or directory). |
|
|
Returns the path of the file (or directory). (Not to be confused with |
|
|
Get the size of the partition that contains the file path in bytes or 0 if the path is invalid. |
|
|
Get the number of bytes of user-accessible unallocated space on the partition holding this path or 0 if the path is invalid. This method attempts to take into account user write permissions. |
|
|
Is the filename (or directory name) absolute? |
|
|
Is the item a directory? |
|
|
Is the item a file? |
|
|
Is the item hidden? (System-dependent.) |
|
|
Returns the last modification time of the file (or directory). |
|
|
Returns the length of the file. |
|
|
Returns a list of files in the directory. |
|
|
Returns the contents of the directory as an array of |
|
|
Returns array of root filesystems if any (e.g., C:/, D:/). |
|
|
Creates the directory. |
|
|
Creates all directories in the path. |
|
|
Renames the file (or directory). |
|
|
Sets execute permissions for the file. |
|
|
Sets the last-modified time of the file (or directory). |
|
|
Sets read permissions for the file. |
|
|
Sets the file to read-only status. |
|
|
Sets the write permissions for the file. |
|
|
Convert the File to an NIO File Path (see the NIO File API). (Not to be confused with |
|
|
Generates a URL object for the file (or directory). |
OK, you’re probably sick of hearing about files already and we haven’t even written a byte yet! Well, now the fun begins. Java provides two fundamental streams for reading from and writing to files: FileInputStream
and FileOutputStream
. These streams provide the basic byte-oriented InputStream
and OutputStream
functionality that is applied to reading and writing files. They can be combined with the filter streams described earlier to work with files in the same way as other stream communications.
You can create a FileInputStream
from a String
pathname or a File
object:
FileInputStream
in
=
new
FileInputStream
(
"/etc/passwd"
);
When you create a FileInputStream
, the Java runtime system attempts to open the specified file. Thus, the FileInputStream
constructors can throw a FileNotFoundException
if the specified file doesn’t exist or an IOException
if some other I/O error occurs. You must catch these exceptions in your code. Wherever possible, it’s a good idea to get in the habit of using the Java 7 try
-with-resources construct to automatically close files for you when you are finished with them:
try
(
FileInputStream
fin
=
new
FileInputStream
(
"/etc/passwd"
)
)
{
....
// fin will be closed automatically if needed upon exiting the try clause.
}
When the stream is first created, its available()
method and the File
object’s length()
method should return the same value.
To read characters from a file as a Reader
, you can wrap an InputStreamReader
around a FileInputStream
. You can also use the FileReader
class instead, which is provided as a convenience. FileReader
is just a FileInputStream
wrapped in an InputStreamReader
with some defaults.
The following class, ListIt
, is a small utility that sends the contents of a file or directory to standard output:
//file: ListIt.java
import
java.io.*
;
class
ListIt
{
public
static
void
main
(
String
args
[]
)
throws
Exception
{
File
file
=
new
File
(
args
[
0
]
);
if
(
!
file
.
exists
()
||
!
file
.
canRead
()
)
{
System
.
out
.
println
(
"Can't read "
+
file
);
return
;
}
if
(
file
.
isDirectory
()
)
{
String
[]
files
=
file
.
list
();
for
(
String
file
:
files
)
System
.
out
.
println
(
file
);
}
else
try
{
Reader
ir
=
new
InputStreamReader
(
new
FileInputStream
(
file
)
);
BufferedReader
in
=
new
BufferedReader
(
ir
);
String
line
;
while
((
line
=
in
.
readLine
())
!=
null
)
System
.
out
.
println
(
line
);
}
catch
(
FileNotFoundException
e
)
{
System
.
out
.
println
(
"File Disappeared"
);
}
}
}
ListIt
constructs a File
object from its first command-line argument and tests the File
to see whether it exists and is readable. If the File
is a directory, ListIt
outputs the names of the files in the directory. Otherwise, ListIt
reads and outputs the file, line by line.
For writing files, you can create a FileOutputStream
from a String
pathname or a File
object. Unlike FileInputStream
, however, the FileOutputStream
constructors don’t throw a FileNotFoundException
. If the specified file doesn’t exist, the FileOutputStream
creates the file. The FileOutputStream
constructors can throw an IOException
if some other I/O error occurs, so you still need to handle this exception.
If the specified file does exist, the FileOutputStream
opens it for writing. When you subsequently call the write()
method, the new data overwrites the current contents of the file. If you need to append data to an existing file, you can use a form of the constructor that accepts a Boolean append
flag:
FileInputStream
fooOut
=
new
FileOutputStream
(
fooFile
);
// overwrite fooFile
FileInputStream
pwdOut
=
new
FileOutputStream
(
"/etc/passwd"
,
true
);
// append
Another way to append data to files is with RandomAccessFile
, which we’ll discuss shortly.
Just as with reading, to write characters (instead of bytes) to a file, you can wrap an OutputStreamWriter
around a FileOutputStream
. If you want to use the default character-encoding scheme, you can use the FileWriter
class instead, which is provided as a convenience.
The following example reads a line of data from standard input and writes it to the file /tmp/foo.txt:
String
s
=
new
BufferedReader
(
new
InputStreamReader
(
System
.
in
)
).
readLine
();
File
out
=
new
File
(
"/tmp/foo.txt"
);
FileWriter
fw
=
new
FileWriter
(
out
);
PrintWriter
pw
=
new
PrintWriter
(
fw
)
pw
.
println
(
s
);
pw
.
close
();
Notice how we wrapped the FileWriter
in a PrintWriter
to facilitate writing the data. Also, to be a good filesystem citizen, we called the close()
method when we’re done with the FileWriter
. Here, closing the PrintWriter
closes the underlying Writer
for us. We also could have used try
-with-resources here.
The java.io.RandomAccessFile
class provides the ability to read and write data at a specified location in a file. RandomAccessFile
implements both the DataInput
and DataOutput
interfaces, so you can use it to read and write strings and primitive types at locations in the file just as if it were a DataInputStream
and DataOutputStream
. However, because the class provides random, rather than sequential, access to file data, it’s not a subclass of either InputStream
or OutputStream
.
You can create a RandomAccessFile
from a String
pathname or a File
object. The constructor also takes a second String
argument that specifies the mode of the file. Use the string r
for a read-only file or rw
for a read/write file.
try
{
RandomAccessFile
users
=
new
RandomAccessFile
(
"Users"
,
"rw"
)
}
catch
(
IOException
e
)
{
...
}
When you create a RandomAccessFile
in read-only mode, Java tries to open the specified file. If the file doesn’t exist, RandomAccessFile
throws an IOException
. If, however, you’re creating a RandomAccessFile
in read/write mode, the object creates the file if it doesn’t exist. The constructor can still throw an IOException
if another I/O error occurs, so you still need to handle this exception.
After you have created a RandomAccessFile
, call any of the normal reading and writing methods, just as you would with a DataInputStream
or DataOutputStream
. If you try to write to a read-only file, the write method throws an IOException
.
What makes a RandomAccessFile
special is the seek()
method. This method takes a long
value and uses it to set the byte offset location for reading and writing in the file. You can use the getFilePointer()
method to get the current location. If you need to append data to the end of the file, use length()
to determine that location, then seek()
to it. You can write or seek beyond the end of a file, but you can’t read beyond the end of a file. The read()
method throws an EOFException
if you try to do this.
Here’s an example of writing data for a simplistic database:
users
.
seek
(
userNum
*
RECORDSIZE
);
users
.
writeUTF
(
userName
);
users
.
writeInt
(
userID
);
...
In this snippet, we assume that the String
length for userName
, along with any data that comes after it, fits within the specified record size.
We are now going to turn our attention from the original, “classic” Java File API to the new, NIO, File API introduced with Java 7. As we mentioned earlier, the NIO File API can be thought of as either a replacement for or a complement to the classic API. Included in the NIO package, the new API is nominally part of an effort to move Java toward a higher performance and more flexible style of I/O supporting selectable and asynchronously interruptable channels. However, in the context of working with files, the new API’s strength is that it provides a fuller abstraction of the filesystem in Java.
In addition to better support for existing, real world, filesystem types—including for the first time the ability to copy and move files, manage links, and get detailed file attributes like owners and permissions—the new File API allows entirely new types of filesystems to be implemented directly in Java. The best example of this is the new ZIP filesystem provider that makes it possible to “mount” a ZIP archive file as a filesystem and work with the files within it directly using the standard APIs, just like any other filesystem. Additionally, the NIO File package provides some utilities that would have saved Java developers a lot of repeated code over the years, including directory tree change monitoring, filesystem traversal (a visitor pattern), filename “globbing,” and convenience methods to read entire files directly into memory.
We’ll cover the basic NIO File API in this section and return to the topic of buffers and channels at the end of the chapter. In particular, we’ll talk about ByteChannel
s and FileChannel
, which you can think of as alternate, buffer-oriented streams for reading and writing files and other types of data.
The main players in the java.nio.file
package are: the FileSystem
, which represents an underlying storage mechanism and serves as a factory for Path
objects; the Path
, which represents a file or directory within the filesystem; and the Files
utility, which contains a rich set of static methods for manipulating Path
objects to perform all of the basic file operations analogous to the classic API.
The FileSystems
(plural) class is our starting point. It is a factory for a FileSystem
object:
// The default host computer filesystem
FileSystem
fs
=
FileSystems
.
getDefault
();
// A custom filesystem for ZIP files, no special properties
Map
<
String
,
String
>
props
=
new
HashMap
<>();
URI
zipURI
=
URI
.
create
(
"jar:file:/Users/pat/tmp/MyArchive.zip"
);
FileSystem
zipfs
=
FileSystems
.
newFileSystem
(
zipURI
,
props
)
);
As shown in this snippet, often we’ll simply ask for the default filesystem to manipulate files in the host computer’s environment, as with the classic API. But the FileSystems
class can also construct a FileSystem
by taking a URI (a special identifier similar to a URL) that references a custom filesystem type. Here we use jar:file
as our URI protocol to indicate we are working with a JAR or ZIP file. We pass the URI along with We’ll show an example of working with the ZIP filesystem provider later in this chapter when we discuss data compression.
FileSystem
implements Closeable
and when a FileSystem
is closed, all open file channels and other streaming objects associated with it are closed as well. Attempting to read or write to those channels will throw an exception at that point. Note that the default filesystem (associated with the host computer) cannot be closed.
Once we have a FileSystem
, we can use it as a factory for Path
objects that represent files or directories. A Path
can be constructed using a string representation just like the classic File
, and subsequently used with methods of the Files
utility to create, read, write, or delete the item.
Path
fooPath
=
fs
.
getPath
(
"/tmp/foo.txt"
);
OutputStream
out
=
Files
.
newOutputStream
(
fooPath
);
This example opens an OutputStream
to write to the file foo.txt. By default, if the file does not exist, it will be created and if it does exist, it will be truncated (set to zero length) before new data is written—but you can change these results using options. We’ll talk more about Files
methods in the next section.
The Path
object implements the java.lang.Iterable
interface, which can be used to iterate through its literal path components (e.g., the slash separated “tmp” and “foo.txt” in the preceding snippet). Although if you want to traverse the path to find other files or directories, you might be more interested in the DirectoryStream
and FileVisitor
that we’ll discuss later. Path
also implements the java.nio.file.Watchable
interface, which allows it to be monitored for changes. We’ll also discuss watching file trees for changes in an upcoming section.
Path has convenience methods for resolving paths relative to a file or directory.
Path
patPath
=
fs
.
getPath
(
"/User/pat/"
);
Path
patTmp
=
patPath
.
resolve
(
"tmp"
);
// "/User/pat/tmp"
// Same as above, using a Path
Path
tmpPath
=
fs
.
getPath
(
"tmp"
);
Path
patTmp
=
patPath
.
resolve
(
tmpPath
);
// "/User/pat/tmp"
// Resolving a given absolute path against any path just yields given path
Path
absPath
=
patPath
.
resolve
(
"/tmp"
);
// "/tmp"
// Resolve sibling to Pat (same parent)
Path
danPath
=
patPath
.
resolveSibling
(
"dan"
);
// "/Users/dan"
In this snippet, we’ve shown the Path
resolve()
and resolveSibling()
methods used to find files or directories relative to a given Path
object. The resolve()
method is generally used to append a relative path to an existing Path
representing a directory. If the argument provided to the resolve()
method is an absolute path, it will just yield the absolute path (it acts kind of like the Unix or DOS “cd” command). The resolveSibling()
method works the same way, but it is relative to the parent of the target Path
; this method is useful for describing the target of a move()
operation.
To bridge the old and new APIs, corresponding toPath()
and toFile()
methods have been provided in java.io.File
and java.nio.file.Path
, respectively, to convert to the other form. Of course, the only types of Path
s that can be produced from File are paths representing files and directories in the default host filesystem.
Path
tmpPath
=
fs
.
getPath
(
"/tmp"
);
File
file
=
tmpPath
.
toFile
();
File
tmpFile
=
new
File
(
"/tmp"
);
Path
path
=
tmpFile
.
toPath
();
Once we have a Path
, we can operate on it with static methods of the Files
utility to create the path as a file or directory, read and write to it, and interrogate and set its properties. We’ll list the bulk of them and then discuss some of the more important ones as we proceed.
Table 11-2 summarizes these methods of the java.nio.file.Files
class. As you might expect, because the Files
class handles all types of file operations, it contains a large number of methods. To make the table more readable, we have elided overloaded forms of the same method (those taking different kinds of arguments) and grouped corresponding and related types of methods together.
Method | Return type | Description |
---|---|---|
|
long or |
Copy a stream to a file path, file path to stream, or path to path. Returns the number of bytes copied or the target |
|
|
Create a single directory or all directories in a specified path. |
|
|
Creates an empty file. The operation is atomic and will only succeed if the file does not exist. (This property can be used to create flag files to guard resources, etc.) |
|
|
Create a temporary, guaranteed, uniquely named directory or file with the specified prefix. Optionally place it in the system default temp directory. |
|
void |
Delete a file or an empty directory. |
|
boolean |
Determine whether the file exists ( |
|
boolean |
Tests basic file features: whether the path exists, is a directory, and other basic attributes. |
|
boolean or |
Create a hard or symbolic link, test to see if a file is a symbolic link, or read the target file pointed to by the symbolic link. Symbolic links are files that reference other files. Regular (“hard”) links are low-level mirrors of a file where two filenames point to the same underlying data. If you don’t know which to use, use a symbolic link. |
|
|
Get or set filesystem-specific file attributes such as access and update times, detailed permissions, and owner information using implementation-specific names. |
|
|
Get a |
|
|
Get or set the last modified time of a file or directory. |
|
|
Get or set a |
|
|
Get or set the full POSIX user-group-other style read and write permissions for the path as a Set of |
|
boolean |
Test to see whether the two paths reference the same file (which may potentially be true even if the paths are not identical). |
|
|
Move a file or directory by renaming or copying it, optionally specifying whether to replace any existing target. Rename will be used unless a copy is required to move a file across file stores or filesystems. Directories can be moved using this method only if the simple rename is possible or if the directory is empty. If a directory move requires copying files across file stores or filesystems, the method throws an |
|
|
Open a file for reading via a |
|
|
Create a new file or open an existing file as a seekable byte channel. (See the full discussion of NIO later in this chapter.) Consider using |
|
|
Return a |
|
|
Open a file for reading via an |
|
|
Returns the MIME type of the file if it can be determined by installed |
|
byte[] or |
Read all data from the file as a byte [] or all characters as a list of strings using a specified character encoding. |
|
long |
Get the size in bytes of the file at the specified path. |
|
|
Apply a |
|
|
Write an array of bytes or a collection of strings (with a specified character encoding) to the file at the specified path and close the file, optionally specifying append and truncation behavior. The default is to truncate and write the data. |
With the preceding methods, we can fetch input or output streams or buffered readers and writers to a given file. We can also create paths as files and dirctories and iterate through file hierarchies. We’ll discuss directory operations in the next section.
As a reminder, the resolve()
and resolveSibling()
methods of Path
are useful for constructing targets for the copy()
and move()
operations.
// Move the file /tmp/foo.txt to /tmp/bar.txt
Path
foo
=
fs
.
getPath
(
"/tmp/foo.txt"
);
Files
.
move
(
foo
,
foo
.
resolveSibling
(
"bar.txt"
)
);
For quickly reading and writing the contents of files without streaming, we can use the various readAll…
and write
methods that move byte arrays or strings in and out of files in a single operation. These are very convenient for files that easily fit into memory.
// Read and write collection of String (e.g. lines of text)
Charset
asciiCharset
=
Charset
.
forName
(
"US-ASCII"
);
List
<
String
>
csvData
=
Files
.
readAllLines
(
csvPath
,
asciiCharset
);
Files
.
write
(
newCSVPath
,
csvData
,
asciiCharset
);
// Read and write bytes
byte
[]
data
=
Files
.
readAllBytes
(
dataPath
);
Files
.
write
(
newDataPath
,
data
);
We are now going to complete our introduction to core Java I/O facilities by returning to the java.nio
package. As previously mentioned, the name NIO stands for “New I/O” and, as we saw earlier in this chapter in our discussion of java.nio.file
, one aspect of NIO is simply to update and enhance features of the legacy java.io
package. Much of the general NIO functionality does indeed overlap with existing APIs. However, NIO was first introduced to address specific issues of scalability for large systems, especially in networked applications. The following section outlines the basic elements of NIO, which center on working with buffers and channels.
Most of the need for the NIO package was driven by the desire to add nonblocking and selectable I/O to Java. Prior to NIO, most read and write operations in Java were bound to threads and were forced to block for unpredictable amounts of time. Although certain APIs such as Sockets (which we’ll see in “Sockets”) provided specific means to limit how long an I/O call could take, this was a workaround to compensate for the lack of a more general mechanism. In many languages, even those without threading, I/O could still be done efficiently by setting I/O streams to a nonblocking mode and testing them for their readiness to send or receive data. In a nonblocking mode, a read or write does only as much work as can be done immediately—filling or emptying a buffer and then returning. Combined with the ability to test for readiness, this allows a single-threaded application to continuously service many channels efficiently. The main thread “selects” a stream that is ready and works with it until it blocks and then moves on to another. On a single-processor system, this is fundamentally equivalent to using multiple threads. It turns out that this style of processing has scalability advantages even when using a pool of threads (rather than just one). We’ll discuss this in detail in Chapter 12 when we discuss web programming and building servers that can handle many clients simultaneously.
In addition to nonblocking and selectable I/O, the NIO package enables closing and interrupting I/O operations asynchronously. As discussed in Chapter 9, prior to NIO there was no reliable way to stop or wake up a thread blocked in an I/O operation. With NIO, threads blocked in I/O operations always wake up when interrupted or when the channel is closed by anyone. Additionally, if you interrupt a thread while it is blocked in an NIO operation, its channel is automatically closed. (Closing the channel because the thread is interrupted might seem too strong, but usually it’s the right thing to do. Leaving it open could result in unexpected behavior or subject the channel to unwanted manipulation.)
Channel I/O is designed around the concept of buffers, which are a sophisticated form of array, tailored to working with communications. The NIO package supports the concept of direct buffers—buffers that maintain their memory outside the Java VM in the host operating system. Because all real I/O operations ultimately have to work with the host OS by maintaining the buffer space there, some operations can be made much more efficient. Data moving between two external endpoints can be transferred without first copying it into Java and back out.
NIO provides two general-purpose file-related features not found in java.io
: memory-mapped files and file locking. We’ll discuss memory-mapped files later, but suffice it to say that they allow you to work with file data as if it were all magically resident in memory. File locking supports the concept of shared and exclusive locks on regions of files—useful for concurrent access by multiple applications.
While java.io
deals with streams, java.nio
works with channels. A channel is an endpoint for communication. Although in practice channels are similar to streams, the underlying notion of a channel is more abstract and primitive. Whereas streams in java.io
are defined in terms of input or output with methods to read and write bytes, the basic channel interface says nothing about how communications happen. It simply has the notion of being open or closed, supported via the methods isOpen()
and close()
. Implementations of channels for files, network sockets, or arbitrary devices then add their own methods for operations, such as reading, writing, or transferring data. The following channels are provided by NIO:
FileChannel
Pipe.SinkChannel
, Pipe.SourceChannel
SocketChannel
, ServerSocketChannel
, DatagramChannel
We’ll cover FileChannel
in this chapter. The Pipe
channels are simply the channel equivalents of the java.io Pipe
facilities. Additionally, in Java 7 there are now asynchronous versions of both the file and socket channels: AsynchronousFileChannel
, AsynchronousSocketChannel
, AsynchronousServerSocketChannel
, and AsynchronousDatagramChannel
. These asynchronous versions essentially buffer all of their operations through a thread pool and report results back through an asynchronous API. We’ll talk about the asynchronous file channel later in this chapter.
All these basic channels implement the ByteChannel
interface, designed for channels that have read and write methods like I/O streams. ByteChannel
s read and write ByteBuffer
s, however, as opposed to plain byte arrays.
In addition to these channel implementations, you can bridge channels with java.io
I/O streams and readers and writers for interoperability. However, if you mix these features, you may not get the full benefits and performance offered by the NIO package.
Most of the utilities of the java.io
and java.net
packages operate on byte arrays. The corresponding tools of the NIO package are built around ByteBuffer
s (with character-based buffer CharBuffer
for text). Byte arrays are simple, so why are buffers necessary? They serve several purposes:
They formalize the usage patterns for buffered data, provide for things like read-only buffers, and keep track of read/write positions and limits within a large buffer space. They also provide a mark/reset facility like that of java.io.BufferedInputStream
.
They provide additional APIs for working with raw data representing primitive types. You can create buffers that “view” your byte data as a series of larger primitives, such as short
s, int
s, or float
s. The most general type of data buffer, ByteBuffer
, includes methods that let you read and write all primitive types just like DataOutputStream
does for streams.
They abstract the underlying storage of the data, allowing for special optimizations by Java. Specifically, buffers may be allocated as direct buffers that use native buffers of the host operating system instead of arrays in Java’s memory. The NIO Channel
facilities that work with buffers can recognize direct buffers automatically and try to optimize I/O to use them. For example, a read from a file channel into a Java byte array normally requires Java to copy the data for the read from the host operating system into Java’s memory. With a direct buffer, the data can remain in the host operating system, outside Java’s normal memory space until and unless it is needed.
A buffer is a subclass of a java.nio.Buffer
object. The base Buffer
class is something like an array with state. It does not specify what type of elements it holds (that is for subtypes to decide), but it does define functionality that is common to all data buffers. A Buffer
has a fixed size called its capacity. Although all the standard Buffer
s provide “random access” to their contents, a Buffer
generally expects to be read and written sequentially, so Buffer
s maintain the notion of a position where the next element is read or written. In addition to position, a Buffer
can maintain two other pieces of state information: a limit, which is a position that is a “soft” limit to the extent of a read or write, and a mark, which can be used to remember an earlier position for future recall.
Implementations of Buffer
add specific, typed get and put methods that read and write the buffer contents. For example, ByteBuffer
is a buffer of bytes and it has get()
and put()
methods that read and write bytes and arrays of bytes (along with many other useful methods we’ll discuss later). Getting from and putting to the Buffer
changes the position marker, so the Buffer
keeps track of its contents somewhat like a stream. Attempting to read or write past the limit marker generates a BufferUnderflowException
or BufferOverflowException
, respectively.
The mark, position, limit, and capacity values always obey the following formula:
mark
<=
position
<=
limit
<=
capacity
The position for reading and writing the Buffer
is always between the mark, which serves as a lower bound, and the limit, which serves as an upper bound. The capacity represents the physical extent of the buffer space.
You can set the position and limit markers explicitly with the position()
and limit()
methods. Several convenience methods are provided for common usage patterns. The reset()
method sets the position back to the mark. If no mark has been set, an InvalidMarkException
is thrown. The clear()
method resets the position to 0
and makes the limit the capacity, readying the buffer for new data (the mark is discarded). Note that the clear()
method does not actually do anything to the data in the buffer; it simply changes the position markers.
The flip()
method is used for the common pattern of writing data into the buffer and then reading it back out. flip
makes the current position the limit and then resets the current position to 0
(any mark is thrown away), which saves having to keep track of how much data was read. Another method, rewind()
, simply resets the position to 0
, leaving the limit alone. You might use it to write the same size data again. Here is a snippet of code that uses these methods to read data from a channel and write it to two channels:
ByteBuffer
buff
=
...
while
(
inChannel
.
read
(
buff
)
>
0
)
{
// position = ?
buff
.
flip
();
// limit = position; position = 0;
outChannel
.
write
(
buff
);
buff
.
rewind
();
// position = 0
outChannel2
.
write
(
buff
);
buff
.
clear
();
// position = 0; limit = capacity
}
This might be confusing the first time you look at it because here, the read from the Channel
is actually a write to the Buffer
and vice versa. Because this example writes all the available data up to the limit, either flip()
or rewind()
have the same effect in this case.
As stated earlier, various buffer types add get and put methods for reading and writing specific data types. Each of the Java primitive types has an associated buffer type: ByteBuffer
, CharBuffer
, ShortBuffer
, IntBuffer
, LongBuffer
, FloatBuffer
, and DoubleBuffer
. Each provides get and put methods for reading and writing its type and arrays of its type. Of these, ByteBuffer
is the most flexible. Because it has the “finest grain” of all the buffers, it has been given a full complement of get and put methods for reading and writing all the other data types as well as byte
. Here are some ByteBuffer
methods:
byte
get
()
char
getChar
()
short
getShort
()
int
getInt
()
long
getLong
()
float
getFloat
()
double
getDouble
()
void
put
(
byte
b
)
void
put
(
ByteBuffer
src
)
void
put
(
byte
[]
src
,
int
offset
,
int
length
)
void
put
(
byte
[]
src
)
void
putChar
(
char
value
)
void
putShort
(
short
value
)
void
putInt
(
int
value
)
void
putLong
(
long
value
)
void
putFloat
(
float
value
)
void
putDouble
(
double
value
)
As we said, all the standard buffers also support random access. For each of the aforementioned methods of ByteBuffer
, an additional form takes an index; for example:
getLong
(
int
index
)
putLong
(
int
index
,
long
value
)
But that’s not all. ByteBuffer
can also provide “views” of itself as any of the coarse-grained types. For example, you can fetch a ShortBuffer
view of a ByteBuffer
with the asShortBuffer()
method. The ShortBuffer
view is backed by the ByteBuffer
, which means that they work on the same data, and changes to either one affect the other. The view buffer’s extent starts at the ByteBuffer
’s current position, and its capacity is a function of the remaining number of bytes, divided by the new type’s size. (For example, short
s consume two bytes each, float
s four, and long
s and double
s take eight.) View buffers are convenient for reading and writing large blocks of a contiguous type within a ByteBuffer
.
CharBuffer
s are interesting as well, primarily because of their integration with String
s. Both CharBuffer
s and String
s implement the java.lang.CharSequence
interface. This is the interface that provides the standard charAt()
and length()
methods. Because of this, newer APIs (such as the java.util.regex
package) allow you to use a CharBuffer
or a String
interchangeably. In this case, the CharBuffer
acts like a modifiable String
with user-configurable, logical start and end positions.
Because we’re talking about reading and writing types larger than a byte, the question arises: in what order do the bytes of multibyte values (e.g., short
s and int
s) get written? There are two camps in this world: “big endian” and “little endian.”3 Big endian means that the most significant bytes come first; little endian is the reverse. If you’re writing binary data for consumption by some native application, this is important. Intel-compatible computers use little endian, and many workstations that run Unix use big endian. The ByteOrder
class encapsulates the choice. You can specify the byte order to use with the ByteBuffer order()
method, using the identifiers ByteOrder.BIG_ENDIAN
and ByteOrder.LITTLE_ENDIAN
like so:
byteArray
.
order
(
ByteOrder
.
BIG_ENDIAN
);
You can retrieve the native ordering for your platform using the static ByteOrder.nativeOrder()
method. (We know you’re curious.)
You can create a buffer either by allocating it explicitly using allocate()
or by wrapping an existing plain Java array type. Each buffer type has a static allocate()
method that takes a capacity (size) and also a wrap()
method that takes an existing array:
CharBuffer
cbuf
=
CharBuffer
.
allocate
(
64
*
1024
);
ByteBuffer
bbuf
=
ByteBuffer
.
wrap
(
someExistingArray
);
A direct buffer is allocated in the same way, with the allocateDirect()
method:
ByteBuffer
bbuf2
=
ByteBuffer
.
allocateDirect
(
64
*
1024
);
As we described earlier, direct buffers can use operating system memory structures that are optimized for use with some kinds of I/O operations. The tradeoff is that allocating a direct buffer is a little slower and heavier weight operation than a plain buffer, so you should try to use them for longer-term buffers.
Character encoders and decoders turn characters into raw bytes and vice versa, mapping from the Unicode standard to particular encoding schemes. Encoders and decoders have long existed in Java for use by Reader
and Writer
streams and in the methods of the String
class that work with byte arrays. However, early on there was no API for working with encoding explicitly; you simply referred to encoders and decoders wherever necessary by name as a String
. The java.nio.charset
package formalized the idea of a Unicode character set encoding with the Charset
class.
The Charset
class is a factory for Charset
instances, which know how to encode character buffers to byte buffers and decode byte buffers to character buffers. You can look up a character set by name with the static Charset.forName()
method and use it in conversions:
Charset
charset
=
Charset
.
forName
(
"US-ASCII"
);
CharBuffer
charBuff
=
charset
.
decode
(
byteBuff
);
// to ascii
ByteBuffer
byteBuff
=
charset
.
encode
(
charBuff
);
// and back
You can also test to see if an encoding is available with the static Charset.isSupported()
method.
The following character sets are guaranteed to be supplied:
US-ASCII
ISO-8859-1
UTF-8
UTF-16BE
UTF-16LE
UTF-16
You can list all the encoders available on your platform using the static availableCharsets()
method:
Map
map
=
Charset
.
availableCharsets
();
Iterator
it
=
map
.
keySet
().
iterator
();
while
(
it
.
hasNext
()
)
System
.
out
.
println
(
it
.
next
()
);
The result of availableCharsets()
is a map because character sets may have “aliases” and appear under more than one name.
In addition to the buffer-oriented classes of the java.nio
package, the InputStreamReader
and OutputStreamWriter
bridge classes of the java.io
package have been updated to work with Charset
as well. You can specify the encoding as a Charset
object or by name.
You can get more control over the encoding and decoding process by creating an instance of CharsetEncoder
or CharsetDecoder
(a codec) with the Charset newEncoder()
and newDecoder()
methods. In the previous snippet, we assumed that all the data was available in a single buffer. More often, however, we might have to process data as it arrives in chunks. The encoder/decoder API allows for this by providing more general encode()
and decode()
methods that take a flag specifying whether more data is expected. The codec needs to know this because it might have been left hanging in the middle of a multibyte character conversion when the data ran out. If it knows that more data is coming, it does not throw an error on this incomplete conversion. In the following snippet, we use a decoder to read from a ByteBuffer bbuff
and accumulate character data into a CharBuffer cbuff
:
CharsetDecoder
decoder
=
Charset
.
forName
(
"US-ASCII"
).
newDecoder
();
boolean
done
=
false
;
while
(
!
done
)
{
bbuff
.
clear
();
done
=
(
in
.
read
(
bbuff
)
==
-
1
);
bbuff
.
flip
();
decoder
.
decode
(
bbuff
,
cbuff
,
done
);
}
cbuff
.
flip
();
// use cbuff. . .
Here, we look for the end of input condition on the in
channel to set the flag done
. Note that we take advantage of the flip()
method on ByteBuffer
to set the limit to the amount of data read and reset the position, setting us up for the decode operation in one step. The encode()
and decode()
methods also return a result object, CoderResult
, that can determine the progress of encoding (we do not use it in the previous snippet). The methods isError()
, isUnderflow()
, and isOverflow()
on the CoderResult
specify why encoding stopped: for an error, a lack of bytes on the input buffer, or a full output buffer, respectively.
Now that we’ve covered the basics of channels and buffers, it’s time to look at a real channel type. The FileChannel
is the NIO equivalent of the java.io.RandomAccessFile
, but it provides several core new features in addition to some performance optimizations. In particular, use a FileChannel
in place of a plain java.io
file stream if you wish to use file locking, memory-mapped file access, or highly optimized data transfer between files or between file and network channels.
A FileChannel
can be created for a Path
using the static FileChannel
open()
method.
FileSystem
fs
=
FileSystems
.
getDefault
();
Path
p
=
fs
.
getPath
(
"/tmp/foo.txt"
);
// Open default for reading
try
(
FileChannel
channel
=
FileChannel
.
open
(
p
)
{
...
}
// Open with options for writing
import
static
java
.
nio
.
file
.
StandardOpenOption
.*;
try
(
FileChannel
channel
=
FileChannel
.
open
(
p
,
WRITE
,
APPEND
,
...
)
)
{
...
}
By default, open()
creates a read-only channel for the file. We can open a channel for writing or appending and control other more advanced features such as atomic create and data syncing by passing additional options as shown in the second part of the previous example. Table 11-3 summarizes these options.
Option | Description |
---|---|
|
Open the file for read-only or write-only (default is read-only). Use both for read-write. |
|
Open the file for writing; all writes are positioned at the end of the file. |
|
Use with |
|
Use with |
|
Attempt to delete the file when it is closed or, if open, when the VM exits. |
|
Wherever possible, guarantee that write operations block until all data is written to storage. |
|
Use when creating a new file, requests the file be sparse. On filesystems where this is supported, a sparse file handles very large, mostly empty files without allocating as much real storage for empty portions. |
|
Use |
A FileChannel
can also be constructed from a classic FileInputStream
, FileOutputStream
, or RandomAccessFile
:
FileChannel
readOnlyFc
=
new
FileInputStream
(
"file.txt"
).
getChannel
();
FileChannel
readWriteFc
=
new
RandomAccessFile
(
"file.txt"
,
"rw"
)
.
getChannel
();
FileChannel
s created from these file input and output streams are read-only or write-only, respectively. To get a read/write FileChannel
, you must construct a RandomAccessFile
with read/write options, as in the previous example.
Using a FileChannel
is just like a RandomAccessFile
, but it works with a ByteBuffer
instead of byte arrays:
ByteBuffer
bbuf
=
ByteBuffer
.
allocate
(
...
);
bbuf
.
clear
();
readOnlyFc
.
position
(
index
);
readOnlyFc
.
read
(
bbuf
);
bbuf
.
flip
();
readWriteFc
.
write
(
bbuf
);
You can control how much data is read and written either by setting buffer position and limit markers or using another form of read/write that takes a buffer starting position and length. You can also read and write to a random position by supplying indexes with the read and write methods:
readWriteFc
.
read
(
bbuf
,
index
)
readWriteFc
.
write
(
bbuf
,
index2
);
In each case, the actual number of bytes read or written depends on several factors. The operation tries to read or write to the limit of the buffer, and the vast majority of the time that is what happens with local file access. The operation is guaranteed to block only until at least one byte has been processed. Whatever happens, the number of bytes processed is returned, and the buffer position is updated accordingly, preparing you to repeat the operation until it is complete if needed. This is one of the conveniences of working with buffers; they can manage the count for you. Like standard streams, the channel read()
method returns -1
upon reaching the end of input.
The size of the file is always available with the size()
method. It can change if you write past the end of the file. Conversely, you can truncate the file to a specified length with the truncate()
method.
FileChannel
s are safe for use by multiple threads and guarantee that data “viewed” by them is consistent across channels in the same VM. Unless you specify the SYNC
or DSYNC
options, no guarantees are made about how quickly writes are propagated to the storage mechanism. If you only intermittently need to be sure that data is safe before moving on, you can use the force()
method to flush changes to disk. The force()
method takes a Boolean argument indicating whether or not file metadata, including timestamp and permissions, must be written (sync or dsync). Some systems keep track of reads on files as well as writes, so you can save a lot of updates if you set the flag to false
, which indicates that you don’t care about syncing that data immediately.
As with all Channel
s, a FileChannel
may be closed by any thread. Once closed, all its read/write and position-related methods throw a ClosedChannelException
.
FileChannel
s support exclusive and shared locks on regions of files through the lock()
method:
FileLock
fileLock
=
fileChannel
.
lock
();
int
start
=
0
,
len
=
fileChannel2
.
size
();
FileLock
readLock
=
fileChannel2
.
lock
(
start
,
len
,
true
);
Locks may be either shared or exclusive. An exclusive lock prevents others from acquiring a lock of any kind on the specified file or file region. A shared lock allows others to acquire overlapping shared locks but not exclusive locks. These are useful as write and read locks, respectively. When you are writing, you don’t want others to be able to write until you’re done, but when reading, you need only to block others from writing, not reading concurrently.
The no-args lock()
method in the previous example attempts to acquire an exclusive lock for the whole file. The second form accepts a starting and length parameter as well as a flag indicating whether the lock should be shared (or exclusive). The FileLock
object returned by the lock()
method can be used to release the lock:
fileLock
.
release
();
Note that file locks are only guaranteed be a cooperative API; they do not necessarily prevent anyone from reading or writing to the locked file contents. In general, the only way to guarantee that locks are obeyed is for both parties to attempt to acquire the lock and use it. Also, shared locks are not implemented on some systems, in which case all requested locks are exclusive. You can test whether a lock is shared with the isShared()
method.
FileChannel
locks are held until the channel is closed or interrupted, so performing locks within a try
-with-resources statement will help ensure that locks are released more robustly.
try
(
FileChannel
channel
=
FileChannel
.
open
(
p
,
WRITE
)
)
{
channel
.
lock
();
...
}
The network is the soul of Java. Most of what is interesting about Java centers on the potential for dynamic, networked applications. As Java’s networking APIs have matured, Java has also become the language of choice for implementing traditional client/server applications and services. In this section, we start our discussion of the java.net
package, which contains the fundamental classes for communications and working with networked resources. Networking is a big topic, though! Chapter 12 will cover more networking goodies, focusing on Internet-related topics.
The classes of java.net
fall into two general categories: the Sockets API for working with low-level Internet protocols and higher-level, web-oriented APIs that work with uniform resource locators (URLs). Figure 11-3 shows the java.net
package.
Java’s Sockets API provides access to the standard network protocols used for communications between hosts on the Internet. Sockets are the mechanism underlying all other kinds of portable networked communications. Sockets are the lowest-level tool in the general networking toolbox—you can use sockets for any kind of communications between client and server or peer applications on the Net, but you have to implement your own application-level protocols for handling and interpreting the data. Higher-level networking tools, such as remote method invocation, HTTP, and web services are implemented on top of sockets.
These days, web services is the term for the more general technology that provides platform-independent, loosely coupled invocation of services on remote servers using web standards such as HTTP and JSON. We talk about web services in Chapter Chapter 12 when we discuss programming for the web.
In this chapter, we’ll provide some simple, practical examples of both high- and low-level Java network programming using sockets. In Chapter 12, we’ll look at the other half of the java.net
package, which lets clients work with web servers and services via URLs. It also introduces Java servlets and the tools that allow you to write your own web applications and services.
Sockets are a low-level programming interface for networked communications. They send streams of data between applications that may or may not be on the same host.
Sockets originated in BSD Unix and are, in some programming languages, hairy, complicated things with lots of small parts that can break off and endanger little children. The reason for this is that most socket APIs can be used with almost any kind of underlying network protocol. Since the protocols that transport data across the network can have radically different features, the socket interface can be quite complex.4
The java.net
package supports a simplified, object-oriented socket interface that makes network communications considerably easier. If you’ve done network programming using sockets in other languages, you should be pleasantly surprised at how simple things can be when objects encapsulate the gory details. If this is the first time you’ve come across sockets, you’ll find that talking to another application over the network can be as simple as reading a file or getting user input. Most forms of I/O in Java, including most network I/O, use the stream classes described in “Streams”. Streams provide a unified I/O interface so that reading or writing across the Internet is similar to reading or writing on the local system. In addition to the stream-oriented interfaces, the Java networking APIs can work with the Java NIO buffer-oriented API for highly scalable applications. We’ll see both in this chapter.
Java provides sockets to support three distinct classes of underlying protocols: Socket
s, DatagramSocket
s, and MulticastSocket
s. In this section, we look at Java’s basic Socket
class, which uses a connection-oriented and reliable protocol. A connection-oriented protocol provides the equivalent of a telephone conversation. After establishing a connection, two applications can send streams of data back and forth and the connection stays in place even when no one is talking. Because the protocol is reliable, it also ensures that no data is lost (resending data as necessary) and that whatever you send always arrives in the order in which you sent it.
We’ll have to leave the DatagramSocket
class, which uses a connectionless, unreliable protocol, for you to explore on your own. (You could start with Java Network Programming by Elliotte Rusty Harold, O’Reilly.) A connectionless protocol is like the postal service. Applications can send short messages to each other, but no end-to-end connection is set up in advance and no attempt is made to keep the messages in order. It’s not even guaranteed that the messages will arrive at all. A MulticastSocket
is a variation of a DatagramSocket
that performs multicasting—simultaneously sending data to multiple recipients. Working with multicast sockets is very much like working with datagram sockets.
In theory, just about any protocol can be used underneath the socket layer (old-schoolers will remember things like Novell’s IPX, Apple’s AppleTalk, etc.). But in practice, there’s only one important protocol family used on the Internet, and only one protocol family that Java supports: the Internet Protocol (IP). The Socket
class speaks TCP, the connection-oriented flavor of IP, and the DatagramSocket
class speaks UDP, the connectionless kind.
When writing network applications, it’s common to talk about clients and servers. The distinction is increasingly vague, but the side that initiates the conversation is usually considered the client. The side that accepts the request is usually the server. In the case where two peer applications use sockets to talk, the distinction is less important, but for simplicity we’ll use this definition.
For our purposes, the most important difference between a client and a server is that a client can create a socket to initiate a conversation with a server application at any time, while a server must be prepared in advance to listen for incoming conversations. The java.net.Socket
class represents one side of an individual socket connection on both the client and server. In addition, the server uses the java.net.ServerSocket
class to listen for new connections from clients. In most cases, an application acting as a server creates a ServerSocket
object and waits, blocked in a call to its accept()
method, until a connection arrives. When it arrives, the accept()
method creates a Socket
object that the server uses to communicate with the client. A server may carry on conversations with multiple clients at once; in this case, there is still only a single ServerSocket
, but the server has multiple Socket
objects—one associated with each client, as shown in Figure 11-4.
At the socket level, a client needs two pieces of information to locate and connect to a server on the Internet: a hostname (used to find the host computer’s network address) and a port number. The port number is an identifier that differentiates between multiple clients or servers on the same host. A server application listens on a prearranged port while waiting for connections. Clients use the port number assigned to the service they want to access. If you think of the host computers as hotels and the applications as guests, the ports are like the guests’ room numbers. For one person to call another, he or she must know the other party’s hotel name and room number.
A client application opens a connection to a server by constructing a Socket
that specifies the hostname and port number of the desired server:
try
{
Socket
sock
=
new
Socket
(
"wupost.wustl.edu"
,
25
);
}
catch
(
UnknownHostException
e
)
{
System
.
out
.
println
(
"Can't find host."
);
}
catch
(
IOException
e
)
{
System
.
out
.
println
(
"Error connecting to host."
);
}
This code fragment attempts to connect a Socket
to port 25 (the SMTP mail service) of the host wupost.wustl.edu. The client handles the possibility that the hostname can’t be resolved (UnknownHostException
) and that it might not be able to connect to it (IOException
). In the preceding case, Java used DNS, the standard Domain Name Service, to resolve the hostname to an IP address for us. The constructor can also accept a string containing the host’s raw IP address:
Socket
sock
=
new
Socket
(
"22.66.89.167"
,
25
);
After a connection is made, input and output streams can be retrieved with the Socket getInputStream()
and getOutputStream()
methods. The following (rather arbitrary) code sends and receives some data with the streams:
try
{
Socket
server
=
new
Socket
(
"foo.bar.com"
,
1234
);
InputStream
in
=
server
.
getInputStream
();
OutputStream
out
=
server
.
getOutputStream
();
// write a byte
out
.
write
(
42
);
// write a newline or carriage return delimited string
PrintWriter
pout
=
new
PrintWriter
(
out
,
true
);
pout
.
println
(
"Hello!"
);
// read a byte
byte
back
=
(
byte
)
in
.
read
();
// read a newline or carriage return delimited string
BufferedReader
bin
=
new
BufferedReader
(
new
InputStreamReader
(
in
)
);
String
response
=
bin
.
readLine
();
server
.
close
();
}
catch
(
IOException
e
)
{
...
}
In this exchange, the client first creates a Socket
for communicating with the server. The Socket
constructor specifies the server’s hostname (foo.bar.com) and a prearranged port number (1234). Once the connection is established, the client writes a single byte to the server using the OutputStream
’s write()
method. To send a string of text more easily, it then wraps a PrintWriter
around the OutputStream
. Next, it performs the complementary operations: reading a byte back from the server using InputStream
’s read()
method and then creating a BufferedReader
from which to get a full string of text. The client then terminates the connection with the close()
method. All these operations have the potential to generate IOException
s; our application will deal with these using the catch
clause.
After a connection is established, a server application uses the same kind of Socket
object for its side of the communications. However, to accept a connection from a client, it must first create a ServerSocket
, bound to the correct port. Let’s recreate the previous conversation from the server’s point of view:
// Meanwhile, on foo.bar.com...
try
{
ServerSocket
listener
=
new
ServerSocket
(
1234
);
while
(
!
finished
)
{
Socket
client
=
listener
.
accept
();
// wait for connection
InputStream
in
=
client
.
getInputStream
();
OutputStream
out
=
client
.
getOutputStream
();
// read a byte
byte
someByte
=
(
byte
)
in
.
read
();
// read a newline or carriage-return-delimited string
BufferedReader
bin
=
new
BufferedReader
(
new
InputStreamReader
(
in
)
);
String
someString
=
bin
.
readLine
();
// write a byte
out
.
write
(
43
);
// say goodbye
PrintWriter
pout
=
new
PrintWriter
(
out
,
true
);
pout
.
println
(
"Goodbye!"
);
client
.
close
();
}
listener
.
close
();
}
catch
(
IOException
e
)
{
...
}
First, our server creates a ServerSocket
attached to port 1234. On some systems, there are rules about which ports an application can use. Port numbers below 1024 are usually reserved for system processes and standard, well-known services, so we pick a port number outside of this range. The ServerSocket
is created only once; thereafter, we can accept as many connections as arrive.
Next, we enter a loop, waiting for the accept()
method of the ServerSocket
to return an active Socket
connection from a client. When a connection has been established, we perform the server side of our dialog, then close the connection and return to the top of the loop to wait for another connection. Finally, when the server application wants to stop listening for connections altogether, it calls the close()
method of the ServerSocket
.
This server is single-threaded; it handles one connection at a time, not calling accept()
to listen for a new connection until it’s finished with the current connection. A more realistic server would have a loop that accepts connections concurrently and passes them off to their own threads for processing or perhaps use a non-blocking ServerSocketChannel
.
The previous examples presuppose that the client has permission to connect to the server and that the server is allowed to listen on the specified socket. If you’re writing a general, standalone application, this is normally the case (and you can probably skip this section). However, untrusted applications run under the auspices of a security policy that can impose arbitrary restrictions on what hosts they may or may not talk to and whether or not they can listen for connections.
If you are going to run your own application under a security manager, you should be aware that the default security manager dissallows all network access. So in order to make network connections, you would have to modify your policy file to grant the appropriate permissions to your code (see Chapter 3 for details). The following policy file fragment sets the socket permissions to allow connections to or from any host on any nonprivileged port:
grant
{
permission
java
.
net
.
SocketPermission
"*:1024-"
,
"listen,accept,connect"
;
};
When starting the Java runtime, you can install the security manager and use this file (call it mysecurity.policy):
%java -Djava.security.manager
-Djava.security.policy=mysecurity.policy MyApplication
In the past, many networked computers ran a simple time service that dispensed their clock’s local time on a well-known port. This was a precursor of NTP, the more general Network Time Protocol.5 The next example, DateAtHost
, includes a subclass of java.util.Date
that fetches the time from a remote host instead of initializing itself from the local clock. (See Chapter 8 for a discussion of the Date
class which is still good for some uses but has been largely replaced by its newer, more flexible cousins, LocalDate
and LocalTime
.)
DateAtHost
connects to the time service (port 37) and reads four bytes representing the time on the remote host. These four bytes have a peculiar specification that we decode to get the time. Here’s the code:
//file: DateAtHost.java
import
java.net.Socket
;
import
java.io.*
;
public
class
DateAtHost
extends
java
.
util
.
Date
{
static
int
timePort
=
37
;
// seconds from start of 20th century to Jan 1, 1970 00:00 GMT
static
final
long
offset
=
2208988800L
;
public
DateAtHost
(
String
host
)
throws
IOException
{
this
(
host
,
timePort
);
}
public
DateAtHost
(
String
host
,
int
port
)
throws
IOException
{
Socket
server
=
new
Socket
(
host
,
port
);
DataInputStream
din
=
new
DataInputStream
(
server
.
getInputStream
()
);
int
time
=
din
.
readInt
();
server
.
close
();
setTime
(
(((
1L
<<
32
)
+
time
)
-
offset
)
*
1000
);
}
}
That’s all there is to it. It’s not very long, even with a few frills. We have supplied two possible constructors for DateAtHost
. Normally we’d expect to use the first, which simply takes the name of the remote host as an argument. The second constructor specifies the hostname and the port number of the remote time service. (If the time service were running on a nonstandard port, we would use the second constructor to specify the alternate port number.) This second constructor does the work of making the connection and setting the time. The first constructor simply invokes the second (using the this()
construct) with the default port as an argument. Supplying simplified constructors that invoke their siblings with default arguments is a common and useful pattern in Java; that is the main reason we’ve shown it here.
The second constructor opens a socket to the specified port on the remote host. It creates a DataInputStream
to wrap the input stream and then reads a four-byte integer using the readInt()
method. It’s no coincidence that the bytes are in the right order. Java’s DataInputStream
and DataOutputStream
classes work with the bytes of integer types in network byte order (most significant to least significant). The time protocol (and other standard network protocols that deal with binary data) also uses the network byte order, so we don’t need to call any conversion routines. Explicit data conversions would probably be necessary if we were using a nonstandard protocol, especially when talking to a non-Java client or server. In that case, we’d have to read byte by byte and do some rearranging to get our four-byte value. After reading the data, we’re finished with the socket, so we close it, terminating the connection to the server. Finally, the constructor initializes the rest of the object by calling Date
’s setTime()
method with the calculated time value.
The four bytes of the time value are interpreted as an integer representing the number of seconds since the beginning of the 20th century. DateAtHost
converts this to Java’s notion of absolute time—the count of milliseconds since January 1, 1970 (an arbitrary date standardized by C and Unix). The conversion first creates a long
value, which is the unsigned equivalent of the integer time
. It subtracts an offset to make the time relative to the epoch (January 1, 1970) rather than the century, and multiplies by 1,000 to convert to milliseconds. The converted time is used to initialize the object.
The DateAtHost
class can work with a time retrieved from a remote host almost as easily as Date
is used with the time on the local host. The only additional overhead is dealing with the possible IOException
that can be thrown by the DateAtHost
constructor:
try
{
Date
d
=
new
DateAtHost
(
"time.nist.gov"
);
System
.
out
.
println
(
"The time over there is: "
+
d
);
}
catch
(
IOException
e
)
{
...
}
This example fetches the time at the host time.nist.gov and prints its value.
We can use our newfound networking skills to extend our apple tossing game and go multiplayer. We’ll have to keep this foray simple, but you might be surprised how quickly we can get a proof of concept off the ground. While there are several mechanisms two players could use to get connected for a shared experience, our example will use the basic client/server model we’ve been discussing in this chapter. One user will start the server and the second user will be able to contact that server as the client to “join”. Once both players are connected, they’ll race to see who can clear their trees the fastest!
Let’s start by adding a menu to our game. Recall from “Menus” that menus live in a menubar and work with ActionEvent
objects much like standard buttons. We need an option for starting a server and another for joining a game at a server someone has already started. The core code for these menu items is straightforward; we can use another helper method in the AppleToss
class:
private
void
setupNetworkMenu
()
{
JMenu
netMenu
=
new
JMenu
(
"Multiplayer"
);
multiplayerHelper
=
new
Multiplayer
();
JMenuItem
startItem
=
new
JMenuItem
(
"Start Server"
);
startItem
.
addActionListener
(
new
ActionListener
()
{
public
void
actionPerformed
(
ActionEvent
e
)
{
multiplayerHelper
.
startServer
();
}
});
netMenu
.
add
(
startItem
);
JMenuItem
joinItem
=
new
JMenuItem
(
"Join Game..."
);
joinItem
.
addActionListener
(
new
ActionListener
()
{
public
void
actionPerformed
(
ActionEvent
e
)
{
String
otherServer
=
JOptionPane
.
showInputDialog
(
AppleToss
.
this
,
"Enter server name or address:"
);
multiplayerHelper
.
joinGame
(
otherServer
);
}
});
netMenu
.
add
(
joinItem
);
JMenuItem
quitItem
=
new
JMenuItem
(
"Disconnect"
);
quitItem
.
addActionListener
(
new
ActionListener
()
{
public
void
actionPerformed
(
ActionEvent
e
)
{
multiplayerHelper
.
disconnect
();
}
});
netMenu
.
add
(
quitItem
);
// build a JMenuBar for the application
JMenuBar
mainBar
=
new
JMenuBar
();
mainBar
.
add
(
netMenu
);
setJMenuBar
(
mainBar
);
}
The use of anonymous inner classes for each menu’ ActionListener
should look familiar. (Or pop ahead to “Method References” and read about how you could use a feature introduced in Java 8 for a more compact setup.) We also use the JOptionPane
discussed in “Input dialogs” to ask the second player for the name or IP address of the server where the first player is waiting. The networking logic is handled in a separate class. We’ll look at the Mutliplayer
class in more detail in the coming sections, but you can see the methods we’ll be implementing.6
As before in “Servers”, we need to pick a port and set up a socket that is listening for an incoming connection. We’ll use port 8677—“TOSS” on a phone number pad. We can create a Server
inner class in our Multiplayer
class to drive a thread ready for network communications. Hopefully the other bits in the snippet below look familiar. The reader
and writer
variables will be used to send and receive the actual game data; more on that in “The Game Protocol”.
class
Server
implements
Runnable
{
ServerSocket
listener
;
public
void
run
()
{
Socket
socket
=
null
;
try
{
listener
=
new
ServerSocket
(
gamePort
);
while
(
keepListening
)
{
socket
=
listener
.
accept
();
// wait for connection
InputStream
in
=
socket
.
getInputStream
();
BufferedReader
reader
=
new
BufferedReader
(
new
InputStreamReader
(
in
)
);
OutputStream
out
=
socket
.
getOutputStream
();
PrintWriter
writer
=
new
PrintWriter
(
out
,
true
);
// ... game protocol logic starts here
We setup our ServerSocket
and then wait for a new client inside a loop. While we only plan to play one opponent at a time, this allows us to accept subsequent clients without going through all the network setup again. To actually start the server listening the first time, we just need a new thread that uses our Server
class.
// from Multiplayer
Server
server
;
// ...
public
void
startServer
()
{
keepListening
=
true
;
// ... other game state can go here
server
=
new
Server
();
serverThread
=
new
Thread
(
server
);
serverThread
.
start
();
}
We keep a reference to the instance of Server
in our Multiplayer
class so that we have ready access to shutting down the connections if the user selects the “disconnect” option from the menu like so:
// from Multiplayer
public
void
disconnect
()
{
disconnecting
=
true
;
keepListening
=
false
;
// Are we in the middle of a game and regularly checking these flags?
// If not, just close the server socket to interrupt the blocking accept() method.
if
(
server
!=
null
&&
keepPlaying
==
false
)
{
server
.
stopListening
();
}
// ... clean up other game state here
}
The keepPlaying
flag is mainly used once we’re in our game loop, but it comes in handy above, too. If we have a valid server
reference but we’re not currently playing a game (so keepPlaying
is false) we know to shut down the listener socket. The stopListening()
method in the Server
inner class is straightforward:
public
void
stopListening
()
{
if
(
listener
!=
null
&&
!
listener
.
isClosed
())
{
try
{
listener
.
close
();
}
catch
(
IOException
ioe
)
{
System
.
err
.
println
(
"Error disconnecting listener: "
+
ioe
.
getMessage
());
}
}
}
The setup and teardown of the client side is similar—without the listening ServerSocket
of course. We’ll mirror the Server
inner class with a Client
inner class and build a smart run()
method to implement our client logic.
class
Client
implements
Runnable
{
String
gameHost
;
boolean
startNewGame
;
public
Client
(
String
host
)
{
gameHost
=
host
;
keepPlaying
=
false
;
startNewGame
=
false
;
}
public
void
run
()
{
try
(
Socket
socket
=
new
Socket
(
gameHost
,
gamePort
))
{
InputStream
in
=
socket
.
getInputStream
();
BufferedReader
reader
=
new
BufferedReader
(
new
InputStreamReader
(
in
)
);
OutputStream
out
=
socket
.
getOutputStream
();
PrintWriter
writer
=
new
PrintWriter
(
out
,
true
);
// ... game protocol logic starts here
We use a constructor for Client
to pass the name of the server we will connect to and rely on the common gamePort
variable used by Server
to setup the listening socket. We use the “try with resource” technique discussed in “Try with Resources” to create our socket and make sure it gets cleaned up when we’re done. Inside that resource try block, we create our reader
and writer
instances for the client’s half of the conversation as shown in Figure 11-5.
To get this all going, we’ll add another handy method to our Multiplayer
helper class.
// from Multiplayer
public
void
joinGame
(
String
otherServer
)
{
clientThread
=
new
Thread
(
new
Client
(
otherServer
));
clientThread
.
start
();
}
There’s no need for a separate disconnect()
method as the state variables used by the server can also drive the polite shutdown of the client. For the client, the server
reference will be null
so there won’t be any attempt to shutdown a nonexistant listener.
You likely noticed we left out the bulk of the run()
method for both the Server
and Client
classes. After we build and connect our data streams, the remaining work is all about collaboratively sending and receiving information about the state of our game. This structured communication is the game’s protocol. Every network service has some protocol or other. Think of the “P” in HTTP. Even our simple DateAtHost
example uses a (very simple) protocol so that clients and servers know who is expected to talk and who must listen at any given moment. If each of the two sides end up waiting for the other side to say something (e.g. both the server and the client are blocking on a reader.readLine()
call) then the connection will appear to hang.
Managing those communication expectations is the core of any protocol, but what to say and how to respond are also important. Indeed, this portion of a protocol often requires the most work on the part of the developer. Part of the difficulty is that you really need both sides to test your work as you go. You can’t test a server without a client and vice versa. Building up both sides as you go can feel tedious but it is worth the extra effort. As with other debugging advice, fixing a small incremental change is much simpler than figuring out what might be wrong with a large block of code.
In our example, we will have the server steer the conversation. This choice is arbitrary—we could have used the client or we could have built a fancier foundation and allowed both the client and the server to be in charge of certain things simultaneously. But with the “server in charge” decision made, we can try a very simple first step in our protocol. We will have the server send a “NEW_GAME” command and then wait for the client to respond with an “OK” answer. The server side code might look like so:
// Create a new game with the client
writer
.
println
(
"NEW_GAME"
);
// If the client agrees, send over the location of the trees
String
response
=
reader
.
readLine
();
if
(
response
!=
null
&&
response
.
equals
(
"OK"
))
{
System
.
out
.
println
(
"Starting a new game!"
)
// ... write tree data here
}
else
{
System
.
err
.
println
(
"Unexpected start response: "
+
response
);
System
.
err
.
println
(
"Skipping game and waiting again."
);
keepPlaying
=
false
;
}
If we get the expected “OK” response, we can proceed with setting up a new game and sharing the tree locations with our opponent—but more on that in a minute. The corresponding client-side code for this first step flows similarly:
// We expect to see the NEW_GAME command first
String
response
=
reader
.
readLine
();
// If we don't see that command, disconnect and quit
if
(
response
==
null
||
!
response
.
equals
(
"NEW_GAME"
))
{
System
.
err
.
println
(
"Unexpected initial command: "
+
response
);
System
.
err
.
println
(
"Disconnecting"
);
writer
.
println
(
"DISCONNECT"
);
return
;
}
// Yay! We're going to play a game. Acknowledge this command
writer
.
println
(
"OK"
);
If you compile and run the game at this point, you could start your server from one system and then join that game from a second system. (You could also just launch a second copy of the game from a separate terminal window. In that case, the “other host” would be the networking keyword localhost
.) Almost immediately after joining from the second game instance, you should see the “Starting a new game!” confirmation printed in the terminal of the first game. Congratulations! You’re on your way to designing a game protocol. Let’s keep going.
Once we know we’re starting a new game, we need to even the playing field—quite literally. The server will tell the game to build a new field and then it can ship the coordinates of all the new trees to the client. The client, in turn, can accept all the incoming trees and place them on a clean field. Once the server has sent all of the trees, it can send a “START” command and play can begin. We’ll stick to using strings to communicate our messages. Here’s one way we can pass our tree details to the client:
gameField
.
setupNewGame
();
for
(
Tree
tree
:
gameField
.
trees
)
{
writer
.
println
(
"TREE "
+
tree
.
getPositionX
()
+
" "
+
tree
.
getPositionY
());
}
// ...
// Start the action!
writer
.
println
(
"START"
);
response
=
reader
.
readLine
();
keepPlaying
=
response
.
equals
(
"OK"
);
On the client side, we can call readLine()
in a loop for “TREE” lines until we see the “START” line like so (with a little error handling thrown in):
// And now gather the trees and set up our field
gameField
.
trees
.
clear
();
response
=
reader
.
readLine
();
while
(
response
.
startsWith
(
"TREE"
))
{
String
[]
parts
=
response
.
split
(
" "
);
int
x
=
Integer
.
parseInt
(
parts
[
1
]);
int
y
=
Integer
.
parseInt
(
parts
[
2
]);
Tree
tree
=
new
Tree
();
tree
.
setPosition
(
x
,
y
);
gameField
.
trees
.
add
(
tree
);
response
=
reader
.
readLine
();
}
if
(!
response
.
equals
(
"START"
))
{
// Hmm, we should have ended the list of trees with a START,
// but didn't. Bail out.
System
.
err
.
println
(
"Unexpected start to the game: "
+
response
);
System
.
err
.
println
(
"Disconnecting"
);
writer
.
println
(
"DISCONNECT"
);
return
;
}
else
{
// Yay again! We're starting a game. Acknowledge this command
writer
.
println
(
"OK"
);
keepPlaying
=
true
;
gameField
.
repaint
();
}
At this point both games should have the same trees and can begin playing to clear them. The server will enter a polling loop and send the current score twice a second. The client will reply with its current score. Note that there are certainly other options for how to share changes in the score. While polling is straightforward, more advanced games or games that require more immediate feedback regarding remote players will likely use more direct communication options. For now, we mainly want to concentrate on a good network back-and-forth so polling keeps our code simpler.
The server should keep sending the current score until the local player has cleared out all of the trees or we see a game-ending response from the client. We’ll need to parse the client’s response to update the other player’s score and watch for them ending the game or simply disconnecting. That loop would look something like this:
while
(
keepPlaying
)
{
try
{
if
(
gameField
.
trees
.
size
()
>
0
)
{
writer
.
(
"SCORE "
);
}
else
{
writer
.
(
"END "
);
keepPlaying
=
false
;
}
writer
.
println
(
gameField
.
getScore
(
1
));
response
=
reader
.
readLine
();
if
(
response
==
null
)
{
keepPlaying
=
false
;
disconnecting
=
true
;
}
else
{
String
parts
[]
=
response
.
split
(
" "
);
switch
(
parts
[
0
])
{
case
"END"
:
keepPlaying
=
false
;
case
"SCORE"
:
gameField
.
setScore
(
2
,
parts
[
1
]);
break
;
case
"DISCONNECT"
:
disconnecting
=
true
;
keepPlaying
=
false
;
break
;
default
:
System
.
err
.
println
(
"Warning. Unexpected command: "
+
parts
[
0
]
+
". Ignoring."
);
}
}
Thread
.
sleep
(
500
);
}
catch
(
InterruptedException
e
)
{
System
.
err
.
println
(
"Interrupted while polling. Ignoring."
);
}
}
And again, the client will mirror these actions. Fortunately for the client, it is just reacting to the commands coming from the server. We don’t need a separate polling mechanism here. We block waiting to read a line, parse it, and then build our response.
while
(
keepPlaying
)
{
response
=
reader
.
readLine
();
String
[]
parts
=
response
.
split
(
" "
);
switch
(
parts
[
0
])
{
case
"END"
:
keepPlaying
=
false
;
case
"SCORE"
:
gameField
.
setScore
(
2
,
parts
[
1
]);
break
;
case
"DISCONNECT"
:
disconnecting
=
true
;
keepPlaying
=
false
;
break
;
default
:
System
.
err
.
println
(
"Unexpected game command: "
+
response
+
". Ignoring."
);
}
if
(
disconnecting
)
{
// We're disconnecting or they are. Acknowledge and quit.
writer
.
println
(
"DISCONNECT"
);
return
;
}
else
{
// If we're not disconnecting, reply with our current score
if
(
gameField
.
trees
.
size
()
>
0
)
{
writer
.
(
"SCORE "
);
}
else
{
keepPlaying
=
false
;
writer
.
(
"END "
);
}
writer
.
println
(
gameField
.
getScore
(
1
));
}
}
When a player has cleared all of their trees, they send (or respond with) an “END” command that includes their final score. At that point, we ask if the same two players want to play again. If so, we can continue using the same reader
and writer
instances for both the server and the client. If not, we’ll let the client disconnect and the server will go back to listening for another player to join.
// If we're not disconnecting, ask about playing again with the same player
if
(!
disconnecting
)
{
String
message
=
gameField
.
getWinner
()
+
" Would you like to ask them to play again?"
;
int
myPlayAgain
=
JOptionPane
.
showConfirmDialog
(
gameField
,
message
,
"Play Again?"
,
JOptionPane
.
YES_NO_OPTION
);
if
(
myPlayAgain
==
JOptionPane
.
YES_OPTION
)
{
// If they haven't disconnected, ask if they want to play again
writer
.
println
(
"PLAY_AGAIN"
);
String
playAgain
=
reader
.
readLine
();
if
(
playAgain
!=
null
)
{
switch
(
playAgain
)
{
case
"YES"
:
startNewGame
=
true
;
break
;
case
"DISCONNECT"
:
keepPlaying
=
false
;
startNewGame
=
false
;
disconnecting
=
true
;
break
;
default
:
System
.
err
.
println
(
"Warning. Unexpected response: "
+
playAgain
+
". Not playing again."
);
}
}
}
}
And one last reciprocal bit of code for the client:
if
(!
disconnecting
)
{
// Check to see if they want to play again
response
=
reader
.
readLine
();
if
(
response
!=
null
&&
response
.
equals
(
"PLAY_AGAIN"
))
{
// Do we want to play again?
String
message
=
gameField
.
getWinner
()
+
" Would you like to play again?"
;
int
myPlayAgain
=
JOptionPane
.
showConfirmDialog
(
gameField
,
message
,
"Play Again?"
,
JOptionPane
.
YES_NO_OPTION
);
if
(
myPlayAgain
==
JOptionPane
.
YES_OPTION
)
{
writer
.
println
(
"YES"
);
startNewGame
=
true
;
}
else
{
// Not playing again so disconnect.
disconnecting
=
true
;
writer
.
println
(
"DISCONNECT"
);
}
}
}
Table 11-4 summarizes our simple protocol.
Server Command | Args (optional) | Client Response | Args (optional) |
---|---|---|---|
NEW_GAME |
OK |
||
TREE |
x y |
||
START |
OK |
||
SCORE |
score |
SCORE |
score |
END |
score |
SCORE |
score |
PLAY_AGAIN |
YES |
||
DISCONNECT |
We could spend much more time on our game. We could expand the protocol to allow multiple opponents. We could change the objective to clear the trees and destroy your opponent. We could make the protocol more bidirectional allowing the client to initiate some of the updates. We could use alternate lower-level protocols supported by Java such as UDP rather than TCP. Indeed, there are entire books devoted to games, or to network programming, or to programming networked games!
As always we have to leave those explorations to you, but hopefully you have a sense of Java’s strong support for networked applications. If you do explore some of those advanced topics, you’ll undoubtedly start with a web search. The World Wide Web is perhaps the greatest example of a networked environment. Given Java’s broad support for networking, it should come as no surprise that Java has some great features devoted to working with the web. The next chapter introduces some of those features for both the client or front end and the server or back end.
1 While NIO was introduced with Java 1.4—so not very new anymore—it was newer than the original, basic package and the name has stuck.
2 Standard error is a stream that is usually reserved for error-related text messages that should be shown to the user of a command-line application. It is differentiated from the standard output, which often might be redirected to a file or another application and not seen by the user.
3 The terms big endian and little endian come from Jonathan Swift’s novel Gulliver’s Travels, where it denoted two camps of Lilliputians: those who eat their eggs from the big end and those who eat them from the little end.
4 For a discussion of sockets in general, see Unix Network Programming by Richard Stevens (Prentice-Hall).
5 Indeed, the publically available site we use from NIST strongly encourages users to upgrade. See the introductory notes at https://tf.nist.gov/tf-cgi/servers.cgi for more information.
6 The game code for this chapter (in the ch11/game
folder) contains the setupNetworkMenu()
method but the anonymous inner action listeners just pop up an info dialog to indicate which menu item was selected. You get to build the Multiplayer
class and call the actual multiplayer methods! But do feel free to check out the completed game—including the networking parts— in the top-level game
folder of the examples for the book.