Tango offers a broad range of functionality for handling input/output (I/O). In this chapter, we introduce four principal features of the tango.io
package:
Console I/O
Stream I/O
Network I/O
File handling.
First, let's get a couple of definitions out of the way. If you already familiar with the notion of streams, feel free to skip ahead.
Tango I/O is primarily stream-oriented. A stream represents a contiguous flow of data without any particular format or discernible feature. Applications may subsequently apply type or semantic structure (meaning) to the data, perhaps treating it as a set of records or lines of text. However, the flow is just a meaningless set of bytes at the raw stream level. Perhaps one common characteristic of a stream is that it usually terminates at some point.
One end of each stream in a Tango program typically connects to some external device, such as a file, network connection, or console. Within the Tango library, these stream end points are known as conduits, and each plays host to both an input and output stream for the specific device. If you conceptualize that each device has a pair of streams attached, you have the notion of a conduit all squared away.
The console is where text output from your program will often be displayed. Console support in Tango is stream-oriented, and includes a high-level type-conversion layer along with a lightweight UTF-8 interface. The former includes Stdout
and Stderr
; the latter consists of Cout
, Cerr
, and Cin
. We discuss each of them here, starting with the lightweight interface.
Perhaps the simplest way to display text on the console is via either Cout
or Cerr
. These are predefined entities in tango.io.Console
that route char[]
content to the appropriate output device. Here's an example:
import tango.io.Console; Cout ("the quick brown fox").newline;
The newline
appended in this example causes the output to be flushed. Line breaks may be embedded within the literal also, using the traditional
syntax, though these are simply passed along by Tango; no explicit processing is performed. Console output is buffered, so without a newline
, the text would not be sent to the destination immediately. Where line breaks are inappropriate, you can achieve immediate flushing to the console by using Cout("hello").flush
or the shortcut variant Cout("hello")()
, where the empty parentheses indicate a flush operation.
Console methods return a chaining reference, enabling the following style for those who prefer it:
auto action = "jumps over the lazy dog"; Cout ("the quick brown fox ")(action).newline;
The console does not directly support formatted output. Each argument is processed independently and rendered to the output in a simple left-to-right order. You can use a Layout
or Print
instance to generate formatted Cout
output, which is just what Stdout
does on your behalf.
Object references may be passed to Cout
, where the object will be queried to obtain a literal:
auto o = new Object; Cout ("the name of Object is ")(o).newline;
Console elements expose their underlying stream in order to permit more expressive usage. For example, the following is a shortcut for copying a text file to the console:
import tango.io.Console; import tango.io.stream.FileStream; Cout.stream.copy (new FileInput ("myfile"));
The Cout.copy
pattern is often used when communicating with other executing processes where, for example, the output of a child process is relayed to the console of the parent.
Console input is enabled in a manner similar to Cout
, but using the predefined entity Cin
instead. Tango waits for some input to be available and returns all of it to the caller. With interactive console usage, this is usually one line of input, returned by the operating system whenever the Enter key is pressed. Here is an example:
import tango.io.Console; Cout ("What is your name? ").flush; auto name = Cin.readln; Cout ("Hello ")(name).newline;
Where console redirection has been applied (via a pipe or equivalent mechanism), Cin
will relay large quantities of redirected input back to the caller for each invocation. You can split the resultant input stream into lines of text, for example, by applying a LineStream
to the stream exposed by Cin
:
import tango.io.Console; import tango.io.stream.LineStream; foreach (line; new LineInput(Cin.stream)) Cout (line).newline;
As you can see, this shows the console being used purely as a streaming input source. Cin
already has embedded support for reading discrete lines of text via its readln
and copyln
methods, but a LineInput
can often be convenient. Consider using a stream iterator from the tango.text.stream
package when you need to split text on boundaries other than lines.
In order to ensure platform independence, and to accommodate console redirection across a variety of use cases, all console-based interaction should be UTF-8 only. The console device interface handles potential conversion between UTF-8 and a platform-specific encoding. Instances of IOException
are thrown when the underlying operating system determines an error occurred, such as when redirection ran into a problem with a remote file.
You can transfer console input to output by copying Cin
to Cout
as follows: Cout.stream.copy (Cin.stream)
.
Stdout
is a general-purpose formatter, sitting atop Cout
. There's also a Stderr
tied to Cerr
, and both are predefined within the tango.io.Stdout
module. Where Cout
supports UTF-8 only, Stdout
handles a wide range of types, converting from native representation to text for display purposes and translating text represented by UTF-16 and/or UTF-32 into the format expected by the console.
The core formatting functionality is provided by tango.text.convert.Layout
and exposed through Stdout
via a number of convenience methods. These methods return a chaining reference (like much of the library does) and accept multiple arguments in the conventional variable argument (vararg) style. For example, a formatted vararg call looks like this:
import tango.io.Stdout; Stdout.format ("{} green bottles, sitting on the wall", 10).newline;
Flushing the output without a line break is similar to Cout
, using either flush
or an empty set of parentheses like so:
Stdout.format ("{} green bottles, sitting on the wall", 10) ();
There is also a variant to append a newline
, which itself implies a flush:
Stdout.formatln ("{} green bottles, sitting on the wall", 10);
A newline
implies a flush when console I/O is not redirected, causing immediate rendering of Stdout
text to a typical console. I/O redirection inhibits this automatic flush behavior, in order to increase throughput. Cout
operates in a similar manner.
Table 7-1 lists some of the variations available for Stdout
, with the results shown on the right.
Table 7.1. Some Stdout Variations
Usage Style | Result |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Stdout.format
supports a variety of options. Unlike C's printf
, Stdout
already knows the type of each argument provided, rendering the printf
type specifier obsolete. Instead, Stdout
supports an optional format descriptor. A portion of the descriptor is generic across all types, while the rest is specific to a particular argument type. It has the following structure:
'{' [index] [',' alignment] [':' format] '}'
The curly braces are required. alignment
indicates a minimum layout width that should be negative to indicate left alignment. The colon is a required prefix for any type-specific option. An optional index
indicates which argument is being addressed. The latter can become important when considering internationalization and localization, where the format strings (including embedded indices) might be externalized and adjusted for locale specifics. Table 7-2 shows some examples of formatted output.
Table 7.2. Formatting Examples
Format Syntax | Result |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Although Stdout.format("abc ", x)
does not reference the provided argument x
, it does not produce an error, since there are cases where dropping an argument is legitimate.
Like Cout
and Cerr
, Stdout
and Stderr
expose the underlying output stream, which may be used directly where appropriate. Revisiting a Cout
example:
import tango.io.Stdout; import tango.io.stream.FileStream; auto file = new FileInput ("myfile"); Stdout.stream.copy (file);
Or, you could sidestep all formatting conversion and append content directly to the underlying stream:
Stdout.stream.write ("the quick brown fox jumps over the lazy dog");
Stdout
also exposes the Layout
instance in use, so that you can invoke it directly. For example, it can sometimes be useful to construct an interim array of formatted output:
char[128] output; auto string = Stdout.layout.sprint (output, "{} green bottles", 10);
It is also possible to substitute the attached Layout
instance, which is useful for configuring Stdout
and Stderr
with an alternate formatting handler. For example, Tango provides a locale-configurable layout, which can be substituted for the default one:
import tango.io.Stdout; import tango.text.locale.Locale; Stdout.layout = new Locale (Culture.getCulture ("fr-FR"));
As a layout replacement, Locale
supports additional, regional-specific formatting options for currency, decimal and numeric representation, and date/time formatting. Both Layout
and Locale
are discussed in Chapter 6.
Stdout
functionality is supported via a module named tango.io.Print
, which may be used to bind similar functionality to streams other than those tied to Cout
and Cerr
(such as a file or a network connection).
Issues arising during formatting are generally injected into the output stream in place of a formatted argument and delimited by a pair of braces. To demonstrate, try printing a null
reference:
import tango.io.Stdout; Object o = null; Stdout.formatln ("using a {} object", o);
The following should appear on the console:
using a {null} object
Streams are utilized throughout tango.io
, so a little knowledge of how they operate will put you in good stead. The two stream types represent input from a device and output to a device, named InputStream
and OutputStream
, respectively. Both have a simple interface, where an output stream is represented by five methods and an input stream by just four. Both stream types have a close
method, which closes the stream, and both have a conduit
method to return a representation of the attached device.
An input stream has two more methods called read
and clear
. The former reads content into a supplied array and returns the number of bytes consumed from the input device. The latter ensures that any buffered input data is cleared away.
An output stream has three additional methods: write
, flush
, and copy
. You've already seen copy
applied a few times (from prior examples in the "Console I/O" section) where an input stream was being copied to the console output stream. Connecting streams together in this way is quite common, and copy
is there to handle the chore efficiently. The other two methods are for writing an array of content to an attached device and for flushing buffered output.
Like the read
method, write
returns the number of bytes consumed by the output device. In both cases, the quantity consumed may be less than offered; that is, read
may not fill the provided array entirely, and write
may not consume all of its provided data in one gulp. In particular, both methods will return the reserved value Eof
when a stream concludes.
One of the more useful aspects of streams is that they can be connected together to form processing chains, or conversion chains, utilizing a design called the decorator pattern. In many cases, as you'll see shortly, a predefined chain will be available for use. You can easily construct your own chains (and your own stream derivatives) if the need arises.
With all that out of the way, let's dig into the tango.io.stream
facilities. For the most part, they represent predecorated streams, or wrappers around other I/O functionality. We'll start with file access, since that's a common need, and quickly step through the other modules. Here, we copy a file to the console output:
import tango.io.Console, tango.io.stream.FileStream; auto input = new FileInput ("myfile"); Cout.stream.copy (input); input.close;
This snippet copies console input to a file:
import tango.io.Console, tango.io.stream.FileStream; auto output = new FileOutput ("myfile");
output.copy (Cin.stream); output.close;
Note that you should always close the stream when finished with it. In this next example, we use TextFileStream
to read lines of text, one at a time, and write each to the console:
import tango.io.Console, tango.io.stream.TextFileStream; auto input = new TextFileInput ("myfile"); foreach (line; input) Cout(line).flush; input.close;
Here, we write formatted text to an output file, again using TextFileStream
. The formatting functionality mirrors that of Stdout
:
import tango.io.Console, tango.io.stream.TextFileStream; auto output = new TextFileOutput ("myfile"); output.formatln ("{} green bottles", 10); output.flush.close;
Flushing an output stream before closing it is generally a good idea, unless the stream is being discarded. In this next variation, an input stream is copied explicitly, so you can see how to use the read
function:
import tango.io.Console, tango.io.stream.FileStream; char[1024] tmp; int len; auto input = new FileInput("myfile"); while ((len = input.read(tmp)) != input.Eof) Cout (tmp[0 .. len]).flush; input.close;
It is certainly less work to use copy
instead! An alternative would be to decorate the stream with a filter that consumes everything it is asked to read
or write
:
import tango.io.Console, tango.io.stream.FileStream, tango.io.stream.GreedyStream;
auto output = new GreedyOutput (new FileOutput("myfile")); output.write ("the quick brown fox jumps over the lazy dog"); output.flush.close;
We wrapped the file stream inside a greedy stream (so they are now chained together), and each request to the greedy stream is relayed to its contained file stream. It's called greedy because it consumes everything asked of it, instead of consuming only what it can. We still flush the output before closing, though in this case, it is not strictly necessary.
This next example demonstrates binary I/O using DataFileStream
, which also supports random access (file seeking). Because of the random-access aspect, this code requires a bit of knowledge not discussed until later in this chapter, in the "Accessing File Content Using FileConduit" section, but we'll go ahead and show how it operates anyway:
import tango.io.FileConduit, tango.io.stream.DataFileStream; auto file = new FileConduit ("myfile", FileConduit.ReadWriteCreate); auto input = new DataFileInput (file); auto output = new DataFileOutput (file); int x=10, y=20; output.putInt(x); output.putInt(y); output.seek (0); x = input.getInt; y = input.getInt; file.close;
The interesting elements here include the FileConduit
(a device) representing a seekable read-write file, and the data-oriented streams that operate on it.
DateFileStream
also buffers the I/O (you may set the buffer size via the streams), and the seek
method flushes the output before adjusting the file location.
Lastly, the following is an example of a stream decorator to read and write maps (hash tables) of name/value pairs. We use it to write to a text file and then read the pairs back again into another map. The content of the file will be a series of name=value
tuples, separated by a line ending.
import tango.io.stream.MapStream,
tango.io.stream.FileStream; char[][char[]] settings; settings ["server-port"] = "8080"; settings ["log-file"] = "log/all.log"; settings ["multicast-address"] = "225.0.0.9"; auto output = new MapOutput (new FileOutput("myfile")); output.append(settings).flush.close; char[][char[]] copy; auto input = new MapInput (new FileInput("myfile")); input.load (copy); input.close;
There are more streaming facilities where those just discussed came from, and they can be applied directly to any old stream, rather than just a file. For example, you can also attach MapStream
onto the console, a network connection, or a memory-based stream. The same goes for LineStream
, FormatStream
, GreedyStream
, TypedStream
, UtfStream
, SnoopStream
, DigestStream
, DataStream
, and EndianStream
. These are all decorators intended for chaining onto another stream, and we'll briefly describe a handful of them here.
FormatStream
provides an output formatter just like Stdout
, but for binding to any other stream. The TextFileOutput
discussed earlier derives from FormatStream
, which itself is derived from tango.io.Print
.
TypedStream
treats streams as a set of (templated) types. In other words, it enables you to address a stream as a set of discrete characters, integers, structs, or whatever the stream comprises.
UtfStream
converts from one UTF encoding to another. For instance, you can use it to convert a UTF-8 stream into a UTF-32 stream and vice versa. Use it in conjunction with an EndianStream
in order to add endian (byte-order) conversion where applicable.
DigestStream
attaches a digest to a provided stream, and updates the digest as data flows by. The tango.io.digest
package contains a selection of message-digest algorithms, including MD2, MD4, MD5, SHA0, SHA1, SHA01, SHA256, SHA512, and Tiger. These can be hooked to DigestStream
as a convenient means of stream processing.
SnoopStream
generates debug messages describing the operations being performed on it. By doing so, SnoopStream
can provide introspection into stream behavior; it snoops on the data flow.
BufferStream
provides the equivalent of a cache for streaming content, enabling an underlying device to be read and written in large chunks instead of discrete data elements. This can optimize many common operations, such as token parsing or output concatenation, and can support efficient mapping of data-record content into program memory.
BufferStream
itself is a shallow wrapper around a generic buffering module called Buffer
, which can be used directly also. In addition to supporting the pedestrian stream operations, Buffer
exposes a range of functionality from simple append operations to more prosaic producer/consumer balancing between streams of different bandwidths. Buffer
also supports multiple downstream consumers. For instance, several contiguous stream iterators may be attached at one time (see the next section), and Buffer
will maintain common state across them. However, typical usage is just regular buffering of either file or network I/O:
import tango.io.stream.FileStream, tango.io.stream.BufferStream; auto input = new BufferInput (new FileInput("myfile"));
Buffer
is data type-agnostic and operates as a smart array, flushing and/or refilling itself via connected upstreams as necessary. Buffer
is actually a conduit instance for a pseudodevice, and exposes both an input stream and an output stream. You can thus connect various stream decorators in order to imply structure over the content therein.
When using TextFileStream
, DataFileStream
, or various other decorators, buffering will be initiated on your behalf. In other cases, you may find that manually applying BufferStream
is a more convenient option (perhaps for your own decorators). The wrappers discussed exist so you generally don't need to glue the various pieces together yourself, yet Buffer
is the mechanism they apply under the covers.
While a Buffer
is often attached to an upstream device (such as a network connection or file), it can happily be used in stand-alone fashion as a memory-based accumulator. There is one distinction between the two usage scenarios: without an upstream device (such as a file), the buffer cannot be automatically flushed when filled to capacity, so an exception will be raised when this occurs. You can use an expanding buffer called GrowBuffer
to handle cases where no upstream device is connected and output content is expected to grow in size. Here's an example of both buffer variations used in a stand-alone mode:
import tango.io.Buffer, tango.io.Console; auto buffer = new Buffer ("Hello World "); Cout.stream.copy (buffer);
buffer = new GrowBuffer; buffer.append("Hello ").append(" World "); Cout.stream.copy (buffer);
Another Buffer
variant, MappedBuffer
, wraps operating system facilities to expose memory-mapped files. The buffer memory is mapped directly onto a (usually) large file, which you can then treat as just another stream. You can even treat MappedBuffer
content directly as a native array.
Tango has a set of classes to split streaming text input into elements bounded by a specific pattern. These classes reside in tango.text.stream
and are templated for char
, wchar
, and dchar
data types. They include an iterator for producing lines of text based on finding embedded end-of-line markers, as well as iterators for isolating text patterns, delimiters, quoted strings, and so on.
For example, you might convert an input stream of characters into a series of lines. We use a Buffer
instance here, but you can use any input stream:
import tango.io.Console; import tango.text.stream.LineIterator; auto input = new Buffer("Hello World "); foreach (line; new LineIterator!(char)(input)) Cout(line).newline;
If this looks familiar, then it's likely because the LineStream
example shown previously is a simple wrapper around the templated class.
Each stream iterator will generate an IOException
when the size of a single element is larger than the containing Buffer
. This might be because that buffer is too small or an element is overly large. You could increase the buffer size to accommodate very large elements (a typical I/O buffer is 8KB or 16KB).
Iterator results are usually aliased directly from an underlying buffer, thus avoiding considerable heap activity where an application doesn't need it. When the resultant element is to be retained by the application, it should be copied before iteration continues (using .dup
or equivalent).
In this section, you'll get a general overview of network support and how it fits into the overall I/O framework. The support provided by Tango is intended to expose aspects of network programming in a simple and straightforward manner, while providing the means to reach underlying mechanisms where necessary.
A number of excellent generic network-programming tutorials are available on the Internet. To learn more, try http://beej.us/guide/bgnet
.
An address/port pair describes an Internet location, and this is encapsulated within an InternetAddress
. The address may refer to the hosting computer (a local address) or to some machine on the other side of the globe.
Local addresses are often used by applications acting as a destination for other machines (as a listener, or server), whereas remote addresses are generally used to enable a program (client) to make resource requests of a server machine. While the address and port of a remote location must be described explicitly, both may be implied for a local location by leaving them unspecified. Sometimes it is convenient to use the hosting machine as both a listening server and a client. In such cases, the remote address would effectively map to the local address (directly or indirectly).
InternetAddress
is a required parameter for a number of other network-oriented classes, and it is simple to construct. Creating an Internet reference to the Digital Mars web server, for example, can be accomplished like so, using the domain name along with the port number reserved for HTTP servers:
import tango.net.InternetAddress; auto addr = new InternetAddress ("www.digitalmars.com", 80);
Alternatively, this could be constructed using a numeric address:
auto addr = new InternetAddress ("65.204.18.192", 80);
A third, and often convenient, alternative is to append the port number to the address descriptor itself:
auto addr = new InternetAddress ("www.digitalmars.com:80");
Each of these alternatives maps to the same remote location. On the server end, to create a local address for listening purposes, you could use this variation with no arguments:
auto addr = new InternetAddress;
With no explicit arguments, both the address and port will be assigned and configured on your behalf. To request a specific port (such as port 80), use this signature instead:
auto addr = new InternetAddress (80);
Each of these address instances represents potential connectivity within your program, since there's no connection made at this time. It may turn out that a remote location is not listening or is otherwise unavailable, or it may be that a local listening address is already in use by some other server. These issues do not arise within InternetAddress
itself, since it is purely an encapsulation of attributes. Instead, they may surface when using bind
or connect
at a later stage.
Tango wraps its underlying network functionality in a cross-platform faade named Socket
, and exposes it as an attribute of higher-level constructs such as SocketConduit
. As such, Socket
represents a low-level network interface in Tango.
Within Socket
, you'll find methods to directly manipulate the socket transport layer and associated controls. For example, switching between blocking and nonblocking sockets remains in the realm of Socket
, as does domain-name lookup, select
requests, socket-connectivity options, and other lower-level machinations.
In general, it is preferable to rely on higher-level constructs such as SocketConduit
and ServerSocket
to configure and manipulate the underlying socket on your behalf. However, the higher levels provide access to the Socket
instance for use when you need to dig a little deeper.
SocketConduit
represents the gateway to a TCP/IP network. It is oriented toward blocking operations (for example, it will potentially stall while waiting for input to arrive), although it can operate in a limited nonblocking configuration. It is a true instance of a Tango conduit and thus exposes both an input stream and an output stream for general usage. For instance, you could copy file content to a network location in the following manner:
import tango.net.SocketConduit, tango.net.InternetAddress; import tango.io.stream.FileStream; auto host = new SocketConduit; host.connect (new InternetAddress ("www.myhost.com", myport)); host.output.copy (new FileInput ("myfile"));
When reading from a network, you can use a similar approach to copy content to a file, to the console, to another instance of SocketConduit
, or to any other stream derivative. You could display the raw response of a web server on the console like so (using path /index.html
):
import tango.io.Console; import tango.net.SocketConduit, tango.net.InternetAddress; auto host = new SocketConduit; auto addr = new InternetAddress ("www.myhost.com:80"); host.connect(addr).output.write ("GET /index.html HTTP/1.0 "); Cout.stream.copy (host.input);
Unlike FileConduit
and the console, SocketConduit
may require an explicit connect
step in order to instantiate input and output streams. This is shown in both client examples here, whereas a server program is typically handed a SocketConduit
with the streams already primed and active.
SocketConduit
has methods to connect to an address, bind to a local adapter for listening purposes, shut down one or both associated streams, test for timeout conditions, and expose the underlying Socket
instance via a socket
property.
Each read
operation is made under a timeout period, thus avoiding endless waiting for a remote listener to respond. The default timeout period is set at 3 seconds, which can be adjusted via a timeout
method.
Because it is a stream host, SocketConduit
can be used in conjunction with many of the other I/O elements such as filters, streaming iterators, buffers, and so on.
A datagram is a message sent over a network as a discrete User Datagram Protocol (UDP) packet, with a maximum size dictated by the network and/or the operating system. Unlike TCP/IP, UDP is an unreliable protocol, meaning that messages might be lost. Where TCP/IP will manage retransmission on your behalf, UDP is completely devoid of such overhead. Where TCP/IP is stream-oriented, UDP is oriented around discrete packets.
UDP can be a blessing for some data streams, where a certain amount of dropped information does not significantly impact the end result. For instance, sending real-time video over a UDP connection can save a lot of grief (in terms of maintaining the real-time aspect), as dropping a video packet here and there will often not adversely affect the desired result. On the other hand, relying on UDP to support networked financial transactions is likely to result in serious discontent for someone. Thus, UDP is useful as a low-overhead means of network communicationcommunication that does not require guaranteed delivery.
DatagramConduit
is a derivative of SocketConduit
, so you may treat it in much the same manner for most operations. The principal difference is in the distinction between data streams and discrete data packets when reading and writing. The second distinction is that both read
and write
accept an optional address. The write
method can direct each sent packet to a different address, and the read
method may receive from different addresses; that is, read
returns the packet originator via an optional address provided to it. Here's an example that sends itself a datagram, using bind
to initiate listening:
import tango.io.Console; import tango.net.InternetAddress, tango.net.DatagramConduit; // Listen for datagrams on the local address auto gram = new DatagramConduit; auto addr = new InternetAddress ("127.0.0.1", 8080); gram.bind (addr); // Write to the address. Retrieve and display message also, since we are listening gram.write ("hello", addr); char[8] tmp; auto size = gram.read (tmp); Cout (tmp[0..size]).newline;
Network hardware is often configured with a facility to distribute datagrams across clients that have registered interest in a particular set. These subscribers register their interest by joining a multicast group, which is one of a reserved set of network addresses. Subscription is different from its broadcast predecessor in that the former does not receive any datagrams until a subscription has taken place. Subscription can also be relinquished and reinstated as appropriate.
Multicast subscription (and dispatch) operates on a reserved set of network addresses, called class D addresses, which range from 225.0.0.0 to 239.255.255.255 inclusive.
A notable benefit of both broadcast and multicast is that datagram dispersal can take place at the hardware level, making it a particularly efficient means of pushing information from a central point to many recipients. However, their application has some fairly narrow limitations, due to the network-traffic pressure that broad usage may cause. Thus, multicast distribution is usually scoped to occur within a specific number of network hops known as the time-to-live (TTL) of a transmission, which is typically within the local network subnet or site. Related choices for the ttl
method include those listed in Table 7-3.
Table 7.3. Multicast TTL Options
Name | Domain |
---|---|
| Restricted to the same host |
| Restricted to the same subnet |
| Restricted to the same site |
| Restricted to the same region |
| Restricted to the same continent |
Other than ttl
, you use MulticastConduit
in a manner similar to DatagramConduit
. It must be bound to an address before incoming datagrams will be noted, though this can be handled on your behalf by a class constructor. MulticastConduit
adds join
and leave
methods, which subscribe and cancel, respectively. You can use multicast yourself in the following manner:
import tango.io.Console; import tango.net.InternetAddress, tango.net.MulticastConduit; auto group = new InternetAddress ("225.0.0.10:8080"); // Listen for datagrams on our group address auto multi = new MulticastConduit (group); // Subscribe and multicast on the group multi.join.write ("hello", group); // We are subscribed, and can thus see the multicast ourselves char[8] tmp; auto len = multi.read (tmp); Cout (tmp[0 .. len]).newline;
The example shows receipt of dispatched messages, which you can disable via a loopback
method.
A server program typically listens for network requests from clients and responds to them in some manner. In order to simplify some of the work of setup and management, Tango wraps a common listening approach in a class called ServerSocket
. For instance, you can create a simple network server using ServerSocket
like so:
import tango.net.ServerSocket, tango.net.InternetAddress; auto server = new ServerSocket (new InternetAddress (80)); auto request = server.accept;
Each time a request is made on the server, a network connection to the requesting client is returned from accept
as a SocketConduit
instance. Your server would read and write that instance in a manner understood by the client, and the connection would be terminated at some point thereafter.
Communicating with a server is straightforward, as the following example demonstrates:
import tango.io.Console; import tango.core.Thread; import tango.net.ServerSocket, tango.net.SocketConduit, tango.net.InternetAddress; const int port = 8080; void serve() { auto server = new ServerSocket (new InternetAddress(port)); // Wait for a request, and respond with a greeting server.accept.output.write ("server replies 'hello'"); } // Create server in a separate thread, and pause slightly for it to begin (new Thread (&serve)).start; Thread.sleep (0.250); // Make a request of our server auto request = new SocketConduit; request.connect (new InternetAddress ("localhost", port)); // Wait for and display response (there is an optional timeout) char[32] response; auto len = request.input.read (response); Cout (response[0 .. len]).newline;
This example is both a server and client within the same program, where the server runs as a separate thread of execution.
File handling in Tango includes path manipulation, high-level wrappers to expose simplified file access, and Unicode support, along with inspection and control over the file system itself.
In this section, the words directory and folder are used interchangeably.
The gateway to file content is through a FileConduit
, which provides both streaming and random-access support. Being a conduit instance, both input and output streams are exposed. When working with file content, you'll often be leveraging a FileConduit
instance without knowing it, since it is wrapped by various other constructs within Tango. Regardless, you may need to reference this explicitly when, for example, random file access is required.
Opening a file for reading is performed as follows:
auto conduit = new FileConduit ("myFilePath");
Opening a file for writing requires one of the styles to be specified (indicating how the file is expected to be manipulated):
auto conduit = new FileConduit ("myFilePath",FileConduit.WriteCreate);
There are a variety of predefined styles, including appending, read-only, read-write, create-always, and so on. You can define additional styles using a combination of a dozen system-level flags.
FileConduit
enables direct, type-agnostic streaming access to file content. In this example, we open a file and copy it directly to the console using Cout.stream.copy
:
import tango.io.Console, tango.io.FileConduit; auto from = new FileConduit ("test.txt"); Cout.stream.copy (from);
And here we copy one file to another, using a similar approach:
import tango.io.FileConduit; auto to = new FileConduit ("copy.txt", FileConduit.WriteCreate); to.output.copy (new FileConduit ("test.txt"));
To load an entire file into memory, you might consider the following approach, where we open a file, create an array to house the content, and then read that content:
import tango.io.FileConduit;
auto file = new FileConduit ("test.txt") auto content = new char[file.length]; auto bytesRead = file.input.read (content);
Conversely, you may write directly to a FileConduit
, like so:
import tango.io.FileConduit; auto to = new FileConduit ("text.txt", FileConduit.WriteCreate); auto bytesWritten = to.output.write (content);
Both these examples represent the essence of what File
(covered in the next section) performs on your behalf.
FileConduit
supports random-access I/O also. The next example relocates the current file position using seek
, and utilizes a DataFileInput
/DataFileOutput
pair to perform simple typed input and output:
import tango.io.FileConduit, tango.io.stream.DataFileStream; // Open a file for reading and writing auto file = new FileConduit ("myfile", FileConduit.ReadWriteCreate); auto input = new DataFileInput (file); auto output = new DataFileOutput (file); // Write data int x=10, y=20; output.putInt(x); output.putInt(y); // Rewind to file start output.seek (0); // Read data back again x = input.getInt; y = input.getInt; file.close;
Each FileConduit
should be explicitly closed when no longer needed. It can often be convenient to use a scope expression for this purpose:
auto file = new FileConduit ("myFile"); scope (exit) file.close;
An IOException
will be raised where a read or write operation fails entirely, or where a copy operation fails to complete. This might happen if, for example, a remote file were to suddenly become unavailable while in use.
File
combines FileConduit
and FilePath
together to provide a convenient means of accessing both attributes and content. The content of a file can be read, appended, or written in a single method call. For example, to read all file content, do this:
import tango.io.File; auto file = new File ("myfile"); auto content = file.read;
The underlying file is closed before the call returns. File
must avoid making assumptions about the file content, so the preceding example returns an array of type void
. When working with File
, it may be necessary to cast the return value to represent the correct data type, and for text files, this is often a char[]
. In this example, we take advantage of a syntactic shortcut to avoid the need for new
:
import tango.io.File; auto content = cast(char[]) File("myfile").read;
To convert a text file into a set of lines, try the following:
import tango.io.File; import Text = tango.text.Util; auto content = cast(char[]) File("myfile").read; auto lines = Text.splitLines (content);
Or you can use a foreach
to iterate instead:
foreach (line; Text.lines (content)) Cout (line).newline;
Files may be set to the content of an array, or text string:
import tango.io.File; auto file = new File ("myfile"); file.write ("the quick brown fox");
Content may be appended in a similar fashion:
file.append (" jumps over the lazy dog");
Methods belonging to FilePath
are exposed via the path
attribute, so you can retrieve the file size, relocate it, remove it, and so on:
auto size = file.path.fileSize;
File
will throw an IOException
where an underlying operating system or file system error occurs. This might happen, for example, when an attempt is made to write to a read-only file.
Unicode is the standard that assigns every known symbol a unique number. The Unicode Transformation Formats, commonly referred to as UTF-8, UTF-16, and UTF-32, describe how Unicode text is actually encoded. All three of these formats are supported directly by the D language via the char[]
, wchar[]
, and dchar[]
data types.
When working with Unicode files, you need to know which encoding was applied when the file was written so that it may be decoded correctly. Some text files might be of a "known" encoding, where the generating application is subsequently used to read the content. Conversely, the content of other text files might be generated by one application and read by another, where the latter is not explicitly aware of the encoding applied. Such a scenario would appear to require some kind of metadata associated with each file, in order for subsequent reading applications to "know" how the content was originally written.
Without general, cross-platform support for file-oriented metadata being available, other schemes have been applied to file content in order to identify the encoding in use. One such scheme uses the first few bytes of a file to identify the encoding, called a byte-order mark (BOM). For better or worse, this particular scheme has become reasonably prevalent and is thus supported within the Tango library as a convenient way to deal with Unicode-based files.
UnicodeFile
combines facilities from the previously described File
class with a capability to recognize and translate file content between various Unicode encodings and their native D representations. UnicodeFile
can be explicitly told which encoding should be applied to a file, or it can discover an existing encoding via file inspection. For example, to read UTF-8 content from a file with unknown encoding, do this:
import tango.io.UnicodeFile; auto file = new UnicodeFile!(char)("myfile.txt", Encoding.Unknown); char[] content = file.read;
The UnicodeFile
class is templated for types of char
, wchar
, and dchar
, representing UTF-8, UTF-16, and UTF-32 encodings. Those are considered to be the internal encoding, while the file itself is described by an external encoding. In the preceding example, our external encoding is stipulated as Encoding.Unknown
, indicating that it should be discovered instead. Alternatives include a set of both explicit and implicit encodings, where the former describe exactly the format of contained text, and the latter indicate that file inspection is required. For example, Encoding.UTF8N
, Encoding.UTF16LE
, and Encoding.UTF32BE
are explicit encodings; Encoding.Unknown
and Encoding.UTF16
are of the implicit variety.
When writing to a UnicodeFile
, the encoding must, at that time, be known in order to transform the output appropriately (injecting a BOM header is optional when writing). When reading, the encoding may be declared as known or unknown.
The read
method returns the current content of the file. The write
method sets the file content and file length to the provided array. The append
method adds content to the tail of the file. When appending, it is your responsibility to ensure the existing and current encodings are correctly matched. Methods to inspect and manipulate the underlying file hierarchy and to check the status of a file or folder are made available via the path
attribute in a manner similar to File
.
UnicodeFile
will relay exceptions when an underlying operating system or file system error occurs, or when an error occurs while content is being decoded.
FileSystem
is where various file system controls are exposed. At this time, tango.io.FileSystem
provides facilities for retrieving and setting the current working directory, and for converting a path into its absolute form. To retrieve the current directory name, do this:
auto name = FileSystem.getDirectory;
Changing the current directory is similar in operation:
FileSystem.setDirectory (name);
FileSystem.toAbsolute
accepts a FilePath
instance and converts it into absolute form relevant to the current working directory. Absolute form generally begins with a path separator, or a storage device identifier, and contains no instances of a dot (.
) or double dot (..
) anywhere in the path. If the provided path is already absolute, it is returned untouched.
Failing to set or retrieve the current directory will cause an exception to be thrown. Passing an invalid path to FileSystem.toAbsolute
will also result in an exception being thrown.
The storage devices of the file system are exposed via the FileRoots
module. On Win32, roots represent drive letters; on Linux, they represent devices located via /etc/mtab
. To list the file storage devices available, try this:
import tango.io.Console, tango.io.FileRoots; foreach (name; FileRoots.list) Cout (name).newline;
An IOException
will be thrown where an underlying operating system or file system error occurs.
The FileScan
module wraps the file traversal functionality from FilePath
in order to provide something more concrete. The principal distinction is that FileScan
visits each discovered folder and generates a list of both the files and the folders that contain those files.
To generate a list of D files and the folders where they reside, you might try this:
import tango.io.Stdout, tango.io.FileScan; char[] root = "."; Stdout.formatln ("Scanning '{}'", root); auto scan = (new FileScan)(root, ".d"); Stdout.format (" {} Folders ", scan.folders.length); foreach (folder; scan.folders) Stdout.format ("{} ", folder); Stdout.format (" {0} Files ", scan.files.length);
foreach (file; scan.files) Stdout.format ("{} ", file); Stdout.formatln (" {} Errors", scan.errors.length); foreach (error; scan.errors) Stdout (error).newline;
The example executes a sweep across all files ending with .d
, beginning at the current folder and extending across all subfolders. Each folder that contains at least one located file is displayed on the console, followed by a list of the located files themselves. The output would look something like this abbreviated listing:
Scanning 'dimport angoio' 8 Folders dimport angoiocompress dimport angoiostream dimport angoiovfs . . . dimport angoio 40 Files dimport angoioBuffer.d dimport angoiocompressBzipStream.d dimport angoiocompresslibStream.d dimport angoioConsole.d dimport angoioFile.d dimport angoioFileConduit.d dimport angoioStdout.d . . . dimport angoiostreamDataFileStream.d dimport angoiostreamDataStream.d dimport angoiostreamFileStream.d dimport angoiostreamFormatStream.d dimport angoiostreamLineStream.d dimport angoiostreamTextFileStream.d dimport angoiostreamTypedStream.d dimport angoiostreamUtfStream.d 0 Errors
For more sophisticated file filtering, FileScan
may be customized via a delegate:
bool delegate (FilePath path, bool folder)
The return value of the delegate should be true
to add the instance, or false
to ignore it. The parameter folder
indicates whether the instance is a directory or a file.
FileScan
throws no explicit exceptions, but those from FilePath.toList
will be gathered up and exposed to the user via scan.errors
instead. These are generally file system failures reported by the underlying operating system.
In the Tango library, file and folder locations are typically described by a FilePath
instance. In some cases, a method accepting a textual file name will wrap it with a FilePath
before continuing.
A number of common file and folder operations are exposed via FilePath
including creation, renaming, removal, and the generation of folder content listsalong with a handful of attributes such as file size and various timestamps. You can check to see if a path exists, whether it is write-protected, and whether it represents a file or a folder.
Creating a FilePath
is straightforward: you provide the constructor with a char[]
. File paths containing non-ASCII characters should be UTF-8 encoded:
import tango.io.FilePath; auto path = new FilePath ("/dev/tango/io/FilePath.d");
With a FilePath
instance in hand, each component can be efficiently inspected and adjusted. You can retrieve or replace each individual component of the path, such as the file name, the extension, the folder segment, the root, and so on. FilePath
can be considered to be a specialized string editor, with hooks into the file system. Using the previous example, Table 7-4 highlights each component.
Table 7.4. Inspecting FilePath Components
Component | Content |
---|---|
|
|
|
|
|
|
|
|
|
|
|
|
Changing component values is straightforward, too, as Table 7-5 illustrates. In the table, we are both adjusting a component and showing the resultant change to the path itself.
Table 7.5. Adjusting FilePath Components
Component | Content |
---|---|
|
|
|
|
|
|
|
|
|
|
You can also append
and prepend
text to a FilePath
, and appropriate separators will be inserted where required. Another useful tool is the pop
function, which removes the rightmost text (in place) such that a parent folder segment is exposed. Successive use of pop
will result in a root folder, or just a simple name. Another handy one is dup
, which can be used to make a copy of another FilePath
, like so:
import tango.io.FilePath; auto path = FilePath ("/dev/tango/io/FilePath.d"); auto other = path.dup.name ("other");
The original path
is left intact, while other
has the same components except for a different name.
When you are creating "physical" files and folders, a distinction is required between the two. Use path.createFile
to create a new file and path.createFolder
to create a new folder. The full path to a folder can be constructed using path.create
, which checks for the existence of each folder in the hierarchy and creates it where not present.
An exception will be raised if path.create
encounters an existing file with the same name as a provided path segment.
Renaming a file can also move it from one place to another:
path.rename ("/directory/otherfilename");
Copying a file retains the original timestamps:
path.copy ("source");
You can remove a file or a folder like this:
path.remove;
List the content of a folder like this:
import tango.io.Console, tango.io.FilePath; foreach (name; path.toList) Cout (name).newline;
You can customize the generated results by passing toList
a filter delegate with the same signature noted in the previous section. Returning false
from the filter causes a specific path to be ignored. An additional, lower-level foreach
iterator exposes further detail:
import tango.io.Stdout, tango.io.FilePath; foreach (info; path) Stdout.formatln("path {}, name {}, size {}, is folder {}", info.path, info.name, info.size, info.folder);
When using FilePath
, any errors produced by the underlying file system will cause an IOException
to be raised. For example, attempting to remove a nonexistent or read-only file will generate an exception.
FilePath
assumes both path and name are present within the provided file path, and therefore may split what is otherwise a logically valid path. Specifically, the name
attribute of a FilePath
is considered to be the segment following a rightmost path separator, and thus a folder identifier can become mapped to the name
property instead of explicitly remaining with the path
property. This follows the intent of treating file and folder paths in an identical manner: as a name with an optional ancestral structure. When you do not want this assumption about the path and name to be made, it is possible (and legitimate) to bias the interpretation by adding a trailing path separator. Doing so will result in an empty name
attribute and a longer path
attribute.
This concludes our look at some of the I/O facilities in Tango, and yet we've barely scratched the surface! Tango I/O offers various network-oriented packages to support HTTP and FTP protocols, for example. It also hosts a digest-message package, nonblocking I/O support, a data-compression package, and more.
In the next (and last) chapter, you'll find a general overview of additional packages within the Tango library.