Although most application networking will involve downloading content over standard protocols such as HTTP(S) and using standard representations, there are times when having a specific data stream protocol is required. In this case, a stream oriented process will allow individual bytes to be read or written, or a datagram or packet oriented process can be used to send individual packets of data.
There are networking libraries to support both; an NSStream
higher-level Objective-C based class provides the mechanism to drive stream-based responses, and although lower-level packet connections are possible with the CoreFoundation
or the POSIX
layer, local multiplayer gaming using the MultipeerConnectivity
module is often appropriate.
Local networking with the MultipeerConnectivity
module involves creating an MCSession
, followed by sendData
to send NSData
objects to connected peers and using the MCSessionDelegate
to receiveData
from connected peers. This is often used to synchronize the state of the world, such as the player's current location or health.
A stream is a reliable, ordered sequence of bytes, which is used by most internet protocols. Streams can be created from a network host and port using the NSStream
class method getStreamsToHostWithName
, added in iOS 8 and part of Mac OS X. This allows an NSInputStream
and NSOutputStream
to be acquired at the same time.
The input and output streams can then be used to send data asynchronously or synchronously. Asynchronous mechanisms involve scheduling the data processing on the application's run-loop and is covered in the Asynchronous reading and writing section. Synchronous mechanisms use read
and write
to receive or send buffers of data.
To simplify the process of acquiring streams, the following can be created as an extension of the NSStream
class. An extension makes a method appear to come from an original class but is implemented externally to that class. Add the following StreamExtensions.swift
file to the CustomViews
project with the following content:
extension NSStream { class func open(host:String,_ port:Int) -> (NSInputStream, NSOutputStream)? { var input:NSInputStream? var output:NSOutputStream? NSStream.getStreamsToHostWithName( host, port: port, inputStream: &input, outputStream: &output) if input == nil || output == nil { return nil } else { output!.open() input!.open() return (input!,output!) } } }
A connection to a remote host can be obtained by calling NSStream.open(host,port)
, which returns an open pair of input/output streams.
The NSInputStream
method read
allows bytes to be read from a stream synchronously, while the NSOutputStream
method write
allows bytes to be written to a stream. These take different types, but the most common approach is to create an array of bytes [UInt8]
in Swift as the buffer, and then read into it or from it with an UnsafeMutablePointer
(equivalent to an ampersand in C).
The read
and write
methods both return a number of bytes read/written. This can be negative (in the case of an error), zero, or positive in the case of bytes having been processed. Both calls take a buffer and a maximum length, though it is not guaranteed that the full maximum length will be processed.
To make it easier to write NSData
content to streams, an extension method on NSOuptutStream
can be created that performs a full write, based on the size of the data:
extension NSOutputStream { func writeData(data:NSData) -> Int { let size = data.length var completed = 0 while completed < size { var wrote = write(UnsafePointer(data.bytes) + completed, maxLength:size - completed) if wrote < 0 { return wrote } else { completed += wrote } } return completed } }
The code takes an NSData
and writes it to the underlying stream, returning the number of bytes written (or a negative value if there are problems). The return value of the write
method is checked, and if the value is negative, returned to the caller directly. Otherwise, the completed
counter is incremented with the number of bytes written.
If the number of written bytes reaches the size of the data requested, then the value is returned. Otherwise the loop recurs, this time starting at the point it left off.
A similar approach can be used to read a full buffer from an NSInputStream
by creating a readBytes
method that returns an array of bytes of a known size and a means to convert this to an NSData
for easier processing/parsing:
extension NSInputStream { func readBytes(size:Int) -> [UInt8]? { let buffer = Array<UInt8>(count:size,repeatedValue:0) var completed = 0 while completed < size { let read = self.read( UnsafeMutablePointer(buffer) + completed, maxLength: size - completed) if read < 0 { return nil } else { completed += read } } return buffer } func readData(size:Int) -> NSData? { if let buffer = readBytes(size) { return NSData( bytes: UnsafeMutablePointer(buffer), length: buffer.count) } else { return nil } } }
The readData
method returns an NSData
, while the readBytes
method returns an array of UInt8
values. The NSData
approach is useful in some cases (particularly, for creating a String
from the returned data), and in other cases being able to process the bytes directly is useful (for example, parsing binary formats). Having both allows either of these to be used as appropriate.
Being able to process data as UTF8 values or hexadecimal values can be useful in some protocols. Although both NSString
and NSData
provide means to convert to and from UTF8, the syntax is overly verbose, as it is based on preexisting Objective-C methods.
To facilitate the conversions, extension methods can be created to provide a simple way of converting to and from UTF8 representations. In addition to class and instance functions, it is possible to use extensions to add dynamic properties to an existing object. This can be used to create the utf8data
and utf8string
properties on NSData
and String
by adding extensions in a file Extensions.swift
as follows:
extension NSData { var utf8string:String { return NSString(data:self, encoding:NSUTF8StringEncoding)! } } extension String { var utf8data:NSData { return self.dataUsingEncoding( NSUTF8StringEncoding, allowLossyConversion: false)! } }
This allows expressions such as data.utf8string
and string.utf8data
, which are much more compact. Each time the expression is evaluated, the associated getter function will be called.
There is no standard convention for naming extensions in Swift at the time this book was written. If there are extensions to a single type of data—such as the streams previously—then the file can be named [Type]Extensions.swift
. Alternatively, the name can be used for the type of methods called; for example, in this case UTF8Extensions.swift
could have been used.
Parsing hexadecimal data from strings and integers can also be added to the String
and Int
types, as follows:
extension String { func fromHex() -> Int { var result = 0 for c in self { result *= 16 switch c { case "0":result += 0 case "1":result += 1 case "2":result += 2 case "3":result += 3 case "4":result += 4 case "5":result += 5 case "6":result += 6 case "7":result += 7 case "8":result += 8 case "9":result += 9 case "a","A":result += 10 case "b","B":result += 11 case "c","C":result += 12 case "d","D":result += 13 case "e","E":result += 14 case "f","F":result += 15 default: break } } return result; } } extension Int { func toHex(digits:Int) -> String { return NSString(format:"%0(digits)x",self) } }
This allows hex values to be created with int.toHex
and string.fromHex
.
It is possible to write a client to query a remote git server using the git://
protocol to determine the hashes of remote tags/branches/references.
The git://
protocol works by sending
packet lines of data, with each line prefixed with four hexadecimal digits in ASCII, indicating the length of the rest of the data (including the four initial digits). Sending a git-upload-pack
request will return a list of references on the remote repository.
Since the git://
protocol uses packet lines, create a PacketLineExtensions.swift
file with the following content:
extension NSOutputStream { func writePacketLine(_ message:String = "") -> Int { let data = message.utf8data let length = data.length if length == 0 { return writeData("0000".utf8data) } else { let prefix = (length + 4).toHex(4).utf8data return self.writeData(prefix) + self.writeData(data) } } }
When an empty NSData
object is passed, the special packet line 0000
is written, indicating the end of the conversation. When a non-empty NSData
is written, the length of the data is written as a hexadecimal value (including the 4 bytes for the length) followed by the data itself.
This will result in a protocol conversation like:
> 004egit-upload-pack /alblue/com.packtpub.swift.essentials.git host=github.com < 00dfadaa46b98ce211ff819f0bb343395ad6a2ec6ef1 HEAD multi_ack thin-pack side-band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed symref=HEAD:refs/heads/master agent=git/2:2.1.1+github-611-gd89bd9f < 003fadaa46b98ce211ff819f0bb343395ad6a2ec6ef1 refs/heads/master > 0000 < 0000
Reading a packet line is similar:
extension NSInputStream { func readPacketLine() -> NSData? { if let data = readData(4) { let length = data.utf8string.fromHex() if length == 0 { return nil } else { return readData(length - 4) } } else { return nil } } func readPacketLineString() -> NSString? { if let data = self.readPacketLine() { return data.utf8string } else { return nil } } }
In this case, the first 4 bytes are read to determine what the remaining length is. If it is zero, a nil
value is returned to indicate the end of the stream. If it is non-zero, the data is read (less the 4 that is used for the packet line length header). An additional readPacketLineString
is provided to allow an easy creation of the packet line as an NSString
.
To remotely query a git repository for references, the git-upload-pack
command needs to be sent, along with a reference to the repository in question and optionally a host as well. To provide an API to query this programmatically, create a RemoteGitRepository
class with an initializer that stores the host, port, and repository; and a lsRemote
function, which returns the value of the references:
class RemoteGitRepository { let host:String let repo:String let port:Int init(host:String, repo:String, _ port:Int = 9418) { self.host = host self.repo = repo self.port = port } func lsRemote() -> [String:String] { var refs = [String:String]() // load the data return refs } }
To load the data from the repository, a connection to the remote host needs to be made on the default port (in this case, 9418
is the default for the git://
protocol). Once the streams have been opened, the git-upload-pack [repository] host=[host]
packet line is sent, and subsequently, lines can be read of the form hash reference
:
if let (input,output) = NSStream.open(host,port) { output.writePacketLine( "git-upload-pack (repo) host=(host) ") while true { if let response = input.readPacketLineString() { let hash = String(response.substringToIndex(41)) let ref = String(response.substringFromIndex(41)) if ref.hasPrefix("HEAD") { continue } else { refs[ref] = hash } } else { break } } output.writePacketLine() input.close() output.close() }
Calling the lsRemote
function on a RemoteGitRepository
instance with an appropriate host
and repo
will return a list of hashes by reference.
Since the network can introduce delays or can even result in a complete failure, network calls should never be performed on the UI thread. Previously, the SampleTable
was used to introduce a runOnUIThread
function. A similar approach can be used to run a function on a background thread:
func runOnBackgroundThread(fn:()->()) { dispatch_async( dispatch_get_global_queue( DISPATCH_QUEUE_PRIORITY_DEFAULT, 0) ,fn) }
This will permit viewDidLoad
to invoke a call in order to query the remote references from the repository and add them to the table. As before, the call to update the table must be called from the UI thread. Add the following to the end of the viewDidLoad
method:
runOnBackgroundThread { let repo = RemoteGitRepository(host: "github.com", repo: "/alblue/com.packtpub.swift.essentials.git") for (ref,hash) in repo.lsRemote() { self.items += [(ref,hash)] } self.runOnUIThread(self.tableView.reloadData) }
Now, when the application is launched, entries corresponding to the branches and tags in the remote repository should be added to the table.
Besides synchronous reading and writing, it is also possible to perform asynchronous reading and writing. Instead of spinning in a while
loop, the application can use callbacks scheduled on the application's run loop.
To receive callbacks, a class that implements NSStreamDelegate
must be created and assigned to the stream's delegate
field. When events occur, the stream
method is called, to which the type of event as well as the associated stream are passed.
The stream is registered with scheduleInRunLoop
(using NSRunLoop.mainRunLoop()
with a NSDefaultRunLoopMode
mode). Finally, the stream can be opened.
Events are defined in the NSStreamEvent
class and include HasSpaceAvailable
(for output streams) and HasBytesAvailable
(for input streams). By responding to callbacks, the application can process results asynchronously.
When using Swift, the NSStreamDelegate
is treated as a weak
delegate on the input or output stream. This poses problems when using an inline class to provide input parsing; doing so will result in an EXC_BAD_ACCESS
, as the delegate is automatically reclaimed by the runtime. This can be avoided by storing a strong circular reference to self
in the initializer and assigning it to nil
when the streams are closed.
This is especially useful for asynchronous protocols, such as XMPP, which might send additional messages at arbitrary times. It also allows battery-powered devices to not spin the CPU, should the remote server be slow or hang.
To receive data asynchronously, a delegate must implement the NSStreamDelegate
method stream(stream:handleEvent)
. When data is available, the HasBytesAvailable
event will be sent, and data can be read accordingly.
To convert the previous example to an asynchronous form, a few changes need to be made. Firstly, the open
extension method created in the Opening a stream connection section needs to be augmented with a connect
method, which does not perform the open
immediately:
class func open(host:String,_ port:Int) -> (NSInputStream, NSOutputStream)? { if let (input,output) = connect(host,port) {input.open() output.open() return (input,output)} else { return nil } } class func connect(host:String,_ port:Int) -> (NSInputStream, NSOutputStream)? { … // as before but with open commented out // input!.open() // output!.open() … }
To create a stream delegate, create a file called PacketLineParser.swift
with the following content:
class PacketLineParser: NSObject, NSStreamDelegate { let output:NSOutputStream let callback:(NSString)->() var capture:PacketLineParser? init(_ output:NSOutputStream, _ callback:(NSString) -> ()) { self.output = output self.callback = callback super.init() capture = self } func stream(stream: NSStream, handleEvent: NSStreamEvent) { let input = stream as NSInputStream if handleEvent == NSStreamEvent.HasBytesAvailable { if let line = input.readPacketLineString() { callback(line) } else { output.writePacketLine() input.close() output.close() capture = nil } } } }
This parser has a callback that is invoked for each packet line read; when the HasBytesAvailable
event is sent, the line is read (using the same synchronous mechanism as before) and then passed to the callback. Unlike the synchronous approach, there is no while
loop here—when data is available, it triggers the parsing of the data.
Since this will be assigned to an input stream delegate (which holds a weak reference), it is necessary to capture a cyclic reference to itself with capture = self
in order to avoid the runtime from evicting the instance. When the streams are closed, the capture
will be set to nil
, which will release the instance.
The readPacketLine
returns a nil
to indicate either an error or a completed stream; in this case, an empty packet line is sent (to tell the remote server that no further interaction is required) and then both the streams are closed.
It is necessary to clean up the streams and remove them from run loops, both when the stream content is successful or when communication errors occur. In addition to the HasBytesAvailable
event, there are events that are sent when the stream's end is encountered or an error occurs.
These should be handled in the same way as when the connection comes to a natural end; resources should be tidied, and in particular, the streams should be removed from run loop processing. Finally, the cyclic reference should be removed to permit the delegate
object to be removed.
The existing close
code can be moved to its own separate function, and additional cases of the stream ending or errors occurring can perform the same cleanup:
func stream(stream: NSStream, handleEvent: NSStreamEvent) { let input = stream as NSInputStream if handleEvent == NSStreamEvent.HasBytesAvailable { if let line = input.readPacketLineString() { callback(line) } else { closeStreams(input,output) } } if handleEvent == NSStreamEvent.EndEncountered || handleEvent == NSStreamEvent.ErrorOccurred { closeStreams(input,output) } } func closeStreams(input:NSInputStream,_ output:NSOutputStream) { if capture != nil { capture = nil output.removeFromRunLoop(NSRunLoop.mainRunLoop(), forMode: NSDefaultRunLoopMode) input.removeFromRunLoop(NSRunLoop.mainRunLoop(), forMode: NSDefaultRunLoopMode) input.delegate = nil output.delegate = nil if output.streamStatus != NSStreamStatus.Closed { output.writePacketLine() output.close() } if input.streamStatus != NSStreamStatus.Closed { input.close() } } }
To provide a list of references asynchronously, the delegate has to be set up with a suitable callback that will parse the returned data. Instead of the method returning a dictionary (which would require synchronous blocking), a callback will be passed which can be called with references as they are found.
To start the process, the git-upload-pack
needs to be sent synchronously, after which subsequent responses will be processed asynchronously. This can be done by creating a new method, lsRemoteAsync
, which takes a callback function for the (String,String)
tuple:
func lsRemoteAsync(fn:(String,String) -> ()) { if let (input,output) = NSStream.connect(host,port) { input.delegate = PacketLineParser(output) { (response:NSString) -> () in let hash = String(response.substringToIndex(41)) let ref = String(response.substringFromIndex(41)) if !ref.hasPrefix("HEAD") { fn(ref,hash) } } input.scheduleInRunLoop(NSRunLoop.mainRunLoop(), forMode: NSDefaultRunLoopMode) input.open() output.open() output.writePacketLine( "git-upload-pack (repo) host=(host) ") } }
This creates a connection (but without opening the streams), sets the delegate
and schedules the run loop for the input stream, and finally opens both the streams for interaction. Once this is done, the initial git-upload-pack
message is sent as before. At this point, the lsRemoteAsync
method returns, and subsequent events occur when input data is received from the server.
When a line is received through the PacketLineParser
callback, it is split into a reference and a hash and then hands the results to the callback passed into the argument in the first place.
Asynchronous programming often involves many callbacks. Instead of a synchronous program that might look like A;B;C;
an asynchronous program often looks like A(callback:B(callback:C))
. When an input trigger occurs—a network request, user interaction, or timer firing—a sequence of actions can occur via these nested callbacks.
Asynchronous pipelines are generally preferred for battery performance reasons, as blocking in a while
spin loop will waste CPU energy until the condition is satisfied.
To display the asynchronous data to the screen, the callback must be modified to allow individual elements to update the GUI.
In SampleTable
, instead of calling repo.lsRemote
(which performs a synchronous lookup) use repo.lsRemoteAsync
. This requires a callback, which can be used to update the table data and can cause the view to reload the contents:
// for (ref,hash) in repo.lsRemote() { // self.items += [(ref,hash)] // } repo.lsRemoteAsync() { (ref:String,hash:String) in self.items += [(ref,hash)] self.runOnUIThread(self.tableView.reloadData) }
Now when the application is run, the references will be updated asynchronously and the UI will not be blocked by a slow or hung server.
Asynchronous sending is not as useful as asynchronous reading unless large uploads are required. If there is a lot of data, then it is unlikely to be written synchronously in a single write
call. It is better to perform any additional writes asynchronously.
To write data asynchronously requires storing the completed
count as a variable outside the function. The write
method can be used to replace the while
loop as before, by writing a segment of the data on each iteration of the stream method. Although code isn't needed in this example, this is how the code could look:
… self.data = data // initial write to kick off subsequent events completed = output.write(UnsafePointer(data.bytes), maxLength: data.length … var completed:Int var data:NSData? func stream(stream: NSStream, handleEvent: NSStreamEvent) { let output = stream as NSOutputStream if handleEvent == NSStreamEvent.HasSpaceAvailable && data != nil { let size = data!.length completed += output.write( UnsafePointer(data!.bytes) + completed, maxLength: size – completed) if completed == size { completed = 0 data = nil } } }
Asynchronous data always starts with a call to synchronously write the data. If not all of the data is written (in other words, completed < size
), then subsequent callbacks will occur on the NSStreamDelegate
. This can then pick up where the data
value left off, using a similar technique to the synchronous case but without a while
loop. Instead of the iteration blocking to write the whole data value, the stream call will be called multiple times (in effect replacing each iteration of the while
loop). In the final run, when completed == size
, the data is released and the completion counter is reset.
The stream callback is called enough number of times to write all the data. If no data is written, then events are no longer called. New data is only written when an additional value is passed. Care must be taken when writing data from different threads, since the data value is processed as an instance variable and overwriting it might cause data to be lost. The reader is invited to extend the single element data into an array of outstanding data elements so that they can be queued up appropriately.