The Personal Cloud

If you need the benefits of cloud sync, storage, and collaboration, but want better control over security and privacy, you can set up your own personal cloud, which means you own and control the hardware and software that provides the services. This lets you set your own privacy policy, storage quotas, and service settings—and you can make your system as secure as you like.

Although a personal cloud can solve many problems, it comes with a couple of gotchas:

• Time, money, and effort: I said earlier that what cloud computing means to me is that it’s somebody else’s problem. You don’t have to buy hardware, install or update software, maintain equipment, and so on. Somebody else takes care of all that. But in a personal cloud, all that stuff is back on you. Even in cases where the setup itself is simple enough for a child, you’ll be responsible for troubleshooting, which may require technical skill.
• Flexibility: If you want to use a personal cloud for file syncing and sharing, backups, or syncing contacts and calendars, you can choose from numerous convenient solutions. But if you’re looking to replace cloud apps, your choices are limited. Although it’s possible to find cloud apps for things like image editing, spreadsheets, project management, and video chat that you can install on your own equipment and then access in a Web browser, those sorts of things require a nontrivial amount of time, effort, and technical skill to set up and maintain. Allowing all that to be someone else’s problem is, after all, part of the appeal of using publicly available cloud services!

Depending on how you approach your personal cloud, you may encounter other barriers too, as I discuss ahead. I say all this not to discourage you from creating a personal cloud, but to set your expectations properly. You’re gaining control and privacy but giving up the simplicity of paying someone else to figure everything out.

Like the rest of this book, this chapter omits step-by-step instructions. There isn’t just one way to set up a personal cloud; the options are unlimited. You’ll have to decide which capabilities you need, which hardware and software you prefer, what trade-offs you’re willing to make, and what cost you can bear—and the biggest consideration of all will be your level of technical expertise. If it’s high, you’ll have more options available; if it’s low, you’ll want to gravitate toward simpler, more consumer-friendly products that lack fancier features.

Instead, the remainder of this chapter looks at the major questions you’ll have to think through when designing your personal cloud: What services do you want to provide? What sort of hardware and software will you use? And where will you put the equipment? In some cases, the obvious answer turns out to be more complicated than you might imagine.

Personal Cloud Basics

Every service in the public cloud is provided by a server of some kind—in most cases, a distributed cluster of virtual servers. But whatever the server’s specific form, it has to reside on a device somewhere running server software of some sort and be connected to the Internet. So if you’re going to create a personal cloud, then by definition, you have to set up one or more servers of your own.

However, a server doesn’t have to be a stand-alone computer. It could be software running on a computer you already own. It could be a tiny appliance that you plug in to your home network (or someone else’s network) and configure in a browser window. It could even be a virtual private server you rent monthly. Just ahead I explore these options in more detail.

Choosing Personal Cloud Services

In this book I’ve talked about lots of different cloud services. If you want to take just one or two of them—say, file storage and sharing—out of the public cloud and under your own control, you can do so fairly easily. You can create personal versions of most cloud services, but there comes a point of diminishing returns. For example, my personal experience was that running my own email server was slightly less fun than having a rock in my shoe all day, every day. I can do it, but life is too short for me to deal with the complications of email servers if I don’t absolutely have to.

Generally speaking, cloud services associated with files—storage, syncing, sharing, and media streaming—are good candidates for a personal cloud. Or at least they’re the easiest place to start. Running your own simple CalDAV (calendars), CardDAV (contacts), or Web server isn’t out of the question for someone with modest technical skills. But if you want to do fancy stuff on the Web, run your own email server, or try to replicate the full functionality of something like Google Drive apps, well, all I can say is that current technology doesn’t make those tasks excessively straightforward for anyone lacking a computer science degree. Sorry. But hey: start with the low-hanging fruit, and in time, we’ll probably get there.

Personal Cloud Software and Equipment

What do you need to create a personal cloud? Well, if you have a computer that isn’t too busy and can be left on (and connected to the Internet) all of the time, you can install software on it that provides cloud services. Or, you can opt for a gadget (see Personal Cloud Server Appliances) that doesn’t rely on a computer. Either way, you may need to overcome some hurdles that could affect the availability of your equipment over the Internet—we’ll get to that later in this chapter.

Personal Cloud Server Software

Here’s a small sampling of software packages you can install on your own computer or on a server under your control to provide various services over the Internet:

• Resilio Sync: This software, previously known as BitTorrent Sync, uses the BitTorrent protocol to sync the files of your choice across your own devices, and also lets you share folders with others. So, in operation, it’s very much like Dropbox, except that all the data stays on your own computers; nothing is stored in the Cloud. Resilio Sync costs $59.99 for personal use or $99.99 for a family of up to five.
• macOS Server: If you’re a Mac user, you can pay Apple $19.99 for an add-on to macOS called macOS Server. It offers email, contact, calendar, Web, file sharing, and several other services, all in a (relatively) easy-to-use package.
• ownCloud, and a newer offshoot called Nextcloud, are private cloud server packages that run on top of an existing Linux installation—for example, a computer you own, a virtual server you rent, or a shared hosting plan offered by a Web host. After a quick setup procedure, you can then run your own file storage, syncing, and sharing service (much like Dropbox), calendar (CalDAV) and contacts (CardDAV) servers, a collaborative editing environment for office documents, audio and video chat, and more. All these services are available in both Web browsers and mobile apps. Because you can control where they’re installed, who’s allowed to connect, how data is encrypted, and so on, they can form the core of a personal cloud. (Nextcloud also sells stand-alone appliances that enable you to run the service from your home or office, and thus keep total control over the hardware even if you don’t have a dedicated Linux server.) Both packages let you add on any of numerous free apps to provide additional cloud services. And, for personal use, both ownCloud and NextCloud are free.
• Tonido Desktop: If you want to turn your existing Windows, Linux, or Mac computer into a cloud server, one easy way to do so is to install the free Tonido Desktop software, which includes file sharing and syncing, media streaming, and other features—all accessible via the Web or client apps for most desktop and mobile platforms.

Personal Cloud Server Appliances

You can also create a personal cloud (for some services, at least) without installing any software. Just pick up a little gadget, connect it to your network, and carry on. Here’s a handful of devices in this rapidly expanding category:

• Apollo Cloud: Containing a 2 TB ($199) or 4 TB ($249) hard drive along with USB 3 and Ethernet connections, this compact device lets you store and securely share files, photos, and videos—and access them from any computer or mobile device. It can also function as a network backup device; it’s even compatible with Apple’s Time Machine app. If you have two or more of these devices, they can automatically mirror each other’s contents, even if they’re located in different cities. (So in some ways it’s much like the now-discontinued Transporter device I recommended in the first edition of this book.)
• Lima Ultra: This tiny device, which costs $129.99, connects to a USB hard drive and your home Ethernet network, and lets you store files and folders securely. You can then connect to it (even remotely) using your Mac or PC, or a mobile app. Like Apollo Cloud, the Lima Ultra can automatically mirror its contents to a second device if you have one. Although you can use Lima to share individual files, so far it doesn’t offer folder sharing. On the plus side, however, it does support streaming audio and video, much like more expensive NAS devices.

• Cloud-enabled NAS devices: Numerous vendors of network-attached storage—basically one or more hard drives in a box with a network interface and a CPU—have products that are accessible not only on your local network but also across the Internet, making them personal cloud server appliances. The specific features vary from one product to the next, but common capabilities include file sharing, backup, and media streaming.

  A few examples:

  • Akitio’s MyCloud personal cloud servers
  • D-Link’s ShareCenter Cloud Storage Enclosures
  • Drobo’s 5N2
  • Netgear’s ReadyNAS devices
  • QNAP’s Home and SOHO products
  • Synology’s J Series and Value Series NAS devices
  • Western Digital’s My Cloud Mirror devices

A quick Web search should turn up numerous other examples of inexpensive devices that provide personal cloud services.

There’s also the option of using neither a physical computer nor an appliance, but rather a virtual server—something I discuss shortly in A Virtual Server.

Choosing Locations for Personal Cloud Servers

Where are you going to put your personal cloud equipment? The answer may not be straightforward. You have several options—and you may choose to use more than one of them (Figure 7).

Your Own Network

You probably already have broadband Internet access in your home or office, so that’s a natural location to consider locating personal cloud equipment. In theory, you can simply plug a server into power and your network, and be off and running. However, before you jump to the conclusion that your existing network is an ideal home, consider the following potential issues:

• ISP restrictions: Many residential Internet providers have terms of service that explicitly forbid users from running servers of any kind. Some ISPs are more relaxed about these restrictions than others. That is, if you run a low-volume server for personal use, your ISP may ignore it, even if it’s technically against the rules. On the other hand, if your ISP is a stickler, it could block any server you run from being seen outside your home—or even cancel your service altogether for breaking the rules. So, it’s a good idea to go read the fine print of your ISP’s service agreement before attempting to set up something of your own.

  In many cases, even though residential users can’t run servers, business users can. You could ask your ISP to upgrade you to business-class service to remove the no-server restriction. You’ll probably continue to use the same network wiring and perhaps even the same cable modem or DSL router. But, business service is invariably more expensive than residential service, and you often get significantly less bandwidth, even at that higher price.

• IP address and DNS issues: Most residential broadband services don’t include static IP addresses—so your server’s IP address may change from time to time. That can make it difficult to locate and reach your server from the outside. (Business broadband connections more frequently offer static IP addresses.)

  One way to work around this problem is to use a dynamic DNS (domain name system) service such as No-IP that assigns a domain name to your IP address, monitors the address for changes, and then modifies the DNS entry to point to the new address automatically. Then you can access your server by domain name without ever having to worry about its actual IP address.

  Most personal server devices and apps (mentioned earlier in this chapter) have other methods of enabling outside devices to discover and reach them.

• NAT problems: Most residential and small-business Internet connections have a router or modem that gets its own public IP address, such as 12.34.56.78—that’s the address the ISP assigns to it, and the address at which the outside world can reach it. But you have a dozen individual devices on your network, so your router uses a system called NAT (network address translation) to give each one of them a private IP address, such as 192.168.0.1.

  This system works pretty well most of the time, but it causes problems for servers, because a device outside your network can’t see private IP addresses, only your public address. So even if you have a dynamic DNS system set up, a client app trying to contact a server on your home network on port 1234 may be unable to figure out how to locate and communicate with the specific device that handles that port.

  Again, there are ways of working around this problem, sometimes requiring fiddling with your home router’s settings for port forwarding, port triggering, or DMZ address (and no, I’m not going to explain that stuff here—look it up on the Web or in your router’s documentation).

• Business policies: If you run your own company, you probably won’t get in trouble for connecting personal server equipment to your company’s network. But if you’re not in charge—and haven’t received special permission from your company’s IT department—putting your own server on the company’s network could lead to severe penalties, because you may be putting your company’s network security at risk even as you’re trying to improve your own. If in doubt, be sure to talk it over with the Powers That Be first.

Someone Else’s Network

If your home or office network is unsuitable as a location for a personal server—or if you want to have equipment in Multiple Locations for redundancy—think about other people you know who have networks of their own. Perhaps a friend, relative, or neighbor will be gracious enough to let you park a device in their home and hook it up to their network. (Freshly baked cookies can often overcome resistance.) Of course, the same potential qualifications apply—there may be ISP restrictions, for example—but the story could be entirely different if your friend uses a different ISP or lives in a different city.

A Data Center

A data center is a building or room that houses servers by the hundreds or thousands—usually mounted in racks that hold dozens of servers in about the space of a refrigerator—and has multiple high-speed connections to the Internet. Data centers have heavy-duty ventilation systems to deal with all the heat from those servers, fancy routers to handle all the traffic, emergency backup power systems, and other systems designed to keep all the data flowing without interruption. All the big Internet companies and ISPs run their own data centers, but many data centers offer their services to any individual or business that needs to rent a bit of space, bandwidth, and power.

When you colocate your own hardware in someone else’s data center, you own and control the hardware, but the data center is responsible for the infrastructure and the physical security.

The great thing about colocation is that you’ll have a far faster and more reliable Internet connection for your server than you could ever get at home. You’ll also benefit from the data center’s backup power and expert staff.

The downside is that even though you retain ownership and legal control over your equipment, it’s outside your physical control. Although a data center likely has strict internal policies about accessing its customers’ equipment, and physical barriers to keep out unauthorized people, the fact remains that someone could, in theory, walk up to your server, plug in a cable, and pull data from it (or intercept the data flowing to and from it). The risk of eavesdropping is lower with colocation than with a public cloud service, but higher than with equipment in your house.

Thousands of companies offer colocation services. Some of them even let you rent or buy a physical server directly from them, so you never have to touch the hardware yourself. Although colocated servers are usually rack-mounted slabs, some companies will colocate a Mac mini, a Transporter (described earlier), an Apple Time Capsule, a NAS (network-attached storage) device, or other network hardware.

A few examples to get you started:

• Macminicolo.net is a Las Vegas-based data center that specializes in—you guessed it—Mac mini and Mac Pro servers, but also colocates servers of other kinds. (Its parent company, MacStadium, offers a wider range of services, operating systems, and locations.) This is just the best-known of numerous similar companies—do a Web search for “Mac mini colocation” and you’ll see what I mean.
• Rackspace offers colocation as well as VPS, shared hosting, and other options.
• ServerPronto advertises itself as “The World’s Most Affordable Colocation.”

As you know, however, a server need not be a physical server. You can also install and run an entire operating system as a virtual server.

A Virtual Server

If you’re thinking about colocating a stand-alone server in a data center, you might instead consider running your own virtual server (see Virtual Private Servers). Like a physical server in a data center, your virtual server would have access to terrific bandwidth and uninterruptible power. But a VPS can be scaled up or down as your needs change, and the hardware layer—things like malfunctioning hard drives and worn-out power supplies—would all still be somebody else’s problem.

Of course, like a physical server, it would be under someone else’s physical control, so the same security and privacy considerations would apply—your server will be safer than a public service, but less safe than hardware in your house.

Nearly every operating system is available as a VPS. Linux (in its numerous distributions) is the most popular choice, but macOS and Windows can also run as virtual machines, so you’re free to choose the operating system you’re most comfortable with.

If you don’t want to start from scratch assembling all the necessary cloud software, you can find prebuilt virtual machines with a full range of cloud services built in and ready to run on your VPS. For Macs, you’ll want a VPS host that offers macOS Server (such as HostMyApple VPS, vmOSX, or Xcloud). If Linux is more to your taste, check out Citadel, Cozy, or Sovereign. (And don’t forget that if you have access to a real or virtual Linux server, you can also run ownCloud or Nextcloud, discussed earlier under Personal Cloud Server Software.)

Multiple Locations

To this point, I’ve largely pretended that running a single server of your own that provides services similar to those available in the public cloud is all it takes to have a “personal cloud.”

But one server does not truly make a cloud. To my way of thinking, a personal server is a useful object that can substitute for cloud services to some extent. But it you really want a personal cloud, you’ll want multiple devices, in multiple locations, in order to ensure that you can still access your services if your home Internet connection goes offline, hardware wears out, a power outage occurs, or some other random thing goes wrong.

In some cases, using multiple servers is trivially simple. For example, if you buy two or more Lima Ultra or Drobo 5N2 devices, then after configuring them to use your account, you can attach them to different networks anywhere in the world—say, relatives’ houses far and wide—and without further ado (excepting the odd occasional need to futz with router settings), you have your own cloud of storage devices. They automatically replicate their data to each other, and as long as one of them, somewhere in the world, is online, you’ll be able to access your data without any extra steps. (If you use CrashPlan for online backups, you can set up a peer-to-peer backup arrangement that functions much the same way—using several friends’ computers, rather than the company’s data center—as a self-sufficient private cloud.)

With most other servers, unfortunately, you’re looking at tons of additional work to approximate that functionality. It’s possible to achieve this effect with, say, a set of Macs running macOS Server, or VPS instances running Linux. But first you’ll have to concoct scripts to keep all the data on the servers in sync with each other, so that if one has to take over, it’s always up to date. Then you must set up your DNS service to route requests for data to one server or another intelligently.

One way to do this is called round robin DNS—the first request goes to Server A, the second goes to Server B, and so on with as many servers as you have until you get back to Server A. Another approach (which you can use separately or in combination with round robin DNS) is automatic failover, which means that if Server A becomes unreachable, the DNS records are changed to point all requests for that server at Server B until Server A comes back online.

If that last paragraph made your head spin…first of all, sorry! And secondly, just wait till you try doing that all yourself. Unless you’re a serious network geek, that sort of thing can drive you half mad.

For most ordinary folks, who really are just never going to mess with their own DNS settings, Linux installations, and syncing scripts, the choice boils down to this:

• Use a personal cloud only in extremely limited ways, such as only for file syncing and sharing.
• Settle for a “cloud” that’s really just one server in one place, with its attendant risks and limitations.
• Give up and use public cloud services after all.

And you know what? Any of those is a perfectly valid conclusion. Your choice is not a matter of what’s ideal as much as which set of risks and complications you can tolerate. For many people, relying on the public cloud is absolutely the right choice, and if I’ve done nothing in this chapter but convince you of that, I consider it pages well spent.

Joe’s Recommendations: Personal Cloud

Should you create a personal cloud at all? If so, which approach should you take, and which product(s) should you use? There’s no right answer for everyone, but allow me to offer a few thoughts:

• Personal cloud products give you significantly more control over privacy and security than public cloud services, doubly so when referring to appliances you can have under your physical control as opposed to software running on a remote (real or virtual) server you own. If you want capabilities roughly comparable to those in Dropbox, iCloud, or G Suite but don’t trust the big corporations with your data, a personal cloud might give you greater peace of mind.
• You can likely save money with personal cloud products, at least in the long run. For example, you might pay $200 for a Lima Ultra and an external hard drive to go with it, about the same as two years of service from Dropbox. But the Lima hardware is a one-time purchase rather than an ongoing subscription, and you get to decide how much storage you have.
• The flip side of the previous point is that you become responsible for maintenance—replacing worn-out hard drives, installing software updates, troubleshooting errors, and so on. If you’re lucky, your personal cloud products might require only minutes of maintenance per year. If you’re unlucky, the sky’s the limit.
• Anything you install on your home network that you or other people need to access from other locations will be subject to the speed and availability of your home broadband connection. I can pretty much guarantee you that neither will come anywhere close to what a commercial cloud provider can offer.

Keeping the above facts in mind, if your main goal is making files available to yourself across multiple devices, Lima Ultra is a good choice. If you want to do fancier sharing with more people, or add apps and media streaming, an inexpensive NAS device might be more to your liking. (I happen to like the Drobo 5N2, which is easier to set up than most, but there are many fine choices.)

If your personal cloud aspirations extend to apps like word processing and video conferencing—and you have the technical chops to install Linux software on a VPS or shared host, I’d give the nod to Nextcloud.