Video on the Web is based on container formats and codecs. A container is a wrapper that stores all the necessary data that comprises the video file being accessed, much like a ZIP file wraps or contains files. Some examples of well-known video containers include Flash Video (.flv), MPEG-4 (.mp4 or .m4v), and AVI (.avi).

The video container houses data, including a video track, an audio track with markers that help synchronize the audio and video, language information, and other bits of metadata that describe the content.

The video container formats relevant to HTML5 are MPEG-4, Ogg, and WebM.

A video codec defines an algorithm for encoding and decoding a multimedia data stream. A codec can encode a data stream for transmission, storage, or encryption, or it can decode it for playback or editing. For the purpose of HTML5 video, we’re concerned with the decoding and playback of a video stream. The video codecs that are most pertinent to HTML5 video are H.264, Theora, and VP8.

An audio codec in theory works the same as a video codec, except it’s concerned with the streaming of sound, rather than video frames. The audio codecs that are most pertinent to HTML5 video are AAC and Vorbis.

It would be nice if browser support allowed us to choose a single container, video codec, and audio codec to create a standard way of embedding video using the new video element in HTML5. Unfortunately, it’s not quite that simple—although things are improving.

In Table 5.1, we’ve outlined video container and codec support in the most popular browser versions. This chart only includes browser versions that offer support for the HTML5 video element.

Opera Mini and Opera Mobile currently offer no support for HTML5 video, but Opera has announced there are plans to include support in upcoming releases.

No embedded video would be complete without giving the user the ability to play, pause, stop, seek through the video, or adjust the volume. HTML5’s video element includes a controls attribute that does just that:

<video src="example.webm" width="375" height="280" controls></video>

controls is a Boolean attribute, so no value is required. Its inclusion in the markup tells the browser to make the controls visible and accessible to the user.

Each browser is responsible for the look of the built-in video controls. Figure 5.1 to Figure 5.4 show how these controls differ in appearance from browser to browser.

Figure 5.1. The native video controls in Firefox 4

Figure 5.2. … in IE9

Figure 5.3. … in Opera 11

Figure 5.4. … and in Chrome

We’d love to omit reference to this particular attribute, since its use will be undesirable for the most part. However, there are cases where it can be appropriate. The Boolean autoplay attribute does exactly what it says: it tells the web page to play the video as soon as possible.

Normally, this is a bad practice; most of us know too well how jarring it can be if a website starts playing video or audio as soon as it loads—especially if our speakers are turned up. Usability best practices dictate that sounds and movement on web pages should only be triggered when requested by the user. But this doesn’t mean that the autoplay attribute can never be used.

For example, if the page in question contains nothing but a video—that is, the user clicked on a link to a page for the sole purpose of viewing a specific video—it may be acceptable to allow it to play automatically, depending on the video’s size, any surrounding content, and the audience.

Here’s how you’d do that:

<video src="example.webm" width="375" height="280" controls 
↵autoplay></video>

Another attribute that you should think twice about before using is the Boolean loop attribute. Again, it’s fairly self-explanatory: according to the spec, this attribute, when present, will tell the browser to “seek back to the start of the media resource upon reaching the end.”

So if you created a web page whose sole intention was to annoy its visitors, it might contain code like this:

<video src="example.webm" width="375" height="280" controls 
↵autoplay loop></video>

Autoplay and an infinite loop! We just need to remove the native controls and we’d have a trifecta of worst practices.

Of course, there are some situations where loop can be useful: imagine a browser-based game, in which ambient sounds and music should play continuously as long as the page is open.

In contrast to the two previous attributes, preload could definitely come in handy in a number of cases. The preload attribute accepts one of three values:

This particular attribute does not have a spec-defined default in cases where it’s omitted; each browser decides which of those three values should be the default state, which makes sense. It allows desktop browsers to preload the video and/or metadata automatically (having no real adverse effect) while permitting mobile browsers to default to either metadata or none, as many mobile users have restricted bandwidth and will prefer to have the choice of downloading the video or not.

When you go to view a video on the Web, normally a single frame of the video will display in order to provide a teaser of its content. The poster attribute makes it easy to choose such a teaser. This attribute, similar to src, will point to an image file on the server by means of a URL.

Here’s how our video element would look with a poster attribute defined:

<video src="example.webm" width="375" height="280" 
↵poster="teaser.jpg" controls></video>

Although the poster attribute is useful, there’s a bug in iOS3 (corrected in iOS4) that prevents playback of the video if this attribute is present. If you know that many of your visitors use iOS 3.x, you should either avoid using the poster attribute, or remove it for those devices specifically.

The audio attribute attribute controls the default state of the audio track for the video element, and currently accepts only a single possible value: muted. The spec states that other values are likely to be added in the future, for specifying the default audio track or volume, for example.

A value of muted will cause the video’s audio track to default to muted, potentially overriding any user preferences. This will only control the default state of the element—the user interacting with the controls, or JavaScript can change this.

Here it is added to our video element:

<video src="example.webm" width="375" height="280" 
↵poster="teaser.jpg" audio="muted"></video>

As we discussed earlier, using a single container format to serve your video is not currently an option, even though that’s really the ultimate idea behind having the video element, and one which we hope will be realized in the future. To allow inclusion of multiple video formats, the video element allows source elements to be defined so that you can allow every user agent to display the video using the format of its choice. These elements serve the same function as the src attribute on the video element; so if you’re providing source elements, there’s no need to specify a src for your video.

Taking current browser support into consideration, here’s how we might declare our source elements:

<source src="example.mp4" type="video/mp4">
<source src="example.webm" type="video/webm">
<source src="example.ogv" type="video/ogg">

The source element (oddly enough) takes a src attribute that specifies the location of the video file. It also accepts a type attribute that specifies the container format for the resource being requested. This latter attribute allows the browser to determine if it can play the file in question, thus preventing the browser from unnecessarily downloading an unsupported format.

The type attribute also allows a codec parameter to be specified, which defines the video and audio codecs for the requested file. Here’s how our source elements will look with the codecs specified:

<source src="example.mp4" 
↵type='video/mp4; codecs="avc1.42E01E, mp4a.40.2"'>
<source src="example.webm" type='video/webm; codecs="vp8, vorbis"'>
<source src="example.ogv" type='video/ogg; codecs="theora, vorbis"'>

You’ll notice that the syntax for the type attribute has been slightly modified to accommodate the container and codec values. The double quotes surrounding the values have been changed to single quotes, and another set of nested double quotes is included specifically for the codecs.

This can be a tad confusing at first glance, but in most cases you’ll just be copying and pasting those values once you have a set method for encoding the videos (which we’ll touch on later in this chapter). The important point is that you define the correct values for the specified file to ensure that the browser can determine which (if any) file it will be able to play.

In our example above, the MP4/H.264/AAC container/codec combination is included first. This is to ensure that the video will play on the iPad. On that device, a bug causes only the first source element to be recognized. It’s safe to assume that this bug is fixed in subsequent versions of the iPad, but for now it’s necessary to include the MP4/H.264 file first to ensure compatibility.

The first source element will be recognized by IE9, Safari, and older versions of Chrome, so that covers quite a large chunk of our HTML5-ready audience.

The next element in the list defines the WebM/VP8/Vorbis container/codec combination. This is supported by later versions of Chrome that will eventually drop support for H.264. In addition to Chrome, WebM video will also play in Firefox 4 and Opera 10.6.

Finally, the third source element declares the Ogg/Theora/Vorbis container/codec combination, which is supported by Firefox 3.5 and Opera 10.5. Although other browsers also support this combination, they’ll be using the other formats since they appear ahead of this one in the source order. The browsers that support only this combination are older versions of browsers whose current versions support other formats, so it will be possible to drop this format once those versions become sufficiently rare.

These three source elements are placed as children of the video element, so with our three file formats declared, our code will now look like this:

<video width="375" height="280" poster="teaser.jpg" audio="muted">
  <source src="example.mp4" type='video/mp4; 
↵codecs="avc1.42E01E, mp4a.40.2"'>
  <source src="example.webm" type='video/webm; 
↵codecs="vp8, vorbis"'>
  <source src="example.ogv" type='video/ogg; 
↵codecs="theora, vorbis"'>
</video>

You’ll notice that our code above is now without the src attribute on the video element. As well as being redundant, it would also override any video files defined in the source elements.

The three source elements that we included inside our video element will cover all modern browsers. But we’re yet to ensure that our video will play for a potentially large portion of our audience. As mentioned earlier, a significant percentage of users are still using browsers without native support for HTML5 video. Most of those users are on some version of Internet Explorer prior to version 9.

In keeping with the principle of graceful degradation, the HTML5 video element has been designed so that older browsers can access the video by some alternate means. Older browsers that fail to recognize the video element will simply ignore it, along with its source children. But if the video element contains content that the browser recognizes as valid HTML, it will read and display that content instead.

What kind of content can we serve to those nonsupporting browsers? According to Adobe, 99% of users have the Flash plugin installed on their systems. Additionally, most of those instances of the Flash plugin are version 9 or later, which offer support for the MPEG-4 video container format. To allow Internet Explorer 6 to 8 (and other older browsers lacking support for HTML5 video) to play the video, we declare an embedded Flash video to use as a fallback. Here’s the completed code for the video on The HTML5 Herald, with the Flash fallback code included:

<video width="375" height="280" poster="teaser.jpg" audio="muted">
  <source src="example.mp4" type='video/mp4; 
↵codecs="avc1.42E01E, mp4a.40.2"'>
  <source src="example.webm" type='video/webm; 
↵codecs="vp8, vorbis"'>
  <source src="example.ogv" type='video/ogg; 
↵codecs="theora, vorbis"'>
  <!-- fallback to Flash: -->
  <object width="375" height="280" type="application/x-shockwave-
↵flash" data="mediaplayer-5.5/player.swf">
    <param name="movie" value="mediaplayer-5.5/player.swf">
    <param name="allowFullScreen" value="true">
    <param name="wmode" value="transparent">
    <param name="flashvars" value="controlbar=over&amp;image=
↵images/teaser.jpg&amp;file=example.mp4">
    <!-- fallback image -->
    <img src="teaser.jpg" width="375" height="280" alt="" 
↵title="No video playback capabilities">
  </object>
</video>

We’ll avoid going into a detailed discussion of how this newly added code works (this isn’t a Flash book, after all!), but here are a few points to note about this addition to our markup:

The last thing we’ll mention here is that, in addition to the Flash fallback content, you could also provide an optional “download video” link that allows the user to get a local copy of the video and view it at their leisure. This would ensure that nobody is left without a means to view the video.

If you find that you’ve followed our instructions closely and your videos still won’t play from your server, the issue could be related to the content-type information being sent.

Content-type, also known as the MIME type, tells the browser what kind of content they’re looking at. Is this a text file? If so, what kind? HTML? JavaScript? Is this a video file? The content-type answers these questions for the browser. Every time your browser requests a page, the server sends “headers” to your browser before sending any files. These headers tell your browser how to interpret the file that follows. Content-type is an example of one of the headers the server sends to the browser.

The MIME type for each video file that you include via the source element is the same as the value of the type attribute (minus any codec information). For the purpose of HTML5 video, we’re concerned with three MIME types. To ensure that your server is able to play all three types of video file, place the following lines of code in your .htaccess file (or the equivalent if you’re using a web server other than Apache):

AddType video/ogg  .ogv
AddType video/mp4  .mp4
AddType video/webm .webm

If this fails to fix your problem, you may have to talk to your host or server administrator to find out if your server is using the correct MIME types. To learn more about configuring other types of web servers, read the excellent article “Properly Configuring Server MIME Types” from the Mozilla Developer Network.

For our sample site, we’re going to build a very simple set of video controls to demonstrate the power of the new HTML5 video API. To start off, Figure 5.5 shows a screenshot of the set of controls we’ll be using to manipulate the video.

Figure 5.5.  The simple set of video controls we’ll be building

Both of those buttons have alternate states: Figure 5.6 shows how the controls will look if the video is playing and the sound has been muted.

Figure 5.6.  Our controls again, this time with the sound muted and the video playing

Our controls have three components:

In most cases, your custom video controls should have all the features of the default controls that various browsers natively provide. If your set of controls introduces fewer or inferior features, it’s likely you’ll end up frustrating your users.

For the purpose of introducing the API, rather than trying to mimic what the browsers natively do, we want to introduce the important parts of the video API gradually; this will allow you to get your feet wet while establishing a foundation from which to work.

We’ll be creating a very simple, yet usable, set of controls for our video. The main feature missing from our set of controls is the seek bar that lets the user “scrub” through the video to find a specific part. This means there will be no way to go back to the start of the video aside from refreshing the page. Other than that, the controls will function adequately—they’ll allow the user to play, pause, mute, or unmute the video.

Here’s the HTML we’ll be using to represent the different parts of the video controls:

<div id="controls" class="hidden">
  <a id="playPause">Play/Pause</a>
  <span id="timer">00:00</span>
  <a id="muteUnmute">Mute/Unmute</a>
</div>

We’ll avoid going into the CSS in great detail, but here’s a summary of what we’ve done (you can view the demo page’s source in the code archive if you want to see how it’s all put together):

If you’re following along building the example, go ahead and style the three elements however you like. The appearance of the controls is really secondary to what we’re accomplishing here, so feel free to fiddle until you have a look you’re happy with.

Let’s go through the steps needed to create our custom controls, and in the process we’ll introduce you to some aspects of the video API. Afterwards, we’ll summarize some other methods and attributes from the API that we won’t be using in our controls, so you can have a good overview of what the API includes.

In order to work with our new custom controls, we’ll first cache them by placing them into JavaScript variables. Here are the first few lines of our code:

var videoEl = $('video')[0],
  playPauseBtn = $('#playPause'),
  vidControls = $('#controls'),
  muteBtn = $('#muteUnmute'),
  timeHolder = $('#timer'),

Of course, caching our selections isn’t necessary, but it’s always best practice (for maintainability and performance) to work with cached objects, rather than needlessly repeating the same code to target various elements on the page.

The first line is targeting the video element itself. We’ll be using this videoEl variable quite a bit when using the API—since most API methods need to be called from the media element. The next four lines of code should be fairly familiar to you if you took note of the HTML that comprises our controls. Those are the four elements on the page that we’ll be manipulating based on user interaction.

Our first task is make sure the native controls are hidden. We could do this easily by simply removing the controls attribute from the HTML. But since our custom controls are dependent on JavaScript, visitors with JavaScript disabled would be deprived of any way of controlling the video. So we’re going to remove the controls attribute in our JavaScript, like this:

videoEl.removeAttribute("controls");

The next step is to make our own custom controls visible. As mentioned earlier, we’ve used CSS to remove our controls from view by default. By using JavaScript to enable the visibility of the custom controls, we ensure that the user will never see two sets of controls.

So our next chunk of code will look like this:

videoEl.addEventListener('canplaythrough', function () {
  vidControls.removeClass("hidden");
}, false);

This is the first place we’ve used a feature from the HTML5 video API. First, take note of the addEventListener method. This method does exactly what its name implies: it listens for the specified event occurring on the targeted element.

In this case, we’re targeting the video element itself. The event we’re registering to be listened for is the canplaythrough event from the video API. According to the definition of this event in the spec:

There are other events we can use to check if the video is ready, each of which has its own specific purpose. We’ll touch on some of those other events later in this chapter. This particular one ensures continuous playback, so it’s a good fit for us as we’d like to avoid choppy playback.

When the canplaythrough event fires, a callback function is run. In that function, we’ve put a single line of code that removes the hidden class from the controls wrapper, so now our controls are visible. Now we want to add some functionality to our controls. Let’s bind a click event handler to our play/pause button:

playPauseBtn.bind('click', function () {
  if (videoEl.paused) {
    videoEl.play();
  } else {
    videoEl.pause();
  }
});

When the button is clicked, we run an if/else block that’s using three additional features from the video API. Here’s a description of all three:

The paused attribute is being accessed to see if the video is currently in the “paused” state. This doesn’t necessarily mean the video has been paused by the user; it could equally just represent the start of the video, before it’s been played. So this attribute will return true if the video isn’t currently playing.

Since we’ve now determined that the play/pause button has been clicked, and the video is not currently playing, we can safely call the play() method on the video element. This will play the video from its last paused location.

Finally, if the paused attribute doesn’t return true, the else portion of our code will fire, and this will trigger the pause() method on the video element, stopping the video.

You may have noticed that our custom controls have no “stop” button (customarily represented by a square icon). You could add such a button if you feel it’s necessary, but many video players don’t use it since the seek bar can be used to move to the beginning of the video. The only catch is that the video API has no “stop” method; to counter this, you can cause the video to mimic the traditional “stop” behavior by pausing it and then sending it to the beginning (more on this later).

You’ll notice that something’s missing from our if/else construct. Earlier, we showed you a couple of screenshots displaying the controls in their two states. We need to use JavaScript to alter the background position of our sprite image; we want to change the button from “play me” to “pause me.”

Here’s how we’ll do that:

videoEl.addEventListener('play', function () {
  playPauseBtn.addClass("playing");
}, false);
videoEl.addEventListener('pause', function () {
  playPauseBtn.removeClass("playing");
}, false);

Here we have two more uses of the addEventListener method (you’ll need to get used to it if you’re going to use the video and audio APIs!). The first block is listening for play events. So if the click handler we wrote triggers the play() method (or if something else causes the video to play, such as some other code on the page), the play event will be detected by the listener and the callback function will run.

The same thing is happening in the second block of code, except that it’s listening for the pause event (not to be confused with the paused attribute).

If the element has been played, the first block will add the class playing to our play/pause button. This class will change the background position of the sprite on the play/pause button to make the “pause me” icon appear. Similarly, the second block of code will remove the playing class, causing the state of the button to go back to the default (the “play me” state).

You’re probably thinking, “why not just add or remove the playing class in the code handling the button click?” While this would work just fine for when the button is clicked (or accessed via the keyboard), there’s another behavior we need to consider here, demonstrated in Figure 5.7.

Figure 5.7. Some video controls are accessible via the context menu

The menu above appears when you bring up the video element’s context menu. As you can see, clicking the controls on the video element isn’t the only way to play/pause or mute/unmute the video.

To ensure that the button states are changed no matter how the video element’s features are accessed, we instead listen for play and pause events (and, as you’ll see in a moment, sound-related events) to change the states of the buttons.

The next bit of functionality we want to add to our script is the mute/unmute button. This piece of code is virtually the same as what was used for the play/pause button. This time, we’ve bound the click event to the mute/unmute button, following with a similar if/else construct:

muteBtn.bind('click', function () {
  if (videoEl.muted) {
    videoEl.muted = false;
  } else {
    videoEl.muted = true;
  }
});

This block of code introduces a new part of the API: the muted attribute. After the mute button is clicked, we check to see the status of this attribute. If it’s true (meaning the sound is muted), we set it to false (which unmutes the sound); if it’s false, we set its status to true.

Again, we haven’t done any button state handling here, for the same reasons mentioned earlier when discussing the play/pause buttons; the context menu allows for muting and unmuting, so we want to change the mute button’s state depending on the actual muting or unmuting of the video, rather than the clicking of the button.

But unlike the play/pause button, we don’t have the ability to listen for mute and unmute events. Instead, the API offers the volumechange event:

videoEl.addEventListener('volumechange', function () {
  if (videoEl.muted) {
    muteBtn.addClass("muted");
  } else {
    muteBtn.removeClass("muted");
  }
}, false);

Again, we’re using an event listener to run some code each time the specified event (in this case a change in volume) takes place. As you can probably infer from the name of this event, the volumechange event isn’t limited to detecting muting and unmuting; it can detect volume changes.

Once we have detected the change in volume, we check the status of the video element’s muted attribute, and we change the class on the mute/unmute button accordingly.

The code we’ve written so far will allow the user to play and pause the video, as well as mute and unmute the sound. All of this is done using our custom controls.

At this point, if you let the video play to the end, it will stop on the last frame. We think it’s best to send the video back to the first frame, ready to be played again. This gives us the opportunity to introduce two new features of the API:

videoEl.addEventListener('ended', function () {
  videoEl.currentTime = 0;
  videoEl.pause();
}, false);

This block of code listens for the ended event, which tells us that the video has reached its end and stopped. Once we detect this event, we set the video’s currentTime property to zero. This property represents the current playback position, expressed in seconds (with decimal fractions).

Which brings us to the next step in our code.

Now for the last step: we want our timer to update the current playback time as the video plays. We’ve already introduced the currentTime property; we can use it to update the content of our #timeHolder element. Here’s how we do it:

videoEl.addEventListener('timeupdate', function () {
  timeHolder[0].innerHTML = secondsToTime(videoEl.currentTime);
}, false);

In this case, we’re listening for timeupdate events. The timeupdate event fires each time the video’s time changes, which means even a fraction of a second’s change will fire this event.

This alone would suffice to create a bare-bones timer. Unfortunately, the time would be unhelpful, and ugly on the eye because you’d see the time changing every millisecond to numerous decimal places, as shown in Figure 5.8.

Figure 5.8.  Using the currentTime property directly in our HTML is less than ideal

In addition, the timer in this state will not display minutes or hours, just seconds—which could end up being in the hundreds or thousands, depending on the length of the video. That’s impractical, to say the least.

To format the seconds into a more user-friendly time, we’ve written a function called secondsToTime(), and called it from our timeupdate handler above. We don’t want to show the milliseconds in this case, so our function rounds the timer to the nearest second. Here’s the start of our function:

var h = Math.floor(s / (60 * 60)),
  dm = s % (60 * 60),
  m = Math.floor(dm / 60),
  ds = dm % 60,
  secs = Math.ceil(ds);

After those five lines of code, the final variable secs will hold a rounded number of seconds, calculated from the number of seconds passed into the function.

Next, we need to ensure that a single digit amount of seconds or minutes is expressed using 05 instead of just 5. The next code block will take care of this:

if (secs === 60) {
  secs = 0;
  m = m + 1;
}

if (secs < 10) {
  secs = "0" + secs;
}

if (m === 60) {
  m = 0;
  h = h + 1;
}

if (m < 10) {
  m = "0" + m;
}

Finally, we return a string that represents the current time of the video in its correct format:

if (h === 0) {
  fulltime = m + ":" + secs;
} else {
  fulltime = h + ":" + m + ":" + secs;
}

return fulltime;

The if/else construct is included to check if the video is one hour or longer; if so, we’ll format the time with two colons. Otherwise, the formatted time will use a single colon that divides minutes from seconds, which will be the case in most circumstances.

Remember where we’re running this function. We’ve included this inside our timeupdate event handler. The function’s returned result will become the content of the timeHolder element (which is the cached element with an id of timer):

timeHolder[0].innerHTML = secondsToTime(videoEl.currentTime);

Because the timeupdate event is triggered with every fraction of a second’s change, the content of the timeHolder element will change rapidly. But because we’re rounding the value to the nearest second, the visible changes will be limited to a time update every second, even though technically the content of the timer element is changing more rapidly.

And that’s it, our custom controls are done. The buttons work as expected and the timer runs smoothly. As we mentioned at the top, this isn’t quite a fully functional set of controls. But you should at least have a good handle on the basics of interacting with HTML5 video from JavaScript, so have a tinker and see what else you can add.

The API has much more to it than what we’ve covered here. Here’s a summary of some events and attributes that you might want to use when building your own custom controls, or when working with video and audio elements.

One point to remember is that these API methods and properties can be used anywhere in your JavaScript—they don’t need to be linked to custom controls. If you’d like to play a video when the mouse hovers over it, or use audio elements to play various sounds associated with your web application or game, all you need to do is call the appropriate methods.