Examples

Let’s consider a few examples. Assuming that you have a simple JSON file, like this one:

{ "val": 5, "txt": "Hello" }

then you can read it (and access its properties) like so:

d3.json("simple.json").then(res=>console.log(res.val,res.txt));

JSON parsers are picky; make sure your JSON is correctly formed! (In particular, JSON property keys must be double-quoted strings, in marked contrast to the syntax for JavaScript object initializers.)

It is also easy to fetch a bitmap image and attach it to the document (or page):

d3.image( "image.jpg" ).then( function(res) {
    d3.select( "#figure" ).append( () => res ) } );

This code assumes that the page contains a placeholder with an appropriate id attribute (for example, <div id="figure">...</div>). Note the argument to the append() function: it is a function, taking no arguments, and returning the result of the fetch! The reason for this roundabout route is that append() can handle either a string or a function returning a node, but not a node by itself. The result of the fetch is a node, hence it is necessary to “wrap it into a function” like this.

Finally, let’s assume that you have an SVG file, for example, containing the definition of a symbol that you would like to reuse:

<svg xmlns="http://www.w3.org/2000/svg"
     xmlns:svg="http://www.w3.org/2000/svg">
  <defs>
    <g id="heart">
      <path d="M0 -3 A3 6.6 -35 1 0 0 6 A3 6.6 35 1 0 0 -3Z" />
    </g>
  </defs>
</svg>

You can insert the <defs> section into the current document and use the defined symbol like this:

d3.svg( "heart.svg" ).then( function(res) {
    d3.select("svg").insert( ()=>res.firstChild,":first-child");
    d3.select("svg").append( "use" ).attr( "href", "#heart" )
        .attr( "transform", "translate(100,100) scale(2)" );
} );

Again, note how the result of the fetch is “wrapped into a function.” In this snippet, res is a DOM SVGDocument instance (not a D3 data type), hence you must use the native DOM API to extract the <defs> element (the first and only child of this SVG document).

The external SVG file in this example contains declarations for XML namespaces. We have not worried about XML namespaces so far (because D3 for the most part takes care of them for us). Here, they are required, because otherwise the SVG parser will not work correctly.

Controlling Fetches with the RequestInit Object

Most of the time, the functions in Table 6-1 just work without any further tweaking. But the simplicity and convenience of the API hides underlying complexity (and sometimes obstructs proper diagnosis if something goes wrong). The means to control aspects of the remote communication is the RequestInit object that all functions in Table 6-1 accept as an optional parameter.²

d3.svg( "heart.svg", { cache: "no-store" } ).then( ... );

function upload() {
    var out = new FormData();                                     

    d3.selectAll( "svg" ).each( function() {                      
        var id = d3.select( this ).attr( "id" );                  
        if( id ) {
            out.set( "filename", id );
            out.set( "data", this.outerHTML );                    

            d3.text( "http://localhost:8080/upload",              
                     { method: "POST", body: out } )
                .then( function(r) { console.log("Succ:", id) },  
                       function(r) { console.log("FAIL:", id) } );
        } } );
}

Field Value Conversions

Whether you use parse() or parseRows(), the field values are strings; values are not automatically converted to numbers. This does cause problems occasionally when other parts of the program require numerical input; for this reason it is good practice to always convert input to numbers explicitly.

You can supply an optional second argument to either parse() or parseRows() to perform such conversions or other desired clean-ups. This function will be invoked for each input line after the line has been split into fields and turned into an object or array. It is therefore not intended to parse each row into fields but to apply conversions to the individual field values.

The conversion function will be passed three arguments:

• The field values in the current row (as object or array)

• The line number of the current row (starting at zero, not counting the header line)

• An array of column names (for parse() only)

The conversion function should return an object or array representing the current line (or null or undefined to skip the current line).

For data files containing only strings, numbers, and dates, you can use the built-in conversion function d3.autoType(), which converts entries that “look like” numbers or dates to the appropriate type. For more complicated situations, you have to write your own conversion function. Consider the following CSV file:

Year,Month,Name,Weight (kg)
2005,1,Peter,86.3
2007,7,Paul,72.5

The following code will turn it into an array of objects with lowercase member names and using appropriate data types:

d3.text( "csv.csv" ).then( function(res) {                    
    var data = d3.csvParse( res, (d,i,cs) => {
        return {
            date: new Date( d.Year, d.Month-1 ),              
            name: d.Name,                                     
            weight: +d["Weight (kg)"]                         
        };
    } );
    console.log( data );
} );

Load as plain text, parse, and convert in callback.

Combine two columns into Date type. (JavaScript’s month index is zero-based.)

Convert property name to lowercase.

Eliminate invalid property name, convert value to number.

Parsing Input Containing Arbitrary Delimiters

The preceding code snippet uses the convenience function d3.csvParse(), which assumes a comma-separated file. Because comma- and tab-separated files are so common, D3 provides a set of shorthands for them (see Table 6-3). For arbitrary delimiters you must first instantiate a parser-formatter instance using d3.dsvFormat(delim), then invoke parse(), format() (or parseRows(), and formatRows()) on this instance, while supplying the input string to parse (or the array to format into a string). The explicit delimiter argument is mandatory; it should be a single character. On the other hand, you can parse a resource when fetching it in one fell swoop using d3.csv(). Here are three ways to achieve the same effect (data will always be an array of objects, use d3.csvParseRows() or parser.parseRows() otherwise):

d3.csv( "csv.csv" ).then( function(res) {
    var data = res;
} );

d3.text( "csv.csv" ).then( function(res) {
    var data = d3.csvParse( res );
} );

d3.text( "csv.csv" ).then( function(res) {
    var parser = d3.dsvFormat( "," );
    var data = parser.parse( res );
} );

Generating Tabular Output

The functions format() and formatRows() implement the inverse operation: serializing a data structure into a string. The input must be an array of objects (for format()) or an array of arrays (for formatRows()). The format() function takes as additional, optional arguments a list of object property names to be included in the output; if this is omitted, a union of all property names found across the entire input is used. Fields in the created string are separated using the specified delimiter, records are separated using newlines ( ), and fields are quoted as necessary.

Using Regular Expressions to Parse Whitespace-Separated Data

Even if data files do not conform to the formats required by the methods just described, the infrastructure provided by them can still be useful. For example, consider the case of a data file with columns separated by whitespace: any combination of tabs and spaces. This is a situation calling for regular expressions, and the following snippet shows how to use them in conjunction with the overall framework:

d3.text( "txt.txt" ).then( function(res) {
    var parser = d3.dsvFormat( "" );                          
    var rows = parser.parseRows( res, function(d, i, cs) {    
        return d[0].split( /s+/g ).map( x => +x );         
    } );
    console.log( rows );
} );

Create a parser-formatter instance, selecting a delimiter character that you are certain does not occur in the input file. (The empty string seems to work, but the ASCII NUL character "", or any other character that you are sure won’t occur, present alternatives.)

Because the delimiter does not occur in the input, no separation into columns is performed (but the input is correctly broken into lines or records)…

… so that the array d of “column” values has only a single element. The split() function is invoked on it with a regular expression that matches any combination of whitespace characters.

Finally, all resulting field values are converted to numbers.

Locales

There are two functions to obtain a locale object (see Table 6-4). Both require a locale definition as input. A locale definition specifies details such as the currency symbol, prevailing number formats, names of months and weekdays, and so on (see the D3 Reference Documentation for details). Locale definitions in a format suitable for D3 are available from https://unpkg.com, a repository for content packaged using the JavaScript package manager npm. The following snippet shows how to fetch and use a new locale definition. It prints the string 3,1316 to the console, following the German convention of using a comma (not a point) as a decimal indicator:

d3.json( "https://unpkg.com/d3-format/locale/de-DE.json" ).then(
    function( res ) {
        var loc = d3.formatLocale( res );
        console.log( loc.format( ".4f" )( Math.PI ) );
    },
    function( err ) {
        throw err;
    }
);

Formatters

A locale object serves as a factory for formatters. Once you have chosen a locale (either a specific one or the default locale), you use it to obtain a formatter instance by providing the intended output format as a string. The formatter is a function object, which takes a single number and returns a formatted string. (It is not possible to create a string containing multiple formatted values simultaneously, in contrast to printf().) All the factory functions that produce a formatter instance are listed in Table 6-5.

In addition to the usual formatter, there is also a special formatter that expresses all quantities in a fixed engineering unit. For instance, it will express all quantities in “kilos,” “millis,” or whichever scale you choose. (See the D3 Reference Documentation or the Wikipedia Metric Prefixes page for all available prefixes.) This behavior is not indicated through the supplied output format specifier; instead, you must use a special factory function to obtain a formatter with this behavior. For example:

d3.formatPrefix( ".4f", 10e-3 )(Math.PI) === "3141.5927m"
d3.formatPrefix( ".4f", 10e3 )(Math.PI)  === "0.0031k"

A formatter object can be reused to format several values. This snippet turns an array of numbers into an array of formatted strings:

var f = d3.format( ".3f" );
[ Math.E, Math.PI ].map( f );

Format or Conversion Specifiers

The format specifier string can consist of up to nine different fields:

[[fill]align][sign][symbol][zero][width][,][.precision][type]

See Table 6-6 for the permissible values and their effects. Be aware that you don’t use % in the format specification (in contrast to what’s customary for printf()).

D3 includes some functions that can help construct format specifiers. First among these is the function d3.formatSpecifier(fmt) that parses the specifier fmt into its constituent fields. You can now inspect individual values, even change some of the fields (possibly programmatically), and then glue the values together again to obtain a new specifier based upon the old one. Other functions (d3.precisionFixed(), d3.precisionPrefix(), and d3.precisionRound()) help you find the proper value for the precision field in a specifier, given the finest resolution (that is, the smallest difference between consecutive values) that you still would like to be visible. These functions are used internally (for instance, to determine the appropriate format for axis tick marks), but they are available for general uses as well.