The Elsevier Article of the Future project: a novel experience of online reading
Elena Zudilova-Seinstra, Martijn Klompenhouwer, Frans Heeman and IJsbrand Jan Aalbersberg
Abstract:
The Article of the Future initiative aims to revolutionize the traditional linear format of the academic paper to make it more dynamic and user-friendly. In this chapter, we present the outcomes of the Article of the Future project, with an emphasis on its final designs, the user feedback collected, and how the Article of the Future handles the rising need of connecting the formal scientific record with associated discipline-specific data. Our main goal was to offer new content and tools that would help researchers in their specific scientific needs. The project team worked with more than 150 people in the scientific community to come up with the article prototypes for the simple-to-read online design. The redesigned article format is now available on ScienceDirect, while our journey continues. Right now, we are gradually bringing new domain-specific functionalities to online articles on ScienceDirect.
Introduction
With the transition from print to online publishing, the layout and presentation of scientific articles have remained relatively unchanged, still following a centuries-old format. Meanwhile, the amount of available scientific supplementary content on the Internet grows exponentially every year (Anderson, 2008). The integration of all this scientific information with the article is necessary to keep publishing efficient and to maintain control over the process. Yet, in most cases, publications and related content are processed separately as single objects and the connections between them are not easy to find (Bourne, 2005). Hence, the infrastructure around a scientific publication is becoming a more prominent feature as the focus of attention is expanding from the formal publication record to include the structure and pattern of linkage between the various research elements associated with the publication (Smit, 2011). However, providing the user with such additional content at the right moment and in the right place remains a challenge. Elsevier’s Article of the Future is an ongoing project focusing on these (and other) important issues.
The Elsevier project explores better ways of presenting online journal articles and enriching their content (Aalbersberg and Kahler, 2011). It was launched in 2009 as an initiative of the life sciences journals of Cell Press, an Elsevier imprint (Marcus, 2010). After the successful completion of this initial stage, Elsevier expanded the project by exploring how the concept could be used in other domains: parasitology and tropical diseases, electrochemistry, materials science, psychology and cognitive science, mathematics and theoretical computer science, paleontology, and business management. In June 2011, Elsevier publicly released 13 Article of the Future prototypes for these domains. Since then, the prototypes have attracted a lot of positive attention from members of the scientific community.
In this chapter, we provide an overview of the main project findings that led us to the development of the new article design layout, which was introduced on the ScienceDirect publishing platform in 2012, and we report the results of several independent user studies conducted by the Article of the Future project team.
Redesign of the article presentation
Through one-to-one user interviews, group discussion sessions and observations, usability tests and on-site visits, the Article of the Future project team inventoried the needs of the scientific community and gathered in-depth knowledge of user tasks and workflows specific to each of the selected subject domains. Our main goal was to offer new content and tools that would help researchers in their specific scientific needs, thus avoiding the current ‘one-size-fits-all’, print-focused publishing solution.
More than 150 researchers, authors and editors participated in the concept phase of the project. They suggested ideas for how articles in their field could be improved by taking full advantage of today’s online capabilities. We set up online user forums to discuss and evaluate these ideas and, later, prototypes were developed from the ideas that were highly rated by the domain experts. Prototypes varied from simple sketches to complete functioning systems.
The final Article of the Future prototypes, publicly released by Elsevier in June 2011 (and still available to view at http://www.articleofthefuture.com), were designed to secure maximal user value out of the online article in the (current) HTML format. In close collaboration with domain experts, the following guiding design principles were formulated, which helped us in making design decisions:
Optimal reading of the article is the basis. The Article of the Future format supports the well-accepted and highly appreciated ‘PDF look and feel’ for easy online reading.
In-place content enrichments. To benefit contextual presence, article enrichments are unobtrusively integrated in-place, providing a clear layout and interactive design.
Additional content and context presented in the right sidebar. Supplementary content and features, as well as information from external databases, are offered next to the article text – enabling sideby-side viewing of article content and associated context.
Clear navigation. The outline of the article links to sections and figures in the text.
Clean design. The visual load is minimized to encourage users to explore the article.
Customized view. The user interface automatically adjusts to the available screen size.
From the first stages to the end of the project, reliable and structured user feedback was used by the project team to shape the design of the Article of the Future. Our findings indicate that there is a definite need for discipline-specific improvements but also that scientists tend to prefer traditional PDF format over HTML when reading scientific journals. In fact, researchers use PDFs to store and organize articles, make annotations while reading, quote by referring to a specific page, and mail articles to colleagues. Finally, the PDF article format prints and reads much more easily than the current online versions.
Therefore, our starting point in the Article of the Future design was to introduce the discipline-specific content enrichments that only the online format could provide over and above the advantages of a PDF-like reading experience. In this way, we wanted to ensure that the new article format would be used and appreciated, and that users would not disregard it solely for readability reasons.
Readability
Extreme care was taken to ensure that the article layout is as easy to read as possible. Users often noted that they would rather read an article in PDF format than the online version. With this in mind, the Article of the Future prototypes recreated what people liked most about reading articles in PDF format.
The alignment of type is justified. A wider line spacing and comfortable negative (white) space were implemented to maintain the focus on the actual article rather than on the surrounding framework (Ling and van Schaik, 2002). The number of words per line has been reduced to provide a more comfortable reading experience. Links do not stand out, allowing for a better reading flow. Highlighting of links is an extra feature that is available as a personalization option.
Due to the increasing popularity of the tablet PC among researchers, specific attention was paid to ensuring that all prototypes can be easily explored by those users who prefer the iPad to a standard desktop or laptop computer.
Discoverability and extensibility
Emphasis was placed on the interactive and discipline-specific content elements within the article, clearly pointing out the added value of the online version. By providing a clean, simplified and minimalist look while adding hierarchy to article elements, we are encouraging users to focus on the article while making the format suitable for long periods of on-screen reading.
Instead of trying to push all the available features to the same level of attention, we allow users to explore the content and discover related content and features in a natural way, while they read the article. For instance, whenever an interactive version of a figure is available, this can be opened within the article, switching between the static and the interactive versions. Examples include videos, interactive graphs, tables and geospatial maps.
In order to maximize the readability of the article, any additional content and features are presented separately. At any time, only one set of features is displayed, with an option to switch between feature sets. Depending on the article content, different feature sets can be displayed (e.g., subject-specific features). This approach ensures that there is no extreme growth of features and elements that clutter the page and detract from the readability of the article.
Three-pane-based content exploration
All of our final prototypes support the three-pane presentation layout: navigation bar on the left, main content area in the middle pane, and a right sidebar. Since each pane can be scrolled independently, it is possible to have both the text (in the middle pane) and an image (in the right sidebar) in view at the same time (see Figure 15.1).
Left navigation pane
The left navigation pane displays a table of contents with clickable section headers and thumbnails of images and tables. It provides a high-level overview of the article and allows easy and clear navigation within the article. There are two options for navigation: with figure and table thumbnails displayed, or without thumbnails displayed.
Middle content pane
The middle content pane displays the traditional article and provides the ‘PDF-like look and feel’ for easy online reading, while the original article is enriched with the capability to interact with and explore the underlying scientific data.
Thus, interactive (Google) maps provide the reader with interactive access to the supplementary geospatial data within the article, which is usually hidden behind a static (often black and white) figure. Such maps are based on KML geographic annotation files (Gibson and Erle, 2006) created by the authors. For earth scientists and paleontologists, access to related ‘geo-time’ content (see, for example, Figure 15.2) is often crucial for gaining an understanding of the research described in the article and for drawing correct conclusions.
Functionality of interactive graphs and charts was prompted by scientists from three different domains: electrochemistry, materials sciences, and psychology. Finding exact values of the data points represented in a graph is often necessary in order to be able to understand, interpret and compare results. Interactive graphs provide automatic tools to support such needs.
Often, scientists print an article and then use a ruler to draw horizontal and vertical lines on a printed figure to determine what values correspond to a specific point of interest on the graph. Depending on the resolution and quality of the image, those measurements are not always very accurate. The interactive crosshairs chart or position graph (Milne, 1974) shown in Figure 15.3 uses intersecting lines in the shape of a cross to allow readers to get interactive access to the underlying data at any location on the graph.
Paleontologists often print and annotate (by hand) individual images from big composite plates. Normally, those plates have one long caption available for a complete plate, which can be positioned on a different page (if the plate and the caption do not fit on a single page). Having one long caption makes it extremely difficult to know which individual image belongs to which part of the long caption, and vice versa. To address this concern, we implemented interactive plates (see Figure 15.4) for the Article of the Future prototypes in paleontology. Displayed in the middle pane, those plates allow the user to read each individual caption by keeping the mouse cursor over the image of interest. It is also possible to zoom into the image of interest and to view it in a full-screen mode.
Right pane
The right pane in the Article of the Future prototypes provides access to supplementary information and additional features which are usually task or content specific. All extra features can be accessed directly from the article or a dropdown menu. Since the right pane uses a dedicated area of the screen new features can be added easily without competing for the same screen estate.
Depending on the domain specificity and the article content, different functionality can be offered. In the Article of the Future prototypes for electrochemistry, structures of chemical compounds are displayed and information on the experimental set-ups is given (see Figure 15.5 [left] – the 3D viewer was developed using CanvasMol [http://alteredqualia.com/canvasmol/]). Psychology prototypes provide access to the experimental flowcharts (see Figure 15.5 [right]) and stimuli examples. Paleontology prototypes feature 3D fossil models and provide information on spatial distribution of the places where they were found.
Features currently presented in the right pane of the Article of the Future prototypes were selected because they were highly rated by the scientists either via the user forum and/or a series of individual interview sessions conducted as part of the Article of the Future project.
For instance, many mathematicians say that it is ‘a pain to search for formulas in the article’ as it requires one to flip back and forth constantly through the article. This makes reading tedious and inconvenient. In the Article of the Future prototypes for mathematics and theoretical computer science, all mathematical expressions can be viewed in the right sidebar (one by one) and accessed directly from the article via the cross-referenced links in the text, as seen in Figure 15.6. This feature was very highly rated by the scientists after the prototypes were publicly released.
In articles related to parasitology and tropical diseases proteins are usually only briefly referred to in the text, even though it is often extremely useful to be able to get more information on those proteins. Displaying additional protein information in the right sidebar (see Figure 15.7) is appreciated highly by scientists as they can get access to all important data without having to interrupt their reading and search for the relevant record in the Protein Data Bank (http://www.rcsb.org/pdb/home/home.do).
Comparing the Article of the Future with traditional publications
In order to compare the Article of the Future presentation layout with the pre-2012 ScienceDirect online articles (as available online until early 2012), we conducted several comparative user tests – where users were asked to perform trial tasks in either layout. Many participants performed these tasks in their usual office settings. However, if possible, we also ran user tests at the Elsevier Usability Lab in Amsterdam, as well as at other specially equipped locations, for example in the Vrije Universiteit (VU University) Intertain Lab (http://www.cs.vu.nl/en/about-the-department/intertain-lab/index.asp). To monitor and analyse the observed user behaviour optimally, we applied state-of-the-art evaluation techniques and methods.
The eye tracking study
In our first preliminary study with a relatively small number of subjects, we applied the eye tracking technique to identify and analyse people’s patterns of visual attention when reading scientific articles on-screen (Zudilova-Seinstra, 2013). The participants had a background in various sub-domains of the social sciences.
The experimental set-up for this study was provided by the VU University Intertain Lab. The Tobii eye trackers available in the lab were calibrated individually for each participant prior to the start of the experiment. Although the experiment was completely automated, it was possible to observe user behaviour from a different room using a VNC server and to assist participants when necessary. We used Tobii Studio 2.1 (http://www.tobii.com) to collect, process and analyse the experimental data.
Participants were asked to search for specific information using the same article in the Article of the Future format and in the pre-2012 ScienceDirect format. Due to set-up limitations the Article of the Future prototype was always shown first. Consequently, participants had a slightly better understanding of the task when they had to read the article in the pre-2012 ScienceDirect format, which was displayed second. It is possible that this had some beneficial effect on task performance for the pre-2012 ScienceDirect condition. Despite this, our first pilot study showed a time improvement of up to 40 per cent for participants searching for information with the Article of the Future prototype.
We believe that this improvement is due to the fact that participants explored the same article in two alternative formats differently. In the Article of the Future format, all participants first spent some time scanning research highlights and viewing the graphical abstract. Then they used the left navigation bar to get quick access to the section of the article in which the relevant information could be found (see Figure 15.8 [top]). In the pre-2012 ScienceDirect format, participants either quickly scanned through the complete article or first scrolled to the article outline and then clicked on the link that could bring them to the section in which the information could be found (see Figure 15.8 [bottom]).
The differences in the article page design leading to these different searching and scanning strategies resulted in better performance, demonstrating that the new Article of the Future format is an improvement in efficiency over the pre-2012 ScienceDirect layout. This performance improvement provided the motivation to conduct a more serious comparative study (described in the next subsection) specifically focusing on user performance and behaviour for the tasks related to the literature study.
Furthermore, the eye tracking data indicated that in the Article of the Future format the participants’ main focus remains on the middle article pane, investigating the left and right panes only when looking for additional information: participants scanned the central area of the article page first, then the left pane and then the right sidebar (for further details see the two heat maps shown in Figure 15.9).
Evaluation of user behaviour and performance
Inspired by our first pilot study described above, we conducted a follow-up empirical study to compare the latest design variant of the Article of the Future and the traditional (pre-2012) ScienceDirect article design. Sixty-four scientists (30 male, 34 female) in the first-half of their scientific career participated in this study. To avoid practice and carry-over effects the design of the experiment was between-subjects for the type of the article presentation (pre-2012 ScienceDirect vs. Article of the Future prototype environment).
Researchers from various fields (e.g., cell biology, chemical engineering, ecology, earth and planetary sciences, food technology, psychology) were asked to perform three trial tasks related to how they usually use information when performing a literature study. Based on their research background, participants were assigned to the Article of the Future prototype or the pre-2012 ScienceDirect condition in such a way that a similar number of participants from the same research domain was assigned to both. Most of the researchers were employed at a Ph.D. level or higher, and in the age range 25–35. Forty individual test sessions took place in the Elsevier Usability Lab in Amsterdam; the remaining 24 sessions were held at various university locations in the Netherlands.
Performance measurements were the key focus of this study. Quantitative measurements were taken on the number of times certain functionalities were used, as well as on the time spent on various tasks and functionality usage. Test sessions were video recorded using Camtasia Studio 7 (see Figure 15.10) and the user behaviour was processed manually based on the list of variables defined prior to the study. To analyse quantitative data, we applied one-way analysis of variance (ANOVA) (Christensen, 2011).
The experimental data showed that users were spending more time reading and scanning journal articles online in the new article format. As can be seen in Figure 15.11 (left), there is a huge difference in the usage of HTML compared to PDF with the Article of the Future prototype: on average, 20 times more time was spent on using HTML (F1, 30=152.36 p<0.001).
Also, ANOVA found that per article in the Article of the Future format more time was spent on the HTML usage (F1, 61=11.11 p=0.001) and less time on the PDF usage (F1, 61=13.55 p=0.005) than in the pre-2012 ScienceDirect format (Figure 15.11 [right]).
These two findings suggest that the HTML article presentation in the Article of the Future format can definitely compete with the traditional PDF format generally preferred by scientists.
We also found that additional content associated with the article was used more often with the Article of the Future prototype compared to pre-2012 ScienceDirect (Figure 15.12 [left], F1, 61=9.72 p=0.003). This suggests that additional article content or context is better integrated and more easily accessible in the new article design. Also, many participants (59 per cent) said that they would definitely choose the Article of the Future presentation layout over existing alternatives (Figure 15.12 [right]).
Following on from our first pilot (described in the previous subsection), we investigated task-related performance differences. Due to the large variability in the background and expertise of the participants, in this study we focused on generic tasks related to the literature study.
We discovered that some tasks could be performed much faster with the Article of the Future prototype. For instance, thanks to the additional functionality offered by the new article presentation format (e.g., left navigation bar, full-size in-place images, easy access to references, and so on) participants were able to determine more quickly whether the article displayed on-screen was irrelevant to their research. The experimental data showed a 34 per cent average time improvement on this task with the Article of the Future prototype (see Figure 15.13 [left], F1, 44=4.51 p=0.038).
Another interesting finding was a reduction in the printing of articles (PDF downloads) with the Article of the Future prototype compared to pre-2012 ScienceDirect (Figure 15.13 [right], F1, 41=4.33 p=0.044). Thanks to the new article design, participants – who regularly perform literature studies – were willing to spend more time reading articles on-screen in the HTML format, and hence we were told by many people that in this new format they would only print articles that they would have to explore in detail (‘because otherwise reading from screen could be tiring’).
User feedback through surveys
With the public release of the Article of the Future prototypes we provided an online survey to be (optionally) completed by visitors. From more than 800 responses received between June 2011 and January 2012, we learnt that:
1. Seventy-five per cent of the respondents agree that the Article of the Future design contributes to obtaining a better understanding of the research described in the article (see Figure 15.14).
2. However, these percentages do vary from discipline to discipline. For instance, while in parasitology and tropical diseases 85 per cent agree with this statement, in mathematics and theoretical computer science only 65 per cent agree (with a much higher percentage of ‘neutrals’).
3. The main elements of the three-pane presentation layout are highly valued by scientists. Left, middle and right panes received positive feedback with average scores of 5.7–6.5 on scale 1–8 (see Figure 15.15). However, since the main content pane represents the article itself, it was rated slightly higher than the other elements (by all domain-specific user groups).
Here is just a small sample of comments on the Article of the Future prototypes left by scientists on its website (http://www.articleofthefuture.com/):
‘Improved hyperlinks between references, equations, etc …are very helpful. Also having the sidebar to see equations or references without loosing your place is very nice.’
‘Easy to find way around article. Don’t have to scroll through pages and pages. Interactive figures are FANTASTIC!’
‘Tablet computing with the Article of the Future is going to replace PDF!’
‘The chance of having handy additional information in the sidebar is very useful, making the argumental line of the article easier to understand, while it improves readability.’
‘… I get more the feeling that I can really approach the data and check it in order to come to the same conclusion (or not) …’
‘Fantastic! A revolutionary and easy way to be up to the minute!’
‘I’m delighted by not needing to flip back and forth to the end of the article to see what the references are.’
‘It is easier to reach the interesting spot and then go back reading.’
Overall, the survey feedback on the Article of the Future design was very positive. User appreciation covered all areas of the design: the easy navigation provided through the left-side pane; the side-by-side display of text and images; the additional information on references on the right; and the interactivity of graphs and plots. Negative feedback mostly concentrated around longer download times and a still-too-cluttered presentation; the former was addressed at release time in January 2012, while improving the latter was an essential part of the right-hand pane release in July 2012.
Conclusion
Elsevier’s Article of the Future project is an example of how design goes much deeper than simply a superficial visual layer. Design drives how we think about our products, based on solid user data. In this project, we harmonized content and context enrichments with presentation, layout, typography and functionality. The Article of the Future prototypes were designed to get maximal user value out of the online article. From the very first stages through to the end of the project, we used reliable and structured user feedback to shape the Article of the Future design direction.
A large number of one-to-one interview sessions, user tests and behavioural and survey studies demonstrate that the Article of the Future format improves both the efficiency and understanding of scientists when browsing and reading scientific articles. The experimental data from our various studies consistently indicate that the new article format, introduced by the Article of the Future project, is very much appreciated by potential users. Furthermore, these users are willing to spend more time reading articles in the new HTML format as it turns the article displayed in a traditional ‘PDF-like’ format into a modern and powerful research tool.
For authors, the Article of the Future delivers greater exposure and a better opportunity to showcase their work. By enabling a higher level of communication and engagement with the scientific community, it enriches and extends the reach of research, giving it more credibility and increasing the likelihood of use and citation.
For readers, the Article of the Future makes it possible to interact with the content and explore subjects further and in more detail, providing deeper insights in an intuitive and flexible way. Researchers save time because they are able to determine the relevancy of an article more quickly, and can use improved linking and navigation to secure immediate access to specific information in the article, images, references, and to externally stored supplementary material. Furthermore, the Article of the Future format is beneficial in the sense of less article prints and hence it also contributes to environmental conservation.
The new article design was introduced into the ScienceDirect publishing platform in 2012, while the Article of the Future project continues to bring new ideas to journal articles (Aalbersberg et al., 2012).