“You can optimize everything and still fail, because you have to optimize for the right things. That’s where reflection and qualitative approaches come in. By asking why, we can see the opportunity for something better beyond the bounds of the current best.
—Erika Hall, Just Enough Research
Whether you’re a designer, writer, developer, or marketer, you are a person with a goal: you want to make sure your website is findable. You want the digital experience you’re creating to actually meet people’s search intent—to give them what they’re looking for.
Including an SEO lens in the research process is one of the most important steps you can take toward achieving your goal. As we’ve seen, Google is a database of human psychology. Once you start leveraging this database to account for user search behavior, you won’t be able to stop seeing how search ties into everything. You’ll naturally find ways to use search data to build a clearer picture of your audience and their needs, and notice gaps in the experience where you’re not meeting search intent.
A caveat: we’re going to discuss flexible strategies for adding a search lens to common user research tactics, not how to approach research practices as a whole. If you’re just getting started with research and aren’t yet familiar with the methods we discuss here, we highly recommend checking out Erika Hall’s invaluable Just Enough Research. It’ll give you a solid foundation around how to approach research more holistically.
While you could certainly add a search lens to nearly any form of research you’re already doing, we recommend integrating search considerations into these four key activities:
In this chapter, we’ll walk through strategies for conducting research activities with SEO in mind.
We once worked on a project with a company that had moved its entire blog off their main site and onto a subdomain, not realizing the negative impact such a move would have on search visibility. Microsites can’t leverage the authority and links of the main domain, so they generally don’t perform as well in organic search.
Many internal constraints led up to the microsite decision. The marketing department wanted the blog to use plain language, but the legal department imposed restrictions on terminology used on the main site. The UX department wanted the blog to have a different look and feel, but the IT department lacked the developer resources to build out new page templates and navigation. Faced with these roadblocks, the blog team decided a microsite was their only option.
We talked to team leads and higher-up stakeholders to dig deeper and learn more. It turned out that leadership didn’t realize that the legal team was being so inflexible, or that the IT department was so underresourced. They definitely didn’t know that building content on a microsite would have negative implications for SEO—and once they did know, they were determined to remove the roadblocks that had kept the content off the main site in the first place. That’s real progress that would never have happened if we hadn’t asked about search considerations in our stakeholder interviews.
As in any stakeholder interview process, the first step is to identify whom you need to talk to. Who are your SEO stakeholders? The biggest challenge there is that most people won’t realize that they are, in fact, SEO stakeholders. They might not see the connection between their goals and the role search plays in reaching those goals. Asking the right questions can help them see those connections.
Start by making a list of all of your project stakeholders, noting the roles they play, as well as their actual names and titles. Then categorize your list to help identify gaps and overlaps. We use the seven stakeholder categories created by content strategist Meghan Casey (http://bkaprt.com/seo38/03-01):
While having stakeholders in these different roles will go a long way toward making your endeavor successful, it’s also important to ensure you have balanced departmental representation involved in the website strategy, from marketing to technology to paid search.
Once your list of stakeholders includes the right mix of people to move your search-friendly agenda forward, it’s time to add an SEO lens to your interview questions. The questions we ask aren’t set in stone, but we typically start with an interview script that will uncover anything that could have a major impact on our work and help us neutralize politics, understand priorities, set expectations, and identify opportunities.
In addition to asking people for their name and title—and any other research questions you might have—here are some basic questions to draw out SEO-relevant insights.
To learn about expectations, ask:
To identify opportunities, ask:
To understand priorities, ask:
To neutralize politics, ask:
Inevitably in your interviews, you’ll come across workflow red flags, conflicting priorities, or unhealthy expectations that could derail your SEO efforts. Make note of those and plan out action items to mitigate them.
As you talk to managers and implementers, you will very likely discover that content, design, and development teams have no idea how intertwined their work really is with SEO outcomes. You might discover that people are wary of SEO in general, and thus actively avoid serious efforts to include it. Even if people are open to it, you might see challenges in workflow, timing, or coordination that you’ll need to get ahead of.
Instead of avoiding troublemakers, invite any potential derailers (especially influential ones) to the initial strategy and project-planning work sessions. (If that’s too much involvement, at least share your project goals and plans with them. Invite them to discuss where their work might intersect, any key areas where they need to provide feedback, or other ways they could participate.) You want them to take as much ownership of the process as they can, so they feel heard and invested in success. If you get stuck, lean on sponsors and champions to help soften attitudes or change perspectives to set the stage for process and workflow changes when necessary.
“Doing SEO” might involve conducting search intent research, making changes to site navigation, writing new content, optimizing meta descriptions and titles, and taking on a whole range of technical tasks that we aren’t even covering in this book. Because it’s usually not feasible to do everything at once, focus your SEO efforts on high-priority areas of your site—but note that this only works if those areas have already been clearly established and prioritized. If stakeholders are unable to agree upon which areas of the site are most important in terms of earning organic visibility, it will be difficult to focus your SEO efforts.
High-priority areas of the site are usually those tied to real business outcomes like generating leads, selling a product or service, or even just drumming up awareness around specific information. Business outcomes are not site-performance metrics, like increasing traffic or reducing the bounce rate. If you hear metrics like that stated as business goals during interviews, dig deeper to get to the desired business outcomes those metrics are meant to drive, and then map them to areas of the website that most clearly support those outcomes.
After your interviews, you might find that different people have different ideas about which areas of the site are most important in terms of SEO. To clarify competing priorities, hold a working session with key stakeholders:
Knowing what people expect SEO to do can help you communicate appropriate expectations early on. Here are some of the most common holdups we see:
To address unrealistic expectations, it’s helpful to demonstrate how your site is doing in organic search right now. This could happen by way of an informal presentation where you teach stakeholders how modern search works and highlight any current organic search performance issues. You can use this opportunity to dispel SEO myths, clarify misconceptions, and set more realistic shared expectations. See Chapter 6 for more ideas about sharing search considerations with your team.
We once worked with a hotel client that was redesigning its website. The client wanted to use SEO to encourage people to book rooms directly on its site, rather than on third-party travel aggregators (like Travelocity). They saw organic traffic as a zero-sum game: users either went directly to their site (good for the client) or booked on third-party sites (not good for the client).
But as we conducted user interviews, we learned that both the hotel website and the third-party sites were an important part of the user journey. Most users we talked to started their hotel-booking process on a third-party travel site to get an idea of their options and costs. But once they identified a hotel that seemed like a fit, they almost always visited the hotel site directly to see more details about the rooms or to look at pictures.
This meant we didn’t have to try to outrank the third-party travel sites for the most popular keyword terms (which is nearly impossible to do anyway). Instead, we focused on improving the user booking journey on the site itself by optimizing the local visitors-guide content and map results to show up earlier, improving key messaging on the homepage, and elevating direct booking deals in other areas of the site.
Talking to real users about the reasons behind their searches helped us fill in the gaps from the search intent analysis and make smarter design decisions that aligned with what people actually wanted to know about the hotel. Search intent analysis and user interviews complement each other quite well, because together they provide the whole picture of search behavior. The data tells you what people are searching for; the interviews tell you why they are searching for it—and the combination can be pretty empowering.
You might be thinking, “Hey, now…user interviews are about way more than just understanding how and why people search!” And you’d be absolutely right. Keep on doing what you do in user interviews—just add in a few SEO-related questions to learn about search behavior, too.
Depending on your specific project needs, you might choose to conduct user interviews before researching search intent, or vice versa. If you research search intent first, you can use interviews as a way to close knowledge gaps around the data. But if you’re doing research for an industry that is technical, niche, or just very unfamiliar, doing the interviews with real users first can help break down complex search paths and expose insider lingo and potential root terms and keywords to research.
The nice thing about adding a search lens to user interviews is you don’t have to add any new groups of people or user types to the list—everyone uses search! You’re simply adding questions to your interview guide that will evoke search insights from the people you’ve already planned to talk to.
Of course, different user segments have different goals and priorities, and your organization might have different goals or content segmented by audience type, too. Because of this, it’s possible you’ll need to tailor your search-focused questions to your user types.
The list of questions we ask users varies from project to project. The exact questions you ask will be determined by the context of your product, service, or subject matter. You’ll also want to consider whether the search has a transactional, informational, navigational, or visit-in-person context, like we discussed in Chapter 2.
Here are some sample search-focused questions we typically ask—you can modify them to meet your needs.
To learn about how people discover websites like yours, ask:
To learn what their search journey looks like, ask:
To learn how search plays into the experience of current customers or users, ask:
Even if the questions you’re asking aren’t specifically designed to draw out search insights, you can still get search-relevant insights by listening to users tell you about their context or challenges. You’re listening for how search plays into their experience, as well as the terminology and language they use to describe your topic.
As you go through your interview notes or call recordings, notice things users said that could impact your site’s search strategy, content, or design. In particular, be on the lookout for how users might be expressing search behaviors and search goals.
Search behaviors are actions or decisions that users make when performing a search. These behaviors can help you understand how and where search fits into the user journey, and which pages or areas of the site you should prioritize for search visibility. In user interviews, search behaviors sound something like this:
Search goals are users’ desired outcomes for a search—the tasks they’re trying to accomplish. Search goals help you understand which pages need to exist and what information they need to deliver. In user interviews, search goals sound like this:
Try to capture the remarks as near to verbatim as possible. People’s exact words and phrases are useful for keyword research, copywriting, and stakeholder presentations. Don’t lose that valuable language behind summaries and paraphrases.
Once you’ve identified search behaviors and goals, you need to translate them into actionable next steps for content, design, and code.
Start by grouping search behaviors and goals into thematic categories. You’ll start to notice which search needs are the most common or crucial—or which search needs you had assumed were relevant but were actually mentioned by very few users. Categorizing will also help you prioritize your recommendations and action items.
For example, during user interviews for our hotel client, we heard some folks mention specific search terms like “family-friendly hotels.” Others identified search behaviors like seeking out user-generated photos of hotel rooms to gauge whether the layout would be spacious enough to share with family members. Other users emphasized the importance of swimming pools and play areas when traveling with young children, and how having a mini-fridge and private bathroom vanity area made sharing rooms with family easier. When we looked at those search behaviors and search goals together, the theme became clear: users wanted family-friendly hotel rooms.
Themes made it easier to see what design and content changes needed to happen to better support the search experience—and ultimately create a better user experience, too. To help users find the family-friendly rooms on the hotel site, we made some recommendations to our client:
h1
and h2
tags) where relevant. As this small sample of recommendations demonstrates, SEO can touch anything from the site navigation to the copywriting to the imagery. When making your own search recommendations for a project, consider every element of the digital experience: What user needs did you hear? What questions did people ask? What does your website need to do in order to solve their problems? It will take a combination of content, design, IA, functionality, markup, and marketing to pull everything off.
You probably already know the mix of brands in your market basket, but what about your search competition? When someone searches for your product, service, or information, they will always have choices. You should know what those choices are—whom you’re competing against, directly and indirectly, for search visibility. With a bit of manual sleuthing and a little help from inexpensive tools, you can better understand the competitive landscape in terms of search performance.
Here’s what you’ll scope out in your competitive analysis:
If you’ve gone through the search intent research process outlined in Chapter 2, you’ll already have a list of root terms that represent the core information, products, or services on your site. Search for these terms in private mode (you don’t want to be logged into your browser) to get a cursory list of organizations your site competes with in search.
Make a list of all of the organizations you see in the first two pages of search results. Then look at what they offer, the problem they’re solving, and the experience their site provides to narrow the list down to your three to five closest competitors.
Add these top search competitors to your existing list of competitors, and use a tool like Ahrefs or Semrush to get a sense of comparative rankings (Fig 3.1).
Many of the same keyword research tools we used in Chapter 2, such as Ahrefs and Semrush, can provide valuable competitive insights (Fig 3.2). These tools have access to an extensive set of keyword data, monthly search volume associated with each query, and insight on which sites rank for each query—which means they can provide an estimate of monthly organic search traffic for a site.
These estimates vary in accuracy, especially if your organization is regional or outside the United States. To gauge how accurate these estimates might be, first compare what each tool says about your site to what you actually know—you’ll probably see some inconsistencies. Then, when you look at the data for your competitors’ sites, keep that same margin of error in mind.
As we’ve discussed, a high volume of inbound links from authoritative websites can help improve your site’s rank in search results, so it’s smart to know how you and your competitors measure up. Using your keyword research tool, look at the number of:
Looking into your competitors’ referring domains and backlinks can help you gauge how their authority might factor into SERP rankings. For example, if a competitor with comparatively subpar content is outranking your site and excelling in their referring domains and backlinks, you might deduce that those factors contribute to their competitive edge in search results. How did a weak website earn such quality backlinks? Maybe their site is just older and more established, and therefore more sites link to it. Maybe their brand is so well known that Google believes they should appear in search results, so they can get away with satisfying less search intent than you. Whatever the reason, if your site lacks referring domains and backlinks by comparison, it might be a good time to chat with your PR folks about developing a solid organic link-building strategy.
On the other hand, if you have the link advantage but aren’t getting more visibility than your competitor, this may mean your content doesn’t satisfy search intent as well. Your next step should be to look for potential gaps in your content.
Look at the information architecture and content of your competitors’ websites to understand their topics. Identify the sections, subsections, and individual pages within each part of the site; don’t worry about keywords, but try to notice the coverage and subject matter. Which content topics do they address that you don’t?
Take, for example, baking supply companies King Arthur Flour and Arrowhead Mills. Although they both sell grain-free flours that could be used in gluten-free, low-carb, or paleo diets, only King Arthur uses those terms in their navigation (Fig 3.3). Arrowhead Mills positions the same products as “gluten-free,” without using terms like “low-carb” or “paleo” (Fig 3.4). For reference, “paleo flour” has ten times the search volume of “grain-free flour”—shouldn’t products that fit that description rank for that search term?
King Arthur also includes recipes and blog posts with words like “keto” and “paleo” (Fig 3.5), but no high-level category pages to group those recipes together. Two dedicated URLs with links to all their paleo and keto recipes, respectively, would improve their search visibility for those topics even more.
Tool features like Content Gap (in Ahrefs) and Kombat (in SpyFu) can also help identify content gaps. These can show which keywords you share with your competitors and which you don’t (Fig 3.6), which can highlight potential opportunities for your site. You can also see the keywords ranked in order from highest search volume to lowest. Think of these keywords as potential content topics rather than searches. When a site has visibility for a keyword that you don’t, they likely have content on a topic you don’t cover on your site.
You can also use some tools, like Ahrefs, to understand which pages, sections, or subdomains on your competitors’ sites generate the most organic search traffic. To access this information, run a Top Pages report and study the list of Top Keywords (Fig 3.7). Do you see any topics that you haven’t already identified? This report is handy because it doesn’t just identify content gaps for you; it identifies the most popular content on your competitors’ sites, the content their users find the most helpful and actively search for. Taking this extra step is a good way to help prioritize which content gaps you should tackle first.
Analyzing the competitive landscape is an important part of content research and a prudent precursor to content auditing. Sure, you can audit the content you’ve got now against SEO best practices and your own set of criteria, but if you’re trying to rank in search results, you also have to audit your content against the sites you’ll compete with for search visibility. You’ll identify content gaps, learn more about why domains are ranking (and possibly outranking you), and hopefully find opportunities to meet search intent in more useful and relevant ways.
If you’re working with content you already have (as opposed to planning for all new content), content audits are essential, especially for your search strategy. Although it’s common to see separate audits for UX, content, and SEO, that can lead to conflicting recommendations or duplicated work. You can avoid that by considering these three intertwined disciplines all in one audit.
Ideally, someone with a search background can pair up with a content strategist or UX designer to cocreate the audit criteria and tackle the work together. That way, the SEO specialist can ensure the audit covers the technical aspects of SEO. (If you want to learn more about what technical aspects to consider, check out Chapter 5.)
But even if you’re not a seasoned search professional, or this kind of collaboration isn’t possible on your team, you should still audit content to ensure it meets basic SEO best practices. It’s not just a matter of checking for on-page optimization and keyword targeting—it’s about using search metrics to home in on valuable content that should be included in your site audit, making sure that content satisfies search intent, and uncovering where search optimization can make the biggest impact.
At its most basic, a content inventory looks like a spreadsheet listing all of the URLs on your site, often including individual files and PDFs. This is the most thorough way to capture and assess all available content.
In addition to a column for URLs—and any other columns you use in your content or UX audits—you’ll want to add columns for each metric you’ll use to analyze search performance. Start with some metrics that are available in your site’s analytics platform:
We also recommend including some metrics and data that can be found in other tools:
title
tag, a short description that tells search engines what the page is about; the title appears at the top of the browser window and in SERPs. Titles should be no longer than sixty characters; longer titles are truncated and could look weird or, worse, misleading in SERP results. The website crawling tool Screaming Frog SEO Spider can show you all the page titles for your site.Now, you could export all of this data from each platform separately, but if you want to save time, Screaming Frog SEO Spider can actually collect all of it for you. Their platform has API access to nearly every other SEO tool and analytics platform out there, including Google Analytics, Google Search Console, Page Speed Insights, Ahrefs, and Moz. So instead of running five separate reports manually, you can collect several different platforms’ worth of data from a single source.
In addition to getting a complete list of everything living on your site, the main reason to create an inventory is to determine what content you’ll actually audit. Unless you have a website that has fewer than a hundred pages, there’s a good chance you’ll only audit a portion of your site’s content, such as a particular section, a set of key pages, a specific content type, or a sampling of subsections.
If you want to audit a sample of the overall site experience, choose pages that represent a range of content types, page templates, and site sections. Be sure to include:
Once you’ve narrowed down your inventory, you should have a more manageable spreadsheet of URLs to audit. What’s more, you can be confident that your audit won’t accidentally overlook pages on your site that are important to users, pages that are playing an important role in your site’s organic search visibility, or underperforming pages that need attention.
Now you’re ready to take a closer look at your content’s search visibility factors—to actually audit. You’re auditing your selected content in order to answer the following questions:
title
tag and meta description, check for applicable Schema.org markup, and consider content formatting and presentation. If all of this sounds like an exact science, please remember it isn’t. While there are a few “yes” or “no” boxes to tick, most of this is gut-based and tied to your own assessment skill level. We’ll walk you through best practices for optimizing many of these elements in Chapter 4.
When it comes to the nuts and bolts of the audit, adding a single column to your spreadsheet for every single SEO element you’re looking at would be overwhelming and unwieldy. Instead, you can make aggregate evaluation easier by adding columns for each of the three categories we just mentioned: on-page optimization, search intent, and SERP display.
Stick to whatever grading scale you’re using for the rest of your audit work, or feel free to use ours: we use a scale of 1–3, where 1 is the worst score (none of the category criteria are met for a specific element) and 3 is the best score (all of the criteria are met). We sometimes use half-points if we need more nuanced scores. Use a notes column to cite exactly what’s weak or missing. You can also download an audit template with the criteria we use (http://bkaprt.com/seo38/03-02) (Fig 3.8).
Whatever grading system you use, remember that it takes time and experience to understand which items in your audit will make the most impact for organic search, and how to weigh that impact against the effort it will take to implement those improvements. Your goal is to include an assessment of your content’s SEO strengths and weaknesses in search alongside your general content audit in order to avoid duplicative or even conflicting recommendations. This will help you identify opportunities for improvement in meeting users’ search intent holistically through content, features, functionality, and other aspects of user experience.
Your newly acquired SEO data can help validate (or push back on) content, design, and business decisions for your project. Without that data, you could end up removing pages that are important to users—satisfying multiple forms of search intent, answering critical questions, and providing helpful context about complex services. If such pages are deemed irrelevant to the project’s strategy, instead of simply deleting them, you can reframe them to support your strategy while retaining their search value.
Adding a search lens to your content audits will not only make you incredibly familiar with the ins and outs of your site’s search elements, but will also give you a sense of what’s working well, what needs improvement, and which patterns point to sitewide issues rather than one-page problems.
As you know by now, SEO is about so much more than keywords and headlines. It’s about using all of the tools we have to communicate meaning and context to search engines while making information easier for humans to find. In addition to serving as a holistic starting point for improving content, SEO, and UX, your audit findings will help you set performance and success metrics for your search strategy moving forward.
Adding a search lens to stakeholder interviews, user interviews, content audits, and competitive analysis is useful for making design decisions that lead to successful search outcomes. But it’s not the only way you can go about incorporating SEO into your research practice. Once you get comfortable with how SEO works and grok the basic metrics used to measure search behavior, you’ll find new ways to add SEO considerations to whatever forms of user research your team already uses. And most important, you’ll start to see that optimizing for search is truly tied to creating a better user experience overall.
In the next chapter, we’ll discuss practical ways to use what you’ve learned in research to make search-friendly design decisions that make humans happy, too.