Chapter 8: Applying AI for Innovation – Consumer Goods Deep Dive
The consumer goods industry is an important part of the global economy. It's worth more than $635 billion in the US alone – a figure that is only going to increase as consumers become wealthier and have greater access to products and information.
Since the global financial crisis of 2008 and the more recent COVID-19 pandemic, the consumer goods sector has faced a number of headwinds. The big three – shopping, travel, and leisure – are all under pressure. Fewer people are going out to eat or visiting malls, and more are choosing to stay in and watch TV instead.
At the same time, e-commerce is booming and even quickly overtook in-store sales due to the pandemic. And with consumers becoming increasingly comfortable with digital platforms for shopping, the growth of e-commerce is set to continue unabated.
As such, consumer goods companies that fail to adapt risk losing customers altogether, as shoppers seek out new ways to buy their favorite products quickly and easily online.
In this chapter, we'll explore how consumer goods firms can use data and artificial intelligence (AI) to innovate and adapt to these changing trends, enabling them to stay ahead of the competition.
In this chapter, we'll cover the following topics:
- Understanding the challenges facing consumer goods brands
- Analyzing product data for consumer goods brands
- Using Commerce.AI for consumer goods brands
Technical requirements
You can download the latest code samples for this chapter from the book's GitHub repository:
https://github.com/PacktPublishing/AI-Powered-Commerce/tree/main/Chapter08
Understanding the challenges facing consumer goods brands
The consumer goods industry has been under significant pressure in recent years from rising input costs, changing consumer habits, and new technology, as well as other factors that are outside of its control (for example, supply chain uncertainty).
Companies within the industry must operate in an increasingly complex regulatory landscape while also dealing with increasing competition from online players, including e-commerce platforms such as Amazon and Flipkart. In today's rapidly evolving world of retailing, no retailer can afford to be complacent or risk falling behind in their market.
As a result, many smaller brands have filed for bankruptcy protection or have seen their stock prices hit rock bottom due to the challenges they face across multiple fronts, including the following:
- Competitive consumer goods
- Consumer goods market intelligence
- Inventory management
- Creating the right product mix
- Creating consumer goods content at scale
- Consumer goods review analysis
Let's dive into each of these challenges in detail, which will help inform the need to use data and AI to stay solvent and ultimately become successful.
Competitive consumer goods
Companies in this industry are competing against each other for market share within a very crowded space, with hundreds of new products launched every year. These companies are also facing an increasingly competitive landscape, with more players entering the consumer goods space and consumers becoming more demanding and discerning about what they buy.
One of the major challenges faced by companies in this industry is that they have to constantly innovate to stand out from their competitors and deliver better experiences for their customers. Innovation is critical for these companies to stay relevant in an ever-changing environment and ensure that they continue to be able to attract and retain customers.
Innovation is a broad term that can mean different things to different people. For some, it might simply mean creating a new product or service, while others may think of innovation as process improvement or organizational transformation. Regardless of the definitions used, all successful consumer goods companies are constantly looking for ways to improve their products to stay ahead of the competition and satisfy customer needs.
Consumer goods market intelligence
Market intelligence is a critical component of successful product innovation in the consumer goods industry. Understanding how to create and leverage insights is essential for creating products that meet consumer needs, and it's also an important tool for guiding long-term strategic decisions about the future of your business.
But what exactly is market intelligence, and how can you use it to help your organization create customer-centric products with the potential to drive growth? Market intelligence is a broad term that describes a variety of different sources of information that can be used to understand your customers.
It includes data from traditional market research methods, such as focus groups and surveys, as well as more creative approaches such as social media monitoring. One important aspect of any market intelligence program is ensuring that it's regularly refreshed – otherwise, you risk falling behind the pace of industry changes and developments.
Understanding your current and potential customers is critical for creating customer-centric products that meet their needs. And while consumer goods companies have long relied on data to inform their product development decisions, many still struggle with the challenge of conducting effective market intelligence programs.
Inventory management
Managing stock is one of the most pressing issues facing consumer goods companies today. With e-commerce, consumers can order products whenever and wherever they want. This has led to a boom in online retail sales, but it also poses a challenge for traditional brick-and-mortar stores that have to compete with on-demand shopping options.
Inventory management has become a key differentiating factor for retail stores. A company's ability to manage its inventory effectively can determine whether it can succeed on its own, become an acquisition target, or fail. For this reason, many companies today are dedicating significant resources to making sure they have the best possible inventory management systems in place.
The consumer goods industry is no different, and there are several factors at play that make managing inventories even more challenging. These include the following:
- Product obsolescence
- Rapidly changing consumer preferences
- Short product life cycles
Let's briefly explore how each of these creates challenges for managing inventory:
- Product obsolescence: This one is obvious, but it's worth noting nonetheless. The products you buy now may be obsolete next year – or even sooner if innovations come along and render your product obsolete in just one season. If products become obsolete sooner than expected, say, because of a shift in consumer interest, then you'll be left with excess product.
- Rapidly changing consumer preferences: People have short attention spans nowadays and prefer instant gratification over delayed gratification. As such, what customers want today may not be what they want tomorrow – especially as innovations emerge that appeal to them on an emotional level.
- Short product life cycles: In many cases these days, consumers see a product for only a few months before they move on to the next thing – which means retailers need to constantly refresh their offerings or risk losing sales. This makes inventory management all the more crucial.
Creating the right product mix
The product mix is a hot topic in the consumer goods industry. A few years ago, analysts and investors were obsessed with finding ways to increase product mix by acquiring complementary brands or entering new markets. Now, as we enter an era of changing consumption habits of emerging consumers, the focus on product mix has shifted from simply acquiring complementary brands to creating new ones.
As companies look for new ways to create value in their businesses, one way they're considering is by looking at their portfolio of brands through a lens of product innovation. The idea here is that if they can create more unique products that are differentiated from competitors' offerings – whether it be through attributes such as design or function – then they can grow their businesses by attracting more customers.
This approach also offers some practical benefits; by creating more differentiated offerings across multiple brands, companies can leverage scale and distribution capabilities across all their brands to generate higher returns on investment for shareholders by driving sales growth or increasing margins per sale. This makes sense since you're not only growing your business but creating financial value as well.
It stands to reason that if a company creates more unique products that are differentiated from its competitors' offerings, then it will attract more customers. This is why we believe this shift in thinking around product innovation may lead to an increase in investing in consumer goods companies as well, over time.
Creating consumer goods content at scale
Product pages are an important place for any brand to reach customers and tell its story. This is where consumers can get a better sense of what the brand stands for and how they should use the product.
To drive conversions and increase sales, it's necessary for brands to invest more time and energy into creating engaging product pages than they did just a few years ago. However, creating high-quality content can be difficult due to time constraints and limited resources.
In the next section, Analyzing product data for consumer goods brands, we'll discuss how companies can leverage data science tools such as machine learning in order to create effective product pages that will help them win over customers online – ultimately increasing sales volumes across the board.
Consumer goods review analysis
Product reviews are one of the most popular and extensive user-generated data sources for many companies. The majority of consumers' purchase decisions are influenced by product reviews.
Product reviews are also a major source of competitive intelligence for consumer goods manufacturers – they can learn what products consumers like best, as well as what features and functions consumers value the most.
Companies use this information to inform their product development and market strategy. For example, if a company is launching a new deodorant that uses plant-based active ingredients, it's likely that they'd look into existing customer reviews about other brands' deodorants for insights on how consumers perceive this new technology.
In order to gain actionable insights from these reviews, companies have traditionally relied on manual methods or paid third parties to manually help with the review process. However, there are now several tools on the market that aim to automate some or all aspects of analyzing product review data online.
Traditionally, automated sentiment analysis has focused on identifying negative language (euphemisms used instead of the word boring, for example) in product reviews. This approach generates an opinion score that represents how positive or negative a reviewer feels about a given product based on its content.
The problem with this approach is that it misses many subtle nuances in reviewer language, which can lead to skewed results when applied consistently across different industries or review platforms.
Furthermore, it's not always clear whether certain terms used by reviewers are intended as compliments or criticisms (for example, that's just great). A more nuanced approach is needed! Reducing bias in automated sentiment analysis requires training datasets with diverse examples drawn from multiple industries and review platforms – but where can we find such a diverse set? Machine learning is the answer, and in the following section, we'll explore how to use it.
Now that we understand some of the major challenges facing consumer goods brands, let's explore how to analyze product data to overcome these challenges and get ahead.
Analyzing product data for consumer goods brands
The elephant in the room when it comes to innovation in consumer goods today is data – there are vast amounts of customer data available today. So why aren't all consumer goods companies using this data to develop better products? It turns out that many don't know where to start with product innovation, given the complexity involved and the lack of available resources within their organizations.
The following are a few of the main ways in which consumer goods brands can use product data:
- Consumer goods content generation
- Analyzing consumer goods reviews
- Lead time analysis
- Demand forecasting
- Maintaining adequate cash flow
- Analyzing the impact of discounts
- Identifying seasonal trends
- Social media analytics
We will explore these methods in-depth in the following sections.
Consumer goods content generation
Creating clear, concise product copy is key to successful products. After all, consumers have millions of products to choose from, but only a limited amount of time and attention. This massive volume of products is a double-edged sword – it means that product teams have to spend time writing copy across multiple marketplaces, and constantly making updates based on new features and product iterations.
All this takes time away from product innovation, but the good news is that it can be automated with AI. In particular, we can create a product copy generator powered by natural language processing (NLP) algorithms using GPT-J.
GPT-J is a large language model, or a machine learning model that was trained on large amounts of text, released by a group called Eleuther AI. We'll demonstrate it as follows:
- First, install GPT-J and import the required library:
!pip install gptj
from GPTJ.Basic_api import SimpleCompletion
Since these large language models use pre-training, the model was already trained on a lot of text data, and we only need a small amount of data to tune the model for a specific task.
- Next, we define this task by providing prompt, which includes examples of product descriptions being generated from a product name and features:
prompt = "Write one sentence descriptions for products based on a list of features. ## Product: Sundef Features: - Sunscreen for athletes - Unique formula to prevent burning eyes - Can be worn on the body and on the face One sentence description: Sundef face & body sunscreen for athletes keeps your skin protected without hurting your eyes, so you can keep your head in the game. ## Product: " + product + " Features: " + features + " One sentence description:"
This prompt is crucial for the language model because language models have a broad range of use cases, including classification, generation, translation, transformation, and more. So, they have to be guided to complete specific tasks. Using the prompt variable, we can guide the model to act as a product copy generator.
- Further, we'll want to pass a number of parameters, primarily temperature (or randomness), max_length (or the maximum output size of the model), and product (or what the user types in, such as SlimWallet):
temperature = 0.4
top_probability = 1.0
max_length = 5
product = "SlimWallet"
- Finally, we can now pass the prompt variable and the parameters to the model to create a recommendation. We'll also grab just the first line of text generated, in case the model goes overboard:
query = SimpleCompletion(prompt, length=max_length, t=temperature, top=top_probability)
Query = query.simple_completion()
lines = Query.splitlines()
results = []
In doing so, giving the model an input about a wallet with three features generates product copy such as SlimWallet is the most stylish, thin, and durable wallet you've ever seen.
There are other ways to try out this same concept even without using any code at all, such as with AI21 Studio (https://studio.ai21.com). In Figure 8.1, we use the same prompt in a visual canvas instead of code, and given the SlimWallet item, AI21 Studio recommends SlimWallet holds 15 cards in a thin shape that's 5 times thinner than a traditional leather wallet.
Figure 8.1 – The AI21 Studio canvas for product description generation
As with GPT-J, we'll need to provide a number of settings, which is done in AI21 Studio through a Configuration panel shown here:
Figure 8.2 – The AI21 Studio Configuration panel
The settings (as seen in Figure 8.2) are almost identical and include maximum completion length, temperature, and stop sequences.
Analyze consumer goods reviews
There are literally hundreds of millions of Amazon product reviews (https://nijianmo.github.io/amazon/index.html), which makes it impossible for any product team to manually read and analyze reviews to extract insights at scale.
Fortunately, we can again use large language models, and automatically extract insights from reviews. Let's look at how to extract user-desired features from a product review. As we've explored in the Consumer goods content generation section, we can provide pre-trained language models with a prompt to guide them toward a specific use case.
In Figure 8.3, we provide AI21 Studio with a prompt that extracts Areas for improvement from a product review:
Figure 8.3 – The AI21 Studio canvas for product review analysis
Now, we can provide the model with any product review, and it will extract an area for improvement, such as Battery life. We can then quickly scan hundreds or thousands of product reviews and tally up the frequency of any given item in order to prioritize it. For example, perhaps 50 reviews request a better battery life, while only 15 reviews request waterproofing, which would inform product teams to prioritize battery life in the next product iteration.
This process can be scaled to any number of products and reviews, enabling instant insights at scale.
Just as we've generated product descriptions programmatically, we can use this new prompt variable (seen in Figure 8.3) to programmatically extract areas for improvement from product reviews. An example of this is given in the GitHub repository for this chapter, which can be found in the Technical requirements section.
Lead time analysis
Understanding how long it takes for a product to be manufactured or produced can be very revealing about its quality and manufacturing processes. An overly long lead time can indicate that there are issues with planning, production capacity, or the shortage of materials, whereas shorter lead times may point to a more efficient supply chain and better planning on the part of the manufacturer.
As an example, if you see that something such as batteries take 3 days to make whereas your competitors need 2 weeks, you have a competitive advantage over them because they are not utilizing their full capacity.
Demand forecasting
It is important for companies who produce consumer goods such as foodstuffs and apparel items to maintain adequate stockpiles of their products at all times so that they will always have something ready for sale when consumers come looking for them.
However, this requires having large inventories relative to demand at any given time, which means being able to accurately predict demand is critical in order to keep costs under control while still maintaining adequate levels of inventory to hand.
AI is a powerful tool for forecasting as it can be trained on vast amounts of historical data and find patterns that influence demand automatically.
Maintaining adequate cash flow
Maintaining sufficient levels of available cash flow during periods where sales volumes are low helps ensure there is enough liquidity available within a business so that it doesn't need external financing. Having too much debt within a business could create problems down the road if revenues suddenly fall below what was originally budgeted due to changes in spending patterns among customers (for example, people buying fewer pairs of shoes than expected).
On the other hand, having too little cash flow during slower periods may mean delaying investments or purchases necessary for future growth until economic conditions improve; this could mean missing out on market opportunities if those investments had started earlier.
Just as AI can be used for accurate demand forecasting, given historical cash flow data, consumer goods brands can forecast cash flow and make changes accordingly. Better inventory management plays a role as well, as overages and stockouts negatively impact cash flow.
Analyzing the impact of discounts
In many consumer goods industries, consumers can often get deep discounts on their purchases throughout the year thanks to promotions that retailers offer. Discounts can also be offered to entice new customers who may not have been aware of such deals before becoming regular buyers at that retailer.
Discounts directly impact both gross margins and net margins for companies that sell similar products since they effectively lower prices. So, while discounts do help boost sales in the short term by increasing volume, they should not be relied on by companies for long-term planning since they tend to eventually wear off.
With AI, consumer goods brands can predict the impact of discounts on any given product or product line, both now and in the future.
Identifying seasonal trends
Seasonal fluctuations are very common within most industries, as well as specific sectors within those industries. In some cases, these fluctuations result from cultural preferences regarding holidays or other events, such as sports tournaments that may draw large crowds for a period of time.
Comparing current sales figures with past years' figures can give a company insight into which periods were best. This information can then be used to help determine whether current expectations for upcoming months are likely to meet or beat those previous expectations.
Social media analytics
Social media platforms offer businesses many ways to interact with their target audiences – including the ability for companies to distribute product information via posts made by employees or contractors who work remotely but still interact with followers online (for example, via Twitter posts about new products being added to product catalogs).
This kind of interaction between employees and followers offers another opportunity for brands looking to engage with their target audiences, even when they aren't physically present in person – and social media platforms make this easy for companies who utilize them effectively.
Further, consumers are constantly talking about products and brands on social media. With AI, brands can more effectively conduct social listening, and they can analyze consumer sentiment and wish lists from social media, in real time and at scale.
Now that we understand how to analyze product data across a range of settings, let's look more closely at using Commerce.AI for consumer goods brands.
Using Commerce.AI for consumer goods brands
As we now know, consumer goods brands are facing greater challenges than ever, and the solutions to these challenges can be found in the data. However, data alone is not enough, and consumer goods brands need to find a way to turn that data into actionable insights.
Let's explore five main ways in which Commerce.AI can be used for consumer goods brands by gaining value from that data:
- Measuring product attributes and trends
- Predicting revenue opportunities
- Analyzing user personas and customer segments
- Analyzing the customer journey
- Generating consumer goods product ideas
Let's explore each of these in the following sections.
Measuring product attributes and trends
Product innovation is the best way to grow a consumer goods business. However, many managers struggle with understanding what makes their products different and better, which can result in missed opportunities.
The reasons for this situation are complex, but they include barriers such as complexity of measurement, limited datasets being available for analysis, a lack of domain expertise required to understand patterns, and a lack of reliable metrics that can be applied across multiple product lines. These challenges have made it difficult for companies to effectively develop new products and services.
Let's focus on one particular dimension of Commerce.AI – sentiment analysis associated with product attributes – through the lens of innovation in consumer goods. One intriguing finding from our research is that attributes such as taste or brand expectations play an important role in shaping purchase behavior behind consumer goods brands.
For example, one reason why upscale consumers buy Prada is that they expect a higher price tag alongside an interesting design aesthetic rather than the standard knockoff experience you'd expect from buying a bag off the discount rack at a department store.
Similarly, one reason Starbucks has been so successful is not just its focus on premium coffee but also the expectation among its customers that it will create interesting new flavors through its roasting technology, rather than it rehashing old favorites time after time. The same logic applies when looking at technology firms versus commodity vendors – there's something interesting happening here that warrants further investigation.
When you strip out all the noise, there are two types of meaningful signals emerging in product data:
- Signals about underlying quality and perceived value creation being performed by CPGs versus their competitors
- Signals about how likely customers are to pay certain prices
In other words, purchase intent ripples through both product attributes and attributes that lead to higher price points, such as service levels or technology features. In short, AI offers real promise in this context by augmenting traditional market surveillance capabilities.
Figure 8.4 visualizes a mockup of Commerce.AI for measuring product attributes and trends, which it fills with the relevant brand and product data:
Figure 8.4 – A Commerce.AI mockup for measuring product attributes and trends
While it's great to understand product attributes and trends, revenue is an undoubtedly crucial measure to consider as well. Let's take a closer look at measuring revenue opportunity.
Predicting revenue opportunity
The art and science of product innovation have always been about finding new ways to satisfy customer needs and desires – and generating revenue along the way. Disruptive innovations often succeed by taking market share away from depleted or disrupted incumbents. Such disruption requires an ability to identify opportunities by their revenue opportunity.
This can help uncover new growth drivers, discover business models that aren't obvious in retrospect, help you to prioritize investments in uncertain areas, and provide early warning when competitors enter an untested space (which could put you at risk of industry disruption if you don't react quickly enough with superior offerings).
With AI as the enabling technology, it is possible now for organizations of all sizes to gain this type of competitive advantage over their peers – all with relative ease; all without having to hire expensive consultants or spend a fortune on R&D labs; all without needing prior experience mining through masses of data manually; all while gaining instant visibility into the path ahead, as well as useful insights into where others are going wrong (or right).
Figure 8.5 visualizes a mockup of Commerce.AI for measuring revenue opportunity, which includes a confidence score to indicate the accuracy of the forecast:
Figure 8.5 – A Commerce.AI mockup for measuring revenue opportunity
Once the revenue opportunity is understood, it's time to better understand user personas and customer segments within that market.
Analyzing user personas and customer segments
Segmentation is an essential aspect of the product-market fit. And it's something that simply can't be ignored by entrepreneurs and product managers.
That said, knowing when to segment users (or potential users) and build features around them can be tricky. The key here is finding the right balance, which requires understanding your user personas as well as your customer segments.
So, what should you know before segmenting your customers? What are the best approaches for building features around user personas, and what pitfalls can you avoid? Read on for tips on how to make this seemingly difficult process easier.
At their core, user personas are a simple exercise in empathy. You simply try to understand who exactly uses your product and what they're trying to accomplish with it. By doing this, you'll be able to build features – or at least concepts for building features – that resonate with the users you already have, rather than seeking out new users that might not even exist yet.
Figure 8.6 visualizes a mockup of Commerce.AI for measuring personas and segments, which it fills with the relevant consumer data:
Figure 8.6 – A Commerce.AI mockup for analyzing personas and segments
Understanding user personas and segments is a critical part of gaining a better understanding of the market, but we also need to take it to the next level by analyzing the customer journey.
Analyzing the customer journey
The customer journey is the path that a customer takes from first learning about your product to becoming an active user of it. Understanding this journey and how to influence it is critical for driving innovation, increasing engagement, and reducing churn.
Traditional analytics methods focus on measuring changes in key metrics such as revenue, user engagement, or cost of customer acquisition (CAC). While these are useful metrics to track over time, they often fail to tell the whole story. In particular, CAC fails to account for the value of having engaged users (that is, those who remain active every day) further down the line.
That's why we need new ways of looking at customer behavior. Using AI allows us to identify patterns and make inferences that would be too complex or costly for any human analyst to produce. We can then use this insight to inform our strategy and improve our product design and development process.
Figure 8.7 visualizes a mockup of Commerce.AI for measuring the customer journey:
Figure 8.7 – A Commerce.AI mockup for analyzing customer journeys
Once we've gained this deep understanding of the market and its potential customers, it's time to come up with product ideas.
Generating consumer goods product ideas
In Chapter 6, Applying AI for Innovation – Consumer Electronics Deep-Dive, we used large language models to generate consumer electronics product ideas with Commerce.AI. Here, we can use the same techniques to generate consumer goods product ideas, which can help product teams speed up the brainstorming process.
Using this approach, we've been able to generate thousands of unique product ideas with little effort. For example, in Figure 8.8, we can see three product ideas generated around healthy snacks:
Figure 8.8 – AI-generated product ideas around healthy snacks
As we've seen, Commere.AI can be used to measure product attributes and trends, predict revenue opportunities, analyze customer data, and generate product ideas. This is a powerful tool to speed up the product development life cycle, which is key to getting ahead of your competition and being first to market.
Summary
In this chapter, we've learned about the key challenges facing consumer goods brands, and how to overcome them by analyzing product data and using the Commerce.AI data engine.
Data can be used to overcome significant challenges in the consumer goods space, which is why many leading brands and product teams are investing in AI to gain an advantage.
In particular, we've learned how to use AI for tasks such as consumer goods content generation, analyzing consumer goods reviews, and demand forecasting. We've also looked at how consumer goods product teams can use Commerce.AI to measure product attributes and trends, predict revenue opportunities, analyze user personas, and more. In doing so, consumer goods brands can better innovate and launch successful products.
In the next chapter, we'll explore in detail how Commerce.AI's Product AI can be used for product concept and development, product launches, and product management. These insights will be useful for any product firm.