11. Data Mining with Advanced Filter

Read this chapter.

Although very few people use Advanced Filter in Excel, it is a workhorse in Excel VBA.

I will estimate that I end up using one of these filtering techniques as the core of a macro in 80 percent of the macros I develop for clients. Given that Advanced Filter is used in less than 1 percent of Excel sessions, this is a dramatic statistic.

So even if you hardly ever use Advanced Filter in regular Excel, you should study this chapter for powerful VBA techniques.

Replacing a Loop with AutoFilter

In Chapter 4, “Looping and Flow Control,” you read about several ways to loop through a dataset to format records that match certain criteria. By using the Filter, you can achieve the same result much faster. Although Microsoft changed the name of AutoFilter to Filter in Excel 2007, the VBA code still refers to AutoFilter.

When the AutoFilter was added to Excel, the team at Microsoft added extra care and attention to the AutoFilter. Items hidden because of an AutoFilter are not simply treated like other hidden rows. AutoFilter gets special treatment. You’ve likely run into the frustrating situation in the past where you have applied formatting to visible rows and the formatting gets applied to the hidden rows. This is certainly a problem when you’ve hidden rows by clicking the #2 Group and Outline button after using the Subtotal command. This is always a problem when you manually hide rows. But it is never a problem when the rows are hidden because of the AutoFilter.

Once you’ve applied an AutoFilter to hide rows, any action performed on the CurrentRegion is only applied to the visible rows. You can apply bold. You can change the font to red. You can even CurrentRegion.EntireRow.Delete to delete the visible rows and the rows hidden by the filter are not impacted.

Let’s say that you have a dataset as shown in Figure 11.1, and you want to perform some action on all the records that match a certain criteria, such as all Ford records.

Figure 11.1. Find all Ford records and mark them.

In Chapter 5, “R1C1-Style Formulas,” you learned to write code like this, which we could use to color all the Ford records green:

Sub OldLoop()
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    For i = 2 To FinalRow
        If Cells(i, 4) = "Ford" Then
            Cells(i, 1).Resize(1, 8).Interior.Color = RGB(0,255,0)
        End If
    Next i
End Sub

If you needed to delete records, you had to be careful to run the loop from the bottom of the dataset to the top using code like this:

Sub OldLoopToDelete()
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    For i = FinalRow To 2 Step -1
        If Cells(i, 4) = "Ford" Then
            Rows(i).Delete
        End If
    Next i
End Sub

The AutoFilter method, however, enables you to isolate all the Ford records in a single line of code:

Range("A1").AutoFilter Field:=4, Criteria1:= "Ford"

After isolating the matching records, you do not need to use the VisibleCellsOnly setting to format the matching records. Instead, the following line of code will format all the matching records to be green:

Range("A1").CurrentRegion.Interior.Color = RGB(0,255,00)

There are two problems with the current two-line macro. First, the program leaves the AutoFilter drop-downs in the dataset. Second, the heading row is also formatted in green.

If you want to turn off the AutoFilter drop-downs and clear the filter, this single line of code will work:

Range("A1").AutoFilter

If you want to leave the AutoFilter drop-downs on but clear the Column D drop-down from showing Ford, you can use this line of code:

ActiveSheet.ShowAllData

The second problem is a bit more difficult. After you apply the filter and select Range("A1").CurrentRegion, the selection includes the headers automatically in the selection. Any formatting is also applied to the header row.

If you did not care about the first blank row below the data, you could simply add an OFFSET(1) to move the current region down to start in A2. This would be fine if your goal were to delete all the Ford records:

Sub DeleteFord()
    ' skips header, but also deletes blank row below
    Range("A1").AutoFilter Field:=4, Criteria1:="Ford"
    Range("A1").CurrentRegion.Offset(1).EntireRow.Delete
    Range("A1").AutoFilter

End Sub

The preceding code works because you do not mind if the first blank row below the data is deleted. However, if you are applying a green format to those rows, the code will apply the green format to the blank row below the dataset, which would not look right.

If you will be doing some formatting, you can determine the height of the dataset and use .Resize to reduce the height of the current region while you use OFFSET:

Sub ColorFord()
    DataHt = Range("A1").CurrentRegion.Rows.Count
    Range("A1").AutoFilter Field:=4, Criteria1:="Ford"

    With Range("A1").CurrentRegion.Offset(1).Resize(DataHt - 1)
    ' No need to use VisibleCellsOnly for formatting
        .Interior.Color = RGB(0,255,0)
        .Font.Bold = True
    End With
    ' Clear the AutoFilter & remove drop-downs
    Range("A1").AutoFilter

End Sub

Using New AutoFilter Techniques

Excel 2007 introduced the possibility of selecting multiple items from a filter, filtering by color, filtering by icon, filtering by top 10, and filtering to virtual date filters. Excel 2010 introduces the new search box in the filter drop-down. All these new filters have VBA equivalents, although some of them are implemented in VBA using legacy filtering methods.

Selecting Multiple Items

Legacy versions of Excel allowed you to select two values, joined by AND or OR. In this case, you would specify xlAnd or xlOr as the operator:

Range("A1").AutoFilter Field:=4, _
    Criteria1:="Ford", _
    Operator:=xlOr, _
    Criteria2:="General Motors"

As the AutoFilter command became more flexible, Microsoft continued to use the same three parameters, even if they didn’t quite make sense. For example, Excel lets you filter a field by asking for the top five items or the bottom 8 percent of records. To use this type of filter, specify either "5" or "8" as the Criteria1 argument, and then specify xlTop10Items, xlTop10Percent, xlBottom10Items, or xlBottom10Percent as the operator. The following code produces the top 12 revenue records:

Sub Top10Filter()
    ' Top 12 Revenue Records
    Range("A1").AutoFilter Field:=6, _
        Criteria1:="12", _
        Operator:=xlTop10Items
End Sub

There are a lot of numbers (6, 12, 10) in the code for this AutoFilter. Field 6 indicates that you are looking at the sixth column. xlTop10Items is the name of the filter, but the filter is not limited to 10 items. The criteria of 12 indicates the number of items that you want the filter to return.

Excel offers several new filter options. It continues to force these filter options to fit in the old object model where the filter command must fit in an operator and up to two criteria fields.

If you want to choose three or more items, change the operator to Operator: =xlFilterValues and specify the list of items as an array in the Criteria1 argument:

Range("A1").AutoFilter Field:=4, _
    Criteria1:=Array("General Motors", "Ford", "Fiat"), _
    Operator:=xlFilterValues

Selecting Using the Search Box

Excel 2010 introduced the new Search box in the AutoFilter drop-down. After typing something in the Search box, you can use the Select All Search Results item.

The macro recorder does a poor job of recording the Search box. The macro recorder hard-codes a list of customers who matched the search at the time you ran the macro.

Think about the Search box. It is really a shortcut way of selecting Text Filters, Contains. In addition, the Contains filter is actually a shortcut way of specifying the search string surrounded by asterisks. Therefore, to filter to all the records that contain “AT,” use this:

Range("A1").AutoFilter, Field:=4, Criteria1:="*at*"

Filtering by Color

To find records that have a particular font color, use an operator of xlFilterFontColor and specify a particular RGB value as the criteria. This code finds all cells with a red font in Column F:

Sub FilterByFontColor()
    Range("A1").AutoFilter Field:=6, _
        Criteria1:=RGB(255, 0, 0), Operator:=xlFilterFontColor
End Sub

To find records that have no particular font color, use an operator of xlFilterAutomaticFillColor and do not specify any criteria.

Sub FilterNoFontColor()
    Range("A1").AutoFilter Field:=6, _
        Operator:=xlFilterAutomaticFontColor
End Sub

To find records that have a particular fill color, use an operator of xlFilterCellColor and specify a particular RGB value as the criteria. This code finds all red cells in Column F:

Sub FilterByFillColor()
    Range("A1").AutoFilter Field:=6, _
        Criteria1:=RGB(255, 0, 0), Operator:=xlFilterCellColor
End Sub

To find records that have no fill color, use an operator of xlFilterNoFill and do not specify any criteria.

Filtering by Icon

If you are expecting the dataset to have an icon set applied, you can filter to show only records with one particular icon by using the xlFilterIcon operator.

For the criteria, you have to know which icon set has been applied and which icon within the set. The icon sets are identified using the names shown in Column A of Figure 11.2. The items range from 1 through 5. The following code filters the Revenue column to show the rows containing an upward-pointing arrow in the 5 Arrows Gray icon set:

Sub FilterByIcon()
    Range("A1").AutoFilter Field:=6, _
        Criteria1:=ActiveWorkbook.IconSets(xl5ArrowsGray).Item(5), _
        Operator:=xlFilterIcon
End Sub

Figure 11.2. To search for a particular icon, you need to know the icon set from Column A and the item number from Row 1.

To find records that have no conditional formatting icon, use an operator of xlFilterNo Icon and do not specify any criteria.

Selecting a Dynamic Date Range Using AutoFilters

Perhaps the most powerful feature in Excel filters is the dynamic filters. These filters enable you to choose records that are above average or with a date field to select virtual periods, such as Next Week or Last Year.

To use a dynamic filter, specify xlFilterDynamic as the operator and then use one of 34 values as Criteria1. The following code finds all dates that are in the next year:

Sub DynamicAutoFilter()
    Range("A1").AutoFilter Field:=3, _
        Criteria1:=xlFilterNextYear, _
        Operator:=xlFilterDynamic
End Sub

The following lists all the dynamic filter criteria options. Specify these values as Criteria1 in the AutoFilter method:

• Criteria for values—Use xlFilterAboveAverage or xlFilterBelowAverage to find all the rows that are above or below average. Note that in Lake Wobegon, using xlFilterBelowAverage will likely return no records.

• Criteria for future periods—Use xlFilterTomorrow, xlFilterNextWeek, xlFilterNextMonth, xlFilterNextQuarter, or xlFilterNextYear to find rows that fall in a certain future period. Note that “next week” starts on Sunday and ends on Saturday.

• Criteria for current periods—Use xlFilterToday, xlFilterThisWeek, xlFilterThisMonth, xlFilterThisQuarter, or xlFilterThisYear to find rows that fall within the current period. Excel will use the system clock to find the current day.

• Criteria for past periods—Use xlFilterYesterday, xlFilterLastWeek, xlFilterLastMonth, xlFilterLastQuarter, xlFilterLastYear, or xlFilterYearToDate to find rows that fall within a previous period.

• Criteria for specific quarters—Use xlFilterDatesInPeriodQuarter1, xlFilterDatesInPeriodQuarter2, xlFilterDatesInPeriodQuarter3, or xlFilterDatesInPeriodQuarter4 to filter to rows that fall within a specific quarter. Note that these filters do not differentiate based on a year. If you ask for quarter 1, you might get records from this January, last February, and next March.

• Criteria for specific months—Use xlFilterDatesInPeriodJanuary through xlFilterDatesInPeriodDecember to filter to records that fall during a certain month. Like the quarters, the filter does not filter to any particular year.

Unfortunately, you cannot combine criteria. You might think that you can specify xlFilterDatesInPeriodJanuary as Criteria1 and xlFilterDatesNextYear as Criteria2. Even though this is a brilliant thought, Microsoft does not support this syntax (yet).

Selecting Visible Cells Only

After you apply a filter, most commands operate only on the visible rows in the selection. If you need to delete the records, format the records, or apply a conditional format to the records, you can simply refer to the .CurrentRegion of the first heading cell and perform the command.

However, if you have a dataset in which the rows have been hidden using the Hide Rows command, any formatting applied to the .CurrentRegion will apply to the hidden rows, too. In these cases, you should use the Visible Cells Only option of the Go To Special dialog, as shown in Figure 11.3.

Figure 11.3. If rows have been manually hidden, use Visible Cells Only in the Go To Special dialog.

To use Visible Cells Only in code, use the SpecialCells property:

Range("A1").CurrentRegion.SpecialCells(xlCellTypeVisible)

Advanced Filter Is Easier in VBA Than in Excel

Using the arcane Advanced Filter command is so difficult in the Excel user interface that it is pretty rare to find someone who enjoys using it regularly.

However, in VBA, advanced filters are a joy to use. With a single line of code, you can rapidly extract a subset of records from a database or quickly get a unique list of values in any column. This is critical when you want to run reports for a specific region or customer. Two advanced filters are used most often in the same procedure—one to get a unique list of customers and a second to filter to each individual customer, as shown in Figure 11.4. The rest of this chapter builds toward such a routine.

Figure 11.4. A typical macro uses two advanced filters.

Using the Excel Interface to Build an Advanced Filter

Because not many people use the Advanced Filter feature, this section walks you through examples using the user interface to build an advanced filter and then shows you the analogous code. You will be amazed at how complex the user interface seems and yet how easy it is to program a powerful advanced filter to extract records.

One reason Advanced Filter is hard to use is that you can use the filter in several different ways. You must make three basic choices in the Advanced Filter dialog box. Because each choice has two options, there are eight (2 × 2 × 2) possible combinations of these choices. The three basic choices are shown in Figure 11.5 and described here:

• Action—You can select Filter the List, In-Place or you can select Copy to Another Location. If you choose to filter the records in place, the nonmatching rows are hidden. Choosing to copy to a new location copies the records that match the filter to a new range.

• Criteria—You can filter with or without criteria. Filtering with criteria is appropriate for getting a subset of rows. Filtering without criteria is still useful when you want a subset of columns or when you are using the Unique Records Only option.

• Unique—You can choose to request Unique Records Only or request all matching records. The Unique option makes the Advanced Filter command one of the fastest ways to find a unique list of values in one field. By placing the “Customer” heading in the output range, you will get a unique list of values for that one column.

Figure 11.5. The Advanced Filter dialog is complicated to use in the Excel user interface. Fortunately, it is much easier in VBA.

Using Advanced Filter to Extract a Unique List of Values

One of the simplest uses of Advanced Filter is to extract a unique list of a single field from a dataset. In this example, you want to get a unique list of customers from a sales report. You know that Customer is in Column D of the dataset. You have an unknown number of records starting in cell A2, and Row 1 is the header row. There is nothing located to the right of the dataset.

Extracting a Unique List of Values with the User Interface

To extract a unique list of values, follow these steps:

1. With the cursor anywhere in the data range, select Advanced from the Sort & Filter group on the Data tab. The first time you use the Advanced Filter command on a worksheet, Excel automatically populates the List Range text box with the entire range of your dataset. On subsequent uses of the Advanced Filter command, this dialog box remembers the settings from the prior advanced filter.

2. Select the Unique Records Only check box at the bottom of the dialog.

3. In the Action section, select Copy to Another Location.

4. Type J1 in the Copy To text box.

By default, Excel copies all the columns in the dataset. You can filter just the Customer column either by limiting the List Range to include only Column D or by specifying one or more headings in the Copy To range. Either method has its own drawbacks.

Change the List Range to a Single Column

Edit the List Range to point to the Customer column. In this case, it means changing the default $A$1:$H$1127 to $D$1:$D$1127. The Advanced Filter dialog should appear.

The drawback of this method is that Excel remembers the list range on subsequent uses of the Advanced Filter command. If you later want to get a unique list of regions, you will be constantly specifying the list range.

Copy the Customer Heading Before Filtering

With a little forethought before invoking the Advanced Filter command, you can allow Excel to keep the default list range of $A$1:$H$1127. In cell J1, type the Customer heading as shown in Figure 11.6. You leave the List Range field pointing to Columns A through H. Because the Copy To range of J1 already contains a valid heading from the list range, Excel copies data only from the Customer column. This is the preferred method, particularly if you will be doing multiple advanced filters. Because Excel remembers the prior settings from the preceding advanced filter, it is more convenient to always filter the entire columns of the list range and limit the columns by setting up headings in the Copy To range.

Figure 11.6. The advanced filter extracted a unique list of customers from the dataset and copied it to Column J.

After you use either of these methods to perform the advanced filter, a concise list of the unique customers appears in Column J (see Figure 11.6).

Extracting a Unique List of Values with VBA Code

In VBA, you use the AdvancedFilter method to carry out the Advanced Filter command. Again, you have three choices to make:

• Action—Choose to either filter in place with the parameter Action:=xlFilterInPlace or copy with Action:=xlFilterCopy. If you want to copy, you also have to specify the parameter CopyToRange:=Range("J1").

• Criteria—To filter with criteria, include the parameter CriteriaRange:=Range("L1:L2"). To filter without criteria, omit this optional parameter.

• Unique—To return only unique records, specify the parameter Unique:=True.

The following code sets up a single-column output range two columns to the right of the last-used column in the data range:

Sub GetUniqueCustomers()
    Dim IRange As Range
    Dim ORange As Range

    ' Find the size of today's dataset
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    NextCol = Cells(1, Columns.Count).End(xlToLeft).Column + 2

    ' Set up output range. Copy heading from D1 there
    Range("D1").Copy Destination:=Cells(1, NextCol)
    Set ORange = Cells(1, NextCol)

    ' Define the Input Range
    Set IRange = Range("A1").Resize(FinalRow, NextCol - 2)

    ' Do the Advanced Filter to get unique list of customers
    IRange.AdvancedFilter Action:=xlFilterCopy, CopyToRange:=ORange, _
      Unique:=True

End Sub

By default, an advanced filter copies all columns. If you just want one particular column, use that column heading as the heading in the output range.

The first bit of code finds the final row and column in the dataset. Although it is not necessary to do so, you can define an object variable for the output range (ORange) and for the input range (IRange).

This code is generic enough that it will not have to be rewritten if new columns are added to the dataset later. Setting up the object variables for the input and output range is done for readability rather than out of necessity. The previous code could be written just as easily like this shortened version:

Sub UniqueCustomerRedux()
    ' Copy a heading to create an output range
    Range("J1").Value = Range("D1").Value
    ' Do the Advanced Filter
    Range("A1").CurrentRegion.AdvancedFilter xlFilterCopy, _
        CopyToRange:=Range("J1"), Unique:=True
End Sub

When you run either of the previous blocks of code on the sample dataset, you get a unique list of customers off to the right of the data. The key to getting a unique list of customers is copying the header from the Customer field to a blank cell and specifying this cell as the output range.

After you have the unique list of customers, you can sort the list and add a SUMIF formula to get total revenue by customer. The following code gets the unique list of customers, sorts it, and then builds a formula to total revenue by customer. Figure 11.7 shows the results:

Sub RevenueByCustomers()
    Dim IRange As Range
    Dim ORange As Range

    ' Find the size of today's in
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    NextCol = Cells(1, Columns.Count).End(xlToLeft).Column + 2

    ' Set up output range. Copy heading from D1 there
    Range("D1").Copy Destination:=Cells(1, NextCol)
    Set ORange = Cells(1, NextCol)

    ' Define the Input Range
    Set IRange = Range("A1").Resize(FinalRow, NextCol - 2)

    ' Do the Advanced Filter to get unique list of customers
    IRange.AdvancedFilter Action:=xlFilterCopy, _
        CopyToRange:=ORange, Unique:=True

    ' Determine how many unique customers we have
    LastRow = Cells(Rows.Count, NextCol).End(xlUp).Row

    ' Sort the data
    Cells(1, NextCol).Resize(LastRow, 1).Sort Key1:=Cells(1, NextCol), _
        Order1:=xlAscending, Header:=xlYes

    ' Add a SUMIF formula to get totals
    Cells(1, NextCol + 1).Value = "Revenue"
    Cells(2, NextCol + 1).Resize(LastRow - 1).FormulaR1C1 = _
        "=SUMIF(R2C4:R" & FinalRow & _
        "C4,RC[-1],R2C6:R" & FinalRow & "C6)"

End Sub

Figure 11.7. This macro produced a summary report by customer from a lengthy dataset. Using AdvancedFilter is the key to powerful macros such as these.

Another use of a unique list of values is to quickly populate a list box or a combo box on a userform. For example, suppose that you have a macro that can run a report for any one specific customer. To allow your clients to choose which customers to report, create a simple userform. Add a list box to the userform and set the list box’s MultiSelect property to 1-fmMultiSelectMulti. In this case, the form is named frmReport. In addition to the list box, there are four command buttons: OK, Cancel, Mark All, and Clear All. The code to run the form follows. Note that the Userform_Initialize procedure includes an advanced filter to get the unique list of customers from the dataset:

Private Sub CancelButton_Click()
    Unload Me
End Sub

Private Sub cbSubAll_Click()
    For i = 0 To lbCust.ListCount - 1
        Me.lbCust.Selected(i) = True
    Next i
End Sub

Private Sub cbSubClear_Click()
    For i = 0 To lbCust.ListCount - 1
        Me.lbCust.Selected(i) = False
    Next i
End Sub

Private Sub OKButton_Click()
    For i = 0 To lbCust.ListCount - 1
        If Me.lbCust.Selected(i) = True Then
            ' Call a routine to produce this report
            RunCustReport WhichCust:=Me.lbCust.List(i)
        End If
    Next i
    Unload Me
End Sub

Private Sub UserForm_Initialize()
    Dim IRange As Range
    Dim ORange As Range

    ' Find the size of today's dataset
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    NextCol = Cells(1, Columns.Count).End(xlToLeft).Column + 2

    ' Set up output range. Copy heading from D1 there
    Range("D1").Copy Destination:=Cells(1, NextCol)
    Set ORange = Cells(1, NextCol)

    ' Define the Input Range
    Set IRange = Range("A1").Resize(FinalRow, NextCol - 2)

    ' Do the Advanced Filter to get unique list of customers
    IRange.AdvancedFilter Action:=xlFilterCopy, _
        CopyToRange:=ORange, Unique:=True

    ' Determine how many unique customers we have
    LastRow = Cells(Rows.Count, NextCol).End(xlUp).Row

    ' Sort the data
    Cells(1, NextCol).Resize(LastRow, 1).Sort Key1:=Cells(1, NextCol), _
        Order1:=xlAscending, Header:=xlYes

With Me.lbCust
    .RowSource = ""
    .List = Cells(2, NextCol).Resize(LastRow - 1, 1).Value
End With

    ' Erase the temporary list of customers
    Cells(1, NextCol).Resize(LastRow, 1).Clear
End Sub

Launch this form with a simple module such as this:

Sub ShowCustForm()
    frmReport.Show
End Sub

Your clients are presented with a list of all valid customers from the dataset. Because the list box’s MultiSelect property is set to allow it, they can select any number of customers.

Getting Unique Combinations of Two or More Fields

To get all unique combinations of two or more fields, build the output range to include the additional fields. This code sample builds a list of unique combinations of two fields, Customer and Product:

Sub UniqueCustomerProduct()
    Dim IRange As Range
    Dim ORange As Range

    ' Find the size of today's dataset
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    NextCol = Cells(1, Columns.Count).End(xlToLeft).Column + 2

    ' Set up output range. Copy headings from D1 & B1
    Range("D1").Copy Destination:=Cells(1, NextCol)
    Range("B1").Copy Destination:=Cells(1, NextCol + 1)
    Set ORange = Cells(1, NextCol).Resize(1, 2)

    ' Define the Input Range
    Set IRange = Range("A1").Resize(FinalRow, NextCol - 2)

    ' Do the Advanced Filter to get unique list of customers & product
    IRange.AdvancedFilter Action:=xlFilterCopy, _
        CopyToRange:=ORange, Unique:=True

    ' Determine how many unique rows we have
    LastRow = Cells(Rows.Count, NextCol).End(xlUp).Row

    ' Sort the data
    Cells(1, NextCol).Resize(LastRow, 2).Sort Key1:=Cells(1, NextCol), _
        Order1:=xlAscending, Key2:=Cells(1, NextCol + 1), _
        Order2:=xlAscending, Header:=xlYes

End Sub

In the result shown in Figure 11.8, you can see that Enhanced Eggbeater buys only one product, and Agile Aquarium buys three products. This might be useful as a guide in running reports on either customer by product or product by customer.

Figure 11.8. By including two columns in the output range on a Unique Values query, we get every combination of Customer and Product.

Using Advanced Filter with Criteria Ranges

As the name implies, Advanced Filter is usually used to filter records—in other words, to get a subset of data. You specify the subset by setting up a criteria range. Even if you are familiar with criteria, be sure to check out using the powerful Boolean formula in criteria ranges later in this chapter, in the section “The Most Complex Criteria: Replacing the List of Values with a Condition Created as the Result of a Formula.”

Set up a criteria range in a blank area of the worksheet. A criteria range always includes two or more rows. The first row of the criteria range contains one or more field header values to match the one(s) in the data range you want to filter. The second row contains a value showing which records to extract. In Figure 11.9, Range J1:J2 is the criteria range, and Range L1 is the output range.

Figure 11.9. To learn a unique list of products purchased by Cool Saddle Traders, set up the criteria range shown in J1:J2.

In the Excel user interface, to extract a unique list of products that were purchased by a particular customer, select Advanced Filter and set up the Advanced Filter dialog, as shown in Figure 11.9. Figure 11.10 shows the results.

Figure 11.10. The results of the advanced filter that uses a criteria range and asks for a unique list of products. Of course, more complex and interesting criteria can be built.

In VBA, you use the following code to perform an equivalent advanced filter:

Sub UniqueProductsOneCustomer()
    Dim IRange As Range
    Dim ORange As Range
    Dim CRange As Range

    ' Find the size of today's dataset
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    NextCol = Cells(1, Columns.Count).End(xlToLeft).Column + 2

    ' Set up the Output Range with one customer
    Cells(1, NextCol).Value = Range("D1").Value
    ' In reality, this value should be passed from the userform
    Cells(2, NextCol).Value = Range("D2").Value
    Set CRange = Cells(1, NextCol).Resize(2, 1)

    ' Set up output range. Copy heading from B1 there
    Range("B1").Copy Destination:=Cells(1, NextCol + 2)
    Set ORange = Cells(1, NextCol + 2)

    ' Define the Input Range
    Set IRange = Range("A1").Resize(FinalRow, NextCol - 2)

    ' Do the Advanced Filter to get unique list of customers & product
    IRange.AdvancedFilter Action:=xlFilterCopy, _
        CriteriaRange:=CRange, CopyToRange:=ORange, Unique:=True
    ' The above could also be written as:
    'IRange.AdvancedFilter xlFilterCopy, CRange, ORange, True

    ' Determine how many unique rows we have
    LastRow = Cells(Rows.Count, NextCol + 2).End(xlUp).Row

    ' Sort the data
    Cells(1, NextCol + 2).Resize(LastRow, 1).Sort Key1:=Cells(1, _
  NextCol + 2), Order1:=xlAscending, Header:=xlYes

End Sub

Joining Multiple Criteria with a Logical OR

You might want to filter records that match one criteria or another. For example, you can extract customers who purchased either product M556 or product R537. This is called a logical OR criteria.

When your criteria should be joined by a logical OR, place the criteria on subsequent rows of the criteria range. For example, the criteria range shown in J1:J3 of Figure 11.11 tells you which customers order product M556 or product W435.

Figure 11.11. Place criteria on successive rows to join them with an OR. This criteria range gets customers who ordered either product M556 or W435.

Joining Two Criteria with a Logical AND

Other times, you will want to filter records that match one criterion and another criterion. For example, you might want to extract records in which the product sold was W435 and the region was the West region. This is called a logical AND.

To join two criteria by AND, put both criteria on the same row of the criteria range. For example, the criteria range shown in J1:K2 of Figure 11.12 gets the customers who ordered product W435 in the West region.

Figure 11.12. Place criteria on the same row to join them with an AND. The criteria range in J1:K2 gets customers from the West region who ordered product W435.

Other Slightly Complex Criteria Ranges

The criteria range shown in Figure 11.13 is based on two different fields that are joined with an OR. The query finds all records that either are from the West region or in which the product is W435.

Figure 11.13. The criteria range in J1:K3 returns records in which either the Region is West or the Product is W435.

The Most Complex Criteria: Replacing the List of Values with a Condition Created as the Result of a Formula

It is possible to have a criteria range with multiple logical AND and logical OR criteria joined together. Although this might work in some situations, in other scenarios it quickly gets out of hand. Fortunately, Excel allows for criteria in which the records are selected as the result of a formula to handle this situation.

Setting Up a Condition as the Result of a Formula

Amazingly, there is an incredibly obscure version of Advanced Filter criteria that can replace the 44,000-row criteria range in the previous case study. In the alternative form of criteria range, the top row is left blank. There is no heading above the criteria. The criterion set up in Row 2 is a formula that results in True or False. If the formula contains any relative references to Row 2 of the data range, Excel compares that formula to every row of the data range, one by one.

For example, if you want all records in which the Gross Profit Percentage is below 53 percent, the formula built in J2 will reference the Profit in H2 and the Revenue in F2. To do this, leave J1 blank to tell Excel that you are using a computed criterion. Cell J2 contains the formula =(H2/F2)<0.53. The criteria range for the advanced filter would be specified as J1:J2.

As Excel performs the advanced filter, it logically copies the formula and applies it to all rows in the database. Anywhere that the formula evaluates to True, the record is included in the output range.

This is incredibly powerful and runs remarkably fast. You can combine multiple formulas in adjacent columns or rows to join the formula criteria with AND or OR, just as you do with regular criteria.

Using Formula-Based Conditions with VBA

Referring back to the userform shown in Figure 11.14, you can use formula-based conditions to filter the report using the userform. The following is the code for this userform. Note the logic in OKButton_Click that builds the formula. Figure 11.17 shows the Excel sheet just before the advanced filter is run.

Figure 11.17. The worksheet just before the macro runs the advanced filter.

The following code initializes the user form. Three advanced filters find the unique list of customers, products, and regions:

Private Sub UserForm_Initialize()
    Dim IRange As Range
    Dim ORange As Range

    ' Find the size of today's dataset
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    NextCol = Cells(1, Columns.Count).End(xlToLeft).Column + 2

    ' Define the input range
    Set IRange = Range("A1").Resize(FinalRow, NextCol - 2)

    ' Set up output range for Customer. Copy heading from D1 there
    Range("D1").Copy Destination:=Cells(1, NextCol)
    Set ORange = Cells(1, NextCol)

    ' Do the Advanced Filter to get unique list of customers
    IRange.AdvancedFilter Action:=xlFilterCopy, CriteriaRange:="", _
        CopyToRange:=ORange, Unique:=True

    ' Determine how many unique customers we have
    LastRow = Cells(Rows.Count, NextCol).End(xlUp).Row

    ' Sort the data
    Cells(1, NextCol).Resize(LastRow, 1).Sort Key1:=Cells(1, NextCol), _
        Order1:=xlAscending, Header:=xlYes

    With Me.lbCust
        .RowSource = ""
        .List = Application.Transpose(Cells(2,NextCol).Resize(LastRow-1,1))
    End With

    ' Erase the temporary list of customers
    Cells(1, NextCol).Resize(LastRow, 1).Clear

    ' Set up output range for product. Copy heading from D1 there
    Range("B1").Copy Destination:=Cells(1, NextCol)
    Set ORange = Cells(1, NextCol)

    ' Do the Advanced Filter to get unique list of customers
    IRange.AdvancedFilter Action:=xlFilterCopy, _
        CopyToRange:=ORange, Unique:=True

    ' Determine how many unique customers we have
    LastRow = Cells(Rows.Count, NextCol).End(xlUp).Row

    ' Sort the data
    Cells(1, NextCol).Resize(LastRow, 1).Sort Key1:=Cells(1, NextCol), _
        Order1:=xlAscending, Header:=xlYes

    With Me.lbProduct
        .RowSource = ""
        .List = Application.Transpose(Cells(2,NextCol).Resize(LastRow-1,1))
    End With

    ' Erase the temporary list of customers
    Cells(1, NextCol).Resize(LastRow, 1).Clear

    ' Set up output range for Region. Copy heading from A1 there
    Range("A1").Copy Destination:=Cells(1, NextCol)
    Set ORange = Cells(1, NextCol)

    ' Do the Advanced Filter to get unique list of customers
    IRange.AdvancedFilter Action:=xlFilterCopy, CopyToRange:=ORange, _
        Unique:=True

    ' Determine how many unique customers we have
    LastRow = Cells(Rows.Count, NextCol).End(xlUp).Row

    ' Sort the data
    Cells(1, NextCol).Resize(LastRow, 1).Sort Key1:=Cells(1, NextCol), _
        Order1:=xlAscending, Header:=xlYes

    With Me.lbRegion
        .RowSource = ""
        .List = Application.Transpose(Cells(2,NextCol).Resize(LastRow-1,1))
    End With

    ' Erase the temporary list of customers
    Cells(1, NextCol).Resize(LastRow, 1).Clear

End Sub

These tiny procedures run when someone clicks Mark All or Clear All in the userform in Figure 11.14:

Private Sub CancelButton_Click()
    Unload Me
End Sub

Private Sub cbSubAll_Click()
    For i = 0 To lbCust.ListCount - 1
        Me.lbCust.Selected(i) = True
    Next i
End Sub

Private Sub cbSubClear_Click()
    For i = 0 To lbCust.ListCount - 1
        Me.lbCust.Selected(i) = False
    Next i
End Sub

Private Sub CommandButton1_Click()
    ' Clear all products
    For i = 0 To lbProduct.ListCount - 1
        Me.lbProduct.Selected(i) = False
    Next i
End Sub

Private Sub CommandButton2_Click()
    ' Mark all products
    For i = 0 To lbProduct.ListCount - 1
        Me.lbProduct.Selected(i) = True
    Next i
End Sub

Private Sub CommandButton3_Click()
    ' Clear all regions
    For i = 0 To lbRegion.ListCount - 1
        Me.lbRegion.Selected(i) = False
    Next i
End Sub

Private Sub CommandButton4_Click()
    ' Mark all regions
    For i = 0 To lbRegion.ListCount - 1
        Me.lbRegion.Selected(i) = True
    Next i
End Sub

The following code is attached to the OK button. This code builds three ranges in O, P, and Q that list the selected customers, products, and regions. The actual criteria range is composed of three blank cells in J1:L1 and then three formulas in J2:L2.

Private Sub OKButton_Click()
    Dim CRange As Range, IRange As Range, ORange As Range
    ' Build a complex criteria that ANDs all choices together
    NextCCol = 10
    NextTCol = 15

    For j = 1 To 3
        Select Case j
            Case 1
                MyControl = "lbCust"
                MyColumn = 4
            Case 2
                MyControl = "lbProduct"
                MyColumn = 2
            Case 3
                MyControl = "lbRegion"
                MyColumn = 1
        End Select
        NextRow = 2
        ' Check to see what was selected.
        For i = 0 To Me.Controls(MyControl).ListCount - 1
            If Me.Controls(MyControl).Selected(i) = True Then
                Cells(NextRow, NextTCol).Value = _
                    Me.Controls(MyControl).List(i)
                NextRow = NextRow + 1
            End If
        Next i
        ' If anything was selected, build a new criteria formula
        If NextRow > 2 Then
            ' the reference to Row 2 must be relative in order to work
            MyFormula = "=NOT(ISNA(MATCH(RC" & MyColumn & ",R2C" & _
                NextTCol & ":R" & NextRow - 1 & "C" & NextTCol & ",0)))"
            Cells(2, NextCCol).FormulaR1C1 = MyFormula
            NextTCol = NextTCol + 1
            NextCCol = NextCCol + 1
        End If
    Next j
    Unload Me

    ' Figure 11.19 shows the worksheet at this point
    ' If we built any criteria, define the criteria range
    If NextCCol > 10 Then
        Set CRange = Range(Cells(1, 10), Cells(2, NextCCol - 1))
        Set IRange = Range("A1").CurrentRegion
        Set ORange = Cells(1, 20)
        IRange.AdvancedFilter xlFilterCopy, CRange, ORange

        ' Clear out the criteria
        Cells(1, 10).Resize(1, 10).EntireColumn.Clear
    End If

    ' At this point, the matching records are in T1

End Sub

Figure 11.17 shows the worksheet just before the AdvancedFilter method is called. The user has selected customers, products, and regions. The macro has built temporary tables in Columns O, P, and Q to show which values the user selected. The criteria range is J1:L2. That criteria formula in J2 looks to see whether the value in $D2 is in the list of selected customers in O. The formulas in K2 and L2 compare $B2 to Column P and $A2 to Column Q.

Using Formula-Based Conditions to Return Above-Average Records

The formula-based conditions formula criteria are cool but are a rarely used feature in a rarely used function. Some interesting business applications use this technique. For example, this criteria formula would find all the above-average rows in the dataset:

=$A2>Average($A$2:$A$1048576)

Using Filter in Place in Advanced Filter

It is possible to filter a large dataset in place. In this case, you do not need an output range. You would normally specify a criteria range—otherwise, you return 100 percent of the records and there is no need to do the advanced filter!

In the user interface of Excel, running a Filter in Place makes sense: You can easily peruse the filtered list looking for something in particular.

Running a Filter in Place in VBA is a little less convenient. The only good way to programmatically peruse the filtered records is to use the xlCellTypeVisible option of the SpecialCells method. In the Excel user interface, the equivalent action is to select Find & Select, Go to Special from the Home tab. In the Go to Special dialog, select Visible Cells Only.

To run a Filter in Place, use the constant XLFilterInPlace as the Action parameter in the AdvancedFilter command and remove the CopyToRange from the command:

IRange.AdvancedFilter Action:=xlFilterInPlace, CriteriaRange:=CRange, _
    Unique:=False

Then, the programmatic equivalent to loop through Visible Cells Only is this code:

For Each cell In Range("A2:A" & FinalRow).SpecialCells(xlCellTypeVisible)
    Ctr = Ctr + 1
Next cell
MsgBox Ctr & " cells match the criteria"

If you know that there would be no blanks in the visible cells, you could eliminate the loop with

Ctr = Application.Counta(Range("A2:A" & _
FinalRow).SpecialCells(xlCellTypeVisible))

Catching No Records When Using Filter in Place

Just as when using Copy, you have to watch out for the possibility of having no records match the criteria. However, in this case it is more difficult to realize that nothing is returned. You generally find out when the .SpecialCells method returns a Runtime Error 1004—no cells were found.

To catch this condition, you have to set up an error trap to anticipate the 1004 error with the SpecialCells method.

→ See Chapter 24, “Handling Errors,” for more information on catching errors.

On Error GoTo NoRecs
    For Each cell In Range("A2:A" & FinalRow).SpecialCells(xlCellTypeVisible)
        Ctr = Ctr + 1
    Next cell
    On Error GoTo 0
    MsgBox Ctr & " cells match the criteria"
    Exit Sub
NoRecs:
    MsgBox "No records match the criteria"
End Sub

This error trap works because it specifically excludes the header row from the SpecialCells range. The header row is always visible after an advanced filter. Including it in the range would prevent the 1004 error from being raised.

Showing All Records After Filter in Place

After doing a Filter in Place, you can get all records to show again by using the ShowAllData method:

ActiveSheet.ShowAllData

The Real Workhorse: xlFilterCopy with All Records Rather Than Unique Records Only

The examples at the beginning of this chapter talked about using xlFilterCopy to get a unique list of values in a field. You used unique lists of customer, region, and product to populate the list boxes in your report-specific userforms.

However, a more common scenario is to use an advanced filter to return all records that match the criteria. After the client selects which customer to report, an advanced filter can extract all records for that customer.

In all the examples in the following sections, you want to keep the Unique Records Only check box cleared. You do this in VBA by specifying Unique:=False as a parameter to the AdvancedFilter method.

This is not difficult to do, and you have some powerful options. If you need only a subset of fields for a report, copy only those field headings to the output range. If you want to resequence the fields to appear exactly as you need them in the report, you can do this by changing the sequence of the headings in the output range.

The next sections walk you through three quick examples to show the options available.

Copying All Columns

To copy all columns, specify a single blank cell as the output range. You will get all columns for those records that match the criteria, as shown in Figure 11.18:

Sub AllColumnsOneCustomer()
    Dim IRange As Range
    Dim ORange As Range
    Dim CRange As Range

    ' Find the size of today's dataset
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    NextCol = Cells(1, Columns.Count).End(xlToLeft).Column + 2

    ' Set up the criteria range with one customer
    Cells(1, NextCol).Value = Range("D1").Value
    ' In reality, this value should be passed from the userform
    Cells(2, NextCol).Value = Range("D2").Value
    Set CRange = Cells(1, NextCol).Resize(2, 1)

    ' Set up output range. It is a single blank cell
    Set ORange = Cells(1, NextCol + 2)

    ' Define the Input Range
    Set IRange = Range("A1").Resize(FinalRow, NextCol - 2)

    ' Do the Advanced Filter to get unique list of customers & product
    IRange.AdvancedFilter Action:=xlFilterCopy, _
        CriteriaRange:=CRange, CopyToRange:=Orange

End Sub

Figure 11.18. When using xlFilterCopy with a blank output range, you get all columns in the same order as they appear in the original list range.

Copying a Subset of Columns and Reordering

If you are doing the advanced filter to send records to a report, it is likely that you might need only a subset of columns and you might need them in a different sequence.

This example finishes the frmReport example that was presented earlier in this chapter. As you recall, frmReport allows the client to select a customer. The OK button then calls the RunCustReport routine, passing a parameter to identify for which customer to prepare a report.

Imagine this is a report being sent to the customer. The customer really does not care about the surrounding region, and you do not want to reveal your cost of goods sold or profit. Assuming that you will put the customer’s name in the title of the report, the fields that you need in order to produce the report are Date, Quantity, Product, and Revenue.

The following code copies those headings to the output range. The advanced filter produces data, as shown in Figure 11.19. The program then goes on to copy the matching records to a new workbook. A title and a total row are added, and the report is saved with the customer’s name. Figure 11.20 shows the final report.

Sub RunCustReport(WhichCust As Variant)
    Dim IRange As Range
    Dim ORange As Range
    Dim CRange As Range
    Dim WBN As Workbook
    Dim WSN As Worksheet
    Dim WSO As Worksheet

    Set WSO = ActiveSheet
    ' Find the size of today's dataset
    FinalRow = Cells(Rows.Count, 1).End(xlUp).Row
    NextCol = Cells(1, Columns.Count).End(xlToLeft).Column + 2

    ' Set up the criteria range with one customer
    Cells(1, NextCol).Value = Range("D1").Value
    Cells(2, NextCol).Value = WhichCust
    Set CRange = Cells(1, NextCol).Resize(2, 1)

    ' Set up output range. We want Date, Quantity, Product, Revenue
    ' These columns are in C, E, B, and F
    Cells(1, NextCol + 2).Resize(1, 4).Value = _
        Array(Cells(1, 3), Cells(1, 5), Cells(1, 2), Cells(1, 6))
    Set ORange = Cells(1, NextCol + 2).Resize(1, 4)

    ' Define the Input Range
    Set IRange = Range("A1").Resize(FinalRow, NextCol - 2)

    ' Do the Advanced Filter to get unique list of customers & products
    IRange.AdvancedFilter Action:=xlFilterCopy, _
        CriteriaRange:=CRange, CopyToRange:=ORange

    ' Create a new workbook with one blank sheet to hold the output
    ' xlWBATWorksheet is the template name for a single worksheet
    Set WBN = Workbooks.Add(xlWBATWorksheet)
    Set WSN = WBN.Worksheets(1)

    ' Set up a title on WSN
    WSN.Cells(1, 1).Value = "Report of Sales to " & WhichCust

    ' Copy data from WSO to WSN
    WSO.Cells(1, NextCol + 2).CurrentRegion.Copy Destination:=WSN.Cells(3, 1)
    TotalRow = WSN.Cells(Rows.Count, 1).End(xlUp).Row + 1
    WSN.Cells(TotalRow, 1).Value = "Total"
    WSN.Cells(TotalRow, 2).FormulaR1C1 = "=SUM(R2C:R[-1]C)"
    WSN.Cells(TotalRow, 4).FormulaR1C1 = "=SUM(R2C:R[-1]C)"

    ' Format the new report with bold
    WSN.Cells(3, 1).Resize(1, 4).Font.Bold = True
    WSN.Cells(TotalRow, 1).Resize(1, 4).Font.Bold = True
    WSN.Cells(1, 1).Font.Size = 18

    WBN.SaveAs ThisWorkbook.Path & Application.PathSeparator & _
        WhichCust & ".xlsx"
    WBN.Close SaveChanges:=False

    WSO.Select

    ' clear the output range, etc.
    Range("J:Z").Clear

End Sub

Figure 11.19. Immediately after the advanced filter, you have just the columns and records needed for the report.

Figure 11.20. After copying the filtered data to a new sheet and applying some formatting, you have a good-looking report to send to each customer.

Excel in Practice: Turning Off a Few Drop-Downs in the AutoFilter

One cool feature is available only in Excel VBA. When you AutoFilter a list in the Excel user interface, every column in the dataset gets a field drop-down in the heading row. Sometimes you have a field that does not make a lot of sense to AutoFilter. For example, in your current dataset, you might want to provide AutoFilter drop-downs for Region, Product, and Customer, but not the numeric or date fields. After setting up the AutoFilter, you need one line of code to turn off each drop-down that you do not want to appear. The following code turns off the drop-downs for Columns C, E, F, G, and H:

Sub AutoFilterCustom()
    Range("A1").AutoFilter Field:=3, VisibleDropDown:=False
    Range("A1").AutoFilter Field:=5, VisibleDropDown:=False
    Range("A1").AutoFilter Field:=6, VisibleDropDown:=False
    Range("A1").AutoFilter Field:=7, VisibleDropDown:=False
    Range("A1").AutoFilter Field:=8, VisibleDropDown:=False
End Sub

Using this tool is a fairly rare treat. Most of the time, Excel VBA lets you do things that are possible in the user interface—and lets you do them very rapidly. The VisibleDropDown parameter actually enables you to do something in VBA that is generally not available in the Excel user interface. Your knowledgeable clients will be scratching their heads trying to figure out how you set up the cool auto filter with only a few filterable columns (see Figure 11.21).

Figure 11.21. Using VBA, you can set up an auto filter where only certain columns have the AutoFilter drop-down.

To clear the filter from the customer column, use this code:

Sub SimpleFilter()
    Worksheets("SalesReport").Select
    Range("A1").AutoFilter
    Range("A1").AutoFilter Field:=4
End Sub

Next Steps

Using techniques from this chapter, you have many reporting techniques available via the arcane Advanced Filter tool. Chapter 12, “Using VBA to Create Pivot Tables,” introduces the most powerful feature in Excel: the pivot table. The combination of advanced filters and pivot tables creates reporting tools that enable amazing applications.