Searching LDAP
In this chapter you will learn
- The basics of LDAP search
- LDAP search using filters
- Creating custom search filter
Searching for information is the most common operation performed against LDAP. A client application initiates an LDAP search by passing in search criteria, the information that determines where to search and what to search for. Upon receiving the request, the LDAP server executes the search and returns all the entries that match the criteria.
LDAP Search Criteria
The LDAP search criteria are made up of three mandatory parameters—base, scope, and filter and several optional parameters. Let’s look at each of these parameters in detail.
The base portion of the search is a Distinguished Name (DN) that identifies the branch of the tree that will be searched. For example, a base of "ou=patrons, dc=inflinx, dc=com" indicates that the search will start in the Patron branch and move downwards. It is also possible to specify an empty base, which will result in searching the root DSE entry.
The scope parameter determines how deep, relative to the base, an LDAP search needs to be performed. LDAP protocol defines three possible search scopes: base, one level, and subtree. Figure 6-1 illustrates the entries that get evaluated as part of the search with different scopes.
Figure 6-1. Search scopes
- The base scope restricts the search to the LDAP entry identified by the base parameter. No other entries will be included as part of the search. In your Library application schema, with a base DN dc=inflinx,dc=com and scope of base, a search would just return the root organization entry, as shown in Figure 6-1.
The one level scope indicates searching all the entries one level directly below the base. The base entry itself is not included in the search. So, with base dc=inflinx,dc=com and scope one level, a search for all entries would return employees and patrons organizational unit.
Finally, the subtree scope includes the base entry and all of its descendent entries in the search. This is the slowest and most expensive option of the three. In your Library example, a search with this scope and base dc=inflinx, dc=com would return all the entries.
In your Library application LDAP Server, let’s say you want to find all the patrons that live in the Midvale area. From the LDAP schema, you know that patron entries have the city attribute that holds the city name they live in. So this requirement essentially boils down to retrieving all entries that have the city attribute with a value of “Midvale”. This is exactly what a search filter does. A search filter defines the characteristics that all the returning entries possess. Logically speaking, the filter gets applied to each entry in the set identified by base and scope. Only the entries that match the filter become part of the returned search results.
An LDAP search filter is made up of three components: an attribute type, an operator, and a value (or range of values) for the attribute. Depending on the operator, the value part can be optional. These components must always be enclosed inside parentheses, like so:
Filter = (attributetype operator value)
With this information in hand, the search filter to locate all patrons living in Midvale would look like this:
(city=Midvale)
Now, let’s say you want to find all the patrons who live in Midvale area and have an e-mail address so that you can send them news of occasional library events. The resulting search filter is essentially a combination of two filter items: an item that identifies patrons in the city of Midvale and an item that identifies patrons that have an e-mail address. You have already seen the first item of the filter. Here is the other portion of the filter:
(mail=*)
The =* operator indicates the presence of an attribute. So the expression mail=* will return all entries that have a value in their mail attribute. The LDAP specification defines filter operators that can be used to combine multiple filters and create complex filters. Here is the format for combining the filters:
Filter = (operator filter1 filter2)
Notice the use of prefix notation, where the operator is written before their operands for combining the two filters. Here is the required filter for your use case:
(&(city=Midvale)(mail=*))
The & in this filter is an And operator. The LDAP specification defines a variety of search filter operators. Table 6-1 lists some of the commonly used operators.
Table 6-1. Search Filter Operators
In addition to the above three parameters, it is possible to include several optional parameters to control search behavior. For example, the timelimit parameter indicates the time allowed to complete the search. Similarly, the sizelimit parameter places an upper bound on the number of entries that can be returned as part of the result.
A very commonly used optional parameter involves providing a list of attribute names. When a search is performed, the LDAP server by default returns all the attributes associated with entries found in the search. Sometimes this might not be desirable. In those scenarios, you can provide a list of attribute names as part of the search, and the LDAP server would return only entries with those attributes. Here is an example of search method in the LdapTemplate that takes an array of attribute names (ATTR_1, ATTR_2, and ATTR_3):
ldapTemplate.search("SEARCH_BASE", "uid=USER_DN", 1, new String[]{"ATTR_1", "ATTR_2", ATTR_3}, new SomeContextMapperImpl());
When this search is performed, the entries returned will only have ATTR_1, ATTR_2, and ATTR_3. This could reduce the amount of data transferred from the server and is useful in high traffic situations.
Since version 3, LDAP servers can maintain attributes for each entry for purely administrative purposes. These attributes are referred to as operational attributes and are not part of the entry’s objectClass. When an LDAP search is performed, the returned entries will not contain the operational attributes by default. In order to retrieve operational attributes, you need to provide a list of operational attributes names in the search criteria.
Note Examples of operational attributes include createTimeStamp, which holds the time when the entry was created, and pwdAccountLockedTime, which records the time when a user’s account was locked.
LDAP injection is a technique where an attacker alters an LDAP query to run arbitrary LDAP statements against the directory server. LDAP injection can result in unauthorized data access or modifications to the LDAP tree. Applications that don’t perform proper input validation or sanitize their input are prone to LDAP injection. This technique is similar to the popular SQL injection attack used against databases.
To better understand LDAP injection, consider a web application that uses LDAP for authentication. Such applications usually provide a web page that lets a user enter his user name and password. In order to verify that username and password match, the application would then construct an LDAP search query that looks more or less like this:
(&(uid=USER_INPUT_UID)(password=USER_INPUT_PWD))
Let’s assume that the application simply trusts the user input and doesn’t perform any validation. Now if you enter the text jdoe)(&))( as the user name and any random text as password, the resulting search query filter would look like this:
(&(uid=jdoe)(&))((password=randomstr))
If the username jdoe is a valid user id in LDAP, then regardless of the entered password, this query will always evaluate to true. This LDAP injection would allow an attacker to bypass authentication and get into the application. The “LDAP Injection & Blind LDAP Injection” article available at www.blackhat.com/presentations/bh-europe-08/Alonso-Parada/Whitepaper/bh-eu-08-alonso-parada-WP.pdf discusses various LDAP injection techniques in great detail.
Preventing LDAP injection, and any other injection techniques in general, begins with proper input validation. It is important to sanitize the entered data and properly encode it before it is used in search filters.
In the previous section, you learned that LDAP search filters are very important for narrowing down the search and identifying entries. However, creating LDAP filters dynamically can be tedious, especially when trying to combine multiple filters. Making sure that all the braces are properly closed can be error-prone. It is also important to escape special characters properly.
Spring LDAP provides several filter classes that make it easy to create and encode LDAP filters. All these filters implement the Filter interface and are part of the org.springframework.ldap.filter package. Listing 6-1 shows the Filter API interface.
package org.springframework.ldap.filter;
public interface Filter {
String encode();
StringBuffer encode(StringBuffer buf);
boolean equals(Object o);
int hashCode();
}
The first encode method in this interface returns a string representation of the filter. The second encode method accepts a StringBuffer as its parameter and returns the encoded version of the filter as a StringBuffer. For your regular development process, you use the first version of encode method that returns String.
The Filter interface hierarchy is shown in Figure 6-2. From the hierarchy, you can see that AbstractFilter implements the Filter interface and acts as the root class for all other filter implementations. The BinaryLogicalFilter is the abstract superclass for binary logical operations such as AND and OR. The CompareFilter is the abstract superclass for filters that compare values such as EqualsFilter and LessThanOrEqualsFilter.
Figure 6-2. Filter hierarchy
Note Most LDAP attribute values by default are case-insensitive for searches.
In the coming sections, you will look at each of the filters in Figure 6-2. Before you do that, let’s create a reusable method that will help you test your filters. Listing 6-2 shows the searchAndPrintResults method that uses the passed-in Filter implementation parameter and performs a search using it. It then outputs the search results to the console. Notice that you will be searching the Patron branch of the LDAP tree.
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.ldap.core.DirContextOperations;
import org.springframework.ldap.core.simple.AbstractParameterizedContextMapper;
import org.springframework.ldap.core.simple.SimpleLdapTemplate;
import org.springframework.ldap.filter.Filter;
import org.springframework.stereotype.Component;
@Component("searchFilterDemo" )
public class SearchFilterDemo {
@Autowired
@Qualifier("ldapTemplate" )
private SimpleLdapTemplate ldapTemplate;
public void searchAndPrintResults(Filter filter) {
List<String> results = ldapTemplate.search("ou=patrons,dc=inflinx,dc=com", filter.encode(),
new AbstractParameterizedContextMapper<String>() {
@Override
protected String doMapFromContext(DirContextOperations context) {
return context.getStringAttribute("cn");
}
});
System.out.println("Results found in search: " + results.size());
for(String commonName: results) {
System.out.println(commonName);
}
}
}
}
An EqualsFilter can be used to retrieve all entries that have the specified attribute and value. Let’s say you want to retrieve all patrons with the first name Jacob. To do this, you create a new instance of EqualsFilter.
EqualsFilter filter = new EqualsFilter("givenName", "Jacob");
The first parameter to the constructor is the attribute name and the second parameter is the attribute value. Invoking the encode method on this filter results in the string (givenName=Jacob).
Listing 6-3 shows the test case that invokes the searchAndPrintResults with the above EqualsFilter as parameter. The console output of the method is also shown in the Listing. Notice that the results have patrons with first name jacob (notice the lowercase j). That is because the sn attribute, like most LDAP attributes, is defined in the schema as being case-insensitive.
@Test
public void testEqualsFilter() {
Filter filter = new EqualsFilter("givenName", "Jacob");
searchFilterDemo.searchAndPrintResults(filter);
}
Results found in search: 2
Jacob Smith
jacob Brady
The LikeFilter is useful for searching LDAP when only a partial value of an attribute is known. The LDAP specification allows the usage of the wildcard * to describe these partial values. Say you want to retrieve all users whose first name begins with “Ja.” To do this, you create a new instance of LikeFilter and pass in the wildcard substring as attribute value.
LikeFilter filter = new LikeFilter("givenName", "Ja*");
Invoking the encode method on this filter results in the string (givenName=Ja*). Listing 6-4 shows the test case and the results of invoking the searchAndPrintResults method with the LikeFilter.
@Test
public void testLikeFilter() {
Filter filter = new LikeFilter("givenName", "Ja*");
searchFilterDemo.searchAndPrintResults(filter);
}
Results found in search: 3
Jacob Smith
Jason Brown
jacob Brady
The wildcard * in the substring is used to match zero or more characters. However, it is very important to understand that LDAP search filters do not support regular expressions. Table 6-2 lists some substring examples.
Table 6-2. LDAP Substring Examples
| LDAP Substring | Description |
|---|---|
| (givenName=*son) | Matches all patrons whose first name ends with son. |
| (givenName=J*n) | Matches all patrons whose first name starts with J and ends with n. |
| (givenName=*a*) | Matches all patrons with first name containing the character a. |
| (givenName=J*s*n) | Matches patrons whose first name starts with J, contains character s, and ends with n. |
You might be wondering about the necessity of a LikeFilter when you can accomplish the same filter expression by simply using the EqualsFilter, like this:
EqualsFilter filter = new EqualsFiler("uid", "Ja*");
Using EqualsFilter in this scenario will not work because the encode method in EqualsFilter considers the wildcard * in the Ja* as a special character and properly escapes it. Thus, the above filter when used for a search would result in all entries that have a first name starting with Ja*.
PresentFilters are useful for retrieving LDAP entries that have at least one value in a given attribute. Consider the earlier scenario where you wanted to retrieve all the patrons that have an e-mail address. To do this, you create a PresentFilter, as shown:
PresentFilter presentFilter = new PresentFilter("email");
Invoking the encode method on the presentFilter instance results in the string (email=*). Listing 6-5 shows the test code and the result when the searchAndPrintResults method is invoked with the above presentFilter.
@Test
public void testPresentFilter() {
Filter filter = new PresentFilter("mail");
searchFilterDemo.searchAndPrintResults(filter);
}
Results found in search: 97
Jacob Smith
Aaren Atp
Aarika Atpco
Aaron Atrc
Aartjan Aalders
Abagael Aasen
Abagail Abadines
.........
.........
NotPresentFilters are used to retrieve entries that don’t have a specified attribute. Attributes that do not have any value in an entry are considered to be not present. Now, let’s say you want to retrieve all patrons that don’t have an e-mail address. To do this, you create an instance of NotPresentFilter, as shown:
NotPresentFilter notPresentFilter = new NotPresentFilter("email");
The encoded version of the notPresentFilter results in the expression !(email=*). Running the searchAndPrintResults results in the output shown in Listing 6-6. The first null is for the organizational unit entry "ou=patrons,dc=inflinx,dc=com".
@Test
public void testNotPresentFilter() {
Filter filter = new NotPresentFilter("mail");
searchFilterDemo.searchAndPrintResults(filter);
}
Results found in search: 5
null
Addons Achkar
Adeniyi Adamowicz
Adoree Aderhold
Adorne Adey
A NotFilter is useful for retrieving entries that do not match a given condition. In the “LikeFilter” section, you looked at retrieving all entries that start with Ja. Now let’s say you want to retrieve all entries that don’t start with Ja. This is where a NotFilter comes into picture. Here is the code for accomplishing this requirement:
NotFilter notFilter = new NotFilter(new LikeFilter("givenName", "Ja*"));
Encoding this filter results in the string !(givenName=Ja*). As you can see, the NotFilter simply adds the negation symbol (!) to the filter passed into its constructor. Invoking the searchAndShowResults method results in the output shown in Listing 6-7.
@Test
public void testNotFilter() {
NotFilter notFilter = new NotFilter(new LikeFilter("givenName", "Ja*"));
searchFilterDemo.searchAndPrintResults(notFilter);
}
Results found in search: 99
Aaren Atp Aarika
Atpco Aaron Atrc
Aartjan Aalders
Abagael Aasen
Abagail Abadines
.........................
It is also possible to combine NotFilter and PresentFilter to create expressions that are equivalent to NotPresentFilter. Here is a new implementation that gets all the entries that don’t have an e-mail address:
NotFilter notFilter = new NotFilter(new PresentFilter("email"));
The GreaterThanOrEqualsFilter is useful for matching all entries that are lexicographically equal to or higher than the given attribute value. For example, a search expression (givenName >= Jacob) can be used to retrieve all entries with given name alphabetically after Jacob, in addition to Jacob. Listing 6-8 shows this implementation along with the output results.
@Test
public void testGreaterThanOrEqualsFilter() {
Filter filter = new GreaterThanOrEqualsFilter("givenName", "Jacob");
searchFilterDemo.searchAndPrintResults(filter);
}
Results found in search: 3
Jacob Smith
jacob Brady
Jason Brown
The LessThanOrEqualsFilter can be used to match entries that are lexicographically equal or lower than the given attribute. So, the search expression (givenName<=Jacob) will return all entries with first name alphabetically lower or equal to Jacob. Listing 6-9 shows the test code that invokes searchAndPrintResults implementation of this requirement along with the output.
@Test
public void testLessThanOrEqualsFilter() {
Filter filter = new LessThanOrEqualsFilter("givenName", "Jacob");
searchFilterDemo.searchAndPrintResults(filter);
}
Results found in search: 100
Jacob Smith
Aaren Atp
Aarika Atpco
Aaron Atrc
Aartjan Aalders
Abagael Aasen
Abagail Abadines
Abahri Abazari
....................
As mentioned, the search includes entries with first name James. The LDAP specification does not provide a less than (<) operator. However, it is possible to combine NotFilter with GreaterThanOrEqualsFilter to obtain “less than” functionality. Here is an implementation of this idea:
NotFilter lessThanFilter = new NotFilter(new GreaterThanOrEqualsFilter("givenName", "James"));
The AndFilter is used to combine multiple search filter expressions to create complex search filters. The resulting filter will match entries that meet all the sub-filter conditions. For example, the AndFilter is suitable for implementing an earlier requirement to get all the patrons that live in the Midvale area and have an e-mail address. The following code shows this implementation:
AndFilter andFilter = new AndFilter();
andFilter.and(new EqualsFilter("postalCode", "84047"));
andFilter.and(new PresentFilter("email"));
Invoking the encode method on this filter results in (&(city=Midvale)(email=*)). Listing 6-10 shows the test case that creates the AndFilter and calls the searchAndPrintResults method.
@Test
public void testAndFilter() {
AndFilter andFilter = new AndFilter();
andFilter.and(new EqualsFilter("postalCode", "84047"));
andFilter.and(new PresentFilter("mail"));
searchFilterDemo.searchAndPrintResults(andFilter);
}
Results found in search: 1
Jacob Smith
Like the AndFilter, an OrFilter can be used to combine multiple search filters. However, the resulting filter will match entries that meet any of the sub-filter conditions. Here is one implementation of the OrFilter:
OrFilter orFilter = new OrFilter();
orFilter.add(new EqualsFilter("postalcode", "84047"));
orFilter.add(new EqualsFilter("postalcode", "84121"));
This OrFilter will retrieve all patrons that live in either the 84047 or the 84121 postal code. The encode method returns the expression (|(postalcode =84047)(postalcode=84121)). The test case for the OrFilter is shown in Listing 6-11.
@Test
public void testOrFilter() {
OrFilter orFilter = new OrFilter();
orFilter.or(new EqualsFilter("postalCode", "84047"));
orFilter.or(new EqualsFilter("postalCode", "84121"));
searchFilterDemo.searchAndPrintResults(orFilter);
}
Results found in search: 2
Jacob Smith
Adriane Admin-mtv
The HardcodedFilter is a convenience class that makes it easy to add static filter text while building search filters. Let’s say you are writing an admin application that allows the administrator to enter a search expression in a text box. If you want to use this expression along with other filters for a search, you can use HardcodedFilter, as shown:
AndFilter filter = new AndFilter();
filter.add(new HardcodedFilter(searchExpression));
filter.add(new EqualsFilter("givenName", "smith"));
In this code, the searchExpression variable contains the user-entered search expression. HardcodedFilter also comes in very handy when the static portion of a search filter comes from a properties file or a configuration file. It is important to remember that this filter does not encode the passed-in text. So please use it with caution, especially when dealing with user input directly.
The WhitespaceWildcardsFilter is another convenience class that makes creation of sub-string search filters easier. Like its superclass EqualsFilter, this class takes an attribute name and a value. However, as the name suggests, it converts all whitespaces in the attribute value to wildcards. Consider the following example:
WhitespaceWildcardsFilter filter = new WhitespaceWildcardsFilter("cn", "John Will");
This filter results in the following expression: (cn=*John*Will*). This filter can be useful while developing search and lookup applications.
Creating Custom Filters
Even though the filter classes provided by Spring LDAP are sufficient in most cases, there might be scenarios where the current set is inadequate. Thankfully, Spring LDAP has made it easy to create new filter classes. In this section, you will look at creating a custom approximate filter.
Approximate filters are used to retrieve entries with attribute values approximately equal to the specified value. Approximate expressions are created using the ∼= operator. So a filter of (givenName ∼= Adeli) will match entries with first name such as Adel or Adele. The approximate filter is useful in search applications when the actual spelling of the value is not known to the user at the time of the search. The implementation of the algorithm to find phonetically similar values varies from one LDAP server implementation to another.
Spring LDAP does not provide any out-of-the-box class to create an approximate filter. In Listing 6-12, you create an implementation of this filter. Notice that the ApproximateFilter class extends the AbstractFilter. The constructor is defined to accept the attribute type and the attribute value. In the encode method, you construct the filter expression by concatenating the attribute type, operator, and the value.
import org.springframework.ldap.filter.AbstractFilter;
private class ApproximateFilter extends AbstractFilter {
private static final String APPROXIMATE_SIGN = "∼=";
private String attribute;
private String value;
public ApproximateFilter(String attribute, String value) {
this.attribute = attribute;
this.value = value;
}
@Override
public StringBuffer encode(StringBuffer buff) {
buff.append('('),
buff.append(attribute).append(APPROXIMATE_SIGN).append(value);
buff.append(')'),
return buff;
}
}
Listing 6-13 shows the test code for running the searchAndPrintResults method with ApproximateFilter class.
@Test
public void testApproximateFilter() {
ApproximateFilter approx = new ApproximateFilter("givenName", "Adeli");
searchFilterDemo.searchAndPrintResults(approx);
}
Here is the output of running the test case:
Results found in search: 6
Adel Acker
Adela Acklin
Adele Acres
Adelia Actionteam
Adella Adamczyk
Adelle Adamkowski
Handling Special Characters
There will be times when you need to construct search filters with characters such as ( or a * that have special meanings in LDAP. To execute these filters successfully, it is important to escape the special characters properly. Escaping is done using the format xx where xx denotes the hexadecimal representation of the character. Table 6-3 lists all of the special characters along with their escape values.
Table 6-3. Special Characters and Escape Values
| Special Character | Escape Value |
|---|---|
| ( | 28 |
| ) | 29 |
| * | 2a |
| 5c | |
| / | 2f |
In addition to the above characters, if any of the following characters are used in a DN, they also need to be properly escaped: comma (,), equals sign (=), plus sign (+), less than (<), greater than (>), pound sign (#), and semi-colon (;).
Summary
In this chapter, you learned how to simplify LDAP searches using search filters. I started the chapter with an overview of LDAP search concepts. Then you looked at different search filters that you can use to retrieve data in a variety of ways. You also saw how Spring LDAP makes it easy to create custom search filters.
In the next chapter, you will look at sorting and paging the results obtained from an LDAP server.