It's all about the data, right? You can have the latest and greatest software tools available, perfectly customized to meet the needs of your organization; but if your data is incomplete, virtually non-existent, or just plain lousy, all of those wonderful tools will mean nothing.
In the previous chapter, we walked through the ins and outs of the CiviCRM interface, understanding the different types of records that will be attached to contacts. However, how do we collect data, organize it, and import it into the system?
In this chapter we will:
Out of the box (or more accurately, off the website), the fields included in CiviCRM to store your data are pretty basic. They include standard communication fields (phone, e-mail, website, and so on), address fields (street, city, state/province, country), demographics (gender, birth date, and so on), and a few additional miscellaneous fields.
Whether you are migrating from an existing database or starting from scratch, you will inevitably have additional data you need to store in fields. CiviCRM provides for this need through its custom data tools.
Custom data consists of groups and fields. You first define a group and assign it to an existing data object; you then create fields within that group.
By data object, we mean an existing type of record. Your custom fields will extend some type of record in your system, such as individual contacts, membership records, or groups. You configure what record type will be extended, along with any associated subtypes or options, when you create the custom data group.
Let's take a look at the custom data interface and walk through the creation of new custom fields to get a better sense of how this works. We can access custom data through Administer | Customize | Custom Data.
Begin by clicking Add Set of Custom Fields. Enter a name for the group and select how it will be used (what record type it will extend). This is the most important step in the process—once you begin collecting data in these fields, you will not be able to change the type of data it is connected to. Many of the options available will provide additional sub-options. For example, if you create a custom data group attached to events, you may then specify one or more event types that the data set will be used for (select- Any- to expose the custom fields to all event types).
Let's briefly review the uses available:
At the risk of being repetitive, we want to underscore the importance of thinking through exactly how your data will be used and where it should be attached before creating your custom data set. Once defined and in use, you cannot change the way the custom data group is used.
Note that you are not limited to one custom data set per used-by type. You may create as many custom data sets as you wish and may create multiple sets attached to the same record type for the purpose of organizing your fields into different tabs or inline collapsible sets.
After naming your group and choosing how it will be used, you can set the order (helpful if you have multiple groups used for the same record type), add descriptive text before and after the fields, and choose from several configuration options. These include:
If your group is attached to one of the contact record options (Contacts, Individual, Household, or Organization):
There are currently some limitations when using the multiple-record options, most notably that there is no way to expose those fields to website forms (profiles) without custom coding. Multiple-record sets must be displayed in a tab, not inline, and only the first of multiple records will be exported. In addition, be aware that once you flag a custom group for use with multiple records, you cannot undo that setting and change it back to a single-record structure. This is because you may have added multiple data records and CiviCRM would have no way of determining which of the multiple records should be retained as the single set of data.
Save your custom data set and advance to the form for creating your first custom field associated with this set. You may return at any time to adjust the settings for your group or add/delete/modify settings for fields.
As with the configuration for custom data sets, it is important to think carefully as to how your fields will be used. While you may return at a later time to alter some of the settings, the Field Type cannot be changed once it is set. This is for a simple reason—once data is collected for a specific field type, it will be stored in ways specific to that type. Changing the type could easily result in data loss. For example, if I create an Alphanumeric Text field and later want to change it to a Date type field, existing data would be lost, as it cannot reliably convert a string to a date-type format.
After naming your field, select the data type and corresponding display type. The options available will depend on what data type you've selected. The following data types are available:
CiviCRM silently rounds integer values entered outside the allowable range to the minimum or maximum value without displaying an error.
Currently, all date fields make use of a popup widget to select the date. While this is helpful for ensuring accurate data storage (the widget constructs the value from the selection and places it in the required database format), it means partial date fields (such as selecting year only) will still operate by selecting a specific month/day/year. This is unfortunately non-intuitive from a data-entry standpoint. It also impacts searches on these custom date fields, and may yield unexpected results. If you need month or year-only fields, you may find it easier to use an alphanumeric select box field as it provides more control over the display and searching.
By default, the contact reference type field will pull contact matches from the entire database. In most cases, you will not want to do this, as it would expose all your contact records to whoever is using the form. CiviCRM provides a hook for customizing the list of records available to the auto complete field: hook_civicrm_contactListQuery
. For more details about using hooks to customize CiviCRM, visit http://wiki.civicrm.org/confluence/display/CRMDOC/CiviCRM+hook+specification.
After selecting your field type, choose your preferred display format. The options available depend on the field type, and several field types have only one option value. We will list and describe all of the display types that may be available.
When choosing a display type that includes a list of options (such as select/checkbox/radio buttons), you will be provided a form for creating the option labels and values. You can enter up to 10 options initially; after saving them, you can add more. You are also given the option of reusing a list previously created for another field. The label is what will appear in the forms; the value is what will be recorded in the database and exported from the system.
Label text should be descriptive and meaningful to the person completing the form. Most of the times you will enter the same value for both label and value, unless you have a good reason to make them different.
The display types that appear are as follows:
Proceed with completing the field creation form, entering a default value (if desired and available), providing help text, and determining if the field is required. Note that if you designate the field as required it will impact all administrative forms—not simply forms exposed to the front of your site. When creating frontend forms you will have another opportunity to designate which fields are required (during profile construction).
If you would like this field included in the advanced search tool, designate it as searchable. Typically you will want most of your fields to be searchable, though it's worth considering the question. For fields that are unlikely to be searched, keeping this option off streamlines the advanced search interface.
The last option on the field creation form is to mark a field as view-only. There are two situations where this can be helpful:
Note that you cannot import data into a field designated as View Only using the Import Contacts tool. You should leave the field as editable, import your data, and then return and mark it as view-only.
On the subject of importing legacy data, you may also find it useful to initially import and then later completely hide some fields (not just mark as view-only). As with most tools in CiviCRM, you can easily disable custom fields. Doing so retains whatever data is stored in the fields, but completely removes it from the interface.
Are you interested in understanding how CiviCRM handles custom fields in the MySQL database? Open up your database in phpMyAdmin, SSH, or your preferred database management tool. The creation of a custom group adds a record to the civicrm_custom_group
table containing your configuration options, and creates a new table called civicrm_value_[unique text based on group name and id]
. Each custom field adds a record to civicrm_custom_field
and creates a new column in the appropriate table.
The table created by the custom group will always have an entity_id
column that is a foreign key to the object's table it extends. For example, a custom group used for individuals will use the entity_id
to link the custom data record to the civicrm_contact
table's ID
field.
This is helpful to know should you ever need to run custom queries on your data. The structure makes it very easy to build relationship diagrams and queries between custom data tables and the records they extend.
Custom data sets and fields are a powerful tool in your CRM strategy. You are likely to find the need to construct them during your initial implementation to meet defined data recording needs, to import legacy data, and throughout your ongoing use of the system as data needs continually evolve.
FPAGM will use custom fields to collect information about the food pantries they serve and businesses supplying food to the organization.
Since the information collected for these two types of contacts are significantly different, the association will define two custom field sets and designate them for use with each contact subtype respectively. This is done by selecting Organization as the Used For value and choosing the appropriate contact subtype in the multi-select box that appears.
One important thing to keep in mind as it relates to the impact of custom fields on your database structure is that if you migrate your site to a new server, you will need to make sure the additional tables created by custom groups are migrated. In other words, it is not sufficient to export only your data; you must also export the database structure. Importing only the data into a clean installation of CiviCRM on the new server is insufficient as you will be missing the custom data tables from your old database.