The process of immune cell differentiation plays a central role in orchestrating immune responses. It is based on the differentiation of naïve immune cells that, upon activation of their transcriptional machinery through a variety of signaling cascades, become phenotypically and functionally different entities capable of responding to a wide range of viruses, bacteria, parasites, or cancer cells. Functionally, immune cells have been classified into either regulatory or effector cell subsets. The cell differentiation process involves a series of sequential and complex biochemical reactions within the intracellular compartment of each cell. The Systems Biology Markup Language (SBML) is an Extensible Markup Language (XML)–based format widely used to represent as well as store models of biological processes. SBML allows the encoding of biological process including their dynamics. This information can be unambiguously converted into a system of ordinary differential equations (ODEs). Of note, ODE models are extensively used to model biological processes such as cell differentiation, immune responses toward specific pathogens, autoimmune processes, or intracellular activation of specific cellular pathways (Carbo et al., 2013, 2014a, b). Several equations are usually required to adequately represent these complex immunological processes, being either at the level of the whole organism, tissue, cells, or molecules.

Carbo et al. (2014b) published the first comprehensive ODE model of CD4 + T cell differentiation, which encompassed both effector T helper (Th1, Th2, Th17) and regulatory Treg cell phenotypes. CD4 + T cells play an important role in regulating adaptive immune functions as well as orchestrating other subsets to maintain homeostasis (Zhu and Paul, 2010). They interact with other immune cells by releasing cytokines that could further promote, suppress, or regulate immune responses. CD4 + T cells are essential in B-cell antibody class switching, in the activation and growth of CD8 + cytotoxic T cells, and in maximizing bactericidal activity of phagocytes such as macrophages. Mature T helper cells express the surface protein CD4, for which this subset is referred to as CD4 + T cells. Upon antigen presentation, naïve CD4 + T cells become activated and undergo a differentiation process controlled by the cytokine milieu in the tissue environment. The cytokine environmental composition, therefore, represents a critical factor in CD4 + T cell differentiation. As an example, a naïve CD4 + T cell in an environment rich in IFNγ or IL-12 will differentiate into Th1. In contrast, an environment rich in IL-4 will induce a Th2 phenotype. Some other phenoptypes are also balanced by each other: Th17 cells, induced by IL-6, IL-1β, and TGF-β, are closely balanced by regulatory T cells (induced by TGFβ only) (Eisenstein and Williams, 2009). Furthermore, competition for cytokines by competing clones of CD4 + T cells within an expanding cell population (proliferation), cell death, and expression of other selective activation factors such as the T cell receptor, OX40, CD28, ICOS, and PD1 are key steps that influence CD4 + T cell differentiation.

Computational approaches allow concurrent multiparametric analysis of biological processes and computational algorithms and models have become powerful and widely used tools to improve the efficiency and reduce the cost of the knowledge discovery process. Systems modeling approaches, combined with experimental immunology studies in vivo, can integrate existing knowledge and provide novel insights on rising trends and behaviors in biological processes such as CD4 + T cell differentiation and function. The CD4 + T cell differentiation model was built upon the current paradigms of molecular interactions that occur in CD4 + T cells, which consists of 60 ODEs, 53 reactions, and 94 species. The mathematical model ensures proper modulation of intracellular pathways and cell phenotypes via external cytokines representing the cytokine milieu. Two types of kinetic equations were employed to mathematically compute dynamic biological processes in the CD4 + T cell model: (i) mass action and (ii) Hill equation kinetics. Despite their simplicity, mass-action kinetics are widely accepted and extensively validated in biological systems due to their inherent ability to accurately represent elementary reactions and species degradation (Goldbeter, 1991). Mass-action rates are also extremely reliable for stochastic modeling simulations. In the CD4 + T cell model, the natural loss of model species due to messenger RNA (mRNA) and protein decay was fit using mass-action rate laws. On the other hand, sigmoidal Hill equations were used to model more complex molecular processes that behave via an on/off switch mechanism, including protein phosphorylation, cytokine-receptor binding, and transcription. Extensive studies have demonstrated the benefits of the Hill equation for studying combinatorial regulation, especially in sigmoidal Hill equations (Mangan and Alon, 2003); thus, this equation set captures complexities arising when a particular model species can be modified by more than one input. Results from modeling the pleotropic and highly dynamic regulation of CD4 + T cell differentiation has guided experimentation to elucidate underlying regulatory mechanisms, identify novel putative CD4 + T cell subsets or potential targets, and enrich our understanding of the dynamics of the process (Kidd et al., 2014; Yosef et al., 2013).

ODE-based modeling approaches require detailed knowledge of kinetic parameters, some of which can be estimated from the research literature and some from in silico experiments. However, models that are based on a large parameter set will be subject to more inaccuracies. Thus, the use of novel modeling approaches applicable to the immune system, and specifically to the CD4 + cell differentiation, has a high value for investigation.

1. B MSM and model reduction

Current biomedical research involves performing experiments and developing hypotheses that link different scales of biological systems, such as intracellular signaling or transcriptional interactions, cellular behavior, and cell population behavior, tissue, and organism-level events. Computational modeling efforts exploring such multiscale systems quantitatively have to incorporate an array of techniques due to the different time and space scales involved. In a previous study, Mei et al. (2012) presented the Enteric Immunity Simulator (ENISI), an agent-based simulator for modeling mucosa immune responses to enteric pathogens. ENISI uses a rule-based approach and can simulate cells, cytokines, cell movement, and cell-cell interactions. To be able to model fine-grained intracellular behaviors, a multiscale modeling (MSM) approach that embeds intracellular models into the intercellular tissue level models is needed. Indeed, the MSM approach includes four scales: Intracellular, Chemokine/Cytokine Diffusion (Intercellular), Cellular, and Tissue. Our current version of ENISI incorporates the Cellular, Chemokine, and Tissue Scales. The cellular scale represents how the cells interact with nearby cells and incorporates the plasticity of a cell based on stochastic and temporal rules. The chemokine scale represents the chemokine concentration and diffusion process. Finally, the tissue scale represents the spatial and compartmental information (Figure 1.1).

Fine-grained ODE models of intracellular pathways controlling immune cell differentiation are adequate for studying the mechanisms of cell differentiation. However, they can be highly complex and expensive from a computational standpoint, especially when embedded within large-scale, agent-based simulations. ENISI Visual models a large number of cells and microbes in the gastrointestinal mucosa. If each agent is represented by 60 ODEs, for example, the simulation will be hardly scalable. Therefore, to be able to develop efficient, agent-based, multiscale models, model reduction needs to be performed. In addition, multiscale models usually do not require all the internal details of intracellular scales to have predictive value. In essence, novel model reduction strategies could be used to address the multiscale scalability requirements to reduce molecular models before integrating them into large-scale, agent-based, tissue-level models.

1. C ANN algorithm and its applications

The artificial neural networks (ANN) algorithm, inspired by the biological neural systems, is powerful in modeling and data-mining tools based upon the theory of connectionism (Yegnanarayana, 2009). In biological systems, neurons are connected to each other through synapses. A neuron receives inputs from multiple neurons and outputs a value based upon the activation function. The perceptron is one of the easiest data structures for the study of neural networking. The perceptron models a neuron’s behavior in the following way: First, the perceptron receives several input values. The connection for each input has a weight in the range of 0 to 1, these values are randomly picked and have an arbitrary unit (a.u.). The threshold unit then sums the inputs, and if the sum exceeds the threshold value, a signal is sent to the output node; otherwise, no signal is sent. The perceptron can “learn” by adjusting the weights to approach the desired output (Nielsen, 2001).

Building on the algorithm of the simple perceptron, the multilayer perceptron (MLP) model not only gives a perceptron structure for representing more than two classes, it also defines a learning rule for this kind of networks. The MLP is divided into three layers: the input layer, the hidden layer, and the output layer, where each layer in this order processes inputs and deliver outputs to the next layer (Nielsen, 2001). The extra layers give the structure needed to recognize nonlinearly separable classes (Figure 1.2). The network structures and the parameters of the activation function are important factors when developing neural network models. Feedforward neural networks are frequently used structures in modeling. There are effective learning algorithms for the parameters once the structures are set in the feedforward ANNs.

Neural network algorithms are widely used for data mining tasks such as classification and pattern recognition. Neural network algorithms are especially effective in modeling nonlinear relationships, which makes them ideal candidates for differentiation processes. Importantly, this process is scalable. However, there are also some practical challenges. It is not possible to know in advance the ideal network topology; therefore, ANN-based methods require testing several network settings or topologies in order to find the best solution. This technical challenge triggers an extended training period. Our initial pilot study was the first to apply neural network algorithms to studying immune cell differentiation (Mei et al., 2013). Based on the initial success of this approach, the study was systematized and expanded. More specifically, the effect of artificial noise on the data is assessed. We use neural network algorithms to reduce the intracellular CD4 + T cell differentiation ODE model into a neural network model with four inputs, five outputs, and one hidden layer of seven nodes. The four input nodes represent the four external cytokines that regulate the cell differentiation; the five output nodes represent the five cytokines that are externalized and secreted by the T-cell subsets. After training, the model achieves high accuracy in predicting the concentrations of output cytokines. For instance, if the outputs are in the range of [0, 1], then the largest average prediction error is 0.0562 for IL17.

3.1 T cell differentiation process as a use case

This study focuses on the T-cell differentiation. However, the techniques and algorithms developed herein can be applied to differentiations of other types of immune cells, such as macrophages, dendritic cells, B cells, etc. The input cytokines are internalized by the naïve T cells and regulate the T-cell differentiation process. The output cytokines are externalized and secreted.

3.2 Data for training and testing models

The data for modeling the relationship from the input and output cytokines can be derived from the T-cell differentiation ODE model (Carbo et al., 2013), which was calibrated using data from biological experiments. By changing the concentrations of the input cytokines, the steady state of the ODE model is calculated. The steady-state results provide a measure of the output cytokines that can be used in the model. Creation of data sets was achieved by using the parameter scan task of COPASI tool (Hoops et al., 2006). COPASI is a software application designed for the simulation and analysis of biochemical networks and their dynamics, which supports models in the SBML standard and can simulate their behavior using ODEs. A five-dimensional scan was performed, where five output cytokines were independently measured. All the data is normalized to the range of [0, 1]. The method used to create data sets is equal-distance sampling. For each input cytokine, five values were chosen (0, 0.25, 0.5, 0.75, and 1). Since there are five input cytokines, 625 data points were created by the parameter scan process. A total of 100 of the data points were selected randomly for training, and the remaining 525 data points were used for testing. Additionally, uniformly distributed noise was added to the output for a quantitative analysis. Table 1.1 shows an example of data points used in the study.

3.3 ANN model

The ANN model can be used to model nonlinear relationships. We developed the ANN model for T-cell differentiation using a package in R named neuralnet (Günther and Fritsch, 2010). The learning algorithm used is back-propagation. The function neuralnet is used for training neural networks, which provides the opportunity to define the required number of hidden layers and hidden neurons. The most important arguments of the neuralnet function include formula, a symbolic description of the model to be fitted; data, a data frame containing the variables specified in formula; and a hidden vector, specifying the number of hidden layers and hidden neurons in each layer (Günther and Fritsch, 2010). To optimize the performance of the ANN model, we tested different sizes of hidden layers, including 1, 2, 4, 5, 6, 7, 8, 10, and 11 hidden neurons. By comparing the average absolute difference between the model predictions and real outputs from the test data, the neural network model with seven hidden neurons was identified to perform best (Table 1.2). The number of hidden layers is a critical model parameter. If the number of layers is too small, under-learning can occur, whereas too many layers can cause overlearning or overfitting. In this study, our results demonstrated that with the network of four inputs and five outputs, seven hidden neurons were necessary to best model the complex nonlinear system of cell differentiation using back-propagation (Figure 1.4).

3.4 Comparative analysis with the LR model and SVM

The LR model was tested for its simplicity. R has a linear regression module lm, which was adapted and used in this study. The lm function is used to fit linear models, which can be used to carry out regression, single stratum analysis of variance, and analysis of covariance (Ihaka and Gentleman, 1996). The result of the linear regression model can be summarized as a linear transformation from the input cytokines to the output cytokines, as shown by Eq. (1.2). The transformation matrix, M_Tran [Eq. (1.3)], summarizes the relationship between input and output cytokine concentrations.

$\left[\begin{array}{c}\hfill \mathit{FOXP}3\hfill \\ \hfill {IFN}_{\gamma i}\hfill \\ \hfill IL17\hfill \\ \hfill {ROR}_{\gamma t}\hfill \\ \hfill \mathit{Tbet}\hfill \end{array}\right]=M_{\mathit{Tran}}\times \left[\begin{array}{c}1\\ {IFN}_{\gamma o}\\ IL12\\ IL6\\ {TFG}_{\beta }\end{array}\right]$

$M_{\mathit{Tran}}=\left[\begin{array}{c}0.0386\\ -0.0259\\ -0.0303\\ -0.0191\\ 0.00558\end{array}\begin{array}{c}0.531\\ -0.0536\\ 0.297\\ -0.568\\ 0.0551\end{array}\begin{array}{c}0.408\\ 0.155\\ -0.0466\\ 0.773\\ -0.130\end{array}\begin{array}{c}0.387\\ 0.146\\ -0.0592\\ 0.811\\ -0.132\end{array}\begin{array}{c}0.663\\ 0.0267\\ 0.129\\ -0.302\\ -0.198\end{array}\right]\text{,}$

where the rows represent IFNγ, IL12, IL6, and TGFβ, respectively.

Furthermore, a model was created using the SVM algorithm, which is another widely used supervised learning algorithm for classification and regression problems. SVM contains all the main features that characterize a maximum margin algorithm (Smola and Schölkopf, 2004). The R package, e1071 (Dimitriadou et al., 2008), was applied to build the SVM models using the same training data and test data as used by our previous modeling approaches. To optimize the performance of the SVM model, we tested different widths of radial kernel, including baseline (0.25), 1, 0.1, 0.01, and 0.001 (Table 1.3).

The prediction errors, average absolute difference between the model predictions, and real outputs from the test data of the different models are shown in Table 1.4. Considering that the data are normalized within [0, 1], the prediction error of linear regression model is obviously larger than that of neural network model. This corroborates that the T-cell differentiation process is highly nonlinear and linear regression will not be an appropriate method for this highly complex and nonlinear process. Additionally, the neural network model with seven hidden neurons was identified to perform best. By calculating the prediction error, it is concluded that the performance of the SVM model is better than the LR model, but worse than the ANN model (Table 1.4).

3.5 Capability of ANN model to analyze data with noise

Stochasticity is an inherent component of biological processes and an important aspect in modeling such systems (Kaern et al., 2005; Munsky et al., 2012; Frank, 2013; Hebenstreit et al., 2012). Thus, we incorporated noise to the output data points. A uniformly distributed noise in the range of [− 0.5%, 0.5%] was added to all five output data points independently in order to assess whether the ANN model could model the system with the same level of accuracy. The ANN model with seven hidden neurons was used to test the data set with noise. In a similar manner, 100 data points were selected randomly as the training data set. The remaining 525 data points were used for testing. Table 1.5 shows that the ANN model still outperforms the LR model and the SVM model when noise is added to the data. However, the performance of the ANN model deteriorates slightly when compared to data without added noise.