CHAPTER 6

Quantum Filtering and Control

In classical robotics, filtering and control algorithms go together, often as software routines in a joint sensing and action loop. Filtering algorithms help infer a robot’s state from noisy perceptual data of the environment that its sensors provide. Control algorithms utilize current state estimates to help facilitate the robot’s trajectory.

Hidden Markov Models (HMMs) and Kalman Filters are two common classical filtering strategies that have served useful in smoothing time series of sensor data and correcting for statistical noise factors involved. PID (which stands for “Proportional-Derivative-Integral”) is a classical control algorithm that helps facilitate a robot’s actions, allowing for precise robot motion and correction for errors in execution.

An interesting research direction in quantum robotics is development of filtering and control algorithms that can operate on quantum phenomena as opposed to classical percepts. Sensing and control of quantum phenomena is a very different problem from the classical case. Directly observing an object governed by quantum dynamics runs the risk of changing the system’s behavior, so direct sensing is generally not feasible. Control algorithms in quantum environments must take into account the unique nature of quantum mechanics in order to successfully manipulate quantum-scale objects. We will discuss approaches to sensing and controlling quantum phenomena.

6.1 QUANTUM MEASUREMENTS

Measuring and extracting information from a quantum system is necessary to implement quantum filtering and control algorithms. However, as thematically seen before, extracting information from a quantum system without collapsing it can be nontrivial. Two suggested ways in the literature of extracting information from a quantum system are projective and continuous measurements. In-depth background on quantum measurement can be found in Deutsch [1983], Gisin [1984].

6.1.1 PROJECTIVE MEASUREMENTS

In quantum mechanics, an observable physical quantity can be represented by a Hermitian operator in a Hilbert space. The Heisenberg uncertainty principle, however, suggests that two non-commutative observables cannot be simultaneously measured. For example, there are fundamental limits to the precision with which the position and momentum of a particle can be simultaneously known. A common measurement model to overcome some of challenges is projective measurements (also known as von Neumann measurements).

Because an observable M is an Hermitian operator, it can be represented as a weighted sum of orthogonal projectors acting on the state space. Thus, one can write:

where P_i is a projector onto the eigenspace M that has eigenvalue m_i. When a measurement occurs, eigenvalue m_i is returned with probability p(m_i) = ψ| P_i |ψ, and the state of the system collapses to .

Projective measurements are considered to be instantaneous, which is a reasonable assumption when the measurement signal is sufficiently strong and short-duration (the closer to an impulse, the better). In practice, it is not always feasible to build instantaneous quantum measurement methods. Thus, continuous measurements are often used to measure information from quantum systems.

6.1.2 CONTINUOUS MEASUREMENTS

Continuous measurements allow one to monitor an observable quantity over time. Continuous measurements monitor signals that appear on a longer-term time-scale, which is useful for extracting control feedback information [Jacobs and Steck, 2006].

An example of the continuous measurement methodology is letting the master quantum system of interest act upon an ancilla quantum system that can be observed using projective measurements. The system of interest can be transitively observed via the ancilla quantum system. While the ancilla quantum system is measured directly, the system of interest is not. Thus, the system of interest is allowed to evolve with minimal disruption of its quantum state. Hidden Quantum Markov Models (discussed in upcoming Section 6.2.2), Quantum Kalman Filters (discussed in upcoming Section 6.3.2), and the Stochastic Master Equation control strategy (discussed in upcoming Section 6.4.5) all leverage continuous measurement methodologies.

6.2 HIDDEN MARKOV MODELS AND QUANTUM EXTENSION

In a classical sensing scenario, a robot’s sensors provide measurements that can be used to infer the robot-environment state. Reasoning over a time series of measurements in a dynamic environment can be a nontrivial statistical inference problem. The Hidden Markov Model (HMM) is a tool to help simplify this inference problem into one that is more computationally tractable.

6.2.1 CLASSICAL HIDDEN MARKOV MODELS

Hidden Markov Models (HMMs) are a method for representing probability distributions over data sequences. Imagine one has a time series of sensor data. In a robotics context, one may like to know the state of the robot and world from the sensor data. The true state of the system is “hidden” (i.e., unknown) at all times, though the sensor data received provides clues as to the time-evolution of the system. The HMM provides probabilistic reasoning capability to attempt to infer the state of the world from the time series of percept data.

Figure 6.1: Graphical structure of Hidden Markov Model.

The graphical model of an HMM is shown in Figure 6.1. The graphical model of the HMM encodes the Markov assumption that knowledge of the current state means future events are independent of events prior to the current state:

The evolution of the system state can thus be described by the sequential application of the transition matrix. Let s_n be the |S| × 1 state distribution at a particular time n. Using the Markov assumption, it can be computed as:

where p₀ is the distribution of system start states. The probability of making observation a at time n is:

where O_a is the 1 × |S| row vector corresponding to observation a in matrix O. Through sequential application of these equations, one can obtain the probability of a sequence of independent observations o_a,1, o_b,2, o_c,3:

Though the true state of the system may be unknown at each time step, it can be inferred from the probabilities of obtaining particular observations in a given state based on the observation model, the distribution of initial system states, and the dynamics of the system expressed by the transition matrix. The Forward-Backward algorithm of the HMM provides not only the forward prediction described in Equations (6.3) and (6.4) but retrospective smoothing on state estimates upon input of latest data points.

An excellent treatment of HMMs is given by Rabiner and Juang [1986]. In robotics, HMMs have found a myriad of applications in sensor fault diagnosis [Verma et al., 2004], terrain mapping and classification [Wolf et al., 2005], understanding of human intent [Kelley et al., 2008], learning from human demonstration [Hovland et al., 1996], and robot introspection [Fox et al., 2006]. Given the widespread use of the classical HMM in representing various sensing and denoising scenarios in robotics, one may expect Hidden Quantum Markov Models to provide similarly significant contributions to the field of quantum robotics.

6.2.2 HIDDEN QUANTUM MARKOV MODELS

A Hidden Quantum Markov Model (HQMM) [Clark et al., 2015] is the quantum version of the HMM, and operates at quantum scale. The HQMM helps filter and make sense of a system governed by quantum dynamics rather than classical dynamics.

The HQMM starts with initial state probabilities of the quantum system that can be described by a density matrix, ρ_s. At each time step, the quantum system evolves and produces an output symbol. The HQMM is governed by a set of Kraus operations {K_m}. Recall from Section 4.3, a set of Kraus matrices {K₁, … , K_K} satisfies . Given this, the time evolution of the initial density matrix is given by:

where t is time and δt is a small change in time. The observation probabilities of the sequence of observations o_a,1, o_b,2, o_c,3 is given by:

This rule is similar to the classical case in that sequential application of matrices yields the final result. The key difference is the Kraus matrices represent both the state transition matrix and observation matrix jointly. For the HQMM, these are not independent processes but very much intertwined.

A key ongoing research challenge is to find ways to implement Kraus operators for HQMMs. One potential way is to utilize an ancilla quantum system whose internal state interacts with the HQMM’s qubits. The ancilla system’s state is read out via projective measurements at each time step to provide indirect information about the primary quantum system. The primary quantum system’s internal state remains hidden, though clues are given through its entanglement with this ancilla system.

Some proposed methods of implementing the overall HQMM include using the successive, non-adaptive readout of entangled many-body states [Monras et al., 2011], the time evolution of an open quantum system on a coarse-grained time scale [Sweke et al., 2014], and using an open quantum system with instantaneous feedback with interactions from the surrounding environment [Ralph, 2011].

Interestingly, the HQMM may be able to represent some stochastic processes more efficiently than HMMs [Monras et al., 2010]. The “realization problem” for HMMs [Vidyasagar, 2005] is: Suppose m is a positive integer and let M = {1, … , m}. Suppose Y_t is a stationary process assuming values in M. Does there exist an HMM that reproduces the statistics of the process?

There exist some stochastic processes that have a more compact representation with HQMMs than HMMs. Monras et al. [2010] provides the following example of such a stochastic process. Consider the 4-symbol stochastic process (s ∈ {0, 1, 2, 3}) given by the transition matrices in Equation (6.8):

Using a Hankel matrix bound [Anderson, 1999], [Monras et al., 2010] proves that the stochastic process cannot be represented by a two-state HMM and requires three states. However, the stochastic process can be represented by a HQMM with two states.

6.3 KALMAN FILTERING AND QUANTUM EXTENSION

A robot’s sensors allow it to measure and quantify phenomena in its environment. Some of what is sensed is actual useful signal from sources of interest in the world. Much of what is sensed, however, is noise due to external factors such as the background. The goal of Kalman filtering [Kalman, 1960] is to improve resolution of signal from noise. Kalman filtering combines information from multiple sensor observations over time to improve measurement precision from percepts containing statistical noise. This section will discuss the details of the Kalman filter as well as how this tool can be extended to quantum (rather than classical) sensing scenarios.

6.3.1 CLASSIC KALMAN FILTERING

Figure 6.2: Kalman Filter can help infer true underlying system state from multiple noisy sensor observations.

Imagine a signal starting at time t = 0 and traveling to points 1 and 2 (Figure 6.2). Even though the signal has a clear trajectory, the underlying sensor data indicates that the trajectory is noisy. The purpose of the Kalman Filter is to improve estimation of the true state of the system based on combining information from multiple (potentially noisy) measurements and previous system state estimates. Kalman filtering is particularly common in navigation and control of mobile robots because of the noisy measurements that are obtained in these contexts that make state estimation of the robot challenging.

Mathematically, the Kalman Filter is described by the following equations:

The first equation describes evolution of the system state, while the second equation describes the process by which measurements are created. The system state at time k is denoted as x_k. The matrix A is the state transition model that describes how the state evolves from time step to time step, B is the control-input model applied to an input control signal u_k, w_k−1 ~ N(0, Q_k) is the process noise model using a zero mean normal distribution with covariance Q_k, z_k is the received measurement, H is an observation model that maps the state space onto the measurement space, and υ_k ~ N(0, R_k) is a zero mean Gaussian white noise variable that represents measurement noise.

The following example illustrates the concept of the Kalman Filter. Consider a scenario where measurement yields a constant signal, there is no control input, but state value and noise are still measured. These conditions imply that A = 1, u_k = 0, and H = 1. Simplifying the previous equation set yields:

From these equations, one can see that both the state estimate x_k and measurement z_k consist of some state plus sensor noise. During each iteration of the filter, the current state estimate is updated using:

where k represents the current time (typically a discrete-time step), K_k represents the Kalman gain, z_k represents the measured value, and represents the previous state estimate.

The current state estimate is a fusion of the current measured value and the previous estimate. The Kalman gain K_k is an averaging factor that is computed using a covariance matrix describing state variability.

Figure 6.3: Graphical model structure of Kalman Filter.

The graphical model structure of a Kalman Filter (shown in Figure 6.3) describes the relationship between the filter parameters and demonstrates the alternating prediction (time update) and update (measurement correction) steps. P_k is the posterior error covariance matrix which indicates the estimated state accuracy and is used to compute the Kalman gain.

6.3.2 QUANTUM KALMAN FILTERING

Since measurement of a quantum system can cause it to decohere, gaining knowledge via direct measurement is generally infeasible. The Quantum Kalman Filter [Ruppert, 2012] utilizes “non-demolition” measurements, a continuous measurement methodology. Non-demolition measurements allow indirect information acquisition of a quantum system S by coupling it with an ancilla quantum system, M, that interacts with S. Directly measuring quantum system M provides information about S, without decoherence of S.

Figure 6.4: System diagram of quantum Kalman Filter. (Reprinted from Process Control Research Group, retrieved from http://daedalus.scl.sztaki.hu, by Ruppert [2012]. Reprinted with permission.)

The operational pipeline for the Quantum Kalman Filter is shown in Figure 6.4 for a single discrete time step. At the beginning of each time step k, an ancilla quantum system is prepared with a known state θ_M. The ancilla system M is coupled with the quantum system S, whose state θ_S is unknown. The two quantum systems are then allowed to evolve according to their joint dynamics for a preset sampling time duration. Afterward, the ancilla qubit is measured and the qubit’s new state will be correlated with that of the unknown quantum system. By running the filter for multiple time steps and aggregating information across multiple measurements, one can achieve an increasingly accurate state estimation of S.

The state and measurement equations of the Quantum Kalman Filter provide the mathematical machinery for fusing information from multiple observations. The equations are analogous to their classical counterparts. However, because the measurement process is indirect as opposed to direct, the quantum and classical filters are quite different mathematical objects. The Quantum Kalman Filter equation system is given by:

where x_k is the current state, is the current state estimate, y_k is the measurement, ω_k is the measurement noise, N is the number of time steps, c is a parameter that characterizes the state of the ancilla qubit and its interaction with the qubit being monitored, and K_k is the Kalman gain (whose computation is extensively described in Ruppert [2012]). Ruppert [2012] also provides a sensitivity analysis of parameters of the Quantum Kalman Filter in simulation.

One of the key areas of exploration for moving from theory to practice is quantifying what conditions and requirements constitute a viable ancillary system to monitor a master quantum system. Ideally, the ancilla system should be able to provide sufficient information about the quantum system of interest through coupling, though must not significantly alter the state of the original system or its natural evolution. Much progress must still be made to develop practical Quantum Kalman Filters. One of the most useful applications of the technology could be in quantum control, which is discussed next.

6.4 CLASSICAL AND QUANTUM CONTROL

The study of controlling quantum phenomena has been a concurrent goal of quantum mechanics alongside much of quantum physics and chemistry. In robotics, a primary goal is to develop methods for active manipulation and feedback control of quantum systems. The nuances of entanglement and coherence make control of quantum systems more challenging than their classical counterparts. In particular, classical models are not transferable to quantum systems because both sensing and actuation behave differently. Thus, new techniques are required. Quantum control methods could extend the reach of robotics to operate in quantum-scale environments and to manipulate and control quantum phenomena. Some of the key results in classical and quantum control are discussed in this section. Dong and Petersen [2010] provide a comprehensive survey of quantum control theory and applications, and their paper is suggested for further reading.

6.4.1 OVERVIEW OF CLASSICAL CONTROL

There are two major types of classical control approaches: open loop and closed loop. In open loop control, the controller commands are computed using only system state and a state transition model, with no additional input. Open loop systems do not incorporate nor compensate for errors during execution of the control commands. Closed loop (or feedback control) systems, in contrast, incorporate measurement information into the controller and attempt to drive the system to a specified desired state. Closed loop systems compute an error, often based on the difference between a current state and the system settling point, and use this error to adjust inputs to help drive a system to the desired system state.

Proportional-Integral-Derivative (PID) control is a common strategy for implementing closed loop robot systems [Siciliano et al., 2010]. The mathematical form for PID is:

where u(t) is the time-varying controller input and e(t) is the error signal (defined as the desired point minus the measurement point). There are three gains, one for each mathematical term in the PID equation, that give rise to PID’s namesake. K_p is the proportional gain, K_i is the integral gain, and K_d is the derivative gain.

The proportional mathematical term specifies how quickly the controller corrects for deviations from the desired point. The correction factor in the control is in proportion to the error. Larger time-varying errors given by e(t), the error signal, will generally receive larger correction with the next control signal. The proportional gain K_p specifies how much of a correction factor is applied per time step.

If one uses solely the proportional term in the controller, the input signal may oscillate about the desired state, alternating between over-compensating and then under-compensating for the desired system state. The derivative term specifies a damping factor that, over time, smooths such oscillations toward the desired state. The K_d gain specifies how much damping is applied.

On their own, the previous two mathematical terms are not enough for the system to guarantee reaching the desired state. Due to errors that accumulate over time while the controller is trying to correct itself, the controller may converge to a steady state too quickly that is not the desired state. To propel the system state to the desired state, the integral term accounts for cumulative error, and is gated by the K_i gain. In additive combination, these three mathematical terms yield a sufficiently robust control strategy for many applications.

6.4.2 OVERVIEW OF QUANTUM CONTROL MODELS

Quantum control models break down across several dimensions. One dimension is whether the quantum system being controlled is open or closed. Open quantum systems interact with their external environment. Closed quantum systems, in contrast, do not. Another dimension is whether feedback is provided by the measurement process or not. Finally, there are two major views of quantum mechanics, the Schrödinger picture and the Heisenberg picture, under which quantum control algorithms can be formulated. In the Schrödinger picture (as previously seen with the Schrödinger equation in Section 2.3), the state vector of the quantum system |ψ evolves in time but the operators and observables are generally constant with respect to time. The Heisenberg picture takes the opposite view in that the state vector |ψ does not change with time but the observables do.

In upcoming sections, we discuss quantum control methods that apply to these different types of control problems. Bilinear Model (BLM) approaches are designed to control closed quantum systems. Markovian Master Equation (MME) approaches are designed to control open quantum systems. Stochastic Master Equation (SME) approaches are designed for quantum feedback control. All three of these models use the Schrödinger model for quantum mechanics. Linear Quantum Stochastic Differential Equation (LQSDE) approaches are designed to control quantum systems that use the Heisenberg formulation. Jointly, these methods allow extending classical control ideas to quantum control.

6.4.3 BILINEAR MODELS (BLM)

Bilinear Models (BLMs) are used to control closed quantum systems. A BLM expresses the state equations for system control in the form y^T A_ix = g_i where A_i are matrices and g_i are real numbers.

Recall from Chapter 2 that in a closed quantum system, the quantum state |ψ (t) evolves as described by the Schrödinger equation: (where i = . The goal is to find a final time t > 0 and control inputs u_k (t) ∈ which drive the quantum system from its initial state |ψ₀ to a target state |ψ_f. Define the total Hamiltonian acting on the system as H(t) = H_o + E_k u_k(t)H_k and the unitary transformation U(t) that describes the quantum state as a function of time: |ψ_t = U(t) |ψ₀. Plugging these definitions into the Schödinger equation, one obtains:

The Hamiltonian H(t) describes the controlled evolution of the system given its time-varying control inputs u_k(t). Solving these equations leads to a controller for the closed quantum system.

BLMs have many application areas. They are typically utilized to describe closed quantum control systems such as molecular systems in physical chemistry and spin systems in nuclear magnetic resonance [Alessandro and Dahleh, 2001].

6.4.4 MARKOVIAN MASTER EQUATION (MME)

Not all quantum systems are closed. In many applications, the quantum system is open and thus unavoidably interacts with its external environment. One way of modeling these types of systems is to use Markovian Master Equations (MME).

Consider an arbitrary quantum system that can take finite discrete states described by a time-varying density matrix ρ(t). The time evolution of the system can thus be expressed using what is referred to as a Master equation for the system:

When the state transition matrix A is time-independent (i.e., the Markov assumption), Equation (6.15) can be simplified to (t) = A_ρ(t). A commutation operator, defined as [X, ρ] = X_ρ — _ρX, can be used to write the quantum Liouville equation:

where i = . In an open quantum system, the overall system-environment Hamiltonian can be written as:

where H_s is the Hamiltonian of the quantum system, H_E is the Hamiltonian of the external environment, and H_SE is the interaction between the system and the environment. The MME strategy is to treat the interaction term H_SE as a perturbation of the original state of the system and environment H_o = H_S + H_E. Define the time-varying interaction equation:

Treating the combined system (quantum system + environment) as closed, the Liouville equation can be written as:

The formal solution is:

MME can be applied when a system has a short environmental correlation time, and negligible memory effects. Unlike BLMs, MMEs can be used for open quantum systems as opposed to just closed quantum systems. MMEs have been applied to quantum error correction [Ahn et al., 2002] and spin squeezing [Thomsen et al., 2002].

6.4.5 STOCHASTIC MASTER EQUATION (SME)

Stochastic Master Equations (SMEs) are a tool to develop quantum feedback controllers that can utilize measurement information from a quantum system to robustify control of the system. Let ρ(t) be the time-varying density matrix of the quantum system and H the Hamiltonian that acts on the system. Assuming a continuous measurement scheme is used to measure an observable X of the system, the evolution of the quantum system’s density matrix can be described by:

where κ is a parameter related to measurement strength, X = tr(X_ρ), and dW is a Wiener increment with zero mean and unit variance equal to dt and satisfies:

where y is the measurement value. The role of the Wiener increment is to incorporate new measurement information. As the measurement value changes over time, the Wiener increment changes and is incorporated into the evolution of the overall quantum system.

In general, many types of SMEs exist, and the form which the master equation takes depends on the measurement process that is being modeled. The SME is useful for modeling atomic interaction with electromagnetic fields and controlling quantum optical systems [van Handel et al., 2005].

6.4.6 LINEAR QUANTUM STOCHASTIC DIFFERENTIAL EQUATION (LQSDE)

The previously discussed models all use the Schrödinger interpretation of quantum mechanics where equations describing the time-dependence of quantum states are given. In contrast, the Linear Quantum Stochastic Differential Equation (LQSDE) approach uses the Heisenberg interpretation, which can be more convenient when time-dependent operators on Ή describe the quantum dynamics. In the Heisenberg interpretation of quantum mechanics, the operators acting on the system evolve in time instead of quantum states.

The general form for the LQSDE is:

where x(t) is a vector of time-varying self-adjoint¹ possibly noncommutative system operators. A, B, C, and D are appropriately sized matrices specifying the system model. The initial conditions of the system, x (0) = x₀ are operators that satisfy the following commutation relations:

where Θ is a real antisymmetric matrix. In the LQSDE model, the input signal can be decomposed as:

where (t) is quantum noise and βw(t) is a self-adjoint, adapted² process [Parthasarathy, 2012].

LQSDEs are useful because they describe some non-commutative linear stochastic systems, especially in linear quantum optics [James et al., 2007] and nanoscale technology [Zhang et al., 2009].

6.4.7 VERIFICATION OF QUANTUM CONTROL ALGORITHMS

When designing classical robot controllers, analysis is often done to understand key properties of the control algorithm such as its controllability, stabilizability, and reachability [Siciliano et al., 2010]. Such verification analysis helps ensure a robot controller performs within its intended specification. Similar verification analysis is likely necessary for quantum control algorithms, though the probabilistic nature of quantum mechanics makes analysis of controller properties more challenging than classical control. Model checking of quantum systems is a developing area of research [Ying et al., 2014, Ying and Ying, 2014].

6.5 CHAPTER SUMMARY

Table 6.1: Summary of quantum operating principles discussed in quantum filtering and control

¹An operator A is self-adjoint if A = A^†.

²For our purposes, this just means β_w (t) is known at time t (though not necessarily before time t).