(17.47)

This model of skew variations is used to analyze the effects of process variations on various 3-D clock trees. This model is extended to include variations of the horizontal interconnects, as described in Appendix F.

The accuracy of the model is demonstrated through a comparison with Monte Carlo simulations. The structure used for this purpose is an H-tree clock distribution network. This H-tree is placed in a circuit with a total area of 10 mm×10 mm. The circuit is assumed to be manufactured in a 45 nm CMOS technology. The parameters of the transistors and the interconnects are extracted from the PTM 45 nm CMOS and global interconnect models [252] and International Technology Roadmap for Semiconductors (ITRS) [643]. The clock buffers consist of two inverters connected in series. The circuit parameters are listed in Table 17.2. The ratio of the width to the channel length is denoted by W_n/L_n and W_p/L_p for, respectively, the NMOS and PMOS transistors. The interconnect resistance and capacitance per unit length are, respectively, denoted by r_int and c_int. The physical and electrical characteristics of the TSVs are also listed in Table 17.2 and are based on the data reported in [181,58] (see Chapter 4, Electrical Properties of Through Silicon Vias). The diameter and length of the TSVs are notated, respectively, as ø_TSV and l_TSV.

The variation of the effective channel length of transistors L_eff is considered in this comparison, which is the most significant source of device variations [622,633,638,640]. Note that the effects of other sources of process variations are also represented by the circuit illustrated in Fig. 17.6 and described by the analytic model in (17.13)–(17.20). The corresponding nominal L_eff, D2D variation $(3{\sigma }_{L_{\mathrm{eff}}}^{\mathrm{D}2\mathrm{D}})$, and WID variation $(3{\sigma }_{L_{\mathrm{eff}}}^{\mathrm{WID}})$ are, respectively, 27, 2.2, and 2.7 nm [643]. The resulting variations of R_b, C_b, and D_b are listed in Table 17.1, which are obtained as discussed in Section 17.2.1 with an input slew rate of 47, 16, and 6 mV/ps. The mean value and standard deviation are denoted, respectively, by μ and σ. The ratio σ/μ indicates the importance of variations [633]. The Monte Carlo simulation is repeated 1,000 times. As reported in Table 17.1, σ of R_b, C_b, and D_b all depend on the input slew rate. Simultaneously considering both the slew rate and the load is therefore necessary when evaluating buffer delay variations.

Two H-tree topologies are used to verify the accuracy of the skew variation model. The first topology (multi-via) is illustrated in Fig. 17.5. The second topology (single via) is illustrated in Fig. 17.9. These topologies are compared in the following subsection. The H-tree spans four tiers. The clock source is located at the center of the first tier. There are 128 clock sinks in total, 32 in each tier. The clock buffers, marked with are inserted following the technique described in [602]. The clock frequency is 1 GHz and the constraint on the input slew rate is 16 mV/ps (the transition time is 5% of the clock period). The number of buffers inserted into the multi-via and single via topologies are, respectively, 168 and 540. Only a few buffers are illustrated in Figs. 17.5 and 17.9 for improved readability. The wire segments between the two buffers are simulated using a standard RC π model.

Both skew variations with uncorrelated (independent) WID variations and correlated WID variations modeled by the multi-level spatial correlation approach are simulated where five levels are assumed (l=5). The error of the skew variation model between any pair of sinks in the target clock tree is below 6%.

17.2.4 Skew Variations in Three-Dimensional Clock Tree Topologies

The skew variation for two types of 3-D global H-trees is investigated in this subsection. The first topology (a multi-via topology) is shown in Fig. 17.5, where the clock source and buffers (except for the buffers at the last level) of a 3-D H-tree are located in a single physical tier (e.g., the first tier). In this topology, the clock signal is propagated to the sinks in other tiers by multiple TSVs. The vertical lines at each leaf correspond to a cluster of TSVs. The second clock tree topology (a single via topology) is illustrated in Fig. 17.9, where a 2-D H-tree is replicated in each tier. The clock signal is propagated by a single via (or a group of TSVs to prevent TSV failure and to lower the resistance of this vertical path), connecting the clock source to each H-tree replica.

To better understand the distribution of skew within these topologies, an intratier (pair of sinks placed in the same tier) and intertier (pair of sinks placed in different tiers) distribution of skew is considered separately. In 3-D H-trees and for intratier paths, the number, size, and location of the buffers along these paths are equal for a single tier since multi-via and single via topologies are both symmetric topologies (at least within the x and y directions). D2D variations in each tier, therefore, affect these clock paths equally. Consequently, according to (17.40), for both the multi-via and single via topologies, only WID variations affect the variation of skew between sinks located on the same tier. For both the single via and multi-via tree topologies, the variation of skew between the buffers located in the same tier exhibits the same behavior as in 2-D circuits.

For a 3-D clock tree, as described by (17.44), if R_in and C_l of each buffer remain unchanged, $\Delta {\mathrm{s}}_{(u,v)}^{\mathrm{WID}}$ between two sinks decreases as the number of nonshared clock buffers (e.g., the buffers after the n_u,v buffers shown in Fig. 17.8) decreases. For a 3-D circuit with area A, the side length of each tier is $L\propto \sqrt{A/N_p}$. Consequently, the number of buffers in one tier decreases as L decreases for an increasing number of tiers forming a 3-D circuit. For the single via topology, all of the clock sinks within a tier are connected to the clock source by the same TSV. The length of this TSV and the increasing number of buffers vertically connected to this TSV do not affect the intratier skew. Consequently, for the single via topology, the distribution of skew between clock sinks in the same tier becomes narrower as the number of tiers increases.

For the multi-via topology, however, the clock sinks in the same tier connect to different TSVs. As the number of tiers increases, both the number of buffers connecting to a TSV and the length of the TSVs increase. The input slew rate decreases since the load is increasing. As reported in Table 17.1, the delay variations of the buffers after the TSVs increase. Moreover, the load of the buffers driving the TSVs increases. These changes in the topological characteristics increase σ_d(i), as described by (17.29). This increase, consequently, counteracts and can surmount the decrease in variations due to the fewer clock buffers along the clock paths.

The buffers inserted into the 3-D clock trees are reported in Table 17.3. The number of inserted buffers within one tier in the single via topology is lower than the multi-via topology, which introduces a lower WID skew variation than the multi-via topology. The total number of buffers in a single via topology is, however, much higher than the multi-via topology. The numbers of buffers in each tier for both topologies decreases as the number of tiers increases. The increasing number of buffers connected to TSVs (due to the greater number of tiers) increases σ_su,v in the multi-via topology but does not affect σ_su,v in the single via topology. Consequently, the decrease in the number of buffers leads to a reduction in skew variation within the same tier for the single via topology, as shown by the (♦) and (■) curves in Fig. 17.10A. Nevertheless, for the multi-via topology, as shown by the Δ and × curves, σ_su,v within the same tier changes nonmonotonically with the number of tiers. For the short distance sinks, σ_s1,2 increases with the number of tiers. As a result, for multi-via 3-D H-trees, simply increasing the number of tiers does not necessarily lower the clock skew. By employing the proposed skew variation model, the number of tiers that produces the lowest skew variation can be determined. Based on this behavior, in a 3-D circuit, if the data related sinks are located mostly within the same tier, the single via topology is more efficient in reducing skew variations and can therefore support a higher clock frequency.

A different behavior is observed when considering intertier skew. As described by (17.40) through (17.42), when the target clock sinks are located in different tiers, the corresponding clock skew is also affected by D2D variations. For the single via topology, the skew variation for the intertier pairs of sinks remains approximately the same irrespective of the tiers to which the sinks belong. This behavior occurs since the paths to the sinks located in different tiers do not share any common segments (see Fig. 17.9). When the number of tiers is greater than two, the skew variation decreases as the number of tiers increases, as also shown in Fig. 17.10B. Since the paths lay in different tiers, according to (17.42), the effect of D2D variations on the 3-D single via topology is much greater than in planar H-trees.

The skew variation under the multi-via topology varies significantly from the single via topology, as illustrated in Fig. 17.10B. The skew variation between tiers depends upon the location of the related sinks. According to (17.40)–(17.42), the effects of D2D variations increase as the number of buffers located in different tiers increases. For the multi-via topology, all of the clock paths preceding the TSVs are in the first tier. The effects of D2D variations on the multi-via topology, therefore, are much smaller than the single via topology, as shown in Fig. 17.10B. Nevertheless, the skew variation of the multi-via topology changes nonmonotonically with the number of tiers. In a 3-D circuit, if the data related sinks are widely distributed on several tiers, the multi-via topology is more efficient in reducing skew variations and can therefore support a higher clock frequency.

These results indicate that the performance improvement in a 3-D clock network depends significantly on the distribution of the sinks (and, consequently, the clock paths) among the tiers. When the data related sinks are distributed on different tiers, the skew of the single via 3-D clock trees is affected more by process variations than a corresponding 2-D clock tree. This behavior is consistent with the conclusions made in Section 17.1 for data paths in 3-D circuits. The effects of process variations on 3-D clock distribution networks can be mitigated in this case by employing a multi-via topology. This topology can better exploit the traits of vertical integration (i.e., shorter wires) to significantly increase the operating frequency.

Based on these observations, a hybrid H-tree topology (multi-group topology) combining the advantages of both the single and multi-via topologies exhibits the lowest clock skew variability. The multi-group topology is illustrated in Fig. 17.11. The key concept is that the m tiers forming a 3-D circuit are divided into Q groups of “data related tiers.” The data related tiers are the physical tiers containing data related registers. The i^th group of data related tiers consists of h_i (≤m) physical tiers. The clock signal is distributed within these h_i tiers by a multi-via topology.

An example of this H-tree topology is illustrated in Fig. 17.11. This H-tree includes two groups of data related tiers (Q=2). The first group spans three tiers (h₁=3), and the second group spans two physical tiers (h₂=2). The buffers contained in each group of data related tiers are denoted by empty “triangles” and “dots.” The TSVs connecting these buffers are called “sink TSVs.” The root of the multi-via topologies is connected with a “root TSV” (or a cluster of TSVs), as illustrated by the segment at the center of the tiers.

For a 3-D circuit, if all the data related clock sinks cannot be located within the same tier but in adjacent tiers, the multi-group topology is more efficient in reducing skew variations than the other topologies. Compared with the single via topology, using Q instead of m H-trees, the multi-group topology significantly reduces skew variations between data related tiers. As compared with the multi-via topology, the multi-group topology requires fewer buffers connected to the sink TSVs than buffers connected to the TSVs of the multi-via topology. Both skew variations within a single tier and skew variations between data related tiers are therefore reduced.

To quantify the ability of a multi-group tree to reduce skew variations among the clock sinks, a 3-D circuit with eight tiers is evaluated. Two variants of the multi-group topology are considered, including two groups (hybrid 2, Q=2, h_i=4) and four groups (hybrid 4, Q=4, h_i=2) of data related tiers. Simulation results are shown in Fig. 17.12.

In Fig. 17.12, skews s_1,2 and s_1,3 (illustrated in Fig. 17.5B) and s_1,6 and s_1,7 (illustrated in Fig. 17.11) are depicted exhibiting skew variations between the nearest and farthest sinks. The results, based on independent and multi-level correlated WID variations, are denoted, respectively, by (I) and (II). σ_s1,2 and σ_s1,3 produced by the multi-group topology are lower than the multi-via topology and decrease as the number of sub-H-trees increases. For the topology with four sub-H-trees (hybrid 4), s_1,2(I), s_1,3(I), s_1,2(II), and s_1,3(II) are reduced, respectively, by 55%, 23%, 44%, and 10% as compared with the multi-via topology. Although σ_s1,3 within the same tier of the multi-group topology is greater (4% for hybrid 4) than the single via topology, the intertier skews σ_s1,6 and σ_s1,7 within a group of data related tiers of the multi-group topology are significantly reduced, as shown in Fig. 17.12B. This reduction is also greater than the multi-via topology. The number of sub-H-trees within a multi-group 3-D topology is determined by the distribution of the data related sinks. Consequently, if the data related sinks are located in adjacent tiers of a 3-D circuit, the multi-group 3-D clock tree topology is more efficient in reducing skew variations than both the single and multi-via topologies.

17.3.1 Delay Variation of Buffer Stages

The distribution of the delay of a buffer stage is modeled in this subsection. The delay of a buffer stage d consists of the delay of the buffer d_b and the interconnect (horizontal wire and/or through silicon via) d_I. The variation of d is modeled in this subsection as a random variable affected by both process variations and power supply noise.

Since the variation of parameters due to process variations is typically within a small range, the delay of a buffer stage considering parameter variations can be approximated by a first order Taylor series expansion [637]

$d\left(t_r,\overrightarrow{P},C_{\mathit{lw}}\right)=d_b\left(t_r,\overrightarrow{P},C_{\mathit{lb}}\right)+d_I\left(\overrightarrow{P},C_{\mathit{lw}}\right)\approx \overline{d}+\sum _{p\in \overrightarrow{P}}\left({\frac{\partial d}{\partial p}|}_0\Delta p\right).$ (17.48)

(17.48)

The input slew rate of this buffer stage is denoted by t_r. The capacitive load at the output of the buffer and wire is denoted, respectively, by C_lb and C_lw. The nominal delay is $\overline{d}$ and the subscript “0” denotes the partial derivative assuming nominal parameters. The set of parameters affected by process variations is denoted by $\overrightarrow{P}$. Each parameter is modeled as a random variable. For instance, if the variation of the channel length of three buffers is considered, $\overrightarrow{P}$ is {L_b,1, L_b,2, L_b,3}. The variation of a parameter $\Delta p$ consists of WID and D2D variations,

$\Delta p=\Delta p_{\mathrm{WID}}+\Delta p_{\mathrm{D}2\mathrm{D}},$ (17.49)

(17.49)

where $\Delta p_{\mathrm{D}2\mathrm{D}}$ is constant among the buffers and interconnects within the same die, while $\Delta p_{\mathrm{WID}}$ varies among the components within the same die [638]. The individual partial derivatives in (17.48) are

$\frac{\partial d}{\partial p}=\frac{\partial d_b}{\partial t_r}\frac{\partial t_r}{\partial p}+\frac{\partial d_b}{\partial C_{\mathit{lb}}}\frac{\partial C_{\mathit{lb}}}{\partial p}+\frac{\partial d_b}{\partial p}+\frac{\partial d_I}{\partial C_{\mathit{lw}}}\frac{\partial C_{\mathit{lw}}}{\partial p}+\frac{\partial d_I}{\partial p}.$ (17.50)

(17.50)

The partial derivatives in (17.50) are determined from the expression for $d_b$ and $d_I$. The expression of $d_b\left({\mathit{t}}_r,\overrightarrow{P},C_{\mathit{lb}}\right)$ is obtained through analytic formulas [653] or an adjoint sensitivity analysis with SPICE-based simulations [637]. To achieve higher accuracy, the latter is used. For horizontal wires, the expression of $d_I\left(\overrightarrow{P},C_{\mathit{lw}}\right)$ is based on the RLC interconnect model described in [654,655].

The variations introduced by the TSVs are discussed in [656,657], where TSV stress induced delay variations of buffers are modeled. In this chapter, the keep out zone around the TSVs is assumed to be sufficiently large (≤10 μm [657,658]) to mitigate the effects of TSV stress. Consequently, the TSVs are modeled as RLC wires with different electrical characteristics than the horizontal interconnects

In addition to process variations, the power noise also affects the buffer delay. The supply voltage $V_{\mathit{dd}}$ is affected by the power noise v, $V_{\mathit{dd}}=V_{\mathit{dd}0}+v$, where V_dd0 is the nominal power supply voltage of the circuit. As the power noise contains several components across a large range of frequencies, the focus of this section is on the frequency noise component due to the capacitance of the on-chip power distribution network and the inductance of the package. This noise is typically between tens and several hundred MHz (e.g., 400 MHz) [649].

This noise is modeled as a sinusoidal waveform where the amplitude of the waveform is the worst case noise observed in the clock network. Assuming a clock edge arrives at the source of a clock path at time zero, t_j is the time when this clock edge arrives at buffer j. The supply noise at buffer j at time t_j is

$v\left(t_j\right)=V_n{e}^{-\epsilon t_j}\sin \left(2\pi f_n{t}_j+\varphi \right),$ (17.51)

(17.51)

$t_j=\sum _{i=1}^{j-1}{d}_i.$ (17.52)

(17.52)

The clock frequency is much higher than the resonant noise frequency, and the clock path delay is typically lower than the period of the resonant noise. Due to the significant voltage drop, the first period of the resonant noise causes the greatest clock jitter [649]. Consequently, to investigate the effects of the worst case power noise on clock distribution networks, (17.51) is approximated by an undamped sinusoidal waveform [649]

$v\left(t_j\right)\approx V_n\sin \left(2\pi f_n{t}_j+\varphi \right).$ (17.53)

(17.53)

According to (17.48) and (17.52), d_i, t_j, and v(t_j) are all random variables. Since $\Delta t_j$ is low as compared with $\overline{t_j}$, v(t_j) can also be approximated by a first order Taylor series expansion,

$v\left(t_j\right)=\overline{v}\left(t_j\right)+\Delta v\left(t_j\right)\approx \overline{v}\left(t_j\right)+{\frac{\partial v\left(t_j\right)}{\partial t_j}|}_0\Delta t_j,$ (17.54)

(17.54)

$\Delta v\left(t_j\right)\approx 2\pi V_n{f}_n\cos (2\pi f_n\overline{t_j}+\mathit{\varphi })\sum _{i=1}^{j-1}\Delta d_i.$ (17.55)

(17.55)

The amplitude V_n and frequency f_n are determined by the switching current and the circuit characteristics. The initial phase ϕ is the phase of the resonant noise when the clock edge arrives at the source of the clock path.

In 3-D circuits, the current dissipated by the tiers can differ due to the different number and size of the devices. The amplitude and frequency of the resonant supply noise change with the current within different tiers. To demonstrate this behavior, a 1-D model of a power distribution network for a three tier circuit, shown in Fig. 17.13, is considered under different switching scenarios. Four cases of switching current are considered for this power network, which are listed in Table 17.4. The pulse width and transition times of the switching current are both 1 ns. The resulting V_n and f_n are depicted in Fig. 17.14. As illustrated in this figure, different current distributions introduce a nonnegligible difference in V_n among tiers ΔV_n. The IR drop and resonance impedances among the tiers both contribute to this ΔV_n. The resonant frequency is similar among tiers (Δf_n≤3 MHz) and does not change significantly with the current.

The electrical characteristics of the TSVs depend upon the manufacturing technology [201,181]. The change of the power noise with the total resistance of the TSVs (R_tsv) is illustrated in Fig. 17.15. In this figure, the amplitude and frequency of the overall supply noise are denoted, respectively, by V3–V1 and f3–f1. The DC IR drop is denoted by V3_dc–V1_dc. Since the resonant noise is stimulated by a current pulse, the effect of the IR drop is also included in V3–V1.

A larger R_TSV introduces a higher IR drop in the first and second tiers. In the third tier, the DC IR drop is not affected by R_TSV since this tier is directly connected to the package (see Fig. 17.13). Nevertheless, a higher R_TSV decreases the quality factor (Q factor) of the resonant circuit, which decreases the amplitude of the resonance. Consequently, V_n in the third tier decreases with R_TSV. In the first and second tiers, V_n is determined by both the resonance and the IR drop. Consequently, V2 and V1 increase with R_TSV due to the significantly higher IR drop. Nevertheless, the increase in V2 and V1 is not as high as the increase in the DC IR drop due to the lower Q factor.

The resonant noise for different number of tiers in a 3-D IC is plotted in Fig. 17.16. The switching current and on-die capacitance are assumed identical for all tiers. As shown in Fig. 17.16, ΔV_n between the bottom and top tiers increases with the number of tiers. As more dies are vertically stacked, the difference in resonant noise among tiers increases.

According to (17.48), the delay variation Δd is also affected by the input slew $\Delta t_r$, which is determined by the previous buffer stage. Considering the effects of $\Delta v$ and $\Delta t_r$ on $\Delta d,$ the delay variation of the jth buffer stage is modeled as

$\Delta d_j\approx \sum _{p\in {\overrightarrow{P}}_j}\left({\frac{\partial d_j}{\partial p}|}_0\Delta p\right)+{\frac{\partial d_j}{\partial v}|}_0\Delta v\left(t_j\right)+{\frac{\partial d_j}{\partial tr}|}_0\Delta {\mathit{t}}_{\mathit{rj}}.$ (17.56)

(17.56)

The set of statistical parameters of the j^th buffer stage is denoted by ${\overrightarrow{P}}_j$, which is a subset of the entire parameter set, ${\overrightarrow{P}}_j\subseteq \overrightarrow{P}$. The input slew of the j^th buffer stage $\Delta {\mathit{tr}}_j$ is determined similarly to (17.56),

$\Delta {\mathit{tr}}_j\approx \sum _{p\in {\overrightarrow{P}}_j}\left({\frac{\partial {\mathit{t}}_{\mathit{rj}}}{\partial p}|}_0\Delta p\right)+{\frac{\partial {\mathit{t}}_{\mathit{rj}}}{\partial v}|}_0\Delta v\left(t_{j-1}\right)+{\frac{\partial t_{\mathit{rj}}}{\partial {\mathit{t}}_{\mathit{rj}-1}}|}_0\Delta {\mathit{t}}_{\mathit{rj}-1}.$ (17.57)

(17.57)

Substituting (17.55) and (17.57) into (17.56), $\Delta d_j$ can be recursively determined considering both process variations and power noise. The coefficients in (17.56) and (17.57) are obtained through an adjoint sensitivity analysis, as previously mentioned. The resulting expression, (17.56), determines the skitter, as described in the following subsection.

17.3.2 Model of Skitter in Three-Dimensional Clock Trees

The definitions of clock skew, period jitter, and skitter are illustrated in Fig. 17.17. The clock signal is fed into the 3-D clock tree from the primary clock driver. Two flip flops are driven by this clock signal, denoted, respectively, as FF₁ and FF₂. The waveforms, clk₁ and clk₂, shown in Fig. 17.17B correspond to the clock signal driving, respectively, FF₁ and FF₂. The time when the first rising edge in Fig. 17.17B arrives at the clock input is defined as the origin. The time when this clock edge arrives at FF₁ and FF₂ is, respectively, denoted by t₁ and t₂. The arrival time of the next rising edge is t′₁ and t′₂. The number of buffers from the clock input to FF₁ and FF₂ is denoted, respectively, by n₁+n₂ and n₃+n₄. The skew between the first edge of clk₁ and clk₂ is S_1,2. The clock period after the first edge for FF₁ and FF₂ are, respectively, T₁ and T₂. The clock period is T_clk. The corresponding period jitter is J₁=T₁−T_clk and J₂=T₂−T_clk.

Assuming the data is transferred from FF₁ to FF₂ within one clock cycle, T_1,2 is the time interval that determines the highest clock frequency supported by the circuit. The setup time requirement needs to be satisfied for the system to operate correctly [577,645]. The setup time slack notated as slack_setup is defined as

${\mathrm{slack}}_{\mathrm{setup}}\equiv T_{1,2}-\max \left(D_{1,2}\right)-t_{\mathrm{setup}},$ (17.58)

(17.58)

$T_{1,2}={(t}_2-t_1)+T_2=S_{1,2}+J_2+T_{\mathrm{clk}},$ (17.59)

(17.59)

where $\max \left(D_{\mathrm{1,2}}\right)$ denotes the longest data transfer time from FF₁ to FF₂. The setup time for FF₂ is $t_{\mathrm{setup}}$. Consequently, the variation of ${\mathrm{slack}}_{\mathrm{setup}}$ is affected by the variation of $T_{\mathrm{1,2}}$, called “setup skitter” and notated as $J_{\mathrm{1,2}}$,

$J_{1,2}=S_{1,2}+J_2=t_2^{\prime }-t_1-T_{\mathrm{clk}}.$ (17.60)

(17.60)

To avoid setup time violations, ${\mathrm{slack}}_{\mathrm{setup}}\ge 0$ is required under all operating conditions.

According to (17.52) and (17.60), skitter $J_{\mathrm{1,2}}$ is the linear combination of the delay of the buffer stages,

$J_{1,2}=\sum _{k=1}^{n_3+n_4}{d^{\prime }}_{2,k}-\sum _{k=1}^{n_1+n_2}{d}_{1,k},$ (17.61)

(17.61)

${\overline{J}}_{\mathrm{1,2}}=\sum _{k=1}^{n_3+n_4}{\overline{d^{\prime }}}_{2,k}-\sum _{k=1}^{n_1+n_2}{\overline{d}}_{1,k},$ (17.62)

(17.62)

$\Delta J_{1,2}=\sum _{k=1}^{n_3+n_4}\Delta {d\prime }_{2,k}-\sum _{k=1}^{n_1+n_2}\Delta d_{1,k}\approx \sum _{p\in \overrightarrow{P}}\left({\frac{\partial J_{1,2}}{\partial p}|}_0\Delta p\right),$ (17.63)

(17.63)

where $d_{2,k}^{\prime }$ is the delay of the k^th buffer stage along the path to FF₂ for the second clock edge. The mean skitter ${\overline{J}}_{\mathrm{1,2}}$ is the mean delay of all of the buffer stages considering the mean voltage supply noise (without process variations). Substituting (17.56) into (17.63), the partial derivative $({\partial J_{\mathrm{1,2}}/\partial p)|}_0$ is obtained. Consequently, skitter J_1,2 is approximated by a first order Taylor series expansion. Assuming all of the parameters are characterized by a Gaussian distribution, ΔJ_1,2 can also be approximated by a Gaussian distribution,

$\Delta J_{1,2}\mathcal{~}\mathcal{N}(0,{\sigma }_{J_{1,2}}^2),$ (17.64)

(17.64)

${\sigma }_{J_{1,2}}^2=\sum _{p\in \overrightarrow{P}}\left({\frac{\partial J_{1,2}}{\partial p}|}_0^2{\sigma }_p^2\right)+2\sum _{p,q\in \overrightarrow{P}}\left({\frac{\partial J_{1,2}}{\partial p}|}_0{\frac{\partial J_{1,2}}{\partial q}|}_0\text{cov}(p,q)\right),$ (17.65)

(17.65)

where $\text{cov}(p,q)$ denotes the covariance between two parameters. Assuming D2D variations are independent from WID variations [632,637], ${\sigma }_p^2={\sigma }_{p(\mathrm{D}2\mathrm{D})}^2+{\sigma }_{p(\mathrm{WID})}^2$. The covariance between the two parameters is determined according to the tiers to which these parameters are related and the spatial correlation between these parameters,

$\text{cov}\left(p,q\right)=\{\begin{array}{l}0,\text{if}p\text{,}q\text{are of different type or belong to different tiers}\\ \mathrm{cov}{\left(p,q\right)}_{\mathrm{WID}}+{\sigma }_{p\left(\mathrm{D}2\mathrm{D}\right)}{\sigma }_{q\left(\mathrm{D}2\mathrm{D}\right)},\mathrm{otherwise},\end{array}$ (17.66a-b)

(17.66a-b)

where the WID covariance ${\mathrm{cov}\left(p,q\right)}_{\mathrm{WID}}$ is determined by the spatial correlation between parameters p and q within the same tier. Statistically, the devices (and wires) close to each other exhibit a higher correlation than those devices far from each other. This spatial correlation can be obtained from fabricated wafers [659] or through a spatial correlation model [637,660].

As shown in (17.65) and (17.66), the variance of the setup skitter ${\sigma }_{J_{\mathrm{1,2}}}^2$ depends on the covariance between the process induced parameters. In 2-D circuits, the change of cov(p,q) is primarily determined by cov(p,q)_WID since the parameters are affected by the same D2D variations. The distribution of the clock paths therefore only affects ${\sigma }_{J_{\mathrm{1,2}}}^2$ by changing the WID covariance. In 3-D circuits, however, D2D variations vary among tiers, and the WID covariance among tiers is zero. Consequently, the distribution of the clock paths affects the skitter variation in a highly complicated manner.

In addition to the setup time slack, the hold time slack also significantly affects circuit performance. The hold time violation can also cause the failure of the entire system [645]. Moreover, this failure cannot be removed by lowering the system wide clock frequency. As illustrated in Fig. 17.17B, the hold time slack is modeled as

${\mathrm{slack}}_{\mathrm{hold}}={\min \left(D_{1,2}\right)-S}_{1,2}-t_{\mathrm{hold}},$ (17.67)

(17.67)

where $t_{\mathrm{hold}}$ is the hold time requirement. The “hold skitter” affecting ${\mathrm{slack}}_{\mathrm{hold}}$ is determined by $S_{\mathrm{1,2}}$, which is the skew between clk₁ and clk₂. Note that $S_{\mathrm{1,2}}$ is affected by both process variations and power noise.

To correctly latch the data in FF₂, ${\mathrm{slack}}_{\mathrm{hold}}\ge 0$ is required to avoid hold time violations under any operating condition. From Fig. 17.17B, S_1,2 is

$S_{1,2}=t_2-t_1=\sum _{k=1}^{n_3+n_4}{d}_{2,k}-\sum _{k=1}^{n_1+n_2}{d}_{1,k}\approx \sum _{k=1}^{n_3+n_4}{\overline{d}}_{2,k}-\sum _{k=1}^{n_1+n_2}{\overline{d}}_{1,k}+\sum _{p\in \overrightarrow{P}}\left({\frac{\partial S_{1,2}}{\partial p}|}_0\Delta p\right).$ (17.68)

(17.68)

Similar to (17.52) and (17.65), the distribution of $\Delta S_{\mathrm{1,2}}$ can be modeled as

$\Delta S_{1,2}\mathcal{~}\mathcal{N}(0,{\sigma }_{S_{1,2}}^2),$ (17.69)

(17.69)

${\sigma }_{S_{1,2}}^2=\sum _{p\in \overrightarrow{P}}\left({\frac{\partial S_{1,2}}{\partial p}|}_0^2{\sigma }_p^2\right)+2\sum _{p,q\in \overrightarrow{P}}\left({\frac{\partial S_{1,2}}{\partial p}|}_0{\frac{\partial S_{1,2}}{\partial q}|}_0\text{cov}(p,q)\right),$ (17.70)

(17.70)

where the partial derivatives are obtained similarly to the coefficients in (17.63). As shown by (17.48)–(17.70), both the setup and hold time violations are simultaneously affected by process variations and power noise. This effect and the accuracy of this model are discussed in the following subsection.

17.3.3 Skitter Related Tradeoffs in Three-Dimensional ICs

The variation of skitter for diverse characteristics of a clock distribution network is discussed in this subsection. The number and size of the buffers change with the number of tiers spanned by the clock networks which also affects the dissipated power. The related tradeoffs are discussed in Section 17.3.3.1. The change in skitter as a function of the phase and magnitude of the power noise frequency is summarized in Section 17.3.3.2.

The electrical parameters of the transistors are based on a 32 nm PTM model [252]. The parameters of the interconnects are based on an Intel 32 nm interconnect technology [649]. The parameters of the TSVs are based on data from [181]. Both the horizontal wires and TSVs are modeled by π segments in SPICE-based simulations. The variation aware model of skitter is based on Matlab. All of the simulations are performed in a Scientific Linux server (Intel Xeon 2.67 GHz, 24 cores, 24 GB memory).

The variations considered in the simulations are listed in Table 17.5. The D2D and WID ΔL_b are extracted based on ITRS data [661]. The wire variations and ΔV_th are based on [637]. The variations of the TSVs are based on [656]. Note that other sources of variations can also be described by this modeling approach. For example, the TSV stress induced delay variation in [657] can be included. In this case, the distribution of d_B in (17.48) is based on the distance between the buffer and the TSVs and the stress induced buffer delay.

17.3.3.1 Skitter versus length of clock paths, number of tiers, and power dissipation

The change in setup skitter with the length of the clock paths is the topic of this subsection. The length ranges from 0.5 to 12.5 mm within two and three tier circuits. Buffers are inserted to produce a 10% T_clk input slew for the next stage. To emphasize the relation between skitter and the length of the clock paths, all tiers are assumed to experience similar supply noise (V_n=90 mv, f_n=400 MHz, and ϕ=270^° [649]). Each pair of paths is distributed across different tiers, as shown in Fig. 17.17A. The resulting ${\mu }_{J_{\mathrm{1,2}}}$and ${\sigma }_{J_{\mathrm{1,2}}}$are illustrated in Fig. 17.18, where the suffixes “2” and “3” denote the results for, respectively, the two and three tier circuits.

The results from SPICE-based Monte Carlo simulations and the semi-analytic model (labeled as (m)) are both depicted in Fig. 17.18. As shown in this figure, both ${\mu }_{J_{\mathrm{1,2}}}$and ${\sigma }_{J_{\mathrm{1,2}}}$ deteriorate with longer clock paths. This behavior is described by the model and exhibits reasonable accuracy. The error of the model is below 11% for ${\mu }_{J_{\mathrm{1,2}}}$ and below 12% for ${\sigma }_{J_{\mathrm{1,2}}}$. Not surprisingly, long clock paths introduce high skitter in 3-D clock trees. Consequently, both the mean and standard deviation of the setup skitter increase with the length of the clock paths.

The skitter has been evaluated for no TSV variations, 5% TSV variations (σ/µ=5%), and 15% TSV variations. The difference in ${\sigma }_{J_{\mathrm{1,2}}}$ among these three cases is around 1 ps for all of the clock paths. This situation shows that TSV variations are a second order effect, consistent with the results reported in [656].

In addition to the length of the clock paths, the number of tiers spanned by these paths determines the skitter of the clock distribution network. Due to the different switching currents in power supply networks and the vertical resistance of the P/G TSVs among tiers, the devices in different tiers are subjected to different ΔV_n, as shown in Figs. 17.14 and 17.15. The tier closer to the P/G pads experiences a lower power noise, as also discussed in Chapter 18, Power Delivery for Three-Dimensional ICs.

Clock paths spanning two tiers with 20 buffers (n₁+n₂=n₃+n₄=20, see Fig. 17.17A) are considered as an example. The clock source is located on tier 2. The total length of each path is 5 mm. The initial phase ϕ (270°) and frequency f_n (400 MHz) are assumed to be the same for both tiers. Two distributions of clock paths are discussed: (A) n₁=n₂=n₃=n₄=10 and (B) n₁=n₃=15, n₂=n₄=5. Distribution (A) denotes equally divided 3-D clock paths. In distribution (B), the longest segment of the clock paths is placed in tier 2. To depict the accuracy of the model, the simulation results of the setup skitter J_1,2 for V_n1=90 mV and different V_n2 are shown in Fig. 17.19. As noted in this figure, ${\mu }_{J_{\mathrm{1,2}}}$ changes significantly with V_n2 while ${\sigma }_{J_{\mathrm{1,2}}}$varies only slightly with V_n2.

The change in setup skitter J_1,2 with (V_n2, V_n1) is illustrated in Fig. 17.20. As shown in Figs. 17.20A and B, for distribution (A), μ_J1,2 increases significantly with both V_n2 and V_n1, since higher supply noise introduces greater period jitter. The clock paths of (A) are equally distributed among the tiers. μ_J1,2 is therefore similarly affected by V_n1 and V_n2. For distribution (B), however, the situation is different. As shown in Figs. 17.20C and D, μ_J1,2 is primarily determined by V_n2 since the longest segment of the clock paths in (B) is placed in tier 2. Consequently, for unequally distributed clock paths, the mean skitter is mainly determined by the tier containing the longest part of the clock paths.

As shown in Figs. 17.20A and B, assuming V_n1=0.09 mV, distribution (A) produces higher μ_J1,2 than (B) for different V_n2. This difference in μ_J1,2 increases with ΔV_n (ΔV_n=V_n1−V_n2), from 1% to 42% of μ_JA. The reason for this significant difference is that the majority of the buffers in (B) is located in tier 2, which is more sensitive to V_n2. More generally, assuming V_n1>V_n2, the mean skitter of (B) is always lower than (A).

Consequently, the distribution of the clock paths in 3-D ICs significantly affects the mean skitter due to the different V_n among tiers. However, in 2-D circuits, this mean skitter does not vary significantly with the distribution of clock paths due to the global resonant noise at low frequencies [662]. The standard deviation σ_J1,2 of (A) and (B) is illustrated, respectively, in Figs. 17.20E and F. Similar to μ_J1,2, σ_J1,2 also increases with V_n1 and V_n2. Nevertheless, Δσ_J1,2 is relatively low as compared with Δμ_J1,2.

Alternatively, the mean value of the hold skitter S_1,2 is relatively low (≤0.5 ps) since the two clock paths have the same number, size, and distribution of buffers. Nevertheless, σ_S1,2 is nonnegligible for both distributions (A) and (B), as illustrated, respectively, in Figs. 17.21A and B. Similar to σ_J1,2, σ_S1,2 increases with V_n1 and V_n2 but Δσ_S1,2 is lower than 1.5 ps. These results demonstrate that the standard deviation of the setup and hold skitter increases with the amplitude of the resonant power noise.

The power consumed by the clock distribution networks constitutes a significant portion of the total power consumed by a complex integrated circuit [645]. Clock skitter also depends on the traits of the buffers, which in turn affect the power of the clock network. This power is evaluated under different constraints on the skitter. A pair of clock paths with a length of 5 mm is evaluated. These paths are both equally distributed across two tiers, where V_n1=0.09 volts and V_n2=0.08 volts. The skitter and power are determined from Monte Carlo simulations. A different number (14 to 40) and size (W_n) of the clock buffers are inserted along the clock paths.

Considering the Gaussian distribution of the setup skitter J_1,2 in (17.64), J_1,2 falls into the range [μ_J1,2−3σ_J1,2, μ_J1,2+3σ_J1,2] with a probability of 99.7%. Within this range, max(J_1,2) indicates the worst (maximum) skitter. For improved readability the absolute value of max(J_1,2) is shown, where max(J_1,2)=|μ_J1,2|+3σ_J1,2. The total power consumption under different constraints on max(J_1,2) for these clock paths is illustrated in Fig. 17.22. The shaded area depicts inferior buffer solutions. Point A denotes the lowest skitter that can be obtained for this example circuit. In the unshaded area, the skitter decreases as the buffer size and power increase. For the same constraint in skitter, those clock paths with fewer buffers are more power efficient.

As shown within the unshaded area, the clock paths with fewer buffers produce lower skitter. For the clock paths with 14 buffers, as the constraint becomes lower than 68 ps, significant power is required. For example, to decrease the max(J_1,2) from 68 to 58 ps (15% improvement), the buffer size is increased from 4 to 10 μm. The resulting power consumption increases from 6.9 to 14.4 mW (a 109% increase). Consequently, pursuing extreme constraints on clock skitter dissipates high power.

17.3.3.2 Skitter versus phase and frequency of the power supply noise

Another parameter that skitter depends upon is the phase difference between the clock signal and the power supply noise. Assuming that the phase difference is the same between the two tiers, meaning that ϕ₁=ϕ₂, the change in J_1,2 and S_1,2 with phase is illustrated in Fig. 17.23 where V_n1=0.09 volts and V_n2=0.07 volts.

As shown in Figs. 17.23A and B, the difference in ϕ produces a significant change not only in μ_J1,2 but also in σ_J1,2. For instance, the highest σ_J1,2 is 41% greater than the lowest σ_J1,2 for distribution (A) (see Fig. 17.23B). The worst case μ_J1,2 occurs when ϕ₁ and ϕ₂ are both approximately 270°, similar to 2-D circuits [649]. The worst σ_J1,2, however, occurs when ϕ≈205°. Therefore, if the initial phase is not 270°, the skitter can be high due to the high σ_J1,2. The difference in σ_J1,2 is low between distributions (A) and (B) since, in either case, the clock path to FF₁ and FF₂ is the same.

The behavior of the hold skitter is different. The effect of ϕ₁ and ϕ₂, on S_1,2 is shown in Fig. 17.23C. Due to the similarity between the two clock paths, the resulting S_1,2 is relatively low. The standard deviation, however, is significantly affected by ϕ. As illustrated in Figs. 17.23B and C, the change of σ_S1,2 is similar to σ_J1,2. Consequently, both for setup and hold skitter, σ changes considerably with the phase of the power supply noise. The highest σ and μ of the skitter do not occur at the same initial phase of the supply noise.

Considering the clock paths and waveforms shown in Fig. 17.17, ϕ is the time when the first clock edge arrives at the input of the clock paths. The worst case σ can be obtained by traversing all possible ϕ. Due to the excessive time required for Monte Carlo simulations, this model is highly efficient in determining the worst case skitter and the corresponding ϕ for multi-tier circuits, as compared with Monte Carlo simulations.

As the effect of the noise phase can be significant both for setup and hold skitter, several techniques, such as RC filtered buffers and “stacked” phase-shifted buffers [663], have been proposed to shift the ϕ seen by the clock paths. In 3-D clock distribution networks, these techniques can be applied to a portion of the clock paths in a different tier to increase Δϕ among the tiers. The change in σ_J1,2 versus the shifted (ϕ₁=ϕ₂) for distribution (A) is shown in Figs. 17.24A and B. As shown in Fig. 17.24B, the dashed line depicts σ_J1,2 for ϕ₁=ϕ₂, which denotes the skitter without phase shifting. As shown by the arrow, the highest σ_J1,2 decreases with Δϕ=ϕ₂−ϕ₁. In this case, since ϕ₂ and ϕ₁ are not simultaneously equal to 270°, the worst case μ_J1,2 also decreases.

In Fig. 17.24C, however, σ_J1,2 of distribution (B) depends strongly on ϕ₂. This behavior occurs since σ_J1,2 is dominated by the supply noise in the second tier. In this case, shifting ϕ among tiers provides less than a 1.5 ps decrease in σ_J1,2, as shown by the dashed line with arrows. Thus, for equally distributed clock paths across 3-D ICs, the worst case skitter can be decreased by shifting ϕ among those tiers with phase-shifted clock distribution networks.

Note that the proper Δϕ is determined by traversing all of the combinations of ϕ in different tiers. The number of combinations increases exponentially with the number of tiers, requiring a large number of simulations. This unified model provides a highly efficient way to determine a valid shift in ϕ to decrease skitter in multi-tier circuits.

In addition to phase. the frequency of the power supply noise also affects the skitter. This frequency is usually considered to be the same among tiers, as shown in Figs. 17.14 and 17.15. The frequency f_n is varied to evaluate the change in skitter with the frequency of the supply noise. The amplitude V_n and phase ϕ are assumed to be the same among tiers, where V_n1=V_n2=90 mV and ϕ₁=ϕ₂=270°. The simulation results are illustrated in Fig. 17.25.

Similar to the effects of V_n, f_n greatly affects μ_J1,2. For instance, μ_J1,2 increases with f_n by up to 70% for distribution (B). The variation of skitter, however, decreases with f_n. The resulting Δσ_J1,2 and Δσ_S1,2 reach 15% for both distributions (A) and (B). This behavior is due to the lower voltage seen by the clock buffers when the clock propagates along the path. The change of μ_d and σ_d for the delay of two serial inverters (a clock buffer) is illustrated in Fig. 17.26A. Both μ_d and σ_d decrease with V_dd. As shown in Fig. 17.26A, assume that the clock edge with the worst case σ_J arrives at the input of the clock path at t₀. When f_n increases from f_n1 to f_n2, the propagation time of this edge decreases from t₁ to t₂ and the supply voltage within this duration increases. This higher supply voltage introduces lower σ in the buffer delay, which lowers σ_J1,2 and σ_S1,2, see (17.65) and (17.70). Consequently, the mean setup skitter increases significantly with the frequency of the power noise, while both σ_J1,2 and σ_S1,2 decrease with this frequency.

As shown in Fig. 17.25, the statistical model for skitter is reasonably accurate as compared with SPICE-based simulations for different frequencies of the power noise. For the worst case μ_J1,2 (σ_J1,2) shown in Fig. 17.25, the error is, respectively, −11% (−12%), −7% (−10%), −8% (−4%), and −10% (−9%). Since σ_J1,2 varies with the power noise, process variations and power noise need to be simultaneously modeled to correctly characterize the clock delay uncertainty. The difference in mean skitter varies by up to 60% due to the different V_n among tiers. σ_J1,2 can vary by up to 51% due to the different ϕ (see Figs. 17.23B and C). Decreasing the variations as well as the mean skitter therefore improves the robustness of 3-D clock distribution networks.

17.3.4 Effect of Skitter on Synthesized Clock Trees

A set of guidelines is provided to support the design of robust 3-D clock distribution networks. The objective of these guidelines is to decrease skitter in 3-D circuits.

To illustrate the utility of these guidelines, several examples of synthesized 3-D clock trees are discussed here. The 3-D circuits are evaluated on IBM benchmark clock circuits [664], generated by randomly distributing the clock sinks to different tiers [656]. The 3-D clock trees are synthesized based on the means-and-medians (MMM)-TB algorithm [593] (see Chapter 15, Synchronization in Three-Dimensional ICs). The buffers are inserted assuming a constraint of 50 fF on the capacitive load. Each clock buffer is formed by an inverter (W_n=4.83 μm W_p=2.1 W_n). An example of the resulting three tier clock trees for the “r1” benchmark circuit (267 sinks) is illustrated in Fig. 17.27A. The clock source, clock sinks, and TSVs are denoted, respectively, by the triangle “,” cross “×,” and dot “” markers. The clock networks in tiers 1, 2, and 3 are denoted, respectively, in blue (black in print), red (dark grey in print), and green (light grey in print).

The skitter is measured within two different regions, as illustrated in Fig. 17.27C. For both regions, A₁ and A₂, the skitter is reported between the pair of the farthest sinks. The three largest IBM benchmark circuits r3, r4, and r5, are evaluated. SPICE simulations are performed on the paths of interest with 2,000 Monte Carlo simulations. The primary features of these benchmarks are listed in Table 17.6, where the computational time is also listed. Note that the simulation time is only for the selected clock paths, not for the entire clock tree. The initial phase and frequency of the power supply noise are assumed to be the same among the three tiers (f_n1=f_n2=f_n3=400 MHz). The amplitude V_n is assumed to differ among the tiers (V_n1=0.09 volts, V_n2=0.08 volts, and V_n3=0.065 volts).

The skitter is reported in Table 17.7. The highest mean skitter occurs when ϕ₁=ϕ₂=ϕ₃=270°, and the highest σ is reported for ϕ₁=ϕ₂=ϕ₃=200°. Four design practices are compared with each other.

In C1 and C2, most of the clock buffers are placed in tier 1, adjacent to the heat sink, to constrain the increase in temperature. In C3, the majority of the clock tree is placed in the middle tier to decrease both the number of TSVs and the power consumption, as suggested in [593]. In C4, based on Guideline 1, the majority of the clock tree is located in tier 3 (with the lowest V_n), as illustrated in Fig. 17.27B. As listed in Table 17.7, μ_J1,2 in Case 1 is similar to Case 2. Nevertheless, σ_J1,2 and σ_S1,2 are significantly underestimated in Case 1 for both regions A₁ and A₂. As compared to Case 2, the difference in σ_J1,2 and σ_S1,2 reaches 36%. This difference demonstrates the necessity for simultaneously modeling process variations and power noise, since separately modeling process variations and power noise significantly underestimates the variations due to skitter.

The difference between the analytic model and SPICE-based Monte Carlo simulations is listed in the error column of Table 17.7. For all σ_J1,2 and σ_S1,2, the error of the proposed model is below 10%, as compared to Monte Carlo simulations. The error in μ is below 7% for J_1,2. Considering the greater than 129× speedup in computational time, as reported in Table 17.6, this analytic model provides an efficient way to accurately estimate skitter.

In Case 2, the majority of the clock distribution network is placed in the tier adjacent to the heat sink. In Case 3, the majority of the clock distribution network is placed in the middle tier to reduce the number of TSVs and dissipate less power [593].

The number of TSVs and power consumption of the entire tree for Cases 2 to 4 are illustrated in Fig. 17.28. The results are normalized with respect to Case 4. As proposed in [593], Case 3 produces the fewest TSVs (see #TSV(C2/C4) and #TSV(C3/C4) in Fig. 17.28). The total power is similar among the three cases due to the similar number of clock buffers, as shown by power (C2/C4) and power (C3/C4) in Fig. 17.28. The power per tier, however, differs due to the different distribution of buffers among the tiers.

Case 4 mitigates the mean skitter, lowering the number of TSVs and the power consumed by the tiers. As illustrated in Figs. 17.14 and 17.15, the tier next to the package exhibits the lowest V_n. Consequently, μ_J1,2 of Case 4 is significantly improved as compared to Cases 2 and 3, as shown by the first three rows of Impr1 and Impr2 in Table 17.7. This improvement ranges from 16% to 31%. This comparison illustrates the efficiency of Guideline 1 in decreasing the mean skitter. For several paths, however, σ_J1,2 and σ_S1,2 in Case 4 increase as compared to Cases 2 and 3. This situation is due to the change in the topology of the clock trees. For example, for the pair of paths in A2 and circuit r5, the number of buffers after the merging point of these paths increases as compared to Case 2. These buffers are located in different tiers. Consequently, σ_J1,2 and σ_S1,2 both increase.