Ultrahigh-Performance Radio Frequency System-on-Chip Implementation of a Kalman Filter-Based High-Precision Time and Frequency Synchronization for Networked Integrated Sensing and Communication Systems

The integration of radar sensing and imaging capabilities into future integrated sensing and communication (ISAC) networks enables advanced use cases, including autonomous vehicle navigation, real-time health monitoring, and smart city management. However, ultraprecise time and frequency synchronization is crucial for unlocking the full potential of such networked ISAC systems. In this article, a novel real-time wireless time and frequency synchronization scheme is developed and fully implemented on a high-end radio frequency system-on-chip field-programmable gate array (FPGA) platform. The excellent performance and robustness of the proposed solution in practical applications are demonstrated. It is evidenced that the recursive nature of the Kalman filter is well suited to the dynamic capabilities of FPGA-based simultaneous synchronization. Observed values obtained through the precision time protocol (PTP) are iteratively refined, thus effectively compensating for uncertainties encountered during a synchronization packet exchange. Due to the deterministic processing time inherent in the FPGA, the proposed synchronization method achieves exceptional precision, with clock offset deviations in the nanosecond range and clock rate deviations limited to only a few parts per billion, even across considerable distances between the network nodes.

SECTION I. Introduction

Synchronization of distributed sensors is a fundamental technological challenge in wireless sensor networks (WSNs) [1] and integrated sensing and communication systems (ISAC) [2], particularly in scenarios involving spatially distributed and cooperatively operating networked units. The inherent nature of distributed systems complicates the deployment of coherent system clocks, posing significant challenges to network efficiency and reliability. Achieving a unified and coherent system clock requires high-precision time and frequency synchronization, which is indispensable for the real-time operation of distributed wireless systems [3]. In radar networks, clock-synchronized systems significantly enhance the accuracy of target localization and imaging [4]. The potential for high-resolution radar imaging over extensive surveillance areas with frequent observation intervals can be achieved through autonomous aerial vehicle (AAV) swarms for which precise synchronization is essential for transforming this vision into reality [5]. Devices, such as base stations, AAVs, and cell phones, serve as sensors, providing pervasive sensing capabilities to support a wide range of intelligent applications in ISAC systems. The sensing performance in these systems is significantly influenced by time and frequency synchronization accuracy, particularly in future implementations where high sensing precision is crucial [6]. As illustrated in Fig. 1, multi-AAV-enabled ISAC networks enhance sensing and communication performance, supporting applications, such as localization, traffic management, and collision avoidance. This synchronized network facilitates real-time, coordinated interactions across diverse smart city components. However, this improvement is accompanied by increased system complexity and a stringent requirement for precise synchronization, which is crucial for maintaining effective coordination and overall system efficiency [7]. In orthogonal frequency-division multiplexing (OFDM)-based radio interfaces, precise synchronization of time and frequency among multiple users is critical to ensuring the integrity and reliability of data transmission [8]. Moreover, in contemporary computer and control networks, accurate synchronization is indispensable for preserving the safety and reliability of time-sensitive information [9]. However, achieving and maintaining precise clock synchronization in distributed wireless networks is particularly challenging in scenarios where access to the global positioning system (GPS) is restricted or unreliable, and atomic oscillators are unavailable [10]. Several factors exacerbate this challenge—including unintended arbitrary deviation of the oscillator’s nominal frequency, nonlinear frequency deviation induced by environmental influences (such as temperature, humidity, vibration, and aging), as well as noise-induced random deviations [11]. These factors introduce clock distortion in distributed communication and sensor networks and, thus, complicate the attainment of stable and consistent synchronization across the entire network. Consequently, it is imperative to develop and implement effective solutions capable of accurately estimating and compensating for these obstacles to ensure reliable and consistent clock synchronization in distributed communication and sensor networks.

FIGURE 1.

Illustration of AAV-enabled ISAC networks with synchronization, localization, and radar imaging capability.

Energy consumption is a critical factor in clock synchronization for WSNs. In WSNs, clock synchronization is primarily achieved by exchanging timestamps between nodes through one-way or two-way time transfer protocols. One-way protocols, such as reference broadcast synchronization (RBS) and the flooding time synchronization protocol (FTSP), utilize single broadcasts, enabling simpler implementation and lower energy consumption. However, these protocols offer reduced precision due to unaccounted transmission delays. In contrast, two-way time transfer protocols, such as the precision time protocol (PTP), the timing-sync protocol for sensor networks (TPSNs), and lightweight time synchronization (LTS), enhance accuracy by compensating for transmission delays, though they typically demand more energy [12]. In two-way centralized synchronization mechanisms, all nodes synchronize directly to a single reference node, ensuring precise alignment across the entire system. The main advantage of centralized protocols is their ability to eliminate the need for coarse time alignment across the network. However, reliance on this single reference node introduces vulnerabilities, particularly if it becomes unreliable. In contrast, decentralized approaches require local nodes to delay clock adjustments until clock parameters are precisely estimated and the synchronization communication phase is complete. Premature readjustments in decentralized systems can lead to significant synchronization errors, making a dedicated time window for readjustment necessary—and thus coarse time alignment becomes essential. While a decentralized mechanism is generally more stable than a centralized one, network latency and jitter in packet exchanges between multiple nodes introduce cumulative uncertainties, ultimately reducing overall synchronization accuracy.

In centralized approach, the IEEE 1588 PTP protocol facilitates network nodes in synchronizing their clocks with a reference clock to achieve highly accurate timing and enables the tracking of time variations through the exchange of time information [13]. Therefore, by employing the PTP protocol along with a well-designed frequency offset estimation mechanism, real-time measurements of time and frequency offset among nodes are provided. This mechanism enhances the integrity and efficiency of WSN systems by aligning the local clocks of all nodes to the reference clock of one specific node that serves as the reference node [14]. Synchronization resilience can be improved by designating an additional node as a backup reference node, thereby ensuring continuous synchronization in the event of unforeseen issues. All nodes transmit time synchronization messages to the reference node, adjusting their clocks to match the reference time. This reference time then becomes the global network time, thus establishing a unified time standard across the network. An accurately synchronized clock ensures the integrity and efficiency of pervasive computing, communication, and localization crucial for maintaining stringent standards in complex network systems.

The PTP protocol’s susceptibility to network latency and jitter is a significant disadvantage. These factors can introduce inaccuracies in the synchronization process by causing delays or variations in the timing of packet exchanges between nodes, thereby making precise synchronization challenging, particularly in networks with fluctuating or unpredictable traffic patterns. This limitation emphasizes the need for careful network management and optimization to mitigate the effects of latency and jitter on synchronization accuracy [15].

A promising approach to enhancing time and frequency synchronization involves employing a Kalman filter-based technique in coordination with PTP, which results in significant improvements. The Kalman filter accurately estimates system state variables in real-time by utilizing observed measurements over time. The process entails a two-way message exchange for time offset estimation, where each round includes a synchronization message sent by the node to be synchronized and a reply message returned by the reference node. This consecutive exchange, combined with continuous monitoring of time stamps across all nodes, ensures precise timing tracking and improves the accuracy of observational analysis. Ultimately, high-precision clock synchronization is achieved through multiple rounds of timestamp message exchanges and the Kalman filter, which fuses measurements with a system model to accurately estimate the state of the dynamic system in real time.

In [16], the frequency offset and time offset between a client and a remote server are calculated using NTP exchange over the network, employing three distinct algorithms: 1) Kalman filter; 2) linear programming; and 3) averaging method. Among these, the Kalman filter demonstrates superior performance in scenarios characterized by Gaussian noise, while linear programming exhibits better results in the presence of bursty traffic, particularly in self-similar network conditions. Consecutive measurements are independent, and performance improves with an increasing number of packets. The empirical results are presented with time offset estimation in the millisecond range and frequency offset estimation in the parts-per-million (ppm) range.

In [17], packet networks that utilize the IEEE 1588 v2 PTP standard for frequency distribution experience a decline in precision primarily due to packet delay variation along the transmission path. To address this issue, the study proposes a modified Kalman filter technique for clock skew estimation and investigates the impact of windowing on system performance. Simulation operates on experimentally measured data rather than performing real-time synchronization. This method is demonstrated on a one-way communication path and achieves performance that is superior to 16 parts per billion (ppb) on telecom networks.

In [18], the method addresses resource-constrained networks characterized by limited energy, unstable processors, and unreliable low-bandwidth communication. It offers adaptability with synchronization intervals and provides probabilistic performance and demonstrates efficiency through simulation. This method assumes a time-varying clock skew and focuses on modeling a clock. The Kalman filter is specifically designed to track clock uncertainty as well as variations in clock skew and offset, thereby facilitating clock synchronization within such networks.

In [19], simulations are conducted to analyze the impact of local clock instability, time exchange interval rate, and precision of correction estimates on the design of a PTP scheme. It investigates how the synchronization message rate influences synchronization capability and clock instability. Additionally, this article presents a state-variable clock model that enables realistic parameterization based on experimental measurements of Allan variance plots for various types of clocks. It incorporates a Kalman filter in this model and evaluates its performance. With a timestamping uncertainty of $10\,\mathrm {ns}$ , the observed standard deviations for clock offset and skew are $220\,\mathrm {ns}$ and $60\,\mathrm {ppb}$ , respectively.

The event-based Kalman filter proposed by [20] presents a novel approach to clock synchronization, simulating a servo clock with an efficiently implemented Kalman filter. The sensitivity of the Kalman filter to specific parameters is analyzed, and with careful initialization, this approach achieves reliable estimation of synchronization uncertainty and precise synchronized timestamping. Additionally, it demonstrates improved stability in managing jitter.

In [14], we demonstrate a field-programmable gate array (FPGA)-based real-time synchronization scheme for wireless networks, utilizing PTP for time offset estimation and compensation. An adjustable crystal oscillator is employed for precise frequency tuning, and when the frequencies are aligned, the time offset deviation slightly exceeds $\pm 20\,\mathrm {ns}$ .

This article enhances synchronization by integrating an optimized Kalman filter, tailored for precise estimation of time offset and frequency skew on the target hardware platform. This integration significantly improves time and frequency synchronization, thereby delivering unprecedented synchronization accuracy in the experimental study. This exceptional performance is particularly noteworthy given the limited experimental work in this field. Table 1 presents a concise comparison of selected research findings, highlighting the superior precision in our approach compared to the state of the art.

TABLE 1 Comparison of Time and Frequency Synchronization Precision Presented in the Literature

The remainder of this article is organized as follows. Section II models random phase and frequency fluctuations in distributed system clocks. Section III presents an analysis of Kalman filter-based synchronization. Section IV provides a comprehensive description of the implemented FPGA design. Section V outlines the practical design approach for the entire scheme and discusses the results. Finally, Section VI concludes this article with a summary of the findings and implications of the study.

SECTION II. Clock Model

The oscillators employed in WSNs lead to nondeterministic behavior and stochastic variations in both time and frequency, which contribute to clock drift relative to true time. Modeling the random phase and frequency fluctuations inherent to these oscillators is crucial for designing optimal combining algorithms. These algorithms generate uniform time scales and provide a stable, accurate reference frequency, which is essential for complex systems that require high-precision synchronization. In this approach, the primary oscillator is assumed to be the reference oscillator with a reference frequency, $f_{0}$ , while the secondary oscillators are modeled relative to the primary one.

The instantaneous value of the electronic clock signal at time t on secondary nodes can be modeled as given below [11], [21] $$\begin{equation*} V(t) = V_{0} \sin \left ({{ 2\pi f_{0} t + \varphi (t) }}\right ). \tag {1}\end{equation*}$$ $V_{0}$ is the nominal amplitude of the clock signal, $f_{0}$ represents the frequency of the reference node, and $\varphi (t)$ denotes the random phase fluctuations between the secondary and reference nodes. The equation $$\begin{align*} f(t)=& \frac {1}{2\pi } \frac {\mathrm {d}}{\mathrm {d}t} \left ({{ 2\pi f_{0} t + \varphi (t) }}\right ) = f_{0} + \frac {1}{2\pi } \frac {{\mathrm {d}}\varphi (t)}{\mathrm {d}t} \\=& f_{0} + \Delta f(t) \tag {2}\end{align*}$$ shows that the instantaneous frequency $f(t)$ consists of the reference frequency $f_{0}$ and the deviation $$\begin{equation*} \Delta f(t) = \frac {1}{2\pi } \frac {{\mathrm {d}}\varphi (t)}{\mathrm {d}t} \tag {3}\end{equation*}$$ which results from the time derivative of the phase fluctuations. In the following equation, the dimensionless measure $$\begin{equation*} \gamma (t) = \frac {\Delta f(t)}{f_{0}} = \frac {1}{2\pi f_{0}} \frac {{\mathrm {d}}\varphi (t)}{\mathrm {d}t} \tag {4}\end{equation*}$$ is defined as the ratio of the frequency deviation $\Delta f(t)$ to the reference frequency $f_{0}$ .

The time deviation of a clock is expressed as a function of the phase deviation $$\begin{equation*} \theta (t) = \frac {\varphi (t)}{2\pi f_{0}}. \tag {5}\end{equation*}$$

The final equation ties these concepts together, thereby revealing that the dimensionless frequency deviation $\gamma (t)$ is the derivative of the time deviation $\theta (t)$ and illustrates that the rate of change of time deviation (how fast the clock is drifting) directly influences the observed frequency deviation $$\begin{equation*} \gamma (t) = \frac {{\mathrm {d}}\theta (t)}{\mathrm {d}t}. \tag {6}\end{equation*}$$

The clock offset indicates how much the clock has deviated from the true or reference time. The clock offset $\theta (t)$ at any given time t can be expressed as [22] $$\begin{equation*} \theta (t) = \int _{0}^{t} \gamma \left ({{\tau }}\right ) \, {\mathrm {d}}\tau + \theta _{0} + w(t). \tag {7}\end{equation*}$$ This equation demonstrates that the time deviation $\theta (t)$ is obtained by integrating the frequency deviation $\gamma (\tau)$ over time, beginning from an initial time deviation $\theta _{0}$ , and accounting for additive white Gaussian noise (AWGN) $w(t)$ .

The discrete-time model of oscillators provides the framework for describing the evolution of phase and frequency over time. In discrete time, the clock offset $\theta _{k}$ is expressed as [22] $$\begin{equation*} \theta _{k} = \sum _{n=1}^{k} \gamma _{n} \Delta \tau _{n} + \theta _{0} + w_{k} \tag {8}\end{equation*}$$ where k represents the current time index and n is the summation index that runs from 1 to k. Then, the following recursive expression is obtained: $$\begin{equation*} \theta _{k} = \theta _{k - 1} + \gamma _{k} \Delta \tau _{k} + w_{k} - w_{k-1}. \tag {9}\end{equation*}$$ In the equation, $\Delta \tau _{k}$ is the constant sampling interval between two synchronization messages and is denoted by $T_{\mathrm {s}}$ and the term $w_{k} - w_{k-1}$ has been substituted with the newly defined term $w_{\theta ,k}$ $$\begin{equation*} \theta _{k} = \theta _{k - 1} + \gamma _{k} T_{\mathrm {s}} + w_{\theta ,k}. \tag {10}\end{equation*}$$

By taking into account the nearly constant skew in the sampling interval, the frequency deviation $\gamma _{k}$ can be modeled as follows [23], [24]: $$\begin{equation*} \gamma _{k} = \gamma _{k-1} + w_{\gamma ,k}. \tag {11}\end{equation*}$$ $w_{\theta }$ and $w_{\gamma }$ are two uncorrelated Gaussian-distributed, mean-free white noise.

These recursive equations model the evolution of clock offset and skew over discrete time steps in systems in which the clocks are susceptible to drift due to stochastic variations. Observations are taken at almost equally spaced time intervals, and discrete-time recursion is generated at these observation points, thus enabling systematic updates of state estimates based on new measurements. By developing a recursive state-variable clock model, it becomes logical to consider implementing a recursive estimator based on the Kalman filter for clock synchronization. The recursive nature of this equation aligns seamlessly with the iterative process of the Kalman filter, which leverages such recursions to provide real-time state estimation. The Kalman filter utilizes this structure to predict and correct the state estimate, effectively managing uncertainties and noise in the system, thereby enhancing synchronization accuracy in applications in which precise timing is critical.

SECTION III. Kalman Filter-Based Synchronization

This article employs the Kalman filter to synchronize time and frequency among wireless communication nodes, which is essential for accurate data integration and fusion in sensor network applications. The Kalman filter serves as a technique to fuse measurements with a system model and estimate the state of the dynamic system in real time. In a statistically optimal manner, the Kalman filter is a recursive algorithm designed to estimate the state of a linear dynamic system from a sequence of uncertain measurements. It provides an optimal estimate when the measurement and process noise are modeled as AWGN. The advantage of the Kalman filter lies in its capability to iteratively update an estimate of an evolving state, enhancing the accuracy, consistency, and adaptability of dynamic system estimation. It provides a probabilistic estimate of the system state and updates it in real time through a two-step process that involves prediction and correction. These steps elucidate how the Kalman filter combines predictions from the system model with measurements to yield optimal estimates of the system state [25].

The Kalman filter framework characterizes the initial, predicted, and final corrected states as Gaussian-distributed random variables, each defined by their respective means and covariances. The representation of the system state using only the mean and covariance significantly enhances computational efficiency, thereby providing a key advantage for real-time processing applications. At each time step, the Kalman filter estimates the system’s state by integrating information from the initial probabilistic estimate, the predicted state based on a dynamic model, and the correction derived from new observations that consider the uncertainty inherent in both the model and measurement. The filter operates in a recursive manner, where the initial estimate is updated through prediction and correction. During the prediction phase, the filter utilizes the system model to forecast the state and its associated uncertainty, which is captured by the predicted state mean and covariance. Subsequently, the correction phase incorporates measurements subject to noise and inaccuracies to adjust and refine the predicted state [25].

The primary objective is to accurately estimate and correct the time offset and frequency skew of secondary nodes relative to the primary reference node using a 2-D linear system model. The process begins from the initial probabilistic estimate at time $t_{k-1}$ , and a time offset estimation model predicts the state at the next time step. This prediction is then updated and corrected using observations derived from PTP measurements at time $t_{k}$ . The real-time tracking of time offset via the PTP protocol may exhibit inaccuracies; however, by integrating PTP measurements with the predictive phase of the Kalman filter, the system can anticipate and compensate for variations in the time offset, thus ensuring more precise and reliable synchronization between nodes. This method enables dynamic adjustment and optimization of the synchronization process, thereby enhancing its performance in time-sensitive applications.

The mathematical model for synchronization using the Kalman filter can be described in the following approach. The estimation model at $t_{k}$ is a linear combination of the estimate, control input, and zero-mean process noise vector, all at the previous time step $t_{k-1}$ $$\begin{equation*} \boldsymbol {x}_{k}=\boldsymbol {F}_{k-1} \boldsymbol {x}_{k-1} + \boldsymbol {B}_{k-1} \boldsymbol {u}_{k-1} + \boldsymbol {w}_{k-1} \tag {12}\end{equation*}$$ where $\boldsymbol {x}_{k}$ denotes the state vector at time k, $\boldsymbol {F}_{k-1}$ is the state transition matrix, $\boldsymbol {B}_{k-1}$ represents the control input matrix, $\boldsymbol {u}_{k-1}$ is the control input vector, and $\boldsymbol {w}_{k-1}$ is the process noise vector [25], [26] $$\begin{align*} F_{k-1}=& \begin{bmatrix} 1 & T_{\mathrm {s}} \\ 0 & 1\end{bmatrix}, \quad x_{k-1} = \begin{bmatrix} \theta _{k-1} \\ \gamma _{k-1}\end{bmatrix} \\ \\ B_{k-1}=& \begin{bmatrix} -1 & -T_{\mathrm {s}} \\ 0 & -1\end{bmatrix}, \quad u_{k-1} = \begin{bmatrix} u_{\theta ,k-1} \\ u_{\gamma ,k-1}\end{bmatrix}. \tag {13}\end{align*}$$ The input, $u_{k-1}$ , represents the external corrections applied to the clock model of the secondary node at each time interval. When the clock offset is positive, indicating that the timestamp of the secondary node is ahead of that of the primary node, it is necessary to decrease this offset to achieve clock alignment between the nodes. To accomplish this, a correction that is the inverse of the current time offset is applied. Consequently, the $\boldsymbol {B}_{k-1}$ matrix, which defines the relationship between the control input and the system, is negative to reflect this inverse relationship. The system model used for synchronization in this article is $$\begin{align*} \theta _{k}=& \left ({{\theta _{k-1} - u_{\theta ,{k-1}}}}\right ) + T_{\mathrm {s}} \left ({{\gamma _{k-1} - u_{\gamma ,{k-1}}}}\right ) \text {and} \tag {14}\\ \gamma _{k}=& \gamma _{k-1} - u_{\gamma ,{k-1}}. \tag {15}\end{align*}$$ These equations can be expressed in a compact matrix-vector form, as given below $$\begin{align*} \begin{bmatrix} \theta _{k} \\ \gamma _{k} \end{bmatrix} = \begin{bmatrix} 1 & T_{\mathrm {s}} \\ 0 & 1 \end{bmatrix} \begin{bmatrix} \theta _{k-1} \\ \gamma _{k-1} \end{bmatrix} + \begin{bmatrix} -1 & -T_{\mathrm {s}} \\ 0 & -1 \end{bmatrix} \begin{bmatrix} u_{\theta ,{k-1}} \\ u_{\gamma ,{k-1}} \end{bmatrix}. \tag {16}\end{align*}$$

The linear measurement model, as defined by [25] and [26], is expressed as $$\begin{align*} y_{k}=& H_{k}x_{k} + v_{k} \\ H_{k}=& \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix}. \tag {17}\end{align*}$$ $H_{k}$ is the identity matrix, $x_{k}$ is the true state vector, and $v_{k}$ is the observation noise vector.

The prediction equations, as expressed by [25] and [26], are given by $$\begin{align*} \check {x}_{k}=& F_{k-1}\hat {x}_{k-1} + B_{k-1}u_{k-1}~\text {and} \tag {18}\\ \check {P}_{k}=& F_{k-1}\hat {P}_{k-1}F^{T}_{k-1} + Q_{k-1} \tag {19}\end{align*}$$ where $x_{k}$ is the state estimate, $P_{k}$ is the error covariance matrix, and $Q_{k-1}$ is the process noise covariance.

The Kalman gain, as defined by [25] and [26], is expressed as $$\begin{equation*} K_{k}=\check {P}_{k}H^{T}_{k}\left ({{H_{k}\check {P}_{k}H^{T}_{k}+R_{k}}}\right )^{-1} \tag {20}\end{equation*}$$ where $R_{k}$ represents the measurement noise covariance, which indicates the reliability of the sensor. Finally, the posteriori state estimate is a combination of the a priori state estimate and the measured data, and is given by the following equation [25], [26]: $$\begin{align*} \hat {x}_{k}=& \check {x}_{k} + K_{k}\left ({{y_{k}-H_{k}\check {x}_{k}}}\right ) \tag {21}\\ \hat {P}_{k}=& \left ({{1-K_{k}H_{k}}}\right )\check {P}_{k}. \tag {22}\end{align*}$$

$\check {x}_{k}$ , $\check {P}_{k}$ : priori/$\hat {x}_{k}$ , $\hat {P}_{k}$ : posteriori.

After obtaining the posteriori state estimate from the Kalman filter, the timestamp at the secondary node is updated accordingly.

This iterative process ensures continuous estimation of both time offset and frequency skew. The Kalman filter utilizes these updates to minimize the variance in the time offsets, thereby optimizing the synchronization process. By applying the Kalman-corrected time offset to the timestamp, rather than relying solely on raw PTP measurements, the precision of synchronization is significantly enhanced. An important aspect is that the Kalman filter enables high-precision frequency skew estimation without requiring a determined skew input.

SECTION IV. Design Overview

A. FPGA Implementation

In this article, three sets of Zynq UltraScale+ RFSoC FPGA boards are utilized to orchestrate a real-time FPGA-based implementation for synchronization in wireless communication networks. Within this configuration, one of the boards serves as the primary reference node that provides a reference clock for the entire network, while the remaining two boards function as secondary nodes that actively participate in the synchronization process through the exchange of timestamps with the primary node. In this framework presented in Fig. 2, secondary nodes take the lead in initiating synchronization by transmitting timestamps encapsulated within sync messages to the reference node. These timestamps are encoded with Reed-Solomon encoder to rectify potential burst errors in communication links [27]. In each sync message, a pattern of known bits (preamble) is included with the encoded timestamps to mark the beginning of each frame. This preamble acts as a distinctive identifier, ensuring accurate frame delineation within the communication system [28]. The frames are exchanged between nodes using quadrature phase-shift keying (QPSK) modulation, where binary data is combined with a carrier signal generated by the transmitter’s local oscillator. This modulation technique involves manipulating the phase of the carrier signal to encode the binary data, thereby enabling efficient data transmission and reception between nodes [29]. On-chip digital-to-analog converters (DACs) and RF data converters facilitate data exchange among nodes on distinct frequency slots and employ frequency-division multiple access (FDMA) to effectively utilize the spectrum. The FDMA approach is a widely employed technique in wireless communication systems. This methodology aims to facilitate simultaneous transmissions and reduce interference among nodes. To achieve this, each node transmits on distinct frequency slots determined by their individual identification, thereby ensuring that transmissions occur concurrently. The system is designed to accommodate additional nodes, which allows them to freely join or leave the network without disrupting overall functionality.

FIGURE 2.

Overview of the FPGA design implementation scheme for the delivery of synchronization packet flow based on the PTP protocol using the developed Kalman filter.

In the hardware implementation of the receiver, a finite impulse response (FIR) filter and automatic gain control (AGC) are utilized. These components serve critical roles in noise cancellation, signal modification, and amplification. The FIR filter selectively attenuates unwanted frequency components, while AGC dynamically adjusts signal gain to maintain optimal levels, collectively ensuring signal integrity and quality [30]. In WSNs, independent wireless units have individual oscillators with slight frequency variations. Thus, achieving zero carrier frequency offset between the receiver and transmitter is crucial. A Costas loop clock recovery mechanism is utilized to compensate for nonzero frequency offset. The Costas loop extracts phase information from the received signal by employing a numerically controlled oscillator (NCO) and modifies the phase offset. It generates a nominal frequency that closely matches the frequency range of the incoming signal. The Costas loop ensures precise demodulation despite frequency discrepancies [31]. Following demodulation, the frame synchronizer block meticulously aligns the sync frame with a predetermined fixed binary pattern (preamble) to precisely discern the frame’s inception. This process enables the extraction of timestamp information and effectively mitigates phase ambiguity, thus ensuring optimal synchronization within the system [32]. In the pursuit of enhancing reliability in the extraction of timestamp information, the Reed-Solomon decoder serves as a block-based error detection and recovery algorithm. Upon extracting the timestamp information in this state, the secondary node employs the PTP engine to compute the time offset relative to the reference node. The computed time offset, derived from each iteration of the time exchange process, is immediately processed by the Kalman filter to achieve clock unification across the system. The system model underlying the Kalman filter is thoroughly described in Section III. The process encompasses both offset and skew estimation and correction to ensure accurate time and frequency alignment with respect to the reference node, as detected in Fig. 3. Time offset correction is performed by immediately adjusting the timestamp once in each synchronization round. In contrast, skew correction, which involves adjusting the frequency of the DCTCXO crystal oscillator, is applied after the analysis of every 4000 measurement results. Since programming the DCTCXO during each synchronization interval is not practical, real-time skew estimation is implemented with the skew compensation parameter $u_{\gamma ,{k-1}}$ fixed at zero. This approach mitigates the risk of the phase-locked loop (PLL) losing lock, thereby ensuring stable and precise frequency control.

FIGURE 3.

Illustration of the offset and skew of the secondary node’s local clock relative to the primary node’s local clock, based on an optimized Kalman filter.

B. Timeline of Sync Message in Communication Link

Fig. 4 illustrates the structure of synchronization messages, which are transmitted every $54.613\,\mathrm {ms}$ . Each sync message occupies $0.426\,\mathrm {ms}$ —equivalent to $1/128$ of the transmission interval—and comprises 512 bits divided into eight subframes of 64 bits each. Every frame commences with a 64-bit preamble that serves as the frame header. Employing unique and orthogonal preamble codes for each secondary node effectively mitigates cross-channel interference and enhances signal detection. This strategy ensures the distinctiveness of each node’s transmission, thereby reducing data transmission errors. In a time information exchange process, the subframes of the sync messages record timestamp information from each round of PTP exchange, thus enabling the alignment of the time of the secondary node with that of the primary node. Fig. 5 illustrates the time alignment of transmission (TX) events between the secondary and primary nodes, occurring at $0.426\,\mathrm {ms}$ intervals, with potential deviations of one timestamp due to Kalman filter-based synchronization. The time delay between the TX event of one node and reception (RX) at the other, primarily depends on their separation distance. Given the propagation speed of electromagnetic waves in the air, this delay is typically on the order of a few nanoseconds for short-range applications. Timeliness in real-time systems guarantees that results are delivered within specified deadlines. This is achieved through precise synchronization and efficient task scheduling. As shown in Figs. 4 and 5, synchronization messages are transmitted every 54.613 ms, with the process completed within 1 ms, ensuring timely coordination. The remaining communication interval of $54.613\,\mathrm {ms}$ is allocated for additional data transmission, enabling parallel communication alongside synchronization.

FIGURE 4.

Signal structure of the sync messages in the communication link.

FIGURE 5.

Timing diagram of synchronization message exchange between primary and secondary nodes, with Kalman filter processing, all completing within 1 ms.

For synchronization messages to be transmitted between secondary and primary nodes, their bit-streams must be modulated onto a carrier wave. In QPSK modulation, the binary data stream is divided into pairs of bits, with each pair corresponding to one of four possible phase shifts in the carrier wave. QPSK modulation employs two orthogonal carrier signals—the in-phase (I) and quadrature (Q) components—generated by an I/Q modulator. This technique allows efficient data transmission by encoding two bits per symbol through distinct phase shifts, as illustrated in Fig. 4.

C. IEEE 1588 PTP-Based Methodology

In this study, a technique similar to that used in IEEE 1588 PTP [13], which is designed for computer networks, is employed. The IEEE 1588 PTP provides a mechanism for estimating the offset of a local clock relative to a reference clock in networks. The time offset is determined by transmitting synchronization messages and receiving timing information. Each node calculates the time difference, $\theta$ , relative to the primary reference node and subsequently adjusts its local clock based on the timing information received from the reference clock.

As illustrated in Fig. 6, the secondary node transmits its local time $T_{1}(t_{1})$ to the primary reference node within the allocated frequency band to initiate synchronization. Then, the primary node receives these messages and records the reception time of the synchronization messages based on its local time $T_{2}(t_{ref})$ . After processing the information, the primary node sends timestamps at $T_{3}(t_{ref})$ back to the secondary node. The secondary node then records the reception time of the synchronization feedback as $T_{4}(t_{1})$ and subsequently computes the time offset. In this protocol, the time offset $\theta$ and time of flight $\tau _{\mathrm {ToF}}$ are determined mathematically as follows: $$\begin{align*} \theta =& \frac {\left ({{T_{2}(t_{ref}) - T_{1}(t_{1})}}\right ) +\left ({{T_{3}(t_{ref}) - T_{4}(t_{1})}}\right )}{2} \tag {23}\\ \tau _{\mathrm {ToF}}=& \frac {\left ({{T_{4}(t_{1}) - T_{1}(t_{1})}}\right ) - \left ({{T_{3}(t_{ref}) - T_{2}(t_{ref})}}\right )}{2}. \tag {24}\end{align*}$$

FIGURE 6.

Closed-loop PTP-based time information packet exchange using the FDMA technique in specifically allocated frequency bands.

In the described communication system, DACs at the secondary nodes are configured to operate at a carrier frequency of $3.8\,\mathrm {GHz}$ . Due to the FDMA scheme, each secondary node is allocated a distinct frequency slot, which results in 30.72-MHz frequency offsets among them. Subsequently, feedback messages from the primary node are transmitted to the secondary nodes at a carrier frequency of $3.2\,\mathrm {GHz}$ . The FDMA allocation also introduces frequency variations of $30.72\,\mathrm {MHz}$ among different secondary nodes. These deliberate frequency offsets are implemented to ensure orthogonal communication channels, thereby mitigating interference and enabling the independent operation of each secondary node with the primary node.

The proposed synchronization method begins by calculating the time offset using the IEEE 1588 PTP. This measured time offset is continuously updated with the Kalman filter at each synchronization message interval. Then, the frequency skew is assessed by evaluating consecutive skew estimates with the Kalman filter. Based on this assessment, adjustments are made to the frequency of the DCTCXO crystal oscillator to align the frequency of the secondary node with that of the primary node.

SECTION V. FPGA Implementation Results

FPGAs are ideally suited for implementing complex real-time algorithms. The high-speed and parallel processing capabilities, combined with the robustness and dynamic nature of FPGAs, make them an excellent platform for executing the real-time synchronization process in wireless networked systems. A Zynq UltraScale+ RFSoC FPGA board—integrated with a high-frequency RF front-end module—is employed for this synchronization process, as depicted in Fig. 7 and 8. The RF module upconverts the carrier frequencies to 25.7 and $26.3\,\mathrm {GHz}$ for signal transmission. Integrating the RF module and upconverting to these frequencies aligns with our objective of incorporating this system into an FPGA-based radar system.

FIGURE 7.

Depiction of the synchronization hardware.

FIGURE 8.

Schematic of the hardware structure featuring a Zynq UltraScale+ FPGA, PLL, DCTCXO, RF module, and patch antennas, with explicit connections among the modules.

Multiple experiments validated the proposed synchronization scheme, focusing on accurately estimating and compensating for time offset and frequency skew.

A. Measurement Setup in Anechoic Chamber

In the first experiment, as depicted in Fig. 9, the wireless synchronization test is performed in an anechoic chamber to ensure precise, interference-free performance, with secondary nodes positioned within 3 m of the primary node. In this configuration, the primary node is positioned at the apex of the isosceles triangle, while the two secondary nodes are located at the other two vertices.

FIGURE 9.

Wireless synchronization measurement in an anechoic chamber.

The optimized Kalman filter illustrated in Fig. 10, significantly improves the accuracy of synchronization measurements. Without the Kalman filter, time offset deviations with the PTP protocol exceeded $\pm 20\,\mathrm {ns}$ , as reported in previous work [14]. By continuously refining estimated time offsets, the Kalman filter enhances the stability of the PTP system. It achieves this by weighting each update according to the uncertainties in the prior estimate and the new PTP measurement. This method minimizes large fluctuations, balancing real-time data with historical trends, resulting in a more consistent and reliable time offset, with observed improvements in practical cases. The frequency skew is calculated as $(\theta _{m,k}-\theta _{m,k-1})/T_{s}$ ; however, inherent uncertainties in the PTP protocol—even with the application of the Kalman filter—as well as the integer nature of time in digital timestamp exchanges, introduce variability in time offset measurements at both k and $k-1$ . This variability results in uncertainty in the measured frequency skew. Experimental results indicate that whether using this calculation or assigning zero as an undetermined input for frequency skew, applying increased weighting toward the Kalman filter’s estimated value yields comparable outcomes.

FIGURE 10.

Block diagram of optimized Kalman filter.

The project design operates at a clock frequency of $307.2\,\mathrm {MHz}$ , thereby yielding a clock cycle duration of $3.2552083\,\mathrm {ns}$ . Consequently, the system’s time step is constrained to $3.2552083\,\mathrm {ns}$ , which represents the minimum achievable precision. This limitation is intrinsic to the digital hardware, which is restricted by its clock frequency, and allows for timing adjustments only in discrete increments of $3.2552083\,\mathrm {ns}$ , as dictated by the design’s clocking process.

As depicted in Fig. 11, the compensation protocol is triggered when the Kalman filter’s estimated time offset exceeds $\pm 3.2552083\,\mathrm {ns}$ . When the estimated offset surpasses this threshold, it signifies a time discrepancy that exceeds the system’s precision limit, thus necessitating correction. The compensation protocol then intervenes to adjust the timing.

FIGURE 11.

PTP-calculated time offset, Kalman filter estimation, and control unit compensation in the opposite direction for frequency-aligned nodes in an anechoic chamber.

The control signal u represents the external action taken to compensate for the time offset. When the Kalman filter’s estimated offset exceeds the threshold of $\pm 3.2552083\,\mathrm {ns}$ , the control unit activates the compensation mechanism. This action adjusts the timing in discrete steps, as constrained by the hardware’s clock frequency, to correct the offset and maintain synchronization accuracy within the system’s precision limits. When the estimated time offset crosses the negative threshold, it indicates that the timestamp is lagging behind the reference time and requires an increase in the timestamp to align it with the reference. Conversely, if the estimated time offset crosses the positive threshold, it suggests that the frequency of the secondary node is slightly higher and causes the timestamp to be ahead of the reference. In this case, the control unit reduces the timestamp to bring it back into alignment.

This explanation indicates that the time offset between the primary and secondary nodes is constrained within a range of $\pm 3.2552083\,\mathrm {ns}$ , as optimized by a tailored Kalman filter. This tight range is closely monitored by capturing clock samples generated by timestamps from the general-purpose input/output (GPIO) pins of the FPGA on each node. An oscilloscope tracks the offset to ensure the time offset remains within this constrained range. As illustrated in Fig. 12, the primary node’s clock signal appears as a stable waveform, while the two secondary nodes’ clock signals show a slight phase shift, demonstrating the time offset. This narrow deviation indicates precise time and frequency synchronization, achieved through an optimized Kalman filter in an anechoic chamber. In the histogram presented in Fig. 13, the standard deviation of the skew estimation is depicted, which implies that the secondary node’s frequency is closely aligned with the reference node.

FIGURE 12.

Oscilloscope illustration showing the offset of two secondary nodes’ local clocks relative to the primary node’s local clock in an anechoic chamber. Time scale: $10\,\mathrm {ns}$ /DIV, voltage scale: $5\,\mathrm {V}$ /DIV.

FIGURE 13.

Kalman filter skew estimation inside an anechoic chamber with aligned frequency.

The frequency skew $\gamma$ between the primary and secondary nodes can be determined using the following equation: $$\begin{equation*} \gamma = \frac {\Delta f}{f}. \tag {25}\end{equation*}$$ For example, by configuring the DCTCXO of the primary node to $307.2\,\mathrm {MHz}$ and the secondary node to a relative frequency of $307.200010\,\mathrm {MHz}$ , a 10-Hz difference is introduced. A frequency offset of $10.01\,\mathrm {Hz}$ , as presented in Fig. 14, is determined by applying a fast Fourier transform (FFT) to the signal recorded by an oscilloscope, resulting from the mixing of the sample clocks of the primary and secondary nodes. This offset corresponds to a frequency skew of 32.59 ppb. The Kalman filter skew estimation, shown in Fig. 15, indicates a skew of 32.79 ppb. This experiment confirms that the frequency skew between the primary and secondary nodes can be estimated with the optimized Kalman filter at an accuracy level significantly better than 1 ppb and subsequently corrected by applying the estimated frequency skew to the DCTCXO of the secondary node.

FIGURE 14.

Measured frequency offset, obtained by performing FFT on the mixed clock signals of the primary and secondary nodes with a 10-Hz difference, recorded by an oscilloscope.

FIGURE 15.

Kalman filter skew estimation inside an anechoic chamber with a frequency offset of $10\,\mathrm {Hz}$ .

An alternative approach for calculating the frequency offset, which complements the previously presented method, involves accumulating the external correction units, represented as $\sum _{n=1}^{k} I(u_{\theta ,n})$ . In this context, $I(u_{\theta ,n})$ is defined as 1 when the compensation condition is satisfied and 0 when it is not. Consequently, $\sum _{n=1}^{k} I(u_{\theta ,n})$ quantifies the number of instances in which the compensation condition has been met. Fig. 16 indicates the control unit is activated more frequently to compensate for the assigned frequency offset. This frequency offset is quantified by $$\begin{equation*} \Delta f = \frac {\sum _{n=1}^{k} I\left ({{u_{\theta ,n}}}\right )}{N_{\text {sync}} \cdot T_{s}} \tag {26}\end{equation*}$$ where $N_{sync}$ is the number of synchronization events, equal to 4000 and $T_{s}$ is the constant time interval between synchronization messages, equal to $54.613\,\mathrm {ms}$ .

FIGURE 16.

PTP-calculated time offset, Kalman filter estimation, and control unit compensation in the opposite direction with a frequency offset of $10\,\mathrm {Hz}$ inside an anechoic chamber.

Table 2 presents numerical results for the corresponding skew estimation from the Kalman filter, the accumulation of compensation events, and the measured frequency offset using FFT. The table demonstrates a high degree of consistency among these results.

TABLE 2 Comparison of Estimated and Measured Frequency Offset

Figs. 17 and 18 illustrate the variances of the time offset and frequency skew, respectively, as represented in the error covariance matrix observed in the conducted experiment.

FIGURE 17.

Variance of time offset $\sigma _{\theta }^{2}$ in the error covariance matrix.

FIGURE 18.

Variance of frequency skew $\sigma _{\gamma }^{2}$ in the error covariance matrix.

Figs. 19 and 20 illustrate the Kalman gains associated with time offset and frequency skew, respectively.

FIGURE 19.

Kalman gain associated with the time offset.

FIGURE 20.

Kalman gain associated with the frequency skew.

In wireless network environments, inherent uncertainties can occasionally result in clearly erroneous data. These anomalous data points are systematically excluded to preserve the performance of the Kalman filter. However, to evaluate the filter’s resilience, intentionally large and erroneous measurement data were introduced to the Kalman filter. The results, illustrated in Fig. 21, demonstrate that despite external corrections of control unit u during this disturbance, a minor accumulation of $\sum _{n=1}^{k} I(u_{\theta ,n})\cdot (-1)$ emerged at the end of the process. This indicates that the system is highly resilient to such disturbances. The Kalman filter rapidly stabilizes following these perturbations; within the first 50 samples after the disturbance, the filter effectively recovers and returns to its nominal state, thereby demonstrating its robustness in managing disruptions.

FIGURE 21.

System reaction with erroneous and significantly large PTP-calculated time offset.

B. Measurement Setup in Outdoor Environment

In the second experiment, as depicted in Figs. 22 and 23, the measurements are conducted in an outdoor environment, with the secondary nodes positioned within 12 and 15 m of the primary node.

FIGURE 22.

Wireless synchronization measurement in an outdoor environment with a distance of 15 m between the primary node and each secondary node.

FIGURE 23.

Wireless synchronization measurement in an outdoor environment with a distance of 12 and 15 m.

At shorter distances, the results obtained in the outdoor environment align closely with those from the anechoic chamber, showing no significant differences. However, as the distance increases, slight deviations in frequency (ppb) and increased time offset variations become evident, as depicted in Figs. 24 and 25. Additionally, the control unit u is triggered more frequently to compensate for these increased deviations. This behavior is linked to the decrease in the signal-to-noise ratio (SNR) as distance increases. Consequently, the variance of the PTP estimates rises, thereby necessitating more frequent adjustments by the filter to maintain accurate synchronization.

FIGURE 24.

PTP-calculated time offset, Kalman filter estimation, and control unit compensation in the opposite direction at a distance of 15 m in an outdoor environment.

FIGURE 25.

Kalman filter skew estimation in an outdoor environment with a distance of 15 m.

The synchronization method begins with a presynchronization step, in which the initial PTP message aligns the joining node’s timestamp with the primary node’s time scale. Following this, a Kalman filter—based on the clock model—continuously refines both the time offset and frequency skew. This approach has been validated through empirical measurements in the present study and demonstrates high accuracy and reliability based on the reported measurement results.

SECTION VI. Conclusion

This work successfully optimized time and frequency synchronization protocols in WSNs using a tailored Kalman filter specifically designed for the given task and hardware platform, with performance rigorously validated through measurement results. The hardware implementation, particularly notable due to the scarcity of experimental work in this area, demonstrated exceptional performance. The optimized protocol, leveraging the Kalman filter, effectively suppressed noise and observation errors and significantly enhanced synchronization precision. These findings highlight the system’s potential as a highly precise and stable solution for environments that demand accurate synchronization, such as dynamic radar networks and future ISAC systems. The current synchronization protocol is centralized, which introduces certain limitations in scenarios where the primary node becomes unavailable. In future work, we plan to transition to a decentralized structure to enhance the robustness and reliability of the system. Furthermore, this optimized synchronization protocol will be integrated with FPGA-implemented radar systems mounted on AAV swarms, specifically for missions involving ground area exploration and monitoring. This integration will enable precise, real-time data acquisition and improved coordination for applications, such as environmental surveillance, glacial research related to climate change studies, as well as agricultural and forest monitoring.