Yeunwoong Kyung

In heterogeneous wireless networked control systems (WNCSs), the age of information (AoI) of the actuation update and actuation update cost are important performance metrics. To reduce the monetary cost, the control system can wait for the availability of a WiFi network for the actuator and then conduct the update using a WiFi network in an opportunistic manner, but this leads to an increased AoI of the actuation update. In addition, since there are different AoI requirements according to the control priorities (i.e., robustness of AoI of the actuation update), these need to be considered when delivering the actuation update. To jointly consider the monetary cost and AoI with priority, this paper proposes a priority-aware actuation update scheme (PAUS) where the control system decides whether to deliver or delay the actuation update to the actuator. For the optimal decision, we formulate a Markov decision process model and derive the optimal policy based on Q-learning, which aims to maximize the average reward that implies the balance between the monetary cost and AoI with priority. Simulation results demonstrate that the PAUS outperforms the comparison schemes in terms of the average reward under various settings.

The observation satellite can exploit its own storage and computational resources to reduce the transmission delay. However, too excessive usage of these resources can have negative effects on the queuing delay at the relay satellite and/or on conducting other tasks at each observation satellite. In this paper, we proposed a new resource- and neighbor-aware observation transmission scheme (RNA-OTS). In RNA-OTS, each observation satellite decides whether to use its resources and the resources of the relay satellite at each time epoch by considering its resource utilization and transmission policies of neighbor observation satellites. For the optimal decision of each observation satellite in a distributed manner, the operation of observation satellites is modeled by means of a constrained stochastic game, and a best-response-dynamics-based algorithm is devised to find the Nash equilibrium. The evaluation results demonstrate that RNA-OTS can decrease the delay to deliver the observation to the destination by up to 87% compared to a relay-satellite-based scheme while guaranteeing a sufficiently low average utilization of the resources of the observation satellite.