On-board Deep Q-Network for UAV-assisted Online Power Transfer and Data Collection
Ref: CISTER-TR-191003 Publication Date: 2019
On-board Deep Q-Network for UAV-assisted Online Power Transfer and Data Collection
Ref: CISTER-TR-191003 Publication Date: 2019Abstract:
Unmanned Aerial Vehicles (UAVs) with Microwave
Power Transfer (MPT) capability provide a practical
means to deploy a large number of wireless powered sensing
devices into areas with no access to persistent power supplies.
The UAV can charge the sensing devices remotely and harvest
their data. A key challenge is online MPT and data collection
in the presence of on-board control of a UAV (e.g., patrolling
velocity) for preventing battery drainage and data queue
overflow of the devices, while up-to-date knowledge on battery
level and data queue of the devices is not available at the UAV.
In this paper, an on-board deep Q-network is developed to
minimize the overall data packet loss of the sensing devices, by
optimally deciding the device to be charged and interrogated
for data collection, and the instantaneous patrolling velocity
of the UAV. Specifically, we formulate a Markov Decision
Process (MDP) with the states of battery level and data
queue length of devices, channel conditions, and waypoints
given the trajectory of the UAV; and solve it optimally with
Q-learning. Furthermore, we propose the on-board deep Qnetwork
that enlarges the state space of the MDP, and a
deep reinforcement learning based scheduling algorithm that
asymptotically derives the optimal solution online, even when
the UAV has only outdated knowledge on the MDP states.
Numerical results demonstrate that our deep reinforcement
learning algorithm reduces the packet loss by at least 69.2%,
as compared to existing non-learning greedy algorithms.
Kai Li
Wei Ni
Eduardo TovarPublished in IEEE Transactions on Vehicular Technology, IEEE, Volume 68, Issue 12, pp 12215-12226.
DOI:10.1109/TVT.2019.2945037.
ISSN: 0018-9545.
Record Date: 15, Oct, 2019