The potential of millimetre wave (mmWave) to provide dependability in 5G cellular networks has prompted the number of wireless communication network users to rise, indicating the progress of telecommunication standards. Despite the fact that the mmWave bands can support massive data rates, the effort required to deliver these data rates for end-to-end services while maintaining ultra-low-latency performance and reliability to support developing applications and use cases will necessitate a re-evaluation of all layers of the protocol stack. As a result, this paper examines millimetre wave characteristics and how they impact end-to-end transmission. In addition, this study examines the difficulties and potential solutions for offering dependable, end-to-end, and ultra-low-latency services in mmWave cellular networks. Aside from comparing mmWave and microwave, this article will also examine scheduling in order to mitigate the negative effects of intermittent connection in mmWave and fully use the capacity. This project's challenge and solution will be modelled using Deep Reinforcement Learning in order to do mathematical modelling and provide delay-propagation characteristics. Finally, the simulation results of physical modelling are provided to demonstrate the effectiveness of the suggested technique. Keywords: Delay, millimetre wave, Scheduling