Landauer's principle states that it costs at least kTln2 of work to reset one bit in the presence of a heat bath at temperature T. The bound of kTln2 is achieved in the unphysical infinite-time limit. Here we ask what is possible if one is restricted to finite-time protocols. We prove analytically that it is possible to reset a bit with a work cost close to kTln2 in a finite time. We construct an explicit protocol that achieves this, which involves changing the system's Hamiltonian avoiding quantum coherences, and thermalising. Using concepts and techniques pertaining to single-shot statistical mechanics, we further develop the limit on the work cost, proving that the heat dissipated is close to the minimal possible not just on average, but guaranteed with high confidence in every run. Moreover we exploit the protocol to design a quantum heat engine that works near the Carnot efficiency in finite time.