We present a method for solving the time-limited coverage problem with the objective of autonomously exploring an unknown environment. Here, the robot is tasked with planning a path over a horizon such that the accumulated area swept out by its sensor footprint is maximized. Because this problem exhibits a diminishing returns property known as submodularity, we choose to formulate it as a sequential decision making process. This formulation allows us to reason about the effect of the robot’s actions on future world coverage states using a rollout-based search algorithm. To quickly find near-optimal solutions, we propose an effective approximation to the coverage sensor model which adapts to the local environment. We validate our proposed coverage planner with high-fidelity dynamic simulations in diverse environments and on physical robots across various real-world environments.