Historically, the field of plasmonics has been relying on the framework of classical electrodynamics, with the local-response approximation of material response being applied even when dealing with nanoscale metallic structures. However, when the confinement of electromagnetic radiation approaches atomic scales, mesoscopic effects are anticipated to become observable, e.g., those associated with the nonlocal electrodynamic surface response of the electron gas. Here, we investigate nonlocal effects in propagating gap surface plasmon modes in ultrathin metal–dielectric–metal planar waveguides, exploiting monocrystalline gold flakes separated by atomic-layer-deposited aluminum oxide. We use scanning near-field optical microscopy to directly access the near-field of such confined gap plasmon modes and measure their dispersion relation via their complex-valued propagation constants. We compare our experimental findings with the predictions of the generalized nonlocal optical response theory to unveil signatures of nonlocal damping, which becomes appreciable for few-nanometer-sized dielectric gaps.