Infrared growth of geometrical fluctuations in inflationary spacetimes is investigated. The problem of gauge-invariant characterization of growth of perturbations, which is of interest also in other spacetimes such as black holes, is addressed by studying evolution of the lengths of curves in the geometry. These may either connect freely falling "satellites," or wrap non-trivial cycles of geometries like the torus, and are also used in diffeomorphism- invariant constructions of two-point functions of field operators. For spacelike separations significantly exceeding the Hubble scale, no spacetime geodesic connects two events, but one may find geodesics constrained to lie within constant-time spatial slices. In inflationary geometries, metric perturbations produce significant and growing corrections to the lengths of such geodesics, as we show in both quantization on an inflating torus and in standard slow-roll inflation. These become large, signaling breakdown of a perturbative description of the geometry via such observables, and consistent with perturbative instability of de Sitter space. In particular, we show that the geodesic distance on constant time slices during inflation becomes non-perturbative a few e-folds after a given scale has left the horizon, by distances \sim 1/H^3 \sim RS, obstructing use of such geodesics in constructing IR-safe observables based on the spatial geometry. We briefly discuss other possible measures of such geometrical fluctuations.