Changes in moisture content of single pulp fibers have an immense influence on the behavior of paper and paper products. Here, an atomic force microscopy (AFM)-based method is applied to investigate the viscoelastic properties of pulp fibers at varying relative humidity (RH) in the transverse direction. Pulp fibers have not only anisotropic properties, but also a very rough surface due to their hierarchical structure. For this reason, we have developed a specific load schedule for the AFM-based test method to overcome uncertainties and limitations due to surface roughness of the pulp fibers. The evaluation of the experimental data combines contact mechanics and viscoelastic models which consist of springs and dashpots in series or parallel describing elastic and viscous behavior. Here, it will be demonstrated that the so-called Generalized Maxwell (GM) model yields comparable results for single pulp fibers at five different RH values and in water. The moisture changes lead to a decrease in the elastic modulus but increase in the relaxation effects with increasing RH. All the determined parameters for the elastic and viscous behavior exhibit a gradual decrease with increasing RH from 10 to 75% RH. The elastic moduli decrease by a factor of 10 and the viscosities are decreasing by a factor of 10–20. In water, there is an even more pronounced decrease of the elastic moduli by a factor 100, and the viscosities decrease by at least three orders of magnitude compared to 10% RH. This indicates that the mechanical response of pulp fibers in water is significantly different than in humid air. This is also illustrated by the fact that a GM model of order two suffices to describe the material behavior in humid air but a GM model of order three is necessary to fit the material behavior in water. A possible interpretation is an additional relaxation effect of the pulp fiber wall in water.