In this paper, we review constitutive models for soft materials. We specifically focus on physically based models accounting for hyperelasticity, visco-hyperelasticity, and damage phenomena. For completeness, we include the thermodynamically based viscohyperelastic and damage models as well as the so-called mixed models. The models are put in the frame of statistical mechanics and thermodynamics. Based on the available experimental data, we provide a quantitative comparison of the hyperelastic models. This information can be used as guidance in the selection of suitable constitutive models. Next, we consider visco-hyperelasticity in the frame of the thermodynamic theory and molecular chain dynamics. We provide a concise summary of the viscohyperelastic models including specific strain energy density function, the evolution laws of internal variables, and applicable conditions. Finally, we review the models accounting for damage phenomenon in soft materials. Various proposed damage criteria are summarized and discussed in connection with the physical interpretations that can be drawn from physically based damage models. The discussed mechanisms include the breakage of polymer chains, debonding between polymer chains and fillers, disentanglement, and so on.