Gauge theories form the basis of our understanding of modern physics—ranging from the description of quarks and gluons to effective models in condensed matter physics. In the nonperturbative regime, gauge theories are conventionally treated discretely as lattice gauge theories. The resulting systems are evaluated with path-integral based Monte Carlo methods. These methods, however, can suffer from the sign problem and do not allow for a direct evaluation of real-time dynamics. In this work, we present a unified and comprehensive framework for gauged Gaussian projected entangled pair states (PEPS), a variational ansatz based on tensor networks. We review the construction of Hamiltonian lattice gauge theories, explain their similarities to PEPS, and detail the construction of the ansatz state. The estimation of ground states is based on a variational Monte Carlo procedure with the PEPS as an ansatz state. This sign-problem-free ansatz can be efficiently evaluated in any dimension with arbitrary gauge groups, and can include dynamical fermionic matter, suggesting new options for the simulation of nonperturbative regimes of gauge theories, including quantum chromodynamics.