Tensor networks states allow one to find the low-energy states of local lattice Hamiltonians through variational optimization. Recently, a construction of such states in the continuum was put forward, providing a first step towards the goal of solving quantum field theories (QFTs) variationally. However, the proposed manifold of continuous tensor network states (CTNSs) is difficult to study in full generality, because the expectation values of local observables cannot be computed analytically. In this paper we study a tractable subclass of CTNSs, the Gaussian CTNSs (GCTNSs), and benchmark them on simple quadratic and quartic bosonic QFT Hamiltonians. We show that GCTNSs provide arbitrarily accurate approximations to the ground states of quadratic Hamiltonians and decent estimates for quartic ones at weak coupling. Since they capture the short distance behavior of the theories we consider exactly, GCTNSs even allow one to renormalize away simple divergences variationally. In the end our study makes it plausible that CTNSs are indeed a good manifold to approximate the low-energy states of QFTs.