Fermionic Gaussian Projected Entangled Pair States (PEPS) are fermionic tensor network state constructions which describe the physics of ground states of non-interacting fermionic Hamiltonians. As non-interacting states, one may study and analyze them very efficiently, in both analytical and numerical means. Recently it was shown that they may be used as the starting point - after applying so-called PEPS gauging mechanisms for variational study of non-trivial, interacting states of lattice gauge theories. This is done using sign-problem free variational Monte-Carlo. In this work we show how to generalize such states from two to three spatial dimensions, focusing on spin representations and requirements of lattice rotations. We present constructions which are crucial for the application of the above mentioned variational Monte-Carlo techniques for studying non-perturbative lattice gauge theory physics, with fermionic matter, in 2 + 1-d and 3 + 1-d models. Thus, the constructions presented here are crucial for the study of non-trivial lattice gauge theories with fermionic tensor network states.