Modern face recognition approaches target successful person identification in challenging scenarios, where uncooperative subjects are captured under unconstrained imaging conditions. With the introduc-tion of a new generation of 3D acquisition devices capable of dynamic acquisitions, this trend is now emerging also in 3D based approaches. Motivated by these considerations, in this paper we propose an original and effective framework to address face recognition from 3D temporal sequences acquired in adverse conditions, which include internal and ex-ternal occlusions, pose and expression variations, and talking. Due to the novelty of the proposed scenario, a new database has been collected using a single-view structured light scanner with a large field of view, which allows free movement of the acquired subjects. The 3D temporal sequences are divided into fragments each modeled as a linear subspace in order to embody the shape and the motion of the facial surfaces. In virtue of the Riemannian geometry of the space of real k-dimensional linear subspaces, called Grassmann manifold, a new formulation of the matching between 3D temporal sequences has been developed. An unsu-pervised clustering over the Grassmann manifold is also introduced for efficient recognition. The proposed approach achieves promising results, without requiring any prior training or manual intervention.