Let $f$ and $g$ be two smooth vector fields on a manifold $M$. Given a submanifold $S$ of $M$, we study the local structure of time-optimal trajectories for the single-input control-affine system $\dot q =f(q)+u\, g(q)$ with initial condition $q(0)\in S$. When the codimension $s$ of $S$ in $M$ is small ($s\leq 4$) and the system has a small codimension singularity at a point $q_0\in S$, we prove that all time-optimal trajectories contained in a sufficiently small neighborhood of $q_0$ are finite concatenations of bang and singular arcs. The proof is based on an extension of index theory to the case of general boundary conditions.