Many scientific applications require to solve successively linear systems Ax=b with different right-hand sides b and a symmetric positive definite matrix A. The Conjugate Gradient method applied to the first system generates a Krylov subspace which can be efficiently recycled thanks to orthogonal projections in subsequent systems. A modified Conjugate Gradient method is then applied with a specific initial guess and initial descent direction and a modified descent direction during the iterations. This paper gives new theoretical results for this method and proposes a new version which seems robust as far as loss of orthogonality is concerned. Numerical experiments show the efficacy of our method even for quite different right-hand sides.