The optimized effective potential (OEP) approach has so far mainly been used in benchmark studies and for the evaluation of band gaps. In this work, we extend the application of the OEP by determining the analytical ionic forces within the plane-wave and pseudopotential framework. It is first shown that, due to the constrained optimization inherent to the OEP approach, an extra term needs to be added to the standard Hellmann-Feynman expression for the forces, whenever nonlocal pseudopotentials are employed. Computing this term for functionals based on Hartree-Fock and the hybrid PBE0 functional yields forces with excellent numerical accuracy. Furthermore, results for equilibrium geometries and vibrational frequencies on a set of molecules and solids confirm that the local exchange OEP is able to reproduce results obtained with the nonlocal exchange potential. Our work opens up the possibility to study lattice dynamics using advanced orbital functionals for describing exchange and correlation effects.