Purpose The purpose of our study was to use time-lapse in order to evaluate the impact of sperm origin (fresh ejaculate or surgically retrieved) on embryo morphokinetic parameters and clinical outcome in intracytoplasmic sperm injection (ICSI) cycles. Methods This retrospective monocentric study was conducted in 485 unselected couples undergoing 604 ICSI cycles with embryo culture in the Embryoscope®. Among them, 445 couples underwent ICSI cycle with fresh ejaculated sperm and 40 with surgically retrieved sperm (26 with testicular sperm and 14 with epididymal sperm). Embryo morphokinetic parameters and clinical cycle outcome were compared between fresh ejaculated sperm and surgically retrieved sperm. A subgroup analysis was also conducted between testicular and epididymal sperm ICSI cycles. Results Clinical outcome was comparable between groups according to sperm origin. Although most early morphokinetic parameters were comparable between ejaculated and surgical sperm groups, a few parameters were significantly different between both groups, but with a considerable overlap in their distribution. Late cellular events occurred significantly later in the surgical sperm group than in the ejaculated sperm group. Conclusions Morphokinetic analysis did not allow us to identify clinically relevant differences between fresh ejaculate and surgically retrieved sperm groups. Further studies are needed, especially concerning the relationship between sperm origin and late morphokinetic parameters, such as blastocyst development.