We are all living with hundreds of anthropogenic chemicals in our bodies every day, a situation that threatens the reproductive health of present and future generations. This review focuses on endocrine disrupting compounds (EDCs), both naturally occurring and man-made, and summarizes how they interfere with the neuroendocrine system to adversely impact pregnancy outcomes, semen quality, age at puberty, and other aspects of human reproductive health. While obvious malformations of the genitals and other reproductive organs are a clear sign of adverse reproductive health outcomes, injury to brain sexual differentiation and thus the hypothalamic-pituitary-gonadal (HPG) axis can be much more difficult to discern, particularly in humans. It is well-established that, over the course of development, gonadal hormones shape the vertebrate brain such that sex specific reproductive physiology and behaviors emerge. Decades of work in neuroendocrinology has elucidated many of the discrete and often very short developmental windows across pre- and postnatal development in which this occurs. This has allowed toxicologists to probe how EDC exposures in these critical windows can permanently alter the structure and function of the HPG axis. Included in this review are discussion of key EDC principles including how latency between exposure and the emergence of consequential health effects can be long, along with a summary of the most common and less well understood EDC modes of action. Also provided are extensive examples of how EDCs are impacting human reproductive health, and evidence that they have the potential for multi-generational physiological and behavioral effects.