Oral consumption of inorganic nitrate, which is abundant in green leafy vegetables and roots, has been shown to increase circulating plasma nitrite concentration, which can be converted to nitric oxide in low oxygen conditions. The associated beneficial physiological effects include a reduction in blood pressure, modification of platelet aggregation, and increases in limb blood flow. There have been numerous studies of nitrate supplementation in healthy recreational and competitive athletes; however, the ergogenic benefits are currently unclear due to a variety of factors including small sample sizes, different dosing regimens, variable nitrate conversion rates, the heterogeneity of participants' initial fitness levels, and the types of exercise tests used. In clinical populations, the study results seem more promising, particularly in patients with cardiovascular diseases who typically present with disruptions in the ability to transport oxygen from the atmosphere to working tissues and reduced exercise tolerance. Many of these disease-related, physiological maladaptations, including endothelial dysfunction, increased reactive oxygen species, reduced tissue perfusion, and muscle mitochondrial dysfunction, have been previously identified as potential targets for nitric oxide restorative effects. This review is the first of its kind to outline the current evidence for inorganic nitrate supplementation as a therapeutic intervention to restore exercise tolerance and improve quality of life in patients with cardiovascular diseases. We summarize the factors that appear to limit or maximize its effectiveness and present a case for why it may be more effective in patients with cardiovascular disease than as ergogenic aid in healthy populations.