The nucleon axial coupling, $g_A$, is a fundamental property of protons and neutrons, dictating the strength with which the weak axial current of the Standard Model couples to nucleons, and hence, the lifetime of a free neutron. The prominence of $g_A$ in nuclear physics has made it a benchmark quantity with which to calibrate lattice QCD calculations of nucleon structure and more complex calculations of electroweak matrix elements in one and few nucleon systems. There were a number of significant challenges in determining $g_A$, notably the notorious exponentially-bad signal-to-noise problem and the requirement for hundreds of thousands of stochastic samples, that rendered this goal more difficult to obtain than originally thought. I will describe the use of an unconventional computation method, coupled with "ludicrously'" fast GPU code, access to publicly available lattice QCD configurations from MILC and access to leadership computing that have allowed these challenges to be overcome resulting in a determination of $g_A$ with 1% precision and all sources of systematic uncertainty controlled. I will discuss the implications of these results for the convergence of $SU(2)$ Chiral Perturbation theory for nucleons, as well as prospects for further improvements to $g_A$ (sub-percent precision, for which we have preliminary results) which is part of a more comprehensive application of lattice QCD to nuclear physics. This is particularly exciting in light of the new CORAL supercomputers coming online, Sierra and Summit, for which our lattice QCD codes achieve a machine-to-machine speed up over Titan of an order of magnitude.