The continental shelf of the Northeast United States and Nova Scotia is a productive marine ecosystem that supports a robust biomass of living marine resources. Understanding marine ecosystem sensitivity to changes in the physical environment can start with the first-order response of phytoplankton (i.e., chlorophyll a), the base of the marine food web. However, the primary physical associations to the interannual variability of chlorophyll a in these waters are unclear. Here we used ocean color satellite measurements and identified the local and remote physical associations to interannual variability of spring surface chlorophyll a from 1998 to 2013. The highest interannual variability of chlorophyll a occurred in March and April on the northern flank of Georges Bank, the western Gulf of Maine, and Nantucket Shoals. Complex interactions between winter wind speed over the Shelf, local winter water levels, and the relative proportions of Atlantic versus Labrador Sea source waters entering the Gulf of Maine from the previous summer/fall were associated with the variability of March/April chlorophyll a in Georges Bank and the Gulf of Maine. Sea surface temperature and sea surface salinity were not robust correlates to spring chlorophyll a. Surface nitrate in the winter was not a robust correlate to chlorophyll a or the physical variables in every case suggesting that nitrate limitation may not be the primary constraint on the interannual variability of the spring bloom throughout all regions. Generalized linear models suggest that we can resolve 88% of March chlorophyll a interannual variability in Georges Bank using lagged physical data.