||A key unresolved question is the role that galaxy mergers play in driving stellar mass growth over cosmic time. Recent observational work hints at the possibility that the overall contribution of 'major' mergers (mass ratios greater than or similar to 1 : 4) to cosmic stellar mass growth may be small, because they enhance star formation rates by relatively small amounts at high redshift, when much of today's stellar mass was assembled. However, the heterogeneity and relatively small size of today's data sets, coupled with the difficulty in identifying genuine mergers, makes it challenging to empirically quantify the merger contribution to stellar mass growth. Here, we use Horizon-AGN, a cosmological hydrodynamical simulation, to comprehensively quantify the contribution of mergers to the star formation budget over the lifetime of the Universe. We show that (1) both major and minor mergers enhance star formation to similar amounts, (2) the fraction of star formation directly attributable to merging is small at all redshifts (e.g. similar to 35 and similar to 20 per cent at z similar to 3 and z similar to 1, respectively) and (3) only similar to 25 per cent of today's stellar mass is directly attributable to galaxy mergers over cosmic time. Our results suggest that smooth accretion, not merging, is the dominant driver of stellar mass growth over the lifetime of the Universe.