||We study the measurement of Higgs boson self-couplings through 2 -> 3 vector boson scattering (VBS) processes in the framework of Standard Model effective field theory (SMEFT) at both proton and lepton colliders. The SMEFT contribution to the amplitude of the 2 -> 3 VBS processes, taking WLWL -> W(L)W(L)h and WLWL -> hhh as examples, exhibits enhancement with the energy A(BSM)/A(SM) similar to E-2/Lambda(2), which indicates the sensitivity of these processes to the related dimension-six operators in SMEFT. Simulation of the full processes at both hadron and lepton colliders with a variety of collision energies are performed to estimate the allowed region on c(6) and c(Phi 1). Especially we find that, with the help of exclusively choosing longitudinal polarizations in the final states and suitable pT cuts, WWh process is as important as the more widely studied triple Higgs production (hhh) in the measurement of Higgs self-couplings. Our analysis indicates that these processes can play important roles in the measurement of Higgs self-couplings at future 100TeV pp colliders and muon colliders. However, their cross sections are generally tiny at low energy machines, which makes them much more challenging to explore.