First of all, referendum are advisory and not legally binding, government can but dont have to follow the outcome. Of course if they wont follow it will be political suicide for such party and on the next general election party that favoured the outcome which won, will be voted in to carry out will of the people.
But yes, even if that was a vote by one person its still a vote. It is majority, no matter how you look at it. Democracy is about majority, not how big majority is or not. There always will be morons with loud mouths that didnt bother to vote and now disagree with the referendum outcome.