I can't invite users to social groups either. The option is not available.
But an easy way to go about it is to simply give the person the link and ask that they join. For example…
Hey Rollyco, why don't you join us over at the Music Appreciation Club? desioner