Wiccan and Speed are also Magneto's grandchildren, which is their third connection. They have strongly alluded that Wiccan will inherit the mantle of Sorcerer Supreme from Stephen Strange and become an X-Man at the same time in various recent media, which is a smaller pip, but still a pip.
And honestly splitting hairs about who to exclude isn't really a good idea in any event. If someeone wants to play Hulkling and their explanation is his declaring 'Where Billy goes, I go' then that's that. If Speedball shows up and says 'Can I crash on your couch, somebody stole my wallet' he's not exactly bringing the whole Avengers roster with him.
The X-Men themselves are winding up further and further afield these days (Rogue, Cannonball, and Sunspot are all serving as Avengers right now) and the X-Men to me have been Mutant-flexible. Warlock is not Homo Sapiens Superior but they let him in anyway (it's because he can turn into a spaceship, always handy).