thriving for realism is a bit silly, tho in real life if you charge me with a horse I'll jumpe and stab I'll get hurt as fuck, but the horse'll die in one hit, or, I could simply pick a rock and throw at it's head, the speed should be enough to knock it unconscious and there would be imminent fall! armore? no problem, stretch a sword on the ground and hick it as the horse approaches, if sharpened enough it would cut half way through it's leg! Better yet, throw yourself over the horses head on the blind side(opposite to the spear), and watch the chaotic fall as you and your foe get fucked up... Or I could throw an explosive at the horse(just like a mage's fireball) and watch it tremble and sprint out of control, or, I could aim with a bow and kill it right on...
Cavalry was only strong in certain war moves against certain fighters, knocking off a knight is easy actually, if the horse get's knocked out, dies or you simply pull the knight from it, it would work, the latter having to force the horse to lose speed(not that hard if you are creative), but then again this is a GAME, G A M E, GAME, what you are using as argument is invalid(nothing against u tho, before you start butthurt raging all over like so many other people here)