But I've come to the position that there's not really any reason to use this rule after all.
Realism? Is it really so much more realistic to be suddenly better at whatever it is you're doing after an 8-hour rest than in the middle of doing it?
It's a holdover from the way some video games work? One can probably name as many video games where you level up immediately when you have enough XP (e.g. Angband) as games where you need to return to town to do so (e.g. Mordor/Demise). (And some, like City of Heroes, where you immediately get some benefits of leveling up but have to visit a trainer to get the rest of them.)
Confusion over whether you have your new hit points and prepared spells? That's easy enough to answer: yes, you immediately gain your new hit points. Your new spell slots can be filled if you take the requisite 15-minute downtime for filling empty slots if you're a
Don't want to spend half the gaming session leveling up characters? Especially in the middle of a fight? That's fine, just hold off on delivering XP until the end of the session, and certainly never deliver XP in the middle of a fight. (I've still got some players who, in their own words, "can't be assed" to level up in the week between sessions, and spend the first few minutes of some sessions leveling up, but that's fine, I like to give plenty of time at the beginning of a session for players to settle down and settle in anyway.)
Further discussion can be found here.