Let's put it another way: having to cast from the phone to the TV is like using a normal screw that requires a screwdriver. A TV where you don't need the phone is like one of those tool-less thumbscrews[1]. They are convenient because in order to use one tool (the screw) you don't have to reach for another tool (the screwdriver).

[1] http://www.baaqii.com/promanage/productimage/Ewhole/A/A678-1...