I cannot speak for others but going to a friend's residence often makes me nervous. I know what I have at MY incall. I cannot imagine what a stranger might have in his home.
It leaves me feeling very vulnerable unless we already know each other.
I often do outcalls to hotels. I charge an outcall fee not because I'm missing appointments otherwise, but because there is generally a bunch of travel time, gas and then ... for me, valet at the hotel and tipping.
Back before this whole arena got a little tight, financially, for most ... men would just say to put the valet on their room number and then, the lady could just tip the valet nicely.
Now ... I just don't see this happening hardly at all which is why I charge an outcall fee. It's for the gas, travel time, valet fees and also, packing up what I need for the appointment as well.
It's a heck of a lot easier to greet a man at the door in lingerie and in your own environment.
Interesting question, though! Sometimes, it really is nice to go to another place. It's just "according", though. I don't visit apartments, period.