Your sphincters are muscles just like any other muscles on your body. Think of the "stretching" involved not as the stretching involved with a rubber band (an inanimate object), but more like the stretching involved when you go to the gym and stretch your body. You start off with easy stretches to warm your muscles up, particularly when you are just beginning to stretch. Over time, your muscles become stronger and more flexible, and can do more extreme stretches. Eventually, you can do wild stretches that seem impossible to most who do not stretch (ever seen a lady do a full split??). This type of stretching does not leave your body loose and damaged, rather it makes your muscles powerful & capable, and leaves you in better shape than before. When done CORRECTLY, the type of stretching involved in anal fisting does the same thing. You should work your way up to it developing your relationship with those muscles over time until they are strong and skilled enough to open themselves wide enough to accommodate a fist. This does NOT leave you with a loose asshole! Your sphincters are not objects like rubber bands. They are human muscles and they get stronger the more you work them. You are actually much LESS likely to ever have issues with hemorrhoids or incontinence if you have a regular relationship with exercising your sphincter muscles, whether that's for fingers, dildos, or fists.
The idea that fisting is violent/painful and even the idea that receiving anal penetration is a passive act are both false and dangerous ideas held by those who lack experience. The person receiving anal penetration needs to do an incredible amount of psychological and physical exercise to accept larger objects inside them. Many of the gentlemen I know who are into it think of receiving anal penetration as a sport of sorts and try to train themselves regularly.
But it's a very common misconception that many people hold. And perhaps your joke was even rooted in not fully understanding the mechanics involved in fisting. 