You aren't renting to some stranger from craigslist, or hanging up a sign in the alley? These companies, screen the people, and they have people that are return customers. They aren't just robbing people blind. And if something does happen you are insured through the company.
Back when I lived at my first condo, it was very easy. It was a condo hotel, so you could just put your unit back in the available rental pool and would receive part of the compensation. I used to do that 3 or 4 times a year, NYE, Superbowl weekend, March madness first weekend. Those kind of times.
This is a little different, because it isn't directly through the building (it is frowned upon by the building), so you have to make arrangements with some company. I don't know if this is something I would be interested in too regularly. We have been here 5 years and never even thought about it. But with the Superbowl, it seemed like a prime opportunity. We'll see how it goes.