Hitting a bag with bag gloves only poses about the same risk of sprain. Plus, if your hands sweat a lot, the sweat can soften up the skin on your knuckles, which can lead to tearing your skin. I used to wail on a heavy bag with bag gloves, and about once a month I'd lose a dime-sized chunk of skin off my middle knuckle. Hurt like hell,and I had to stop punching with that hand until the skin grew back.
I've heard that using a heavy punching bag is very dangerous if you do not wear bag mits. On the other hand, I want to condition my hands and wrists(But not break them). Should I wear mits?
Generally, for heavy bag sessions, you should wear heavy bag gloves and wraps. Reasons: injury prevention; joint health; increased work capacity; decreased discomfort; etc.