HOWTO upright capsule

Set the orientation back to upright and the angular velocity to zero after every time step. There is a dBodySetMaxAngularSpeed() function that does it automatically. Just set the maximum angular velocity to zero, and the capsule will never rotate. It won't let you rotate around the up axis, but this is usually done outside the physics simulation. (e.g. a "Camera" entity in a 3D engine whith a yaw() method, attached to the capsule)

For the ability to climb stairs or overcome small, steep obstacles it is useful to use a ray for "feet", and a spring to push the character upwards (with some damping).

Jon Watte has added a capsule/ray based character to his demo application, available at http://www.mindcontrol.org/~hplus/raycar/.

Useful discussions on this topic appear in the mailing list: 1, 2, 3, 4, 5