Are there limitations on minimum cable diameter?
Not really, as long as it's not zero or a negative number. In the current EOD robot sample, there's a thin tether cable that works well. The Offshore sample features a ROV with a 4cm cable tether.
Small cables may be hard to see in the scene unless the camera or viewpoint is placed very close. However, note that the radius of the cable rendering can be set independently of the radius of the dynamics cable. In the robot tether's case, the graphic cable is 5mm in radius, but the dynamic cable is only 2mm (this was done to make sure the tether remains visible whenever the viewpoint is set far from the robot).