Not being able to see your images, it's hard to say too much. I have the 200-500 and use it on a D7100, and find it nice and sharp. When not going freehand, I usually use it on a gimbal (similar to a Wimberly Side-kick, but home made) on a big old Manfrotto tripod, or on a smaller monopod gimbal (again similar to Wimberly but not). VR seems to make no difference at all on the tripod and has a slight, at least occasional, benefit on the monopod. This might vary with distance and whatnot, but I have not noticed a problem with leaving it on, which I usually do, in part just because I forget to turn it off. And of course it might also vary with the tripod. Although my tripod rig is hefty (Manfrotto 3221 with a Kirk BH-1 ball head) it can still vibrate, and I find it smoother with the gimbal.
I will confess that, because of high ISO noise on the D7100, I tend often to shoot at too slow a shutter speed, so I have not exhaustively tested for sharpness at the highest shutter speeds at all distances and conditions, because the VR on this lens is so good. But in general, testing just for the sake of testing even with pixel peeping deeper than I find necessary, I have noted little if any difference leaving VR on when it is not necessary, as opposed to huge difference when it is.
But since circumstances are so often just a little different, I think you're probably on the right track simply to test yours, and set the VR accordingly. Whether it's an issue of the sample variation in the lens, or tripod firmness, or shooting technique, if one way is sharper, there's no downside to using it as long as you remember what you've done.
