The problem isn't that people are wearing their no-gi rank, they just haven't learned the fundamentals of grip breaks to counter-act some of the advantages the Gi gives. Realistically, if you train Jiu Jitsu and you're at least a purple belt, you should know how to break some grips in the Gi, especially if it's a major hole in your game when you go into tournaments. The same goes for learning how to defend against lapel chokes.
http://www.youtube.com/watch?v=bKmXm1-9IpI
Here's another 10th Planet brown belt Ryan Fortin having no problems in the Gi.
I don't think the difference is that great. Not enough for me to have a white belt for when I train with the Gi. If anything, my game is better when I wear it. However, I would respect an instructors request to wear a white belt if I was visiting another gym.