For me, I think it's about finding a balance between personal freedom, the impact personal freedom has on society (both for good and for ill), and the real physical consequences of a drug. We've already recognized (or at least accepted) as a society that it's OK to put certain substances into your body to achieve certain effects. Why is it OK to take a Prozac a day and alter the fundamental structure of your brain, but pot is evil incarnate? The multibillion dollar pharma industry makes sure you feel just fine about taking whatever they're selling, but God forbid you let a plant follow it's natural biological course and grow up in your yard or basement.
Are there societal consequences to personal freedom? Sure. We've decided as a nation that it's better to have personal freedom and sometimes have to deal with the problems that arise from the abuse of that freedom than to take freedom away entirely -- except for pot. The societal problems that come with weed are almost exclusively the result of it's legal status, not it's pharmacological effects. Is it not better for a 50-something nurse to grow a few plants in her basement than to help fund drug cartels by buying weed off the street? What kind of public menace is she? Isn't she actually the opposite? What happened to the bootleggers after Prohibition was repealed? Al Capone didn't keep up the whiskey smuggling business after that because there was no reason to. I expect the Mexican cartels will do the same. They'll find other things to smuggle and kill people for; it's got nothing to do with pot and everything to do with economics. Certainly there are public health concerns that go with smoking anything and pot is no better than tobacco for your lungs (though it's much better for your body chemistry). You also don't have to smoke -- there are vaporizers and other enteral means of ingestion.
There are damn few physical consequences to pot; especially as compared to other recreational and medical drugs. It's impossible to ingest enough THC by smoking to die from an OD -- you can kill yourself the first time you drink alcohol. The same physical effects on the lungs are there as with smoking tobacco but anybody who smokes the pot equivalent of a 20 pack-year tobacco history (the benchmark for smoking-related lung disease) would be baked out of their gourd CONSTANTLY; so it's clear that at the very least it's no more harmful that tobacco and probably less so (owing in no small part as well to the fact that you don't have Big Tobacco companies putting hundreds of chemicals like formaldehyde, ammonia, and fiberglass in your pot.).
So if we can accept that it's OK to ingest a substance to feel better, that the problems created by weed are a result of the law and not the drug, and that it is in fact safer than the two most popular legal substances by far; why do we insist on criminalizing it so much? I don't live in Cali, but come on Prop 19!!!