Deterence is increased both by the severity of the punishment, but also by the certainty of receiving the punishment. I think most major crimes are committed by people who have comitted minor crimes. If getting caught for minor things becomes more likely, it may be less tempting to go down that path.
There's a large body of work trying to evaluate the effects of both causes. Arguably, the US has gone way down in the realm of diminishing returns in terms of severity, especially when it come to drug laws.
Of course this doesn't take away any of the potential downsides in terms of civil liberties of any approach that raises the certainty of getting punishment.
It's interesting that, according to that Wikipedia article, "...Research has shown that increasing the severity of a punishment does not have much effect on crime...". Someone should tell that to the state lawmakers where three strikes sentences are prevalent.
http://en.wikipedia.org/wiki/Deterrence_(legal)