No, a driver having insurance means that I will be compensated by the at fault party if they damage or destroy my vehicle or if I require medical care. A person without auto insurance sentences me to the likelihood of having no compensation for the damages or destruction of my vehicle and any medical bills caused by an unisured driver. (Which is why I carry uninsured motorist on my policy - basically I pay more because of irresponsible people)The unisnsured driver gets away without having to pay a monthly premium or pay for my damages.
Also, insurance works better the more people who are paying in because there is more money going into the pot versus overall risk. If less people were paying in to car insurance, rates might go up because the company is making less money. Insurance companies need lots of people paying in in order to be profitable, they would raise rates if they were not getting enough money in due to reduced insurance policies.