Some caterpillars are deadly, but I haven't heard of a case of anyone being poisoned to death by a caterpillar in the northern part of the world. I did see some web pages in which dogs have been stung by caterpillars. Watch out for caterpillars when you are taking a walk with your dog.
Some rare species of caterpillars are poisonous and they mainly live in the Amazon Rainforest. I have heard of certain species of caterpillar who can kill a human within 5 minutes of being stung (similar to caterpillar on the first image). One of the main reasons why some areas in the Amazon Rainforest have NOT been explored by humans. Poisonous frogs, and other very deadly creatures live in the most dense parts of the rainforest. Humans don't belong there.
I have come in contact with some caterpillars in my native Puerto Rico, that the most they did was hurt a bunch (it burns badly but the injury was local). Many people say that most "hairy" worms will sting. So make sure when you see hairy caterpillars you do not touch them. Leave that for the experts who know how to tell them apart.