Yes and no. The precise laws and requirements vary from country to country.
New Zealand Edit
In New Zealand you must fill out an "Exemption Form" in order to homeschool your children. As long as you know how to fill out the form (your local support group can help with this) it's easy and you can have your children out of school and back into the home learning environment within weeks.
However your children will then be assessed by the Education Review Office, or ERO. The ERO officer will go through your children's work and if you have not satisfied their requirements they can revoke your exemption form, meaning that (legally) you must send your children back to school.
Of course because there are so many homeschoolers and so few ERO officers, they will only visit about once or twice in the child's educational timespan. Indeed some families get missed out altogether.
Some parents, such as the self-titled "unschoolers"--who aim to teach only what the child wants to learn rather than forcing a broad spectrum--continue without an exemption form after their inevitable disqualification (due to inadequate range of education), or begin teaching without one, as their teaching methods cannot possibly meet the ERO's standards.So far, none of these families have (reportedly) been caught doing this, so the repercussions are unknown (although the law makes them sound harsh), but even without a track record of punishments homeschooling without an exemption is not recommended.
This question was asked on the wiki: Homeschooling.