Yes, you must withdraw your child from the public school.
If your child completed the previous school year in a public school and is considered enrolled for the upcoming year, you must formally withdraw your child from the school BEFORE the new school year begins (or as soon as possible) so they will not consider your child truant.