What if you keep ignoring a Census worker?

I forgot to fill out the Census form and now I keep getting these annoying notices left on my door from a Census worker.

If I don’t do anything about it, if I don’t answer the questions, what will happen? I thought the Census was to count the number of people. I’m a little uncomfortable revealing my race.