For those who don't want to make their own products... (this may not be as natural as some of the other solutions, but it sure is simple)
I've been using St. Ives Apricot Scrub on my face, neck, and chest twice a day. It's inexpensive and sold at Walmart (and everywhere else I assume). It's the only thing I've found that cleared my skin up. I think regular products just weren't getting deep enough into my pores to get them clean. This has an exfoliating scrub in it, so it gets deep. I have very sensitive skin and this product isn't bothering my skin at all. I'm very impressed with it and have heard good things from other people I recommended it to.
I can't recommend St. Ives highly enough. A total stranger walked up to me at Walmart and told me about it while I was looking through the facial products.
I use Cetaphil moisturizer as well. My skin has never been so clear. I'm really happy about it.
I've also started washing the oils off my face and neck a couple more times a day (with just a wet rag or even a dry napkin) so nothing can sit there long enough to make it's way inside my skin. I noticed when I sleep at night, I get creases in my neck and those creases were developing zits. Now that I've been scrubbing those creases after I wake up I haven't had any more zits there.