I scrub my dog's teeth with a toothbrush and bakingsoda tooth paste. I then take some clean gauze and soak in Hydrogen Peroxide and wipe along his gums. It really helps
A note of caution: If you have a breed with strong jaws (like a pitbull), he can accidentally chomp your finger while squirming away from you. They generate so much jaw pressure, even accidentally, that they can really injure you if they are trying to get away.
Most vets give them some sort of sedative or anesthesia... I personally don't like the risks involved in that. I have had success with a low dose of valium, which leaves him calm, coherent, and conscious-- but relaxed while I clean his teeth.