Yes, first thing, deep breath. I know how you feel, the exact same thing happened to me except that my dog did not initially get a killing grip on the small dog and I was able to eventually pry his mouth open and the small dog ran away, apparently not seriously hurt. I looked for the dog, but never saw it again. My other two dogs were in the yard and just watched everything calmly, mildly interested. It still upsets me, more than ten years later.
Neither you nor your dog did anything wrong. I would continue with the leash training and always be alert when he is around small animals, although he may not be as reactive on leash as in his own backyard. Or he may be worse, but now you know to be careful.
Fletcher was always very leash reactive to strange dogs of any size, and chased any cats in his yard, but lived peacefully with five indoor cats, so you just never know ahead of time how any individual dog will react.