In most cases, a bullet will lose much more kinetic energy on impact to a human body than a fist will, but it will transfer much less momentum. You're basically right in your conclusions, however, because kinetic energy doesn't really matter at all when your goal is to push over a human.

For example, take a basic 9x19mm load, with a bullet weighing 124 grains (8 grams), entering a human chest at 1150fps (350m/s) and exiting the back at 500fps (152m/s). The bullet lost 295ft-lbs of kinetic energy while penetrating the torso -- some of this went towards bullet deformation, some of it was lost to friction, some deformed and accelerated tissue sideways out of the bullet's path, and a small part went towards pushing the person backwards. According to conservation of momentum, 11.5lbf (1.58kgm/s) of momentum was transferred to the body -- enough to accelerate a 155lb (70kg) person backwards at a whopping 0.074fps (0.023m/s).

A punch might have 500 times as much mass moving forwards at 1/30th the velocity, which would mean it has 5/9th the kinetic energy of the above bullet. Again, some of the energy will be released as deformation, sound, etc. More importantly, the punch has 16.7x as much momentum to transfer to the target.

Figuring out the force with which the projectile or fist accelerates the target and the work done is far more complex than that, and I'm not even going to attempt that.