The penetration difference between small caliber HEAT (high explosive anti-tank)and HEDP (high-explosive dual purpose) rounds is not that big a deal.
The 40mm HEDP round will penetrate around 50mm of
RHA and produce lethal fragments in a roughly five meter radius.
A 40mm HEAT round would not penetrate significantly more RHA and would basically produce no significant fragmentation effect.
A 40mm HE round (i.e. no significant anti armor effect at all) will also not produce significantly more fragments.
Basically, this be because the round is so small.
For comparison, thrown HE grenades will produce lethal fragments over a significantly larger radius (10-20 meters). This is because the thrown grenades are significantly heavier.
If you are asking about a HEAT thrown grenade (something like the
Panzerhandgranaten or Panzerwurfmine). The short answer is that you can make them to have pretty good armor penetration (maybe as much as 250-300mm RHA iwth modern materials) but no one would. They are very difficult to use. Instead expect something like the
AT4,
carl gustav, or
RPG-16