The Lee bullet seater can't handle the Hornady hollow point bullet. The concave seating plug is the same diameter as the top of the hollow point, the sharp edges on each mis-align on each other and cause the bullet to shift during seating.

In this case, a bell on the case during sizing would not help, the mis-alignment of edges would still tip the bullet at the top of the seating stroke.

If I keep this Lee set, I'll have to take that seating plug and grind the end flat. I was going to buy a Hornady set, but the Lee was much cheaper.

The Hornady set comes with two plugs, one for round nose bullets and one for flat. The Lee die is only geared for round nose. A Hornady bullet seater die also has a guide sleeve that keeps the bullet aligned on the way up to seating, which would help a lot here.

If you are not just reloading round nose bullets, I would recommend you spend a bit extra and get Hornady dies, or at least any other make that offers two plugs, otherwise the Lee set is fine.