Realistic energies for vortex pinning in intermediate-density neutron star matter