Try using vastly different value oscillators instead of just removing to see what the difference is. I don't know how close your external oscillator value is in relation to the one on the chip internally.
Oscillators are cheap and you could temp solder in a header pin to try different ones maybe.
Maybe you can run a simulation in MPLABX with the MCP39F511 and see how it behaves as well.
you could also make an external circuit with a display that actually tells you what the clock is running at if it's that crucial.
Sorry if my obvious responses are annoying, it's fun to think about other problems.