Fix sampler tests
I noticed that the sampler tests are being run in the regular CI job in addition to the dedicated jobs and using at least half the time. This should speed up the standard jobs.
It will also remove the sampler running from the coverage, which I think is what we wanted.