Make sure that the extent of the x axis is the same for all plots that are stacked vertically. Now, when the y labels have a different size, each plot panel has a different width