[PATCH OLK-5.10] spi: mpc52xx: Add cancel_work_sync before module remove