[PATCH OLK-5.10 2/3] spi: cadence-quadspi: Handle spi_unregister_master() in remove()