To avoid the issue, set q->limits.max_sectors with the minimum value between dev->max_sectors and q->limits.max_sectors.
dev->max_sectors describes the ATA hardware limitation (similar to shost->max_sectors for SCSI). Whereas q->limit.max_sectors is the block layer soft limit for filesystem I/O. That value should not be used to set blk_queue_max_hw_sectors(). Nor should queue limits currently in effect be used to configure what is essentially a hardware capability.
I suspect you need to clamp the libata dev->max_sectors value to sdev->host->max_sectors.