Some firmware revisions don't notify host about
self-bss-peer removal until after associated vdev
is deleted. This has been observed with qca6174
WLAN.RM.2.0-00073 firmware.

This patch fixes AP teardown slowdowns and
prevents delays and warnings:

 ath10k_pci 0000:00:05.0: failed to remove peer for AP vdev 0: -110
 ath10k_pci 0000:00:05.0: removing stale peer xx:xx:xx:xx:xx:xx from vdev_id 0
 ath10k_pci 0000:00:05.0: peer-unmap-event: unknown peer id 24
 ath10k_pci 0000:00:05.0: peer-unmap-event: unknown peer id 8

Signed-off-by: Michal Kazior <michal.kazior@tieto.com>
Signed-off-by: Kalle Valo <kvalo@qca.qualcomm.com>