The hardware seems to have a race condition when the inactive
channels are in slave mode. We support master mode only, so
we can just switch all channels to master mode.

Signed-off-by: Sascha Hauer <s.hauer@pengutronix.de>
[ukleinek: add more verbose comment about the race]
Signed-off-by: Uwe Kleine-König <u.kleine-koenig@pengutronix.de>
Signed-off-by: Grant Likely <grant.likely@secretlab.ca>