Perhaps it's not so obvious, so I should also mention my array was a raid5 array, so I could only fail and remove ONE disk in this procedure, even though I was dropping from 6 to 3 drives. Failing and removing more than 1 drive in a RAID5 will of course destroy the array and the data.

You should only fail and remove the remaining drives (2 in this case) AFTER the array finishes reshaping.