Do you have any issues with the flapping of a tarp freaking the chickens out or has the tarp ever ripped and flew into the coop? Those were two potential issues I wondered about when considering a tarp. Maybe it's just a matter of securing it well enough.
Or do you have a screen or fencing on the open side? I won't--our open side will be wide open to a secure run/yard.
Such things are secured well, but if they do wave or flap in the wind a bit, the chickens grow used to it.