Yes, but not for to long. Since the grower feed isn't full of the calcium and proper nutrients for them, you should offer oyster shell to the hens. The calcium helps give strong egg shells.
Increased calcium is the main difference between layer and the other types of feeds. Layer feed is never a requirement, especially with oyster shells available. Since I have chicks right now I don't buy layer feed at all, but once everyone is laying I'll do half layer, half grower, since I like the layer pellets but want to supplement protein just a little with the grower.