Silkie coop is progressing, however, I've hit a snag...

I'm repurposing a former doghouse into a coop for my 3 Silkies. I've reached the point where I just need to finish getting it painted, put the hardware back on and the final step will be reshingling the roof. The snag I've run up against is that hubby handed me the calk gun with a tube of white silicone in lieu of paintable calk in it. I wasn't paying attention and used the silicone to seal up some cracks, etc. Now that I'm in the process of painting, the paint doesn't want to stick to the places where the silicone is (gives those areas a kind of "crackled" look). Do I have to rip out the silicone and recalk those areas or, would it suffice just to put a light coating of paintable calk over the silicone so that the paint sticks?

Pictures of the progress thus far:

Front:



Bay window:



Window opposite the bay window:



Doors, back and front (front will be a chicken door in door for easy access/cleaning):



Please pardon the mess in my wood shed. Most of the materials I've used thus far are repurposed. I probably have about $80 total into the project so far, $50 of that was for the doghouse itself.

Thanks!