I haven't found that to be true.
We don't give leftovers every day though. Maybe every 2-3 days.
If the protein content of the total feed drops too much, maybe that could be the cause of your problem. For example, if you give huge amounts of spaghetti- then the next day huge amounts of rice-and a loaf of bread on the third day, it could make the protein level go down enough that they don't lay as well theoretically.
Wheat I have found has around 12-14% protein (white is 12 and red is 14). Layer feed is 16%.
Generally the higher the protein content the larger your eggs will be, according to a fellow BYC'er in commercial egg production.