It's not necessarily that you can't - but if you're trying to breed pure birds you wouldn't want to.
The Lavender gene is a recessive blue dilution gene. Think of it as a coat of paint. The bird is actually black underneath; but the Lavender gene is painting a coat of color overtop it.
Being as Lavender is a recessive, any cross of Lavender over non-Lavender will yield non-Lavender chicks. Instead, the Black will show. Now, Black tends to be quite dominant over most colors; the exception would be Buff. Most Buff/Black crosses tend to show black on the lower half of the body with TONS of Buff leakage on the upper half.
The second issue is that all the offspring are Lavender carries. They carry one copy of the gene, and while they do not physically (phenotypically) show it, they can and will pass it on to their offspring. If these Lav carriers are bred to non-Lav, non-Lav carrier birds, some of the offspring will carry it and none will show it. If the Lav carriers are bred to Lav or Lav-carrier birds, some offspring will show Lavender and the rest will carry it.
Here's the problem with these. Lavender dilutes black to Lavender, but it also dilutes red to a soft yellow-bronze. So these offspring, having had recent Buff inclusion, will have ton of red/gold leakage, which will result in diluted yellow-bronze coloring in the Lavender plumage. This color is considered very undesirable in Lavenders.
