Depending on how much you use, I would be on the lookout about the salt content for a couple of reasons. One is that if you put it in the compost, you have now put sodium in the compost. A little is no big deal, but sodium has a tendency to break down the structure of the soil, and cause plants to dehydrate.
I love the idea, by the way. Simple. Elegant. It just is a matter of watching that you use the least amount that works, and check to make sure there are no repercussions as Denninmi pointed out with the ducks, and with any plants that may receive baking soda in their soil amendments.
For similar reasons, I am thinking hard about anything I add to their water and bedding. I am avoiding ACV in their swim water and it is working out fine - I just scrub the pans every few days, and once every couple of weeks I dry them in the sun. That is because the water gets dumped into a mini canal that carries the fertilized water into my grape arbor. The grapes just adore the fertigation! But vinegar is something I have used to kill weeds, so I don't want vinegar in that water.
For their bedding (which doesn't get really wet because I don't give them water in their house) I add peat moss. Lowering the pH reduces the production of ammonia in compost, and I use a variation of the deep litter method so ammonia is a concern. But not when I use peat moss! That is working out well, and is just fine for the plants when I replace the bedding, which I do not expect to be needed often.