BoldogKennel,
Chromosome number is not the valuable tool used to distinguish species as was once thought and chromosome number is probably not the real mechanism blocking fertility between populations / species that otherwise produce viable offspring.. I work with sunfishes (bluegill, redear and several others from time to time) and make hybrids for one reason or another according to need. Most of the "species" have the same number of chromosome pairs yet fertility issues exist between species. Within the green sunfish Lepomis cyanellus at least, individuals can vary in terms of the number of chromome pairs they have and still produce fertile offspring with normal fertility as far as I can tell.
What blocks fertility of hybrid offspring is how genetic information is arranged on and between chromosomes. When gametes are produced during meiosis the chromosome line up so similar stretches of DNA (coding for same loci) also line up which facilitates rearranging alleles between chromosome pairs. Problem is that with species / populations that are not closely related, complementory stretches are moved about and on different chromosomes making proper line up in hybrid offspring difficult if not impossible. When the swapping does occur it results in some chromosomes loosing stretches of DNA and others gaining. Loosing or gaining too many loci or the wrong ones can be very bad for individuals that might be produced by the fusion of such affected gametes. If problem is bad enough, then the gamete production process is aborted. If extent is lesser, gametes can still be produced but they tend to result in abnormal offspring. In the latter case, when two hybrids are mated together the survival of their offspring tends to be very low but if such individuals are backcrossed to either parental or even crossed with another species, the survival is quite a bit better because the pure species brings into mess half of the intact genetic information needed to make a viable individual which is enough if gametes of hybrid brings most nearly or just a little more than half of what is needed.