No it's furthur down in the back near the tail. You have to dig around to scrape it out. Usually in bony pockets. It's hard to explain but if the chicken is breast up with legs toward you (like when you're gutting it) and you run two fingers down the iside toward you, it looks like the floor of the birds cavity is solid but if you push firmly down with your fingers you'll feel the pockets where the gunk is. Lots of little ligaments in there near the pelvis and spine
Those are the kidneys, it's not a problem to leave them in the back, or you can remove them. They taste like liver (not a positive for many people), but given what I know about the function if kidneys (filter out toxins for one thing) I wouldn't eat them now.
If you buy chicken thighs, you find this livery thing attached to the pocket of the bone near the backbone. I think those are kidneys. I love that piece actually, because its like liver. It sounds like its that piece. Take a whole chicken leg including the thighs and you will usually find it near the backbone.