Well, they do often hang in pairs when they're on a tree. Also, some avocados have a sort of wrinkly skin. So I can see how a squinting Aztec predecessor of ours may have seen a resemblance.

But maybe you're right and we're missing something about Aztec biology. Maybe they had green testicles...

;-)