So I haven't watched Zoey's Extraordinary Playlist, but I did religiously watch Crazy Ex-Girlfriend, which also had full musical numbers every episode. CXG took good advantage of its cast's differing capabilities both vocally and physically, but tbh a number of its lead cast are Broadway, musical theatre vets. But even then, you're gonna have differing levels of capability on the triple threat spectrum. Vincent Rodriguez, who played Josh, has a solid voice but is a dancer, and he was given several dance numbers over the years to show that off. Santino Fontana (the original Josh) and Donna Lynne Champlin (Paula) both have ridiculous voices and they're strong movers, but neither of them are known as dancers in New York. (David Hull, who played White Josh, is a really strong dancer, but actually barely ended up showing it off.) CXG's choreography was also fantastic at evoking and honing in on specific styles. The other thing it did was just -- you know, have backup dancers filling out a number, as solos were the more common. When someone is just there to dance, you get someone who's fantastic at dancing.
I cannot speak to Zoey's choreography. But CXG did it very well! (Hello, I loved CXG so very, very much, and everyone should watch it.)