Even though you think you're only running a single app at a time, that doesn't mean only a single thread is running. Getting the UI from that app onto the screen is actually done by a second process, called Surface Flinger. Even within that one app you're using, there are probably multiple threads: one drawing the UI and responding to your input, another communicating with a server, and perhaps another doing whatever slow computation the app needs to do. Having multiple threads allows the UI to remain responsive while performing these slow tasks. On top of that, other apps run in the background sometimes: WhatsApp is listening for new messages, Gmail is checking for new mail, Google Now is checking updated stock prices, Google Play is downloading an app update, etc.
All in all, there's no single ideal number of CPU cores that suits every situation. A lot of the drive towards increasing number of cores on phones is marketing, just like the clock speed wars on PCs 10 years ago. But there's an extra factor that makes it worthwhile having these huge numbers of cores: dark silicon.
The silicon area on the phone's system-on-chip that's taken up by a single CPU core has got smaller and smaller over the years. Most of the space is taken up by RAM, and even the GPU is several times larger than the whole CPU. It doesn't cost much to just put more CPU cores in the silicon. Most of the time, if your phone isn't doing very much, those extra cores can be turned off to save power. But then when you want to play a mobile game, or run a Snapchat filter, or anything compute-intensive that wants to run several threads, those extra cores can be powered up for a burst of speed.
So really, the ideal number of cores is lots. Even if some people never run the kind of workload that benefits from lots of cores, there's almost zero cost to having unused cores, and there's a huge benefit for some workloads.