TL;DR: COUNT(*) is optimized to be fast, you should use it. You have probably read in a bunch of different places that you shouldn't use SELECT(*) in MySQL when you don't need all the data.
While SELECT(*) selects all the columns, COUNT(*) is specifically optimized to count all the rows.
If a secondary index is not present, InnoDB processes SELECT COUNT(*) statements by scanning the clustered index.
It contains the indexed data and a pointer to the primary key (clustered index) where you can get the rest of the data.