That's pretty much it, my Sony Xperia Z3 Compact came with AVG antivirus preinstalled and I can't see how to uninstall it, since it does not show up in Play Store under My Apps / Installed (or anywhere else). My Android version is 4.4.4 .
Note that my problem is not the one mentioned in this question. In my case under
Settings > Security > Device administrators the checkbox next to AVG was already unchecked.
Instead my problem is simply that I don't know where to start the uninstallation process (and I don't understand why all my installed apps do show up as expected under My Apps but AVG does not).