Zero-shot Learning for Subdiscrimination in Pre-trained Models

Abstract

In deep metric learning (DML) high-level input data are represented in a lower-level representation (embedding) space, such that samples from the same class are mapped close together, while samples from disparate classes are mapped further apart. In this lower-level representation, only a single inference sample from each known class is required to accurately discriminate between classes. To this end, embeddings trained for a specific task may contain additional feature information which can be used to go a level deeper into the discrimination task, i.e. allowing for feature sub-discrimination. This study takes an embedding trained to discriminate faces (identities) and uses the inherent feature information within the embedding to differentiate several attributes such as gender, age, and skin tone, without any additional training. This study is split into two cases; intra class discrimination where all the embeddings considered are from the same identity/in-dividual but with minor attributes such as beard/beardless, glasses/without glasses and emotions; and extra class discrimination where the embeddings represent different identities/people with more prominent attributes such as male/female, pale/dark tone, young/older. In the intra class sub-discriminant scenario, the inference process distinguishes common attributes and several artefacts of different identities, achieving 90.0% and 76.0% accuracy for beards and glasses, respectively. The system can also perform extra class sub-discrimination with a high accuracy rate, notably 99.3%, 99.3% and 94.1% for gender, skin tone, and age, respectively. To sum up, this work investigates the sub-discriminative capabilities of DML models by clustering discriminative features evident within the structure of DML embeddings.