In this work we propose to regularize deep neural nets with a new type of multitask learning where the auxiliary task is formed by agglomerating classes into super-classes. As such, it is possible to jointly train the network on the class-based classification problem AND super-class based classification problem. We study this strategy in settings with small training sets. This regularization, concurrently with a scheme of randomly reinitializing weights in deeper layers, leads to competitive results with respect to the state of the art.