SOTAVerified

Knowledge Distillation

Knowledge distillation is the process of transferring knowledge from a large model to a smaller one. While large models (such as very deep neural networks or ensembles of many models) have higher knowledge capacity than small models, this capacity might not be fully utilized.

Papers

Recently AddedMost HypedMost ActiveNeeds VerificationMost Verified

Showing 2901–2925 of 4240 papers

TitleStatusHype
Classroom-Inspired Multi-Mentor Distillation with Adaptive Learning Strategies—0
Class Similarity Weighted Knowledge Distillation for Continual Semantic Segmentation—0
Class Token and Knowledge Distillation for Multi-head Self-Attention Speaker Verification Systems—0
Collaborative Learning for Enhanced Unsupervised Domain Adaptation—0
CleverDistiller: Simple and Spatially Consistent Cross-modal Distillation—0
CLFace: A Scalable and Resource-Efficient Continual Learning Framework for Lifelong Face Recognition—0
Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learning—0
CLIP-CID: Efficient CLIP Distillation via Cluster-Instance Discrimination—0
CLIPPING: Distilling CLIP-Based Models With a Student Base for Video-Language Retrieval—0
CLoCKDistill: Consistent Location-and-Context-aware Knowledge Distillation for DETRs—0
Closing the Gap between Client and Global Model Performance in Heterogeneous Federated Learning—0
Cloud-Device Collaborative Learning for Multimodal Large Language Models—0
CL-ReKD: Cross-lingual Knowledge Distillation for Multilingual Retrieval Question Answering—0
ClST: A Convolutional Transformer Framework for Automatic Modulation Recognition by Knowledge Distillation—0
CMU’s IWSLT 2022 Dialect Speech Translation System—0
CoCo DistillNet: a Cross-layer Correlation Distillation Network for Pathological Gastric Cancer Segmentation—0
CoDERT: Distilling Encoder Representations with Co-learning for Transducer-based Speech Recognition—0
Cold & Warm Net: Addressing Cold-Start Users in Recommender Systems—0
Collaborative Distillation for Top-N Recommendation—0
Collaborative Distillation in the Parameter and Spectrum Domains for Video Action Recognition—0
Collaborative Inter-agent Knowledge Distillation for Reinforcement Learning—0
Collaborative Learning for Deep Neural Networks—0
Collaborative Multi-Teacher Knowledge Distillation for Learning Low Bit-width Deep Neural Networks—0
Collaborative Teacher-Student Learning via Multiple Knowledge Transfer—0
CoLLD: Contrastive Layer-to-layer Distillation for Compressing Multilingual Pre-trained Speech Encoders—0
Show:102550
← PrevPage 117 of 170Next →
All datasetsImageNetCIFAR-100COCO (Common Objects in Context)COCO 2017 valPASCAL VOCKITTI

Benchmark Results

#ModelMetricClaimedVerifiedStatus
1ScaleKD (T:BEiT-L S:ViT-B/14)Top-1 accuracy %86.43—Unverified
2ScaleKD (T:Swin-L S:ViT-B/16)Top-1 accuracy %85.53—Unverified
3ScaleKD (T:Swin-L S:ViT-S/16)Top-1 accuracy %83.93—Unverified
4ScaleKD (T:Swin-L S:Swin-T)Top-1 accuracy %83.8—Unverified
5KD++(T: regnety-16GF S:ViT-B)Top-1 accuracy %83.6—Unverified
6VkD (T:RegNety 160 S:DeiT-S)Top-1 accuracy %82.9—Unverified
7SpectralKD (T:Swin-S S:Swin-T)Top-1 accuracy %82.7—Unverified
8ScaleKD (T:Swin-L S:ResNet-50)Top-1 accuracy %82.55—Unverified
9DiffKD (T:Swin-L S: Swin-T)Top-1 accuracy %82.5—Unverified
10DIST (T: Swin-L S: Swin-T)Top-1 accuracy %82.3—Unverified
#ModelMetricClaimedVerifiedStatus
1SRD (T:resnet-32x4, S:shufflenet-v2)Top-1 Accuracy (%)79.86—Unverified
2shufflenet-v2(T:resnet-32x4, S:shufflenet-v2)Top-1 Accuracy (%)78.76—Unverified
3MV-MR (T: CLIP/ViT-B-16 S: resnet50)Top-1 Accuracy (%)78.6—Unverified
4resnet8x4 (T: resnet32x4 S: resnet8x4)Top-1 Accuracy (%)78.28—Unverified
5resnet8x4 (T: resnet32x4 S: resnet8x4 [modified])Top-1 Accuracy (%)78.08—Unverified
6ReviewKD++(T:resnet-32x4, S:shufflenet-v2)Top-1 Accuracy (%)77.93—Unverified
7ReviewKD++(T:resnet-32x4, S:shufflenet-v1)Top-1 Accuracy (%)77.68—Unverified
8resnet8x4 (T: resnet32x4 S: resnet8x4)Top-1 Accuracy (%)77.5—Unverified
9resnet8x4 (T: resnet32x4 S: resnet8x4)Top-1 Accuracy (%)76.68—Unverified
10resnet8x4 (T: resnet32x4 S: resnet8x4)Top-1 Accuracy (%)76.31—Unverified
#ModelMetricClaimedVerifiedStatus
1LSHFM (T: ResNet101 S: ResNet50)mAP77.16—Unverified
2LSHFM (T: ResNet101 S: MobileNetV2)mAP73.73—Unverified
3ADLIK-Faster (T: Faster R-CNN vit-base S: Faster R-CNN deit-small) box AP47.6—Unverified
4ADLIK-Mask (T: Mask R-CNN vit-base S: Mask R-CNN deit-small)mask AP42.4—Unverified
#ModelMetricClaimedVerifiedStatus
1ReviewKD++(T: faster rcnn(resnet101), S:faster rcnn(resnet50))AP@0.561.8—Unverified
2ReviewKD++(T: faster rcnn(resnet101), S:faster rcnn(resnet18))AP@0.557.96—Unverified
3ReviewKD++(T: faster rcnn(resnet101), S:faster rcnn(mobilenet-v2))AP@0.555.18—Unverified
#ModelMetricClaimedVerifiedStatus
1LSHFM (T: ResNet101 S: ResNet50)mAP93.17—Unverified
2LSHFM (T: ResNet101 S: MobileNetV2)mAP90.14—Unverified
#ModelMetricClaimedVerifiedStatus
1TIE-KD (T: Adabins S: MobileNetV2)RMSE2.43—Unverified