Model Compression

Model Compression is an actively pursued area of research over the last few years with the goal of deploying state-of-the-art deep networks in low-power and resource limited devices without significant drop in accuracy. Parameter pruning, low-rank factorization and weight quantization are some of the proposed methods to compress the size of deep networks.

Source: KD-MRI: A knowledge distillation framework for image reconstruction and image restoration in MRI workflow

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 276–300 of 1356 papers

Title	Date	Tasks	Status	Score
Paraphrasing Complex Network: Network Compression via Factor Transfer	Feb 14, 2018	Model CompressionTransfer Learning	CodeCode Available	5
Is Modularity Transferable? A Case Study through the Lens of Knowledge Distillation	Mar 27, 2024	Domain AdaptationKnowledge Distillation	CodeCode Available	5
InDistill: Information flow-preserving knowledge distillation for model compression	May 20, 2022	Knowledge DistillationModel Compression	CodeCode Available	5
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression	Jul 6, 2024	Language ModelingLanguage Modelling	CodeCode Available	5
Information-Theoretic Understanding of Population Risk Improvement with Model Compression	Jan 27, 2019	ClusteringModel Compression	CodeCode Available	5
Iterative Filter Pruning for Concatenation-based CNN Architectures	May 4, 2024	Model Compression	CodeCode Available	5
Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning	Nov 19, 2015	Atari GamesDeep Reinforcement Learning	CodeCode Available	5
JavaScript Convolutional Neural Networks for Keyword Spotting in the Browser: An Experimental Analysis	Oct 30, 2018	Keyword SpottingModel Compression	CodeCode Available	5
Bayesian Tensorized Neural Networks with Automatic Rank Selection	May 24, 2019	Model CompressionTensor Decomposition	CodeCode Available	5
I3D: Transformer architectures with input-dependent dynamic depth for speech recognition	Mar 14, 2023	Model Compressionspeech-recognition	CodeCode Available	5
HTR-JAND: Handwritten Text Recognition with Joint Attention Network and Knowledge Distillation	Dec 24, 2024	Computational EfficiencyHandwritten Text Recognition	CodeCode Available	5
HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression	Oct 16, 2021	Few-Shot LearningKnowledge Distillation	CodeCode Available	5
Hybrid Binary Networks: Optimizing for Accuracy, Efficiency and Memory	Apr 11, 2018	BinarizationModel Compression	CodeCode Available	5
Image Classification with CondenseNeXt for ARM-Based Computing Platforms	Jun 26, 2021	Autonomous DrivingClassification	CodeCode Available	5
How does topology of neural architectures impact gradient propagation and model performance?	Jun 16, 2021	Model CompressionNeural Architecture Search	CodeCode Available	5
High-fidelity 3D Model Compression based on Key Spheres	Jan 19, 2022	Model CompressionObject	CodeCode Available	5
A Miniaturized Semantic Segmentation Method for Remote Sensing Image	Oct 27, 2018	Model CompressionSemantic Segmentation	CodeCode Available	5
ImPart: Importance-Aware Delta-Sparsification for Improved Model Compression and Merging in LLMs	Apr 17, 2025	Model CompressionQuantization	CodeCode Available	5
Gradual Channel Pruning while Training using Feature Relevance Scores for Convolutional Neural Networks	Feb 23, 2020	Model Compression	CodeCode Available	5
GSB: Group Superposition Binarization for Vision Transformer with Limited Training Samples	May 13, 2023	BinarizationKnowledge Distillation	CodeCode Available	5
A Brief Review of Hypernetworks in Deep Learning	Jun 12, 2023	Causal InferenceContinual Learning	CodeCode Available	5
AutoMC: Automated Model Compression based on Domain Knowledge and Progressive search strategy	Jan 24, 2022	Model Compression	CodeCode Available	5
Generalizing Teacher Networks for Effective Knowledge Distillation Across Student Architectures	Jul 22, 2024	Knowledge DistillationModel Compression	CodeCode Available	5
GASL: Guided Attention for Sparsity Learning in Deep Neural Networks	Jan 7, 2019	Model CompressionNetwork Pruning	CodeCode Available	5
From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression	Dec 14, 2021	Contrastive LearningLanguage Modeling	CodeCode Available	5

Show:10 25 50

← PrevPage 12 of 55Next →

All datasets ImageNet QNLI

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	ADLIK-MO-ResNet50+W4A4	Top-1	77.88	—	Unverified
2	ADLIK-MO-ResNet50+W3A4	Top-1	77.34	—	Unverified
3	ResNet-18 + 4bit-1dim model compression using DKM	Top-1	70.52	—	Unverified
4	MobileNet-v1 + 4bit-1dim model compression using DKM	Top-1	69.63	—	Unverified
5	ResNet-18 + 2bit-1dim model compression using DKM	Top-1	68.63	—	Unverified
6	MobileNet-v1 + 2bit-1dim model compression using DKM	Top-1	67.62	—	Unverified
7	ResNet-18 + 4bit-4dim model compression using DKM	Top-1	66.1	—	Unverified
8	ResNet-18 + 2bit-2dim model compression using DKM	Top-1	64.7	—	Unverified
9	MobileNet-v1 + 4bit-4dim model compression using DKM	Top-1	61.4	—	Unverified
10	ResNet-18 + 1bit-1dim model compression using DKM	Top-1	59.7	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	MobileBERT + 2bit-1dim model compression using DKM	Accuracy	82.13	—	Unverified
2	MobileBERT + 1bit-1dim model compression using DKM	Accuracy	63.17	—	Unverified