Network Pruning

Network Pruning is a popular approach to reduce a heavy network to obtain a light-weight form by removing redundancy in the heavy network. In this approach, a complex over-parameterized network is first trained, then pruned based on come criterions, and finally fine-tuned to achieve comparable performance with reduced parameters.

Source: Ensemble Knowledge Distillation for Learning Improved and Efficient Networks

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 51–75 of 534 papers

Title	Date	Tasks	Status	Hype	Score
1xN Pattern for Pruning Convolutional Neural Networks	May 31, 2021	CPUNetwork Pruning	CodeCode Available	1	5
Membership Inference Attacks and Defenses in Neural Network Pruning	Feb 7, 2022	Inference AttackMembership Inference Attack	CodeCode Available	1	5
Adversarial Pruning: A Survey and Benchmark of Pruning Methods for Adversarial Robustness	Sep 2, 2024	Adversarial RobustnessNetwork Pruning	CodeCode Available	1	5
DHP: Differentiable Meta Pruning via HyperNetworks	Mar 30, 2020	AutoMLDenoising	CodeCode Available	1	5
AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks	Jun 23, 2021	Network Pruning	CodeCode Available	1	5
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs	Oct 13, 2023	Network Pruning	CodeCode Available	1	5
Neural Pruning via Growing Regularization	Dec 16, 2020	L2 RegularizationModel Compression	CodeCode Available	1	5
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning	Jul 6, 2020	Efficient Neural NetworkNetwork Pruning	CodeCode Available	1	5
Dynamic Channel Pruning: Feature Boosting and Suppression	Oct 12, 2018	Model CompressionNetwork Pruning	CodeCode Available	1	5
NTK-SAP: Improving neural network pruning by aligning training dynamics	Apr 6, 2023	Network Pruning	CodeCode Available	1	5
HYDRA: Pruning Adversarially Robust Neural Networks	Feb 24, 2020	Network Pruning	CodeCode Available	1	5
DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks	Dec 1, 2021	Network Pruning	CodeCode Available	1	5
Discovering Neural Wirings	Jun 3, 2019	Feature EngineeringNetwork Pruning	CodeCode Available	1	5
Automatic Network Pruning via Hilbert-Schmidt Independence Criterion Lasso under Information Bottleneck Principle	Jan 1, 2023	Network Pruning	CodeCode Available	1	5
Effective Sparsification of Neural Networks with Global Sparsity Constraint	May 3, 2021	Network Pruning	CodeCode Available	1	5
A Three-regime Model of Network Pruning	May 28, 2023	Efficient Neural NetworkHyperparameter Optimization	CodeCode Available	1	5
Automatic Neural Network Pruning that Efficiently Preserves the Model Accuracy	Nov 18, 2021	Network Pruning	CodeCode Available	1	5
Efficient Latency-Aware CNN Depth Compression via Two-Stage Dynamic Programming	Jan 28, 2023	Network PruningVocal Bursts Valence Prediction	CodeCode Available	1	5
A Gradient Flow Framework For Analyzing Network Pruning	Sep 24, 2020	Network Pruning	CodeCode Available	1	5
Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch	Mar 21, 2024	Network Pruning	CodeCode Available	1	5
Feather: An Elegant Solution to Effective DNN Sparsification	Oct 3, 2023	Network Pruning	CodeCode Available	1	5
Filter-Pruning of Lightweight Face Detectors Using a Geometric Median Criterion	Nov 28, 2023	Face DetectionNetwork Pruning	CodeCode Available	1	5
Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning	Jun 3, 2024	Network Pruning	CodeCode Available	1	5
Finding Skill Neurons in Pre-trained Transformer-based Language Models	Nov 14, 2022	Network Pruning	CodeCode Available	1	5
How Not to Give a FLOP: Combining Regularization and Pruning for Efficient Inference	Mar 30, 2020	Network Pruning	CodeCode Available	1	5

Show:10 25 50

← PrevPage 3 of 22Next →

All datasets ImageNet ImageNet - ResNet 50 - 90% sparsity CIFAR-100 CIFAR-10 MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	ResNet50-2.3 GFLOPs	Accuracy	78.79	—	Unverified
2	ResNet50-1.5 GFLOPs	Accuracy	78.07	—	Unverified
3	ResNet50 2.5 GFLOPS	Accuracy	78	—	Unverified
4	RegX-1.6G	Accuracy	77.97	—	Unverified
5	ResNet50 2.0 GFLOPS	Accuracy	77.7	—	Unverified
6	ResNet50-3G FLOPs	Accuracy	77.1	—	Unverified
7	ResNet50-2G FLOPs	Accuracy	76.4	—	Unverified
8	ResNet50-1G FLOPs	Accuracy	76.38	—	Unverified
9	TAS-pruned ResNet-50	Accuracy	76.2	—	Unverified
10	ResNet50	Accuracy	75.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Feather	Top-1 Accuracy	76.93	—	Unverified
2	Spartan	Top-1 Accuracy	76.17	—	Unverified
3	ST-3	Top-1 Accuracy	76.03	—	Unverified
4	AC/DC	Top-1 Accuracy	75.64	—	Unverified
5	CS	Top-1 Accuracy	75.5	—	Unverified
6	ProbMask	Top-1 Accuracy	74.68	—	Unverified
7	STR	Top-1 Accuracy	74.31	—	Unverified
8	DNW	Top-1 Accuracy	74	—	Unverified
9	GMP	Top-1 Accuracy	73.91	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	+U-DML*	Inference Time (ms)	675.56	—	Unverified
2	Dense	Accuracy	79	—	Unverified
3	AC/DC	Accuracy	78.2	—	Unverified
4	Beta-Rank	Accuracy	74.01	—	Unverified
5	TAS-pruned ResNet-110	Accuracy	73.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAS-pruned ResNet-110	Accuracy	94.33	—	Unverified
2	ShuffleNet – Quantised	Inference Time (ms)	23.15	—	Unverified
3	AlexNet – Quantised	Inference Time (ms)	5.23	—	Unverified
4	MobileNet – Quantised	Inference Time (ms)	4.74	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	FFN-ShapleyPruned	Avg #Steps	12.05	—	Unverified