Network Pruning

Network Pruning is a popular approach to reduce a heavy network to obtain a light-weight form by removing redundancy in the heavy network. In this approach, a complex over-parameterized network is first trained, then pruned based on come criterions, and finally fine-tuned to achieve comparable performance with reduced parameters.

Source: Ensemble Knowledge Distillation for Learning Improved and Efficient Networks

Papers

Recently Added Most Hyped Most Active Needs Verification Most Verified

Showing 76–100 of 534 papers

Title	Date	Tasks	Status	Hype
Picking the Underused Heads: A Network Pruning Perspective of Attention Head Selection for Fusing Dialogue Coreference Information	Dec 15, 2023	Network Pruning	—Unverified	0
Neural Architecture Codesign for Fast Bragg Peak Analysis	Dec 10, 2023	AutoMLModel Compression	—Unverified	0
Accelerating Convolutional Neural Network Pruning via Spatial Aura Entropy	Dec 8, 2023	Computational EfficiencyNetwork Pruning	—Unverified	0
An End-to-End Network Pruning Pipeline with Sparsity Enforcement	Dec 4, 2023	Network Pruning	—Unverified	0
Towards Higher Ranks via Adversarial Weight Pruning	Nov 29, 2023	Model CompressionNetwork Pruning	CodeCode Available	0
LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS	Nov 28, 2023	Knowledge DistillationNeRF	CodeCode Available	2
Filter-Pruning of Lightweight Face Detectors Using a Geometric Median Criterion	Nov 28, 2023	Face DetectionNetwork Pruning	CodeCode Available	1
Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruning	Nov 23, 2023	Data AugmentationKnowledge Distillation	—Unverified	0
Neural Network Pruning by Gradient Descent	Nov 21, 2023	Computational EfficiencyDeep Learning	CodeCode Available	0
Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks	Nov 21, 2023	Network Pruning	—Unverified	0
Data Augmentations in Deep Weight Spaces	Nov 15, 2023	Data AugmentationNetwork Pruning	—Unverified	0
Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks	Nov 15, 2023	BenchmarkingNetwork Pruning	CodeCode Available	0
Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models	Nov 8, 2023	Language ModellingNetwork Pruning	CodeCode Available	1
Brain-Inspired Efficient Pruning: Exploiting Criticality in Spiking Neural Networks	Nov 5, 2023	Network Pruning	—Unverified	0
Efficient Model-Based Deep Learning via Network Pruning and Fine-Tuning	Nov 3, 2023	Deep LearningImage Reconstruction	CodeCode Available	0
Importance Estimation with Random Gradient for Neural Network Pruning	Oct 31, 2023	Network Pruning	—Unverified	0
SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity	Oct 30, 2023	Network Pruning	CodeCode Available	0
Linear Mode Connectivity in Sparse Neural Networks	Oct 28, 2023	Dataset DistillationLinear Mode Connectivity	—Unverified	0
GraFT: Gradual Fusion Transformer for Multimodal Re-Identification	Oct 25, 2023	Network PruningRepresentation Learning	—Unverified	0
Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs	Oct 13, 2023	Network Pruning	CodeCode Available	1
Samples on Thin Ice: Re-Evaluating Adversarial Pruning of Neural Networks	Oct 12, 2023	Network Pruning	—Unverified	0
Filter Pruning For CNN With Enhanced Linear Representation Redundancy	Oct 10, 2023	channel selectionNetwork Pruning	CodeCode Available	0
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity	Oct 8, 2023	Network Pruning	CodeCode Available	1
SWAP: Sparse Entropic Wasserstein Regression for Robust Network Pruning	Oct 7, 2023	Network Pruningregression	CodeCode Available	0
Feather: An Elegant Solution to Effective DNN Sparsification	Oct 3, 2023	Network Pruning	CodeCode Available	1

Show:10 25 50

← PrevPage 4 of 22Next →

All datasets ImageNet ImageNet - ResNet 50 - 90% sparsity CIFAR-100 CIFAR-10 MNIST

Benchmark Results

#	Model	Metric	Claimed	Verified	Status
1	ResNet50-2.3 GFLOPs	Accuracy	78.79	—	Unverified
2	ResNet50-1.5 GFLOPs	Accuracy	78.07	—	Unverified
3	ResNet50 2.5 GFLOPS	Accuracy	78	—	Unverified
4	RegX-1.6G	Accuracy	77.97	—	Unverified
5	ResNet50 2.0 GFLOPS	Accuracy	77.7	—	Unverified
6	ResNet50-3G FLOPs	Accuracy	77.1	—	Unverified
7	ResNet50-2G FLOPs	Accuracy	76.4	—	Unverified
8	ResNet50-1G FLOPs	Accuracy	76.38	—	Unverified
9	TAS-pruned ResNet-50	Accuracy	76.2	—	Unverified
10	ResNet50	Accuracy	75.59	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	Feather	Top-1 Accuracy	76.93	—	Unverified
2	Spartan	Top-1 Accuracy	76.17	—	Unverified
3	ST-3	Top-1 Accuracy	76.03	—	Unverified
4	AC/DC	Top-1 Accuracy	75.64	—	Unverified
5	CS	Top-1 Accuracy	75.5	—	Unverified
6	ProbMask	Top-1 Accuracy	74.68	—	Unverified
7	STR	Top-1 Accuracy	74.31	—	Unverified
8	DNW	Top-1 Accuracy	74	—	Unverified
9	GMP	Top-1 Accuracy	73.91	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	+U-DML*	Inference Time (ms)	675.56	—	Unverified
2	Dense	Accuracy	79	—	Unverified
3	AC/DC	Accuracy	78.2	—	Unverified
4	Beta-Rank	Accuracy	74.01	—	Unverified
5	TAS-pruned ResNet-110	Accuracy	73.16	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	TAS-pruned ResNet-110	Accuracy	94.33	—	Unverified
2	ShuffleNet – Quantised	Inference Time (ms)	23.15	—	Unverified
3	AlexNet – Quantised	Inference Time (ms)	5.23	—	Unverified
4	MobileNet – Quantised	Inference Time (ms)	4.74	—	Unverified

#	Model	Metric	Claimed	Verified	Status
1	FFN-ShapleyPruned	Avg #Steps	12.05	—	Unverified