COMPUTER VISION

CORE MACHINE LEARNING

Learning from Few Samples: Transformation-Invariant SVMs with Composition and Locality at Multiple Scales

January 04, 2023

Abstract

Motivated by the problem of learning with small sample sizes, this paper shows how to incorporate into support-vector machines (SVMs) those properties that have made convolutional neural networks (CNNs) successful. Particularly important is the ability to incorporate domain knowledge of invariances, e.g., translational invariance of images. Kernels based on the maximum similarity over a group of transformations are not generally positive definite. Perhaps it is for this reason that they have not been studied theoretically. We address this lacuna and show that positive definiteness indeed holds with high probability for kernels based on the maximum similarity in the small training sample set regime of interest, and that they do yield the best results in that regime. We also show how additional properties such as their ability to incorporate local features at multiple spatial scales, e.g., as done in CNNs through max pooling, and to provide the benefits of composition through the architecture of multiple layers, can also be embedded into SVMs. We verify through experiments on widely available image sets that the resulting SVMs do provide superior accuracy in comparison to well-established deep neural network benchmarks for small sample sizes.

Download the Paper

AUTHORS

Written by

Xi Liu

Panganamala Kumar

Ruida Zhou

Tao Liu

Publisher

NeurIPS

Research Topics

Computer Vision

Core Machine Learning

Related Publications

March 09, 2023

COMPUTER VISION

The Casual Conversations v2 Dataset

Bilal Porgali, Vítor Albiero, Jordan Ryda, Cristian Canton Ferrer, Caner Hazirbas

March 09, 2023

February 28, 2023

CORE MACHINE LEARNING

On the duality between contrastive and non-contrastive self-supervised learning

Quentin Garrido, Adrien Bardes, Yann LeCun, Yubei Chen, Laurent Najman

February 28, 2023

February 21, 2023

COMPUTER VISION

CORE MACHINE LEARNING

ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks

Felix Xu, Fuyuan Zhang, Hua Qi, Jianjun Zhao, Jianlang Chen, Lei Ma, Qing Guo, Zhijie Wang

February 21, 2023

January 10, 2023

COMPUTER VISION

CORE MACHINE LEARNING

Online Backfilling with No Regret for Large-Scale Image Retrieval

Gokhan Uzunbas, Joena Zhang, Sara Cao, Ser-Nam Lim, Taipeng Tian, Bohyung Han, Seonguk Seo

January 10, 2023

Help Us Pioneer The Future of AI

We share our open source frameworks, tools, libraries, and models for everything from research exploration to large-scale production deployment.