Machine learning: II. Neural networks

Neural networks #

Neural networks form the backbone of today’s machine learning. At the most fundemental level, a neural network is a novel paradigm for representing functions. The image below represents a simple example of a neural network \(\mathcal{N}:\mathbb{R} \rightarrow \mathbb{R}\):

A basic neural network.

We will focus on two questions:

  • How expressive is the hypothesis class of neural networks? In particular, can we aproximate any continuous function \(f:\mathbb{R}^n \rightarrow \mathbb{R}^m\) via a neural network?
  • How do we effectively find a neural network representation for a given function or data set?