Discriminative Vs Generative Models

Reading through analytics vidhya , following are the major points to take away:

• Machine Learning models can be classified into two categories: (a) Discriminative, and (b) Generative
• Discriminative model tries to draw boundaries in the data space whereas Generative model tries to model how the data is placed/distributed.

---

• Discriminative Models:
  • use (conditional) probability to discriminate the datasets.
  • Data sets can be unseen.
  • Used to make predictions, and used for classification and regression problems.
  • unable to generate new data points, since they are focused only on generating boundaries
  • mostly used for supervised learning tasks
• Generative Models:
  • focuses on the distribution of datasets
  • to find the probability
  • they are able to generate new data points similar to training datasets since they capture the distribution of the dataset
  • mostly suited for unsupervised learning tasks

---

• Presents an example of spam email filtering using the two methods
• Discriminative Models:
  • Tries to define the boundary condition of what defines the spam vs not spam email, using training datasets
  • i.e. somehow find the P(Y|X), Y being probability of email being spam, X being words in the email.
• Generative Models:
  • derives the conditional probability that P(X|Y) and P(Y) from general observation or training datasets
  • uses Bayes Theorem to calculate P(Y|X) after P(X|Y) and P(Y) is estimated

---

• Some more points on each
• Discriminative Models:
  • Some of the popular examples are:
    • Logistic regression, SVMs, NNs, Nearest Neighbour, Desicion Trees, Random Forests etc
  • Impacted negligibly by outlier.
• Generative Models:
  • they often rely on Bayes Theorem, so they can solve complex tasks than analgous discriminative model
  • Examples are:
    • Naiive Bayes, Bayesian Network, Markov Random fields, Hidden Markov Model, Latent Dirichlet Allocation, Generative Adversarial Networks, Autoregressive Model
  • Impacted highly by any outlier.

Reading through turing.com , following are the major take aways:

• They present an exmaple of topper twins, who are both toppers in their class. But, they have a different approach to preparing for tests.
• The first one
  • goes into detail of each topic,
  • once he grasps, never forgets it
  • but the process is cumbersome, and needs more time to prepare
• The second one
  • gets a general idea about any topic
  • learns the differences and patterns of each subtopics,
  • has more flexibility in thinking
• Here the first is analogous to Generative Modeling while the second one is analogous to Discriminative modeling.
• Deep Learning algo like ANN, CNN, RNN use Discriminative approach.
• Deep Learning algo like autoencoder, Botlzmann machine, self organizing maps use Generative approach

Having read through the above topics and after creating a faint idea, the google developer page was a very good source to learn. It can be found at this link .