ObsidianVault/Running Start/CSB320 - Machine Learning Concepts/Class 5-21 (Ensemble Models).md

• The average of guesses of a large number of people will typically be more accurate than an individual can be
  • (wisdom of crowds)
• Bootstrapping
  • generates simulated samples by sampling with replacement from existing sample
  • sampling is random
  • Out of bag sample
    • data that was never picked in bootstrapping
    • is often used for testing data (OOB samples were not in the training samples so the model has never seen them)
    • with common techniques often ~30% of data is not picked (and forms OOB set)
  • distribution of classes with bootstrapping will form a Gaussian distribution
  • when enough samples are used bootstrap distributions will approximate population statistics
    • Assumption is made with bootstrapping that the sample approximates the original population
  • bootstrap samples are used as training sets, OOB samples serve as testing sets

bootstrap_samples = [resample(df, replace=True, random_state=i) for i in range(5)]