#rs/notes #rs/class/csb320
- - -

- Classification models
	- supervised
	- classify instances into class (category)
	- can predict class directly or probabilities
- Each column in data table is feature
	- Each row is sample/tuple/instance
- K-Nearest neighbors
	- lazy learner
		- memorizes data, doesn't create model during training
	- if k=6 it looks at 6 closes neighbors in dataset
		- predicts based on what these are classified as
		- there can be very different predictions based on k
	- gets very expensive as the dataset grows
	- selecting value of k
		- k is a hyperparameter (user set value )
		- typical values of 3-15
		- lower value can be sensitive to noise
		- higher value risks underfitting
	- distance measures
		- euclidean distance
			- good when data is compact and continuous
		- manhattan distance
			- sum of absolute differences in coordinates
			- good when data is discrete or with large distances
		- Minkowski distance
			- includes euclidean and manhattan
			- parameter allows to interpolate between the two
			- can be used for model tuning since distance function can be changed between euclidean and manhattan
		- features should be standardized to ensure fair distance measures
- Logistic regression
	- log-odds: natural log of the probability ratio
	- uses a logistic regression to predict the chances of something being categorized in certain way
	- logistic regression $$\hat{p}=\frac{\exp(w_0 + w_1x_i)}{1 + \exp(w_{0} + w_{1}x_{i})}$$
	- output of logistic regression is compared to threshold T
	- default T = 0.5
	- linear regression will underfit for classifying data, logistic regression is better
	- multiple input features: $$\hat{p}= \frac{\exp(w_{0} + w_{1}x_{1i}+\dots+w_{p}x_{p i})}{1+\exp(w_{0} + w_{1}x_{1i}+\dots+w_{p}x_{p i})}$$
- Gaussian Naive Bayes
	- normal distributions for each outcome
	- Baye's rule: $$P(A|B) = \frac{P(B|A) * P(A)}{P(B)}$$
		- $P(A|B$): Posterior probability
		- Assumptions
			- all input features are independent
			- all input features contribute equally to classification
	- requires that each probability is 0
		- if there is one option that is 0, can add 1 to each to ensure it works
- Linear Discriminant Analysis
	- supervised
	- dimensional reduction
	- maximize the distance between groups
		- within-class variance is minimized
		- maximize the distances between the means of the two categories on the new axis
	- trying to maximize: $$\frac{(\mu_{1}-\mu_{2})^2}{s_{1}^2-s_{2}^2}$$
		- Top of equation is the distance between the averages of the data projected onto the new line
		- bottom is minimizing the scatter within each category
	- discriminant analysis determines decision boundary between classes