30 lines
1.4 KiB
Markdown
30 lines
1.4 KiB
Markdown
#rs/class/csb320 #rs/notes
|
|
- - -
|
|
- Models are trained by learning from their mistakes
|
|
- multi-armed bandits
|
|
- choose action from k possibilities
|
|
- receive a reward
|
|
- reward is dependent on the action taken
|
|
- explore vs. exploit
|
|
- exploit is taking the greedy action, the on that is known to produce the greatest reward
|
|
- explore is taking other random actions to learn values
|
|
- a combination of the two produce the best results
|
|
- $\varepsilon$-greedy strategy
|
|
- with probability $\varepsilon$ take a random action, with probability $1-\varepsilon$ take the best known actions
|
|
- $\varepsilon$ often starts high and decreases over time
|
|
- $\rho$ (regret) can be used to find how good a strategy is
|
|
- this is the difference between how much reward was gotten and the maximum reward possible if distributions are known ahead of time
|
|
- markov decision process
|
|
- future states only depend on the present state, not what came before
|
|
- trying to maximize reward without knowing entire history
|
|
- monte-carlo
|
|
- updates model after every episode
|
|
- temporal difference
|
|
- learns after every step, not every episode
|
|
- q-learning
|
|
- assembles all possible q values on the way to end reward
|
|
- updates q values to find best path
|
|
- $$Q^{new}(s_{t}, a_{t}) \leftarrow (1 - a) * Q(s_{t}, a_{t}) + a * (r_{t} + \gamma * max_{a} Q(s_{t+1}, a))$$
|
|
- ![[BellmanEquation.excalidraw]]
|
|
- This equation can be used to update the q values of the grid
|
|
- it takes the old value and adds |