ece 109 cumsheet

cumulative cheat sheet

i will turn this into a better cheat sheet for the final, just need to know what doesn't need to be on it

week 1: event

i dont have anything to say lmao

P(E \cup F) = P(E) + P(F) - P(EF) "inclusion-exclusion principle," subtracts double counted intersection

P(E) = \frac{|E|}{|S|} but only if it's equiprobable!!

          \binom{n}{k} = \frac{n!}{k! \, (n - k)!}

          G - F = GF^C

does not occur = complement occurred

"not necessarily" is a valid response to "does [event] occur?"

post-quiz note: none of this was used

week 2: given

conditional probability

          P(E | F) = \frac{P(EF)}{P(F)} = \frac{P(F | E) \, P(E)}{P(F)}

          P(EF) = P(E | F) \, P(F) = P(F | E) \, P(E)

          \begin{aligned}
            P(B) &= P(BA) + P(BA^C) = P(B | A) \, P(A) + P(B | A^C) \, P(A^C) ~ \text{splitting into cases} \\
            &= P(B | A_1) \, P(A_1) + \dots + P(B | A_n) \, P(A_n) \\
            P(B | E) &= P(B | A_1 E) \, P(A_1 | E) + \dots + P(B | A_n E) \, P(A_n | E)
          \end{aligned}

P(A_1 \dots A_n) = P(A_1) \, P(A_2 | A_1) \, P(A_3 | A_1 A_2) \dots P(A_n | A_1 \dots A_{n-1}) "chain rule"

P(A_i | B) = \dfrac{P(B | A_i) \, P(A_i)}{\displaystyle\sum_{j = 1}^n P(B | A_j) \, P(A_j)} "bayes theorem"

if E and F disjoint, P(E | F) = 0

if E \subseteq F then EF = E, P(F | E) = 1

split complicated events into cases

post quiz note: we did not use this. did need to know how to interpret gibberish like "ur mom but not my mom"

week 3: independent

If E and F are independent,

```
P(EF) = P(E) \, P(F)
```
P(E | F) = P(E) if P(F) \neq 0
E and F^C, E^C and F, E^C and F^C are also independent

If A and B are independent given C,

P(AB | C) = P(A | C) \, P(B | C) if P(C) \neq 0
```
P(A | BC) = P(A | C)
```
but they aren't independent given C^C!!

If E and F are disjoint and their probabilities aren't 0, they're not independent

If three events are independent to each other ("pairwise independent") and P(ABC) = P(A) \, P(B) \, P(C) (which isn't necessarily true), they're independent

Probability E occurs exactly k times in n independent trials:

          \binom{n}{k} (P(E))^k (1 - P(E))^{n-k}

For disjoint A and B, probability that A occurs before B in independent trials:

          \frac{P(A)}{P(A) + P(B)}

and i guess random vars are fair game too??

"with replacement" = putting it back (→ independent trials)

disjoint and subset events are not independent

don't forget that you know about their complements too

post quiz: P(A | C) is NOT \frac{P(A)}{P(C)}. the conditional independence equations above were useful

week 4: cumdf

(CDF/pmf definitions moved to a table in week 5)

here are a bunch of formulas that are probably kind of easy to remember/figure out on the spot

P(X > u) = 1 - F(u)
P(X < u) = F(u^-)
P(X \geq u) = 1 - F(u^-)
P(X = u) = "jump at u" (if continuous it's 0)
P(a < X \leq b) = F(b) - F(a)
P(a < X < b) = F(b^-) - F(a)
P(a \leq X < b) = F(b^-) - F(a^-)
P(a \leq X \leq b) = F(b) - F(a^-)

cumDF only needs to be right continuous, so F(u) = F(u^+) \neq F(u^-)

when zeger has P([a, b]) = b - a, split into cases based on possible values of X or endpoints of idk (for example, if X could be 0 and 1, then your cases would be u < 0, 0 \leq u < 1, etc, see w4d2 lecture example or winter 22 quiz 4.3)

also, P(E^C | F) = 1 - P(E | F)

i do not imagine any of this will be needed on the quiz

post quiz: nothing was used. the quiz was easy

week 5: expected

function	stands for	typical function name and formula	graph notes	rules/properties
CDF	cumulative distribution function	F_X(u) = P(X \leq u)	filled bubbles always above open circles	never decreases, starts at 0 and ends at 1
pmf	probability mass function	p_X(u) = P(X = u)	bars with dots on them	never negative, bars add to 1
pdf	probability density function	f_X(u) = \frac{dF_X(u)}{du}		never negative, area under curve is 1 P(a < X \leq b) = \int\limits_a^b f_X(u) du

function

stands for

typical function name and formula

graph notes

rules/properties

CDF

cumulative distribution function

F_X(u) = P(X \leq u)

filled bubbles always above open circles

never decreases, starts at 0 and ends at 1

pmf

probability mass function

p_X(u) = P(X = u)

bars with dots on them

never negative, bars add to 1

pdf

probability density function

f_X(u) = \frac{dF_X(u)}{du}

never negative, area under curve is 1

                  P(a < X \leq b) = \int\limits_a^b f_X(u) du

expected value/mean:

          E[X] = \sum_u u f_X(u)

          E[X] = \int\limits^\infty_{-\infty} u f_X(u) du

special rv	pmf/pdf	intuition	mean	variance
uniform discrete rv	`n` equally spaced rvs of probability `\frac{1}{n}`
binary rv (deterministic rv when `q = 0` or 1)	p_X(0) = 1 - q p_X(1) = q	flipping a biased coin	E[X] = q	\sigma_X^2 = q(1-q)
binomial rv	p_X(k) = \binom{n}{k} p^k (1 - p)^{n-k} for `k = 0, 1, \dots, n`	number of heads from flipping biased coin `n` times	E[X] = np
poisson rv	p_X(k) = \dfrac{\lambda^k e^{-\lambda}}{k!} for `k = 0, 1, 2, \dots`		E[X] = \lambda
geometric rv	p_X(k) = p(1 - p)^{k - 1} for `k = 1, 2, \dots`	number of biased coin flips until first head	E[X] = \frac{1}{p}
uniformly distributed rv on `[a, b]`	f_X(u) = \begin{cases} \dfrac{1}{b - a} & \text{if $a < u < b$} \\ 0 & \text{else} \end{cases}		E[X] = \frac{a + b}{2}	\sigma_X^2 = \frac{(a-b)^2}{12}
gaussian/normal rv X \sim \N(m, \sigma^2)	f_X(u) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\dfrac{1}{2}\left(\dfrac{u - m}{\sigma}\right)^2}		E[X] = m	\Var(X) = \sigma^2
exponential rv	f_X(u) = \begin{cases} \lambda e^{-\lambda u} & \text{if $u > 0$} \\ 0 & \text{else} \end{cases}		E[X] = \frac{1}{\lambda}

post-quiz: none of this was very useful, but desmos is!!

week 6: variance

CDF for X \sim \N(0, 1):

          \Phi(u) = \frac{1}{\sqrt{2\pi}} \int\limits_{-\infty}^u e^{-\frac{z^2}{2}} dz = \frac{1}{2} + \frac{1}{2}\erf\left(\frac{u}{\sqrt{2}}\right)

for X \sim \N(m, \sigma^2), F_X(u) = \Phi\left(\dfrac{u-m}{\sigma}\right)

i dont think i need to know this at all, but

          \erf(u) = \frac{2}{\sqrt{\pi}} \int\limits_0^u e^{-t^2} dt

more expected value properties:

          E[g(X)] = \begin{cases}
            \displaystyle\int\limits_{-\infty}^\infty g(u) f_X(u) du & \text{continuous} \\
            \displaystyle\sum_u g(u) p_X(u) & \text{discrete}
          \end{cases}

E[g_1(X) + g_2(X)] = E[g_1(X)] + E[g_2(X)]

nth moment: E[X^n]. first moment is mean

nth central moment: E[(X - E[X])^n]. first central moment is zero, second central moment is variance (sqrt is stddev)

          \sigma_X^2 = \Var(X) = E[(X - E[X])^2] = E[X^2] - E[X]^2

for Y = aX + b:

```
E[aX + b] = aE[X] + b
```
pmf p_Y(u) = p_X\left(\dfrac{u-b}{a}\right)
variance \sigma_Y^2 = a^2 \sigma_X^2
if X \sim \N(m, \sigma^2), then Y \sim \N(am + b, a^2 \sigma^2)

to find pdf of Y in terms of p_X(u) (pdf of X)

set up CDF of Y (get F_Y in terms of F_X)
differentiate

          \frac{d}{du} \int\limits_{h(u)}^{g(u)} f(z, u) dz = f(g(u), u) g'(u) - f(h(u), u) h'(u) + \int\limits_{h(u)}^{g(u)} \left(\frac{\partial}{\partial u} f(z, u)\right) dz ~ \text{(leibniz rule)}

joint cumdfs probably not this week

You can visually use the center of mass of the pdf/pmf for expected value

Variance is never negative

Use Wolfram Alpha!

be prepared to complete the square and stuff to make things look like a gaussian

post quiz note:

know your log properties
\Var(aX^2) = a^2\Var(X^2) because if you let Y = X^2 then \Var(aX^2) = \Var(aY) = a^2 \Var(Y) = a^2 \Var(X^2)
when using desmos to calculate variance of Y = 2 - 3X^2 you can't drop X = 0 when calculating expected value because Y \neq 0

week 7: joint

joint marginal

	joint	marginal
cumDF	F_{X,Y}(u, v) = P(X \leq u, Y \leq v)	F_X(u) = F_{X,Y}(u, \infty) F_Y(u) = F_{X,Y}(\infty, v)
pmf	p_{X,Y}(u, v) = P(X = u, Y = v) to get a probability of a region, just add the probabilities in it P((X, Y) \in T) = \sum_{(u, v) \in T} p_{X,Y}(u, v) the CDF is like an infinite rectangle with top right corner `(a, b)` F_{X,Y}(a, b) = P((X, Y) \in \{(u, v) \| u \leq a, v \leq b\}) \sum_{u, v} p_{X,Y}(u, v) = 1	p_X(u) = \sum_u p_{X,Y}(u, v) p_Y(v) = \sum_u p_{X,Y}(u, v)
pdf	you can take a partial derivative in either order f_{X,Y}(u, v) = \frac{\partial^2 F_{X,Y}(u, v)}{\partial u \, \partial v} derived from P((X, Y) \in T) = \iint\limits_T f_{X,Y}(u, v) \, du \, dv pdf ⇒ CDF: F_{X,Y}(u, v) = \int\limits_{-\infty}^v \int\limits_{-\infty}^u f_{X,Y}(s, t) \, ds \, dt \int\limits_{-\infty}^\infty \int\limits_{-\infty}^\infty f_{X,Y}(u, v) \, du \, dv = 1	f_X(u) = \int\limits_{-\infty}^\infty f_{X,Y}(u, v) \, dv f_Y(v) = \int\limits_{-\infty}^\infty f_{X,Y}(u, v) \, du

cumDF

F_{X,Y}(u, v) = P(X \leq u, Y \leq v)

F_X(u) = F_{X,Y}(u, \infty)

F_Y(u) = F_{X,Y}(\infty, v)

pmf

p_{X,Y}(u, v) = P(X = u, Y = v)

to get a probability of a region, just add the probabilities in it

                  P((X, Y) \in T) = \sum_{(u, v) \in T} p_{X,Y}(u, v)

the CDF is like an infinite rectangle with top right corner (a, b)

                  F_{X,Y}(a, b) = P((X, Y) \in \{(u, v) | u \leq a, v \leq b\})

                  \sum_{u, v} p_{X,Y}(u, v) = 1

p_X(u) = \sum_u p_{X,Y}(u, v)

p_Y(v) = \sum_u p_{X,Y}(u, v)

pdf

you can take a partial derivative in either order

                  f_{X,Y}(u, v) = \frac{\partial^2 F_{X,Y}(u, v)}{\partial u \, \partial v}

derived from

                  P((X, Y) \in T) = \iint\limits_T f_{X,Y}(u, v) \, du \, dv

pdf ⇒ CDF:

                  F_{X,Y}(u, v) = \int\limits_{-\infty}^v \int\limits_{-\infty}^u f_{X,Y}(s, t) \, ds \, dt

                  \int\limits_{-\infty}^\infty \int\limits_{-\infty}^\infty f_{X,Y}(u, v) \, du \, dv = 1

                  f_X(u) = \int\limits_{-\infty}^\infty f_{X,Y}(u, v) \, dv

                  f_Y(v) = \int\limits_{-\infty}^\infty f_{X,Y}(u, v) \, du

(from spencer vid) to find a probability from joint pdf:

figure out what region to integrate over depending on the event given
do integral. lmao

shortcut! if you know that a marginal pdf will be constant (eg when finding marginal pdf of Y where joint pdf only dependent on u (X)) you can just solve for C by setting area = 1

post quiz. desmos is ideal for those double integrals. everything was just pdf stuff, so didnt use this cumsheet for the quiz

week 8: independent rvs

these are equivalent:

rvs X and Y are independent

              P(X \in A, Y \in B) = P(X \in A) \, P(Y \in B)

where A, B \in \mathbb{R}

joint CDF factors: F_{X,Y}(u, v) = F_X(u) \, F_Y(v) for all u, v
joint pdf factors: f_{X,Y}(u, v) = f_X(u) \, f_Y(v) for all u, v
joint pmf factors: p_{X,Y}(u, v) = p_X(u) \, p_Y(v) for all u, v

iid (independent, identically distributed) means same pdf/pmf

rv	pdf
Z = X + Y	\begin{aligned} f_e(w) &= \int\limits_{-\infty}^\infty f_{X,Y}(w - v, v) \, dv \\ &= \int\limits_{-\infty}^\infty f_{X,Y}(u, w - u) \, du \\ &= \int\limits_{-\infty}^\infty f_X(w - v) \, f_Y(v) \, dv ~ \text{(if independent; convolution integral)} \\ &= \int\limits_{-\infty}^\infty f_X(u) \, f_Y(w - u) \, du \end{aligned}
Z = \max(X, Y)	\begin{aligned} f_Z(w) &= \int\limits_{-\infty}^t f_{X,Y}(u, t) \, du + \int\limits_{-\infty}^t f_{X,Y}(t, v) \, dv \\ &= F_X(t) \, f_Y(t) + f_X(t) \, F_Y(t) ~ \text{(if independent)} \\ &= n \, F_X^{n-1}(t) \, f_{X_1}(t) ~ \text{(if $n$ iid rvs)} \\ &= \begin{cases} nt^{n-1} ~ \text{if $t \in [0, 1]$} \\ 0 ~ \text{else} \end{cases} ~ \text{(if rvs uniform on $[0, 1]$)} \end{aligned}
Z = XY	f_Z(a) = \int\limits_{-\infty}^\infty \frac{1}{\|v\|} f_{X,Y}\left(\frac{a}{v}, v\right) \, dv

pdf

Z = X + Y

                  \begin{aligned}
                    f_e(w) &= \int\limits_{-\infty}^\infty f_{X,Y}(w - v, v) \, dv \\
                    &= \int\limits_{-\infty}^\infty f_{X,Y}(u, w - u) \, du \\
                    &= \int\limits_{-\infty}^\infty f_X(w - v) \, f_Y(v) \, dv ~ \text{(if independent; convolution integral)} \\
                    &= \int\limits_{-\infty}^\infty f_X(u) \, f_Y(w - u) \, du
                  \end{aligned}

Z = \max(X, Y)

                  \begin{aligned}
                    f_Z(w) &= \int\limits_{-\infty}^t f_{X,Y}(u, t) \, du + \int\limits_{-\infty}^t f_{X,Y}(t, v) \, dv \\
                    &= F_X(t) \, f_Y(t) + f_X(t) \, F_Y(t) ~ \text{(if independent)} \\
                    &= n \, F_X^{n-1}(t) \, f_{X_1}(t) ~ \text{(if $n$ iid rvs)} \\
                    &= \begin{cases}
                      nt^{n-1} ~ \text{if $t \in [0, 1]$} \\
                      0 ~ \text{else}
                    \end{cases} ~ \text{(if rvs uniform on $[0, 1]$)}
                  \end{aligned}

Z = XY

                  f_Z(a) = \int\limits_{-\infty}^\infty \frac{1}{|v|} f_{X,Y}\left(\frac{a}{v}, v\right) \, dv

from spencer:

P(\max(X, Y) \geq 1) = 1 - P(\max(X, Y) < 1) which is a bit more useful

polar integration

              \iint f(r, \theta) r dr d\theta

when the quiz asks which of a bunch of joint pmf graphs are independent, the rows/columns should be multiples of each other

post-quiz: yeah actually in general, if you have two independent uniform rvs, their joint pdf/pmf will be a rectangle/array

week 9: covariance

          f_{X + Y, X - Y}(a, b) = \frac{1}{2} f_{X,Y}\left(\frac{a + b}{2}, \frac{a - b}{2}\right)

          \begin{aligned}
            E[Z = g(X_1, \dots, X_n)] &= \int\limits_{-\infty}^\infty \int\limits_{-\infty}^\infty \dots \int\limits_{-\infty}^\infty g(u_1, \dots, u_n) \, f_{X_1, \dots, X_n}(u_1, \dots, u_n) \, du_1 \dots du_n \\
            E[g(X, Y)] &= \int\limits_{-\infty}^\infty \int\limits_{-\infty}^\infty g(u, v) \, f_{X, Y}(u, v) \, du dv & \text{(continuous)} \\
            &= \sum_v \sum_u g(u, v) \, p_{X, Y}(u, v) & \text{(discrete)}
          \end{aligned}

          E[X + Y] = E[X] + E[Y] ~ \text{They don't need to be independent}

covariance

          \begin{aligned}
            \Cov(X, Y) &= E[(X - E[X])(Y - E[Y])] \\
            &= E[XY] - E[X] \, E[Y] \\
            \Cov(X, X) &= \Var(X) \\
            \Cov(X, Y + Z) &= \Cov(X, Y) + \Cov(X, Z) \\
            \Cov(aX, bY) &= ab \Cov(X, Y) \\
            \Var(X \pm Y) &= \Var(X) + \Var(Y) \pm 2 \Cov(X, Y) \\
            &= \Var(X) + \Var(Y) ~ \text{if uncorrelated}
          \end{aligned}

for independent rvs, \Var(X_1 \pm \dots \pm X_n) = \Var(X_1) \pm \dots \pm \Var(X_n)

correlation coefficient

          \begin{aligned}
            \rho_{X, Y} &= \frac{\Cov(X, Y)}{\sigma_X \sigma_Y} \\
            &= E\left[\left(\frac{X - E[X]}{\sigma_X}\right)\left(\frac{Y - E[Y]}{\sigma_Y}\right)\right] \\
            |\rho_{X, Y}| &\leq 1
          \end{aligned}

independence → (uncorrelated ↔ E[XY] = E[X] \, E[Y])

then zeger went on to talk about estimation.. which spencer DID mention , so 😱😱😱

mean squared error = E[(X - \hat{X})^2] with error e, true value X, estimate \hat{X}

best linear estimate

          \begin{aligned}
            \hat{X} &= \frac{E[XY]}{E[Y^2]} \cdot Y \\
            &= \rho_{X, Y} Y ~ \text{if $\sigma_X = \sigma_Y = 1$, $E[X] = E[Y] = 0$}
          \end{aligned}

joint gaussian :bervous: emoji

          \begin{aligned}
            f_{X,Y}(u, v) &=
              \frac{1}{2\pi\sigma_X\sigma_Y\sqrt{1-\rho^2}} ~
              e^{\displaystyle -\frac{1}{2(1 - \rho^2)} \left(
                \left(\frac{u - m_X}{\sigma_X}\right)^2
                + \left(\frac{v - m_Y}{\sigma_Y}\right)^2
                - 2\rho\left(\frac{u - m_X}{\sigma_X}\right)\left(\frac{v - m_Y}{\sigma_Y}\right)
              \right)} ~ \text{where $\rho$ is the correlation coeff} \\
            &= \frac{1}{2\pi\sqrt{1-\rho^2}} ~
              e^{\displaystyle -\frac{1}{2(1 - \rho^2)}
              (u^2 + v^2 - 2\rho uv)} ~ \text{if $m_X = m_Y = 0$, $\sigma_X = \sigma_Y = 1$} \\
          \end{aligned}

just for gaussians, independence ↔ uncorrelated

don't mix up stddev and variance!! make sure to sqrt variance for \sigma_X for correlation coefficient, but when the problem asks to sum variances leave it as \sigma_X^2

maybe the infinite integral of e^{-ax} might be nice to have

          \begin{aligned}
            \int\limits_0^\infty e^{-ax} dx &= -\frac{1}{a} e^{-ax} \bigg\rvert_0^\infty \\
            &= -\frac{1}{a} e^{-a\infty} + \frac{1}{a} e^0 \\
            &= -\frac{1}{a} \cdot 0 + \frac{1}{a} \cdot 1 \\
            &= \boxed{\frac{1}{a}}
          \end{aligned}

actually this seems incredibly limited i would just use desmos

at least for fall '21 #2 apparently it is better to use the gaussian pdf not the joint gaussian

post-quiz: needed to double check variance equation