User:Rocky zhu

=Sample Space, Probability, Events=
 * Model 1: The experiment of throwing an equal dice once and anticipating the result (Simple Experiment)
 * Model 2: The experiment of throwing an equal coin once and anticipating the result (Simple Experiment)
 * Model 3: The experiment of throwing an equal dice once and an equal coin once and anticipating the result (Combinational Experiment)

Basic Definition

 * Sample Space: All the possible outcomes for an experiment
 * in Model 1, S1 = {1,2,3,4,5,6}
 * in Model 2, S2 = {H,T}
 * in Model 3,  S3 = {1H,2H,3H,4H,5H,6H,1T,2T,3T,4T,5T,6T}


 * Probability(not-so-good def): A quantative measure of the possibility of each outcome in an experiment.
 * in Model 1, Pr(1) = Pr(2) = Pr(3) = Pr(4) = Pr(5) = Pr(6) = 1/6
 * in Model 2, Pr(H) = Pr(T) 1/2
 * in Model 3, Pr(1H) = Pr(2H) = Pr(3H) = Pr(4H) = Pr(5H) = Pr(6H) = Pr(1T) = Pr(2T) = Pr(3T) = Pr(4T) = Pr(5T) = Pr(6T) = 1/12


 * The definition above is not so easy to use. Assume in model 1, we want to know the probability of occurrence of odd number, or small number, How do we express them? P(1)+P(3)+P(5); P(1)+P(2)+P(3)


 * Event: A set of elements in sample space. An event is said to occur if one element in it happens.
 * Basic Event A event that contains only one outcoem in the sample space. E1, E2, ... Em
 * Probability: A quantative measure of the possibility each event.
 * Clearly, The Pr(EVENT) = Pr(E1)+Pr(E2)+...Pr(En);

Probability of Joint Events

 * Question
 * Q1, In Model 1, we define two events A---even output{2,4,6}, B---small output{1,2,3}. What's the probability of A AND B happening together?
 * Q2, In Model 2, we define two events A: head output{H}, B:tail output{T}. What's the probability of A AND B happening together?
 * Q3, In Model 3, What is the probability that in model 1, even output happens AND in model 2, head occurs?
 * Answer
 * A1, In model 1, A*B = {2} Pr(A*B) = 1/6;
 * A2: In model 2, A*B = {}, Pr(H*T) = 0;
 * A3: even and head = {2H,4H,6H}; Pr = 3/12 = 1/4;


 * Joint Event: The joint event JA of A1,...,Ak is the case in which A1,...,Ak all happen.  JA = A1*A2*...*Ak.

Probability of Union Events

 * Question
 * Q1 In Model 1, we define two events A--- even output{2,4,6}, B---small output{1,2,3}. What's the probability that A OR B happens?
 * Q2 In Model 2, we define two events A: head output{H}, B:tail output{T}. What's the probability that A OR B happens?
 * Q3: What is the probability that in model 1, even output happens OR in model 2, head occurs?
 * Answer
 * A1 In model 1, AUB = {1,2,3,4,6} Pr(AUB) = 5/6;
 * A2: In model 2, AUB = {H,T},     Pr(HUT) = 1;
 * even or head = {1H,2H,3H,4H,5H,6H, 2T, 4T,6T}	Pr = 9/12 = 3/4


 * Definition: The union event UA of A1,...,Ak is the case in which A1,...,Ak all happen.UA = A1UA2U...UAk.

//		Relationship betweeen Joint and Union Events Questions: Can we relate Pr(AUB) and Pr(A*B) with Pr(A), Pr(B)? Answer: YES! Pr(AUB) = Pr(A) + Pr(B) - Pr(A*B) Special case: if A and B are mutually exclusive: Pr(A*B) = 0 then Pr(AUB) = Pr(A) + Pr(B)

Probability of Conditional Events

 * Question
 * Q1 in model 1, What is the probability that B={1,2,3} happens if A={2,4,6} happens
 * Q2: in model 2, What is the probability that B={H} happens if A={T} happens
 * Q3: What is the probability that B={1,2,3} if A={T} happens
 * Answers
 * A1 A = {2}U{4}U{6}, only the event {2} falls in B. Pr(B|A) = Pr({2})/Pr(A) = 1/3 = Pr(AB)/Pr(A)
 * A2: Pr(B|A) = Pr(H|T) = 0;
 * A3: A ={1T,2T,3T,4T,5T,6T} the events{1T,2T,3T} fall in B. Pr(B|A) = Pr({1T})+ Pr({2T}) + Pr({3T})/Pr(A) = 1/2 = Pr(AB)/Pr(A)
 * Conditional Probability: Pr(B|A) = Pr(B*A)/Pr(A)
 * special case---for mutual exclusive events A and B Pr(A|B) = Pr(B|A) = 0; becaue Pr(AB) = 0;

Independent Events

 * In model 1 or 2, the event A and B are dependent of each other because the occurrence of either one of them will affect the other.
 * In model 3, the event A1 and B1 are independent because the occurrence of either one of them does NOT affect the other.

Intuitively speaking, In a simple experiment S, the events A and B are usually dependent. However, in a combinational experiment, if the events A1 only takes into account the result of an simple experiment S1, ,..., An only takes into account the result of an simple experiment Sn, then A1, A2, ... An are usually independent.

Definition: Event A and B are independent if Pr(A|B) = Pr(A)

=Greedy Algorithm= Mathematical Characterization: What kinds of problems are suitable for using Greedy Algorithm?

Example

 * Find the minimum number of processor needed for a set of jobs
 * Give the customers the minimum number of coins for their change
 * Find the minimum spanning tree(MST) of a connected graph

Case Study:MST

 * Blue and Red Rules
 * Blue Rule: for any cut in which no blue edge exists, mark the edge with smallest weight as blue.
 * Red Rule: for any cycle in which no red edge exists, mark the edge with heaviest weight as red.

Mark all edges as uncolored initially. Keep applying Blue Rule util n-1 edges are marked blue.
 * Kruskal's Algorithm
 * Algorithm Description
 * 1) Create an array to indicate which component each node belongs to. Initialize them with each node's "name" or index.
 * 2) Create an array to link to antoher node in the same component. Initialize them with null.
 * 3) Create an array to store blue edges, with a variable "size" to indicate the number of blue edges stored in it. Initialize size with 0
 * 4) Build a min-heap to sort all the edges.
 * 5) Loop: while size < n-1
 * 6) *Pick out the smallest weighted edge in the heap
 * 7) *Check the component fields of the two nodes of this edge
 * 8) *if they are the same
 * 9) **ignore this edge
 * 10) *else
 * 11) **merge all the nodes in the two components into one component.
 * 12) **size++

As to the operation of checking if component fields of the two nodes are the same and the operation of merging all the nodes in two components into one, we come up with two data structures as below:

Once we are sure that two nodes belong to different component, we merge all the nodes in the two component by setting the component filed of the nodes in one component the same as that in the other component.
 * Data Strcture 1 to support this algorithm---Linked List


 * Algorithm Analysis
 * 1) The time to find which component a node belongs to is 1, so total time for FIND operation is n.
 * 2) The time to merge nodes in two components in worst case is:
 * 3) * The worst case happens if
 * 4) *# the component fields of the nodes in the bigger component are always changed(because we are blind of the size of each component) and
 * 5) *#a new node is merged into A constant component in each MERGE operation,.
 * 6) * The time needed for MERGE operation is n(n-1)/2 = O(n^2)
 * 7) The MERGE operation time can be reduced if we keep track of the size of each component
 * 8) * The time needed for MERGE operation when we know the size of each component is O(nlgn)

For two nodes that belong to different component, instead of changing the component fields for nodes in one component, we find the roots of the two component, and check if they are the same. If not, merge the two components by making one node point to the other.
 * Data Strcture 2 to support this algorithm---Tree


 * Algorithm Analysis
 * 1) The time to merge two component is 1, so total time for MERGE operation is n.
 * 2) The time to find the root of two components in worst case with track of the height of the tree for each component is O(nlgn), ie, a Link(thin tall tree) is formed at last
 * 3) The time for FIND operation can be reduced if the root of the tree whose height is smaller point to the root in the other tree whose height is larger(fat short tree)(path compression)

=Lower/Upper Bound=
 * Asymptotic lower bound is a function of the SIZE of the input, which describes the amount of operations ALL algorithm will do to solve a problem on a case(could be worst, average, or worst).


 * Asymptotic upper bound is a function of the SIZE of the input, which describes the amount of operations AN algorithm will do to solve a problem on a certain case(could be worst, average, or worst).

Example
Problem:                                                      Sorting Basic Operations:                                             Pairwise Comparisons The lower bound is given by:                                  best-case: O(n)        worst-case: O(nlgn) The upper bound, when Heap Sort AlgoriThm is used, is:        best-case: O(nlgn)     worst-case: O(nlgn) The upper bound, When Selection Sort Algorithm is used, is:   best-case: O(n^2)      worst-case: O(n^2)

Problem:                                                      Finding MST in a connected graph The upper bound, when Kruskal's Algorithm is used with Linked List data structure, is: worst-case: O(nlgn) The upper bound, when Kruskal's Algorithm is used with Tree data structure, is:        worst-case: O(n)