Fact: log(n) = Θ(n log n).
[Show.]
A lower bound on sorting:
(Rosen 9.2)
computation tree based on comparisons; has n! leaves;
thus must have height lg(n!) = Ω(n log n)
(Recall tree's size-vs-height bound from structural-induction lecture.)
We have a very complete picture for sorting: we know an
lower bound for any algorithm, and and an algorithm which attains it.
Usually we're not so lucky.
For instance, as mentioned, we'd like to know that there's no quick
way to factor numbers (that it's complexity is Θ(n),
not just O(n).)
One counting argument:
Recall that reals in [0,1] are uncountable.
(Consider writing such numbers in binary.)
How many functions are there, in N → {F,T}?
This is isomorphic.
How many programs exist? Countable.
Therefore: There are functions which cannot be computed (!)
Recall from comp210: The Halting Problem.
Note how we're distinguishing between a Problem,
and various algorithms to solve (instances of) the problem.
Def'ns:
P = PTIME: The set of all problems which have a polynomial-time solution.
That is, run times in ∪k∈N O(nk)
where n is the size of the input.
NP = NPTIME: The set of all problems which have a polynomial-time *verifier*.
(much weaker!) (Stands for "non-deterministic polynomial".)
EXPTIME: the set of all problems which have an exponential-time solution.
Note that P is contained in NP is contained in EXPTIME.
Are these containments proper? Unknown (!)
(Despite Homer3's assertion.)
Classify these *problems* as best possible:
- Sorting: given a list of n numbers, return the sorted list.
- Shortest-path: Given a graph and two vertices, what is shortest path btwn?
- Longest-path
- Composite?: Given a number n, is it composite?
- Prime?: Given a number n, is it prime?
- 3-coloring: Partition a graph into three sets R,G,B such that no edges
between elements of the same set(color).
- Chess: given a board, does white have a forced win?
For 8x8 boards, this is a finite problem: O(1).
But for nxn board, it's harder.
- Minesweeper-consistency
- TAUTOLOGY: given phi, is it true under all truth assignments?
- EQUIVALENCE: given phi, psi, are they the same under all truth assignment?
Recall: we showed earlier that each of these *reduced* to the other:
If your friend can quickly solve Tautology,
then you could set up a Equivalence-solving shoppe,
by subcontracting w/ your friend, so Equiv ≤P Tautology.
Conversely, Tautology ≤P Equivalence.
- SAT: given a propositional formula, is it satisfiable?
- Circuit-SAT (as SAT, but w/ a AND/OR/NOT feed-forward circuit.)
- has-PTIME-soln?: Given (the code for) a *verifier*, and an input
for the problem in question, is there a sol'n which appeases the verifier?
Naive algorithm:
Note that this last problem is artificial. It has the property that
it's "NP-complete": if you can solve that problem quickly (in P time),
then you could solve *any* problem in NP quickly.
For reductions, we require that a reduction be computable in P, so that
we get the desired "can solve in polynomial time" relationship.
Cook, 1974: SAT is NP-complete.
Proof sketch: Reduce has-PTIME-soln? to SAT:
Given a verifier and an input problem:
We take the verifier, and model it with a huge propositional formula:
make a proposition for each variable's bits
at each of nk time-steps.
The sequence of code (and the meaning of the language) makes
formulas relating the variables at one time step and the next.
Call this mega-formula φ. It's huge,
but it's still polynomial in the input length.
Now, just pass the question to the SAT solver,
asking outputnk ∧ φ.
This solves whether or not there's some input that makes
the
[Realize, this is very much a sketch.
For instance, is φ really polynomial?
What language constructs are being assumed?
Do we really need to be given k as well as the verifier?
What if k isn't known?, or it takes fewer than nk steps
for some inputs?
There's some funniness about the size of the verifier,
as opposed to the size of the subproblem-input --
what if the verifier were exponential in size of the subproblem-input?
Does it even make sense to talk about that, for a fixed subproblem-input?
Etc.
Comp 482 for more info.]
Many problems of practical interest have been shown NP-complete.
(The 10-coloring problem is assigning students to colleges,
w/o having people from same high school in same college.)
For instance, SAT can be reduced to 3-coloring:
via a clever way of taking a formula,
creating an equivalent formula with exactly 3 variables per clause
(adding extra dummy variables),
then taking these 3-clauses and constructing a weird graph that
can be 3-colored iff the original formula was satisfied.
We know that solving any of these is as difficult as solving all of them,
and probably not worth your effort to try.
Though there is a $1,000,000 cash prize for resolving P vs NP either way.
(Some NP-complete problems allow approximate sol'ns, which might be good
enough; for others NP-complete, finding an approximate sol'n is as
difficult as finding the entire sol'n!)
Note that
Minesweeper is NP-complete.
Reached here, 2004.Apr.20
======== Randomized Algorithms ========
Now, a different tack:
Flipping coins to gain computational power(?!)
Towards randomized algs and their running time:
Running time of quicksort?:
Hmm, look at avg-case time.
Hmm, need to define "average".
[Can set up recurrence, and show that any sol'n is bounded by O(n log n)]
For random partition, what is the size of the larger half?:
1/n*max(0,n-1) + 1/n*max(1,n-2) + 1/n*max(2,n-3) + 1/n*max(3,n-4)
+ ... + 1/n*(max(floor(n/2),ceil(n/2)) + 1/n*(max(floor(n/2),ceil(n/2)) + ...
+ 1/n*max(n-4,3) + 1/n*max(n-3,2) + 1/n*max(n-2,1) + 1/n*max(n-1,0).
= 1/n * [ (n-1) + (n-2) + ... + ceiling(n/2)
+ ceiling(n/2) + ... + (n-2) + (n-1) ]
= 1/n*2*[ (n-1 + n/2) * (n-1)/4 ] = (3n-2)(n-1)/4n = (3n^2-5n+2)/4n = 3n/4 + O(1)
Note how randomization helped our quicksort alg.
Consider too:
- testing primes
For a number n, and 1 < b < n: "The Divisor test: b|n ?"
- If n is prime, then the test always passes. (No false negatives.)
But the converse doesn't always hold: if the test passes,
it may or may not be prime. (Some false positives.)
What if we search for witnesses b < n, uniformly at random?
- If n is non-prime, there might be only two factors,
so the test might fail for only 2/n of the possible tests.
Picking at random, it'd take a long time to have confidence.
(We'd rather try each candidate methodically!)
BUT, Miller came up with a very clever alternate test:
"For a number n and b < n, The Miller test of n with base b:
Let n-1 = 2st, where s non-negative and t odd.
It passes iff:
- bt ≡ 1 (mod n), or
- ∃ 0≤j<s such that b2jt ≡ -1."
(See also Rosen's exercises.)
Pertinent fact:
- If n is prime, then the test always passes.
- If n is non-prime, then there are at least n/4 bases which
will cause the test to fail.
Algorithm: Choose b uniformly at random; if fails test answer
"n definitely not prime"; if it passes answer
"n prime (prob that this is a false positive: < 75%)"
Likelihood of a cosmic ray hitting a transistor with enough energy
to flip a bit while our program is running: < 1060 [guess]
Don't be confused by raging threads on sci.math: "a number is prime
or it isn't; nonsensical to say it's prime with probability ...".
Whenever you give a probability, you have to say what you're
randomizing over!
- flipping mattress, or
- who to pay for dinner?
- choose one of three people randomly; we only have access to random bits.
- Or, we have independent-but-biased coin (but don't know the bias).
How to get random bits from it?
Look at successive pairs of flips -- only take those which differ!
- Measuring pi: throw darts at a square, count how many land in circle.
More generally, measuring volume defined enclosed by wacky surfaces.
These are all trade-offs:
"Monte Carlo": guaranteed fast run time / low memory, but not accurate (false-positives are possible)
"Las Vegas": always acurrate, usually quick but possibly slow.